Python bindings for whatlang using pyo3

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for omarmhaimdat from gravatar.com omarmhaimdat

Unverified details

These details have not been verified by PyPI
Project links
GitHub Statistics

View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery

Meta

License: MIT

Author: Omar MHAIMDAT

Tags whatlang, pyo3, python, bindings, natural language, language detection

Requires: Python >=3.7

Classifiers

Project description

WhatLang ⚡

Python Binding for WhatLang, a blazing fast language detection library

PyPI version License PyPI - Downloads Build Status

Showcase

WhatLang is a Python library for detecting the language of a text. It is based on the WhatLang Rust library.

Installation

pip install whatlang-pyo3

Usage

Detect

>>> from whatlang import detect
>>> info = detect("This is written in English")
"Language: eng - Script: Latin - Confidence: 0.11450955767632877 - Is reliable: false"
>>> info.lang
"eng"
>>> info.script
"Latin"
>>> info.confidence
0.11450955767632877
>>> info.is_reliable
False
>>> info.to_iso()
"Language: en - Script: Latin - Confidence: 0.11450955767632877 - Is reliable: false"

Detect language

You can also detect the language of a text without the script and the confidence.

>>> from whatlang import detect_lang
>>> detect_lang("This is written in English")
"eng"
>>> detect_lang("Ceci est écrit en français")
"fra"
>>> detect_lang("Ceci est écrit en français").iso
"fr"

Detect script

You can also detect the script of a text without the language and the confidence.

>>> from whatlang import detect_script
>>> detect_script("This is written in English")
Name: Latin - Languages: spa, eng, por, ind, fra, deu, jav, vie, ita, tur, pol, ron, hrv, nld, uzb, hun, aze, ces, zul, swe, aka, sna, afr, fin, slk, tgl, tuk, dan, nob, cat, lit, slv, epo, lav, est, lat
>>> detect_script.name
"Latin"
>>> detect_script.langs
['spa', 'eng', 'por', 'ind', 'fra', 'deu', 'jav', 'vie', 'ita', 'tur', 'pol', 'ron', 'hrv', 'nld', 'uzb', 'hun', 'aze', 'ces', 'zul', 'swe', 'aka', 'sna', 'afr', 'fin', 'slk', 'tgl', 'tuk', 'dan', 'nob', 'cat', 'lit', 'slv', 'epo', 'lav', 'est', 'lat']

Batch detection

You can also detect the language of a list of texts, and take advantage of the parallelism with the n_jobs parameter.

Batch detection

>>> from whatlang import batch_detect
>>> batch_detect(["This is written in English", "Ceci est écrit en français"], n_jobs=-1)
[LanguageInfo(lang='eng', script='Latin', confidence=0.11450955767632877, is_reliable=False), LanguageInfo(lang='fra', script='Latin', confidence=0.11450955767632877, is_reliable=False)]

The performance of the batch detection is much better than the detection of a single text, it can be up to 5 times faster.

Contributing

Contributions are welcome! Please open an issue or a pull request.

Install maturin

We use maturin for the development of this library.

pip install maturin

Build

maturin build --release

License

This project is licensed under the MIT License - see the LICENSE file for details

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for omarmhaimdat from gravatar.com omarmhaimdat

Unverified details

These details have not been verified by PyPI
Project links
GitHub Statistics

View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery

Meta

License: MIT

Author: Omar MHAIMDAT

Tags whatlang, pyo3, python, bindings, natural language, language detection

Requires: Python >=3.7

Classifiers

Release history Release notifications | RSS feed

This version

0.6.0

0.5.0

0.4.0

0.3.3

0.3.2

0.3.1

0.3.0

0.2.3

0.2.2

0.2.1

0.2.0

0.1.6

0.1.5

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

whatlang_pyo3-0.6.0.tar.gz (3.3 MB view hashes)

Uploaded Source

Built Distributions

whatlang_pyo3-0.6.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-cp312-none-win_amd64.whl (351.6 kB view hashes)

Uploaded CPython 3.12 Windows x86-64

whatlang_pyo3-0.6.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-cp312-cp312-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.12 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64

whatlang_pyo3-0.6.0-cp311-none-win_amd64.whl (351.6 kB view hashes)

Uploaded CPython 3.11 Windows x86-64

whatlang_pyo3-0.6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-cp311-cp311-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.11 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64

whatlang_pyo3-0.6.0-cp310-none-win_amd64.whl (351.6 kB view hashes)

Uploaded CPython 3.10 Windows x86-64

whatlang_pyo3-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-cp310-cp310-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.10 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64

whatlang_pyo3-0.6.0-cp39-none-win_amd64.whl (352.3 kB view hashes)

Uploaded CPython 3.9 Windows x86-64

whatlang_pyo3-0.6.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-cp39-cp39-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.9 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64

whatlang_pyo3-0.6.0-cp38-none-win_amd64.whl (351.7 kB view hashes)

Uploaded CPython 3.8 Windows x86-64

whatlang_pyo3-0.6.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-cp38-cp38-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.8 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64

whatlang_pyo3-0.6.0-cp37-none-win_amd64.whl (351.6 kB view hashes)

Uploaded CPython 3.7 Windows x86-64

whatlang_pyo3-0.6.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.6.0-cp37-cp37m-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.7m macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page