Skip to main content

No project description provided

Project description

plsfix

PyPI package

plsfix is drop-in replacement for ftfy, written in Rust and ~10x faster.

>>> from plsfix import fix_text
>>> print(fix_text("(ง'⌣')ง"))
('⌣')

Installing

Python

pip install plsfix

Rust

cargo add plsfix

What it does

(Taken from the ftfy README)

Here are some examples (found in the real world) of what plsfix can do:

plsfix can fix mojibake (encoding mix-ups), by detecting patterns of characters that were clearly meant to be UTF-8 but were decoded as something else:

    >>> import plsfix
    >>> plsfix.fix_text('✔ No problems')
    '✔ No problems'

Does this sound impossible? It's really not. UTF-8 is a well-designed encoding that makes it obvious when it's being misused, and a string of mojibake usually contains all the information we need to recover the original string.

plsfix can fix multiple layers of mojibake simultaneously:

    >>> plsfix.fix_text('The Mona Lisa doesn’t have eyebrows.')
    "The Mona Lisa doesn't have eyebrows."

It can fix mojibake that has had "curly quotes" applied on top of it, which cannot be consistently decoded until the quotes are uncurled:

    >>> plsfix.fix_text("l’humanité")
    "l'humanité"

plsfix can fix mojibake that would have included the character U+A0 (non-breaking space), but the U+A0 was turned into an ASCII space and then combined with another following space:

    >>> plsfix.fix_text(\xa0 perturber la réflexion')
    'à perturber la réflexion'
    >>> plsfix.fix_text('à perturber la réflexion')
    'à perturber la réflexion'

plsfix can also decode HTML entities that appear outside of HTML, even in cases where the entity has been incorrectly capitalized:

    >>> # by the HTML 5 standard, only 'PÉREZ' is acceptable
    >>> plsfix.fix_text('PÉREZ')
    'PÉREZ'

These fixes are not applied in all cases, because plsfix has a strongly-held goal of avoiding false positives -- it should never change correctly-decoded text to something else.

The following text could be encoded in Windows-1252 and decoded in UTF-8, and it would decode as 'MARQUɅ'. However, the original text is already sensible, so it is unchanged.

    >>> plsfix.fix_text('IL Y MARQUÉ…')
    'IL Y MARQUÉ…'

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

plsfix-0.1.7.tar.gz (66.0 kB view hashes)

Uploaded Source

Built Distributions

plsfix-0.1.7-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.6 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

plsfix-0.1.7-pp310-pypy310_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl (13.8 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ s390x

plsfix-0.1.7-pp310-pypy310_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (12.4 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ ppc64le

plsfix-0.1.7-pp310-pypy310_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (10.6 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ ARMv7l

plsfix-0.1.7-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (12.2 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

plsfix-0.1.7-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.whl (12.0 MB view hashes)

Uploaded PyPy manylinux: glibc 2.5+ i686

plsfix-0.1.7-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.6 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

plsfix-0.1.7-pp39-pypy39_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl (13.8 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ s390x

plsfix-0.1.7-pp39-pypy39_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (12.3 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ ppc64le

plsfix-0.1.7-pp39-pypy39_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (10.6 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ ARMv7l

plsfix-0.1.7-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (12.2 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

plsfix-0.1.7-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.whl (12.0 MB view hashes)

Uploaded PyPy manylinux: glibc 2.5+ i686

plsfix-0.1.7-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.5 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

plsfix-0.1.7-pp38-pypy38_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl (13.8 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ s390x

plsfix-0.1.7-pp38-pypy38_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (12.3 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ ppc64le

plsfix-0.1.7-pp38-pypy38_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (10.6 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ ARMv7l

plsfix-0.1.7-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (12.2 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

plsfix-0.1.7-pp38-pypy38_pp73-manylinux_2_5_i686.manylinux1_i686.whl (12.0 MB view hashes)

Uploaded PyPy manylinux: glibc 2.5+ i686

plsfix-0.1.7-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl (13.8 MB view hashes)

Uploaded CPython 3.13 manylinux: glibc 2.17+ s390x

plsfix-0.1.7-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (12.4 MB view hashes)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ppc64le

plsfix-0.1.7-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (10.7 MB view hashes)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ARMv7l

plsfix-0.1.7-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (12.2 MB view hashes)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ARM64

plsfix-0.1.7-cp312-none-win_amd64.whl (1.2 MB view hashes)

Uploaded CPython 3.12 Windows x86-64

plsfix-0.1.7-cp312-none-win32.whl (1.2 MB view hashes)

Uploaded CPython 3.12 Windows x86

plsfix-0.1.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.6 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

plsfix-0.1.7-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl (13.8 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.17+ s390x

plsfix-0.1.7-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (12.4 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ppc64le

plsfix-0.1.7-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (10.7 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARMv7l

plsfix-0.1.7-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (12.2 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

plsfix-0.1.7-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl (12.0 MB view hashes)

Uploaded CPython 3.12 manylinux: glibc 2.5+ i686

plsfix-0.1.7-cp312-cp312-macosx_11_0_arm64.whl (1.6 MB view hashes)

Uploaded CPython 3.12 macOS 11.0+ ARM64

plsfix-0.1.7-cp312-cp312-macosx_10_12_x86_64.whl (1.6 MB view hashes)

Uploaded CPython 3.12 macOS 10.12+ x86-64

plsfix-0.1.7-cp311-none-win_amd64.whl (1.2 MB view hashes)

Uploaded CPython 3.11 Windows x86-64

plsfix-0.1.7-cp311-none-win32.whl (1.2 MB view hashes)

Uploaded CPython 3.11 Windows x86

plsfix-0.1.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.6 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

plsfix-0.1.7-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl (13.9 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ s390x

plsfix-0.1.7-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (12.4 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ppc64le

plsfix-0.1.7-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (10.7 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARMv7l

plsfix-0.1.7-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (12.3 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

plsfix-0.1.7-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl (12.0 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.5+ i686

plsfix-0.1.7-cp311-cp311-macosx_11_0_arm64.whl (1.6 MB view hashes)

Uploaded CPython 3.11 macOS 11.0+ ARM64

plsfix-0.1.7-cp311-cp311-macosx_10_12_x86_64.whl (1.6 MB view hashes)

Uploaded CPython 3.11 macOS 10.12+ x86-64

plsfix-0.1.7-cp310-none-win_amd64.whl (1.2 MB view hashes)

Uploaded CPython 3.10 Windows x86-64

plsfix-0.1.7-cp310-none-win32.whl (1.2 MB view hashes)

Uploaded CPython 3.10 Windows x86

plsfix-0.1.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.6 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

plsfix-0.1.7-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl (13.9 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ s390x

plsfix-0.1.7-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (12.4 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ppc64le

plsfix-0.1.7-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (10.7 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARMv7l

plsfix-0.1.7-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (12.3 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

plsfix-0.1.7-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl (12.0 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.5+ i686

plsfix-0.1.7-cp310-cp310-macosx_11_0_arm64.whl (1.6 MB view hashes)

Uploaded CPython 3.10 macOS 11.0+ ARM64

plsfix-0.1.7-cp310-cp310-macosx_10_12_x86_64.whl (1.6 MB view hashes)

Uploaded CPython 3.10 macOS 10.12+ x86-64

plsfix-0.1.7-cp39-none-win_amd64.whl (1.2 MB view hashes)

Uploaded CPython 3.9 Windows x86-64

plsfix-0.1.7-cp39-none-win32.whl (1.2 MB view hashes)

Uploaded CPython 3.9 Windows x86

plsfix-0.1.7-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.6 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

plsfix-0.1.7-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl (13.8 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ s390x

plsfix-0.1.7-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (12.3 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ppc64le

plsfix-0.1.7-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (10.7 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARMv7l

plsfix-0.1.7-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (12.2 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

plsfix-0.1.7-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl (12.0 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.5+ i686

plsfix-0.1.7-cp38-none-win_amd64.whl (1.2 MB view hashes)

Uploaded CPython 3.8 Windows x86-64

plsfix-0.1.7-cp38-none-win32.whl (1.2 MB view hashes)

Uploaded CPython 3.8 Windows x86

plsfix-0.1.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.6 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

plsfix-0.1.7-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl (13.9 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ s390x

plsfix-0.1.7-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (12.4 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ppc64le

plsfix-0.1.7-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (10.7 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARMv7l

plsfix-0.1.7-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (12.2 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

plsfix-0.1.7-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl (12.0 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.5+ i686

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page