This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).
Project description
The author of this package has not provided a project description
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
python-ucto-0.6.7.tar.gz
(104.7 kB
view hashes)
Built Distributions
Close
Hashes for python_ucto-0.6.7-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 75c2814548684701346cdde7d442868553a68758e9cc94aa072fa6180a801593 |
|
MD5 | 97af25262514d0e8168877fe88cf4dcf |
|
BLAKE2b-256 | b4d987f0c20ee5c597bbabfe0ed987fb4100c3c8620071851103f832a0b33555 |
Close
Hashes for python_ucto-0.6.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec97216860d457fc7c5909c1ef775c8b4f2d54702041484efe6b4a1b796f0feb |
|
MD5 | 1400db93fcc3d86d25685c2f7ae18536 |
|
BLAKE2b-256 | 82c78dbea1c60c64ab308345fc86b9797c87496d716cf64b8bc9cbf1bc2497f1 |
Close
Hashes for python_ucto-0.6.7-cp312-cp312-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 903991f4445204d81ab34478dc420ac8139bef0ea9d5a98cb7c4eae85502fa7b |
|
MD5 | b82d59a15a6b7a2df89b753028cead47 |
|
BLAKE2b-256 | 0a9b68cc6bd92421e7934ef8451a10616df3cd6595afbbf6e40f10165380aa86 |
Close
Hashes for python_ucto-0.6.7-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4276dfa686afb5e13f9a60139a4827c1084da9c93320b4e1811f192ea0d1a7b5 |
|
MD5 | 70244c5cac7e96b9138beb09447c9d1e |
|
BLAKE2b-256 | 6da5e52345bc28d317f1e07f760652a3440438b5c767807407392d2cc596e663 |
Close
Hashes for python_ucto-0.6.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e7aa7c96616a4943833c029b8dcfa7b84f5f49c4ab5c7378962ba6197e3d6eae |
|
MD5 | cf9b4fd9f8ae2636505882aa60278223 |
|
BLAKE2b-256 | 69ce830b8ac02580df22348eb6c3f0bf875da9511f6d8c1cc17b8dd3d18b2b32 |
Close
Hashes for python_ucto-0.6.7-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eff7a87da114d242bc8133030793611e954cdea42070e04260026e0e36bedf23 |
|
MD5 | 5bfd42dc824b4a011f78ce48d69cd730 |
|
BLAKE2b-256 | a27fbbed634964eb89ce4095ffb3042980b77b198c48d0ecbbf28872733ca4a9 |
Close
Hashes for python_ucto-0.6.7-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6ff8b7b02d3062c7f445b4c5c19ab35254beca677f1edf38948a76710b35bbc6 |
|
MD5 | 5aa437b9a20dc099df7b514eaa733324 |
|
BLAKE2b-256 | 7187c953313daa1cef14f173c756b75c12574cb4ce8fae69c7cbd9034ec60a36 |
Close
Hashes for python_ucto-0.6.7-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6ac68722012ec18bfb034bb87ce1e1f1fcfd437e749322a52805532045165641 |
|
MD5 | b0685031f8c0c62aa3e5179d797dd5ae |
|
BLAKE2b-256 | f79396f180b26cda8efdb103f851149d8105807f5676cfd61288c969450156c1 |
Close
Hashes for python_ucto-0.6.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dcaef3972756ea90cb146b608c3454e314c2c44d8123486fa085eba257315d3a |
|
MD5 | 17d4402094c5ee6084a471926fa82c5f |
|
BLAKE2b-256 | 40de6149dc61f6893fa8f3953419caa693f16339a93a9d9a640ecba637ff2808 |
Close
Hashes for python_ucto-0.6.7-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3ff20f4b70a4028255d11992930616a64f14f2aa66c29c5634c90f0af4a68b13 |
|
MD5 | 0429d75200e02a18c029f81e659462f8 |
|
BLAKE2b-256 | 1d14ffc23f6e01ae807315e91b1175397aedc30b106c9527044fe54c30c881a7 |
Close
Hashes for python_ucto-0.6.7-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f490a6c9affc8828cd5045d2c10596486ab4766dc8c01d36c5fb97d3cbcf8c08 |
|
MD5 | aff3b426f91a50455328ba6a25db16a0 |
|
BLAKE2b-256 | 92ae44d06147e2316f86e75edf082c8fdafebfa421580deb1838df17785918cf |
Close
Hashes for python_ucto-0.6.7-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 479a08b41d990549c454a7d60998ec40990e60f6330915bc132e48d1530eb736 |
|
MD5 | 4a4a0671aa9c0e53d4a8180577e2cc55 |
|
BLAKE2b-256 | 629053718878bfc2a716cc7af1d0170c85e9fba48bc9693c1bf7513ea9127852 |
Close
Hashes for python_ucto-0.6.7-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3bcb4d50e7faa73fe22658426dab89c4e0ecab0d1c6e8c9aaf39c3b8ebe6ef75 |
|
MD5 | 7beaa8c204f255a508258d08ec555f88 |
|
BLAKE2b-256 | 6cb5e48f0c9def21d227823913fa7f4b057f6a23e3ad3107ef3bbe8521485103 |
Close
Hashes for python_ucto-0.6.7-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 286818db3d6d1db08c7f6415fd7b73a32b06a95b903165306bbfc2e2a65266c4 |
|
MD5 | 480373d5c90be7d741706df251666235 |
|
BLAKE2b-256 | 99923e836f3c1dd3af2caeb0d2522c06a7598ae3249d72e5befd4062b953c91d |
Close
Hashes for python_ucto-0.6.7-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cad15c50b4b89625a0e4adb7d9bf92dbaeb2d17b7ee6d865331980a245796837 |
|
MD5 | f6948f08eaed4071d2484357ac880d6b |
|
BLAKE2b-256 | 9bbb12fb90a36e0142ea297bb406368e189b929fe4934c528bced8aaa0cd6542 |
Close
Hashes for python_ucto-0.6.7-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 49b9bb8d7fb275dac9fc2c034bb4de4765e9a6119df2e6bf658a0d7463e42799 |
|
MD5 | e3325d811cd4574be647adfa47cbafcf |
|
BLAKE2b-256 | aff9c7a62034d2347ce302f245f19f5b4b1bb4ccf021d066eb15cd02496c37f5 |
Close
Hashes for python_ucto-0.6.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 93201e1b5965195db5b2529068ad627effab0b84f7503e4cddfa36e0b3e8ca8a |
|
MD5 | 28ccc523838b4dd57b3f1952a0ab787b |
|
BLAKE2b-256 | 4cd56626559887d3037a2ddf29d75fba25ed94a8446b48b3a3dea8027a114ad9 |
Close
Hashes for python_ucto-0.6.7-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 730d44e0fe8e882bc3307d99950472632c70ba422e2c66f694134c93d5148f86 |
|
MD5 | 0cd03e8e6499243be3fdb61945891e64 |
|
BLAKE2b-256 | 6a04544dca2b2f7e57bd1a1283c0fee7c75c210795e3413574abd6609d657497 |
Close
Hashes for python_ucto-0.6.7-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d9edb110df936b261f6fd8f53b634739bf5ff96531baedec92e9c96ee96467a2 |
|
MD5 | 9270bf9d1165b7b4befff14b31bf803e |
|
BLAKE2b-256 | 8f1f8a2a84555d86ec9469b92386544b4297d19e175c4d58d49fd7204555d0af |
Close
Hashes for python_ucto-0.6.7-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1995ce9bd3eac14ea9287fb6011e6f2bfb5976a31760bd417c6ab3710185a916 |
|
MD5 | 4f7056a244403c8fe686c43b161698d6 |
|
BLAKE2b-256 | da8ca0cf748429ea2d4d566f85e79bc6cc91436a7df38f455b070df83dbca4af |
Close
Hashes for python_ucto-0.6.7-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 47e22ab48ff83b8d78f63a93dc23ead1467b038b7d890347075a5ff7fc8cc469 |
|
MD5 | 9f88545d19eb96e749d1e6025c1c16ab |
|
BLAKE2b-256 | c8c6dfa2ea4ed40bb5da531a271a27b76907c2fdabb7498e8bb4493f9f62f95c |
Close
Hashes for python_ucto-0.6.7-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8befcc84c6f6bce9f181cac7f5aa8c3d19a94cdf37164f1848a9172f2192b136 |
|
MD5 | 060ee0a2653c200bf5f6778e6cc2609b |
|
BLAKE2b-256 | 08c2eef16414931bd25de5f9ebd2175cc93de48faa31e5f4f48af0247820f327 |
Close
Hashes for python_ucto-0.6.7-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 10df0c8e8fce35744c45257d3dc4094550cecb683f5f7ceeb26592e61d51c896 |
|
MD5 | 710486d9c298283d867d3bd592b6730f |
|
BLAKE2b-256 | e3db0b8127a0ed6513bd215936b30775874739c453a96f2ec308e08b023f32cb |
Close
Hashes for python_ucto-0.6.7-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 232eb038a403bc91a912c24419929196b3011a9a810dee5f0f7be964de98021a |
|
MD5 | f34c0334d8425cbe58bafa709e88a7e4 |
|
BLAKE2b-256 | 771a202cc7833d70533cee8f46ae183b3a496057bb0696f8906460de258de1ed |