NeuralTokenizer is a very simple to use text tokenizer which uses neural networks from the SimpleDNN library.
NeuralTokenizer is part of KotlinNLP.
<dependency>
<groupId>com.kotlinnlp</groupId>
<artifactId>neuraltokenizer</artifactId>
<version>0.4.8</version>
</dependency>Try some examples of usage of NeuralTokenizer running the files in the examples folder.
To run the examples you need datasets of test and training that you can find here
The neural model is all contained into a single class which provides simple dump() and load() methods to serialize it and afterwards load it.
This software is released under the terms of the Mozilla Public License, v. 2.0
We greatly appreciate any bug reports and contributions, which can be made by filing an issue or making a pull request through the github page.