Uses of Interface
org.languagetool.tokenizers.Tokenizer
-
Packages that use Tokenizer Package Description org.languagetool org.languagetool.noop org.languagetool.rules.ngrams org.languagetool.tokenizers -
-
Uses of Tokenizer in org.languagetool
Methods in org.languagetool that return Tokenizer Modifier and Type Method Description Tokenizer
Language. getWordTokenizer()
Get this language's word tokenizer implementation. -
Uses of Tokenizer in org.languagetool.noop
Methods in org.languagetool.noop that return Tokenizer Modifier and Type Method Description Tokenizer
NoopLanguage. getWordTokenizer()
-
Uses of Tokenizer in org.languagetool.rules.ngrams
Methods in org.languagetool.rules.ngrams that return Tokenizer Modifier and Type Method Description protected Tokenizer
NgramProbabilityRule. getGoogleStyleWordTokenizer()
-
Uses of Tokenizer in org.languagetool.tokenizers
Subinterfaces of Tokenizer in org.languagetool.tokenizers Modifier and Type Interface Description interface
CompoundWordTokenizer
Interface for components that take compound words and split them into their parts.interface
SentenceTokenizer
Tokenizes text into sentences.Classes in org.languagetool.tokenizers that implement Tokenizer Modifier and Type Class Description class
SimpleSentenceTokenizer
A very simple sentence tokenizer that splits on[.!?…]
followed by whitespace or an uppercase letter.class
SRXSentenceTokenizer
Class to tokenize sentences using rules from an SRX file.class
WordTokenizer
Tokenizes a sentence into words.
-