Uses of Interface
com.yahoo.language.process.Tokenizer
-
Packages that use Tokenizer Package Description com.yahoo.language com.yahoo.language.opennlp com.yahoo.language.process com.yahoo.language.simple -
-
Uses of Tokenizer in com.yahoo.language
Methods in com.yahoo.language that return Tokenizer Modifier and Type Method Description Tokenizer
Linguistics. getTokenizer()
Returns a thread-unsafe tokenizer. -
Uses of Tokenizer in com.yahoo.language.opennlp
Classes in com.yahoo.language.opennlp that implement Tokenizer Modifier and Type Class Description class
OpenNlpTokenizer
Tokenizer using OpenNlpMethods in com.yahoo.language.opennlp that return Tokenizer Modifier and Type Method Description Tokenizer
OpenNlpLinguistics. getTokenizer()
-
Uses of Tokenizer in com.yahoo.language.process
Constructors in com.yahoo.language.process with parameters of type Tokenizer Constructor Description SegmenterImpl(Tokenizer tokenizer)
StemmerImpl(Tokenizer tokenizer)
-
Uses of Tokenizer in com.yahoo.language.simple
Classes in com.yahoo.language.simple that implement Tokenizer Modifier and Type Class Description class
SimpleTokenizer
A tokenizer which splits on whitespace, normalizes and transforms using the given implementations and stems using the kstem algorithm.Methods in com.yahoo.language.simple that return Tokenizer Modifier and Type Method Description Tokenizer
SimpleLinguistics. getTokenizer()
-