Uses of Interface
com.yahoo.language.process.Tokenizer

Packages that use Tokenizer

Package	Description
com.yahoo.language
com.yahoo.language.opennlp
com.yahoo.language.process
com.yahoo.language.simple

Uses of Tokenizer in com.yahoo.language

Methods in com.yahoo.language that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	Linguistics.getTokenizer()	Returns a thread-unsafe tokenizer.

Uses of Tokenizer in com.yahoo.language.opennlp

Classes in com.yahoo.language.opennlp that implement Tokenizer

Modifier and Type	Class	Description
class	OpenNlpTokenizer	Tokenizer using OpenNlp

Methods in com.yahoo.language.opennlp that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	OpenNlpLinguistics.getTokenizer()

Uses of Tokenizer in com.yahoo.language.process

Constructors in com.yahoo.language.process with parameters of type Tokenizer

Modifier	Constructor	Description
	SegmenterImpl(Tokenizer tokenizer)
	StemmerImpl(Tokenizer tokenizer)

Uses of Tokenizer in com.yahoo.language.simple

Classes in com.yahoo.language.simple that implement Tokenizer

Modifier and Type	Class	Description
class	SimpleTokenizer	A tokenizer which splits on whitespace, normalizes and transforms using the given implementations and stems using the kstem algorithm.

Methods in com.yahoo.language.simple that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	SimpleLinguistics.getTokenizer()