Uses of Interface
com.yahoo.language.process.Token

Packages that use Token

Package	Description
com.yahoo.language.process
com.yahoo.language.simple

Uses of Token in com.yahoo.language.process

Methods in com.yahoo.language.process that return Token

Modifier and Type	Method	Description
Token	Token.getComponent(int i)	Returns a component token of this

Methods in com.yahoo.language.process that return types with arguments of type Token

Modifier and Type	Method	Description
Iterable<Token>	Tokenizer.tokenize(String input, Language language, StemMode stemMode, boolean removeAccents)	Returns the tokens produced from an input string under the rules of the given Language and additional options

Uses of Token in com.yahoo.language.simple

Classes in com.yahoo.language.simple that implement Token

Modifier and Type	Class	Description
class	SimpleToken

Methods in com.yahoo.language.simple that return Token

Modifier and Type	Method	Description
Token	SimpleToken.getComponent(int i)

Methods in com.yahoo.language.simple that return types with arguments of type Token

Modifier and Type	Method	Description
Iterable<Token>	SimpleTokenizer.tokenize(String input, Language language, StemMode stemMode, boolean removeAccents)	Tokenize the input, applying the transform of this to each token string.
Iterable<Token>	SimpleTokenizer.tokenize(String input, Function<String,String> tokenProcessor)	Tokenize the input, and apply the given transform to each token string.

Methods in com.yahoo.language.simple with parameters of type Token

Modifier and Type	Method	Description
SimpleToken	SimpleToken.addComponent(Token token)