Package com.yahoo.language.process

Class SpecialTokenRegistry

java.lang.Object

com.yahoo.language.process.SpecialTokenRegistry

public class SpecialTokenRegistry
extends Object

Immutable named lists of "special tokens" - strings which should override the normal tokenizer semantics and be tokenized into a single token.

Author:

bratseth

Constructor Summary

Constructors

Constructor	Description
SpecialTokenRegistry()	Creates an empty special token registry
SpecialTokenRegistry(com.yahoo.vespa.configdefinition.SpecialtokensConfig config)	Create a special token registry from a configuration object.
SpecialTokenRegistry(List<SpecialTokens> specialTokensList)

Method Summary

Modifier and Type	Method	Description
SpecialTokens	getSpecialTokens(String name)	Returns the list of special tokens for a given name.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SpecialTokenRegistry

public SpecialTokenRegistry()

Creates an empty special token registry

SpecialTokenRegistry

public SpecialTokenRegistry(com.yahoo.vespa.configdefinition.SpecialtokensConfig config)

Create a special token registry from a configuration object.

SpecialTokenRegistry

public SpecialTokenRegistry(List<SpecialTokens> specialTokensList)

Method Details

getSpecialTokens

public SpecialTokens getSpecialTokens(String name)

Returns the list of special tokens for a given name.

Parameters:

name - the name of the special tokens to return null, the empty string or the string "default" returns the default ones

Returns:

a read-only list of SpecialToken instances, an empty list if this name has no special tokens