Package com.yahoo.language.process
Class SpecialTokenRegistry
- java.lang.Object
-
- com.yahoo.language.process.SpecialTokenRegistry
-
public class SpecialTokenRegistry extends java.lang.Object
Immutable named lists of "special tokens" - strings which should override the normal tokenizer semantics and be tokenized into a single token.- Author:
- bratseth
-
-
Constructor Summary
Constructors Constructor Description SpecialTokenRegistry()
Creates an empty special token registrySpecialTokenRegistry(com.yahoo.vespa.configdefinition.SpecialtokensConfig config)
Create a special token registry from a configuration object.SpecialTokenRegistry(java.util.List<SpecialTokens> specialTokensList)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description SpecialTokens
getSpecialTokens(java.lang.String name)
Returns the list of special tokens for a given name.
-
-
-
Constructor Detail
-
SpecialTokenRegistry
public SpecialTokenRegistry()
Creates an empty special token registry
-
SpecialTokenRegistry
public SpecialTokenRegistry(com.yahoo.vespa.configdefinition.SpecialtokensConfig config)
Create a special token registry from a configuration object.
-
SpecialTokenRegistry
public SpecialTokenRegistry(java.util.List<SpecialTokens> specialTokensList)
-
-
Method Detail
-
getSpecialTokens
public SpecialTokens getSpecialTokens(java.lang.String name)
Returns the list of special tokens for a given name.- Parameters:
name
- the name of the special tokens to return null, the empty string or the string "default" returns the default ones- Returns:
- a read-only list of SpecialToken instances, an empty list if this name has no special tokens
-
-