SpecialTokens (linguistics 7.527.20 API)

java.lang.Object
- com.yahoo.language.process.SpecialTokens

```
public class SpecialTokens
extends Object
```
An immutable list of special tokens - strings which should override the normal tokenizer semantics and be tokenized into a single token. Special tokens are case insensitive.

Author:

bratseth

Nested Class Summary

Nested Classes
Modifier and Type Class Description

static class SpecialTokens.Token
An immutable special token

Constructor Summary

Constructors
Constructor Description

SpecialTokens(String name, List<SpecialTokens.Token> tokens)

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`Map<String,String>`	`asMap()`	Returns the tokens of this as an immutable map from token to replacement.
`static SpecialTokens`	`empty()`
`String`	`name()`	Returns the name of this special tokens list
`SpecialTokens.Token`	`tokenize(String string, boolean substring)`	Returns the special token starting at the start of the given string, or null if no special token starts at this string

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - SpecialTokens
```
public SpecialTokens(String name,
                     List<SpecialTokens.Token> tokens)
```
- Method Detail
  - name
```
public String name()
```
    Returns the name of this special tokens list
  - asMap
```
public Map<String,String> asMap()
```
    Returns the tokens of this as an immutable map from token to replacement. Tokens which do not have a replacement token maps to themselves.
  - tokenize
```
public SpecialTokens.Token tokenize(String string,
                                    boolean substring)
```
    Returns the special token starting at the start of the given string, or null if no special token starts at this string
    
    Parameters:
    
    string - the string to search for a special token at the start position
    
    substring - true to allow the special token to be followed by a character which does not mark the end of a token
  - empty
```
public static SpecialTokens empty()
```