Package ai.djl.modality.nlp
Interface Vocabulary
-
- All Known Implementing Classes:
DefaultVocabulary
public interface Vocabulary
Vocabulary
is a collection of tokens. The primary purpose of a vocabulary is the map a token to an index.
-
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description boolean
contains(java.lang.String token)
Check if the vocabulary contains a token.long
getIndex(java.lang.String token)
Returns the index of the given token.java.lang.String
getToken(long index)
Returns the token corresponding to the given index.long
size()
Returns the size of theVocabulary
.
-
-
-
Method Detail
-
getToken
java.lang.String getToken(long index)
Returns the token corresponding to the given index.- Parameters:
index
- the index- Returns:
- the token corresponding to the given index
-
contains
boolean contains(java.lang.String token)
Check if the vocabulary contains a token.- Parameters:
token
- String token to be checked- Returns:
- whether this vocabulary contains the token
-
getIndex
long getIndex(java.lang.String token)
Returns the index of the given token.- Parameters:
token
- the token- Returns:
- the index of the given token.
-
size
long size()
Returns the size of theVocabulary
.- Returns:
- the size of the
Vocabulary
-
-