Package ai.djl.basicdataset.utils
Class TextData.Configuration
- java.lang.Object
-
- ai.djl.basicdataset.utils.TextData.Configuration
-
-
Constructor Summary
Constructors Constructor Description Configuration()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description TextData.Configuration
setEmbeddingSize(int embeddingSize)
Sets the size for newTextEmbedding
s.TextData.Configuration
setReservedTokens(java.util.List<java.lang.String> reservedTokens)
Sets the list of reserved tokens.TextData.Configuration
setTextEmbedding(ai.djl.modality.nlp.embedding.TextEmbedding textEmbedding)
Sets theTextEmbedding
to use to embed the text data.TextData.Configuration
setTextProcessors(java.util.List<ai.djl.modality.nlp.preprocess.TextProcessor> textProcessors)
Sets theTextProcessor
s to use for the text data.TextData.Configuration
setUnknownToken(java.lang.String unknownToken)
Sets the default unknown token.TextData.Configuration
setVocabulary(ai.djl.modality.nlp.Vocabulary vocabulary)
Sets theVocabulary
to use to hold the text data.TextData.Configuration
update(TextData.Configuration other)
Updates thisTextData.Configuration
with the non-null values from another configuration.
-
-
-
Method Detail
-
setTextProcessors
public TextData.Configuration setTextProcessors(java.util.List<ai.djl.modality.nlp.preprocess.TextProcessor> textProcessors)
Sets theTextProcessor
s to use for the text data.- Parameters:
textProcessors
- theTextProcessor
s- Returns:
- this configuration
-
setTextEmbedding
public TextData.Configuration setTextEmbedding(ai.djl.modality.nlp.embedding.TextEmbedding textEmbedding)
Sets theTextEmbedding
to use to embed the text data.- Parameters:
textEmbedding
- theTextEmbedding
- Returns:
- this configuration
-
setVocabulary
public TextData.Configuration setVocabulary(ai.djl.modality.nlp.Vocabulary vocabulary)
Sets theVocabulary
to use to hold the text data.- Parameters:
vocabulary
- theVocabulary
- Returns:
- this configuration
-
setEmbeddingSize
public TextData.Configuration setEmbeddingSize(int embeddingSize)
Sets the size for newTextEmbedding
s.- Parameters:
embeddingSize
- the embedding size- Returns:
- this configuration
-
setUnknownToken
public TextData.Configuration setUnknownToken(java.lang.String unknownToken)
Sets the default unknown token.- Parameters:
unknownToken
- theString
value of unknown token- Returns:
- this configuration
-
setReservedTokens
public TextData.Configuration setReservedTokens(java.util.List<java.lang.String> reservedTokens)
Sets the list of reserved tokens.- Parameters:
reservedTokens
- true to train the text embedding- Returns:
- this configuration
-
update
public TextData.Configuration update(TextData.Configuration other)
Updates thisTextData.Configuration
with the non-null values from another configuration.- Parameters:
other
- the other configuration to use to update this- Returns:
- this configuration after updating
-
-