Index (Deep Java Library 0.20.0

A B C D E F G H L N O P Q S T
All Classes All Packages

A

ai.djl.huggingface.tokenizers - package ai.djl.huggingface.tokenizers: Contains classes to interface with the underlying Huggingface tokenizers.
ai.djl.huggingface.tokenizers.jni - package ai.djl.huggingface.tokenizers.jni: Contains classes to interface with the underlying Huggingface tokenizers.
ai.djl.huggingface.translator - package ai.djl.huggingface.translator: Contains Translator classes that leverage Huggingface tokenizers.
ai.djl.huggingface.zoo - package ai.djl.huggingface.zoo: Contains the built-in HfModelZoo.

B

batchDecode(long[][]) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the decoded Strings from the input batch ids.
batchDecode(long[][], boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the decoded Strings from the input batch ids.
batchDecode(long, long[][], boolean) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
batchEncode(long, String[], boolean) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
batchEncode(String[]) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentence in batch.
batchEncode(String[], boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentence in batch.
batchEncode(List<String>) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentence in batch.
batchEncode(List<String>, boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentence in batch.
build() - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Builds the translator.
build() - Method in class ai.djl.huggingface.translator.FillMaskTranslator.Builder: Builds the translator.
build() - Method in class ai.djl.huggingface.translator.QuestionAnsweringTranslator.Builder: Builds the translator.
build() - Method in class ai.djl.huggingface.translator.TextClassificationTranslator.Builder: Builds the translator.
build() - Method in class ai.djl.huggingface.translator.TextEmbeddingTranslator.Builder: Builds the translator.
build() - Method in class ai.djl.huggingface.translator.TokenClassificationTranslator.Builder: Builds the translator.
builder() - Static method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Creates a builder to build a HuggingFaceTokenizer.
builder(HuggingFaceTokenizer) - Static method in class ai.djl.huggingface.translator.FillMaskTranslator: Creates a builder to build a FillMaskTranslator.
builder(HuggingFaceTokenizer) - Static method in class ai.djl.huggingface.translator.QuestionAnsweringTranslator: Creates a builder to build a QuestionAnsweringTranslator.
builder(HuggingFaceTokenizer) - Static method in class ai.djl.huggingface.translator.TextClassificationTranslator: Creates a builder to build a TextClassificationTranslator.
builder(HuggingFaceTokenizer) - Static method in class ai.djl.huggingface.translator.TextEmbeddingTranslator: Creates a builder to build a TextEmbeddingTranslator.
builder(HuggingFaceTokenizer) - Static method in class ai.djl.huggingface.translator.TokenClassificationTranslator: Creates a builder to build a TokenClassificationTranslator.
builder(HuggingFaceTokenizer, Map<String, ?>) - Static method in class ai.djl.huggingface.translator.FillMaskTranslator: Creates a builder to build a FillMaskTranslator.
builder(HuggingFaceTokenizer, Map<String, ?>) - Static method in class ai.djl.huggingface.translator.QuestionAnsweringTranslator: Creates a builder to build a QuestionAnsweringTranslator.
builder(HuggingFaceTokenizer, Map<String, ?>) - Static method in class ai.djl.huggingface.translator.TextClassificationTranslator: Creates a builder to build a TextClassificationTranslator.
builder(HuggingFaceTokenizer, Map<String, ?>) - Static method in class ai.djl.huggingface.translator.TextEmbeddingTranslator: Creates a builder to build a TextEmbeddingTranslator.
builder(HuggingFaceTokenizer, Map<String, ?>) - Static method in class ai.djl.huggingface.translator.TokenClassificationTranslator: Creates a builder to build a TokenClassificationTranslator.
builder(Map<String, ?>) - Static method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Creates a builder to build a HuggingFaceTokenizer.
buildSentence(List<String>) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer

C

CharSpan - Class in ai.djl.huggingface.tokenizers.jni: A class holds character span information.
CharSpan(int, int) - Constructor for class ai.djl.huggingface.tokenizers.jni.CharSpan: Constructs a new CharSpan instance.
checkStatus() - Static method in class ai.djl.huggingface.tokenizers.jni.LibUtils
close() - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer
configure(Map<String, ?>) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Configures the builder with the arguments.
configure(Map<String, ?>) - Method in class ai.djl.huggingface.translator.FillMaskTranslator.Builder: Configures the builder with the model arguments.
configure(Map<String, ?>) - Method in class ai.djl.huggingface.translator.QuestionAnsweringTranslator.Builder: Configures the builder with the model arguments.
configure(Map<String, ?>) - Method in class ai.djl.huggingface.translator.TextClassificationTranslator.Builder: Configures the builder with the model arguments.
configure(Map<String, ?>) - Method in class ai.djl.huggingface.translator.TextEmbeddingTranslator.Builder: Configures the builder with the model arguments.
configure(Map<String, ?>) - Method in class ai.djl.huggingface.translator.TokenClassificationTranslator.Builder: Configures the builder with the model arguments.
createTokenizer(String) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
createTokenizerFromString(String) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary

D

decode(long[]) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the decoded String from the input ids.
decode(long[], boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the decoded String from the input ids.
decode(long, long[], boolean) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
deleteEncoding(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
deleteTokenizer(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
disablePadding(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
disableTruncation(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary

E

encode(long, String, boolean) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
encode(String) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentence.
encode(String[]) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentences.
encode(String[], boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentences.
encode(String, boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentence.
encode(String, String) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentence.
encode(String, String, boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentence.
encode(List<String>) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentences.
encode(List<String>, boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Returns the Encoding of the input sentences.
encodeDual(long, String, String, boolean) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
encodeList(long, String[], boolean) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
Encoding - Class in ai.djl.huggingface.tokenizers: A class holds token encoding information.
Encoding(long[], long[], String[], long[], long[], long[], CharSpan[], Encoding[]) - Constructor for class ai.djl.huggingface.tokenizers.Encoding

F

FillMaskTranslator - Class in ai.djl.huggingface.translator: The translator for Huggingface fill mask model.
FillMaskTranslator.Builder - Class in ai.djl.huggingface.translator: The builder for fill mask translator.
FillMaskTranslatorFactory - Class in ai.djl.huggingface.translator: A TranslatorFactory that creates a FillMaskTranslator instance.
FillMaskTranslatorFactory() - Constructor for class ai.djl.huggingface.translator.FillMaskTranslatorFactory
finalize() - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer

G

getAttentionMask() - Method in class ai.djl.huggingface.tokenizers.Encoding: Returns the attention masks.
getAttentionMask(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getBatchifier() - Method in class ai.djl.huggingface.translator.FillMaskTranslator
getBatchifier() - Method in class ai.djl.huggingface.translator.QuestionAnsweringTranslator
getBatchifier() - Method in class ai.djl.huggingface.translator.TextClassificationTranslator
getBatchifier() - Method in class ai.djl.huggingface.translator.TextEmbeddingTranslator
getBatchifier() - Method in class ai.djl.huggingface.translator.TokenClassificationTranslator
getCharTokenSpans() - Method in class ai.djl.huggingface.tokenizers.Encoding: Returns char token spans.
getEnd() - Method in class ai.djl.huggingface.tokenizers.jni.CharSpan: Returns the end position.
getGroupId() - Method in class ai.djl.huggingface.zoo.HfModelZoo
getIds() - Method in class ai.djl.huggingface.tokenizers.Encoding: Returns the token ids.
getMaxLength(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getModelZoo() - Method in class ai.djl.huggingface.zoo.HfZooProvider
getOverflowing() - Method in class ai.djl.huggingface.tokenizers.Encoding: Returns an array of overflowing encodings.
getOverflowing(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getPaddingStrategy(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getPadToMultipleOf(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getSpecialTokenMask() - Method in class ai.djl.huggingface.tokenizers.Encoding: Returns the special token masks.
getSpecialTokenMask(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getStart() - Method in class ai.djl.huggingface.tokenizers.jni.CharSpan: Returns the start position.
getStride(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getSupportedEngines() - Method in class ai.djl.huggingface.zoo.HfModelZoo
getSupportedTypes() - Method in class ai.djl.huggingface.translator.FillMaskTranslatorFactory
getSupportedTypes() - Method in class ai.djl.huggingface.translator.QuestionAnsweringTranslatorFactory
getSupportedTypes() - Method in class ai.djl.huggingface.translator.TextClassificationTranslatorFactory
getSupportedTypes() - Method in class ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory
getSupportedTypes() - Method in class ai.djl.huggingface.translator.TokenClassificationTranslatorFactory
getTokenCharSpans(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getTokenIds(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getTokens() - Method in class ai.djl.huggingface.tokenizers.Encoding: Returns the tokens.
getTokens(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getTruncationStrategy(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getTypeIds() - Method in class ai.djl.huggingface.tokenizers.Encoding: Returns the token type ids.
getTypeIds(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
getWordIds() - Method in class ai.djl.huggingface.tokenizers.Encoding: Returns the word ids.
getWordIds(long) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary

H

HfModelZoo - Class in ai.djl.huggingface.zoo: HfModelZoo is a repository that contains HuggingFace models.
HfZooProvider - Class in ai.djl.huggingface.zoo: An Huggingface model zoo provider implements the ZooProvider interface.
HfZooProvider() - Constructor for class ai.djl.huggingface.zoo.HfZooProvider
HuggingFaceTokenizer - Class in ai.djl.huggingface.tokenizers: HuggingFaceTokenizer is a Huggingface tokenizer implementation of the Tokenizer interface that converts sentences into token.
HuggingFaceTokenizer.Builder - Class in ai.djl.huggingface.tokenizers: The builder for creating huggingface tokenizer.

L

LIB - Static variable in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
LibUtils - Class in ai.djl.huggingface.tokenizers.jni: Utilities for finding the Huggingface tokenizer native binary on the System.

N

newInstance(InputStream, Map<String, String>) - Static method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Create a pre-trained HuggingFaceTokenizer instance from InputStream.
newInstance(Class, Class<O>, Model, Map<String, ?>) - Method in class ai.djl.huggingface.translator.FillMaskTranslatorFactory
newInstance(Class, Class<O>, Model, Map<String, ?>) - Method in class ai.djl.huggingface.translator.QuestionAnsweringTranslatorFactory
newInstance(Class, Class<O>, Model, Map<String, ?>) - Method in class ai.djl.huggingface.translator.TextClassificationTranslatorFactory
newInstance(Class, Class<O>, Model, Map<String, ?>) - Method in class ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory
newInstance(Class, Class<O>, Model, Map<String, ?>) - Method in class ai.djl.huggingface.translator.TokenClassificationTranslatorFactory
newInstance(String) - Static method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Creates a pre-trained HuggingFaceTokenizer instance from huggingface hub.
newInstance(String, Map<String, String>) - Static method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Create a pre-trained HuggingFaceTokenizer instance from huggingface hub.
newInstance(Path) - Static method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Create a pre-trained HuggingFaceTokenizer instance from existing models.
newInstance(Path, Map<String, String>) - Static method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer: Create a pre-trained HuggingFaceTokenizer instance from existing models.

O

optAddSpecialTokens(boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Sets if add special tokens.
optBatchifier(Batchifier) - Method in class ai.djl.huggingface.translator.FillMaskTranslator.Builder: Sets the Batchifier for the Translator.
optBatchifier(Batchifier) - Method in class ai.djl.huggingface.translator.QuestionAnsweringTranslator.Builder: Sets the Batchifier for the Translator.
optBatchifier(Batchifier) - Method in class ai.djl.huggingface.translator.TextClassificationTranslator.Builder: Sets the Batchifier for the Translator.
optBatchifier(Batchifier) - Method in class ai.djl.huggingface.translator.TextEmbeddingTranslator.Builder: Sets the Batchifier for the Translator.
optBatchifier(Batchifier) - Method in class ai.djl.huggingface.translator.TokenClassificationTranslator.Builder: Sets the Batchifier for the Translator.
optIncludeTokenTypes(boolean) - Method in class ai.djl.huggingface.translator.QuestionAnsweringTranslator.Builder: Sets if include token types for the Translator.
optManager(NDManager) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Sets the optional manager used to manage the lifecycle of the tokenizer.
optMaskToken(String) - Method in class ai.djl.huggingface.translator.FillMaskTranslator.Builder: Sets the id of the mask Translator.
optMaxLength(int) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Sets maxLength for padding and truncation.
optPadding(boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Enables or Disables default padding behavior for the tokenizer.
optPadToMaxLength() - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Enables padding to pad sequences to previously specified maxLength, or modelMaxLength if not specified.
optPadToMultipleOf(int) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Sets padToMultipleOf for padding.
optStride(int) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Sets the stride to use in overflow overlap when truncating sequences longer than the model supports.
optTokenizerName(String) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Sets the name of the tokenizer.
optTokenizerPath(Path) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Sets the file path of the tokenizer.
optTopK(int) - Method in class ai.djl.huggingface.translator.FillMaskTranslator.Builder: Set the topK number of classes to be displayed.
optTruncateFirstOnly() - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Enables truncation to only truncate the first item.
optTruncateSecondOnly() - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Enables truncation to only truncate the second item.
optTruncation(boolean) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.Builder: Enables or Disables default truncation behavior for the tokenizer.

P

prepare(TranslatorContext) - Method in class ai.djl.huggingface.translator.TextClassificationTranslator
prepare(TranslatorContext) - Method in class ai.djl.huggingface.translator.TokenClassificationTranslator
PretrainedConfig - Class in ai.djl.huggingface.translator: A class representing HuggingFace config.json file.
PretrainedConfig() - Constructor for class ai.djl.huggingface.translator.PretrainedConfig
processInput(TranslatorContext, QAInput) - Method in class ai.djl.huggingface.translator.QuestionAnsweringTranslator
processInput(TranslatorContext, String) - Method in class ai.djl.huggingface.translator.FillMaskTranslator
processInput(TranslatorContext, String) - Method in class ai.djl.huggingface.translator.TextClassificationTranslator
processInput(TranslatorContext, String) - Method in class ai.djl.huggingface.translator.TextEmbeddingTranslator
processInput(TranslatorContext, String) - Method in class ai.djl.huggingface.translator.TokenClassificationTranslator
processOutput(TranslatorContext, NDList) - Method in class ai.djl.huggingface.translator.FillMaskTranslator
processOutput(TranslatorContext, NDList) - Method in class ai.djl.huggingface.translator.QuestionAnsweringTranslator
processOutput(TranslatorContext, NDList) - Method in class ai.djl.huggingface.translator.TextClassificationTranslator
processOutput(TranslatorContext, NDList) - Method in class ai.djl.huggingface.translator.TextEmbeddingTranslator
processOutput(TranslatorContext, NDList) - Method in class ai.djl.huggingface.translator.TokenClassificationTranslator

Q

QuestionAnsweringTranslator - Class in ai.djl.huggingface.translator: The translator for Huggingface question answering model.
QuestionAnsweringTranslator.Builder - Class in ai.djl.huggingface.translator: The builder for question answering translator.
QuestionAnsweringTranslatorFactory - Class in ai.djl.huggingface.translator: A TranslatorFactory that creates a QuestionAnsweringTranslator instance.
QuestionAnsweringTranslatorFactory() - Constructor for class ai.djl.huggingface.translator.QuestionAnsweringTranslatorFactory

S

setPadding(long, int, String, int) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary
setTruncation(long, int, String, int) - Method in class ai.djl.huggingface.tokenizers.jni.TokenizersLibrary

T

TextClassificationTranslator - Class in ai.djl.huggingface.translator: The translator for Huggingface text classification model.
TextClassificationTranslator.Builder - Class in ai.djl.huggingface.translator: The builder for token classification translator.
TextClassificationTranslatorFactory - Class in ai.djl.huggingface.translator: A TranslatorFactory that creates a TextClassificationTranslator instance.
TextClassificationTranslatorFactory() - Constructor for class ai.djl.huggingface.translator.TextClassificationTranslatorFactory
TextEmbeddingTranslator - Class in ai.djl.huggingface.translator: The translator for Huggingface text embedding model.
TextEmbeddingTranslator.Builder - Class in ai.djl.huggingface.translator: The builder for token classification translator.
TextEmbeddingTranslatorFactory - Class in ai.djl.huggingface.translator: A TranslatorFactory that creates a TextEmbeddingTranslator instance.
TextEmbeddingTranslatorFactory() - Constructor for class ai.djl.huggingface.translator.TextEmbeddingTranslatorFactory
TokenClassificationTranslator - Class in ai.djl.huggingface.translator: The translator for Huggingface token classification model.
TokenClassificationTranslator.Builder - Class in ai.djl.huggingface.translator: The builder for token classification translator.
TokenClassificationTranslatorFactory - Class in ai.djl.huggingface.translator: A TranslatorFactory that creates a TokenClassificationTranslator instance.
TokenClassificationTranslatorFactory() - Constructor for class ai.djl.huggingface.translator.TokenClassificationTranslatorFactory
tokenize(String) - Method in class ai.djl.huggingface.tokenizers.HuggingFaceTokenizer
TokenizersLibrary - Class in ai.djl.huggingface.tokenizers.jni: A class containing utilities to interact with the Tokenizer JNI layer.

A B C D E F G H L N O P Q S T
All Classes All Packages