Interface TextEmbedding
-
- All Known Implementing Classes:
ModelZooTextEmbedding
,SimpleTextEmbedding
,TrainableTextEmbedding
public interface TextEmbedding
A class to manage 1-DNDArray
representations of multiple words.A text embedding differs from a
WordEmbedding
because the text embedding does not have to be applied to each word independently.A text embedding maps text to a
NDArray
that attempts to represent the key ideas in the words. Each of the values in the dimension can represent different pieces of meaning such as young-old, object-living, etc.These text embeddings can be used in two different ways in models. First, they can be used purely for preprocessing the model. In this case, it is a requirement for most models that use text as an input. The model is not trained. For this use case, use
embedText(ai.djl.ndarray.NDManager, java.util.List<java.lang.String>)
.In the second option, the embedding can be trained using the standard deep learning techniques to better handle the current dataset. For this case, you need two methods. First, call
preprocessTextToEmbed(List)
within your dataset. Then, the first step in your model should be to callembedText(NDManager, long[])
.
-
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description NDArray
embedText(NDArray textIndices)
Embeds the text after preprocessed usingpreprocessTextToEmbed(List)
.default NDArray
embedText(NDManager manager, long[] textIndices)
Embeds the text after preprocessed usingpreprocessTextToEmbed(List)
.default NDArray
embedText(NDManager manager, java.util.List<java.lang.String> text)
Embeds a text.long[]
preprocessTextToEmbed(java.util.List<java.lang.String> text)
Preprocesses the text to embed into an array to pass into the model.java.util.List<java.lang.String>
unembedText(NDArray textEmbedding)
Returns the closest matching text for a given embedding.
-
-
-
Method Detail
-
preprocessTextToEmbed
long[] preprocessTextToEmbed(java.util.List<java.lang.String> text)
Preprocesses the text to embed into an array to pass into the model.Make sure to call
embedText(NDManager, long[])
after this.- Parameters:
text
- the text to embed- Returns:
- the indices of text that is ready to embed
-
embedText
default NDArray embedText(NDManager manager, java.util.List<java.lang.String> text) throws EmbeddingException
Embeds a text.- Parameters:
manager
- the manager for the embedding arraytext
- the text to embed- Returns:
- the embedded text
- Throws:
EmbeddingException
- if there is an error while trying to embed
-
embedText
default NDArray embedText(NDManager manager, long[] textIndices) throws EmbeddingException
Embeds the text after preprocessed usingpreprocessTextToEmbed(List)
.- Parameters:
manager
- the manager to create the embedding arraytextIndices
- the indices of text to embed- Returns:
- the embedded text
- Throws:
EmbeddingException
- if there is an error while trying to embed
-
embedText
NDArray embedText(NDArray textIndices) throws EmbeddingException
Embeds the text after preprocessed usingpreprocessTextToEmbed(List)
.- Parameters:
textIndices
- the indices of text to embed- Returns:
- the embedded text
- Throws:
EmbeddingException
- if there is an error while trying to embed
-
unembedText
java.util.List<java.lang.String> unembedText(NDArray textEmbedding) throws EmbeddingException
Returns the closest matching text for a given embedding.- Parameters:
textEmbedding
- the text embedding to find the matching string text for.- Returns:
- text similar to the passed in embedding
- Throws:
EmbeddingException
- if the input is not unembeddable
-
-