public class TrainableTextEmbedding extends AbstractBlock implements TextEmbedding
TrainableTextEmbedding
is an implementation of TextEmbedding
based on TrainableWordEmbedding
block. This TextEmbedding
is ideal when there are no pre-trained
embeddings available, or when the pre-trained embedding needs to be further trained.children, inputNames, inputShapes, parameters, parameterShapeCallbacks, version
Constructor and Description |
---|
TrainableTextEmbedding(TrainableWordEmbedding wordEmbedding)
Constructs a
TrainableTextEmbedding . |
Modifier and Type | Method and Description |
---|---|
NDArray |
embedText(NDArray textIndices)
Embeds the text after preprocessed using
TextEmbedding.preprocessTextToEmbed(List) . |
NDList |
forward(ParameterStore parameterStore,
NDList inputs,
boolean training,
ai.djl.util.PairList<java.lang.String,java.lang.Object> params)
Applies the operating function of the block once.
|
Shape[] |
getOutputShapes(NDManager manager,
Shape[] inputShapes)
Returns the expected output shapes of the block for the specified input shapes.
|
void |
initializeChildBlocks(NDManager manager,
DataType dataType,
Shape... inputShapes)
Initializes the Child blocks of this block.
|
long[] |
preprocessTextToEmbed(java.util.List<java.lang.String> text)
Preprocesses the text to embed into an array to pass into the model.
|
java.util.List<java.lang.String> |
unembedText(NDArray textEmbedding)
Returns the closest matching text for a given embedding.
|
addChildBlock, addParameter, addParameter, addParameter, beforeInitialize, cast, clear, describeInput, getChildren, getDirectParameters, getParameters, getParameterShape, initialize, isInitialized, loadMetadata, loadParameters, readInputShapes, saveInputShapes, saveMetadata, saveParameters, setInitializer, setInitializer, toString
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
embedText, embedText
forward, forward, validateLayout
public TrainableTextEmbedding(TrainableWordEmbedding wordEmbedding)
TrainableTextEmbedding
.wordEmbedding
- the word embedding to embed each wordpublic long[] preprocessTextToEmbed(java.util.List<java.lang.String> text)
Make sure to call TextEmbedding.embedText(NDManager, long[])
after this.
preprocessTextToEmbed
in interface TextEmbedding
text
- the text to embedpublic NDArray embedText(NDArray textIndices) throws EmbeddingException
TextEmbedding
TextEmbedding.preprocessTextToEmbed(List)
.embedText
in interface TextEmbedding
textIndices
- the indices of text to embedEmbeddingException
- if there is an error while trying to embedpublic java.util.List<java.lang.String> unembedText(NDArray textEmbedding)
unembedText
in interface TextEmbedding
textEmbedding
- the text embedding to find the matching string text for.public NDList forward(ParameterStore parameterStore, NDList inputs, boolean training, ai.djl.util.PairList<java.lang.String,java.lang.Object> params)
Block
public void initializeChildBlocks(NDManager manager, DataType dataType, Shape... inputShapes)
AbstractBlock
initializeChildBlocks
in class AbstractBlock
manager
- the manager to use for initializationdataType
- the requested data typeinputShapes
- the expected input shapes for this blockpublic Shape[] getOutputShapes(NDManager manager, Shape[] inputShapes)
Block
getOutputShapes
in interface Block
manager
- an NDManagerinputShapes
- the shapes of the inputs