public class TransformerEncoderBlock extends AbstractBlock
children, inputNames, inputShapes, parameters, version
Constructor and Description |
---|
TransformerEncoderBlock(int embeddingSize,
int headCount,
int hiddenSize,
float dropoutProbability,
java.util.function.Function<NDList,NDList> activationFunction)
Creates a transformer encoder block.
|
Modifier and Type | Method and Description |
---|---|
protected NDList |
forwardInternal(ParameterStore ps,
NDList inputs,
boolean training,
ai.djl.util.PairList<java.lang.String,java.lang.Object> params)
A helper for
Block.forward(ParameterStore, NDList, boolean, PairList) after
initialization. |
Shape[] |
getOutputShapes(Shape[] inputShapes)
Returns the expected output shapes of the block for the specified input shapes.
|
void |
initializeChildBlocks(NDManager manager,
DataType dataType,
Shape... inputShapes)
Initializes the Child blocks of this block.
|
addChildBlock, addParameter, beforeInitialize, cast, clear, describeInput, forward, forward, forwardInternal, getChildren, getDirectParameters, getParameters, initialize, isInitialized, loadMetadata, loadParameters, prepare, readInputShapes, saveInputShapes, saveMetadata, saveParameters, setInitializer, setInitializer, setInitializer, toString
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
forward, validateLayout
public TransformerEncoderBlock(int embeddingSize, int headCount, int hiddenSize, float dropoutProbability, java.util.function.Function<NDList,NDList> activationFunction)
embeddingSize
- the embedding size for tokensheadCount
- number of attention blockshiddenSize
- the hidden size for fully connected networksdropoutProbability
- dropout probabilityactivationFunction
- activation functionpublic Shape[] getOutputShapes(Shape[] inputShapes)
inputShapes
- the shapes of the inputspublic void initializeChildBlocks(NDManager manager, DataType dataType, Shape... inputShapes)
initializeChildBlocks
in class AbstractBlock
manager
- the manager to use for initializationdataType
- the requested data typeinputShapes
- the expected input shapes for this blockprotected NDList forwardInternal(ParameterStore ps, NDList inputs, boolean training, ai.djl.util.PairList<java.lang.String,java.lang.Object> params)
Block.forward(ParameterStore, NDList, boolean, PairList)
after
initialization.forwardInternal
in class AbstractBlock
ps
- the parameter storeinputs
- the input NDListtraining
- true for a training forward passparams
- optional parameters