public final class ScaledDotProductAttentionBlock extends AbstractBlock
Abbreviations used:
In many use cases F=T. For self attention, the input is equal to the output.
This block can process input in four forms:
Attention masks must contain a 1 for positions to keep and a 0 for positions to mask.
Modifier and Type | Class and Description |
---|---|
static class |
ScaledDotProductAttentionBlock.Builder
A builder for
ScaledDotProductAttentionBlock s. |
children, inputNames, inputShapes, parameters, version
Modifier and Type | Method and Description |
---|---|
static ScaledDotProductAttentionBlock.Builder |
builder()
Creates a new Builder to build an Attention Block with.
|
protected NDList |
forwardInternal(ParameterStore parameterStore,
NDList inputs,
boolean training,
ai.djl.util.PairList<java.lang.String,java.lang.Object> params)
A helper for
Block.forward(ParameterStore, NDList, boolean, PairList) after
initialization. |
Linear |
getKeyProjection()
Pointwise Linear projection of the keys.
|
Shape[] |
getOutputShapes(Shape[] inputShapes)
Returns the expected output shapes of the block for the specified input shapes.
|
Linear |
getQueryProjection()
Pointwise Linear projection of the queries.
|
Linear |
getResultProjection()
Pointwise Linear projection of the results.
|
Linear |
getValueProjection()
Pointwise Linear projection of the values.
|
void |
initializeChildBlocks(NDManager manager,
DataType dataType,
Shape... inputShapes)
Initializes the Child blocks of this block.
|
addChildBlock, addParameter, beforeInitialize, cast, clear, describeInput, forward, forward, forwardInternal, getChildren, getDirectParameters, getParameters, initialize, isInitialized, loadMetadata, loadParameters, prepare, readInputShapes, saveInputShapes, saveMetadata, saveParameters, setInitializer, setInitializer, setInitializer, toString
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
forward, validateLayout
public Linear getKeyProjection()
public Linear getQueryProjection()
public Linear getValueProjection()
public Linear getResultProjection()
public Shape[] getOutputShapes(Shape[] inputShapes)
inputShapes
- the shapes of the inputspublic void initializeChildBlocks(NDManager manager, DataType dataType, Shape... inputShapes)
initializeChildBlocks
in class AbstractBlock
manager
- the manager to use for initializationdataType
- the requested data typeinputShapes
- the expected input shapes for this blockprotected NDList forwardInternal(ParameterStore parameterStore, NDList inputs, boolean training, ai.djl.util.PairList<java.lang.String,java.lang.Object> params)
Block.forward(ParameterStore, NDList, boolean, PairList)
after
initialization.forwardInternal
in class AbstractBlock
parameterStore
- the parameter storeinputs
- the input NDListtraining
- true for a training forward passparams
- optional parameterspublic static ScaledDotProductAttentionBlock.Builder builder()