Attention

Instance Constructors

new Attention(memory: Output, memorySequenceLengths: Output = null, checkInnerDimensionsDefined: Boolean = true, scoreMaskValue: Output = Float.NegativeInfinity, name: String = "Attention")(implicit evAS: Aux[AS, ASS])

memory
Memory to query; usually the output of an RNN encoder. Each tensor in the memory should be shaped [batchSize, maxTime, ...].
memorySequenceLengths
Sequence lengths for the batch entries in the memory. If provided, the memory tensor rows are masked with zeros for values past the respective sequence lengths.
checkInnerDimensionsDefined
If true, the memory argument's shape is checked to ensure that all but the two outermost dimensions of each tensor are fully defined.
scoreMaskValue
Scalar tensor containing the mask value to use for the attention scores before passing them to probability. Defaults to negative infinity. Note that this value is only used if memorySequenceLengths is not null.
name
Name prefix to use for all created ops.

Abstract Value Members

abstract def alignment(query: Output, previousState: AS): (Output, AS)

Computes an alignment tensor given the provided query and previous alignment tensor.
Computes an alignment tensor given the provided query and previous alignment tensor.
The previous alignment tensor is important for attention mechanisms that use the previous alignment to calculate the attention at the next time step, such as monotonic attention mechanisms.
TODO: Figure out how to generalize the "next state" functionality.
query
Query tensor.
previousState
Previous alignment tensor.
returns
Tuple containing the alignment tensor and the next attention state.
abstract def initialState: AS

Initial state value.
Initial state value.
This is important for attention mechanisms that use the previous alignment to calculate the alignment at the next time step (e.g., monotonic attention).
The default behavior is to return the same output as initialAlignment.
abstract def score(query: Output, state: AS): Output

Computes an alignment score for query.
Computes an alignment score for query.
query
Query tensor.
state
Current attention mechanism state (defaults to the previous alignment tensor). The data type of this tensor matches that of values and its shape is [batchSize, alignmentSize], where alignmentSize is the memory's maximum time.
returns
Score tensor.

Attributes
protected
abstract def stateSize: ASS

Concrete Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
lazy val alignmentSize: Output
final def asInstanceOf[T0]: T0

Definition Classes
Any
lazy val batchSize: Output
val checkInnerDimensionsDefined: Boolean

If true, the memory argument's shape is checked to ensure that all but the two outermost dimensions of each tensor are fully defined.
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
lazy val dataType: types.DataType
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
lazy val initialAlignment: Output

Initial alignment value.
Initial alignment value.
This is important for attention mechanisms that use the previous alignment to calculate the alignment at the next time step (e.g., monotonic attention).
The default behavior is to return a tensor of all zeros.
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
lazy val keys: Output
val memory: Output

Memory to query; usually the output of an RNN encoder.
Memory to query; usually the output of an RNN encoder. Each tensor in the memory should be shaped [batchSize, maxTime, ...].

Attributes
protected
val memorySequenceLengths: Output

Sequence lengths for the batch entries in the memory.
Sequence lengths for the batch entries in the memory. If provided, the memory tensor rows are masked with zeros for values past the respective sequence lengths.

Attributes
protected
val name: String

Name prefix to use for all created ops.
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def probability(score: Output, state: AS): Output

Computes alignment probabilities for score.
Computes alignment probabilities for score.
score
Alignment score tensor.
state
Current attention mechanism state (defaults to the previous alignment tensor). The data type of this tensor matches that of values and its shape is [batchSize, alignmentSize], where alignmentSize is the memory's maximum time.
returns
Alignment probabilities tensor.

Attributes
protected
val scoreMaskValue: Output

Scalar tensor containing the mask value to use for the attention scores before passing them to probability.
Scalar tensor containing the mask value to use for the attention scores before passing them to probability. Defaults to negative infinity. Note that this value is only used if memorySequenceLengths is not null.
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
lazy val values: Output
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Docs: object Attention | package attention

abstract class Attention[AS, ASS] extends AnyRef

Instance Constructors

new Attention(memory: Output, memorySequenceLengths: Output = null, checkInnerDimensionsDefined: Boolean = true, scoreMaskValue: Output = Float.NegativeInfinity, name: String = "Attention")(implicit evAS: Aux[AS, ASS])

Abstract Value Members

abstract def alignment(query: Output, previousState: AS): (Output, AS)

abstract def initialState: AS

abstract def score(query: Output, state: AS): Output

abstract def stateSize: ASS

Concrete Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

lazy val alignmentSize: Output

final def asInstanceOf[T0]: T0

lazy val batchSize: Output

val checkInnerDimensionsDefined: Boolean

def clone(): AnyRef

lazy val dataType: types.DataType

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

lazy val initialAlignment: Output

final def isInstanceOf[T0]: Boolean

lazy val keys: Output

val memory: Output

val memorySequenceLengths: Output

val name: String

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def probability(score: Output, state: AS): Output

val scoreMaskValue: Output

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

lazy val values: Output

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from AnyRef

Inherited from Any

Ungrouped