Rank3TensorLayer
is the number of input. (vector1, vector2, entire).
is the number of output
is an activation function to be applied
is initial quadratic-level weight matrix Q for the case that it is restored from JSON (default: Seq())
is initial linear-level weight matrix L for the case that it is restored from JSON (default: Seq())
is initial bias weight matrix b for the case that it is restored from JSON (default: null)
Retrieve first input
Retrieve first input
input to be separated
first input
Retrive second input
Retrive second input
input to be separated
second input
Translate this layer into JSON object (in Play! framework)
Translate this layer into JSON object (in Play! framework)
JSON object describes this layer
weights for update
is an activation function to be applied
is an activation function to be applied
Forward computation
Forward computation
input matrix
output matrix
accumulated delta values
is the number of input.
is the number of input. (vector1, vector2, entire).
is the number of output
is the number of output
Sugar: Forward computation.
Sugar: Forward computation. Calls apply(x)
input matrix
output matrix
Backward computation.
Backward computation.
to be propagated ( dG / dF is propagated from higher layer )
of this layer (in this case, x = entry of dX / dw)
of this layer (in this case, y)
propagated error (in this case, dG/dx )
Let this layer have function F composed with function X(x) = x1'.Q.x2 + L.x + b
and higher layer have function G. (Each output is treated as separately except propagation)
Weight is updated with: dG/dW
and propagate dG/dx
For the computation, we only used denominator layout. (cf. Wikipedia Page of Matrix Computation) For the computation rules, see "Matrix Cookbook" from MIT.
Layer: Basic, Fully-connected Rank 3 Tensor Layer.
v0 = a column vector concatenate v2 after v1 (v11, v12, ... v1in1, v21, ...) Q = Rank 3 Tensor with size out, in1 × in2 is its entry. L = Rank 3 Tensor with size out, 1 × (in1 + in2) is its entry. b = out × 1 matrix. output = f( v1'.Q.v2 + L.v0 + b )