public abstract class BaseWrapperLayer extends Object implements Layer
Layer.TrainingMode, Layer.Type| Modifier and Type | Field and Description |
|---|---|
protected Layer |
underlying |
| Constructor and Description |
|---|
BaseWrapperLayer(@NonNull Layer underlying) |
| Modifier and Type | Method and Description |
|---|---|
INDArray |
activate(boolean training,
LayerWorkspaceMgr workspaceMgr)
Perform forward pass and return the activations array with the last set input
|
INDArray |
activate(INDArray input,
boolean training,
LayerWorkspaceMgr workspaceMgr)
Perform forward pass and return the activations array with the specified input
|
void |
addListeners(TrainingListener... listener)
This method ADDS additional TrainingListener to existing listeners
|
void |
allowInputModification(boolean allow)
A performance optimization: mark whether the layer is allowed to modify its input array in-place.
|
void |
applyConstraints(int iteration,
int epoch)
Apply any constraints to the model
|
Pair<Gradient,INDArray> |
backpropGradient(INDArray epsilon,
LayerWorkspaceMgr workspaceMgr)
Calculate the gradient relative to the error in the next layer
|
int |
batchSize()
The current inputs batch size
|
double |
calcRegularizationScore(boolean backpropParamsOnly)
Calculate the regularization component of the score, for the parameters in this layer
For example, the L1, L2 and/or weight decay components of the loss function |
void |
clear()
Clear input
|
void |
clearNoiseWeightParams() |
void |
close() |
void |
computeGradientAndScore(LayerWorkspaceMgr workspaceMgr)
Update the score
|
NeuralNetConfiguration |
conf()
The configuration for the neural network
|
Pair<INDArray,MaskState> |
feedForwardMaskArray(INDArray maskArray,
MaskState currentMaskState,
int minibatchSize)
Feed forward the input mask array, setting in the layer as appropriate.
|
void |
fit()
All models have a fit method
|
void |
fit(INDArray data,
LayerWorkspaceMgr workspaceMgr)
Fit the model to the given data
|
TrainingConfig |
getConfig() |
int |
getEpochCount() |
INDArray |
getGradientsViewArray() |
LayerHelper |
getHelper() |
int |
getIndex()
Get the layer index.
|
int |
getInputMiniBatchSize()
Get current/last input mini-batch size, as set by setInputMiniBatchSize(int)
|
int |
getIterationCount() |
Collection<TrainingListener> |
getListeners()
Get the iteration listeners for this layer.
|
INDArray |
getMaskArray() |
ConvexOptimizer |
getOptimizer()
Returns this models optimizer
|
INDArray |
getParam(String param)
Get the parameter
|
Gradient |
gradient()
Get the gradient.
|
Pair<Gradient,Double> |
gradientAndScore()
Get the gradient and score
|
void |
init()
Init the model
|
INDArray |
input()
The input/feature matrix for the model
|
boolean |
isPretrainLayer()
Returns true if the layer can be trained in an unsupervised/pretrain manner (AE, VAE, etc)
|
long |
numParams()
the number of parameters for the model
|
long |
numParams(boolean backwards)
the number of parameters for the model
|
INDArray |
params()
Parameters of the model (if any)
|
Map<String,INDArray> |
paramTable()
The param table
|
Map<String,INDArray> |
paramTable(boolean backpropParamsOnly)
Table of parameters by key, for backprop
For many models (dense layers, etc) - all parameters are backprop parameters
|
double |
score()
The score for the model
|
void |
setBackpropGradientsViewArray(INDArray gradients)
Set the gradients array as a view of the full (backprop) network parameters
NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
|
void |
setCacheMode(CacheMode mode)
This method sets given CacheMode for current layer
|
void |
setConf(NeuralNetConfiguration conf)
Setter for the configuration
|
void |
setEpochCount(int epochCount)
Set the current epoch count (number of epochs passed ) for the layer/network
|
void |
setIndex(int index)
Set the layer index.
|
void |
setInput(INDArray input,
LayerWorkspaceMgr workspaceMgr)
Set the layer input.
|
void |
setInputMiniBatchSize(int size)
Set current/last input mini-batch size.
Used for score and gradient calculations. |
void |
setIterationCount(int iterationCount)
Set the current iteration count (number of parameter updates) for the layer/network
|
void |
setListeners(Collection<TrainingListener> listeners)
Set the
TrainingListeners for this model. |
void |
setListeners(TrainingListener... listeners)
Set the
TrainingListeners for this model. |
void |
setMaskArray(INDArray maskArray)
Set the mask array.
|
void |
setParam(String key,
INDArray val)
Set the parameter with a new ndarray
|
void |
setParams(INDArray params)
Set the parameters for this model.
|
void |
setParamsViewArray(INDArray params)
Set the initial parameters array as a view of the full (backprop) network parameters
NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
|
void |
setParamTable(Map<String,INDArray> paramTable)
Setter for the param table
|
Layer.Type |
type()
Returns the layer type
|
void |
update(Gradient gradient)
Update layer weights and biases with gradient change
|
void |
update(INDArray gradient,
String paramType)
Perform one update applying the gradient
|
boolean |
updaterDivideByMinibatch(String paramName)
DL4J layers typically produce the sum of the gradients during the backward pass for each layer, and if required
(if minibatch=true) then divide by the minibatch size.
However, there are some exceptions, such as the batch norm mean/variance estimate parameters: these "gradients" are actually not gradients, but are updates to be applied directly to the parameter vector. |
protected Layer underlying
public BaseWrapperLayer(@NonNull
@NonNull Layer underlying)
public void setCacheMode(CacheMode mode)
LayersetCacheMode in interface Layerpublic double calcRegularizationScore(boolean backpropParamsOnly)
LayercalcRegularizationScore in interface LayerbackpropParamsOnly - If true: calculate regularization score based on backprop params only. If false: calculate
based on all params (including pretrain params, if any)public Layer.Type type()
Layerpublic Pair<Gradient,INDArray> backpropGradient(INDArray epsilon, LayerWorkspaceMgr workspaceMgr)
LayerbackpropGradient in interface Layerepsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C
is cost function a=sigma(z) is activation.workspaceMgr - Workspace managerArrayType.ACTIVATION_GRAD workspace via the workspace managerpublic INDArray activate(boolean training, LayerWorkspaceMgr workspaceMgr)
Layeractivate in interface Layertraining - training or test modeworkspaceMgr - Workspace managerArrayType.ACTIVATIONS workspace via the workspace managerpublic INDArray activate(INDArray input, boolean training, LayerWorkspaceMgr workspaceMgr)
Layeractivate in interface Layerinput - the input to usetraining - train or test modeworkspaceMgr - Workspace manager.ArrayType.ACTIVATIONS workspace via the workspace managerpublic Collection<TrainingListener> getListeners()
LayergetListeners in interface Layerpublic void setListeners(TrainingListener... listeners)
LayerTrainingListeners for this model. If any listeners have previously been set, they will be
replaced by this methodsetListeners in interface LayersetListeners in interface Modelpublic void addListeners(TrainingListener... listener)
ModeladdListeners in interface Modelpublic void fit()
Modelpublic void update(Gradient gradient)
Modelpublic void update(INDArray gradient, String paramType)
Modelpublic double score()
Modelpublic void computeGradientAndScore(LayerWorkspaceMgr workspaceMgr)
ModelcomputeGradientAndScore in interface Modelpublic INDArray params()
Modelpublic long numParams()
Modelpublic long numParams(boolean backwards)
Modelpublic void setParams(INDArray params)
Modelpublic void setParamsViewArray(INDArray params)
ModelsetParamsViewArray in interface Modelparams - a 1 x nParams row vector that is a view of the larger (MLN/CG) parameters arraypublic INDArray getGradientsViewArray()
getGradientsViewArray in interface ModelgetGradientsViewArray in interface Trainablepublic void setBackpropGradientsViewArray(INDArray gradients)
ModelsetBackpropGradientsViewArray in interface Modelgradients - a 1 x nParams row vector that is a view of the larger (MLN/CG) gradients arraypublic void fit(INDArray data, LayerWorkspaceMgr workspaceMgr)
Modelpublic Gradient gradient()
ModelModel.computeGradientAndScore(LayerWorkspaceMgr) } .public Pair<Gradient,Double> gradientAndScore()
ModelgradientAndScore in interface Modelpublic int batchSize()
Modelpublic NeuralNetConfiguration conf()
Modelpublic void setConf(NeuralNetConfiguration conf)
Modelpublic INDArray input()
Modelpublic ConvexOptimizer getOptimizer()
ModelgetOptimizer in interface Modelpublic INDArray getParam(String param)
Modelpublic Map<String,INDArray> paramTable()
ModelparamTable in interface Modelpublic Map<String,INDArray> paramTable(boolean backpropParamsOnly)
ModelparamTable in interface ModelparamTable in interface TrainablebackpropParamsOnly - If true, return backprop params only. If false: return all params (equivalent to
paramsTable())public void setParamTable(Map<String,INDArray> paramTable)
ModelsetParamTable in interface Modelpublic void setParam(String key, INDArray val)
Modelpublic void clear()
Modelpublic void applyConstraints(int iteration,
int epoch)
ModelapplyConstraints in interface Modelpublic void init()
Modelpublic void setListeners(Collection<TrainingListener> listeners)
LayerTrainingListeners for this model. If any listeners have previously been set, they will be
replaced by this methodsetListeners in interface LayersetListeners in interface Modelpublic void setIndex(int index)
Layerpublic int getIndex()
Layerpublic int getIterationCount()
getIterationCount in interface Layerpublic int getEpochCount()
getEpochCount in interface Layerpublic void setIterationCount(int iterationCount)
LayersetIterationCount in interface Layerpublic void setEpochCount(int epochCount)
LayersetEpochCount in interface Layerpublic void setInput(INDArray input, LayerWorkspaceMgr workspaceMgr)
Layerpublic void setInputMiniBatchSize(int size)
LayersetInputMiniBatchSize in interface Layerpublic int getInputMiniBatchSize()
LayergetInputMiniBatchSize in interface LayerLayer.setInputMiniBatchSize(int)public void setMaskArray(INDArray maskArray)
LayerLayer.feedForwardMaskArray(INDArray, MaskState, int) should be used in
preference to this.setMaskArray in interface LayermaskArray - Mask array to setpublic INDArray getMaskArray()
getMaskArray in interface Layerpublic boolean isPretrainLayer()
LayerisPretrainLayer in interface Layerpublic void clearNoiseWeightParams()
clearNoiseWeightParams in interface Layerpublic Pair<INDArray,MaskState> feedForwardMaskArray(INDArray maskArray, MaskState currentMaskState, int minibatchSize)
LayerfeedForwardMaskArray in interface LayermaskArray - Mask array to setcurrentMaskState - Current state of the mask - see MaskStateminibatchSize - Current minibatch size. Needs to be known as it cannot always be inferred from the activations
array due to reshaping (such as a DenseLayer within a recurrent neural network)public void allowInputModification(boolean allow)
LayerallowInputModification in interface Layerallow - If true: the input array is safe to modify. If false: the input array should be copied before it
is modified (i.e., in-place modifications are un-safe)public LayerHelper getHelper()
public TrainingConfig getConfig()
public boolean updaterDivideByMinibatch(String paramName)
TrainableupdaterDivideByMinibatch in interface TrainableparamName - Name of the parameterCopyright © 2020. All rights reserved.