Package org.deeplearning4j.nn.layers
Class BasePretrainNetwork<LayerConfT extends BasePretrainNetwork>
- java.lang.Object
-
- org.deeplearning4j.nn.layers.AbstractLayer<LayerConfT>
-
- org.deeplearning4j.nn.layers.BaseLayer<LayerConfT>
-
- org.deeplearning4j.nn.layers.BasePretrainNetwork<LayerConfT>
-
- All Implemented Interfaces:
Serializable
,Cloneable
,Layer
,Model
,Trainable
- Direct Known Subclasses:
AutoEncoder
public abstract class BasePretrainNetwork<LayerConfT extends BasePretrainNetwork> extends BaseLayer<LayerConfT>
- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.deeplearning4j.nn.api.Layer
Layer.TrainingMode, Layer.Type
-
-
Field Summary
-
Fields inherited from class org.deeplearning4j.nn.layers.BaseLayer
gradient, gradientsFlattened, gradientViews, optimizer, params, paramsFlattened, score, solver, weightNoiseParams
-
Fields inherited from class org.deeplearning4j.nn.layers.AbstractLayer
cacheMode, conf, dataType, dropoutApplied, epochCount, index, input, inputModificationAllowed, iterationCount, maskArray, maskState, preOutput, trainingListeners
-
-
Constructor Summary
Constructors Constructor Description BasePretrainNetwork(NeuralNetConfiguration conf, DataType dataType)
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description Pair<Gradient,INDArray>
backpropGradient(INDArray epsilon, LayerWorkspaceMgr workspaceMgr)
Calculate the gradient relative to the error in the next layerdouble
calcRegularizationScore(boolean backpropParamsOnly)
Calculate the regularization component of the score, for the parameters in this layer
For example, the L1, L2 and/or weight decay components of the loss functionprotected Gradient
createGradient(INDArray wGradient, INDArray vBiasGradient, INDArray hBiasGradient)
INDArray
getCorruptedInput(INDArray x, double corruptionLevel)
Corrupts the given input by doing a binomial sampling given the corruption levellong
numParams()
The number of parameters for the model, for backprop (i.e., excluding visible bias)long
numParams(boolean backwards)
the number of parameters for the modelINDArray
params()
Returns the parameters of the neural network as a flattened row vectorMap<String,INDArray>
paramTable(boolean backpropParamsOnly)
Table of parameters by key, for backprop For many models (dense layers, etc) - all parameters are backprop parametersabstract Pair<INDArray,INDArray>
sampleHiddenGivenVisible(INDArray v)
Sample the hidden distribution given the visibleabstract Pair<INDArray,INDArray>
sampleVisibleGivenHidden(INDArray h)
Sample the visible distribution given the hiddenvoid
setParams(INDArray params)
Set the parameters for this model.protected void
setScoreWithZ(INDArray z)
-
Methods inherited from class org.deeplearning4j.nn.layers.BaseLayer
activate, clear, clearNoiseWeightParams, clone, computeGradientAndScore, fit, fit, getGradientsViewArray, getOptimizer, getParam, getParamWithNoise, gradient, hasBias, hasLayerNorm, layerConf, paramTable, preOutput, preOutputWithPreNorm, score, setBackpropGradientsViewArray, setParam, setParams, setParamsViewArray, setParamTable, toString, update, update
-
Methods inherited from class org.deeplearning4j.nn.layers.AbstractLayer
activate, addListeners, allowInputModification, applyConstraints, applyDropOutIfNecessary, applyMask, assertInputSet, backpropDropOutIfPresent, batchSize, close, conf, feedForwardMaskArray, getConfig, getEpochCount, getHelper, getIndex, getInput, getInputMiniBatchSize, getListeners, getMaskArray, gradientAndScore, init, input, layerId, setCacheMode, setConf, setEpochCount, setIndex, setInput, setInputMiniBatchSize, setListeners, setListeners, setMaskArray, type, updaterDivideByMinibatch
-
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.deeplearning4j.nn.api.Layer
getIterationCount, isPretrainLayer, setIterationCount
-
-
-
-
Constructor Detail
-
BasePretrainNetwork
public BasePretrainNetwork(NeuralNetConfiguration conf, DataType dataType)
-
-
Method Detail
-
getCorruptedInput
public INDArray getCorruptedInput(INDArray x, double corruptionLevel)
Corrupts the given input by doing a binomial sampling given the corruption level- Parameters:
x
- the input to corruptcorruptionLevel
- the corruption value- Returns:
- the binomial sampled corrupted input
-
createGradient
protected Gradient createGradient(INDArray wGradient, INDArray vBiasGradient, INDArray hBiasGradient)
-
numParams
public long numParams(boolean backwards)
Description copied from interface:Model
the number of parameters for the model- Specified by:
numParams
in interfaceModel
- Overrides:
numParams
in classAbstractLayer<LayerConfT extends BasePretrainNetwork>
- Returns:
- the number of parameters for the model
-
sampleHiddenGivenVisible
public abstract Pair<INDArray,INDArray> sampleHiddenGivenVisible(INDArray v)
Sample the hidden distribution given the visible- Parameters:
v
- the visible to sample from- Returns:
- the hidden mean and sample
-
sampleVisibleGivenHidden
public abstract Pair<INDArray,INDArray> sampleVisibleGivenHidden(INDArray h)
Sample the visible distribution given the hidden- Parameters:
h
- the hidden to sample from- Returns:
- the mean and sample
-
setScoreWithZ
protected void setScoreWithZ(INDArray z)
- Overrides:
setScoreWithZ
in classBaseLayer<LayerConfT extends BasePretrainNetwork>
-
paramTable
public Map<String,INDArray> paramTable(boolean backpropParamsOnly)
Description copied from interface:Model
Table of parameters by key, for backprop For many models (dense layers, etc) - all parameters are backprop parameters- Specified by:
paramTable
in interfaceModel
- Specified by:
paramTable
in interfaceTrainable
- Overrides:
paramTable
in classBaseLayer<LayerConfT extends BasePretrainNetwork>
- Parameters:
backpropParamsOnly
- If true, return backprop params only. If false: return all params (equivalent to paramsTable())- Returns:
- Parameter table
-
params
public INDArray params()
Description copied from class:BaseLayer
Returns the parameters of the neural network as a flattened row vector- Specified by:
params
in interfaceModel
- Specified by:
params
in interfaceTrainable
- Overrides:
params
in classBaseLayer<LayerConfT extends BasePretrainNetwork>
- Returns:
- the parameters of the neural network
-
numParams
public long numParams()
The number of parameters for the model, for backprop (i.e., excluding visible bias)- Specified by:
numParams
in interfaceModel
- Specified by:
numParams
in interfaceTrainable
- Overrides:
numParams
in classBaseLayer<LayerConfT extends BasePretrainNetwork>
- Returns:
- the number of parameters for the model (ex. visible bias)
-
setParams
public void setParams(INDArray params)
Description copied from interface:Model
Set the parameters for this model. This expects a linear ndarray which then be unpacked internally relative to the expected ordering of the model- Specified by:
setParams
in interfaceModel
- Overrides:
setParams
in classBaseLayer<LayerConfT extends BasePretrainNetwork>
- Parameters:
params
- the parameters for the model
-
backpropGradient
public Pair<Gradient,INDArray> backpropGradient(INDArray epsilon, LayerWorkspaceMgr workspaceMgr)
Description copied from interface:Layer
Calculate the gradient relative to the error in the next layer- Specified by:
backpropGradient
in interfaceLayer
- Overrides:
backpropGradient
in classBaseLayer<LayerConfT extends BasePretrainNetwork>
- Parameters:
epsilon
- w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C is cost function a=sigma(z) is activation.workspaceMgr
- Workspace manager- Returns:
- Pair
where Gradient is gradient for this layer, INDArray is epsilon (activation gradient) needed by next layer, but before element-wise multiply by sigmaPrime(z). So for standard feed-forward layer, if this layer is L, then return.getSecond() == dL/dIn = (w^(L)*(delta^(L))^T)^T. Note that the returned array should be placed in the ArrayType.ACTIVATION_GRAD
workspace via the workspace manager
-
calcRegularizationScore
public double calcRegularizationScore(boolean backpropParamsOnly)
Description copied from interface:Layer
Calculate the regularization component of the score, for the parameters in this layer
For example, the L1, L2 and/or weight decay components of the loss function- Specified by:
calcRegularizationScore
in interfaceLayer
- Overrides:
calcRegularizationScore
in classBaseLayer<LayerConfT extends BasePretrainNetwork>
- Parameters:
backpropParamsOnly
- If true: calculate regularization score based on backprop params only. If false: calculate based on all params (including pretrain params, if any)- Returns:
- the regularization score of
-
-