Package org.deeplearning4j.nn.layers
Class LossLayer
- java.lang.Object
-
- org.deeplearning4j.nn.layers.AbstractLayer<LayerConfT>
-
- org.deeplearning4j.nn.layers.BaseLayer<LossLayer>
-
- org.deeplearning4j.nn.layers.LossLayer
-
- All Implemented Interfaces:
Serializable
,Cloneable
,Classifier
,Layer
,IOutputLayer
,Model
,Trainable
public class LossLayer extends BaseLayer<LossLayer> implements Serializable, IOutputLayer
- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.deeplearning4j.nn.api.Layer
Layer.TrainingMode, Layer.Type
-
-
Field Summary
Fields Modifier and Type Field Description protected INDArray
labels
-
Fields inherited from class org.deeplearning4j.nn.layers.BaseLayer
gradient, gradientsFlattened, gradientViews, optimizer, params, paramsFlattened, score, weightNoiseParams
-
Fields inherited from class org.deeplearning4j.nn.layers.AbstractLayer
cacheMode, conf, dataType, dropoutApplied, epochCount, index, input, inputModificationAllowed, iterationCount, maskArray, maskState, preOutput, trainingListeners
-
-
Constructor Summary
Constructors Constructor Description LossLayer(NeuralNetConfiguration conf, DataType dataType)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description INDArray
activate(boolean training, LayerWorkspaceMgr workspaceMgr)
Perform forward pass and return the activations array with the last set inputINDArray
activate(INDArray input, boolean training, LayerWorkspaceMgr workspaceMgr)
Perform forward pass and return the activations array with the specified inputPair<Gradient,INDArray>
backpropGradient(INDArray epsilon, LayerWorkspaceMgr workspaceMgr)
Calculate the gradient relative to the error in the next layerdouble
calcRegularizationScore(boolean backpropOnlyParams)
Calculate the regularization component of the score, for the parameters in this layer
For example, the L1, L2 and/or weight decay components of the loss functionvoid
clear()
Clear inputvoid
computeGradientAndScore(LayerWorkspaceMgr workspaceMgr)
Update the scoredouble
computeScore(double fullNetRegTerm, boolean training, LayerWorkspaceMgr workspaceMgr)
Compute score after labels and input have been set.INDArray
computeScoreForExamples(double fullNetRegTerm, LayerWorkspaceMgr workspaceMgr)
Compute the score for each example individually, after labels and input have been set.double
f1Score(INDArray examples, INDArray labels)
Returns the f1 score for the given examples.double
f1Score(DataSet data)
Sets the input and labels and returns a score for the prediction wrt true labelsvoid
fit(INDArray examples, int[] labels)
Fit the modelvoid
fit(INDArray input, LayerWorkspaceMgr workspaceMgr)
Fit the model to the given datavoid
fit(INDArray input, INDArray labels)
Fit the modelvoid
fit(DataSet data)
Fit the modelvoid
fit(DataSetIterator iter)
Train the model based on the datasetiteratorINDArray
getLabels()
Get the labels array previously set withIOutputLayer.setLabels(INDArray)
protected INDArray
getLabels2d()
Gradient
gradient()
Gets the gradient from one training iterationPair<Gradient,Double>
gradientAndScore()
Get the gradient and scoreboolean
isPretrainLayer()
Returns true if the layer can be trained in an unsupervised/pretrain manner (AE, VAE, etc)boolean
needsLabels()
Returns true if labels are required for this output layerint
numLabels()
Returns the number of possible labelsINDArray
params()
Returns the parameters of the neural network as a flattened row vectorint[]
predict(INDArray input)
Returns the predictions for each example in the datasetList<String>
predict(DataSet dataSet)
Return predicted label namesvoid
setLabels(INDArray labels)
Set the labels array for this output layerprotected void
setScoreWithZ(INDArray z)
Layer.Type
type()
Returns the layer type-
Methods inherited from class org.deeplearning4j.nn.layers.BaseLayer
clearNoiseWeightParams, clone, fit, getGradientsViewArray, getOptimizer, getParam, getParamWithNoise, hasBias, hasLayerNorm, layerConf, numParams, paramTable, paramTable, preOutput, preOutputWithPreNorm, score, setBackpropGradientsViewArray, setParam, setParams, setParams, setParamsViewArray, setParamTable, toString, update, update
-
Methods inherited from class org.deeplearning4j.nn.layers.AbstractLayer
addListeners, allowInputModification, applyConstraints, applyDropOutIfNecessary, applyMask, assertInputSet, backpropDropOutIfPresent, batchSize, close, conf, feedForwardMaskArray, getConfig, getEpochCount, getHelper, getIndex, getInput, getInputMiniBatchSize, getListeners, getMaskArray, init, input, layerId, numParams, setCacheMode, setConf, setEpochCount, setIndex, setInput, setInputMiniBatchSize, setListeners, setListeners, setMaskArray, updaterDivideByMinibatch
-
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.deeplearning4j.nn.api.Layer
allowInputModification, clearNoiseWeightParams, feedForwardMaskArray, getEpochCount, getHelper, getIndex, getInputMiniBatchSize, getIterationCount, getListeners, getMaskArray, setCacheMode, setEpochCount, setIndex, setInput, setInputMiniBatchSize, setIterationCount, setListeners, setListeners, setMaskArray
-
Methods inherited from interface org.deeplearning4j.nn.api.Model
addListeners, applyConstraints, batchSize, close, conf, fit, getGradientsViewArray, getOptimizer, getParam, init, input, numParams, numParams, paramTable, paramTable, score, setBackpropGradientsViewArray, setConf, setParam, setParams, setParamsViewArray, setParamTable, update, update
-
Methods inherited from interface org.deeplearning4j.nn.api.Trainable
getConfig, getGradientsViewArray, numParams, paramTable, updaterDivideByMinibatch
-
-
-
-
Field Detail
-
labels
protected INDArray labels
-
-
Constructor Detail
-
LossLayer
public LossLayer(NeuralNetConfiguration conf, DataType dataType)
-
-
Method Detail
-
computeScore
public double computeScore(double fullNetRegTerm, boolean training, LayerWorkspaceMgr workspaceMgr)
Compute score after labels and input have been set.- Specified by:
computeScore
in interfaceIOutputLayer
- Parameters:
fullNetRegTerm
- Regularization score term for the entire networktraining
- whether score should be calculated at train or test time (this affects things like application of dropout, etc)- Returns:
- score (loss function)
-
computeScoreForExamples
public INDArray computeScoreForExamples(double fullNetRegTerm, LayerWorkspaceMgr workspaceMgr)
Compute the score for each example individually, after labels and input have been set.- Specified by:
computeScoreForExamples
in interfaceIOutputLayer
- Parameters:
fullNetRegTerm
- Regularization score term for the entire network (or, 0.0 to not include regularization)- Returns:
- A column INDArray of shape [numExamples,1], where entry i is the score of the ith example
-
computeGradientAndScore
public void computeGradientAndScore(LayerWorkspaceMgr workspaceMgr)
Description copied from interface:Model
Update the score- Specified by:
computeGradientAndScore
in interfaceModel
- Overrides:
computeGradientAndScore
in classBaseLayer<LossLayer>
-
setScoreWithZ
protected void setScoreWithZ(INDArray z)
- Overrides:
setScoreWithZ
in classBaseLayer<LossLayer>
-
gradientAndScore
public Pair<Gradient,Double> gradientAndScore()
Description copied from interface:Model
Get the gradient and score- Specified by:
gradientAndScore
in interfaceModel
- Overrides:
gradientAndScore
in classAbstractLayer<LossLayer>
- Returns:
- the gradient and score
-
backpropGradient
public Pair<Gradient,INDArray> backpropGradient(INDArray epsilon, LayerWorkspaceMgr workspaceMgr)
Description copied from interface:Layer
Calculate the gradient relative to the error in the next layer- Specified by:
backpropGradient
in interfaceLayer
- Overrides:
backpropGradient
in classBaseLayer<LossLayer>
- Parameters:
epsilon
- w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C is cost function a=sigma(z) is activation.workspaceMgr
- Workspace manager- Returns:
- Pair
where Gradient is gradient for this layer, INDArray is epsilon (activation gradient) needed by next layer, but before element-wise multiply by sigmaPrime(z). So for standard feed-forward layer, if this layer is L, then return.getSecond() == dL/dIn = (w^(L)*(delta^(L))^T)^T. Note that the returned array should be placed in the ArrayType.ACTIVATION_GRAD
workspace via the workspace manager
-
gradient
public Gradient gradient()
Gets the gradient from one training iteration
-
calcRegularizationScore
public double calcRegularizationScore(boolean backpropOnlyParams)
Description copied from interface:Layer
Calculate the regularization component of the score, for the parameters in this layer
For example, the L1, L2 and/or weight decay components of the loss function- Specified by:
calcRegularizationScore
in interfaceLayer
- Overrides:
calcRegularizationScore
in classBaseLayer<LossLayer>
- Parameters:
backpropOnlyParams
- If true: calculate regularization score based on backprop params only. If false: calculate based on all params (including pretrain params, if any)- Returns:
- the regularization score of
-
type
public Layer.Type type()
Description copied from interface:Layer
Returns the layer type- Specified by:
type
in interfaceLayer
- Overrides:
type
in classAbstractLayer<LossLayer>
- Returns:
-
fit
public void fit(INDArray input, LayerWorkspaceMgr workspaceMgr)
Description copied from interface:Model
Fit the model to the given data
-
activate
public INDArray activate(boolean training, LayerWorkspaceMgr workspaceMgr)
Description copied from interface:Layer
Perform forward pass and return the activations array with the last set input- Specified by:
activate
in interfaceLayer
- Overrides:
activate
in classBaseLayer<LossLayer>
- Parameters:
training
- training or test modeworkspaceMgr
- Workspace manager- Returns:
- the activation (layer output) of the last specified input. Note that the returned array should be placed
in the
ArrayType.ACTIVATIONS
workspace via the workspace manager
-
activate
public INDArray activate(INDArray input, boolean training, LayerWorkspaceMgr workspaceMgr)
Description copied from interface:Layer
Perform forward pass and return the activations array with the specified input- Specified by:
activate
in interfaceLayer
- Overrides:
activate
in classAbstractLayer<LossLayer>
- Parameters:
input
- the input to usetraining
- train or test modeworkspaceMgr
- Workspace manager.- Returns:
- Activations array. Note that the returned array should be placed in the
ArrayType.ACTIVATIONS
workspace via the workspace manager
-
isPretrainLayer
public boolean isPretrainLayer()
Description copied from interface:Layer
Returns true if the layer can be trained in an unsupervised/pretrain manner (AE, VAE, etc)- Specified by:
isPretrainLayer
in interfaceLayer
- Returns:
- true if the layer can be pretrained (using fit(INDArray), false otherwise
-
params
public INDArray params()
Description copied from class:BaseLayer
Returns the parameters of the neural network as a flattened row vector
-
f1Score
public double f1Score(DataSet data)
Sets the input and labels and returns a score for the prediction wrt true labels- Specified by:
f1Score
in interfaceClassifier
- Parameters:
data
- the data to score- Returns:
- the score for the given input,label pairs
-
f1Score
public double f1Score(INDArray examples, INDArray labels)
Returns the f1 score for the given examples. Think of this to be like a percentage right. The higher the number the more it got right. This is on a scale from 0 to 1.- Specified by:
f1Score
in interfaceClassifier
- Parameters:
examples
- te the examples to classify (one example in each row)labels
- the true labels- Returns:
- the scores for each ndarray
-
numLabels
public int numLabels()
Returns the number of possible labels- Specified by:
numLabels
in interfaceClassifier
- Returns:
- the number of possible labels for this classifier
-
fit
public void fit(DataSetIterator iter)
Description copied from interface:Classifier
Train the model based on the datasetiterator- Specified by:
fit
in interfaceClassifier
- Parameters:
iter
- the iterator to train on
-
predict
public int[] predict(INDArray input)
Returns the predictions for each example in the dataset- Specified by:
predict
in interfaceClassifier
- Parameters:
input
- the matrix to predict- Returns:
- the prediction for the dataset
-
predict
public List<String> predict(DataSet dataSet)
Return predicted label names- Specified by:
predict
in interfaceClassifier
- Parameters:
dataSet
- to predict- Returns:
- the predicted labels for the dataSet
-
fit
public void fit(INDArray input, INDArray labels)
Fit the model- Specified by:
fit
in interfaceClassifier
- Parameters:
input
- the examples to classify (one example in each row)labels
- the example labels(a binary outcome matrix)
-
fit
public void fit(DataSet data)
Fit the model- Specified by:
fit
in interfaceClassifier
- Parameters:
data
- the data to train on
-
fit
public void fit(INDArray examples, int[] labels)
Fit the model- Specified by:
fit
in interfaceClassifier
- Parameters:
examples
- the examples to classify (one example in each row)labels
- the labels for each example (the number of labels must match
-
clear
public void clear()
Description copied from interface:Model
Clear input
-
getLabels
public INDArray getLabels()
Description copied from interface:IOutputLayer
Get the labels array previously set withIOutputLayer.setLabels(INDArray)
- Specified by:
getLabels
in interfaceIOutputLayer
- Returns:
- Labels array, or null if it has not been set
-
needsLabels
public boolean needsLabels()
Description copied from interface:IOutputLayer
Returns true if labels are required for this output layer- Specified by:
needsLabels
in interfaceIOutputLayer
- Returns:
- true if this output layer needs labels or not
-
setLabels
public void setLabels(INDArray labels)
Description copied from interface:IOutputLayer
Set the labels array for this output layer- Specified by:
setLabels
in interfaceIOutputLayer
- Parameters:
labels
- Labels array to set
-
getLabels2d
protected INDArray getLabels2d()
-
-