public class LossMultiLabel extends Object implements ILossFunction
This Loss function requires that the Labels are given as a multi-hot encoded vector. It doesn't require any special Activation method, i.e. the network output doesn't have to be in any specific range.
The loss is calculated based on the classification difference on labels that the examples has, and those that it doesn't have. Assume that each example has a set of labels, these labels are the positive set, the labels that do not belong to the example are in the negative set. This loss function trains the network to produce a higher value for labels that are in the positive set than those that are in the negative set.
Notice that in order to learn anything at all, this loss function requires that your example labels are not all 0 or all 1. In these cases the loss gradient will be 0. If you have to work with examples like that, you should try using a ComputationGraph with two LossLayers, one using LossMultiLabel and the other one using LossBinaryXENT.
For a more detailed explanation and the actual formulas, read the original paper by Zhang and Zhou. The implementation on scoreArray is based on equation 3, while computeGradient is based on equation 11. The main difference being that -(c_k - c_l) = (c_l - c_k) was used to simplify the calculations.
Min-Ling Zhang and Zhi-Hua Zhou, "Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization," in IEEE Transactions on Knowledge and Data Engineering, vol. 18, no. 10, pp. 1338-1351, Oct. 2006. doi: 10.1109/TKDE.2006.162 *
Constructor and Description |
---|
LossMultiLabel() |
Modifier and Type | Method and Description |
---|---|
INDArray |
computeGradient(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask)
Compute the gradient of the loss function with respect to the inputs: dL/dOutput
|
Pair<Double,INDArray> |
computeGradientAndScore(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask,
boolean average)
Compute both the score (loss function value) and gradient.
|
double |
computeScore(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask,
boolean average)
Compute the score (loss function value) for the given inputs.
|
INDArray |
computeScoreArray(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask)
Compute the score (loss function value) for each example individually.
|
String |
name()
The opName of this function
|
INDArray |
scoreArray(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask) |
String |
toString() |
public INDArray scoreArray(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask)
public double computeScore(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask, boolean average)
ILossFunction
computeScore
in interface ILossFunction
labels
- Label/expected preOutputpreOutput
- Output of the model (neural network)activationFn
- Activation function that should be applied to preOutputmask
- Mask array; may be nullaverage
- Whether the score should be averaged (divided by number of rows in labels/preOutput) or not @return Loss function valuepublic INDArray computeScoreArray(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask)
ILossFunction
computeScoreArray
in interface ILossFunction
labels
- Labels/expected outputpreOutput
- Output of the model (neural network)activationFn
- Activation function that should be applied to preOutputmask
- @return Loss function value for each example; column vectorpublic INDArray computeGradient(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask)
ILossFunction
computeGradient
in interface ILossFunction
labels
- Label/expected outputpreOutput
- Output of the model (neural network), before the activation function is appliedactivationFn
- Activation function that should be applied to preOutputmask
- Mask array; may be nullpublic Pair<Double,INDArray> computeGradientAndScore(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask, boolean average)
ILossFunction
ILossFunction.computeScore(INDArray, INDArray, IActivation, INDArray, boolean)
and ILossFunction.computeGradient(INDArray, INDArray, IActivation, INDArray)
individuallycomputeGradientAndScore
in interface ILossFunction
labels
- Label/expected outputpreOutput
- Output of the model (neural network)activationFn
- Activation function that should be applied to preOutputmask
- Mask array; may be nullaverage
- Whether the score should be averaged (divided by number of rows in labels/output) or notpublic String name()
ILossFunction
name
in interface ILossFunction
Copyright © 2020. All rights reserved.