public class LossMultiLabel extends DifferentialFunction implements ILossFunction
This Loss function requires that the Labels are given as a multi-hot encoded vector. It doesn't require any special Activation method, i.e. the network output doesn't have to be in any specific range.
The loss is calculated based on the classification difference on labels that the examples has, and those that it doesn't have. Assume that each example has a set of labels, these labels are the positive set, the labels that do not belong to the example are in the negative set. This loss function trains the network to produce a higher value for labels that are in the positive set than those that are in the negative set.
For a more detailed explanation and the actual formulas, read the original paper by Zhang and Zhou. The implementation on scoreArray is based on equation 3, while computeGradient is based on equation 11. The main difference being that -(c_k - c_l) = (c_l - c_k) was used to simplify the calculations.
Min-Ling Zhang and Zhi-Hua Zhou, "Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization," in IEEE Transactions on Knowledge and Data Engineering, vol. 18, no. 10, pp. 1338-1351, Oct. 2006. doi: 10.1109/TKDE.2006.162 *
dimensions, extraArgs, inPlace, sameDiff, scalarValue
Constructor and Description |
---|
LossMultiLabel() |
Modifier and Type | Method and Description |
---|---|
INDArray |
computeGradient(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask)
Compute the gradient of the loss function with respect to the inputs: dL/dOutput
|
org.nd4j.linalg.primitives.Pair<Double,INDArray> |
computeGradientAndScore(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask,
boolean average)
Compute both the score (loss function value) and gradient.
|
double |
computeScore(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask,
boolean average)
Compute the score (loss function value) for the given inputs.
|
INDArray |
computeScoreArray(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask)
Compute the score (loss function value) for each example individually.
|
List<SDVariable> |
doDiff(List<SDVariable> f1)
The actual implementation for automatic differentiation.
|
void |
initFromOnnx(OnnxProto3.NodeProto node,
SameDiff initWith,
Map<String,OnnxProto3.AttributeProto> attributesForNode,
OnnxProto3.GraphProto graph)
Iniitialize the function from the given
OnnxProto3.NodeProto |
void |
initFromTensorFlow(NodeDef nodeDef,
SameDiff initWith,
Map<String,AttrValue> attributesForNode,
GraphDef graph)
Initialize the function from the given
NodeDef |
String |
name()
The opName of this function
|
String |
onnxName()
The opName of this function in onnx
|
String |
opName()
The name of the op
|
Op.Type |
opType()
The type of the op
|
SDVariable[] |
outputVariables()
Return the output variables for this differential function.
|
SDVariable[] |
outputVariables(String baseName)
Return the output functions for this differential function.
|
INDArray |
scoreArray(INDArray labels,
INDArray preOutput,
IActivation activationFn,
INDArray mask) |
String |
tensorflowName()
The opName of this function tensorflow
|
String |
toString() |
arg, args, asProperties, attributeAdaptersForFunction, calculateOutputShape, configFieldName, diff, dup, equals, f, getValue, hashCode, hasPlaceHolderInputs, isConfigProperties, larg, mappingsForFunction, onnxNames, opNum, propertiesForFunction, rarg, resolvePropertiesFromSameDiffBeforeExecution, setInstanceId, setValueFor, tensorflowNames
public INDArray scoreArray(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask)
public double computeScore(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask, boolean average)
ILossFunction
computeScore
in interface ILossFunction
labels
- Label/expected preOutputpreOutput
- Output of the model (neural network)activationFn
- Activation function that should be applied to preOutputmask
- Mask array; may be nullaverage
- Whether the score should be averaged (divided by number of rows in labels/preOutput) or not @return Loss function valuepublic INDArray computeScoreArray(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask)
ILossFunction
computeScoreArray
in interface ILossFunction
labels
- Labels/expected outputpreOutput
- Output of the model (neural network)activationFn
- Activation function that should be applied to preOutputmask
- @return Loss function value for each example; column vectorpublic INDArray computeGradient(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask)
ILossFunction
computeGradient
in interface ILossFunction
labels
- Label/expected outputpreOutput
- Output of the model (neural network), before the activation function is appliedactivationFn
- Activation function that should be applied to preOutputmask
- Mask array; may be nullpublic org.nd4j.linalg.primitives.Pair<Double,INDArray> computeGradientAndScore(INDArray labels, INDArray preOutput, IActivation activationFn, INDArray mask, boolean average)
ILossFunction
ILossFunction.computeScore(INDArray, INDArray, IActivation, INDArray, boolean)
and ILossFunction.computeGradient(INDArray, INDArray, IActivation, INDArray)
individuallycomputeGradientAndScore
in interface ILossFunction
labels
- Label/expected outputpreOutput
- Output of the model (neural network)activationFn
- Activation function that should be applied to preOutputmask
- Mask array; may be nullaverage
- Whether the score should be averaged (divided by number of rows in labels/output) or notpublic String name()
ILossFunction
name
in interface ILossFunction
public SDVariable[] outputVariables()
DifferentialFunction
outputVariables
in class DifferentialFunction
public SDVariable[] outputVariables(String baseName)
DifferentialFunction
outputVariables
in class DifferentialFunction
public List<SDVariable> doDiff(List<SDVariable> f1)
DifferentialFunction
doDiff
in class DifferentialFunction
public String opName()
DifferentialFunction
opName
in class DifferentialFunction
public Op.Type opType()
DifferentialFunction
opType
in class DifferentialFunction
public void initFromTensorFlow(NodeDef nodeDef, SameDiff initWith, Map<String,AttrValue> attributesForNode, GraphDef graph)
DifferentialFunction
NodeDef
initFromTensorFlow
in class DifferentialFunction
public void initFromOnnx(OnnxProto3.NodeProto node, SameDiff initWith, Map<String,OnnxProto3.AttributeProto> attributesForNode, OnnxProto3.GraphProto graph)
DifferentialFunction
OnnxProto3.NodeProto
initFromOnnx
in class DifferentialFunction
public String onnxName()
DifferentialFunction
onnxName
in class DifferentialFunction
public String tensorflowName()
DifferentialFunction
tensorflowName
in class DifferentialFunction
Copyright © 2018. All rights reserved.