public class NDLoss extends Object
Constructor and Description |
---|
NDLoss() |
Modifier and Type | Method and Description |
---|---|
INDArray |
absoluteDifference(INDArray label,
INDArray predictions,
INDArray weights)
Absolute difference loss:
sum_i abs( label[i] - predictions[i] ) |
INDArray |
absoluteDifference(INDArray label,
INDArray predictions,
INDArray weights,
LossReduce lossReduce)
Absolute difference loss:
sum_i abs( label[i] - predictions[i] ) |
INDArray |
cosineDistance(INDArray label,
INDArray predictions,
INDArray weights,
int dimension)
Cosine distance loss:
1 - cosineSimilarity(x,y) or 1 - sum_i label[i] * prediction[i] , which isequivalent to cosine distance when both the predictions and labels are normalized. Note: This loss function assumes that both the predictions and labels are normalized to have unit l2 norm. If this is not the case, you should normalize them first by dividing by norm2(String, SDVariable, boolean, int...) along the cosine distance dimension (with keepDims=true). |
INDArray |
cosineDistance(INDArray label,
INDArray predictions,
INDArray weights,
LossReduce lossReduce,
int dimension)
Cosine distance loss:
1 - cosineSimilarity(x,y) or 1 - sum_i label[i] * prediction[i] , which isequivalent to cosine distance when both the predictions and labels are normalized. Note: This loss function assumes that both the predictions and labels are normalized to have unit l2 norm. If this is not the case, you should normalize them first by dividing by norm2(String, SDVariable, boolean, int...) along the cosine distance dimension (with keepDims=true). |
INDArray |
ctcLoss(INDArray targetLabels,
INDArray logitInput,
INDArray targetLabelLengths,
INDArray logitInputLengths)
CTC Loss: Connectionist Temporal Classification Loss.
|
INDArray |
hingeLoss(INDArray label,
INDArray predictions,
INDArray weights)
Hinge loss: a loss function used for training classifiers.
Implements L = max(0, 1 - t * predictions) where t is the label values after internally converting to {-1,1}from the user specified {0,1}. |
INDArray |
hingeLoss(INDArray label,
INDArray predictions,
INDArray weights,
LossReduce lossReduce)
Hinge loss: a loss function used for training classifiers.
Implements L = max(0, 1 - t * predictions) where t is the label values after internally converting to {-1,1}from the user specified {0,1}. |
INDArray |
huberLoss(INDArray label,
INDArray predictions,
INDArray weights,
double delta)
Huber loss function, used for robust regression.
|
INDArray |
huberLoss(INDArray label,
INDArray predictions,
INDArray weights,
LossReduce lossReduce,
double delta)
Huber loss function, used for robust regression.
|
INDArray |
l2Loss(INDArray var)
L2 loss: 1/2 * sum(x^2)
|
INDArray |
logLoss(INDArray label,
INDArray predictions)
Log loss, i.e., binary cross entropy loss, usually used for binary multi-label classification.
|
INDArray |
logLoss(INDArray label,
INDArray predictions,
INDArray weights,
LossReduce lossReduce,
double epsilon)
Log loss, i.e., binary cross entropy loss, usually used for binary multi-label classification.
|
INDArray |
logPoisson(INDArray label,
INDArray predictions,
INDArray weights,
boolean full)
Log poisson loss: a loss function used for training classifiers.
Implements L = exp(c) - z * c where c is log(predictions) and z is labels. |
INDArray |
logPoisson(INDArray label,
INDArray predictions,
INDArray weights,
LossReduce lossReduce,
boolean full)
Log poisson loss: a loss function used for training classifiers.
Implements L = exp(c) - z * c where c is log(predictions) and z is labels. |
INDArray |
meanPairwiseSquaredError(INDArray label,
INDArray predictions,
INDArray weights)
Mean pairwise squared error.
MPWSE loss calculates the difference between pairs of consecutive elements in the predictions and labels arrays. For example, if predictions = [p0, p1, p2] and labels are [l0, l1, l2] then MPWSE is: [((p0-p1) - (l0-l1))^2 + ((p0-p2) - (l0-l2))^2 + ((p1-p2) - (l1-l2))^2] / 3 |
INDArray |
meanPairwiseSquaredError(INDArray label,
INDArray predictions,
INDArray weights,
LossReduce lossReduce)
Mean pairwise squared error.
MPWSE loss calculates the difference between pairs of consecutive elements in the predictions and labels arrays. For example, if predictions = [p0, p1, p2] and labels are [l0, l1, l2] then MPWSE is: [((p0-p1) - (l0-l1))^2 + ((p0-p2) - (l0-l2))^2 + ((p1-p2) - (l1-l2))^2] / 3 |
INDArray |
meanSquaredError(INDArray label,
INDArray predictions,
INDArray weights)
Mean squared error loss function.
|
INDArray |
meanSquaredError(INDArray label,
INDArray predictions,
INDArray weights,
LossReduce lossReduce)
Mean squared error loss function.
|
INDArray |
sigmoidCrossEntropy(INDArray label,
INDArray predictionLogits,
INDArray weights)
Sigmoid cross entropy: applies the sigmoid activation function on the input logits (input "pre-sigmoid preductions")
and implements the binary cross entropy loss function. |
INDArray |
sigmoidCrossEntropy(INDArray label,
INDArray predictionLogits,
INDArray weights,
LossReduce lossReduce,
double labelSmoothing)
Sigmoid cross entropy: applies the sigmoid activation function on the input logits (input "pre-sigmoid preductions")
and implements the binary cross entropy loss function. |
INDArray |
softmaxCrossEntropy(INDArray oneHotLabels,
INDArray logitPredictions,
INDArray weights)
Applies the softmax activation function to the input, then implement multi-class cross entropy:
-sum_classes label[i] * log(p[c]) where p = softmax(logits) If LossReduce#NONE is used, returned shape is [numExamples] out for [numExamples, numClasses] predicitons/labels; otherwise, the output is a scalar. |
INDArray |
softmaxCrossEntropy(INDArray oneHotLabels,
INDArray logitPredictions,
INDArray weights,
LossReduce lossReduce,
double labelSmoothing)
Applies the softmax activation function to the input, then implement multi-class cross entropy:
-sum_classes label[i] * log(p[c]) where p = softmax(logits) If LossReduce#NONE is used, returned shape is [numExamples] out for [numExamples, numClasses] predicitons/labels; otherwise, the output is a scalar. |
INDArray |
sparseSoftmaxCrossEntropy(INDArray logits,
INDArray labels)
As per softmaxCrossEntropy(String, SDVariable, SDVariable, LossReduce) but the labels variable
is represented as an integer array instead of the equivalent one-hot array. i.e., if logits are rank N, then labels have rank N-1 |
INDArray |
weightedCrossEntropyWithLogits(INDArray targets,
INDArray inputs,
INDArray weights)
Weighted cross entropy loss with logits
|
public INDArray absoluteDifference(INDArray label, INDArray predictions, INDArray weights, LossReduce lossReduce)
sum_i abs( label[i] - predictions[i] )
label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTpublic INDArray absoluteDifference(INDArray label, INDArray predictions, INDArray weights)
sum_i abs( label[i] - predictions[i] )
label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)public INDArray cosineDistance(INDArray label, INDArray predictions, INDArray weights, LossReduce lossReduce, int dimension)
1 - cosineSimilarity(x,y)
or 1 - sum_i label[i] * prediction[i]
, which islabel
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is use (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTdimension
- Dimension to perform the cosine distance overpublic INDArray cosineDistance(INDArray label, INDArray predictions, INDArray weights, int dimension)
1 - cosineSimilarity(x,y)
or 1 - sum_i label[i] * prediction[i]
, which islabel
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is use (NUMERIC type)dimension
- Dimension to perform the cosine distance overpublic INDArray ctcLoss(INDArray targetLabels, INDArray logitInput, INDArray targetLabelLengths, INDArray logitInputLengths)
targetLabels
- Label array (NUMERIC type)logitInput
- Inputs (NUMERIC type)targetLabelLengths
- Length of the target label (NUMERIC type)logitInputLengths
- Length of the input (NUMERIC type)public INDArray hingeLoss(INDArray label, INDArray predictions, INDArray weights, LossReduce lossReduce)
L = max(0, 1 - t * predictions)
where t is the label values after internally converting to {-1,1}label
- Label array. Each value should be 0.0 or 1.0 (internally -1 to 1 is used) (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTpublic INDArray hingeLoss(INDArray label, INDArray predictions, INDArray weights)
L = max(0, 1 - t * predictions)
where t is the label values after internally converting to {-1,1}label
- Label array. Each value should be 0.0 or 1.0 (internally -1 to 1 is used) (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)public INDArray huberLoss(INDArray label, INDArray predictions, INDArray weights, LossReduce lossReduce, double delta)
L = 0.5 * (label[i] - predictions[i])^2 if abs(label[i] - predictions[i]) < delta
L = delta * abs(label[i] - predictions[i]) - 0.5 * delta^2 otherwise
label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTdelta
- Loss function delta valuepublic INDArray huberLoss(INDArray label, INDArray predictions, INDArray weights, double delta)
L = 0.5 * (label[i] - predictions[i])^2 if abs(label[i] - predictions[i]) < delta
L = delta * abs(label[i] - predictions[i]) - 0.5 * delta^2 otherwise
label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)delta
- Loss function delta valuepublic INDArray l2Loss(INDArray var)
var
- Variable to calculate L2 loss of (NUMERIC type)public INDArray logLoss(INDArray label, INDArray predictions, INDArray weights, LossReduce lossReduce, double epsilon)
-1/numExamples * sum_i (labels[i] * log(predictions[i] + epsilon) + (1-labels[i]) * log(1-predictions[i] + epsilon))
label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTepsilon
- epsilonpublic INDArray logLoss(INDArray label, INDArray predictions)
-1/numExamples * sum_i (labels[i] * log(predictions[i] + epsilon) + (1-labels[i]) * log(1-predictions[i] + epsilon))
label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)public INDArray logPoisson(INDArray label, INDArray predictions, INDArray weights, LossReduce lossReduce, boolean full)
L = exp(c) - z * c
where c is log(predictions) and z is labels.label
- Label array. Each value should be 0.0 or 1.0 (NUMERIC type)predictions
- Predictions array (has to be log(x) of actual predictions) (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTfull
- Boolean flag. true for logPoissonFull, false for logPoissonpublic INDArray logPoisson(INDArray label, INDArray predictions, INDArray weights, boolean full)
L = exp(c) - z * c
where c is log(predictions) and z is labels.label
- Label array. Each value should be 0.0 or 1.0 (NUMERIC type)predictions
- Predictions array (has to be log(x) of actual predictions) (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)full
- Boolean flag. true for logPoissonFull, false for logPoissonpublic INDArray meanPairwiseSquaredError(INDArray label, INDArray predictions, INDArray weights, LossReduce lossReduce)
[((p0-p1) - (l0-l1))^2 + ((p0-p2) - (l0-l2))^2 + ((p1-p2) - (l1-l2))^2] / 3
label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used. Must be either null, scalar, or have shape [batchSize] (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTpublic INDArray meanPairwiseSquaredError(INDArray label, INDArray predictions, INDArray weights)
[((p0-p1) - (l0-l1))^2 + ((p0-p2) - (l0-l2))^2 + ((p1-p2) - (l1-l2))^2] / 3
label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used. Must be either null, scalar, or have shape [batchSize] (NUMERIC type)public INDArray meanSquaredError(INDArray label, INDArray predictions, INDArray weights, LossReduce lossReduce)
(label[i] - prediction[i])^2
- i.e., squared error on a per-element basis.label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTpublic INDArray meanSquaredError(INDArray label, INDArray predictions, INDArray weights)
(label[i] - prediction[i])^2
- i.e., squared error on a per-element basis.label
- Label array (NUMERIC type)predictions
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)public INDArray sigmoidCrossEntropy(INDArray label, INDArray predictionLogits, INDArray weights, LossReduce lossReduce, double labelSmoothing)
-1/numExamples * sum_i (labels[i] * log(sigmoid(logits[i])) + (1-labels[i]) * log(1-sigmoid(logits[i])))
numClasses = labels.size(1);<br> label = (1.0 - labelSmoothing) * label + 0.5 * labelSmoothing
label
- Label array (NUMERIC type)predictionLogits
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTlabelSmoothing
- Label smoothing value. Default value: 0public INDArray sigmoidCrossEntropy(INDArray label, INDArray predictionLogits, INDArray weights)
-1/numExamples * sum_i (labels[i] * log(sigmoid(logits[i])) + (1-labels[i]) * log(1-sigmoid(logits[i])))
numClasses = labels.size(1);<br> label = (1.0 - labelSmoothing) * label + 0.5 * labelSmoothing
label
- Label array (NUMERIC type)predictionLogits
- Predictions array (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)public INDArray softmaxCrossEntropy(INDArray oneHotLabels, INDArray logitPredictions, INDArray weights, LossReduce lossReduce, double labelSmoothing)
-sum_classes label[i] * log(p[c])
where p = softmax(logits)
When label smoothing is > 0, the following label smoothing is used:
numClasses = labels.size(1);<br> oneHotLabel = (1.0 - labelSmoothing) * oneHotLabels + labelSmoothing/numClasses
oneHotLabels
- Label array. Should be one-hot per example and same shape as predictions (for example, [mb, nOut]) (NUMERIC type)logitPredictions
- Predictions array (pre-softmax) (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)lossReduce
- Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNTlabelSmoothing
- Label smoothing value. Default value: 0public INDArray softmaxCrossEntropy(INDArray oneHotLabels, INDArray logitPredictions, INDArray weights)
-sum_classes label[i] * log(p[c])
where p = softmax(logits)
When label smoothing is > 0, the following label smoothing is used:
numClasses = labels.size(1);<br> oneHotLabel = (1.0 - labelSmoothing) * oneHotLabels + labelSmoothing/numClasses
oneHotLabels
- Label array. Should be one-hot per example and same shape as predictions (for example, [mb, nOut]) (NUMERIC type)logitPredictions
- Predictions array (pre-softmax) (NUMERIC type)weights
- Weights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)public INDArray sparseSoftmaxCrossEntropy(INDArray logits, INDArray labels)
logits
- Logits array ("pre-softmax activations") (NUMERIC type)labels
- Labels array. Must be an integer type. (INT type)public INDArray weightedCrossEntropyWithLogits(INDArray targets, INDArray inputs, INDArray weights)
targets
- targets array (NUMERIC type)inputs
- input array (NUMERIC type)weights
- eights array. May be null. If null, a weight of 1.0 is used (NUMERIC type)Copyright © 2021. All rights reserved.