Package ai.djl.nn.transformer

Class BertMaskedLanguageModelLoss

java.lang.Object

ai.djl.training.evaluator.Evaluator

ai.djl.training.loss.Loss

ai.djl.nn.transformer.BertMaskedLanguageModelLoss

public class BertMaskedLanguageModelLoss
extends Loss

The loss for the bert masked language model task.

Field Summary

Fields inherited from class ai.djl.training.evaluator.Evaluator

totalInstances

Constructor Summary

Constructors

Constructor	Description
BertMaskedLanguageModelLoss(int labelIdx, int maskIdx, int logProbsIdx)	Creates an MLM loss.

Method Summary

Modifier and Type	Method	Description
NDArray	accuracy(NDList labels, NDList predictions)	Calculates the percentage of correctly predicted masked tokens.
NDArray	evaluate(NDList labels, NDList predictions)	Calculates the evaluation between the labels and the predictions.

Methods inherited from class ai.djl.training.loss.Loss

addAccumulator, elasticNetWeightedDecay, elasticNetWeightedDecay, elasticNetWeightedDecay, elasticNetWeightedDecay, getAccumulator, hingeLoss, hingeLoss, hingeLoss, l1Loss, l1Loss, l1Loss, l1WeightedDecay, l1WeightedDecay, l1WeightedDecay, l2Loss, l2Loss, l2Loss, l2WeightedDecay, l2WeightedDecay, l2WeightedDecay, maskedSoftmaxCrossEntropyLoss, maskedSoftmaxCrossEntropyLoss, maskedSoftmaxCrossEntropyLoss, resetAccumulator, sigmoidBinaryCrossEntropyLoss, sigmoidBinaryCrossEntropyLoss, sigmoidBinaryCrossEntropyLoss, softmaxCrossEntropyLoss, softmaxCrossEntropyLoss, softmaxCrossEntropyLoss, updateAccumulator

Methods inherited from class ai.djl.training.evaluator.Evaluator

checkLabelShapes, checkLabelShapes, getName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BertMaskedLanguageModelLoss

public BertMaskedLanguageModelLoss(int labelIdx,
                                   int maskIdx,
                                   int logProbsIdx)

Creates an MLM loss.

Parameters:

labelIdx - index of labels

maskIdx - index of mask

logProbsIdx - index of log probs

Method Detail

evaluate

public NDArray evaluate(NDList labels,
                        NDList predictions)

Calculates the evaluation between the labels and the predictions.

Specified by:

evaluate in class Evaluator

Parameters:

labels - the correct values

predictions - the predicted values

Returns:

the evaluation result

accuracy

public NDArray accuracy(NDList labels,
                        NDList predictions)

Calculates the percentage of correctly predicted masked tokens.

Parameters:

labels - expected tokens and mask

predictions - prediction of a bert model

Returns:

the percentage of correctly predicted masked tokens