ai.djl.training.evaluator

Class Evaluator

java.lang.Object

ai.djl.training.evaluator.Evaluator

Direct Known Subclasses:

AbstractAccuracy, BoundingBoxError, Loss

public abstract class Evaluator
extends java.lang.Object

Base class for all Evaluators that can be used to evaluate the performance of a model.

The Evaluators can all be monitored to make an assessment about the performance of the model. However, only ones that further extend Loss are suited to being used to directly optimize a model.

In addition to computing the evaluation, an evaluator can accumulate values to compute a total evaluation. For different purposes, it is possible to have multiple accumulators on a single evaluator. Each accumulator must be added with a String key to identify the accumulator. Before using an accumulator, you must addAccumulator(String). Then, call updateAccumulator(String, NDList, NDList) to add more data to the accumulator. You can use getAccumulator(String) to retrieve the accumulated value and resetAccumulator(String) to reset the accumulator to the same value as when just added.

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.Map<java.lang.String,java.lang.Long>	totalInstances

Constructor Summary

Constructors

Constructor and Description
Evaluator(java.lang.String name) Creates an evaluator with abstract update methods.

Method Summary

Modifier and Type	Method and Description
abstract void	addAccumulator(java.lang.String key) Adds an accumulator for the results of the evaluation with the given key.
protected void	checkLabelShapes(NDArray labels, NDArray predictions) Checks the length of NDArrays.
protected void	checkLabelShapes(NDArray labels, NDArray predictions, boolean checkDimOnly) Checks if the two input NDArray have the same length or shape.
abstract NDArray	evaluate(NDList labels, NDList predictions) Calculates the evaluation between the labels and the predictions.
abstract float	getAccumulator(java.lang.String key) Returns the accumulated evaluator value.
java.lang.String	getName() Returns the name of this Evaluator.
abstract void	resetAccumulator(java.lang.String key) Resets the evaluator value with the given key.
abstract void	updateAccumulator(java.lang.String key, NDList labels, NDList predictions) Updates the evaluator with the given key based on a NDList of labels and predictions.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

totalInstances

protected java.util.Map<java.lang.String,java.lang.Long> totalInstances

Constructor Detail

Evaluator

public Evaluator(java.lang.String name)

Creates an evaluator with abstract update methods.

Parameters:

name - the name of the evaluator

Method Detail

getName

public java.lang.String getName()

Returns the name of this Evaluator.

Returns:

the name of this Evaluator

evaluate

public abstract NDArray evaluate(NDList labels,
                                 NDList predictions)

Calculates the evaluation between the labels and the predictions.

Parameters:

labels - the correct values

predictions - the predicted values

Returns:

the evaluation result

addAccumulator

public abstract void addAccumulator(java.lang.String key)

Adds an accumulator for the results of the evaluation with the given key.

Parameters:

key - the key for the new accumulator

updateAccumulator

public abstract void updateAccumulator(java.lang.String key,
                                       NDList labels,
                                       NDList predictions)

Updates the evaluator with the given key based on a NDList of labels and predictions.

This is a synchronized operation. You should only call it at the end of a batch or epoch.

Parameters:

key - the key of the accumulator to update

labels - a NDList of labels

predictions - a NDList of predictions

resetAccumulator

public abstract void resetAccumulator(java.lang.String key)

Resets the evaluator value with the given key.

Parameters:

key - the key of the accumulator to reset

getAccumulator

public abstract float getAccumulator(java.lang.String key)

Returns the accumulated evaluator value.

Parameters:

key - the key of the accumulator to get

Returns:

the accumulated value

Throws:

java.lang.IllegalArgumentException - if no accumulator was added with the given key

checkLabelShapes

protected void checkLabelShapes(NDArray labels,
                                NDArray predictions,
                                boolean checkDimOnly)

Checks if the two input NDArray have the same length or shape.

Parameters:

labels - a NDArray of labels

predictions - a NDArray of predictions

checkDimOnly - whether to check for first dimension only

checkLabelShapes

protected void checkLabelShapes(NDArray labels,
                                NDArray predictions)

Checks the length of NDArrays.

Parameters:

labels - a NDArray of labels

predictions - a NDArray of predictions