Class Evaluator
- java.lang.Object
-
- ai.djl.training.evaluator.Evaluator
-
- Direct Known Subclasses:
AbstractAccuracy
,BoundingBoxError
,IndexEvaluator
,Loss
public abstract class Evaluator extends java.lang.Object
Base class for allEvaluator
s that can be used to evaluate the performance of a model.The
Evaluator
s can all be monitored to make an assessment about the performance of the model. However, only ones that further extendLoss
are suited to being used to directly optimize a model.In addition to computing the evaluation, an evaluator can accumulate values to compute a total evaluation. For different purposes, it is possible to have multiple accumulators on a single evaluator. Each accumulator must be added with a String key to identify the accumulator. Before using an accumulator, you must
addAccumulator(String)
. Then, callupdateAccumulator(String, NDList, NDList)
to add more data to the accumulator. You can usegetAccumulator(String)
to retrieve the accumulated value andresetAccumulator(String)
to reset the accumulator to the same value as when just added.
-
-
Field Summary
Fields Modifier and Type Field Description protected java.util.Map<java.lang.String,java.lang.Long>
totalInstances
-
Constructor Summary
Constructors Constructor Description Evaluator(java.lang.String name)
Creates an evaluator with abstract update methods.
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description abstract void
addAccumulator(java.lang.String key)
Adds an accumulator for the results of the evaluation with the given key.protected void
checkLabelShapes(NDArray labels, NDArray predictions)
Checks the length of NDArrays.protected void
checkLabelShapes(NDArray labels, NDArray predictions, boolean checkDimOnly)
Checks if the two inputNDArray
have the same length or shape.abstract NDArray
evaluate(NDList labels, NDList predictions)
Calculates the evaluation between the labels and the predictions.abstract float
getAccumulator(java.lang.String key)
Returns the accumulated evaluator value.java.lang.String
getName()
Returns the name of thisEvaluator
.abstract void
resetAccumulator(java.lang.String key)
Resets the evaluator value with the given key.abstract void
updateAccumulator(java.lang.String key, NDList labels, NDList predictions)
Updates the evaluator with the given key based on aNDList
of labels and predictions.
-
-
-
Method Detail
-
getName
public java.lang.String getName()
Returns the name of thisEvaluator
.- Returns:
- the name of this
Evaluator
-
evaluate
public abstract NDArray evaluate(NDList labels, NDList predictions)
Calculates the evaluation between the labels and the predictions.- Parameters:
labels
- the correct valuespredictions
- the predicted values- Returns:
- the evaluation result
-
addAccumulator
public abstract void addAccumulator(java.lang.String key)
Adds an accumulator for the results of the evaluation with the given key.- Parameters:
key
- the key for the new accumulator
-
updateAccumulator
public abstract void updateAccumulator(java.lang.String key, NDList labels, NDList predictions)
Updates the evaluator with the given key based on aNDList
of labels and predictions.This is a synchronized operation. You should only call it at the end of a batch or epoch.
- Parameters:
key
- the key of the accumulator to updatelabels
- aNDList
of labelspredictions
- aNDList
of predictions
-
resetAccumulator
public abstract void resetAccumulator(java.lang.String key)
Resets the evaluator value with the given key.- Parameters:
key
- the key of the accumulator to reset
-
getAccumulator
public abstract float getAccumulator(java.lang.String key)
Returns the accumulated evaluator value.- Parameters:
key
- the key of the accumulator to get- Returns:
- the accumulated value
- Throws:
java.lang.IllegalArgumentException
- if no accumulator was added with the given key
-
checkLabelShapes
protected void checkLabelShapes(NDArray labels, NDArray predictions, boolean checkDimOnly)
Checks if the two inputNDArray
have the same length or shape.- Parameters:
labels
- aNDArray
of labelspredictions
- aNDArray
of predictionscheckDimOnly
- whether to check for first dimension only
-
-