ai.djl.inference

Class BasePredictor<I,O>

java.lang.Object

ai.djl.inference.BasePredictor<I,O>

Type Parameters:

I - the type of the input

O - the type of the output

All Implemented Interfaces:

Predictor<I,O>, java.lang.AutoCloseable

public class BasePredictor<I,O>
extends java.lang.Object
implements Predictor<I,O>

BasePredictor contains common code for implementations of the Predictor interface.

Field Summary

Fields

Modifier and Type	Field and Description
protected NDManager	manager
protected Model	model

Constructor Summary

Constructors

Constructor and Description
BasePredictor(Model model, Translator<I,O> translator, boolean copy) Creates a new instance of BasePredictor with the given Model and Translator.

Method Summary

Modifier and Type	Method and Description
java.util.List<O>	batchPredict(java.util.List<I> inputs) Predicts a batch for inference.
void	close()
protected NDList	forward(NDList ndList)
O	predict(I input) Predicts an item for inference.
void	setMetrics(Metrics metrics) Attaches a Metrics param to use for benchmark.
protected void	waitToRead(NDList list)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

model

protected Model model

manager

protected NDManager manager

Constructor Detail

BasePredictor

public BasePredictor(Model model,
                     Translator<I,O> translator,
                     boolean copy)

Creates a new instance of BasePredictor with the given Model and Translator.

Parameters:

model - the model on which the predictions are based

translator - the translator to be used

copy - whether to copy the parameters to the parameter store

Method Detail

predict

public O predict(I input)
          throws TranslateException

Predicts an item for inference.

Specified by:

predict in interface Predictor<I,O>

Parameters:

input - the input

Returns:

the output object defined by the user

Throws:

TranslateException - if an error occurs during prediction

batchPredict

public java.util.List<O> batchPredict(java.util.List<I> inputs)
                               throws TranslateException

Predicts a batch for inference.

Specified by:

batchPredict in interface Predictor<I,O>

Parameters:

inputs - a list of inputs

Returns:

a list of output objects defined by the user

Throws:

TranslateException - if an error occurs during prediction

setMetrics

public void setMetrics(Metrics metrics)

Attaches a Metrics param to use for benchmark.

Specified by:

setMetrics in interface Predictor<I,O>

Parameters:

metrics - the Metrics class

waitToRead

protected void waitToRead(NDList list)

forward

protected NDList forward(NDList ndList)

close

public void close()

Specified by:

close in interface Predictor<I,O>

Specified by:

close in interface java.lang.AutoCloseable