ai.djl.inference

Interface Predictor<I,O>

Type Parameters:

I - the input type

O - the output type

All Superinterfaces:

java.lang.AutoCloseable

All Known Implementing Classes:

BasePredictor

public interface Predictor<I,O>
extends java.lang.AutoCloseable

The Predictor interface provides a session for model inference.

You can use a Predictor, with a specified Translator, to perform inference on a Model. The following is example code that uses Predictor:

 Model model = Model.load(modelDir, modelName);

 // User must implement Translator interface, read Translator for detail.
 Translator<String, String> translator = new MyTranslator();

 try (Predictor<String, String> predictor = model.newPredictor(translator)) {
   String result = predictor.predict("What's up");
 }
 

See the tutorials on:

• Inference with a custom trained model
• Inference with a model zoo model
• Inference with an MXNet model

For information about running multi-threaded inference, see here.

See Also:

Model, Translator

Method Summary

Modifier and Type	Method and Description
java.util.List<O>	batchPredict(java.util.List<I> inputs) Predicts a batch for inference.
void	close()
O	predict(I input) Predicts an item for inference.
void	setMetrics(Metrics metrics) Attaches a Metrics param to use for benchmark.

Method Detail

predict

O predict(I input)
   throws TranslateException

Predicts an item for inference.

Parameters:

input - the input

Returns:

the output object defined by the user

Throws:

TranslateException - if an error occurs during prediction

batchPredict

java.util.List<O> batchPredict(java.util.List<I> inputs)
                        throws TranslateException

Predicts a batch for inference.

Parameters:

inputs - a list of inputs

Returns:

a list of output objects defined by the user

Throws:

TranslateException - if an error occurs during prediction

setMetrics

void setMetrics(Metrics metrics)

Attaches a Metrics param to use for benchmark.

Parameters:

metrics - the Metrics class

close

void close()

Specified by:

close in interface java.lang.AutoCloseable