Package ai.djl.inference
Class Predictor<I,O>
java.lang.Object
ai.djl.inference.Predictor<I,O>
- Type Parameters:
I
- the input typeO
- the output type
- All Implemented Interfaces:
AutoCloseable
The
Predictor
interface provides a session for model inference.
You can use a Predictor
, with a specified Translator
, to perform inference on
a Model
. The following is example code that uses Predictor
:
Model model = Model.load(modelDir, modelName);
// User must implement Translator interface, read Translator
for detail.
Translator<String, String> translator = new MyTranslator();
try (Predictor<String, String> predictor = model.newPredictor(translator)) {
String result = predictor.predict("What's up");
}
See the tutorials on:
-
Nested Class Summary
-
Field Summary
Modifier and TypeFieldDescriptionprotected Block
protected Dimension
protected NDManager
protected Metrics
protected Model
protected ParameterStore
protected boolean
protected long
protected Translator<I,
O> -
Constructor Summary
ConstructorDescription -
Method Summary
Modifier and TypeMethodDescriptionbatchPredict
(List<I> inputs) Predicts a batch for inference.void
close()
protected void
finalize()
Predicts an item for inference.protected NDList
predictInternal
(TranslatorContext ctx, NDList ndList) Predicts an item for inference.void
setMetrics
(Metrics metrics) Attaches a Metrics param to use for benchmark.streamingPredict
(I input) Predicts an item for inference.boolean
Returns true if streaming is supported by the predictor, block, and translator.
-
Field Details
-
translator
-
timestamp
protected long timestamp -
prepared
protected boolean prepared -
model
-
manager
-
metrics
-
block
-
parameterStore
-
dimension
-
-
Constructor Details
-
Predictor
- Parameters:
model
- the model on which the predictions are basedtranslator
- the translator to be useddevice
- the device for predictioncopy
- whether to copy the parameters to the parameter store. If the device changes, it will copy regardless
-
-
Method Details
-
predict
Predicts an item for inference.- Parameters:
input
- the input- Returns:
- the output object defined by the user
- Throws:
TranslateException
- if an error occurs during prediction
-
predictInternal
Predicts an item for inference.- Parameters:
ctx
- the context for thePredictor
.ndList
- the inputNDList
- Returns:
- the output
NDList
- Throws:
TranslateException
- if an error occurs during prediction
-
batchPredict
Predicts a batch for inference.- Parameters:
inputs
- a list of inputs- Returns:
- a list of output objects defined by the user
- Throws:
TranslateException
- if an error occurs during prediction
-
streamingPredict
Predicts an item for inference.- Parameters:
input
- the input- Returns:
- the output object defined by the user
- Throws:
TranslateException
- if an error occurs during prediction
-
supportsStreaming
public boolean supportsStreaming()Returns true if streaming is supported by the predictor, block, and translator.- Returns:
- true if streaming is supported by the predictor, block, and translator
-
setMetrics
Attaches a Metrics param to use for benchmark.- Parameters:
metrics
- the Metrics class
-
close
public void close()- Specified by:
close
in interfaceAutoCloseable
-
finalize
-