Package ai.djl.inference
Class Predictor<I,O>
- java.lang.Object
-
- ai.djl.inference.Predictor<I,O>
-
- Type Parameters:
I
- the input typeO
- the output type
- All Implemented Interfaces:
java.lang.AutoCloseable
public class Predictor<I,O> extends java.lang.Object implements java.lang.AutoCloseable
ThePredictor
interface provides a session for model inference.You can use a
Predictor
, with a specifiedTranslator
, to perform inference on aModel
. The following is example code that usesPredictor
:Model model = Model.load(modelDir, modelName); // User must implement Translator interface, read
Translator
for detail. Translator<String, String> translator = new MyTranslator(); try (Predictor<String, String> predictor = model.newPredictor(translator)) { String result = predictor.predict("What's up"); }See the tutorials on:
-
-
Field Summary
Fields Modifier and Type Field Description protected Block
block
protected NDManager
manager
protected Metrics
metrics
protected ParameterStore
parameterStore
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.util.List<O>
batchPredict(java.util.List<I> inputs)
Predicts a batch for inference.void
close()
protected void
finalize()
O
predict(I input)
Predicts an item for inference.protected NDList
predictInternal(TranslatorContext ctx, NDList ndList)
Predicts an item for inference.void
setMetrics(Metrics metrics)
Attaches a Metrics param to use for benchmark.
-
-
-
Field Detail
-
manager
protected NDManager manager
-
metrics
protected Metrics metrics
-
block
protected Block block
-
parameterStore
protected ParameterStore parameterStore
-
-
Constructor Detail
-
Predictor
public Predictor(Model model, Translator<I,O> translator, Device device, boolean copy)
- Parameters:
model
- the model on which the predictions are basedtranslator
- the translator to be useddevice
- the device for predictioncopy
- whether to copy the parameters to the parameter store. If the device changes, it will copy regardless
-
-
Method Detail
-
predict
public O predict(I input) throws TranslateException
Predicts an item for inference.- Parameters:
input
- the input- Returns:
- the output object defined by the user
- Throws:
TranslateException
- if an error occurs during prediction
-
predictInternal
protected NDList predictInternal(TranslatorContext ctx, NDList ndList) throws TranslateException
Predicts an item for inference.- Parameters:
ctx
- the context for thePredictor
.ndList
- the inputNDList
- Returns:
- the output
NDList
- Throws:
TranslateException
- if an error occurs during prediction
-
batchPredict
public java.util.List<O> batchPredict(java.util.List<I> inputs) throws TranslateException
Predicts a batch for inference.- Parameters:
inputs
- a list of inputs- Returns:
- a list of output objects defined by the user
- Throws:
TranslateException
- if an error occurs during prediction
-
setMetrics
public void setMetrics(Metrics metrics)
Attaches a Metrics param to use for benchmark.- Parameters:
metrics
- the Metrics class
-
close
public void close()
- Specified by:
close
in interfacejava.lang.AutoCloseable
-
finalize
protected void finalize() throws java.lang.Throwable
- Overrides:
finalize
in classjava.lang.Object
- Throws:
java.lang.Throwable
-
-