Package ai.djl.inference.streaming
Interface StreamingTranslator<I,O>
-
- Type Parameters:
I
- the input typeO
- the output type
- All Superinterfaces:
PostProcessor<O>
,PreProcessor<I>
,Translator<I,O>
- All Known Subinterfaces:
ServingTranslator
public interface StreamingTranslator<I,O> extends Translator<I,O>
An expansion of theTranslator
with postProcessing for theStreamingBlock
(used byPredictor.streamingPredict(Object)
.
-
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description O
processStreamOutput(TranslatorContext ctx, java.util.stream.Stream<NDList> list)
Processes the output NDList to the corresponding output object.-
Methods inherited from interface ai.djl.translate.PostProcessor
processOutput
-
Methods inherited from interface ai.djl.translate.PreProcessor
processInput
-
Methods inherited from interface ai.djl.translate.Translator
getBatchifier, getExpansions, prepare, toBatchTranslator, toBatchTranslator
-
-
-
-
Method Detail
-
processStreamOutput
O processStreamOutput(TranslatorContext ctx, java.util.stream.Stream<NDList> list) throws java.lang.Exception
Processes the output NDList to the corresponding output object.- Parameters:
ctx
- the toolkit used for post-processinglist
- the output NDList after inference, usually immutable in engines like PyTorch. @see Issue 1774- Returns:
- the output object of expected type
- Throws:
java.lang.Exception
- if an error occurs during processing output
-
-