public class PermanentBatchAggregator
extends java.lang.Object
Modifier and Type | Field and Description |
---|---|
protected int |
batchSize |
protected java.util.concurrent.LinkedBlockingDeque<WorkerJob> |
jobQueue |
protected int |
maxBatchDelay |
protected java.util.List<WorkerJob> |
wjs |
Constructor and Description |
---|
PermanentBatchAggregator(ModelInfo model,
java.util.concurrent.LinkedBlockingDeque<WorkerJob> jobQueue)
Constructs a
PermanentBatchAggregator instance. |
Modifier and Type | Method and Description |
---|---|
protected void |
drainTo(java.util.List<WorkerJob> list,
int maxDelay) |
java.util.List<ai.djl.modality.Input> |
getRequest()
Poll the queue and return a list of Input Objects for the model.
|
boolean |
isFinished()
Checks if this
BatchAggregator and the thread can be shutdown or if this aggregator
waits for more data. |
protected java.util.List<WorkerJob> |
pollBatch()
Fills in the list with a batch of jobs.
|
void |
sendError(java.lang.Throwable error)
Completes the job with an error.
|
void |
sendResponse(java.util.List<ai.djl.modality.Output> outputs)
Sends to response to all waiting clients.
|
protected int batchSize
protected int maxBatchDelay
protected java.util.List<WorkerJob> wjs
protected java.util.concurrent.LinkedBlockingDeque<WorkerJob> jobQueue
protected java.util.List<WorkerJob> pollBatch() throws java.lang.InterruptedException
java.lang.InterruptedException
- if interruptedpublic boolean isFinished()
BatchAggregator
and the thread can be shutdown or if this aggregator
waits for more data.public java.util.List<ai.djl.modality.Input> getRequest() throws java.lang.InterruptedException
java.lang.InterruptedException
- if thread gets interrupted while waiting for new data in the
queue.public void sendResponse(java.util.List<ai.djl.modality.Output> outputs)
outputs
- list of model-outputs in same order as the input objects.public void sendError(java.lang.Throwable error)
error
- the exceptionprotected void drainTo(java.util.List<WorkerJob> list, int maxDelay) throws java.lang.InterruptedException
java.lang.InterruptedException