Package ai.djl.serving.wlm
Contains the model server backend which manages worker threads and executes jobs on models.
- See Also:
WorkLoadManager
-
Class Summary Class Description Job<I,O> A class represents an inference job.ModelInfo<I,O> A class represent a loaded model and it's metadata.PermanentBatchAggregator<I,O> a batch aggregator that never terminates by itself.TemporaryBatchAggregator<I,O> a batch aggregator that terminates after a maximum idle time.WorkerIdGenerator class to generate an unique worker id.WorkerThread<I,O> TheWorkerThread
is the worker managed by theWorkLoadManager
.WorkerThread.Builder<I,O> A Builder to construct aWorkerThread
.WorkLoadManager WorkLoadManager is responsible to manage the work load of worker thread. -
Enum Summary Enum Description ModelInfo.Status An enum represents state of a model.WorkerState An enum represents state of a worker.