Package ai.djl.serving.wlm
Contains the model server backend which manages worker threads and executes jobs on models.
- See Also:
WorkLoadManager
-
Interface Summary Interface Description JobFunction<I,O> A function describing the action to take in aJob
. -
Class Summary Class Description Adapter An adapter is a modification producing a variation of a model that can be used during prediction.Job<I,O> A class represents an inference job.LmiUtils A utility class to detect optimal engine for LMI model.ModelInfo<I,O> A class represent a loaded model and it's metadata.PermanentBatchAggregator<I,O> a batch aggregator that never terminates by itself.PyAdapter An overload ofAdapter
for the python engine.SageMakerUtils A utility class to detect optimal engine for SageMaker saved model.TemporaryBatchAggregator<I,O> a batch aggregator that terminates after a maximum idle time.WorkerGroup<I,O> WorkerIdGenerator class to generate an unique worker id.WorkerPool<I,O> Manages the work load for a single model.WorkerPoolConfig<I,O> AWorkerPoolConfig
represents a task that could be run in theWorkLoadManager
.WorkerPoolConfig.ThreadConfig<I,O> The part of theWorkerPoolConfig
for an individualWorkerThread
.WorkerThread<I,O> TheWorkerThread
is the worker managed by theWorkLoadManager
.WorkerThread.Builder<I,O> A Builder to construct aWorkerThread
.WorkLoadManager WorkLoadManager is responsible to manage the work load of worker thread. -
Enum Summary Enum Description WorkerPoolConfig.Status An enum represents state of a worker type.WorkerState An enum represents state of a worker.