Skip navigation links
A B C E G H I J L M O P R S T U V W 

A

ai.djl.serving.wlm - package ai.djl.serving.wlm
Contains the model server backend which manages worker threads and executes jobs on models.
ai.djl.serving.wlm.util - package ai.djl.serving.wlm.util
Contains utilities to support the WorkLoadManager.

B

build() - Method in class ai.djl.serving.wlm.WorkerThread.Builder
Builds the WorkerThread with the provided data.
builder() - Static method in class ai.djl.serving.wlm.WorkerThread
Creates a builder to build a WorkerThread.

C

cleanup() - Method in class ai.djl.serving.wlm.WorkLoadManager.WorkerPool
removes all stopped workers and workers in state error from the pool.
close() - Method in class ai.djl.serving.wlm.ModelInfo
configureModelBatch(int, int) - Method in class ai.djl.serving.wlm.ModelInfo
Sets a new batchSize and returns a new configured ModelInfo object.
configurePool(int) - Method in class ai.djl.serving.wlm.ModelInfo
Sets new configuration for the workerPool backing this model and returns a new configured ModelInfo object.

E

equals(Object) - Method in class ai.djl.serving.wlm.ModelInfo

G

generate() - Method in class ai.djl.serving.wlm.WorkerIdGenerator
generate a new worker id.
getBatchSize() - Method in class ai.djl.serving.wlm.ModelInfo
Returns the configured batch size.
getBegin() - Method in class ai.djl.serving.wlm.Job
Returns the job begin time.
getDefaultWorkers(NDManager, String, int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager
Returns the default number of workers for a new registered model.
getFuture() - Method in class ai.djl.serving.wlm.util.WorkerJob
Returns the future for the job.
getGpuId() - Method in class ai.djl.serving.wlm.WorkerThread
Returns the gpu id used by the thread.
getInput() - Method in class ai.djl.serving.wlm.Job
Returns the input data.
getInstance() - Static method in class ai.djl.serving.wlm.util.WlmConfigManager
Returns the singleton ConfigManager instance.
getJob() - Method in class ai.djl.serving.wlm.util.WorkerJob
Returns the Job.
getJobQueue() - Method in class ai.djl.serving.wlm.WorkLoadManager.WorkerPool
Returns the JobQueue for this model.
getMaxBatchDelay() - Method in class ai.djl.serving.wlm.ModelInfo
Returns the maximum delay in milliseconds to aggregate a batch.
getMaxIdleTime() - Method in class ai.djl.serving.wlm.ModelInfo
returns the configured maxIdleTime of workers.
getMaxWorkers() - Method in class ai.djl.serving.wlm.WorkLoadManager.WorkerPool
Returns the maximum number of workers for a model.
getMinWorkers() - Method in class ai.djl.serving.wlm.WorkLoadManager.WorkerPool
Returns the minimum number of workers for a model.
getModel() - Method in class ai.djl.serving.wlm.Job
Returns the model that associated with this job.
getModel() - Method in class ai.djl.serving.wlm.ModelInfo
Returns the loaded ZooModel.
getModelDir() - Method in class ai.djl.serving.wlm.ModelInfo
Returns the model cache directory.
getModelName() - Method in class ai.djl.serving.wlm.ModelInfo
Returns the model name.
getNumRunningWorkers(ModelInfo) - Method in class ai.djl.serving.wlm.WorkLoadManager
Returns the number of running workers of a model.
getQueueLength(ModelInfo) - Method in class ai.djl.serving.wlm.WorkLoadManager
Returns the current number of request in the queue.
getQueueSize() - Method in class ai.djl.serving.wlm.ModelInfo
Returns the configured size of the workers queue.
getScheduled() - Method in class ai.djl.serving.wlm.Job
Returns the job scheduled time.
getStartTime() - Method in class ai.djl.serving.wlm.WorkerThread
Returns the thread start time.
getState() - Method in class ai.djl.serving.wlm.WorkerThread
Returns the worker state.
getVersion() - Method in class ai.djl.serving.wlm.ModelInfo
Returns the model version.
getWorkerId() - Method in class ai.djl.serving.wlm.WorkerThread
Returns the worker thread ID.
getWorkerPoolForModel(ModelInfo) - Method in class ai.djl.serving.wlm.WorkLoadManager
Returns the WorkLoadManager.WorkerPool for a model.
getWorkers(ModelInfo) - Method in class ai.djl.serving.wlm.WorkLoadManager
Returns the workers for the specific model.
getWorkers() - Method in class ai.djl.serving.wlm.WorkLoadManager.WorkerPool
Returns a list of worker thread.

H

hashCode() - Method in class ai.djl.serving.wlm.ModelInfo

I

inferModelNameFromUrl(String) - Static method in class ai.djl.serving.wlm.ModelInfo
Infer model name form model URL in case model name is not provided.
isDebug() - Method in class ai.djl.serving.wlm.util.WlmConfigManager
Returns if debug is enabled.
isFinished() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator
Checks if this BatchAggregator and the thread can be shutdown or if this aggregator waits for more data.
isFinished() - Method in class ai.djl.serving.wlm.TemporaryBatchAggregator
Checks if this BatchAggregator and the thread can be shutdown or if this aggregator waits for more data.
isFixPoolThread() - Method in class ai.djl.serving.wlm.WorkerThread
check if this worker is instantiate is one of the fix threads of a pool.
isRunning() - Method in class ai.djl.serving.wlm.WorkerThread
Returns true if the worker thread is running.

J

Job - Class in ai.djl.serving.wlm
A class represents an inference job.
Job(ModelInfo, Input) - Constructor for class ai.djl.serving.wlm.Job
Constructs a new Job instance.

L

log() - Method in class ai.djl.serving.wlm.WorkLoadManager.WorkerPool
Logs the current state of this WorkerPool when level "Debug" is enabled.

M

ModelInfo - Class in ai.djl.serving.wlm
A class represent a loaded model and it's metadata.
ModelInfo(String, String, ZooModel<Input, Output>, int, int, int, int) - Constructor for class ai.djl.serving.wlm.ModelInfo
Constructs a new ModelInfo instance.

O

optAggregator(BatchAggregator) - Method in class ai.djl.serving.wlm.WorkerThread.Builder
Sets a BatchAggregator which overrides the instantiated default BatchAggregator.
optFixPoolThread(boolean) - Method in class ai.djl.serving.wlm.WorkerThread.Builder
Sets if the workerThread should be part of the fixed pool.

P

PermanentBatchAggregator - Class in ai.djl.serving.wlm
a batch aggregator that never terminates by itself.
PermanentBatchAggregator(ModelInfo, LinkedBlockingDeque<WorkerJob>) - Constructor for class ai.djl.serving.wlm.PermanentBatchAggregator
Constructs a PermanentBatchAggregator instance.
pollBatch() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator
Fills in the list with a batch of jobs.
pollBatch() - Method in class ai.djl.serving.wlm.TemporaryBatchAggregator
Fills in the list with a batch of jobs.
preBuildProcessing() - Method in class ai.djl.serving.wlm.WorkerThread.Builder
 

R

run() - Method in class ai.djl.serving.wlm.WorkerThread
runJob(Job) - Method in class ai.djl.serving.wlm.WorkLoadManager
Adds an inference job to the job queue of the next free worker.

S

scaleWorkers(String, int, int) - Method in class ai.djl.serving.wlm.WorkLoadManager.WorkerPool
Sets new worker capcities for this model.
self() - Method in class ai.djl.serving.wlm.WorkerThread.Builder
Returns self reference to this builder.
setDebug(boolean) - Method in class ai.djl.serving.wlm.util.WlmConfigManager
Sets debug mode.
setJobQueue(LinkedBlockingDeque<WorkerJob>) - Method in class ai.djl.serving.wlm.WorkerThread.Builder
Sets the jobQueue used to poll for new jobs.
setModel(ModelInfo) - Method in class ai.djl.serving.wlm.WorkerThread.Builder
Sets the ModelInfo the thread will be responsible for.
setScheduled() - Method in class ai.djl.serving.wlm.Job
Marks the job has been scheduled.
shutdown(WorkerState) - Method in class ai.djl.serving.wlm.WorkerThread
Shuts down the worker thread.

T

TemporaryBatchAggregator - Class in ai.djl.serving.wlm
a batch aggregator that terminates after a maximum idle time.
TemporaryBatchAggregator(ModelInfo, LinkedBlockingDeque<WorkerJob>) - Constructor for class ai.djl.serving.wlm.TemporaryBatchAggregator
a batch aggregator that terminates after a maximum idle time.
toString() - Method in class ai.djl.serving.wlm.ModelInfo

U

unregisterModel(ModelInfo) - Method in class ai.djl.serving.wlm.WorkLoadManager
Removes a model from management.

V

validate() - Method in class ai.djl.serving.wlm.WorkerThread.Builder
 
valueOf(String) - Static method in enum ai.djl.serving.wlm.WorkerState
Returns the enum constant of this type with the specified name.
values() - Static method in enum ai.djl.serving.wlm.WorkerState
Returns an array containing the constants of this enum type, in the order they are declared.

W

WlmCapacityException - Exception in ai.djl.serving.wlm.util
Thrown to throttle when a job is run but the job queue capacity is exceeded.
WlmCapacityException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmCapacityException
Constructs a WlmCapacityException with the specified detail message.
WlmCapacityException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.util.WlmCapacityException
Constructs a WlmCapacityException with the specified detail message and cause.
WlmConfigManager - Class in ai.djl.serving.wlm.util
This manages some configurations used by the WorkLoadManager.
WlmConfigManager() - Constructor for class ai.djl.serving.wlm.util.WlmConfigManager
 
WlmException - Exception in ai.djl.serving.wlm.util
Thrown when an exception occurs inside the WorkLoadManager.
WlmException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmException
Constructs a WlmException with the specified detail message.
WlmException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.util.WlmException
Constructs a WlmException with the specified detail message and cause.
WlmShutdownException - Exception in ai.djl.serving.wlm.util
Thrown when a job is run but all workers are shutdown.
WlmShutdownException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmShutdownException
Constructs a WlmShutdownException with the specified detail message.
WlmShutdownException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.util.WlmShutdownException
Constructs a WlmShutdownException with the specified detail message and cause.
WorkerIdGenerator - Class in ai.djl.serving.wlm
class to generate an unique worker id.
WorkerIdGenerator() - Constructor for class ai.djl.serving.wlm.WorkerIdGenerator
 
WorkerJob - Class in ai.djl.serving.wlm.util
A Job containing metadata from the WorkLoadManager.
WorkerJob(Job, CompletableFuture<Output>) - Constructor for class ai.djl.serving.wlm.util.WorkerJob
Constructs a new WorkerJob.
WorkerPool(ModelInfo) - Constructor for class ai.djl.serving.wlm.WorkLoadManager.WorkerPool
Construct and initial data structure.
WorkerState - Enum in ai.djl.serving.wlm
An enum represents state of a worker.
WorkerThread - Class in ai.djl.serving.wlm
The WorkerThread is the worker managed by the WorkLoadManager.
WorkerThread.Builder - Class in ai.djl.serving.wlm
A Builder to construct a WorkerThread.
WorkLoadManager - Class in ai.djl.serving.wlm
WorkLoadManager is responsible to manage the work load of worker thread.
WorkLoadManager() - Constructor for class ai.djl.serving.wlm.WorkLoadManager
Constructs a WorkLoadManager instance.
WorkLoadManager.WorkerPool - Class in ai.djl.serving.wlm
Manages the work load for a single model.
A B C E G H I J L M O P R S T U V W 
Skip navigation links