A B C D E F G H I J L M N O P Q R S T U V W
All Classes All Packages
All Classes All Packages
All Classes All Packages
A
- Adapter - Class in ai.djl.serving.wlm
-
An adapter is a modification producing a variation of a model that can be used during prediction.
- Adapter(String, String) - Constructor for class ai.djl.serving.wlm.Adapter
-
Constructs an
Adapter
. - addConfigJob(WorkerJob<I, O>) - Method in class ai.djl.serving.wlm.WorkerThread
-
Adds a configuration job to this thread.
- ai.djl.serving.wlm - package ai.djl.serving.wlm
-
Contains the model server backend which manages worker threads and executes jobs on models.
- ai.djl.serving.wlm.util - package ai.djl.serving.wlm.util
-
Contains utilities to support the
WorkLoadManager
. - apply(List<I>) - Method in interface ai.djl.serving.wlm.JobFunction
-
Applies this function.
B
- batchSize - Variable in class ai.djl.serving.wlm.WorkerPoolConfig
- build() - Method in class ai.djl.serving.wlm.WorkerThread.Builder
-
Builds the
WorkerThread
with the provided data. - builder(WorkerPoolConfig<I, O>) - Static method in class ai.djl.serving.wlm.WorkerThread
-
Creates a builder to build a
WorkerThread
.
C
- cleanup() - Method in class ai.djl.serving.wlm.WorkerPool
-
removes all stopped workers and workers in state error from the pool.
- close() - Method in class ai.djl.serving.wlm.ModelInfo
-
Close all loaded workers.
- close() - Method in class ai.djl.serving.wlm.ModelInfo.ModelThread
-
Closes the thread type and frees any resources.
- close() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Close all loaded workers.
- close() - Method in class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig
-
Closes the thread type and frees any resources.
- close() - Method in class ai.djl.serving.wlm.WorkLoadManager
-
Close all wpcs related to the
WorkloadManager
. - configJobs - Variable in class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig
- configureWorkers(int, int) - Method in class ai.djl.serving.wlm.WorkerGroup
-
Configures minimum and maximum number of workers.
D
- decreaseRef() - Method in class ai.djl.serving.wlm.WorkerPool
-
Decrease the reference count and return the current count.
E
- equals(Object) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
F
- FAILED - ai.djl.serving.wlm.WorkerPoolConfig.Status
G
- generate() - Method in class ai.djl.serving.wlm.WorkerIdGenerator
-
generate a new worker id.
- getAdapter(String) - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns an adapter on this
ModelInfo
. - getAdapters() - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the adapters for this model.
- getBatchSize() - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Returns the default batchSize for workers.
- getBatchSize() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the configured batch size.
- getConfigJobs() - Method in class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig
-
Gets the configuration jobs for the worker.
- getDevice() - Method in class ai.djl.serving.wlm.WorkerGroup
-
Returns the device of the worker group.
- getDevice() - Method in class ai.djl.serving.wlm.WorkerThread
-
Returns the device used by the thread.
- getEngine() - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the engine.
- getEngineName() - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the engine name.
- getFuture() - Method in class ai.djl.serving.wlm.util.WorkerJob
-
Returns the future for the job.
- getId() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the worker type ID.
- getInput() - Method in class ai.djl.serving.wlm.Job
-
Returns the input data.
- getInputClass() - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the model input class.
- getInstance() - Static method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Returns the singleton
ConfigManager
instance. - getJob() - Method in class ai.djl.serving.wlm.util.WorkerJob
-
Returns the
Job
. - getJobQueue() - Method in class ai.djl.serving.wlm.WorkerPool
-
Returns the
JobQueue
for this model. - getJobQueueSize() - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Returns the default job queue size.
- getLoadOnDevices() - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the devices the worker type will be loaded on at startup.
- getLoadOnDevices() - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Returns the devices the model will be loaded on at startup.
- getLoadOnDevices() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the devices the worker type will be loaded on at startup.
- getMaxBatchDelayMillis() - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Returns the default max batch delay in milliseconds for the working queue.
- getMaxBatchDelayMillis() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the maximum delay in milliseconds to aggregate a batch.
- getMaxIdleSeconds() - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Returns the default max idle time for workers.
- getMaxIdleSeconds() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the configured max idle time in seconds of workers.
- getMaxWorkers() - Method in class ai.djl.serving.wlm.WorkerGroup
-
Returns the max number of workers for the model and device.
- getMaxWorkers() - Method in class ai.djl.serving.wlm.WorkerPool
-
Returns the maximum number of workers for a model across all devices.
- getMaxWorkers(Device) - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the maximum number of workers.
- getMaxWorkers(Device) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the maximum number of workers.
- getMinWorkers() - Method in class ai.djl.serving.wlm.WorkerGroup
-
Returns the min number of workers for the model and device.
- getMinWorkers(Device) - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the minimum number of workers.
- getMinWorkers(Device) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the minimum number of workers.
- getModel(Device) - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the loaded
ZooModel
for a device. - getModels() - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns all loaded models.
- getModelUrl() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the worker type url.
- getName() - Method in class ai.djl.serving.wlm.Adapter
-
Returns the adapter name.
- getNumRunningWorkers(WorkerPoolConfig<?, ?>) - Method in class ai.djl.serving.wlm.WorkLoadManager
-
Returns the number of running workers of a wpc.
- getOutput() - Method in class ai.djl.serving.wlm.Job
-
Returns the output data.
- getOutputClass() - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the model output class.
- getPredictor() - Method in class ai.djl.serving.wlm.ModelInfo.ModelThread
-
Returns the predictor.
- getQueueSize() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the configured size of the workers queue.
- getReservedMemoryMb() - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Returns the default reserved memory in MB.
- getRunner() - Method in class ai.djl.serving.wlm.Job
-
Returns the task to run for the job.
- getSrc() - Method in class ai.djl.serving.wlm.Adapter
-
Returns the adapter src.
- getStartTime() - Method in class ai.djl.serving.wlm.WorkerThread
-
Returns the thread start time.
- getState() - Method in class ai.djl.serving.wlm.WorkerThread
-
Returns the worker state.
- getStatus() - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the worker type loading status.
- getStatus() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the worker type loading status.
- getThreadType() - Method in class ai.djl.serving.wlm.WorkerThread
-
Returns the
WorkerPoolConfig
'sWorkerPoolConfig.ThreadConfig
for this thread. - getVersion() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the worker type version.
- getWaitingMicroSeconds() - Method in class ai.djl.serving.wlm.Job
-
Returns the wait time of this job.
- getWorkerGroups() - Method in class ai.djl.serving.wlm.WorkerPool
-
Returns a map of
WorkerGroup
. - getWorkerId() - Method in class ai.djl.serving.wlm.WorkerThread
-
Returns the worker thread ID.
- getWorkerPool(WorkerPoolConfig<I, O>) - Method in class ai.djl.serving.wlm.WorkLoadManager
-
Returns the
WorkerPool
for a model. - getWorkerPoolById(String) - Method in class ai.djl.serving.wlm.WorkLoadManager
-
Returns the
WorkerPool
for a wpc. - getWorkers() - Method in class ai.djl.serving.wlm.WorkerGroup
-
Returns a list of workers.
- getWorkers() - Method in class ai.djl.serving.wlm.WorkerPool
-
Returns a list of worker thread.
- getWpc() - Method in class ai.djl.serving.wlm.Job
-
Returns the worker pool config that is associated with this job.
- getWpc() - Method in class ai.djl.serving.wlm.WorkerPool
-
Returns the model of the worker pool.
H
- hashCode() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
- hasInputOutputClass(Class<I>, Class<O>) - Method in class ai.djl.serving.wlm.ModelInfo
-
Clarifies the input and output class when not specified.
I
- id - Variable in class ai.djl.serving.wlm.WorkerPoolConfig
- increaseRef() - Method in class ai.djl.serving.wlm.WorkerPool
-
Increases the reference count.
- inferModelNameFromUrl(String) - Static method in class ai.djl.serving.wlm.ModelInfo
-
Infer model name form model URL in case model name is not provided.
- initialize() - Method in class ai.djl.serving.wlm.ModelInfo
-
Initialize the worker.
- initialize() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Initialize the worker.
- initWorkers(String) - Method in class ai.djl.serving.wlm.WorkerPool
-
Initializes new worker capacities for this model.
- initWorkers(String, int, int) - Method in class ai.djl.serving.wlm.WorkerPool
-
Initializes new worker capacities for this model.
- isAllWorkerBusy() - Method in class ai.djl.serving.wlm.WorkerPool
-
Returns
true
if all workers are busy. - isAllWorkerDied() - Method in class ai.djl.serving.wlm.WorkerPool
-
Return if all workers died.
- isDebug() - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Returns if debug is enabled.
- isFinished() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator
-
Checks if this
BatchAggregator
and the thread can be shutdown or if this aggregator waits for more data. - isFinished() - Method in class ai.djl.serving.wlm.TemporaryBatchAggregator
-
Checks if this
BatchAggregator
and the thread can be shutdown or if this aggregator waits for more data. - isFixPoolThread() - Method in class ai.djl.serving.wlm.WorkerThread
-
check if this worker is instantiate is one of the fix threads of a pool.
- isFullyScaled() - Method in class ai.djl.serving.wlm.WorkerPool
-
Returns if the worker groups is fully scaled.
- isParallelLoading() - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns if the worker type can be load parallel on multiple devices.
- isParallelLoading() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns if the worker type can be load parallel on multiple devices.
- isRunning() - Method in class ai.djl.serving.wlm.WorkerThread
-
Returns true if the worker thread is running.
J
- Job<I,O> - Class in ai.djl.serving.wlm
-
A class represents an inference job.
- Job(WorkerPoolConfig<I, O>, I) - Constructor for class ai.djl.serving.wlm.Job
-
Constructs a new
Job
instance. - Job(WorkerPoolConfig<I, O>, I, JobFunction<I, O>) - Constructor for class ai.djl.serving.wlm.Job
-
Constructs a new
Job
instance. - JobFunction<I,O> - Interface in ai.djl.serving.wlm
-
A function describing the action to take in a
Job
.
L
- LmiUtils - Class in ai.djl.serving.wlm
-
A utility class to detect optimal engine for LMI model.
- load(Device) - Method in class ai.djl.serving.wlm.ModelInfo
-
Loads the worker type to the specified device.
- load(Device) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Loads the worker type to the specified device.
M
- maxBatchDelayMillis - Variable in class ai.djl.serving.wlm.WorkerPoolConfig
- maxIdleSeconds - Variable in class ai.djl.serving.wlm.WorkerPoolConfig
- maxWorkers - Variable in class ai.djl.serving.wlm.WorkerPoolConfig
- minWorkers - Variable in class ai.djl.serving.wlm.WorkerPoolConfig
- ModelInfo<I,O> - Class in ai.djl.serving.wlm
-
A class represent a loaded model and it's metadata.
- ModelInfo(String) - Constructor for class ai.djl.serving.wlm.ModelInfo
-
Constructs a new
ModelInfo
instance. - ModelInfo(String, String, Criteria<I, O>) - Constructor for class ai.djl.serving.wlm.ModelInfo
-
Constructs a
ModelInfo
based on aCriteria
. - ModelInfo(String, String, String, String, String, Class<I>, Class<O>, int, int, int, int, int, int) - Constructor for class ai.djl.serving.wlm.ModelInfo
-
Constructs a new
ModelInfo
instance. - ModelInfo.ModelThread - Class in ai.djl.serving.wlm
- ModelThread(Device) - Constructor for class ai.djl.serving.wlm.ModelInfo.ModelThread
- modelUrl - Variable in class ai.djl.serving.wlm.WorkerPoolConfig
N
- name - Variable in class ai.djl.serving.wlm.Adapter
- newInstance(WorkerPoolConfig<?, ?>, String, String) - Static method in class ai.djl.serving.wlm.Adapter
-
Constructs a new
Adapter
. - newThread(Device) - Method in class ai.djl.serving.wlm.ModelInfo
-
Starts a new
WorkerThread
for thisWorkerPoolConfig
. - newThread(Device) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Starts a new
WorkerThread
for thisWorkerPoolConfig
.
O
- optFixPoolThread(boolean) - Method in class ai.djl.serving.wlm.WorkerThread.Builder
-
Sets if the workerThread should be part of the fixed pool.
P
- PENDING - ai.djl.serving.wlm.WorkerPoolConfig.Status
- PermanentBatchAggregator<I,O> - Class in ai.djl.serving.wlm
-
a batch aggregator that never terminates by itself.
- PermanentBatchAggregator(WorkerPoolConfig<I, O>, LinkedBlockingDeque<WorkerJob<I, O>>) - Constructor for class ai.djl.serving.wlm.PermanentBatchAggregator
-
Constructs a
PermanentBatchAggregator
instance. - pollBatch() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator
-
Fills in the list with a batch of jobs.
- pollBatch() - Method in class ai.djl.serving.wlm.TemporaryBatchAggregator
-
Fills in the list with a batch of jobs.
- postWorkflowParsing(String) - Method in class ai.djl.serving.wlm.ModelInfo
-
Performs post workflow parsing initialization.
- PyAdapter - Class in ai.djl.serving.wlm
-
An overload of
Adapter
for the python engine. - PyAdapter(String, String) - Constructor for class ai.djl.serving.wlm.PyAdapter
-
Constructs an
Adapter
.
Q
- queueSize - Variable in class ai.djl.serving.wlm.WorkerPoolConfig
R
- READY - ai.djl.serving.wlm.WorkerPoolConfig.Status
- register(WorkerPool<I, O>) - Method in class ai.djl.serving.wlm.Adapter
-
Registers this adapter in a worker pool.
- registerAdapter(Adapter) - Method in class ai.djl.serving.wlm.ModelInfo
-
Adds an adapter to this
ModelInfo
. - registerJob(WorkerPoolConfig<I, O>, WorkerPoolConfig.ThreadConfig<I, O>) - Method in class ai.djl.serving.wlm.Adapter
-
Creates a
WorkerJob
to register this adapter in aWorkerThread
. - registerPredictor(Predictor<?, ?>) - Method in class ai.djl.serving.wlm.Adapter
- registerPredictor(Predictor<?, ?>) - Method in class ai.djl.serving.wlm.PyAdapter
- registerWorkerPool(WorkerPoolConfig<I, O>) - Method in class ai.djl.serving.wlm.WorkLoadManager
-
Registers a
WorkerPool
(model). - run() - Method in class ai.djl.serving.wlm.WorkerThread
- run(List<Job<I, O>>) - Method in class ai.djl.serving.wlm.ModelInfo.ModelThread
- run(List<Job<I, O>>) - Method in class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig
-
Runs the work on the
WorkerThread
and stores in the job. - runAll(List<Job<I, O>>, JobFunction<I, O>) - Static method in class ai.djl.serving.wlm.Job
-
Runs a
JobFunction
on a batch of jobs and sets the result in their output. - runJob(Job<I, O>) - Method in class ai.djl.serving.wlm.WorkLoadManager
-
Adds an inference job to the job queue of the next free worker.
S
- SageMakerUtils - Class in ai.djl.serving.wlm
-
A utility class to detect optimal engine for SageMaker saved model.
- scaleWorkers(String, int, int) - Method in class ai.djl.serving.wlm.WorkerPool
-
Sets new worker capacities for this model.
- setBatchSize(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Sets the default batchSize for workers.
- setBatchSize(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Sets the configured batch size.
- setDevice(Device) - Method in class ai.djl.serving.wlm.WorkerThread.Builder
-
RSets the device to run operations on.
- setFailOutput(Job<Input, Output>, int, String) - Static method in class ai.djl.serving.wlm.Job
-
Sets a
Job
output to a failure. - setId(String) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Sets the worker type ID.
- setJobQueue(LinkedBlockingDeque<WorkerJob<I, O>>) - Method in class ai.djl.serving.wlm.WorkerThread.Builder
-
Sets the jobQueue used to poll for new jobs.
- setJobQueueSize(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Sets the default job queue size.
- setLoadOnDevices(String) - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Sets the devices the model will be loaded on at startup.
- setMaxBatchDelayMillis(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Sets the default max batch delay in milliseconds for the working queue.
- setMaxBatchDelayMillis(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Sets the maximum delay in milliseconds to aggregate a batch.
- setMaxIdleSeconds(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Sets the default max idle time in seconds for workers.
- setMaxIdleSeconds(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Sets the configured max idle time in seconds of workers.
- setMaxWorkers(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Sets the starting number of max workers.
- setMinMaxWorkers(int, int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Sets the starting minimum and maximum number of workers.
- setMinWorkers(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Sets the starting number of min workers.
- setOutput(O) - Method in class ai.djl.serving.wlm.Job
-
Sets the output of the job.
- setQueueSize(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Sets the configured size of the workers queue.
- setReservedMemoryMb(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager
-
Sets the reserved memory in MB.
- shutdown() - Method in class ai.djl.serving.wlm.WorkerPool
-
Shuts down all the worker threads in the work pool.
- shutdown(WorkerState) - Method in class ai.djl.serving.wlm.WorkerThread
-
Shuts down the worker thread.
- shutdownWorkers() - Method in class ai.djl.serving.wlm.WorkerPool
-
Shutdown all works.
- src - Variable in class ai.djl.serving.wlm.Adapter
T
- TemporaryBatchAggregator<I,O> - Class in ai.djl.serving.wlm
-
a batch aggregator that terminates after a maximum idle time.
- TemporaryBatchAggregator(WorkerPoolConfig<I, O>, LinkedBlockingDeque<WorkerJob<I, O>>) - Constructor for class ai.djl.serving.wlm.TemporaryBatchAggregator
-
a batch aggregator that terminates after a maximum idle time.
- ThreadConfig(Device) - Constructor for class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig
- toString() - Method in class ai.djl.serving.wlm.WorkerPoolConfig
U
- unregister(WorkerPool<I, O>, String) - Static method in class ai.djl.serving.wlm.Adapter
-
Unregisters an adapter in a worker pool.
- unregisterAdapter(String) - Method in class ai.djl.serving.wlm.ModelInfo
-
Removes an adapter from this
ModelInfo
. - unregisterJob(WorkerPoolConfig<I, O>, WorkerPoolConfig.ThreadConfig<I, O>) - Method in class ai.djl.serving.wlm.Adapter
-
Creates a
WorkerJob
to unregister this adapter from aWorkerThread
. - unregisterPredictor(Predictor<?, ?>) - Method in class ai.djl.serving.wlm.Adapter
- unregisterPredictor(Predictor<?, ?>) - Method in class ai.djl.serving.wlm.PyAdapter
- unregisterWorkerPool(WorkerPoolConfig<?, ?>) - Method in class ai.djl.serving.wlm.WorkLoadManager
-
Removes a worker pool from management.
V
- valueOf(String) - Static method in enum ai.djl.serving.wlm.WorkerPoolConfig.Status
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum ai.djl.serving.wlm.WorkerState
-
Returns the enum constant of this type with the specified name.
- values() - Static method in enum ai.djl.serving.wlm.WorkerPoolConfig.Status
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum ai.djl.serving.wlm.WorkerState
-
Returns an array containing the constants of this enum type, in the order they are declared.
- version - Variable in class ai.djl.serving.wlm.WorkerPoolConfig
W
- withDefaultDevice(String) - Method in class ai.djl.serving.wlm.ModelInfo
-
Returns the default device for this model if device is null.
- withDefaultDevice(String) - Method in class ai.djl.serving.wlm.WorkerPoolConfig
-
Returns the default device for this model if device is null.
- WlmCapacityException - Exception in ai.djl.serving.wlm.util
-
Thrown to throttle when a job is run but the job queue capacity is exceeded.
- WlmCapacityException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmCapacityException
-
Constructs a
WlmCapacityException
with the specified detail message. - WlmCapacityException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.util.WlmCapacityException
-
Constructs a
WlmCapacityException
with the specified detail message and cause. - WlmConfigManager - Class in ai.djl.serving.wlm.util
-
This manages some configurations used by the
WorkLoadManager
. - WlmException - Exception in ai.djl.serving.wlm.util
-
Thrown when an exception occurs inside the
WorkLoadManager
. - WlmException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmException
-
Constructs a
WlmException
with the specified detail message. - WlmException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.util.WlmException
-
Constructs a
WlmException
with the specified detail message and cause. - WlmOutOfMemoryException - Exception in ai.djl.serving.wlm.util
-
Thrown when no enough memory to load the model.
- WlmOutOfMemoryException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmOutOfMemoryException
-
Constructs a
WlmOutOfMemoryException
with the specified detail message. - WlmShutdownException - Exception in ai.djl.serving.wlm.util
-
Thrown when a job is run but all workers are shutdown.
- WlmShutdownException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmShutdownException
-
Constructs a
WlmShutdownException
with the specified detail message. - WlmShutdownException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.util.WlmShutdownException
-
Constructs a
WlmShutdownException
with the specified detail message and cause. - WORKER_BUSY - ai.djl.serving.wlm.WorkerState
- WORKER_ERROR - ai.djl.serving.wlm.WorkerState
- WORKER_MODEL_LOADED - ai.djl.serving.wlm.WorkerState
- WORKER_SCALED_DOWN - ai.djl.serving.wlm.WorkerState
- WORKER_STARTED - ai.djl.serving.wlm.WorkerState
- WORKER_STOPPED - ai.djl.serving.wlm.WorkerState
- WorkerGroup<I,O> - Class in ai.djl.serving.wlm
- WorkerIdGenerator - Class in ai.djl.serving.wlm
-
class to generate an unique worker id.
- WorkerIdGenerator() - Constructor for class ai.djl.serving.wlm.WorkerIdGenerator
- WorkerJob<I,O> - Class in ai.djl.serving.wlm.util
-
A
Job
containing metadata from theWorkLoadManager
. - WorkerJob(Job<I, O>, CompletableFuture<O>) - Constructor for class ai.djl.serving.wlm.util.WorkerJob
-
Constructs a new
WorkerJob
. - WorkerPool<I,O> - Class in ai.djl.serving.wlm
-
Manages the work load for a single model.
- WorkerPoolConfig<I,O> - Class in ai.djl.serving.wlm
-
A
WorkerPoolConfig
represents a task that could be run in theWorkLoadManager
. - WorkerPoolConfig() - Constructor for class ai.djl.serving.wlm.WorkerPoolConfig
- WorkerPoolConfig.Status - Enum in ai.djl.serving.wlm
-
An enum represents state of a worker type.
- WorkerPoolConfig.ThreadConfig<I,O> - Class in ai.djl.serving.wlm
-
The part of the
WorkerPoolConfig
for an individualWorkerThread
. - WorkerState - Enum in ai.djl.serving.wlm
-
An enum represents state of a worker.
- WorkerThread<I,O> - Class in ai.djl.serving.wlm
-
The
WorkerThread
is the worker managed by theWorkLoadManager
. - WorkerThread.Builder<I,O> - Class in ai.djl.serving.wlm
-
A Builder to construct a
WorkerThread
. - WorkLoadManager - Class in ai.djl.serving.wlm
-
WorkLoadManager is responsible to manage the work load of worker thread.
- WorkLoadManager() - Constructor for class ai.djl.serving.wlm.WorkLoadManager
-
Constructs a
WorkLoadManager
instance.
All Classes All Packages