Package ai.djl.serving.wlm
Class WorkerPool<I,O>
java.lang.Object
ai.djl.serving.wlm.WorkerPool<I,O>
Manages the work load for a single model.
-
Method Summary
Modifier and TypeMethodDescriptionvoid
cleanup()
removes all stopped workers and workers in state error from the pool.int
Decrease the reference count and return the current count.Returns theJobQueue
for this model.int
Returns the maximum number of workers for a model across all devices.Map<ai.djl.Device,
WorkerGroup<I, O>> Returns a map ofWorkerGroup
.List<WorkerThread<I,
O>> Returns a list of worker thread.getWpc()
Returns the model of the worker pool.void
Increases the reference count.void
initWorkers
(String deviceName) Initializes new worker capacities for this model.void
initWorkers
(String deviceName, int minWorkers, int maxWorkers) Initializes new worker capacities for this model.boolean
Returnstrue
if all workers are busy.boolean
Return if all workers died.boolean
Returns if the worker groups is fully scaled.void
scaleWorkers
(String deviceName, int minWorkers, int maxWorkers) Sets new worker capacities for this model.void
shutdown()
Shuts down all the worker threads in the work pool.void
Shutdown all works.
-
Method Details
-
increaseRef
public void increaseRef()Increases the reference count. -
decreaseRef
public int decreaseRef()Decrease the reference count and return the current count.- Returns:
- the current count
-
getWpc
Returns the model of the worker pool.- Returns:
- the model of the worker pool
-
getWorkerGroups
Returns a map ofWorkerGroup
.- Returns:
- a map of
WorkerGroup
-
getWorkers
Returns a list of worker thread.- Returns:
- the workers
-
getJobQueue
Returns theJobQueue
for this model.- Returns:
- the jobQueue
-
getMaxWorkers
public int getMaxWorkers()Returns the maximum number of workers for a model across all devices.- Returns:
- the maximum number of workers for a model across all devices
-
isAllWorkerDied
public boolean isAllWorkerDied()Return if all workers died.- Returns:
- true if all workers died
-
isAllWorkerBusy
public boolean isAllWorkerBusy()Returnstrue
if all workers are busy.- Returns:
true
if all workers are busy
-
isFullyScaled
public boolean isFullyScaled()Returns if the worker groups is fully scaled.- Returns:
- true if the worker groups is fully scaled
-
initWorkers
Initializes new worker capacities for this model.- Parameters:
deviceName
- the device for the model, null for default devices
-
initWorkers
Initializes new worker capacities for this model.- Parameters:
deviceName
- the device for the model, null for default devicesminWorkers
- minimum amount of workers (-1 for model default).maxWorkers
- maximum amount of workers (-1 for model default).
-
scaleWorkers
Sets new worker capacities for this model.- Parameters:
deviceName
- the device for the model, null for all loaded devicesminWorkers
- minimum amount of workers.maxWorkers
- maximum amount of workers.
-
shutdownWorkers
public void shutdownWorkers()Shutdown all works. -
cleanup
public void cleanup()removes all stopped workers and workers in state error from the pool. -
shutdown
public void shutdown()Shuts down all the worker threads in the work pool.
-