ai.djl.serving.wlm

Class ModelInfo

java.lang.Object

ai.djl.serving.wlm.ModelInfo

All Implemented Interfaces:

java.lang.AutoCloseable

public final class ModelInfo
extends java.lang.Object
implements java.lang.AutoCloseable

A class represent a loaded model and it's metadata.

Constructor Summary

Constructors

Constructor and Description
ModelInfo(java.lang.String modelName, java.lang.String version, java.lang.String modelUrl, ai.djl.repository.zoo.ZooModel<ai.djl.modality.Input,ai.djl.modality.Output> model, int queueSize, int maxIdleTime, int maxBatchDelay, int batchSize) Constructs a new ModelInfo instance.

Method Summary

Modifier and Type	Method and Description
void	close()
ModelInfo	configureModelBatch(int batchSize, int maxBatchDelay) Sets a new batchSize and returns a new configured ModelInfo object.
ModelInfo	configurePool(int maxIdleTime) Sets new configuration for the workerPool backing this model and returns a new configured ModelInfo object.
boolean	equals(java.lang.Object o)
int	getBatchSize() Returns the configured batch size.
int	getMaxBatchDelay() Returns the maximum delay in milliseconds to aggregate a batch.
int	getMaxIdleTime() returns the configured maxIdleTime of workers.
int	getMaxWorkers() Returns the configured maximum number of workers.
int	getMinWorkers() Returns the configured minimum number of workers.
ai.djl.repository.zoo.ZooModel<ai.djl.modality.Input,ai.djl.modality.Output>	getModel() Returns the loaded ZooModel.
java.nio.file.Path	getModelDir() Returns the model cache directory.
java.lang.String	getModelName() Returns the model name.
java.lang.String	getModelUrl() Returns the model url.
int	getQueueSize() Returns the configured size of the workers queue.
java.lang.String	getVersion() Returns the model version.
int	hashCode()
static java.lang.String	inferModelNameFromUrl(java.lang.String url) Infer model name form model URL in case model name is not provided.
ModelInfo	scaleWorkers(int minWorkers, int maxWorkers) Sets new workers capcities for this model and returns a new configured ModelInfo object.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

ModelInfo

public ModelInfo(java.lang.String modelName,
                 java.lang.String version,
                 java.lang.String modelUrl,
                 ai.djl.repository.zoo.ZooModel<ai.djl.modality.Input,ai.djl.modality.Output> model,
                 int queueSize,
                 int maxIdleTime,
                 int maxBatchDelay,
                 int batchSize)

Constructs a new ModelInfo instance.

Parameters:

modelName - the name of the model that will be used as HTTP endpoint

version - the version of the model

modelUrl - the model url

model - the ZooModel

queueSize - the maximum request queue size

maxIdleTime - the initial maximum idle time for workers.

maxBatchDelay - the initial maximum delay when scaling up before giving up.

batchSize - the batch size for this model.

Method Detail

configureModelBatch

public ModelInfo configureModelBatch(int batchSize,
                                     int maxBatchDelay)

Sets a new batchSize and returns a new configured ModelInfo object. You have to triggerUpdates in the ModelManager using this new model.

Parameters:

batchSize - the batchSize to set

maxBatchDelay - maximum time to wait for a free space in worker queue after scaling up workers before giving up to offer the job to the queue.

Returns:

new configured ModelInfo.

scaleWorkers

public ModelInfo scaleWorkers(int minWorkers,
                              int maxWorkers)

Sets new workers capcities for this model and returns a new configured ModelInfo object. You have to triggerUpdates in the ModelManager using this new model.

Parameters:

minWorkers - minimum amount of workers.

maxWorkers - maximum amount of workers.

Returns:

new configured ModelInfo.

configurePool

public ModelInfo configurePool(int maxIdleTime)

Sets new configuration for the workerPool backing this model and returns a new configured ModelInfo object. You have to triggerUpdates in the ModelManager using this new model.

Parameters:

maxIdleTime - time a WorkerThread can be idle before scaling down this worker.

Returns:

new configured ModelInfo.

getModel

public ai.djl.repository.zoo.ZooModel<ai.djl.modality.Input,ai.djl.modality.Output> getModel()

Returns the loaded ZooModel.

Returns:

the loaded ZooModel

getModelName

public java.lang.String getModelName()

Returns the model name.

Returns:

the model name

getVersion

public java.lang.String getVersion()

Returns the model version.

Returns:

the model version

getModelUrl

public java.lang.String getModelUrl()

Returns the model url.

Returns:

the model url

getModelDir

public java.nio.file.Path getModelDir()

Returns the model cache directory.

Returns:

the model cache directory

getMaxIdleTime

public int getMaxIdleTime()

returns the configured maxIdleTime of workers.

Returns:

the maxIdleTime

getMinWorkers

public int getMinWorkers()

Returns the configured minimum number of workers.

Returns:

the configured minimum number of workers

getMaxWorkers

public int getMaxWorkers()

Returns the configured maximum number of workers.

Returns:

the configured maximum number of workers

getBatchSize

public int getBatchSize()

Returns the configured batch size.

Returns:

the configured batch size

getMaxBatchDelay

public int getMaxBatchDelay()

Returns the maximum delay in milliseconds to aggregate a batch.

Returns:

the maximum delay in milliseconds to aggregate a batch

getQueueSize

public int getQueueSize()

Returns the configured size of the workers queue.

Returns:

requested size of the workers queue.

close

public void close()

Specified by:

close in interface java.lang.AutoCloseable

inferModelNameFromUrl

public static java.lang.String inferModelNameFromUrl(java.lang.String url)

Infer model name form model URL in case model name is not provided.

Parameters:

url - the model URL

Returns:

the model name

equals

public boolean equals(java.lang.Object o)

Overrides:

equals in class java.lang.Object

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object