Package ai.djl.serving.wlm

Class ModelInfo<I,​O>

  • public final class ModelInfo<I,​O>
extends WorkerPoolConfig<I,​O>
    A class represent a loaded model and it's metadata.

    • Constructor Detail

      • ModelInfo

        public ModelInfo​(java.lang.String modelUrl)
        Constructs a new ModelInfo instance.
        Parameters:
        modelUrl - the model Url

      • ModelInfo

        public ModelInfo​(java.lang.String id,
                 java.lang.String modelUrl,
                 ai.djl.repository.zoo.Criteria<I,​O> criteria)
        Constructs a ModelInfo based on a Criteria.
        Parameters:
        id - the id for the created ModelInfo
        modelUrl - the model Url
        criteria - the model criteria

      • ModelInfo

        public ModelInfo​(java.lang.String id,
                 java.lang.String modelUrl,
                 java.lang.String version,
                 java.lang.String engineName,
                 java.lang.String loadOnDevices,
                 java.lang.Class<I> inputClass,
                 java.lang.Class<O> outputClass,
                 int queueSize,
                 int maxIdleSeconds,
                 int maxBatchDelayMillis,
                 int batchSize,
                 int minWorkers,
                 int maxWorkers)
        Constructs a new ModelInfo instance.
        Parameters:
        id - the ID of the model that will be used by workflow
        modelUrl - the model url
        version - the version of the model
        engineName - the engine to load the model
        loadOnDevices - the devices to load the model on
        inputClass - the model input class
        outputClass - the model output class
        queueSize - the maximum request queue size
        maxIdleSeconds - the initial maximum idle time for workers
        maxBatchDelayMillis - the initial maximum delay when scaling up before giving up
        batchSize - the batch size for this model
        minWorkers - the minimum number of workers
        maxWorkers - the maximum number of workers

    • Method Detail

      • postWorkflowParsing

        public void postWorkflowParsing​(java.lang.String workflowDir)
        Performs post workflow parsing initialization.
        Parameters:
        workflowDir - the workflow parent directory

      • load

        public void load​(ai.djl.Device device)
          throws ai.djl.ModelException,
                 java.io.IOException
        Loads the worker type to the specified device.
        Specified by:
        load in class WorkerPoolConfig<I,​O>
        Parameters:
        device - the device to load worker type on
        Throws:
        ai.djl.ModelException - if failed to load the specified model
        java.io.IOException - if failed to read worker type file

      • getModels

        public java.util.Map<ai.djl.Device,​ai.djl.repository.zoo.ZooModel<I,​O>> getModels()
        Returns all loaded models.
        Returns:
        all loaded models

      • getModel

        public ai.djl.repository.zoo.ZooModel<I,​O> getModel​(ai.djl.Device device)
        Returns the loaded ZooModel for a device.
        Parameters:
        device - the device to return the model on
        Returns:
        the loaded ZooModel

      • getEngine

        public ai.djl.engine.Engine getEngine()
        Returns the engine.
        Returns:
        the engine

      • getEngineName

        public java.lang.String getEngineName()
        Returns the engine name.
        Returns:
        the engine name

      • getInputClass

        public java.lang.Class<I> getInputClass()
        Returns the model input class.
        Returns:
        the model input class

      • getOutputClass

        public java.lang.Class<O> getOutputClass()
        Returns the model output class.
        Returns:
        the model output class

      • hasInputOutputClass

        public void hasInputOutputClass​(java.lang.Class<I> inputClass,
                                java.lang.Class<O> outputClass)
        Clarifies the input and output class when not specified.

        Warning: This is intended for internal use with reflection.

        Parameters:
        inputClass - the model input class
        outputClass - the model output class

      • getMinWorkers

        public int getMinWorkers​(ai.djl.Device device)
        Returns the minimum number of workers.
        Overrides:
        getMinWorkers in class WorkerPoolConfig<I,​O>
        Parameters:
        device - the device to get the min workers for
        Returns:
        the minimum number of workers

      • getMaxWorkers

        public int getMaxWorkers​(ai.djl.Device device)
        Returns the maximum number of workers.
        Overrides:
        getMaxWorkers in class WorkerPoolConfig<I,​O>
        Parameters:
        device - the device to get the max workers for
        Returns:
        the maximum number of workers

      • initialize

        public void initialize()
                throws java.io.IOException,
                       ai.djl.ModelException
        Initialize the worker.
        Specified by:
        initialize in class WorkerPoolConfig<I,​O>
        Throws:
        java.io.IOException - if failed to download worker
        ai.djl.repository.zoo.ModelNotFoundException - if model not found
        ai.djl.ModelException

      • registerAdapter

        public void registerAdapter​(Adapter adapter)
        Adds an adapter to this ModelInfo.
        Parameters:
        adapter - the adapter to add

      • unregisterAdapter

        public Adapter unregisterAdapter​(java.lang.String name)
        Removes an adapter from this ModelInfo.
        Parameters:
        name - the adapter to remove
        Returns:
        the removed adapter

      • getAdapters

        public java.util.Map<java.lang.String,​Adapter> getAdapters()
        Returns the adapters for this model.
        Returns:
        the adapters for this model

      • getAdapter

        public Adapter getAdapter​(java.lang.String name)
        Returns an adapter on this ModelInfo.
        Parameters:
        name - the adapter name to get
        Returns:
        the adapter

      • inferModelNameFromUrl

        public static java.lang.String inferModelNameFromUrl​(java.lang.String url)
        Infer model name form model URL in case model name is not provided.
        Parameters:
        url - the model URL
        Returns:
        the model name

      • withDefaultDevice

        public ai.djl.Device withDefaultDevice​(java.lang.String deviceName)
        Returns the default device for this model if device is null.
        Overrides:
        withDefaultDevice in class WorkerPoolConfig<I,​O>
        Parameters:
        deviceName - the device to use if it is not null
        Returns:
        a non-null device

      • getLoadOnDevices

        public java.lang.String[] getLoadOnDevices()
        Returns the devices the worker type will be loaded on at startup.
        Specified by:
        getLoadOnDevices in class WorkerPoolConfig<I,​O>
        Returns:
        the devices the worker type will be loaded on at startup

      • isParallelLoading

        public boolean isParallelLoading()
        Returns if the worker type can be load parallel on multiple devices.
        Specified by:
        isParallelLoading in class WorkerPoolConfig<I,​O>
        Returns:
        if the worker type can be load parallel on multiple devices