Module org.elasticsearch.server
Package org.elasticsearch.inference

Interface InferenceService

All Superinterfaces:
AutoCloseable, Closeable
public interface InferenceService extends Closeable

  • Method Details

    • init

      default void init(Client client)

    • name

      String name()

    • parseRequestConfig

      void parseRequestConfig(String modelId, TaskType taskType, Map<String,Object> config, Set<String> platfromArchitectures, ActionListener<Model> parsedModelListener)
      Parse model configuration from the config map from a request and return the parsed Model. This requires that both the secrets and service settings be contained in the service_settings field. This function modifies config map, fields are removed from the map as they are read.

      If the map contains unrecognized configuration option an ElasticsearchStatusException is thrown.

      Parameters:
      modelId - Model Id
      taskType - The model task type
      config - Configuration options including the secrets
      platfromArchitectures - The Set of platform architectures (OS name and hardware architecture) the cluster nodes and models are running on.
      parsedModelListener - A listener which will handle the resulting model or failure

    • parsePersistedConfigWithSecrets

      Model parsePersistedConfigWithSecrets(String modelId, TaskType taskType, Map<String,Object> config, Map<String,Object> secrets)
      Parse model configuration from config map from persisted storage and return the parsed Model. This requires that secrets and service settings be in two separate maps. This function modifies config map, fields are removed from the map as they are read. If the map contains unrecognized configuration options, no error is thrown.
      Parameters:
      modelId - Model Id
      taskType - The model task type
      config - Configuration options
      secrets - Sensitive configuration options (e.g. api key)
      Returns:
      The parsed Model

    • parsePersistedConfig

      Model parsePersistedConfig(String modelId, TaskType taskType, Map<String,Object> config)
      Parse model configuration from config map from persisted storage and return the parsed Model. This function modifies config map, fields are removed from the map as they are read. If the map contains unrecognized configuration options, no error is thrown.
      Parameters:
      modelId - Model Id
      taskType - The model task type
      config - Configuration options
      Returns:
      The parsed Model

    • infer

      void infer(Model model, @Nullable String query, List<String> input, Map<String,Object> taskSettings, InputType inputType, TimeValue timeout, ActionListener<InferenceServiceResults> listener)
      Perform inference on the model.
      Parameters:
      model - The model
      query - Inference query, mainly for re-ranking
      input - Inference input
      taskSettings - Settings in the request to override the model's defaults
      inputType - For search, ingest etc
      timeout - The timeout for the request
      listener - Inference result listener

    • chunkedInfer

      void chunkedInfer(Model model, @Nullable String query, List<String> input, Map<String,Object> taskSettings, InputType inputType, ChunkingOptions chunkingOptions, TimeValue timeout, ActionListener<List<ChunkedInferenceServiceResults>> listener)
      Chunk long text according to chunkingOptions or the model defaults if chunkingOptions contains unset values.
      Parameters:
      model - The model
      query - Inference query, mainly for re-ranking
      input - Inference input
      taskSettings - Settings in the request to override the model's defaults
      inputType - For search, ingest etc
      chunkingOptions - The window and span options to apply
      timeout - The timeout for the request
      listener - Chunked Inference result listener

    • start

      void start(Model model, ActionListener<Boolean> listener)
      Start or prepare the model for use.
      Parameters:
      model - The model
      listener - The listener

    • stop

      default void stop(String modelId, ActionListener<Boolean> listener)
      Stop the model deployment. The default action does nothing except acknowledge the request (true).
      Parameters:
      modelId - The ID of the model to be stopped
      listener - The listener

    • putModel

      default void putModel(Model modelVariant, ActionListener<Boolean> listener)
      Put the model definition (if applicable) The main purpose of this function is to download ELSER The default action does nothing except acknowledge the request (true).
      Parameters:
      modelVariant - The configuration of the model variant to be downloaded
      listener - The listener

    • isModelDownloaded

      default void isModelDownloaded(Model model, ActionListener<Boolean> listener)
      Checks if the modelId has been downloaded to the local Elasticsearch cluster using the trained models API The default action does nothing except acknowledge the request (false). Any internal services should Override this method.
      Parameters:
      model -
      listener - The listener

    • checkModelConfig

      default void checkModelConfig(Model model, ActionListener<Model> listener)
      Optionally test the new model configuration in the inference service. This function should be called when the model is first created, the default action is to do nothing.
      Parameters:
      model - The new model
      listener - The listener

    • isInClusterService

      default boolean isInClusterService()
      Return true if this model is hosted in the local Elasticsearch cluster
      Returns:
      True if in cluster

    • getMinimalSupportedVersion

      TransportVersion getMinimalSupportedVersion()
      Defines the version required across all clusters to use this service
      Returns:
      TransportVersion specifying the version