Annotation Interface RegisterAiService

@Retention(RUNTIME) @Target(TYPE) public @interface RegisterAiService
Used to create LangChain4j's AiServices in a declarative manner that the application can then use simply by using the class as a CDI bean. Under the hood LangChain4j's AiServices.builder(Class) is called while also providing the builder with the proper
invalid reference 
ChatLanguageModel
bean (mandatory), tools bean (optional), ChatMemoryProvider and ContentRetriever beans (which by default are configured if such beans exist).

NOTE: The resulting CDI bean is RequestScoped by default. If you need to change the scope, simply annotate the class with a CDI scope. CAUTION: When using anything other than the request scope, you need to be very careful with the chat memory implementation.

NOTE: When the application also contains the quarkus-micrometer extension, metrics are automatically generated for the method invocations.

See also ToolBox, SeedMemory

  • Element Details

    • streamingChatLanguageModelSupplier

      Class<? extends Supplier<dev.langchain4j.model.chat.StreamingChatModel>> streamingChatLanguageModelSupplier
      Configures the way to obtain the
      invalid reference 
      StreamingChatLanguageModel
      to use. If not configured, the default CDI bean implementing the model is looked up. Such a bean provided automatically by extensions such as quarkus-langchain4j-openai, quarkus-langchain4j-azure-openai or quarkus-langchain4j-hugging-face
      Default:
      io.quarkiverse.langchain4j.RegisterAiService.BeanStreamingChatLanguageModelSupplier.class

    • chatLanguageModelSupplier

      Class<? extends Supplier<dev.langchain4j.model.chat.ChatModel>> chatLanguageModelSupplier
      Configures the way to obtain the
      invalid reference 
      ChatLanguageModel
      to use. If not configured, the default CDI bean implementing the model is looked up. Such a bean provided automatically by extensions such as quarkus-langchain4j-openai, quarkus-langchain4j-azure-openai or quarkus-langchain4j-hugging-face
      Default:
      io.quarkiverse.langchain4j.RegisterAiService.BeanChatLanguageModelSupplier.class

    • modelName

      String modelName
      When chatLanguageModelSupplier is set to BeanChatLanguageModelSupplier.class (which is the default) this allows the selection of the
      invalid reference 
      ChatLanguageModel
      CDI bean to use.

      If not set, the default model (i.e. the one configured without setting the model name) is used. An example of the default model configuration is the following: quarkus.langchain4j.openai.chat-model.model-name=gpt-4-turbo-preview

      If set, it uses the model configured by name. For example if this is set to somename an example configuration value for that named model could be: quarkus.langchain4j.somename.openai.chat-model.model-name=gpt-4-turbo-preview

      If the model needs to be selected at runtime, then ModelName can be used as a method parameter of an AiService.

      Default:
      "<default>"

    • maxSequentialToolInvocations

      int maxSequentialToolInvocations
      Defines the maximum number of sequential calls to tools while handling a single chat request. If this number is exceeded, the chat request will fail. If not specified (left to zero) for a specific AI service, the AI service will use the value of the common quarkus.langchain4j.ai-service.max-tool-executions property. If that property is unset too, the default is 10 invocations.
      Default:
      0

    • tools

      Class<?>[] tools
      Tool classes to use. All tools are expected to be CDI beans.
      Default:
      {}

    • toolHallucinationStrategy

      Class<? extends Function<dev.langchain4j.agent.tool.ToolExecutionRequest,dev.langchain4j.data.message.ToolExecutionResultMessage>> toolHallucinationStrategy
      Strategy to be used when the AI service hallucinates the tool name.
      Default:
      io.quarkiverse.langchain4j.RegisterAiService.BeanIfExistsToolHallucinationStrategy.class

    • chatMemoryProviderSupplier

      Class<? extends Supplier<dev.langchain4j.memory.chat.ChatMemoryProvider>> chatMemoryProviderSupplier
      Configures the way to obtain the ChatMemoryProvider.

      Be default, Quarkus configures a ChatMemoryProvider bean that uses a InMemoryChatMemoryStore bean as the backing store. The default type for the actual ChatMemory is MessageWindowChatMemory and it is configured with the value of the quarkus.langchain4j.chat-memory.memory-window.max-messages configuration property (which default to 10) as a way of limiting the number of messages in each chat.

      If the application provides its own ChatMemoryProvider bean, that takes precedence over what Quarkus provides as the default.

      If the application provides an implementation of ChatMemoryStore, then that is used instead of the default InMemoryChatMemoryStore.

      In the most advances case, an arbitrary ChatMemoryProvider can be used by having a custom Supplier<ChatMemoryProvider> configured in this property. Supplier<dev.langchain4j.memory.chat.ChatMemoryProvider> needs to be provided.

      Default:
      io.quarkiverse.langchain4j.RegisterAiService.BeanChatMemoryProviderSupplier.class

    • retrievalAugmentor

      Class<? extends Supplier<dev.langchain4j.rag.RetrievalAugmentor>> retrievalAugmentor
      Configures the way to obtain the RetrievalAugmentor to use (when using RAG). The Supplier may or may not be a CDI bean (but most typically it will, so consider adding a bean-defining annotation to it). If it is not a CDI bean, Quarkus will create an instance by calling its no-arg constructor.

      If unspecified, Quarkus will attempt to locate a CDI bean that implements RetrievalAugmentor and use it if one exists.

      Default:
      io.quarkiverse.langchain4j.RegisterAiService.BeanIfExistsRetrievalAugmentorSupplier.class

    • moderationModelSupplier

      Class<? extends Supplier<dev.langchain4j.model.moderation.ModerationModel>> moderationModelSupplier
      Configures the way to obtain the ModerationModel to use. By default, Quarkus will look for a CDI bean that implements ModerationModel if at least one method is annotated with @Moderate. If an arbitrary ModerationModel instance is needed, a custom implementation of Supplier<dev.langchain4j.model.moderation.ModerationModel> needs to be provided.
      Default:
      io.quarkiverse.langchain4j.RegisterAiService.BeanIfExistsModerationModelSupplier.class

    • toolProviderSupplier

      Class<? extends Supplier<dev.langchain4j.service.tool.ToolProvider>> toolProviderSupplier
      Configures a toolProviderSupplier. It is possible to use together toolProviderSupplier and "normal" tools.
      Default:
      io.quarkiverse.langchain4j.RegisterAiService.BeanIfExistsToolProviderSupplier.class