Package dev.braintrust

Class Braintrust

java.lang.Object
dev.braintrust.Braintrust
public class Braintrust extends Object
Main entry point for the Braintrust SDK.

This class provides access to all Braintrust functionality. Most users will interact with a singleton instance via get(), though you can create independent instances if needed.

The Braintrust instance also provides methods for enabling Braintrust in open telemetry builders.

Additionally, vendor-specific instrumentation or functionality is provided by Braintrust<Vendor Name>. E.g. BraintrustOpenAI, BraintrustAnthropic, etc.

See Also:

  • Method Summary

    Modifier and Type
    Method
    Description
    BraintrustApiClient
    apiClient()
    Deprecated.
    BraintrustConfig
    config()
     
    <INPUT, OUTPUT>
    Eval.Builder<INPUT,OUTPUT>
    evalBuilder()
    Create a new eval builder
    <INPUT, OUTPUT>
    Dataset<INPUT,OUTPUT>
    fetchDataset(String datasetName)
    Deprecated.
    the INPUT and OUTPUT type parameters are not applied at runtime: case values are returned as raw JSON-decoded objects (e.g.
    <INPUT, OUTPUT>
    Dataset<INPUT,OUTPUT>
    fetchDataset(String datasetName, Class<INPUT> inputClass, Class<OUTPUT> outputClass)
    Fetch the latest version of a dataset from Braintrust, deserializing each case's input and expected values into the given types.
    <INPUT, OUTPUT>
    Dataset<INPUT,OUTPUT>
    fetchDataset(String datasetName, String datasetVersion)
    Deprecated.
    the INPUT and OUTPUT type parameters are not applied at runtime; see fetchDataset(String).
    <INPUT, OUTPUT>
    Dataset<INPUT,OUTPUT>
    fetchDataset(String datasetName, String datasetVersion, Class<INPUT> inputClass, Class<OUTPUT> outputClass)
    Fetch a specific version of a dataset from Braintrust, deserializing each case's input and expected values into the given types.
    <INPUT, OUTPUT>
    Dataset<INPUT,OUTPUT>
    fetchDataset(String datasetName, String datasetVersion, Function<Object,INPUT> inputConverter, Function<Object,OUTPUT> outputConverter)
    Fetch a specific version of a dataset from Braintrust, converting each case's input and expected values with the supplied converter functions.
    <INPUT, OUTPUT>
    Dataset<INPUT,OUTPUT>
    fetchDataset(String datasetName, Function<Object,INPUT> inputConverter, Function<Object,OUTPUT> outputConverter)
    Fetch the latest version of a dataset from Braintrust, converting each case's input and expected values with the supplied converter functions.
    <INPUT, OUTPUT>
    Scorer<INPUT,OUTPUT>
    fetchScorer(String scorerSlug)
    Fetch a scorer from Braintrust by slug, using the default project from configuration.
    <INPUT, OUTPUT>
    Scorer<INPUT,OUTPUT>
    fetchScorer(String scorerSlug, String version)
    Fetch a scorer from Braintrust by slug, using the default project from configuration.
    <INPUT, OUTPUT>
    Scorer<INPUT,OUTPUT>
    fetchScorer(String projectName, String scorerSlug, String version)
    Fetch a scorer from Braintrust by project name and slug.
    <INPUT, OUTPUT>
    Scorer<INPUT,OUTPUT>
    fetchScorer(String projectName, String scorerSlug, String version, Function<INPUT,Object> inputConverter, Function<OUTPUT,Object> outputConverter)
    Fetch a scorer from Braintrust by project name and slug, with custom converters that transform the input, output, and expected scorer argument values into a JSON-serializable form.
    <INPUT, OUTPUT>
    Scorer<INPUT,OUTPUT>
    fetchScorer(String scorerSlug, String version, Function<INPUT,Object> inputConverter, Function<OUTPUT,Object> outputConverter)
    Fetch a scorer from Braintrust by slug, using the default project from configuration, with custom converters that transform the input, output, and expected scorer argument values into a JSON-serializable form.
    static Braintrust
    get()
    get or create the global braintrust instance.
    static Braintrust
    get(BraintrustConfig config)
    get or create the global braintrust instance from the given config
    static Braintrust
    of(BraintrustConfig config)
    Create a new Braintrust instance from the given config
    BraintrustOpenApiClient
    openApiClient()
     
    io.opentelemetry.api.OpenTelemetry
    openTelemetryCreate()
    Quick start method that sets up global OpenTelemetry with this Braintrust.
    io.opentelemetry.api.OpenTelemetry
    openTelemetryCreate(boolean registerGlobal)
    Quick start method that sets up OpenTelemetry with this Braintrust.
    void
    openTelemetryEnable(io.opentelemetry.sdk.trace.SdkTracerProviderBuilder tracerProviderBuilder, io.opentelemetry.sdk.logs.SdkLoggerProviderBuilder loggerProviderBuilder, io.opentelemetry.sdk.metrics.SdkMeterProviderBuilder meterProviderBuilder)
    Add braintrust to existing open telemetry builders

    This method provides the most options for configuring Braintrust and OpenTelemetry.
    URI
    projectUri()
    URI to the configured braintrust org and project
    BraintrustPromptLoader
    promptLoader()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • get

      public static Braintrust get()
      get or create the global braintrust instance. Most users will want to use this method to access the Braintrust SDK.

    • get

      public static Braintrust get(BraintrustConfig config)
      get or create the global braintrust instance from the given config

    • of

      public static Braintrust of(BraintrustConfig config)
      Create a new Braintrust instance from the given config

    • apiClient

      @Deprecated public BraintrustApiClient apiClient()
      Deprecated.
      Deprecated. Please use openApiClient() instead

    • projectUri

      public URI projectUri()
      URI to the configured braintrust org and project

    • openTelemetryCreate

      public io.opentelemetry.api.OpenTelemetry openTelemetryCreate()
      Quick start method that sets up global OpenTelemetry with this Braintrust.

      If you're looking for more options for configuring Braintrust/OpenTelemetry, consult the `enable` method.

    • openTelemetryCreate

      public io.opentelemetry.api.OpenTelemetry openTelemetryCreate(boolean registerGlobal)
      Quick start method that sets up OpenTelemetry with this Braintrust.

      If you're looking for more options for configuring Braintrust and OpenTelemetry, consult the `enable` method.

    • openTelemetryEnable

      public void openTelemetryEnable(@Nonnull io.opentelemetry.sdk.trace.SdkTracerProviderBuilder tracerProviderBuilder, @Nonnull io.opentelemetry.sdk.logs.SdkLoggerProviderBuilder loggerProviderBuilder, @Nonnull io.opentelemetry.sdk.metrics.SdkMeterProviderBuilder meterProviderBuilder)
      Add braintrust to existing open telemetry builders

      This method provides the most options for configuring Braintrust and OpenTelemetry. If you're looking for a more user-friendly setup, consult the `openTelemetryCreate` methods.

      NOTE: if your otel setup does not have any particular builder, pass an instance of the default provider builder. E.g. `SdkMeterProvider.builder()`

      NOTE: This method should only be invoked once. Enabling Braintrust multiple times is unsupported and may lead to undesired behavior

    • evalBuilder

      public <INPUT, OUTPUT> Eval.Builder<INPUT,OUTPUT> evalBuilder()
      Create a new eval builder

    • fetchDataset

      @Deprecated public <INPUT, OUTPUT> Dataset<INPUT,OUTPUT> fetchDataset(String datasetName)
      Deprecated.
      the INPUT and OUTPUT type parameters are not applied at runtime: case values are returned as raw JSON-decoded objects (e.g. LinkedHashMap), and accessing them as INPUT/OUTPUT will throw ClassCastException unless those types are Object or Map. Use fetchDataset(String, Class, Class) instead.
      Fetch the latest version of a dataset from Braintrust, using the default project from configuration.

    • fetchDataset

      public <INPUT, OUTPUT> Dataset<INPUT,OUTPUT> fetchDataset(String datasetName, Class<INPUT> inputClass, Class<OUTPUT> outputClass)
      Fetch the latest version of a dataset from Braintrust, deserializing each case's input and expected values into the given types.

      Values are deserialized using the SDK's shared Jackson mapper (see BraintrustJsonMapper), which may be customized via BraintrustJsonMapper.configure(java.util.function.Consumer<com.fasterxml.jackson.databind.ObjectMapper>). For full control over per-value conversion (custom mappers, fixups, generic types), use fetchDataset(String, Function, Function).

      The returned dataset preserves experiment-to-dataset linking when used with Eval: experiments created by Eval.run() are stamped with this dataset's id and version.

      Parameters:
      datasetName - the name of the dataset within the configured project
      inputClass - the type to deserialize each case's input into
      outputClass - the type to deserialize each case's expected into
      Returns:
      a typed view of the remote dataset; cases are fetched lazily, and deserialization errors surface during iteration

    • fetchDataset

      public <INPUT, OUTPUT> Dataset<INPUT,OUTPUT> fetchDataset(String datasetName, @Nullable String datasetVersion, Class<INPUT> inputClass, Class<OUTPUT> outputClass)
      Fetch a specific version of a dataset from Braintrust, deserializing each case's input and expected values into the given types.

      See fetchDataset(String, Class, Class) for deserialization and experiment-linking semantics.

      Parameters:
      datasetName - the name of the dataset within the configured project
      datasetVersion - the dataset version to pin, or null to fetch the latest version upon every cursor open
      inputClass - the type to deserialize each case's input into
      outputClass - the type to deserialize each case's expected into
      Returns:
      a typed view of the remote dataset; cases are fetched lazily, and deserialization errors surface during iteration

    • fetchDataset

      @Deprecated public <INPUT, OUTPUT> Dataset<INPUT,OUTPUT> fetchDataset(String datasetName, @Nullable String datasetVersion)
      Deprecated.
      the INPUT and OUTPUT type parameters are not applied at runtime; see fetchDataset(String). Use fetchDataset(String, String, Class, Class) instead.
      Fetch a specific version of a dataset from Braintrust, using the default project from configuration.

    • fetchDataset

      public <INPUT, OUTPUT> Dataset<INPUT,OUTPUT> fetchDataset(String datasetName, Function<Object,INPUT> inputConverter, Function<Object,OUTPUT> outputConverter)
      Fetch the latest version of a dataset from Braintrust, converting each case's input and expected values with the supplied converter functions.

      Converters receive the raw JSON-decoded value (typically a Map<String, Object>, List, String, Number, Boolean, or null) and are responsible for producing the typed value. This gives the caller full control over deserialization — e.g. a custom Jackson ObjectMapper, generic types, or row fixups: 

      
 Dataset<MyInput, MyOutput> ds = braintrust.fetchDataset(
         "golden-cases",
         raw -> myMapper.convertValue(raw, MyInput.class),
         raw -> myMapper.convertValue(raw, MyOutput.class));

      The returned dataset preserves experiment-to-dataset linking when used with Eval: experiments created by Eval.run() are stamped with this dataset's id and version.

      Parameters:
      datasetName - the name of the dataset within the configured project
      inputConverter - converts each case's raw input value; must tolerate null
      outputConverter - converts each case's raw expected value; must tolerate null
      Returns:
      a typed view of the remote dataset; cases are fetched lazily, and converter errors surface during iteration

    • fetchDataset

      public <INPUT, OUTPUT> Dataset<INPUT,OUTPUT> fetchDataset(String datasetName, @Nullable String datasetVersion, Function<Object,INPUT> inputConverter, Function<Object,OUTPUT> outputConverter)
      Fetch a specific version of a dataset from Braintrust, converting each case's input and expected values with the supplied converter functions.

      See fetchDataset(String, Function, Function) for converter and experiment-linking semantics.

      Parameters:
      datasetName - the name of the dataset within the configured project
      datasetVersion - the dataset version to pin, or null to fetch the latest version upon every cursor open
      inputConverter - converts each case's raw input value; must tolerate null
      outputConverter - converts each case's raw expected value; must tolerate null
      Returns:
      a typed view of the remote dataset; cases are fetched lazily, and converter errors surface during iteration

    • fetchScorer

      public <INPUT, OUTPUT> Scorer<INPUT,OUTPUT> fetchScorer(String scorerSlug)
      Fetch a scorer from Braintrust by slug, using the default project from configuration.
      Parameters:
      scorerSlug - the unique slug identifier for the scorer
      Returns:
      a Scorer that invokes the remote function

    • fetchScorer

      public <INPUT, OUTPUT> Scorer<INPUT,OUTPUT> fetchScorer(String scorerSlug, @Nullable String version)
      Fetch a scorer from Braintrust by slug, using the default project from configuration.
      Parameters:
      scorerSlug - the unique slug identifier for the scorer
      version - optional version of the scorer to fetch
      Returns:
      a Scorer that invokes the remote function

    • fetchScorer

      public <INPUT, OUTPUT> Scorer<INPUT,OUTPUT> fetchScorer(String scorerSlug, @Nullable String version, Function<INPUT,Object> inputConverter, Function<OUTPUT,Object> outputConverter)
      Fetch a scorer from Braintrust by slug, using the default project from configuration, with custom converters that transform the input, output, and expected scorer argument values into a JSON-serializable form.

      By default, argument values are serialized with the SDK's shared mapper (BraintrustJsonMapper). Use this variant to control serialization, e.g. with a custom ObjectMapper: fetchScorer(slug, null, myMapper::valueToTree, myMapper::valueToTree). Case metadata and eval parameters are always serialized with the OpenAPI client's mapper.

      Parameters:
      scorerSlug - the unique slug identifier for the scorer
      version - optional version of the scorer to fetch
      inputConverter - converts each case's input value into a JSON-serializable form (e.g. a Jackson JsonNode, Map, or scalar); never invoked with null
      outputConverter - converts the task output and case expected values into a JSON-serializable form; never invoked with null
      Returns:
      a Scorer that invokes the remote function

    • fetchScorer

      public <INPUT, OUTPUT> Scorer<INPUT,OUTPUT> fetchScorer(String projectName, String scorerSlug, @Nullable String version)
      Fetch a scorer from Braintrust by project name and slug.
      Parameters:
      projectName - the name of the project containing the scorer
      scorerSlug - the unique slug identifier for the scorer
      version - optional version of the scorer to fetch
      Returns:
      a Scorer that invokes the remote function

    • fetchScorer

      public <INPUT, OUTPUT> Scorer<INPUT,OUTPUT> fetchScorer(String projectName, String scorerSlug, @Nullable String version, Function<INPUT,Object> inputConverter, Function<OUTPUT,Object> outputConverter)
      Fetch a scorer from Braintrust by project name and slug, with custom converters that transform the input, output, and expected scorer argument values into a JSON-serializable form. See fetchScorer(String, String, Function, Function) for converter semantics.
      Parameters:
      projectName - the name of the project containing the scorer
      scorerSlug - the unique slug identifier for the scorer
      version - optional version of the scorer to fetch
      inputConverter - converts each case's input value into a JSON-serializable form; never invoked with null
      outputConverter - converts the task output and case expected values into a JSON-serializable form; never invoked with null
      Returns:
      a Scorer that invokes the remote function

    • config

      public BraintrustConfig config()

    • openApiClient

      public BraintrustOpenApiClient openApiClient()

    • promptLoader

      public BraintrustPromptLoader promptLoader()