Package dev.braintrust.eval

Class ScorerBrainstoreImpl<INPUT,OUTPUT>

java.lang.Object

dev.braintrust.eval.ScorerBrainstoreImpl<INPUT,OUTPUT>

All Implemented Interfaces:

Scorer<INPUT,OUTPUT>

public class ScorerBrainstoreImpl<INPUT,OUTPUT>
extends Object
implements Scorer<INPUT,OUTPUT>

A scorer that invokes a remote Braintrust function to compute scores.

This implementation fetches a scorer function from Braintrust and invokes it via the API for each task result. The remote function receives the input, output, expected, and metadata as arguments.

Supports distributed tracing: if an object ID is available in OTEL baggage and a valid span context exists, the scorer will pass parent span information to the remote function, enabling the remote function's spans to appear as children of the local scorer span.

Constructor Summary

Constructors

Constructor	Description
ScorerBrainstoreImpl(BraintrustApiClient apiClient, String functionId, String version)	Deprecated.
ScorerBrainstoreImpl(BraintrustOpenApiClient apiClient, String functionId, String version)	Create a new remote scorer.
ScorerBrainstoreImpl(BraintrustOpenApiClient apiClient, String functionId, String version, Function<INPUT,Object> inputConverter, Function<OUTPUT,Object> outputConverter)	Create a new remote scorer with custom input/output converters.

Method Summary

Modifier and Type	Method	Description
String	getName()
List<Score>	score(TaskResult<INPUT,OUTPUT> taskResult)	Scores the result of a successful task execution.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface dev.braintrust.eval.Scorer

scoreForScorerException, scoreForTaskException

Constructor Details

ScorerBrainstoreImpl

@Deprecated
public ScorerBrainstoreImpl(BraintrustApiClient apiClient,
 String functionId,
 @Nullable
 String version)

Deprecated.

ScorerBrainstoreImpl

public ScorerBrainstoreImpl(BraintrustOpenApiClient apiClient,
 String functionId,
 @Nullable
 String version)

Create a new remote scorer. Input, output, and expected values are serialized with the SDK's shared mapper (BraintrustJsonMapper); use ScorerBrainstoreImpl(BraintrustOpenApiClient, String, String, java.util.function.Function, java.util.function.Function) to customize serialization.

Parameters:

apiClient - the API client to use for invoking the function

functionId - braintrust function id

version - optional version of the function to invoke. null always invokes latest version.

ScorerBrainstoreImpl

public ScorerBrainstoreImpl(BraintrustOpenApiClient apiClient,
 String functionId,
 @Nullable
 String version,
 Function<INPUT,Object> inputConverter,
 Function<OUTPUT,Object> outputConverter)

Create a new remote scorer with custom input/output converters.

Parameters:

apiClient - the API client to use for invoking the function

functionId - braintrust function id

version - optional version of the function to invoke. null always invokes latest version.

inputConverter - converts each case's input value into a JSON-serializable form (e.g. a Jackson JsonNode, Map, or scalar) before the invoke request is sent; never invoked with null

outputConverter - converts the task output and case expected values into a JSON-serializable form; never invoked with null

Method Details

getName

public String getName()

Specified by:

getName in interface Scorer<INPUT,OUTPUT>

score

public List<Score> score(TaskResult<INPUT,OUTPUT> taskResult)

Description copied from interface: Scorer

Scores the result of a successful task execution.

Specified by:

score in interface Scorer<INPUT,OUTPUT>

Parameters:

taskResult - the task output and originating dataset case

Returns:

one or more scores, each with a value between 0 and 1 inclusive

If this method thows, the error will be recorded on the span and Scorer.scoreForScorerException(java.lang.Exception, dev.braintrust.eval.TaskResult<INPUT, OUTPUT>) will be called as a fallback