Package dev.braintrust.eval

Class ScorerBrainstoreImpl<INPUT,OUTPUT>

java.lang.Object

dev.braintrust.eval.ScorerBrainstoreImpl<INPUT,OUTPUT>

All Implemented Interfaces:

Scorer<INPUT,OUTPUT>

public class ScorerBrainstoreImpl<INPUT,OUTPUT>
extends Object
implements Scorer<INPUT,OUTPUT>

A scorer that invokes a remote Braintrust function to compute scores.

This implementation fetches a scorer function from Braintrust and invokes it via the API for each task result. The remote function receives the input, output, expected, and metadata as arguments.

Supports distributed tracing: if an object ID is available in OTEL baggage and a valid span context exists, the scorer will pass parent span information to the remote function, enabling the remote function's spans to appear as children of the local scorer span.

Constructor Summary

Constructors

Constructor	Description
ScorerBrainstoreImpl(BraintrustApiClient apiClient, String functionId, String version)	Deprecated.
ScorerBrainstoreImpl(BraintrustOpenApiClient apiClient, String functionId, String version)	Create a new remote scorer.

Method Summary

Modifier and Type	Method	Description
String	getName()
List<Score>	score(TaskResult<INPUT,OUTPUT> taskResult)	Scores the result of a successful task execution.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface dev.braintrust.eval.Scorer

scoreForScorerException, scoreForTaskException

Constructor Details

ScorerBrainstoreImpl

@Deprecated
public ScorerBrainstoreImpl(BraintrustApiClient apiClient,
 String functionId,
 @Nullable
 String version)

Deprecated.

ScorerBrainstoreImpl

public ScorerBrainstoreImpl(BraintrustOpenApiClient apiClient,
 String functionId,
 @Nullable
 String version)

Create a new remote scorer.

Parameters:

apiClient - the API client to use for invoking the function

functionId - braintrust function id

version - optional version of the function to invoke. null always invokes latest version.

Method Details

getName

public String getName()

Specified by:

getName in interface Scorer<INPUT,OUTPUT>

score

public List<Score> score(TaskResult<INPUT,OUTPUT> taskResult)

Description copied from interface: Scorer

Scores the result of a successful task execution.

Specified by:

score in interface Scorer<INPUT,OUTPUT>

Parameters:

taskResult - the task output and originating dataset case

Returns:

one or more scores, each with a value between 0 and 1 inclusive

If this method thows, the error will be recorded on the span and Scorer.scoreForScorerException(java.lang.Exception, dev.braintrust.eval.TaskResult<INPUT, OUTPUT>) will be called as a fallback