Package dev.braintrust.eval

Interface TracedScorer<INPUT,OUTPUT>

Type Parameters:

INPUT - type of the input data

OUTPUT - type of the output data

All Superinterfaces:

Scorer<INPUT,OUTPUT>

public interface TracedScorer<INPUT,OUTPUT>
extends Scorer<INPUT,OUTPUT>

A scorer that receives access to the full distributed trace of the task that was evaluated.

Implement this interface when your scorer needs to examine intermediate LLM calls, tool invocations, or other spans produced during task execution — not just the final TaskResult.

Method Summary

Modifier and Type	Method	Description
default List<Score>	score(TaskResult<INPUT,OUTPUT> taskResult)	Scores the result of a successful task execution.
List<Score>	score(TaskResult<INPUT,OUTPUT> taskResult, BrainstoreTrace trace)	Scores the task result using the distributed trace for additional context.

Methods inherited from interface dev.braintrust.eval.Scorer

getName, scoreForScorerException, scoreForTaskException

Method Details

score

List<Score> score(TaskResult<INPUT,OUTPUT> taskResult,
 BrainstoreTrace trace)

Scores the task result using the distributed trace for additional context. Called instead of Scorer.score(TaskResult) when a BrainstoreTrace is available.

Parameters:

taskResult - the task output and originating dataset case

trace - lazy access to the distributed trace spans for this eval case

Returns:

one or more scores, each with a value between 0 and 1 inclusive

score

default List<Score> score(TaskResult<INPUT,OUTPUT> taskResult)

Scores the result of a successful task execution.

When used inside an Eval, this overload is never called — score(TaskResult, BrainstoreTrace) is dispatched instead. This default implementation throws UnsupportedOperationException to surface any accidental direct calls.

Specified by:

score in interface Scorer<INPUT,OUTPUT>

Parameters:

taskResult - the task output and originating dataset case

Returns:

one or more scores, each with a value between 0 and 1 inclusive

If this method thows, the error will be recorded on the span and Scorer.scoreForScorerException(java.lang.Exception, dev.braintrust.eval.TaskResult<INPUT, OUTPUT>) will be called as a fallback