Package dev.braintrust.eval

Interface Task<INPUT,OUTPUT>

Type Parameters:

INPUT - type of the input data

OUTPUT - type of the output data

public interface Task<INPUT,OUTPUT>

A task function that runs against a single dataset case and produces an output.

If the task throws an exception, the error is recorded on the span and each scorer's Scorer.scoreForTaskException(java.lang.Exception, dev.braintrust.eval.DatasetCase<INPUT, OUTPUT>) method is invoked instead of Scorer.score(dev.braintrust.eval.TaskResult<INPUT, OUTPUT>).

Method Summary

Modifier and Type	Method	Description
TaskResult<INPUT,OUTPUT>	apply(DatasetCase<INPUT,OUTPUT> datasetCase)	Executes this task against a single dataset case and returns the result.

Method Details

apply

TaskResult<INPUT,OUTPUT> apply(DatasetCase<INPUT,OUTPUT> datasetCase)
                        throws Exception

Executes this task against a single dataset case and returns the result.

Parameters:

datasetCase - the dataset case to evaluate

Returns:

the task result containing the output and the originating dataset case

Throws:

Exception - if the task fails, the error will be recorded on the span and scoring will fall back to Scorer.scoreForTaskException(java.lang.Exception, dev.braintrust.eval.DatasetCase<INPUT, OUTPUT>)