Package dev.braintrust.devserver

Class RemoteEval.Builder<INPUT,OUTPUT>

java.lang.Object

dev.braintrust.devserver.RemoteEval.Builder<INPUT,OUTPUT>

Enclosing class:

RemoteEval<INPUT,OUTPUT>

public static class RemoteEval.Builder<INPUT,OUTPUT>
extends Object

Method Summary

Modifier and Type	Method	Description
RemoteEval<INPUT,OUTPUT>	build()	Build the RemoteEval
RemoteEval.Builder<INPUT,OUTPUT>	clearParameters()
RemoteEval.Builder<INPUT,OUTPUT>	clearScorers()
RemoteEval.Builder<INPUT,OUTPUT>	inputConverter(Function<Object,INPUT> inputConverter)	Converts raw JSON-decoded input values from eval requests into INPUT.
RemoteEval<INPUT,OUTPUT>	internalBuild()
RemoteEval.Builder<INPUT,OUTPUT>	name(String name)	The name of this evaluator (used as identifier)
RemoteEval.Builder<INPUT,OUTPUT>	outputConverter(Function<Object,OUTPUT> outputConverter)	Converts raw JSON-decoded expected values from eval requests into OUTPUT.
RemoteEval.Builder<INPUT,OUTPUT>	parameter(ParameterDef<?> parameter)
RemoteEval.Builder<INPUT,OUTPUT>	parameters(Collection<? extends ParameterDef<?>> parameters)
RemoteEval.Builder<INPUT,OUTPUT>	scorer(Scorer<INPUT,OUTPUT> scorer)
RemoteEval.Builder<INPUT,OUTPUT>	scorers(Collection<? extends Scorer<INPUT,OUTPUT>> scorers)
RemoteEval.Builder<INPUT,OUTPUT>	task(Task<INPUT,OUTPUT> task)	The task function that performs the evaluation
RemoteEval.Builder<INPUT,OUTPUT>	taskFunction(Function<INPUT,OUTPUT> taskFn)	Convenience builder method to create a RemoteEval with a simple task function.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

taskFunction

public RemoteEval.Builder<INPUT,OUTPUT> taskFunction(Function<INPUT,OUTPUT> taskFn)

Convenience builder method to create a RemoteEval with a simple task function.

Parameters:

taskFn - Function that takes input and returns output

Returns:

this builder

build

public RemoteEval<INPUT,OUTPUT> build()

Build the RemoteEval

name

public RemoteEval.Builder<INPUT,OUTPUT> name(@Nonnull
 String name)

The name of this evaluator (used as identifier)

Returns:

this.

task

public RemoteEval.Builder<INPUT,OUTPUT> task(@Nonnull
 Task<INPUT,OUTPUT> task)

The task function that performs the evaluation

The task function must be thread safe.

Returns:

this.

scorer

public RemoteEval.Builder<INPUT,OUTPUT> scorer(Scorer<INPUT,OUTPUT> scorer)

scorers

public RemoteEval.Builder<INPUT,OUTPUT> scorers(Collection<? extends Scorer<INPUT,OUTPUT>> scorers)

clearScorers

public RemoteEval.Builder<INPUT,OUTPUT> clearScorers()

parameter

public RemoteEval.Builder<INPUT,OUTPUT> parameter(ParameterDef<?> parameter)

parameters

public RemoteEval.Builder<INPUT,OUTPUT> parameters(Collection<? extends ParameterDef<?>> parameters)

clearParameters

public RemoteEval.Builder<INPUT,OUTPUT> clearParameters()

inputConverter

public RemoteEval.Builder<INPUT,OUTPUT> inputConverter(@Nonnull
 Function<Object,INPUT> inputConverter)

Converts raw JSON-decoded input values from eval requests into INPUT. Defaults to an unchecked cast of the raw value.

Returns:

this.

outputConverter

public RemoteEval.Builder<INPUT,OUTPUT> outputConverter(@Nonnull
 Function<Object,OUTPUT> outputConverter)

Converts raw JSON-decoded expected values from eval requests into OUTPUT. Defaults to an unchecked cast of the raw value.

Returns:

this.

internalBuild

public RemoteEval<INPUT,OUTPUT> internalBuild()

toString

public String toString()

Overrides:

toString in class Object