Class EvalCreateParams
-
- All Implemented Interfaces:
-
com.openai.core.Params
public final class EvalCreateParams implements Params
Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the Evals guide.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public final class
EvalCreateParams.Builder
A builder for EvalCreateParams.
public final class
EvalCreateParams.Body
public final class
EvalCreateParams.DataSourceConfig
The configuration for the data source used for the evaluation runs. Dictates the schema of the data used in the evaluation.
public final class
EvalCreateParams.TestingCriterion
A LabelModelGrader object which uses a model to assign labels to each item in the evaluation.
public final class
EvalCreateParams.Metadata
Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
-
Method Summary
Modifier and Type Method Description final EvalCreateParams.DataSourceConfig
dataSourceConfig()
The configuration for the data source used for the evaluation runs. final List<EvalCreateParams.TestingCriterion>
testingCriteria()
A list of graders for all eval runs in this group. final Optional<EvalCreateParams.Metadata>
metadata()
Set of 16 key-value pairs that can be attached to an object. final Optional<String>
name()
The name of the evaluation. final JsonField<EvalCreateParams.DataSourceConfig>
_dataSourceConfig()
Returns the raw JSON value of dataSourceConfig. final JsonField<List<EvalCreateParams.TestingCriterion>>
_testingCriteria()
Returns the raw JSON value of testingCriteria. final JsonField<EvalCreateParams.Metadata>
_metadata()
Returns the raw JSON value of metadata. final JsonField<String>
_name()
Returns the raw JSON value of name. final Map<String, JsonValue>
_additionalBodyProperties()
final Headers
_additionalHeaders()
final QueryParams
_additionalQueryParams()
final EvalCreateParams.Builder
toBuilder()
final EvalCreateParams.Body
_body()
Headers
_headers()
The full set of headers in the parameters, including both fixed and additional headers. QueryParams
_queryParams()
The full set of query params in the parameters, including both fixed and additional query params. Boolean
equals(Object other)
Integer
hashCode()
String
toString()
final static EvalCreateParams.Builder
builder()
Returns a mutable builder for constructing an instance of EvalCreateParams. -
-
Method Detail
-
dataSourceConfig
final EvalCreateParams.DataSourceConfig dataSourceConfig()
The configuration for the data source used for the evaluation runs. Dictates the schema of the data used in the evaluation.
-
testingCriteria
final List<EvalCreateParams.TestingCriterion> testingCriteria()
A list of graders for all eval runs in this group. Graders can reference variables in the data source using double curly braces notation, like
{{item.variable_name}}
. To reference the model's output, use thesample
namespace (ie,{{sample.output_text}}
).
-
metadata
final Optional<EvalCreateParams.Metadata> metadata()
Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
-
_dataSourceConfig
final JsonField<EvalCreateParams.DataSourceConfig> _dataSourceConfig()
Returns the raw JSON value of dataSourceConfig.
Unlike dataSourceConfig, this method doesn't throw if the JSON field has an unexpected type.
-
_testingCriteria
final JsonField<List<EvalCreateParams.TestingCriterion>> _testingCriteria()
Returns the raw JSON value of testingCriteria.
Unlike testingCriteria, this method doesn't throw if the JSON field has an unexpected type.
-
_metadata
final JsonField<EvalCreateParams.Metadata> _metadata()
Returns the raw JSON value of metadata.
Unlike metadata, this method doesn't throw if the JSON field has an unexpected type.
-
_name
final JsonField<String> _name()
Returns the raw JSON value of name.
Unlike name, this method doesn't throw if the JSON field has an unexpected type.
-
_additionalBodyProperties
final Map<String, JsonValue> _additionalBodyProperties()
-
_additionalHeaders
final Headers _additionalHeaders()
-
_additionalQueryParams
final QueryParams _additionalQueryParams()
-
toBuilder
final EvalCreateParams.Builder toBuilder()
-
_body
final EvalCreateParams.Body _body()
-
_headers
Headers _headers()
The full set of headers in the parameters, including both fixed and additional headers.
-
_queryParams
QueryParams _queryParams()
The full set of query params in the parameters, including both fixed and additional query params.
-
builder
final static EvalCreateParams.Builder builder()
Returns a mutable builder for constructing an instance of EvalCreateParams.
The following fields are required:
.dataSourceConfig() .testingCriteria()
-
-
-
-