Class SDParser
- java.lang.Object
-
- com.yahoo.searchdefinition.parser.SDParser
-
- All Implemented Interfaces:
SDParserConstants
public class SDParser extends Object implements SDParserConstants
The schema parser- Author:
- bratseth
-
-
Field Summary
Fields Modifier and Type Field Description Token
jj_nt
Next token.Token
token
Current token.SDParserTokenManager
token_source
Generated Token Manager.-
Fields inherited from interface com.yahoo.searchdefinition.parser.SDParserConstants
ALIAS, ALWAYS, ANNOTATION, ANNOTATIONREFERENCE, ARITY, ARRAY, AS, ASCENDING, ATTRIBUTE, BODY, BOLDING, BRACE_ML_CONTENT, BRACE_ML_LEVEL_1, BRACE_ML_LEVEL_2, BRACE_ML_LEVEL_3, BRACE_SL_CONTENT, BRACE_SL_LEVEL_1, BRACE_SL_LEVEL_2, BRACE_SL_LEVEL_3, BTREE, CASED, COLON, COMMA, COMPRESSION, COMPRESSIONLEVEL, COMPRESSIONTHRESHOLD, CONSTANT, CONSTANTS, CONTEXT, CREATEIFNONEXISTENT, CUTOFF_FACTOR, CUTOFF_STRATEGY, DEFAULT, DENSEPOSTINGLISTTHRESHOLD, DESCENDING, DICTIONARY, DIRECT, DISTANCEMETRIC, DIVERSITY, DOCUMENT, DOCUMENTSUMMARY, DOT, DOUBLE, DYNAMIC, ENABLE_BM25, ENABLEBITVECTORS, ENABLEONLYBITVECTOR, EOF, EVALUATION_POINT, EXACT, EXACTTERMINATOR, EXPRESSION_ML, EXPRESSION_SL, FALSE, FASTACCESS, FASTSEARCH, FIELD, FIELDS, FIELDSET, FILE, FILE_PATH, FILTER, FIRSTPHASE, FROMDISK, FULL, FUNCTION, GRAM, GRAMSIZE, GREATERTHAN, HASH, HEADER, HNSW, HTTP, HUGE, ID, IDENTICAL, IDENTIFIER, IDENTIFIER_WITH_DASH, IGNOREDEFAULTRANKFEATURES, IMPORT, INDEX, INDEXING, INDEXINGREWRITE, INHERITS, INLINE, INTEGER, KEEPRANKCOUNT, LBRACE, LESSTHAN, LITERAL, LOCALE, LONG, LOOSE, LOWERBOUND, LOWERCASE, LZ4, MACRO, MAP, MATCH, MATCHEDELEMENTSONLY, MATCHFEATURES_ML, MATCHFEATURES_ML_INHERITS, MATCHFEATURES_SL, MATCHPHASE, MAXFILTERCOVERAGE, MAXHITS, MAXLENGTH, MAXLINKSPERNODE, MIN_GROUPS, MINHITSPERTHREAD, MODEL, MTOKEN, MULTITHREADEDINDEXING, MUTABLE, MUTATE, NEIGHBORSTOEXPLOREATINSERT, NEVER, NL, NONE, NORMAL, NORMALIZING, NUMSEARCHPARTITIONS, NUMTHREADSPERSEARCH, OFF, OMITSUMMARYFEATURES, ON, ON_FIRST_PHASE, ON_MATCH, ON_SECOND_PHASE, ON_SUMMARY, ONDEMAND, ONNX_INPUT_SL, ONNX_OUTPUT_SL, ONNXMODEL, OPERATION, ORDER, PAGED, PRE_POST_FILTER_TIPPING_POINT, PREFIX, PRIMARY, PROPERTIES, QUATERNARY, QUERYCOMMAND, QUESTIONMARK, QUOTEDSTRING, RANK, RANKDEGRADATION, RANKDEGRADATIONFREQ, RANKFEATURES_ML, RANKFEATURES_SL, RANKPROFILE, RANKPROPERTIES, RANKSCOREDROPLIMIT, RANKTYPE, RAW, RAW_AS_BASE64_IN_SUMMARY, RBRACE, REFERENCE, REMOVEIFZERO, RERANKCOUNT, RPBINLOW, RPBINSIZE, RPPOSBINSIZE, SCHEMA, SEARCH, SEARCHLIB_SKIP, SECONDARY, SECONDPHASE, SINGLE_LINE_COMMENT, SORTING, SOURCE, SSCONTEXTUAL, SSOVERRIDE, SSTITLE, SSURL, STATIC, STEMMING, STRENGTH, STRICT, STRING, STRUCT, STRUCTFIELD, SUBSTRING, SUFFIX, SUMMARY, SUMMARYFEATURES_ML, SUMMARYFEATURES_ML_INHERITS, SUMMARYFEATURES_SL, SUMMARYTO, SYMMETRIC, TENSOR_TYPE, TENSOR_VALUE_ML, TENSOR_VALUE_SL, TERMWISELIMIT, TERTIARY, TEXT, TO, tokenImage, TRUE, TYPE, UCA, UNCASED, UPPERBOUND, URI, URI_PATH, USEDOCUMENT, VARIABLE, WEIGHT, WEIGHTEDSET, WORD
-
-
Constructor Summary
Constructors Constructor Description SDParser(CharStream stream)
Constructor with user supplied CharStream.SDParser(SDParserTokenManager tm)
Constructor with generated Token Manager.SDParser(SimpleCharStream stream, com.yahoo.config.application.api.FileRegistry fileRegistry, com.yahoo.config.application.api.DeployLogger deployLogger, com.yahoo.config.model.api.ModelContext.Properties properties, Application application, RankProfileRegistry rankProfileRegistry, boolean documentsOnly)
Creates a parser
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
alias(FieldOperationContainer container)
void
annotation(Schema schema, SDDocumentType document)
Consumes a annotation block from within either a document element.SDAnnotationType
annotationBody(Schema schema, SDAnnotationType type)
Consumes a single element of an annotation body block.void
annotationOutside(Schema schema)
This rule consumes a annotation block from within either a document element or a schema element.com.yahoo.document.DataType
annotationRefDataType()
Object
attribute(FieldOperationContainer field)
This rule consumes an attribute statement of a field element.Object
attributeSetting(FieldOperationContainer field, AttributeOperation attribute, String attributeName)
This rule consumes a single attribute setting statement of an attribute element.Object
attributeTensorType(AttributeOperation attribute)
This rule consumes a tensor type statement for an attribute element.void
body(SDField field)
This rule consumes a body statement of a field element.void
bodycfg(SDDocumentType document)
Consumes a document body block.void
bolding(FieldOperationContainer field)
This rule consumes a bolding statement of a field element.Boolean
bool()
This rule consumes a boolean value.void
compression(SDDocumentType document)
Consumes a compression block.void
compressionItem()
Consumes the body of a compression block.void
constants(RankProfile profile)
Consumes a set of constants available in ranking expressions in the enclosing profile.void
constantTensor(RankProfile profile, String name)
String
constantTensorErrorMessage(String rankProfileName, String constantTensorName)
void
constantValue(RankProfile profile, String name)
double
consumeFloat()
This rule consumes a floating-point token and returns its numeric value.long
consumeLong()
This rule consumes a long or integer token and returns its numeric value.Number
consumeNumber()
com.yahoo.document.DataType
dataType()
This rule consumes a data type block from within a field element.void
dictionary(FieldOperationContainer field)
This rule consumes a dictionary statement of a field element.void
dictionarySetting(FieldOperationContainer field)
void
disable_tracing()
Disable tracing.void
diversity(RankProfile.MatchPhaseSettings profile)
This rule consumes a diversity block of a rank profile.void
diversityItem(RankProfile.DiversitySettings settings)
void
document(Schema schema)
Consumes a document element.Object
documentBody(SDDocumentType document, Schema schema)
Consumes a document body blockObject
documentSummary(Schema schema)
Consumes a document-summary block from within a schema block.Object
documentSummaryItem(DocumentSummary summary)
Consumes a single document-summary item.void
enable_tracing()
Enable tracing.void
exactTerminator(FieldOperationContainer container)
String
expression()
This rule consumes an expression token and returns its image.void
field(SDDocumentType document, Schema schema)
Consumes a field block from within a document element.String
fieldBody(SDField field, Schema schema, SDDocumentType document)
This rule consumes a single element of a field body block.void
fieldRankFilter(RankProfile profile)
This rule consumes a rank filter statement of a rank profile.void
fieldRankType(RankProfile profile)
This rule consumes a rank-type statement of a rank profile.void
fieldSet(Schema schema)
void
fieldStemming(FieldOperationContainer field)
This rule consumes a stemming block of a field element.void
fieldWeight(RankProfile profile)
This rule consumes a field-weight statement of a rank profile.String
fileItem()
void
firstPhase(RankProfile profile)
Consumes the first-phase block of a rank profile.Object
firstPhaseItem(RankProfile profile)
void
function(RankProfile profile)
This rule consumes a function statement of a rank-profile.ParseException
generateParseException()
Generate ParseException.Token
getNextToken()
Token
getToken(int index)
void
gramSize(FieldOperationContainer container)
void
header(SDField field)
This rule consumes a header statement of a field element.void
headercfg(SDDocumentType document)
Consumes a document head block.void
hnswIndex(IndexOperation index)
void
hnswIndexBody(HnswIndexParams.Builder params)
void
id(FieldOperationContainer field, SDDocumentType document)
Consumes an id statement of a field body block.String
identifier()
Consumes an identifier.String
identifierWithDash()
void
ignoreRankFeatures(RankProfile profile)
This rule consumes a ignore-default-rank-features statement for a rank profile.void
importField(Schema schema)
Object
index(Schema schema, FieldOperationContainer field)
Consumes an index block for a field element.Object
indexBody(IndexOperation index)
Consumes a single index statement for an index block.Object
indexing(FieldOperationContainer field)
This rule consumes an indexing block of a field element.void
indexingOperation(FieldOperationContainer field, boolean multiLine)
This rule consumes an IL script block.void
indexingRewrite(FieldOperationContainer field)
Consumes an indexing-rewrite statement of a field body block.void
inheritsAnnotation(SDAnnotationType annotation)
void
inheritsDocument(SDDocumentType document)
Consumes a document inheritance statement.void
inheritsDocumentSummary(DocumentSummary documentSummary, Schema schema)
This rule consumes an inherits statement of a document summary.void
inheritsRankProfile(RankProfile profile)
This rule consumes an inherits statement of a rank-profile.boolean
inline()
int
integer()
This rule consumes an integer token and returns its numeric value.void
lbrace()
This rule consumes an opening brace with leading and trailing newline tokens.com.yahoo.document.DataType
mapDataType()
FieldOperationContainer
match(FieldOperationContainer field)
Object
matchFeatures(RankProfile profile)
This rule consumes a match-features block of a rank profile.Object
matchItem(FieldOperationContainer field)
This rule consumes a single match item for a match block.void
matchPhase(RankProfile profile)
This rule consumes a match-phase block of a rank profile.void
matchPhaseItem(RankProfile.MatchPhaseSettings settings)
void
matchSize(FieldOperationContainer container)
Object
matchType(FieldOperationContainer container)
void
minHitsPerThread(RankProfile profile)
This rule consumes a min-hits-per-thread statement for a rank profile.void
mutate(RankProfile profile)
This rule consumes an mutate statement of a rank-profile.String
mutate_expr()
void
mutate_operation(RankProfile profile)
void
namedDocument(Schema schema)
Consumes a document element, explicitly namedvoid
normalizing(FieldOperationContainer field)
This rule consumes a normalizing statement of a field element.void
numSearchPartitions(RankProfile profile)
This rule consumes a num-search-partitions statement for a rank profile.void
numThreadsPerSearch(RankProfile profile)
This rule consumes a num-threads-per-search statement for a rank profile.void
onnxModel(Schema schema)
Consumes a onnx-model block of a schema element.Object
onnxModelItem(OnnxModel onnxModel)
This rule consumes an onnx-model block.void
queryCommand(FieldOperationContainer container)
String
quotedString()
Consumes a quoted string token and returns the token image minus the quotes.Object
rank(FieldOperationContainer field)
Consumes a rank statement of a field element.Object
rankDegradation(RankProfile profile)
This rule consumes a rank-degradation statement of a rank profile.void
rankDegradationBinLow()
This rule consumes part of a rank-degradation statement of a rank profile.void
rankDegradationBinSize()
This rule consumes part of a rank-degradation statement of a rank profile.Object
rankDegradationItem()
This rule consumes part of a rank-degradation statement of a rank profile.void
rankDegradationPosbinSize()
This rule consumes part of a rank-degradation statement of a rank profile.Object
rankFeatures(RankProfile profile)
Consumes a rank-features block of a rank profilevoid
rankingConstant(Schema schema)
Consumes a constant block of a schema element.String
rankingConstantErrorMessage(String name)
void
rankProfile(Schema schema)
Consumes a rank-profile block of a schema element.Object
rankProfileItem(RankProfile profile)
This rule consumes a single statement for a rank-profile block.void
rankProperties(RankProfile profile)
This rule consumes a rank-properties block of a rank profile.void
rankProperty(RankProfile profile)
This rule consumes a single rank property pair for a rank profile.String
rankPropertyItem()
This rule consumes a single rank property for a rank-properties block.Object
rankSetting(RankOperation field)
Consumes a single rank setting of a rank statement.void
rankType(FieldOperationContainer field)
This rule consumes a rank-type statement of a field element.void
rawAsBase64(Schema schema)
com.yahoo.document.TemporaryStructuredDataType
referenceType()
void
ReInit(CharStream stream)
Reinitialise.void
ReInit(SDParserTokenManager tm)
ReinitialiseSchema
rootDocument(String dir)
Consumes a schema definition that contains only documents to be used for inheritance, etc.Object
rootDocumentItem(Schema schema)
Consumes a single item from within a root document node.Schema
rootSchema(String dir)
This rule consumes a proper schema block.Object
rootSchemaItem(Schema schema)
Consumes an element of a schema block.Schema
schema(com.yahoo.document.DocumentTypeManager docMan, String dir)
The rule consumes any schema and returns the corresponding object.void
searchStemming(Schema schema)
This rule consumes a stemming statement for a schema element.void
secondPhase(RankProfile profile)
Consumes the second-phase block of a rank profile.Object
secondPhaseItem(RankProfile profile)
Consumes a statement for a second-phase block.Object
sorting(FieldOperationContainer field, String name)
Object
sortingSetting(SortingOperation sorting, String attributeName)
String
string()
Consumes a string token and returns the token image.SDDocumentType
structDefinition(Schema schema, SDDocumentType repo)
This rule consumes a struct block from within a document element.void
structField(FieldOperationContainer field, Schema schema, SDDocumentType document)
This rule consumes a struct subfield from a document field body.String
structFieldBody(FieldOperationContainer field, Schema schema, SDDocumentType document)
This rule consumes a single element of a struct subfield body block.void
structFieldDefinition(SDDocumentType struct)
This rule consumes a field block of a struct body.void
structInside(SDDocumentType document, Schema schema)
This rule consumes a struct block from within a document element.void
structOutside(Schema schema)
This rule consumes a struct block from within a document element.void
summaryBolding(SummaryInFieldLongOperation field)
This rule consumes a bolding statement for a summary field element.void
summaryDestinationList(SummaryInFieldLongOperation field)
This rule consumes a destination-list statement for a summary field element.Object
summaryFeatures(RankProfile profile)
This rule consumes a summary-features block of a rank profile.Object
summaryInDocument(DocumentSummary document)
This rule consumes a summary statement defined inside a document-summary block.Object
summaryInField(FieldOperationContainer field)
The rule consumes a summary statement defined inside a field.SummaryInFieldOperation
summaryInFieldLong(FieldOperationContainer field)
This rule consumes a multi-line summary field.SummaryInFieldOperation
summaryInFieldShort(FieldOperationContainer field)
This rule consumes a single-line summary field.Object
summaryItem(SummaryInFieldLongOperation field)
This rule consumes an item of a summary field block.void
summaryProperties(SummaryInFieldLongOperation field)
This rule consumes properties for a summary field element.void
summaryProperty(SummaryInFieldLongOperation field)
This rule consumes a single summary property pair for a summary field element.void
summarySourceList(SummaryInFieldOperation field)
This rule consumes a source-list statement for a summary field element.void
summaryTo(FieldOperationContainer field)
This rule consumes a summary-to statement of a field element.Object
summaryTransform(SummaryInFieldOperation field)
This rule consumes a transform statement for a summary field element.com.yahoo.tensor.TensorType
tensorType(String errorMessage)
com.yahoo.tensor.TensorType
tensorTypeWithPrefix(String errorMessage)
String
tensorValue()
void
termwiseLimit(RankProfile profile)
This rule consumes a num-threads-per-search statement for a rank profile.boolean
trace_enabled()
String
uriItem()
void
useDocument(Schema schema)
Consumes a use-document statement.void
weight(FieldOperationContainer field)
This rule consumes a weight statement of a field element.Object
weightedset(FieldOperationContainer field)
This rule consumes a weighted set statement of a field element.Object
weightedsetBody(WeightedSetOperation field)
This rule consumes one body item of a weighted set block.com.yahoo.document.DataType
wildCardType()
-
-
-
Field Detail
-
token_source
public SDParserTokenManager token_source
Generated Token Manager.
-
token
public Token token
Current token.
-
jj_nt
public Token jj_nt
Next token.
-
-
Constructor Detail
-
SDParser
public SDParser(SimpleCharStream stream, com.yahoo.config.application.api.FileRegistry fileRegistry, com.yahoo.config.application.api.DeployLogger deployLogger, com.yahoo.config.model.api.ModelContext.Properties properties, Application application, RankProfileRegistry rankProfileRegistry, boolean documentsOnly)
Creates a parser- Parameters:
documentsOnly
- true to only parse the document aspect of a schema (e.g skip rank profiles)
-
SDParser
public SDParser(CharStream stream)
Constructor with user supplied CharStream.- Parameters:
stream
- stream to init with
-
SDParser
public SDParser(SDParserTokenManager tm)
Constructor with generated Token Manager.- Parameters:
tm
- Token manager to use
-
-
Method Detail
-
schema
public final Schema schema(com.yahoo.document.DocumentTypeManager docMan, String dir) throws ParseException
The rule consumes any schema and returns the corresponding object. This is the only production that should ever consume leading newlines.- Parameters:
dir
- the directory containing the file being parsed- Returns:
- the schema object
- Throws:
ParseException
-
rootSchema
public final Schema rootSchema(String dir) throws ParseException
This rule consumes a proper schema block. This and rootDocument() are the only rules that should ever consume trailing newline tokens.- Parameters:
dir
- the directory containing the file being parsed.- Returns:
- the schema definition object.
- Throws:
ParseException
-
rootSchemaItem
public final Object rootSchemaItem(Schema schema) throws ParseException
Consumes an element of a schema block. This and rootSearch() are the only rules that should ever consume trailing newline tokens.- Parameters:
schema
- the schema object to modify.- Returns:
- Null.
- Throws:
ParseException
-
rootDocument
public final Schema rootDocument(String dir) throws ParseException
Consumes a schema definition that contains only documents to be used for inheritance, etc.- Parameters:
dir
- the directory containing the file being parsed.- Returns:
- the schema definition object.
- Throws:
ParseException
-
rootDocumentItem
public final Object rootDocumentItem(Schema schema) throws ParseException
Consumes a single item from within a root document node.- Parameters:
schema
- the schema object to modify.- Returns:
- Null.
- Throws:
ParseException
-
useDocument
public final void useDocument(Schema schema) throws ParseException
Consumes a use-document statement. This currently does nothing.- Parameters:
schema
- the schema object to modify.- Throws:
ParseException
-
document
public final void document(Schema schema) throws ParseException
Consumes a document element. The name defaults to the schema's name, but may be set.- Parameters:
schema
- the schema object to add content to.- Throws:
ParseException
-
namedDocument
public final void namedDocument(Schema schema) throws ParseException
Consumes a document element, explicitly named- Parameters:
schema
- the schema object to add content to- Throws:
ParseException
-
documentBody
public final Object documentBody(SDDocumentType document, Schema schema) throws ParseException
Consumes a document body block- Parameters:
document
- the document type to modify.schema
- the schema object to add content to- Returns:
- Null.
- Throws:
ParseException
-
rawAsBase64
public final void rawAsBase64(Schema schema) throws ParseException
- Throws:
ParseException
-
headercfg
public final void headercfg(SDDocumentType document) throws ParseException
Consumes a document head block.- Parameters:
document
- The document type to modify.- Throws:
ParseException
-
bodycfg
public final void bodycfg(SDDocumentType document) throws ParseException
Consumes a document body block.- Parameters:
document
- The document type to modify.- Throws:
ParseException
-
compression
public final void compression(SDDocumentType document) throws ParseException
Consumes a compression block. This can be set in both document header and -body block.- Parameters:
document
- The document type to modify.- Throws:
ParseException
-
compressionItem
public final void compressionItem() throws ParseException
Consumes the body of a compression block.- Throws:
ParseException
-
inheritsDocument
public final void inheritsDocument(SDDocumentType document) throws ParseException
Consumes a document inheritance statement.- Parameters:
document
- The document type to modify.- Throws:
ParseException
-
field
public final void field(SDDocumentType document, Schema schema) throws ParseException
Consumes a field block from within a document element.- Parameters:
document
- the document type to modifyschema
- the schema object to add content to- Throws:
ParseException
-
fieldSet
public final void fieldSet(Schema schema) throws ParseException
- Throws:
ParseException
-
annotationOutside
public final void annotationOutside(Schema schema) throws ParseException
This rule consumes a annotation block from within either a document element or a schema element.- Parameters:
schema
- the schema object to add content to- Throws:
ParseException
-
annotation
public final void annotation(Schema schema, SDDocumentType document) throws ParseException
Consumes a annotation block from within either a document element.- Parameters:
document
- the document object to add content to- Throws:
ParseException
-
annotationBody
public final SDAnnotationType annotationBody(Schema schema, SDAnnotationType type) throws ParseException
Consumes a single element of an annotation body block.- Parameters:
schema
- the schema object to add content totype
- the type being built- Returns:
- a modified or new AnnotationType instance
- Throws:
ParseException
-
inheritsAnnotation
public final void inheritsAnnotation(SDAnnotationType annotation) throws ParseException
- Throws:
ParseException
-
structInside
public final void structInside(SDDocumentType document, Schema schema) throws ParseException
This rule consumes a struct block from within a document element.- Parameters:
schema
- the schema object to add content to- Throws:
ParseException
-
structOutside
public final void structOutside(Schema schema) throws ParseException
This rule consumes a struct block from within a document element.- Parameters:
schema
- the schema object to add content to- Throws:
ParseException
-
structDefinition
public final SDDocumentType structDefinition(Schema schema, SDDocumentType repo) throws ParseException
This rule consumes a struct block from within a document element.- Parameters:
schema
- the schema object to add content to- Throws:
ParseException
-
dataType
public final com.yahoo.document.DataType dataType() throws ParseException
This rule consumes a data type block from within a field element.- Returns:
- the consumed data type
- Throws:
ParseException
-
referenceType
public final com.yahoo.document.TemporaryStructuredDataType referenceType() throws ParseException
- Throws:
ParseException
-
annotationRefDataType
public final com.yahoo.document.DataType annotationRefDataType() throws ParseException
- Throws:
ParseException
-
mapDataType
public final com.yahoo.document.DataType mapDataType() throws ParseException
- Throws:
ParseException
-
wildCardType
public final com.yahoo.document.DataType wildCardType() throws ParseException
- Throws:
ParseException
-
structFieldDefinition
public final void structFieldDefinition(SDDocumentType struct) throws ParseException
This rule consumes a field block of a struct body.- Parameters:
struct
- The struct to modify.- Throws:
ParseException
-
structField
public final void structField(FieldOperationContainer field, Schema schema, SDDocumentType document) throws ParseException
This rule consumes a struct subfield from a document field body. This is not to be confused with a document struct's fields, but rather this is a subfield of a document field of type struct.- Parameters:
field
- the field to modifyschema
- the schema object to add content todocument
- the document type to modify- Throws:
ParseException
-
fieldBody
public final String fieldBody(SDField field, Schema schema, SDDocumentType document) throws ParseException
This rule consumes a single element of a field body block.- Parameters:
field
- the field being builtschema
- the schema object to add content todocument
- the owning document, or null if this is a search field- Returns:
- Null
- Throws:
ParseException
-
structFieldBody
public final String structFieldBody(FieldOperationContainer field, Schema schema, SDDocumentType document) throws ParseException
This rule consumes a single element of a struct subfield body block. Only elements that are supported in streaming schema and indexed schema (with complex attributes) are allowed.- Parameters:
field
- the field being builtschema
- the schema object to add content todocument
- the owning document, or null if this is a schema field- Returns:
- Null
- Throws:
ParseException
-
indexing
public final Object indexing(FieldOperationContainer field) throws ParseException
This rule consumes an indexing block of a field element.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
indexingOperation
public final void indexingOperation(FieldOperationContainer field, boolean multiLine) throws ParseException
This rule consumes an IL script block. This is expected to consume trailing newlines.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
summaryTo
public final void summaryTo(FieldOperationContainer field) throws ParseException
This rule consumes a summary-to statement of a field element.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
weight
public final void weight(FieldOperationContainer field) throws ParseException
This rule consumes a weight statement of a field element.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
weightedset
public final Object weightedset(FieldOperationContainer field) throws ParseException
This rule consumes a weighted set statement of a field element.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
weightedsetBody
public final Object weightedsetBody(WeightedSetOperation field) throws ParseException
This rule consumes one body item of a weighted set block.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
rankType
public final void rankType(FieldOperationContainer field) throws ParseException
This rule consumes a rank-type statement of a field element.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
attribute
public final Object attribute(FieldOperationContainer field) throws ParseException
This rule consumes an attribute statement of a field element.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
sorting
public final Object sorting(FieldOperationContainer field, String name) throws ParseException
- Throws:
ParseException
-
sortingSetting
public final Object sortingSetting(SortingOperation sorting, String attributeName) throws ParseException
- Throws:
ParseException
-
attributeSetting
public final Object attributeSetting(FieldOperationContainer field, AttributeOperation attribute, String attributeName) throws ParseException
This rule consumes a single attribute setting statement of an attribute element.- Parameters:
field
- The field to modify.attributeName
- The name of the attribute to change.- Returns:
- Null.
- Throws:
ParseException
-
attributeTensorType
public final Object attributeTensorType(AttributeOperation attribute) throws ParseException
This rule consumes a tensor type statement for an attribute element.- Parameters:
attribute
- The attribute to modify.- Returns:
- Null.
- Throws:
ParseException
-
summaryInDocument
public final Object summaryInDocument(DocumentSummary document) throws ParseException
This rule consumes a summary statement defined inside a document-summary block.- Parameters:
document
- The document summary to modify.- Returns:
- Null.
- Throws:
ParseException
-
summaryInField
public final Object summaryInField(FieldOperationContainer field) throws ParseException
The rule consumes a summary statement defined inside a field.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
summaryInFieldShort
public final SummaryInFieldOperation summaryInFieldShort(FieldOperationContainer field) throws ParseException
This rule consumes a single-line summary field.- Parameters:
field
- The field to modify.- Returns:
- The consumed summary field.
- Throws:
ParseException
-
summaryInFieldLong
public final SummaryInFieldOperation summaryInFieldLong(FieldOperationContainer field) throws ParseException
This rule consumes a multi-line summary field.- Returns:
- The consumed summary field.
- Throws:
ParseException
-
summaryItem
public final Object summaryItem(SummaryInFieldLongOperation field) throws ParseException
This rule consumes an item of a summary field block.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
summaryTransform
public final Object summaryTransform(SummaryInFieldOperation field) throws ParseException
This rule consumes a transform statement for a summary field element.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
summaryBolding
public final void summaryBolding(SummaryInFieldLongOperation field) throws ParseException
This rule consumes a bolding statement for a summary field element.- Parameters:
field
- The summary field to modify.- Throws:
ParseException
-
summarySourceList
public final void summarySourceList(SummaryInFieldOperation field) throws ParseException
This rule consumes a source-list statement for a summary field element.- Parameters:
field
- The summary field to modify.- Throws:
ParseException
-
summaryDestinationList
public final void summaryDestinationList(SummaryInFieldLongOperation field) throws ParseException
This rule consumes a destination-list statement for a summary field element.- Parameters:
field
- The summary field to modify.- Throws:
ParseException
-
summaryProperties
public final void summaryProperties(SummaryInFieldLongOperation field) throws ParseException
This rule consumes properties for a summary field element.- Parameters:
field
- The summary field to modify.- Throws:
ParseException
-
summaryProperty
public final void summaryProperty(SummaryInFieldLongOperation field) throws ParseException
This rule consumes a single summary property pair for a summary field element.- Parameters:
field
- The summary field to modify.- Throws:
ParseException
-
fieldStemming
public final void fieldStemming(FieldOperationContainer field) throws ParseException
This rule consumes a stemming block of a field element.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
searchStemming
public final void searchStemming(Schema schema) throws ParseException
This rule consumes a stemming statement for a schema element.- Parameters:
schema
- the schema to modify- Throws:
ParseException
-
normalizing
public final void normalizing(FieldOperationContainer field) throws ParseException
This rule consumes a normalizing statement of a field element. At the moment, this can only be used to turn off normalizing.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
bolding
public final void bolding(FieldOperationContainer field) throws ParseException
This rule consumes a bolding statement of a field element.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
dictionary
public final void dictionary(FieldOperationContainer field) throws ParseException
This rule consumes a dictionary statement of a field element.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
dictionarySetting
public final void dictionarySetting(FieldOperationContainer field) throws ParseException
- Throws:
ParseException
-
body
public final void body(SDField field) throws ParseException
This rule consumes a body statement of a field element.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
header
public final void header(SDField field) throws ParseException
This rule consumes a header statement of a field element.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
queryCommand
public final void queryCommand(FieldOperationContainer container) throws ParseException
- Throws:
ParseException
-
alias
public final void alias(FieldOperationContainer container) throws ParseException
- Throws:
ParseException
-
match
public final FieldOperationContainer match(FieldOperationContainer field) throws ParseException
- Throws:
ParseException
-
matchItem
public final Object matchItem(FieldOperationContainer field) throws ParseException
This rule consumes a single match item for a match block.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
matchType
public final Object matchType(FieldOperationContainer container) throws ParseException
- Throws:
ParseException
-
exactTerminator
public final void exactTerminator(FieldOperationContainer container) throws ParseException
- Throws:
ParseException
-
gramSize
public final void gramSize(FieldOperationContainer container) throws ParseException
- Throws:
ParseException
-
matchSize
public final void matchSize(FieldOperationContainer container) throws ParseException
- Throws:
ParseException
-
rank
public final Object rank(FieldOperationContainer field) throws ParseException
Consumes a rank statement of a field element.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
rankSetting
public final Object rankSetting(RankOperation field) throws ParseException
Consumes a single rank setting of a rank statement.- Parameters:
field
- The field to modify.- Returns:
- Null.
- Throws:
ParseException
-
id
public final void id(FieldOperationContainer field, SDDocumentType document) throws ParseException
Consumes an id statement of a field body block.- Parameters:
field
- The field to modify.document
- The document type to modify.- Throws:
ParseException
-
indexingRewrite
public final void indexingRewrite(FieldOperationContainer field) throws ParseException
Consumes an indexing-rewrite statement of a field body block.- Parameters:
field
- The field to modify.- Throws:
ParseException
-
documentSummary
public final Object documentSummary(Schema schema) throws ParseException
Consumes a document-summary block from within a schema block.- Parameters:
schema
- the schema object to add content to- Returns:
- Null
- Throws:
ParseException
-
inheritsDocumentSummary
public final void inheritsDocumentSummary(DocumentSummary documentSummary, Schema schema) throws ParseException
This rule consumes an inherits statement of a document summary.- Parameters:
documentSummary
- the document summary to modifyschema
- the schema object documentSummary is being added to- Throws:
ParseException
-
documentSummaryItem
public final Object documentSummaryItem(DocumentSummary summary) throws ParseException
Consumes a single document-summary item.- Parameters:
summary
- The document summary to modify.- Returns:
- Null.
- Throws:
ParseException
-
index
public final Object index(Schema schema, FieldOperationContainer field) throws ParseException
Consumes an index block for a field element.- Parameters:
schema
- the schema object to add content tofield
- the field to modify- Returns:
- Null
- Throws:
ParseException
-
indexBody
public final Object indexBody(IndexOperation index) throws ParseException
Consumes a single index statement for an index block.- Parameters:
index
- The index to modify.- Returns:
- Null.
- Throws:
ParseException
-
hnswIndex
public final void hnswIndex(IndexOperation index) throws ParseException
- Throws:
ParseException
-
hnswIndexBody
public final void hnswIndexBody(HnswIndexParams.Builder params) throws ParseException
- Throws:
ParseException
-
onnxModel
public final void onnxModel(Schema schema) throws ParseException
Consumes a onnx-model block of a schema element.- Parameters:
schema
- the schema object to add content to.- Throws:
ParseException
-
onnxModelItem
public final Object onnxModelItem(OnnxModel onnxModel) throws ParseException
This rule consumes an onnx-model block.- Parameters:
onnxModel
- The onnxModel to modify.- Returns:
- Null.
- Throws:
ParseException
-
rankingConstant
public final void rankingConstant(Schema schema) throws ParseException
Consumes a constant block of a schema element.- Parameters:
schema
- the schema object to add content to- Throws:
ParseException
-
fileItem
public final String fileItem() throws ParseException
- Throws:
ParseException
-
uriItem
public final String uriItem() throws ParseException
- Throws:
ParseException
-
rankingConstantErrorMessage
public final String rankingConstantErrorMessage(String name) throws ParseException
- Throws:
ParseException
-
rankProfile
public final void rankProfile(Schema schema) throws ParseException
Consumes a rank-profile block of a schema element.- Parameters:
schema
- the schema object to add content to- Throws:
ParseException
-
rankProfileItem
public final Object rankProfileItem(RankProfile profile) throws ParseException
This rule consumes a single statement for a rank-profile block.- Parameters:
profile
- The rank profile to modify.- Returns:
- Null.
- Throws:
ParseException
-
inheritsRankProfile
public final void inheritsRankProfile(RankProfile profile) throws ParseException
This rule consumes an inherits statement of a rank-profile.- Parameters:
profile
- The profile to modify.- Throws:
ParseException
-
mutate
public final void mutate(RankProfile profile) throws ParseException
This rule consumes an mutate statement of a rank-profile.- Parameters:
profile
- The profile to modify.- Throws:
ParseException
-
mutate_operation
public final void mutate_operation(RankProfile profile) throws ParseException
- Throws:
ParseException
-
mutate_expr
public final String mutate_expr() throws ParseException
- Throws:
ParseException
-
function
public final void function(RankProfile profile) throws ParseException
This rule consumes a function statement of a rank-profile.- Parameters:
profile
- The profile to modify.- Throws:
ParseException
-
inline
public final boolean inline() throws ParseException
- Throws:
ParseException
-
matchPhase
public final void matchPhase(RankProfile profile) throws ParseException
This rule consumes a match-phase block of a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
matchPhaseItem
public final void matchPhaseItem(RankProfile.MatchPhaseSettings settings) throws ParseException
- Throws:
ParseException
-
diversity
public final void diversity(RankProfile.MatchPhaseSettings profile) throws ParseException
This rule consumes a diversity block of a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
diversityItem
public final void diversityItem(RankProfile.DiversitySettings settings) throws ParseException
- Throws:
ParseException
-
firstPhase
public final void firstPhase(RankProfile profile) throws ParseException
Consumes the first-phase block of a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
firstPhaseItem
public final Object firstPhaseItem(RankProfile profile) throws ParseException
- Throws:
ParseException
-
secondPhase
public final void secondPhase(RankProfile profile) throws ParseException
Consumes the second-phase block of a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
secondPhaseItem
public final Object secondPhaseItem(RankProfile profile) throws ParseException
Consumes a statement for a second-phase block.- Parameters:
profile
- The rank profile to modify.- Returns:
- Null.
- Throws:
ParseException
-
summaryFeatures
public final Object summaryFeatures(RankProfile profile) throws ParseException
This rule consumes a summary-features block of a rank profile.- Parameters:
profile
- The rank profile to modify.- Returns:
- Null.
- Throws:
ParseException
-
matchFeatures
public final Object matchFeatures(RankProfile profile) throws ParseException
This rule consumes a match-features block of a rank profile.- Parameters:
profile
- The rank profile to modify.- Returns:
- Null.
- Throws:
ParseException
-
rankFeatures
public final Object rankFeatures(RankProfile profile) throws ParseException
Consumes a rank-features block of a rank profile- Throws:
ParseException
-
ignoreRankFeatures
public final void ignoreRankFeatures(RankProfile profile) throws ParseException
This rule consumes a ignore-default-rank-features statement for a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
numThreadsPerSearch
public final void numThreadsPerSearch(RankProfile profile) throws ParseException
This rule consumes a num-threads-per-search statement for a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
minHitsPerThread
public final void minHitsPerThread(RankProfile profile) throws ParseException
This rule consumes a min-hits-per-thread statement for a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
numSearchPartitions
public final void numSearchPartitions(RankProfile profile) throws ParseException
This rule consumes a num-search-partitions statement for a rank profile.- Parameters:
profile
- the rank profile to modify- Throws:
ParseException
-
termwiseLimit
public final void termwiseLimit(RankProfile profile) throws ParseException
This rule consumes a num-threads-per-search statement for a rank profile.- Parameters:
profile
- the rank profile to modify- Throws:
ParseException
-
rankProperties
public final void rankProperties(RankProfile profile) throws ParseException
This rule consumes a rank-properties block of a rank profile. There is a little trick within this rule to allow the final rank property to skip the terminating newline token.- Parameters:
profile
- the rank profile to modify- Throws:
ParseException
-
rankProperty
public final void rankProperty(RankProfile profile) throws ParseException
This rule consumes a single rank property pair for a rank profile.- Parameters:
profile
- the rank profile to modify- Throws:
ParseException
-
rankPropertyItem
public final String rankPropertyItem() throws ParseException
This rule consumes a single rank property for a rank-properties block.- Returns:
- The token image of the consumed item.
- Throws:
ParseException
-
fieldWeight
public final void fieldWeight(RankProfile profile) throws ParseException
This rule consumes a field-weight statement of a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
fieldRankType
public final void fieldRankType(RankProfile profile) throws ParseException
This rule consumes a rank-type statement of a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
fieldRankFilter
public final void fieldRankFilter(RankProfile profile) throws ParseException
This rule consumes a rank filter statement of a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
rankDegradationBinSize
public final void rankDegradationBinSize() throws ParseException
This rule consumes part of a rank-degradation statement of a rank profile.- Throws:
ParseException
-
rankDegradationBinLow
public final void rankDegradationBinLow() throws ParseException
This rule consumes part of a rank-degradation statement of a rank profile.- Throws:
ParseException
-
rankDegradationPosbinSize
public final void rankDegradationPosbinSize() throws ParseException
This rule consumes part of a rank-degradation statement of a rank profile.- Throws:
ParseException
-
rankDegradationItem
public final Object rankDegradationItem() throws ParseException
This rule consumes part of a rank-degradation statement of a rank profile.- Throws:
ParseException
-
rankDegradation
public final Object rankDegradation(RankProfile profile) throws ParseException
This rule consumes a rank-degradation statement of a rank profile.- Parameters:
profile
- The rank profile to modify.- Throws:
ParseException
-
constants
public final void constants(RankProfile profile) throws ParseException
Consumes a set of constants available in ranking expressions in the enclosing profile.- Throws:
ParseException
-
constantValue
public final void constantValue(RankProfile profile, String name) throws ParseException
- Throws:
ParseException
-
constantTensor
public final void constantTensor(RankProfile profile, String name) throws ParseException
- Throws:
ParseException
-
constantTensorErrorMessage
public final String constantTensorErrorMessage(String rankProfileName, String constantTensorName) throws ParseException
- Throws:
ParseException
-
tensorValue
public final String tensorValue() throws ParseException
- Throws:
ParseException
-
tensorTypeWithPrefix
public final com.yahoo.tensor.TensorType tensorTypeWithPrefix(String errorMessage) throws ParseException
- Throws:
ParseException
-
tensorType
public final com.yahoo.tensor.TensorType tensorType(String errorMessage) throws ParseException
- Throws:
ParseException
-
importField
public final void importField(Schema schema) throws ParseException
- Throws:
ParseException
-
expression
public final String expression() throws ParseException
This rule consumes an expression token and returns its image.- Returns:
- The consumed token image.
- Throws:
ParseException
-
identifierWithDash
public final String identifierWithDash() throws ParseException
- Throws:
ParseException
-
identifier
public final String identifier() throws ParseException
Consumes an identifier. This must be kept in sync with all word tokens that should be parseable as identifiers.- Returns:
- the identifier string
- Throws:
ParseException
-
string
public final String string() throws ParseException
Consumes a string token and returns the token image.- Returns:
- The consumed token image.
- Throws:
ParseException
-
quotedString
public final String quotedString() throws ParseException
Consumes a quoted string token and returns the token image minus the quotes. This does not perform unescaping of the content, it simply removes the first and last character of the image. However, the token itself can contain anything but a double quote.- Returns:
- The unquoted token image.
- Throws:
ParseException
-
bool
public final Boolean bool() throws ParseException
This rule consumes a boolean value.- Returns:
- The consumed boolean value.
- Throws:
ParseException
-
integer
public final int integer() throws ParseException
This rule consumes an integer token and returns its numeric value.- Returns:
- The consumed integer value.
- Throws:
ParseException
-
consumeLong
public final long consumeLong() throws ParseException
This rule consumes a long or integer token and returns its numeric value.- Returns:
- The consumed long value.
- Throws:
ParseException
-
consumeFloat
public final double consumeFloat() throws ParseException
This rule consumes a floating-point token and returns its numeric value.- Returns:
- The consumed value.
- Throws:
ParseException
-
consumeNumber
public final Number consumeNumber() throws ParseException
- Throws:
ParseException
-
lbrace
public final void lbrace() throws ParseException
This rule consumes an opening brace with leading and trailing newline tokens.- Throws:
ParseException
-
ReInit
public void ReInit(CharStream stream)
Reinitialise.- Parameters:
stream
- stream to init with
-
ReInit
public void ReInit(SDParserTokenManager tm)
Reinitialise- Parameters:
tm
- Token manager to use
-
getNextToken
public final Token getNextToken()
- Returns:
- the next Token.
-
getToken
public final Token getToken(int index)
- Parameters:
index
- index to be retrieved- Returns:
- the specific Token.
-
generateParseException
public ParseException generateParseException()
Generate ParseException.- Returns:
- new Exception object. Never
null
-
trace_enabled
public final boolean trace_enabled()
- Returns:
- Always
false
.
-
enable_tracing
public final void enable_tracing()
Enable tracing.
-
disable_tracing
public final void disable_tracing()
Disable tracing.
-
-