Package com.yahoo.searchdefinition.processing

Classes in this package (processors) implements some search definition features by reducing them to simpler features. The processors are run after parsing of the search definition, before creating the derived model. For simplicity, features should always be implemented here rather than in the derived model if possible. New processors must be added to the list in Processing.

Interface Summary

Interface	Description
Processing.ProcessorFactory

Class Summary

Class	Description
AddAttributeTransformToSummaryOfImportedFields	Adds the attribute summary transform (SummaryTransform.ATTRIBUTE to all SummaryField having an imported field as source.
AddExtraFieldsToDocument	This processor creates a SDDocumentType for each Schema object which holds all the data that search associates with a document described in a search definition file.
AdjustPositionSummaryFields
AttributeProperties	Checks that attribute properties only are set for attributes that have data (are created by an indexing statement).
AttributesImplicitWord	Fields that derive to attribute(s) and no indices should use the WORD indexing form, in a feeble attempt to match the most peoples expectations as closely as possible.
Bolding	Checks that bolding or dynamic summary is turned on only for text fields.
BoolAttributeValidator	Validates attribute fields using bool type, ensuring the collection type is supported.
BuiltInFieldSets	Adds field sets for 1) fields defined inside document type 2) fields inside search but outside document
CreatePositionZCurve	Adds a "fieldName_zcurve" long attribute and "fieldName.distance" and "FieldName.position" summary fields to all position type fields.
DictionaryProcessor	Propagates dictionary settings from field level to attribute level.
DisallowComplexMapAndWsetKeyTypes	Non-primitive key types for map and weighted set forbidden (though OK in document model)
DiversitySettingsValidator
ExactMatch	The implementation of exact matching
FastAccessValidator	Validates the use of the fast-access property.
FieldSetSettings	Computes the right "index commands" for each fieldset in a search definition.
FilterFieldNames	Takes the fields and indexes that are of type rank filter, and stores those names on all rank profiles
ImplicitSummaries	Makes implicitly defined summaries into explicit summaries
ImplicitSummaryFields	This processor adds all implicit summary fields to all registered document summaries.
ImportedFieldsResolver	Iterates all imported fields from schema parsing and validates and resolves them into concrete fields from referenced document types.
IndexFieldNames	Because of the way the parser works (allowing any token as identifier), it is not practical to limit the syntax of field names there, do it here.
IndexingInputs	This processor modifies all indexing scripts so that they input the value of the owning field by default.
IndexingOutputs	This processor modifies all indexing scripts so that they output to the owning field by default.
IndexingValidation
IndexingValues
IntegerIndex2Attribute	Replaces the 'index' statement of all numerical fields to 'attribute' because we no longer support numerical indexes.
LiteralBoost	Expresses literal boosts in terms of extra indices with rank boost.
MakeAliases	Takes the aliases set on field by parser and sets them on correct Index or Attribute
MakeDefaultSummaryTheSuperSet	All summary fields which are not attributes must currently be present in the default summary class, since the default summary class also defines the docsum.dat format.
MatchConsistency	Warn on inconsistent match settings for any index
MatchedElementsOnlyResolver	Iterates all summary fields with 'matched-elements-only' and adjusts transform (if all struct-fields are attributes) and validates that the field type is supported.
MatchPhaseSettingsValidator	Validates the match phase settings for all registered rank profiles.
MatchPhaseSettingsValidator.AttributeValidator
MultifieldIndexHarmonizer	Ensures that there are no conflicting types or field settings in multifield indices, either by changing settings or by splitting conflicting fields in multiple ones with different settings.
MutableAttributes
NGramMatch	The implementation of "gram" matching - splitting the incoming text and the queries into n-grams for matching.
OnnxModelConfigGenerator	Processes ONNX ranking features of the form: onnx("files/model.onnx", "path/to/output:1") And generates an "onnx-model" configuration as if it was defined in the schema: onnx-model files_model_onnx { file: "files/model.onnx" } Inputs and outputs are resolved in OnnxModelTypeResolver, which must be processed after this.
OnnxModelTypeResolver	Processes every "onnx-model" element in the schema.
OptimizeIlscript	Run ExpressionOptimizer on all scripts, to get rid of expressions that have no effect.
PagedAttributeValidator	Validates the 'paged' attribute setting and throws if specified on unsupported types.
PredicateProcessor	Validates the predicate fields.
Processing	Executor of processors.
Processor	Abstract superclass of all search definition processors.
RankingExpressionTypeResolver	Resolves and assigns types to all functions in a ranking expression, and validates the types of all ranking expressions under a search instance: Some operators constrain the types of inputs, and first-and second-phase expressions must return scalar values.
ReferenceFieldsProcessor	Class that processes reference fields and removes attribute aspect of such fields from summary.
ReservedDocumentNames
ReservedFunctionNames	Issues a warning if some function has a reserved name.
SearchMustHaveDocument	A search must have a document definition of the same name inside of it, otherwise crashes may occur as late as during feeding
SetRankTypeEmptyOnFilters	All rank: filter fields should have rank type empty.
SortingSettings	Validate conflicting settings for sorting
StringSettingsOnNonStringFields
SummaryConsistency	Ensure that summary field transforms for fields having the same name are consistent across summary classes
SummaryDiskAccessValidator	Emits a warning for summaries which accesses disk.
SummaryDynamicStructsArrays	Fail if: An SD field explicitly says summary:dynamic , but the field is wset, array or struct.
SummaryFieldsMustHaveValidSource	Verifies that the source fields actually refers to a valid field.
SummaryNamesFieldCollisions	Verifies that equally named summary fields in different summary classes don't use different fields for source.
TagType	The implementation of the tag datatype
TensorFieldProcessor	Class that processes and validates tensor fields.
TextMatch
TypedTransformProvider
UriHack
UrlFieldValidator
ValidateFieldTypes	This Processor makes sure all fields with the same name have the same DataType.
ValidateFieldTypesDocumentsOnly
ValidateFieldWithIndexSettingsCreatesIndex	Check that fields with index settings actually creates an index or attribute
ValidateStructTypeInheritance
WordMatch	The implementation of word matching - with word matching the field is assumed to contain a single "word" - some contiguous sequence of word and number characters - but without changing the data at the indexing side (as with text matching) to enforce this.