com.yahoo.search.predicate

Class PredicateIndexBuilder

java.lang.Object

com.yahoo.search.predicate.PredicateIndexBuilder

@Beta
public class PredicateIndexBuilder
extends Object

A builder for PredicateIndex.

When creating a PredicateIndexBuilder, you must specify an arity. This is used for range features, and is a trade-off of index size vs. query speed. Higher arities gives larger index but faster search.

indexDocument(int, Predicate) takes a document id and a predicate to insert into the index. Predicates should be specified using the predicate syntax described in the documentation. Create the Predicate objects using Predicate.fromString(String).

Use build() to create an instance of PredicateIndex.

Author:

bjorncs

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PredicateIndexBuilder.PredicateIndexStats A collection of metrics about the currently built PredicateIndex.

Field Summary

Fields

Modifier and Type	Field and Description
private com.yahoo.search.predicate.index.SimpleIndex.Builder	boundsIndexBuilder
private Config	config
private com.yahoo.search.predicate.index.conjunction.ConjunctionIndexBuilder	conjunctionIndexBuilder
private com.yahoo.search.predicate.index.SimpleIndex.Builder	conjunctionIntervalIndexBuilder
private int	documentIdCounter
private int	highestIntervalEnd
private List<Short>	intervalEndsBuilder
private com.yahoo.search.predicate.index.SimpleIndex.Builder	intervalIndexBuilder
private com.yahoo.search.predicate.index.PredicateIntervalStore.Builder	intervalStoreBuilder
private List<Byte>	minFeatureIndexBuilder
private int	nZStarDocuments
private int	nZStarIntervals
private com.yahoo.search.predicate.index.PredicateOptimizer	optimizer
private Set<Integer>	seenIds
private List<Integer>	zeroConstraintDocuments

Constructor Summary

Constructors

Constructor and Description
PredicateIndexBuilder(Config config) Creates a PredicateIndexBuilder based on a Config object.
PredicateIndexBuilder(int arity) Creates a PredicateIndexBuilder with default upper and lower bounds.
PredicateIndexBuilder(int arity, long lowerBound, long upperBound) Creates a PredicateIndexBuilder.

Method Summary

Modifier and Type	Method and Description
private void	aggregateZStarStatistics(Map<Long,List<Integer>> intervalMap)
PredicateIndex	build()
PredicateIndexBuilder.PredicateIndexStats	getStats() Retrieve metrics about the current index.
int	getZeroConstraintDocCount()
void	indexDocument(int docId, com.yahoo.document.predicate.Predicate predicate) Indexes a predicate with the given id.
private void	indexDocument(int docId, com.yahoo.search.predicate.annotator.PredicateTreeAnnotations annotations)
private void	indexDocumentBoundsFeatures(int docId, Map<Long,List<com.yahoo.search.predicate.index.IntervalWithBounds>> boundsMap)
private void	indexDocumentConjunctions(int docId, Map<com.yahoo.search.predicate.index.conjunction.IndexableFeatureConjunction,List<Integer>> featureConjunctions)
private void	indexDocumentFeatures(int docId, Map<Long,List<Integer>> intervalMap)
private void	indexZeroConstraintDocument(int docId)
private static boolean	isAlwaysMatchingDocument(com.yahoo.document.predicate.Predicate p)
private static boolean	isNeverMatchingDocument(com.yahoo.document.predicate.Predicate p)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

seenIds

private final Set<Integer> seenIds

intervalEndsBuilder

private final List<Short> intervalEndsBuilder

minFeatureIndexBuilder

private final List<Byte> minFeatureIndexBuilder

zeroConstraintDocuments

private final List<Integer> zeroConstraintDocuments

intervalIndexBuilder

private final com.yahoo.search.predicate.index.SimpleIndex.Builder intervalIndexBuilder

boundsIndexBuilder

private final com.yahoo.search.predicate.index.SimpleIndex.Builder boundsIndexBuilder

conjunctionIntervalIndexBuilder

private final com.yahoo.search.predicate.index.SimpleIndex.Builder conjunctionIntervalIndexBuilder

conjunctionIndexBuilder

private final com.yahoo.search.predicate.index.conjunction.ConjunctionIndexBuilder conjunctionIndexBuilder

intervalStoreBuilder

private final com.yahoo.search.predicate.index.PredicateIntervalStore.Builder intervalStoreBuilder

optimizer

private final com.yahoo.search.predicate.index.PredicateOptimizer optimizer

config

private final Config config

documentIdCounter

private int documentIdCounter

nZStarDocuments

private int nZStarDocuments

nZStarIntervals

private int nZStarIntervals

highestIntervalEnd

private int highestIntervalEnd

Constructor Detail

PredicateIndexBuilder

public PredicateIndexBuilder(int arity)

Creates a PredicateIndexBuilder with default upper and lower bounds.

Parameters:

arity - The arity to use when indexing range predicates. Small arity gives smaller index, but more expensive searches.

PredicateIndexBuilder

public PredicateIndexBuilder(int arity,
                             long lowerBound,
                             long upperBound)

Creates a PredicateIndexBuilder. Limiting the range of possible values in range predicates reduces index size and increases search performance.

Parameters:

arity - The arity to use when indexing range predicates. Small arity gives smaller index, but more expensive searches.

lowerBound - The lower bound for the range of values used by range predicates.

upperBound - The upper bound for the range of values used by range predicates.

PredicateIndexBuilder

public PredicateIndexBuilder(Config config)

Creates a PredicateIndexBuilder based on a Config object.

Parameters:

config - Configuration for the PredicateIndexBuilder.

Method Detail

indexDocument

public void indexDocument(int docId,
                          com.yahoo.document.predicate.Predicate predicate)

Indexes a predicate with the given id.

Parameters:

docId - A 32-bit document id, returned in the Hit objects when the predicate matches.

predicate - The predicate to index.

isAlwaysMatchingDocument

private static boolean isAlwaysMatchingDocument(com.yahoo.document.predicate.Predicate p)

isNeverMatchingDocument

private static boolean isNeverMatchingDocument(com.yahoo.document.predicate.Predicate p)

indexZeroConstraintDocument

private void indexZeroConstraintDocument(int docId)

indexDocument

private void indexDocument(int docId,
                           com.yahoo.search.predicate.annotator.PredicateTreeAnnotations annotations)

aggregateZStarStatistics

private void aggregateZStarStatistics(Map<Long,List<Integer>> intervalMap)

indexDocumentFeatures

private void indexDocumentFeatures(int docId,
                                   Map<Long,List<Integer>> intervalMap)

indexDocumentBoundsFeatures

private void indexDocumentBoundsFeatures(int docId,
                                         Map<Long,List<com.yahoo.search.predicate.index.IntervalWithBounds>> boundsMap)

indexDocumentConjunctions

private void indexDocumentConjunctions(int docId,
                                       Map<com.yahoo.search.predicate.index.conjunction.IndexableFeatureConjunction,List<Integer>> featureConjunctions)

build

public PredicateIndex build()

getZeroConstraintDocCount

public int getZeroConstraintDocCount()

getStats

public PredicateIndexBuilder.PredicateIndexStats getStats()

Retrieve metrics about the current index.

Returns:

An object containing metrics.