Class ProbableIntersectionCursor<T>
- java.lang.Object
-
- com.apple.foundationdb.record.provider.foundationdb.cursors.MergeCursor<T,T,com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>>
-
- com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursor<T>
-
- Type Parameters:
T
- the type of element returned by this cursor
- All Implemented Interfaces:
RecordCursor<T>
,AutoCloseable
,Iterator<T>
@API(EXPERIMENTAL) public class ProbableIntersectionCursor<T> extends MergeCursor<T,T,com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>>
A cursor that returns all results that are probably in all of its children. This differs from theIntersectionCursor
in that it does not require that its children produce results in a compatible order. Just like theIntersectionCursor
, this cursor does require a comparison key function for each entry, but it only uses this comparison key for determining whether two results returned by child cursors are equal.This cursor makes very few guarantees about its results. In particular, just as with the
UnorderedUnionCursor
, results are returned as they come, so the exact ordering of returned results is determined only at runtime. It also can produce duplicates if the same result appears multiple times in its child cursors. Additionally, in order to support resuming this cursor using a continuation, Bloom filters are used internally to remember which results the cursors have already seen. However, as Bloom filters can report false positives, it is possible that this cursor can return a result that only appears in a proper subset of the child cursors' result sets. The selectivity of the Bloom filter can be adjusted by setting theexpectedResults
andfalsePositivePercentage
parameters at cursor creation time. These parameters are fed through to the underlying GuavaBloomFilter
initializer.However, this cursor does make the following guarantees:
- All results that are actually in the intersection will be in this cursor's result set.
- Any result of this cursor is the result of at least one child cursor.
This cursor can therefore be used to select a narrow candidate set of "probable" elements of the intersection and then perform a (possibly more expensive) alternative filter to verify each result. For example, if each child corresponds to satisfying some conjunct of an
"and"
predicate by scanning an index, then one could intersect the results with this cursor and then evaluate the predicate on each returned record as a residual filter.- See Also:
BloomFilter
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface com.apple.foundationdb.record.RecordCursor
RecordCursor.NoNextReason
-
-
Field Summary
Fields Modifier and Type Field Description static long
DEFAULT_EXPECTED_RESULTS
The default number of results to expect to be read from each child cursor of this cursor.static double
DEFAULT_FALSE_POSITIVE_PERCENTAGE
The default acceptable false positive percentage when evaluating whether an element is contained with a given cursor's result set.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description protected CompletableFuture<List<com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>>>
computeNextResultStates()
Determine which cursors should have their values included in the next iteration.static <T> ProbableIntersectionCursor<T>
create(Function<? super T,? extends List<Object>> comparisonKeyFunction, List<Function<byte[],RecordCursor<T>>> cursorFunctions, byte[] continuation, FDBStoreTimer timer)
Create a cursor merging the results of two or more cursors.static <T> ProbableIntersectionCursor<T>
create(Function<? super T,? extends List<Object>> comparisonKeyFunction, List<Function<byte[],RecordCursor<T>>> cursorFunctions, long expectedResults, double falsePositivePercentage, byte[] continuation, FDBStoreTimer timer)
Create a cursor merging the results of two or more cursors.protected com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorContinuation
getContinuationObject()
Produce aRecordCursorContinuation
for this cursor.protected T
getNextResult(List<com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>> resultStates)
Determine the next result to return based on the results of the child cursors.protected RecordCursor.NoNextReason
mergeNoNextReasons()
Merge theRecordCursor.NoNextReason
s for child cursors.-
Methods inherited from class com.apple.foundationdb.record.provider.foundationdb.cursors.MergeCursor
accept, checkNextStateTimeout, close, getChildContinuations, getContinuation, getCursorStates, getExecutor, getNoNextReason, getStrongestNoNextReason, getTimer, getWeakestNoNextReason, next, onHasNext, onNext, whenAll, whenAny
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface java.util.Iterator
forEachRemaining, remove
-
Methods inherited from interface com.apple.foundationdb.record.RecordCursor
asIterator, asList, filter, filterAsync, filterAsyncInstrumented, filterAsyncInstrumented, filterInstrumented, filterInstrumented, first, flatMapPipelined, forEach, forEachAsync, forEachResult, forEachResultAsync, getCount, getNext, hasNext, limitRowsTo, limitTo, map, mapEffect, mapEffect, mapPipelined, orElse, reduce, skip, skipThenLimit
-
-
-
-
Field Detail
-
DEFAULT_EXPECTED_RESULTS
public static final long DEFAULT_EXPECTED_RESULTS
The default number of results to expect to be read from each child cursor of this cursor.- See Also:
- Constant Field Values
-
DEFAULT_FALSE_POSITIVE_PERCENTAGE
public static final double DEFAULT_FALSE_POSITIVE_PERCENTAGE
The default acceptable false positive percentage when evaluating whether an element is contained with a given cursor's result set.- See Also:
- Constant Field Values
-
-
Method Detail
-
computeNextResultStates
@Nonnull protected CompletableFuture<List<com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>>> computeNextResultStates()
Description copied from class:MergeCursor
Determine which cursors should have their values included in the next iteration. This list should include the state associated with each cursor, and the returned value should be a (not necessarily proper) sublist of the result states of this cursor. (This list can be retrieved by callingMergeCursor.getCursorStates()
.) To indicate that this cursor is done returning elements, this function should return the empty list.- Specified by:
computeNextResultStates
in classMergeCursor<T,T,com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>>
- Returns:
- the list of cursors to include in the next iteration
-
getNextResult
@Nonnull protected T getNextResult(@Nonnull List<com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>> resultStates)
Description copied from class:MergeCursor
Determine the next result to return based on the results of the child cursors. This will be called with the return value ofMergeCursor.computeNextResultStates()
.- Specified by:
getNextResult
in classMergeCursor<T,T,com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>>
- Parameters:
resultStates
- the list of cursors to be included in the result- Returns:
- a result somehow combining the results of the input cursors
-
mergeNoNextReasons
@Nonnull protected RecordCursor.NoNextReason mergeNoNextReasons()
Description copied from class:MergeCursor
Merge theRecordCursor.NoNextReason
s for child cursors. This will only be called after it is determined that the merge cursor itself will not return aNoNextReason
. Note that this may (or may) not imply that all child cursors are done (depending on the implementation of the extending cursor). ThisNoNextReason
will be the reason returned by the merge cursor.- Specified by:
mergeNoNextReasons
in classMergeCursor<T,T,com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>>
- Returns:
- a
RecordCursor.NoNextReason
based on the child cursors'NoNextReason
s
-
getContinuationObject
@Nonnull protected com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorContinuation getContinuationObject()
Description copied from class:MergeCursor
Produce aRecordCursorContinuation
for this cursor. The super-class itself handles constructing the byte array continuation from this continuation. Most continuations will be based on the continuations of this cursors child cursors. These can be retrieved by callingMergeCursor.getChildContinuations()
. This will only be called afterMergeCursor.computeNextResultStates()
has completed.- Specified by:
getContinuationObject
in classMergeCursor<T,T,com.apple.foundationdb.record.provider.foundationdb.cursors.ProbableIntersectionCursorState<T>>
- Returns:
- a
RecordCursorContinuation
for this cursor based on the state of its child cursors
-
create
@Nonnull public static <T> ProbableIntersectionCursor<T> create(@Nonnull Function<? super T,? extends List<Object>> comparisonKeyFunction, @Nonnull List<Function<byte[],RecordCursor<T>>> cursorFunctions, @Nullable byte[] continuation, @Nullable FDBStoreTimer timer)
Create a cursor merging the results of two or more cursors. This uses the default values forexpectedResults
andfalsePositivePercentage
.- Type Parameters:
T
- the type of elements returned by this cursor- Parameters:
comparisonKeyFunction
- the function evaluated to compare elements from different cursorscursorFunctions
- a list of functions to produceRecordCursor
s from a continuationcontinuation
- any continuation from a previous scantimer
- the timer used to instrument events- Returns:
- a cursor containing any records from any child cursor
- See Also:
create(Function, List, long, double, byte[], FDBStoreTimer)
-
create
@Nonnull public static <T> ProbableIntersectionCursor<T> create(@Nonnull Function<? super T,? extends List<Object>> comparisonKeyFunction, @Nonnull List<Function<byte[],RecordCursor<T>>> cursorFunctions, long expectedResults, double falsePositivePercentage, @Nullable byte[] continuation, @Nullable FDBStoreTimer timer)
Create a cursor merging the results of two or more cursors. The resulting cursor will return all children that are probably in all of the child cursors. Unlike theIntersectionCursor
, this does not require that results be returned in the same order as their values from the comparison key function. However, it does require that if two elements evaluate to the same comparison key that they are equal.Every result from this cursor is guaranteed to be in at least one child, but some results may appear in only a proper subset of the given cursors. This is because a probabilistic data structure is used internally to allow for the cursor to be resumed across continuation boundaries. The caller can adjust the memory usage as well as the false positive rate by tweaking the values of the
expectedResults
andfalsePositivePercentage
parameters. Note that those parameters only matter if the continuation isnull
. A non-null continuation will read those parameters based on serialized versions of the data structure that are included as part of the continuation.- Type Parameters:
T
- the type of elements returned by this cursor- Parameters:
comparisonKeyFunction
- the function evaluated to compare elements from different cursorscursorFunctions
- a list of functions to produceRecordCursor
s from a continuationexpectedResults
- the expected number of results from each child cursorfalsePositivePercentage
- an acceptable false positive percentage for each cursorcontinuation
- any continuation from a previous scantimer
- the timer used to instrument events- Returns:
- a cursor containing any records from any child cursor
-
-