Class StandardIndexMaintainer
- java.lang.Object
-
- com.apple.foundationdb.record.provider.foundationdb.IndexMaintainer
-
- com.apple.foundationdb.record.provider.foundationdb.indexes.StandardIndexMaintainer
-
- Direct Known Subclasses:
AtomicMutationIndexMaintainer
,BitmapValueIndexMaintainer
,RankIndexMaintainer
,TextIndexMaintainer
,TimeWindowLeaderboardIndexMaintainer
,ValueIndexMaintainer
,VersionIndexMaintainer
@API(MAINTAINED) public abstract class StandardIndexMaintainer extends IndexMaintainer
Base class forIndexMaintainer
implementation.
-
-
Field Summary
Fields Modifier and Type Field Description protected static int
TOO_LARGE_VALUE_MESSAGE_LIMIT
-
Fields inherited from class com.apple.foundationdb.record.provider.foundationdb.IndexMaintainer
state
-
-
Constructor Summary
Constructors Modifier Constructor Description protected
StandardIndexMaintainer(IndexMaintainerState state)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description CompletableFuture<Boolean>
addedRangeWithKey(Tuple primaryKey)
Whether this key has been added to some range within theRangeSet
associated with this index.protected void
addUniquenessViolation(Tuple valueKey, Tuple primaryKey, Tuple existingKey)
Add a uniqueness violation within the database.protected static boolean
canDeleteWhere(IndexMaintainerState state, QueryToKeyMatcher.Match match, Key.Evaluated evaluated)
boolean
canDeleteWhere(QueryToKeyMatcher matcher, Key.Evaluated evaluated)
Get whether this index scan delete records matching a particular key query.boolean
canEvaluateAggregateFunction(IndexAggregateFunction function)
Get whether this index can be used to evaluate the given aggregate function.boolean
canEvaluateRecordFunction(IndexRecordFunction<?> function)
Returntrue
if this index be used to evaluate the given record function.protected <M extends Message>
voidcheckKeyValueSizes(FDBIndexableRecord<M> savedRecord, Tuple valueKey, Tuple value, byte[] keyBytes, byte[] valueBytes)
protected <M extends Message>
voidcheckUniqueness(FDBIndexableRecord<M> savedRecord, IndexEntry indexEntry)
protected List<IndexEntry>
commonKeys(List<IndexEntry> oldIndexEntries, List<IndexEntry> newIndexEntries)
protected Tuple
decodeValue(byte[] value)
Decode value portion of key value pair.CompletableFuture<Void>
deleteWhere(Transaction tr, Tuple prefix)
Clear index storage associated with the given key prefix.CompletableFuture<Tuple>
evaluateAggregateFunction(IndexAggregateFunction function, TupleRange range, IsolationLevel isolationLevel)
Evaluate an aggregate function over the given range using this index.protected <M extends Message>
List<IndexEntry>evaluateIndex(FDBRecord<M> record)
Apply the key and value expressions to arecord
.<T,M extends Message>
CompletableFuture<T>evaluateRecordFunction(EvaluationContext context, IndexRecordFunction<T> function, FDBRecord<M> record)
Evaluate a record function on the given record.protected <M extends Message>
List<IndexEntry>filteredIndexEntries(FDBIndexableRecord<M> savedRecord)
Filter out index keys according toIndexMaintenanceFilter
.protected Executor
getExecutor()
protected int
getGroupedCount()
protected int
getGroupingCount()
protected FDBStoreTimer
getTimer()
protected Tuple
indexEntryKey(Tuple valueKey, Tuple primaryKey)
The entire index key to be used, including both the indexed value(s) and the primary key(s), with redundancy removed.boolean
isIdempotent()
Whether updating or removing a record on this index is idempotent.protected static <T> List<T>
makeMutable(List<T> list)
CompletableFuture<IndexOperationResult>
performOperation(IndexOperation operation)
Perform a type-specific operation on index.protected CompletableFuture<Void>
removeUniquenessViolationsAsync(Tuple valueKey, Tuple primaryKey)
Remove a uniqueness violation within the database.protected void
saveIndexEntryAsKeyValue(IndexEntry keyValue)
Manually save an index entry, for example when rebuilding in place with a different storage format.protected RecordCursor<IndexEntry>
scan(TupleRange range, byte[] continuation, ScanProperties scanProperties)
Scan the primary index tree for the given range.RecordCursor<IndexEntry>
scanUniquenessViolations(TupleRange range, byte[] continuation, ScanProperties scanProperties)
Scans through the list of uniqueness violations within the database.boolean
skipUpdateForUnchangedKeys()
protected static String
trimTooLargeTuple(Tuple tuple)
protected IndexEntry
unpackKeyValue(KeyValue kv)
Convert stored key value pair into an index entry.protected IndexEntry
unpackKeyValue(Subspace subspace, KeyValue kv)
Convert stored key value pair located in the given subspace into an index entry.<M extends Message>
CompletableFuture<Void>update(FDBIndexableRecord<M> oldRecord, FDBIndexableRecord<M> newRecord)
Update associated index for a changed record.protected <M extends Message>
CompletableFuture<Void>updateIndexKeys(FDBIndexableRecord<M> savedRecord, boolean remove, List<IndexEntry> indexEntries)
Update index according to record keys.protected <M extends Message>
Function<Void,CompletableFuture<Void>>updateIndexKeysFunction(FDBIndexableRecord<M> savedRecord, boolean remove, List<IndexEntry> indexEntries)
protected <M extends Message>
CompletableFuture<Void>updateOneKeyAsync(FDBIndexableRecord<M> savedRecord, boolean remove, IndexEntry indexEntry)
Store a single key in the index.RecordCursor<InvalidIndexEntry>
validateEntries(byte[] continuation, ScanProperties scanProperties)
Validate the integrity of the index (such as identifying index entries that do not point to records or identifying records that do not point to valid index entries).protected RecordCursor<InvalidIndexEntry>
validateMissingEntries(byte[] continuation, ScanProperties scanProperties)
Validate entries in the index.protected RecordCursor<InvalidIndexEntry>
validateOrphanEntries(byte[] continuation, ScanProperties scanProperties)
Validate entries in the index.-
Methods inherited from class com.apple.foundationdb.record.provider.foundationdb.IndexMaintainer
getIndexSubspace, getSecondarySubspace, scan, unsupportedAggregateFunction, unsupportedRecordFunction
-
-
-
-
Field Detail
-
TOO_LARGE_VALUE_MESSAGE_LIMIT
protected static final int TOO_LARGE_VALUE_MESSAGE_LIMIT
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
StandardIndexMaintainer
protected StandardIndexMaintainer(IndexMaintainerState state)
-
-
Method Detail
-
getTimer
@Nullable protected FDBStoreTimer getTimer()
-
scan
protected RecordCursor<IndexEntry> scan(@Nonnull TupleRange range, @Nullable byte[] continuation, @Nonnull ScanProperties scanProperties)
Scan the primary index tree for the given range.- Parameters:
range
- range of index keys to scancontinuation
- any continuation from previous scanscanProperties
- any limits on the scan- Returns:
- a cursor of index entries within the given range
-
unpackKeyValue
@Nonnull protected IndexEntry unpackKeyValue(@Nonnull KeyValue kv)
Convert stored key value pair into an index entry.- Parameters:
kv
- a raw key-value from the database- Returns:
- an index entry
-
unpackKeyValue
@Nonnull protected IndexEntry unpackKeyValue(@Nonnull Subspace subspace, @Nonnull KeyValue kv)
Convert stored key value pair located in the given subspace into an index entry.- Parameters:
subspace
- the database subspace for the indexkv
- a raw key-value withinsubspace
- Returns:
- an index entry
-
decodeValue
@Nonnull protected Tuple decodeValue(@Nonnull byte[] value)
Decode value portion of key value pair.- Parameters:
value
- the raw value portion of a key value pair- Returns:
- a decoded tuple of any values stored in the value side of the index, which is usually empty
-
skipUpdateForUnchangedKeys
public boolean skipUpdateForUnchangedKeys()
-
update
@Nonnull public <M extends Message> CompletableFuture<Void> update(@Nullable FDBIndexableRecord<M> oldRecord, @Nullable FDBIndexableRecord<M> newRecord)
Description copied from class:IndexMaintainer
Update associated index for a changed record.- Specified by:
update
in classIndexMaintainer
- Type Parameters:
M
- type of message- Parameters:
oldRecord
- the previous stored record ornull
if a new record is being creatednewRecord
- the new record ornull
if an old record is being deleted- Returns:
- a future that is complete when the record update is done
-
filteredIndexEntries
@Nullable protected <M extends Message> List<IndexEntry> filteredIndexEntries(@Nullable FDBIndexableRecord<M> savedRecord)
Filter out index keys according toIndexMaintenanceFilter
. Keys that do not pass the filter will not be stored / removed from the index.- Type Parameters:
M
- the message type of the record- Parameters:
savedRecord
- record for key evaluation- Returns:
- filtered list of index keys for the given record
-
commonKeys
@Nonnull protected List<IndexEntry> commonKeys(@Nonnull List<IndexEntry> oldIndexEntries, @Nonnull List<IndexEntry> newIndexEntries)
-
updateIndexKeysFunction
@Nonnull protected <M extends Message> Function<Void,CompletableFuture<Void>> updateIndexKeysFunction(@Nonnull FDBIndexableRecord<M> savedRecord, boolean remove, @Nonnull List<IndexEntry> indexEntries)
-
updateIndexKeys
protected <M extends Message> CompletableFuture<Void> updateIndexKeys(@Nonnull FDBIndexableRecord<M> savedRecord, boolean remove, @Nonnull List<IndexEntry> indexEntries)
Update index according to record keys. Often this operation returns an already completed future because there is no asynchronous work to be done.- Type Parameters:
M
- the message type of the record- Parameters:
savedRecord
- the record being indexedremove
-true
if removing from indexindexEntries
- the result ofevaluateIndex(FDBRecord)
- Returns:
- a future completed when update is done
-
updateOneKeyAsync
protected <M extends Message> CompletableFuture<Void> updateOneKeyAsync(@Nonnull FDBIndexableRecord<M> savedRecord, boolean remove, @Nonnull IndexEntry indexEntry)
Store a single key in the index.- Type Parameters:
M
- the message type of the record- Parameters:
savedRecord
- the record being indexedremove
-true
if removing from indexindexEntry
- the entry for the index to be updated- Returns:
- a future completed when the key is updated
-
checkUniqueness
protected <M extends Message> void checkUniqueness(@Nonnull FDBIndexableRecord<M> savedRecord, @Nonnull IndexEntry indexEntry)
-
addUniquenessViolation
protected void addUniquenessViolation(@Nonnull Tuple valueKey, @Nonnull Tuple primaryKey, @Nullable Tuple existingKey)
Add a uniqueness violation within the database. This is used to keep track of uniqueness violations that occur when an index is in write-only mode, both during the built itself and by other writes. This means that the writes will succeed, but it will cause a later attempt to make the index readable to fail.- Parameters:
valueKey
- the indexed key that is (apparently) not uniqueprimaryKey
- the primary key of one record that is causing a violationexistingKey
- the primary key of another record that is causing a violation (ornull
if none specified)
-
removeUniquenessViolationsAsync
@Nonnull protected CompletableFuture<Void> removeUniquenessViolationsAsync(@Nonnull Tuple valueKey, @Nonnull Tuple primaryKey)
Remove a uniqueness violation within the database. This is used to keep track of uniqueness violations that occur when an index is in write-only mode, both during the built itself and by other writes. This means that the writes will succeed, but it will cause a later attempt to make the index readable to fail.This will remove the last uniqueness violation entry when removing the second last entry that contains the value key.
- Parameters:
valueKey
- the indexed key that is (apparently) not uniqueprimaryKey
- the primary key of one record that is causing a violation- Returns:
- a future that is complete when the uniqueness violation is removed
-
scanUniquenessViolations
@Nonnull public RecordCursor<IndexEntry> scanUniquenessViolations(@Nonnull TupleRange range, @Nullable byte[] continuation, @Nonnull ScanProperties scanProperties)
Description copied from class:IndexMaintainer
Scans through the list of uniqueness violations within the database. It will return a cursor ofIndexEntry
instances where thegetKey()
will return the primary key of the record causing a problem andgetValue()
will return the index value that is being duplicated.- Specified by:
scanUniquenessViolations
in classIndexMaintainer
- Parameters:
range
- range of tuples to readcontinuation
- any continuation from a previous invocationscanProperties
- row limit and other scan properties- Returns:
- a cursor that will return primary key-index key pairs indicating uniqueness violations
-
validateEntries
@Nonnull public RecordCursor<InvalidIndexEntry> validateEntries(@Nullable byte[] continuation, @Nullable ScanProperties scanProperties)
Validate the integrity of the index (such as identifying index entries that do not point to records or identifying records that do not point to valid index entries). The default implementation provided by theStandardIndexMaintainer
class is a no-op (performs no validation) and should be overridden by implementing classes.- Specified by:
validateEntries
in classIndexMaintainer
- Parameters:
continuation
- any continuation from a previous validation invocationscanProperties
- skip, limit and other properties of the validation (use default values ifnull
)- Returns:
- a cursor over invalid index entries including reasons (the default is an empty cursor)
-
validateOrphanEntries
@Nonnull protected RecordCursor<InvalidIndexEntry> validateOrphanEntries(@Nullable byte[] continuation, @Nonnull ScanProperties scanProperties)
Validate entries in the index. It scans the index and checks if the record associated with each index entry exists.- Parameters:
continuation
- any continuation from a previous validation invocationscanProperties
- skip, limit and other properties of the validation- Returns:
- a cursor over index entries that have no associated records including the reason
-
validateMissingEntries
@Nonnull protected RecordCursor<InvalidIndexEntry> validateMissingEntries(@Nullable byte[] continuation, @Nonnull ScanProperties scanProperties)
Validate entries in the index. It scans the records and checks if the index entries associated with each record exist. Note that it may not work for indexes on synthetic record types (e.g., join indexes).- Parameters:
continuation
- any continuation from a previous validation invocationscanProperties
- skip, limit and other properties of the validation- Returns:
- a cursor over records that have no associated index entries including the reason
-
checkKeyValueSizes
protected <M extends Message> void checkKeyValueSizes(@Nonnull FDBIndexableRecord<M> savedRecord, @Nonnull Tuple valueKey, @Nonnull Tuple value, @Nonnull byte[] keyBytes, @Nonnull byte[] valueBytes)
-
indexEntryKey
@Nonnull protected Tuple indexEntryKey(@Nonnull Tuple valueKey, @Nonnull Tuple primaryKey)
The entire index key to be used, including both the indexed value(s) and the primary key(s), with redundancy removed.- Parameters:
valueKey
- the indexed value(s) for the entryprimaryKey
- the primary key for the record- Returns:
- the key to use for an index entry
-
saveIndexEntryAsKeyValue
protected void saveIndexEntryAsKeyValue(IndexEntry keyValue)
Manually save an index entry, for example when rebuilding in place with a different storage format. Does not check uniqueness or maintain any secondary subspaces.- Parameters:
keyValue
- the entry to save
-
canEvaluateRecordFunction
public boolean canEvaluateRecordFunction(@Nonnull IndexRecordFunction<?> function)
Description copied from class:IndexMaintainer
Returntrue
if this index be used to evaluate the given record function.- Specified by:
canEvaluateRecordFunction
in classIndexMaintainer
- Parameters:
function
- requested function- Returns:
true
if this index can be used to evaluate the given function
-
evaluateRecordFunction
@Nonnull public <T,M extends Message> CompletableFuture<T> evaluateRecordFunction(@Nonnull EvaluationContext context, @Nonnull IndexRecordFunction<T> function, @Nonnull FDBRecord<M> record)
Description copied from class:IndexMaintainer
Evaluate a record function on the given record.- Specified by:
evaluateRecordFunction
in classIndexMaintainer
- Type Parameters:
T
- the result type of the functionM
- the message type of the record- Parameters:
context
- context for evaluationfunction
- the record function to apply to the given recordrecord
- record against which to evaluate- Returns:
- a future that completes with the result of evaluation
-
canEvaluateAggregateFunction
public boolean canEvaluateAggregateFunction(@Nonnull IndexAggregateFunction function)
Description copied from class:IndexMaintainer
Get whether this index can be used to evaluate the given aggregate function.- Specified by:
canEvaluateAggregateFunction
in classIndexMaintainer
- Parameters:
function
- the requested aggregate function- Returns:
true
if this index be used to evaluate the given aggregate function
-
evaluateAggregateFunction
@Nonnull public CompletableFuture<Tuple> evaluateAggregateFunction(@Nonnull IndexAggregateFunction function, @Nonnull TupleRange range, @Nonnull IsolationLevel isolationLevel)
Description copied from class:IndexMaintainer
Evaluate an aggregate function over the given range using this index.- Specified by:
evaluateAggregateFunction
in classIndexMaintainer
- Parameters:
function
- the aggregate function to evaluaterange
- the range over which to accumulate the aggregateisolationLevel
- the isolation level at which to perform the scan- Returns:
- a future that completes with the aggregate result
-
getGroupingCount
protected int getGroupingCount()
-
getGroupedCount
protected int getGroupedCount()
-
isIdempotent
public boolean isIdempotent()
Description copied from class:IndexMaintainer
Whether updating or removing a record on this index is idempotent. In principle, all index updates in the normal case are idempotent as long as the index update and the record insertion or deletion is atomic. However, certain indexes (mostly aggregate indexes) have the property that the index update on its own are not idempotent.- Specified by:
isIdempotent
in classIndexMaintainer
- Returns:
- whether updating this index is idempotent
-
addedRangeWithKey
@Nonnull public CompletableFuture<Boolean> addedRangeWithKey(@Nonnull Tuple primaryKey)
Description copied from class:IndexMaintainer
Whether this key has been added to some range within theRangeSet
associated with this index. This is used within the context of seeing if one should update a non-idempotent write-only index with a new key. If the key is in some range, then it means that one should update the index as it is based off of stale data. If the key is not in some range, then it means that one should not update the index, as the rebuild job will handle it later.- Specified by:
addedRangeWithKey
in classIndexMaintainer
- Parameters:
primaryKey
- the record key of the record to check- Returns:
- a future that will be
true
if some range contains the record andfalse
otherwise
-
canDeleteWhere
protected static boolean canDeleteWhere(@Nonnull IndexMaintainerState state, @Nonnull QueryToKeyMatcher.Match match, @Nonnull Key.Evaluated evaluated)
-
canDeleteWhere
public boolean canDeleteWhere(@Nonnull QueryToKeyMatcher matcher, @Nonnull Key.Evaluated evaluated)
Description copied from class:IndexMaintainer
Get whether this index scan delete records matching a particular key query.- Specified by:
canDeleteWhere
in classIndexMaintainer
- Parameters:
matcher
- the key queryevaluated
- parameters to the key query- Returns:
true
if this index accommodate awhereRecordsWhere
-
deleteWhere
public CompletableFuture<Void> deleteWhere(Transaction tr, @Nonnull Tuple prefix)
Description copied from class:IndexMaintainer
Clear index storage associated with the given key prefix.- Specified by:
deleteWhere
in classIndexMaintainer
- Parameters:
tr
- transaction in which to access the databaseprefix
- prefix of primary key to clear- Returns:
- a future that is complete when the given prefix has been cleared from this index
-
performOperation
public CompletableFuture<IndexOperationResult> performOperation(@Nonnull IndexOperation operation)
Description copied from class:IndexMaintainer
Perform a type-specific operation on index. Allowed operations will vary by index type.- Specified by:
performOperation
in classIndexMaintainer
- Parameters:
operation
- the requested operation- Returns:
- a future that completes with the result of the operation
-
evaluateIndex
@Nonnull protected <M extends Message> List<IndexEntry> evaluateIndex(@Nonnull FDBRecord<M> record)
Apply the key and value expressions to arecord
.- Type Parameters:
M
- the message type of the record- Parameters:
record
- the record from which the index will extract its key and value- Returns:
- a list of index keys and values
-
-