Package com.apple.foundationdb.record.provider.foundationdb.indexes

Class StandardIndexMaintainer

    • Field Detail

      • TOO_LARGE_VALUE_MESSAGE_LIMIT

        protected static final int TOO_LARGE_VALUE_MESSAGE_LIMIT
        See Also:
        Constant Field Values

    • Constructor Detail

    • Method Detail

      • scan

        protected RecordCursor<IndexEntry> scan​(@Nonnull
                                        TupleRange range,
                                        @Nullable
                                        byte[] continuation,
                                        @Nonnull
                                        ScanProperties scanProperties)
        Scan the primary index tree for the given range.
        Parameters:
        range - range of index keys to scan
        continuation - any continuation from previous scan
        scanProperties - any limits on the scan
        Returns:
        a cursor of index entries within the given range

      • unpackKeyValue

        @Nonnull
protected IndexEntry unpackKeyValue​(@Nonnull
                                    KeyValue kv)
        Convert stored key value pair into an index entry.
        Parameters:
        kv - a raw key-value from the database
        Returns:
        an index entry

      • unpackKeyValue

        @Nonnull
protected IndexEntry unpackKeyValue​(@Nonnull
                                    Subspace subspace,
                                    @Nonnull
                                    KeyValue kv)
        Convert stored key value pair located in the given subspace into an index entry.
        Parameters:
        subspace - the database subspace for the index
        kv - a raw key-value within subspace
        Returns:
        an index entry

      • decodeValue

        @Nonnull
protected Tuple decodeValue​(@Nonnull
                            byte[] value)
        Decode value portion of key value pair.
        Parameters:
        value - the raw value portion of a key value pair
        Returns:
        a decoded tuple of any values stored in the value side of the index, which is usually empty

      • skipUpdateForUnchangedKeys

        public boolean skipUpdateForUnchangedKeys()

      • filteredIndexEntries

        @Nullable
protected <M extends Message> List<IndexEntry> filteredIndexEntries​(@Nullable
                                                                    FDBIndexableRecord<M> savedRecord)
        Filter out index keys according to IndexMaintenanceFilter. Keys that do not pass the filter will not be stored / removed from the index.
        Type Parameters:
        M - the message type of the record
        Parameters:
        savedRecord - record for key evaluation
        Returns:
        filtered list of index keys for the given record

      • updateIndexKeys

        protected <M extends Message> CompletableFuture<Void> updateIndexKeys​(@Nonnull
                                                                      FDBIndexableRecord<M> savedRecord,
                                                                      boolean remove,
                                                                      @Nonnull
                                                                      List<IndexEntry> indexEntries)
        Update index according to record keys. Often this operation returns an already completed future because there is no asynchronous work to be done.
        Type Parameters:
        M - the message type of the record
        Parameters:
        savedRecord - the record being indexed
        remove - true if removing from index
        indexEntries - the result of evaluateIndex(FDBRecord)
        Returns:
        a future completed when update is done

      • updateOneKeyAsync

        protected <M extends Message> CompletableFuture<Void> updateOneKeyAsync​(@Nonnull
                                                                        FDBIndexableRecord<M> savedRecord,
                                                                        boolean remove,
                                                                        @Nonnull
                                                                        IndexEntry indexEntry)
        Store a single key in the index.
        Type Parameters:
        M - the message type of the record
        Parameters:
        savedRecord - the record being indexed
        remove - true if removing from index
        indexEntry - the entry for the index to be updated
        Returns:
        a future completed when the key is updated

      • addUniquenessViolation

        protected void addUniquenessViolation​(@Nonnull
                                      Tuple valueKey,
                                      @Nonnull
                                      Tuple primaryKey,
                                      @Nullable
                                      Tuple existingKey)
        Add a uniqueness violation within the database. This is used to keep track of uniqueness violations that occur when an index is in write-only mode, both during the built itself and by other writes. This means that the writes will succeed, but it will cause a later attempt to make the index readable to fail.
        Parameters:
        valueKey - the indexed key that is (apparently) not unique
        primaryKey - the primary key of one record that is causing a violation
        existingKey - the primary key of another record that is causing a violation (or null if none specified)

      • removeUniquenessViolationsAsync

        @Nonnull
protected CompletableFuture<Void> removeUniquenessViolationsAsync​(@Nonnull
                                                                  Tuple valueKey,
                                                                  @Nonnull
                                                                  Tuple primaryKey)
        Remove a uniqueness violation within the database. This is used to keep track of uniqueness violations that occur when an index is in write-only mode, both during the built itself and by other writes. This means that the writes will succeed, but it will cause a later attempt to make the index readable to fail.

        This will remove the last uniqueness violation entry when removing the second last entry that contains the value key.

        Parameters:
        valueKey - the indexed key that is (apparently) not unique
        primaryKey - the primary key of one record that is causing a violation
        Returns:
        a future that is complete when the uniqueness violation is removed

      • scanUniquenessViolations

        @Nonnull
public RecordCursor<IndexEntry> scanUniquenessViolations​(@Nonnull
                                                         TupleRange range,
                                                         @Nullable
                                                         byte[] continuation,
                                                         @Nonnull
                                                         ScanProperties scanProperties)
        Description copied from class: IndexMaintainer
        Scans through the list of uniqueness violations within the database. It will return a cursor of IndexEntry instances where the getKey() will return the primary key of the record causing a problem and getValue() will return the index value that is being duplicated.
        Specified by:
        scanUniquenessViolations in class IndexMaintainer
        Parameters:
        range - range of tuples to read
        continuation - any continuation from a previous invocation
        scanProperties - row limit and other scan properties
        Returns:
        a cursor that will return primary key-index key pairs indicating uniqueness violations

      • validateEntries

        @Nonnull
public RecordCursor<InvalidIndexEntry> validateEntries​(@Nullable
                                                       byte[] continuation,
                                                       @Nullable
                                                       ScanProperties scanProperties)
        Validate the integrity of the index (such as identifying index entries that do not point to records or identifying records that do not point to valid index entries). The default implementation provided by the StandardIndexMaintainer class is a no-op (performs no validation) and should be overridden by implementing classes.
        Specified by:
        validateEntries in class IndexMaintainer
        Parameters:
        continuation - any continuation from a previous validation invocation
        scanProperties - skip, limit and other properties of the validation (use default values if null)
        Returns:
        a cursor over invalid index entries including reasons (the default is an empty cursor)

      • validateOrphanEntries

        @Nonnull
protected RecordCursor<InvalidIndexEntry> validateOrphanEntries​(@Nullable
                                                                byte[] continuation,
                                                                @Nonnull
                                                                ScanProperties scanProperties)
        Validate entries in the index. It scans the index and checks if the record associated with each index entry exists.
        Parameters:
        continuation - any continuation from a previous validation invocation
        scanProperties - skip, limit and other properties of the validation
        Returns:
        a cursor over index entries that have no associated records including the reason

      • validateMissingEntries

        @Nonnull
protected RecordCursor<InvalidIndexEntry> validateMissingEntries​(@Nullable
                                                                 byte[] continuation,
                                                                 @Nonnull
                                                                 ScanProperties scanProperties)
        Validate entries in the index. It scans the records and checks if the index entries associated with each record exist. Note that it may not work for indexes on synthetic record types (e.g., join indexes).
        Parameters:
        continuation - any continuation from a previous validation invocation
        scanProperties - skip, limit and other properties of the validation
        Returns:
        a cursor over records that have no associated index entries including the reason

      • indexEntryKey

        @Nonnull
protected Tuple indexEntryKey​(@Nonnull
                              Tuple valueKey,
                              @Nonnull
                              Tuple primaryKey)
        The entire index key to be used, including both the indexed value(s) and the primary key(s), with redundancy removed.
        Parameters:
        valueKey - the indexed value(s) for the entry
        primaryKey - the primary key for the record
        Returns:
        the key to use for an index entry

      • saveIndexEntryAsKeyValue

        protected void saveIndexEntryAsKeyValue​(IndexEntry keyValue)
        Manually save an index entry, for example when rebuilding in place with a different storage format. Does not check uniqueness or maintain any secondary subspaces.
        Parameters:
        keyValue - the entry to save

      • canEvaluateRecordFunction

        public boolean canEvaluateRecordFunction​(@Nonnull
                                         IndexRecordFunction<?> function)
        Description copied from class: IndexMaintainer
        Return true if this index be used to evaluate the given record function.
        Specified by:
        canEvaluateRecordFunction in class IndexMaintainer
        Parameters:
        function - requested function
        Returns:
        true if this index can be used to evaluate the given function

      • canEvaluateAggregateFunction

        public boolean canEvaluateAggregateFunction​(@Nonnull
                                            IndexAggregateFunction function)
        Description copied from class: IndexMaintainer
        Get whether this index can be used to evaluate the given aggregate function.
        Specified by:
        canEvaluateAggregateFunction in class IndexMaintainer
        Parameters:
        function - the requested aggregate function
        Returns:
        true if this index be used to evaluate the given aggregate function

      • getGroupingCount

        protected int getGroupingCount()

      • getGroupedCount

        protected int getGroupedCount()

      • isIdempotent

        public boolean isIdempotent()
        Description copied from class: IndexMaintainer
        Whether updating or removing a record on this index is idempotent. In principle, all index updates in the normal case are idempotent as long as the index update and the record insertion or deletion is atomic. However, certain indexes (mostly aggregate indexes) have the property that the index update on its own are not idempotent.
        Specified by:
        isIdempotent in class IndexMaintainer
        Returns:
        whether updating this index is idempotent

      • addedRangeWithKey

        @Nonnull
public CompletableFuture<Boolean> addedRangeWithKey​(@Nonnull
                                                    Tuple primaryKey)
        Description copied from class: IndexMaintainer
        Whether this key has been added to some range within the RangeSet associated with this index. This is used within the context of seeing if one should update a non-idempotent write-only index with a new key. If the key is in some range, then it means that one should update the index as it is based off of stale data. If the key is not in some range, then it means that one should not update the index, as the rebuild job will handle it later.
        Specified by:
        addedRangeWithKey in class IndexMaintainer
        Parameters:
        primaryKey - the record key of the record to check
        Returns:
        a future that will be true if some range contains the record and false otherwise

      • deleteWhere

        public CompletableFuture<Void> deleteWhere​(Transaction tr,
                                           @Nonnull
                                           Tuple prefix)
        Description copied from class: IndexMaintainer
        Clear index storage associated with the given key prefix.
        Specified by:
        deleteWhere in class IndexMaintainer
        Parameters:
        tr - transaction in which to access the database
        prefix - prefix of primary key to clear
        Returns:
        a future that is complete when the given prefix has been cleared from this index

      • evaluateIndex

        @Nonnull
protected <M extends Message> List<IndexEntry> evaluateIndex​(@Nonnull
                                                             FDBRecord<M> record)
        Apply the key and value expressions to a record.
        Type Parameters:
        M - the message type of the record
        Parameters:
        record - the record from which the index will extract its key and value
        Returns:
        a list of index keys and values