Class IndexingByRecords
- java.lang.Object
-
- com.apple.foundationdb.record.provider.foundationdb.IndexingBase
-
- com.apple.foundationdb.record.provider.foundationdb.IndexingByRecords
-
@API(INTERNAL) public class IndexingByRecords extends IndexingBase
This indexer scans all records in the record store.
-
-
Field Summary
-
Fields inherited from class com.apple.foundationdb.record.provider.foundationdb.IndexingBase
common
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description CompletableFuture<TupleRange>
buildEndpoints()
Builds (with a retry loop) the endpoints of an index.CompletableFuture<TupleRange>
buildEndpoints(FDBRecordStore store, AtomicLong recordsScanned)
Builds (transactionally) the endpoints of an index.CompletableFuture<Void>
buildRange(Key.Evaluated start, Key.Evaluated end)
Builds (with a retry loop) the index by adding records with primary keys within the given range.CompletableFuture<Void>
buildRange(FDBRecordStore store, Key.Evaluated start, Key.Evaluated end)
Builds (transactionally) the index by adding records with primary keys within the given range.CompletableFuture<Key.Evaluated>
buildUnbuiltRange(FDBRecordStore store, Key.Evaluated start, Key.Evaluated end)
Builds (transactionally) the index by adding records with primary keys within the given range.List<org.apache.commons.lang3.tuple.Pair<Tuple,Tuple>>
splitIndexBuildRange(int minSplit, int maxSplit)
-
Methods inherited from class com.apple.foundationdb.record.provider.foundationdb.IndexingBase
buildCommitRetryAsync, buildIndexAsync, convertOrNull, getLimit, getRunner, indexBuildLockSubspace, indexBuildScannedRecordsSubspace, indexBuildTypeSubspace, iterateRangeOnly, maybeLogBuildProgress, openRecordStore, packOrNull, rebuildIndexAsync, setFallbackMode, throttleDelay
-
-
-
-
Method Detail
-
buildEndpoints
@Nonnull public CompletableFuture<TupleRange> buildEndpoints(@Nonnull FDBRecordStore store, @Nullable AtomicLong recordsScanned)
Builds (transactionally) the endpoints of an index. What this means is that builds everything from the beginning of the key space to the first record and everything from the last record to the end of the key space. There won't be any records within these ranges (except for the last record of the record store), but it does mean that any records in the future that get added to these ranges will correctly update the index. This means, e.g., that if the workload primarily adds records to the record store after the current last record (because perhaps the primary key is based off of an atomic counter or the current time), running this method will be highly contentious, but once it completes, the rest of the index build should happen without any more conflicts. This will return a (possibly null)TupleRange
that contains the primary keys of the first and last records within the record store. This can then be used to either build the range right away or to then divy-up the remaining ranges between multiple agents working in parallel if one desires.- Parameters:
store
- the record store in which to rebuild the indexrecordsScanned
- continues counter- Returns:
- a future that will contain the range of records in the interior of the record store
-
buildEndpoints
@Nonnull public CompletableFuture<TupleRange> buildEndpoints()
Builds (with a retry loop) the endpoints of an index. See thebuildEndpoints()
method that takes anFDBRecordStore
as its parameter for more details. This will retry on that function until it gets a non-exceptional result and return the results back.- Returns:
- a future that will contain the range of records in the interior of the record store
-
buildRange
@Nonnull public CompletableFuture<Void> buildRange(@Nonnull FDBRecordStore store, @Nullable Key.Evaluated start, @Nullable Key.Evaluated end)
Builds (transactionally) the index by adding records with primary keys within the given range. This will look for gaps of keys within the given range that haven't yet been rebuilt and then rebuild only those ranges. As a result, if this method is called twice, the first time, it will build whatever needs to be built, and then the second time, it will notice that there are no ranges that need to be built, so it will do nothing. In this way, it is idempotent and thus safe to use in retry loops. This method will fail if there is too much work to be done in a single transaction. If one wants to handle building a range that does not fit in a single transaction, one should use thebuildRange()
function that takes anFDBDatabase
as its first parameter.- Parameters:
store
- the record store in which to rebuild the rangestart
- the (inclusive) beginning primary key of the range to build (ornull
to go to the end)end
- the (exclusive) end primary key of the range to build (ornull
to go to the end)- Returns:
- a future that will be ready when the build has completed
-
buildRange
@Nonnull public CompletableFuture<Void> buildRange(@Nullable Key.Evaluated start, @Nullable Key.Evaluated end)
Builds (with a retry loop) the index by adding records with primary keys within the given range. This will look for gaps of keys within the given range that haven't yet been rebuilt and then rebuild only those ranges. It will also limit each transaction to the number of records specified by thelimit
parameter of this class's constructor. In the case that that limit is too high (i.e., it can't make any progress or errors out on a non-retriable error liketransaction_too_large
, this method will actually decrease the limit so that less work is attempted each transaction. It will also rate limit itself as to not make too many requests per second.Note that it does not have the protections (synchronized sessions and index state precondition) which are imposed on buildIndexAsync() (or its variations), but it does use the created synchronized session if a buildIndexAsync() is running on the
OnlineIndexer
simultaneously or this range build is used as part of buildIndexAsync internally.- Parameters:
start
- the (inclusive) beginning primary key of the range to build (ornull
to go from the beginning)end
- the (exclusive) end primary key of the range to build (ornull
to go to the end)- Returns:
- a future that will be ready when the build has completed
-
buildUnbuiltRange
@Nonnull public CompletableFuture<Key.Evaluated> buildUnbuiltRange(@Nonnull FDBRecordStore store, @Nullable Key.Evaluated start, @Nullable Key.Evaluated end)
Builds (transactionally) the index by adding records with primary keys within the given range. This requires that the range is initially "unbuilt", i.e., no records within the given range have yet been processed by the index build job. It is acceptable if there are records within that range that have already been added to the index because they were added to the store after the index was added in write-only mode but have not yet been processed by the index build job. Note that this function is not idempotent in that if the first time this function runs, if it fails withcommit_unknown_result
but the transaction actually succeeds, running this function again will result in aOnlineIndexer.RecordBuiltRangeException
being thrown the second time. Retry loops used by theOnlineIndexer
class that call this method handle this contingency. For the most part, this method should only be used by those who know what they are doing. It is included because it is less expensive to make this call if one already knows that the range will be unbuilt, but the caller must be ready to handle the circumstance that the range might be built the second time. Most users should use thebuildRange()
method with the same parameters in the case that they want to build a range of keys into the index. That method is idempotent, but it is slightly more costly as it firsts determines what ranges are have not yet been built before building them.- Parameters:
store
- the record store in which to rebuild the rangestart
- the (inclusive) beginning primary key of the range to build (ornull
to start from the beginning)end
- the (exclusive) end primary key of the range to build (ornull
to go to the end)- Returns:
- a future with the key of the first record not processed by this range rebuild
- Throws:
OnlineIndexer.RecordBuiltRangeException
- if the given range contains keys already processed by the index build
-
-