@API(value=UNSTABLE) public static class OnlineIndexer.Builder extends Object
OnlineIndexer
.
OnlineIndexer.newBuilder().setRecordStoreBuilder(recordStoreBuilder).setIndex(index).build()
OnlineIndexer.newBuilder().setDatabase(fdb).setMetaData(metaData).setSubspace(subspace).setIndex(index).build()
Modifier and Type | Field and Description |
---|---|
protected Function<OnlineIndexer.Config,OnlineIndexer.Config> |
configLoader |
protected Index |
index |
protected int |
limit |
protected int |
maxRetries |
protected int |
recordsPerSecond |
protected FDBRecordStore.Builder |
recordStoreBuilder |
protected Collection<RecordType> |
recordTypes |
protected FDBDatabaseRunner |
runner |
protected boolean |
syntheticIndex |
Modifier | Constructor and Description |
---|---|
protected |
Builder() |
Modifier and Type | Method and Description |
---|---|
OnlineIndexer |
build()
Build an
OnlineIndexer . |
Function<OnlineIndexer.Config,OnlineIndexer.Config> |
getConfigLoader()
Get the function used by the online indexer to load the config parameters on fly.
|
int |
getIncreaseLimitAfter()
Get the number of successful range builds before re-increasing the number of records to process in a single
transaction.
|
Index |
getIndex()
Get the index to be built.
|
long |
getInitialDelayMillis()
Get the initial delay between transaction retry attempts.
|
int |
getLimit()
Get the maximum number of records to process in one transaction.
|
int |
getMaxAttempts()
Get the maximum number of transaction retry attempts.
|
long |
getMaxDelayMillis()
Get the maximum delay between transaction retry attempts.
|
int |
getMaxRetries()
Get the maximum number of times to retry a single range rebuild.
|
Map<String,String> |
getMdcContext()
Get the logging context used in
OnlineIndexer.buildIndex(boolean) . |
FDBTransactionPriority |
getPriority()
Get the priority of transactions used for this index build.
|
long |
getProgressLogIntervalMillis()
Get the minimum time between successful progress logs when building across transactions.
|
int |
getRecordsPerSecond()
Get the maximum number of records to process in a single second.
|
FDBRecordStore.Builder |
getRecordStoreBuilder()
Get the record store builder that will be used to open record store instances for indexing.
|
Collection<RecordType> |
getRecordTypes()
Get the explicit set of record types to be indexed.
|
FDBDatabaseRunner |
getRunner()
Get the runner that will be used to call into the database.
|
FDBStoreTimer |
getTimer()
Get the timer used in
OnlineIndexer.buildIndex(boolean) . |
FDBDatabase.WeakReadSemantics |
getWeakReadSemantics()
Get the acceptable staleness bounds for transactions used by this build.
|
OnlineIndexer.Builder |
setConfigLoader(Function<OnlineIndexer.Config,OnlineIndexer.Config> configLoader)
Set the function used by the online indexer to load the mutable configuration parameters on fly.
|
OnlineIndexer.Builder |
setDatabase(FDBDatabase database)
Set the database in which to run the indexing.
|
OnlineIndexer.Builder |
setFormatVersion(int formatVersion)
Set the store format version to use while building the index.
|
OnlineIndexer.Builder |
setIncreaseLimitAfter(int increaseLimitAfter)
Set the number of successful range builds before re-increasing the number of records to process in a single
transaction.
|
OnlineIndexer.Builder |
setIndex(Index index)
Set the index to be built.
|
OnlineIndexer.Builder |
setIndex(String indexName)
Set the index to be built.
|
OnlineIndexer.Builder |
setIndexMaintenanceFilter(IndexMaintenanceFilter indexMaintenanceFilter)
Set the
IndexMaintenanceFilter to use while building the index. |
OnlineIndexer.Builder |
setInitialDelayMillis(long initialDelayMillis)
Set the initial delay between transaction retry attempts.
|
OnlineIndexer.Builder |
setLimit(int limit)
Set the maximum number of records to process in one transaction.
|
OnlineIndexer.Builder |
setMaxAttempts(int maxAttempts)
Set the maximum number of transaction retry attempts.
|
OnlineIndexer.Builder |
setMaxDelayMillis(long maxDelayMillis)
Set the maximum delay between transaction retry attempts.
|
OnlineIndexer.Builder |
setMaxRetries(int maxRetries)
Set the maximum number of times to retry a single range rebuild.
|
OnlineIndexer.Builder |
setMdcContext(Map<String,String> mdcContext)
Set the logging context used in
OnlineIndexer.buildIndex(boolean) . |
OnlineIndexer.Builder |
setMetaData(RecordMetaDataProvider metaDataProvider)
Set the meta-data to use when indexing.
|
OnlineIndexer.Builder |
setPriority(FDBTransactionPriority priority)
Set the priority of transactions used for this index build.
|
OnlineIndexer.Builder |
setProgressLogIntervalMillis(long millis)
Set the minimum time between successful progress logs when building across transactions.
|
OnlineIndexer.Builder |
setRecordsPerSecond(int recordsPerSecond)
Set the maximum number of records to process in a single second.
|
OnlineIndexer.Builder |
setRecordStore(FDBRecordStore recordStore)
Set the record store that will be used as a template to open record store instances for indexing.
|
OnlineIndexer.Builder |
setRecordStoreBuilder(FDBRecordStore.Builder recordStoreBuilder)
Set the record store builder that will be used to open record store instances for indexing.
|
OnlineIndexer.Builder |
setRecordTypes(Collection<RecordType> recordTypes)
Set the explicit set of record types to be indexed.
|
OnlineIndexer.Builder |
setRunner(FDBDatabaseRunner runner)
Set the runner that will be used to call into the database.
|
OnlineIndexer.Builder |
setSerializer(RecordSerializer<Message> serializer)
Set the
RecordSerializer to use while building the index. |
OnlineIndexer.Builder |
setSubspace(Subspace subspace)
Set the subspace of the record store in which to build the index.
|
OnlineIndexer.Builder |
setSubspaceProvider(SubspaceProvider subspaceProvider)
Set the subspace of the record store in which to build the index.
|
OnlineIndexer.Builder |
setTimer(FDBStoreTimer timer)
Set the timer used in
OnlineIndexer.buildIndex(boolean) . |
OnlineIndexer.Builder |
setWeakReadSemantics(FDBDatabase.WeakReadSemantics weakReadSemantics)
Set the acceptable staleness bounds for transactions used by this build.
|
protected void |
validate() |
@Nullable protected FDBDatabaseRunner runner
@Nullable protected FDBRecordStore.Builder recordStoreBuilder
@Nullable protected Collection<RecordType> recordTypes
@Nonnull protected Function<OnlineIndexer.Config,OnlineIndexer.Config> configLoader
protected int limit
protected int maxRetries
protected int recordsPerSecond
protected boolean syntheticIndex
@Nullable public FDBDatabaseRunner getRunner()
public OnlineIndexer.Builder setRunner(@Nullable FDBDatabaseRunner runner)
setDatabase(com.apple.foundationdb.record.provider.foundationdb.FDBDatabase)
or setRecordStore(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore)
or setRecordStoreBuilder(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore.Builder)
.runner
- the runner that connects to the target databasepublic OnlineIndexer.Builder setDatabase(@Nonnull FDBDatabase database)
setRecordStore(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore)
or setRecordStoreBuilder(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore.Builder)
.database
- the target database@Nullable public FDBRecordStore.Builder getRecordStoreBuilder()
public OnlineIndexer.Builder setRecordStoreBuilder(@Nonnull FDBRecordStore.Builder recordStoreBuilder)
recordStoreBuilder
- the record store buildersetRecordStore(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore)
public OnlineIndexer.Builder setRecordStore(@Nonnull FDBRecordStore recordStore)
recordStore
- the target record store@Nullable public Index getIndex()
@Nonnull public OnlineIndexer.Builder setIndex(@Nullable Index index)
index
- the index to be built@Nonnull public OnlineIndexer.Builder setIndex(@Nonnull String indexName)
indexName
- the index to be built@Nullable public Collection<RecordType> getRecordTypes()
@Nonnull public OnlineIndexer.Builder setRecordTypes(@Nullable Collection<RecordType> recordTypes)
setIndex(com.apple.foundationdb.record.metadata.Index)
.recordTypes
- the record types to be indexed or null
to infer from the index@Nullable public Function<OnlineIndexer.Config,OnlineIndexer.Config> getConfigLoader()
@Nonnull public OnlineIndexer.Builder setConfigLoader(@Nonnull Function<OnlineIndexer.Config,OnlineIndexer.Config> configLoader)
The loader is given the current configuration as input at the beginning of each transaction and should produce the configuration to use in the next transaction.
configLoader
- the functionpublic int getLimit()
@Nonnull public OnlineIndexer.Builder setLimit(int limit)
OnlineIndexer.DEFAULT_LIMIT
= .
Note setConfigLoader(Function)
is the recommended way of loading online index builder's parameters
and the values set by this method will be overwritten if the supplier is set.limit
- the maximum number of records to process in one transactionpublic int getMaxRetries()
getMaxAttempts()
, and it will also retry for other error
codes, such as transaction_too_large
.@Nonnull public OnlineIndexer.Builder setMaxRetries(int maxRetries)
getMaxAttempts()
, it and will also retry for other error
codes, such as transaction_too_large
.
The default number of retries is OnlineIndexer.DEFAULT_MAX_RETRIES
= .
Note setConfigLoader(Function)
is the recommended way of loading online index builder's parameters
and the values set by this method will be overwritten if the supplier is set.maxRetries
- the maximum number of times to retry a single range rebuildpublic int getRecordsPerSecond()
@Nonnull public OnlineIndexer.Builder setRecordsPerSecond(int recordsPerSecond)
OnlineIndexer.DEFAULT_RECORDS_PER_SECOND
= .
Note setConfigLoader(Function)
is the recommended way of loading online index builder's parameters
and the values set by this method will be overwritten if the supplier is set.recordsPerSecond
- the maximum number of records to process in a single second.@Nullable public FDBStoreTimer getTimer()
OnlineIndexer.buildIndex(boolean)
.null
if none is set@Nonnull public OnlineIndexer.Builder setTimer(@Nullable FDBStoreTimer timer)
OnlineIndexer.buildIndex(boolean)
.timer
- timer to use@Nullable public Map<String,String> getMdcContext()
OnlineIndexer.buildIndex(boolean)
.null
if none is set@Nonnull public OnlineIndexer.Builder setMdcContext(@Nullable Map<String,String> mdcContext)
OnlineIndexer.buildIndex(boolean)
.mdcContext
- the logging context to set while runningFDBDatabase.openContext(Map,FDBStoreTimer)
@Nullable public FDBDatabase.WeakReadSemantics getWeakReadSemantics()
null
, which indicates that the transaction should not used any cached version
at all.FDBRecordContext.getWeakReadSemantics()
@Nonnull public OnlineIndexer.Builder setWeakReadSemantics(@Nullable FDBDatabase.WeakReadSemantics weakReadSemantics)
weakReadSemantics
- the acceptable staleness bounds for transactions used by this buildFDBRecordContext.getWeakReadSemantics()
,
FDBDatabase.setTrackLastSeenVersion(boolean)
@Nonnull public FDBTransactionPriority getPriority()
FDBTransactionPriority.BATCH
.FDBRecordContext.getPriority()
@Nonnull public OnlineIndexer.Builder setPriority(@Nonnull FDBTransactionPriority priority)
BATCH
priority level as their work is generally
discretionary and not time sensitive. However, in certain circumstances, it may be
necessary to run at the higher DEFAULT
priority level.
For example, if a missing index is causing some queries to perform additional, unnecessary work that
is overwhelming the database, it may be necessary to build the index at DEFAULT
priority
in order to lessen the load induced by those queries on the cluster.priority
- the priority of transactions used for this index buildFDBRecordContext.getPriority()
public int getMaxAttempts()
RecordCoreRetriableTransactionException
.FDBDatabaseRunner.getMaxAttempts()
public OnlineIndexer.Builder setMaxAttempts(int maxAttempts)
RecordCoreRetriableTransactionException
.maxAttempts
- the maximum number of attemptsFDBDatabaseRunner.setMaxAttempts(int)
public OnlineIndexer.Builder setIncreaseLimitAfter(int increaseLimitAfter)
limit
.
By default this is OnlineIndexer.DO_NOT_RE_INCREASE_LIMIT
, which means it will not re-increase after successes.
Note setConfigLoader(Function)
is the recommended way of loading online index builder's parameters
and the values set by this method will be overwritten if the supplier is set.
increaseLimitAfter
- the number of successful range builds before increasing the number of records
processed in a single transactionpublic int getIncreaseLimitAfter()
OnlineIndexer.DO_NOT_RE_INCREASE_LIMIT
, which means it will not re-increase after successes.limit
public long getMaxDelayMillis()
FDBDatabaseRunner.getMaxDelayMillis()
public OnlineIndexer.Builder setMaxDelayMillis(long maxDelayMillis)
maxDelayMillis
- the maximum delayFDBDatabaseRunner.setMaxDelayMillis(long)
public long getInitialDelayMillis()
FDBDatabaseRunner.getInitialDelayMillis()
public OnlineIndexer.Builder setInitialDelayMillis(long initialDelayMillis)
initialDelayMillis
- the initial delayFDBDatabaseRunner.setInitialDelayMillis(long)
public long getProgressLogIntervalMillis()
for more information on the format of the log
public OnlineIndexer.Builder setProgressLogIntervalMillis(long millis)
This is the count of records scanned as part of successful transactions used by the
multi-transaction methods (e.g. OnlineIndexer.buildIndexAsync()
or
OnlineIndexer.buildRange(Key.Evaluated, Key.Evaluated)
). The transactional methods (i.e., the methods that
take a store) do not count towards this value. Since only successful transactions are included,
transactions that get commit_unknown_result
will not get counted towards this value,
so this may be short by the number of records scanned in those transactions if they actually
succeeded. In contrast, the timer count:
FDBStoreTimer.Counts.ONLINE_INDEX_BUILDER_RECORDS_SCANNED
, includes all records scanned,
regardless of whether the associated transaction was successful or not.
Note setConfigLoader(Function)
is the recommended way of loading online index builder's parameters
and the values set by this method will be overwritten if the supplier is set.
millis
- the number of milliseconds to wait between successful logspublic OnlineIndexer.Builder setIndexMaintenanceFilter(@Nonnull IndexMaintenanceFilter indexMaintenanceFilter)
IndexMaintenanceFilter
to use while building the index.
Normally this is set by setRecordStore(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore)
or setRecordStoreBuilder(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore.Builder)
.indexMaintenanceFilter
- the index filter to usepublic OnlineIndexer.Builder setSerializer(@Nonnull RecordSerializer<Message> serializer)
RecordSerializer
to use while building the index.
Normally this is set by setRecordStore(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore)
or setRecordStoreBuilder(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore.Builder)
.serializer
- the serializer to usepublic OnlineIndexer.Builder setFormatVersion(int formatVersion)
setRecordStore(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore)
or setRecordStoreBuilder(com.apple.foundationdb.record.provider.foundationdb.FDBRecordStore.Builder)
.formatVersion
- the format version to usepublic OnlineIndexer.Builder setMetaData(@Nonnull RecordMetaDataProvider metaDataProvider)
metaDataProvider
- meta-data to usepublic OnlineIndexer.Builder setSubspaceProvider(@Nonnull SubspaceProvider subspaceProvider)
subspaceProvider
- subspace to usepublic OnlineIndexer.Builder setSubspace(@Nonnull Subspace subspace)
subspace
- subspace to usepublic OnlineIndexer build()
OnlineIndexer
.protected void validate()