public interface ColumnFamilyOptionsInterface
Modifier and Type | Field and Description |
---|---|
static long |
DEFAULT_COMPACTION_MEMTABLE_MEMORY_BUDGET
Default memtable memory budget used with the following methods:
optimizeLevelStyleCompaction()
optimizeUniversalStyleCompaction()
|
Modifier and Type | Method and Description |
---|---|
int |
bloomLocality()
Control locality of bloom filter probes to improve cache miss rate.
|
CompactionStyle |
compactionStyle()
Compaction style for DB.
|
java.util.List<CompressionType> |
compressionPerLevel()
Return the currently set
CompressionType
per instances. |
CompressionType |
compressionType()
Compress blocks using the specified compression algorithm.
|
double |
hardRateLimit()
Puts are delayed 1ms at a time when any level has a compaction score that
exceeds hard_rate_limit.
|
boolean |
inplaceUpdateSupport()
Allows thread-safe inplace updates.
|
boolean |
levelCompactionDynamicLevelBytes()
Return if
LevelCompactionDynamicLevelBytes is enabled. |
int |
levelZeroFileNumCompactionTrigger()
The number of files in level 0 to trigger compaction from level-0 to
level-1.
|
int |
levelZeroSlowdownWritesTrigger()
Soft limit on the number of level-0 files.
|
int |
levelZeroStopWritesTrigger()
Maximum number of level-0 files.
|
double |
maxBytesForLevelMultiplier()
The ratio between the total size of level-(L+1) files and the total
size of level-L files for all L.
|
long |
maxCompactionBytes()
Control maximum size of each compaction (not guaranteed)
|
int |
maxMemCompactionLevel()
Deprecated.
|
long |
maxTableFilesSizeFIFO()
FIFO compaction option.
|
java.lang.String |
memTableFactoryName()
Returns the name of the current mem table representation.
|
int |
minPartialMergeOperands()
The number of partial merge operands to accumulate before partial
merge will be performed.
|
int |
minWriteBufferNumberToMerge()
The minimum number of write buffers that will be merged together
before writing to storage.
|
int |
numLevels()
If level-styled compaction is used, then this number determines
the total number of levels.
|
boolean |
optimizeFiltersForHits()
Returns the current state of the
optimize_filters_for_hits
setting. |
ColumnFamilyOptionsInterface |
optimizeForPointLookup(long blockCacheSizeMb)
Use this if you don't need to keep the data sorted, i.e.
|
ColumnFamilyOptionsInterface |
optimizeLevelStyleCompaction()
Default values for some parameters in ColumnFamilyOptions are not
optimized for heavy workloads and big datasets, which means you might
observe write stalls under some conditions.
|
java.lang.Object |
optimizeLevelStyleCompaction(long memtableMemoryBudget)
Default values for some parameters in ColumnFamilyOptions are not
optimized for heavy workloads and big datasets, which means you might
observe write stalls under some conditions.
|
java.lang.Object |
optimizeUniversalStyleCompaction()
Default values for some parameters in ColumnFamilyOptions are not
optimized for heavy workloads and big datasets, which means you might
observe write stalls under some conditions.
|
java.lang.Object |
optimizeUniversalStyleCompaction(long memtableMemoryBudget)
Default values for some parameters in ColumnFamilyOptions are not
optimized for heavy workloads and big datasets, which means you might
observe write stalls under some conditions.
|
boolean |
purgeRedundantKvsWhileFlush()
Purge duplicate/deleted keys when a memtable is flushed to storage.
|
int |
rateLimitDelayMaxMilliseconds()
The maximum time interval a put will be stalled when hard_rate_limit
is enforced.
|
java.lang.Object |
setBloomLocality(int bloomLocality)
Control locality of bloom filter probes to improve cache miss rate.
|
java.lang.Object |
setCompactionStyle(CompactionStyle compactionStyle)
Set compaction style for DB.
|
java.lang.Object |
setComparator(AbstractComparator<? extends AbstractSlice<?>> comparator)
Use the specified comparator for key ordering.
|
java.lang.Object |
setComparator(BuiltinComparator builtinComparator)
Set
BuiltinComparator to be used with RocksDB. |
java.lang.Object |
setCompressionPerLevel(java.util.List<CompressionType> compressionLevels)
Different levels can have different compression
policies.
|
java.lang.Object |
setCompressionType(CompressionType compressionType)
Compress blocks using the specified compression algorithm.
|
java.lang.Object |
setHardRateLimit(double hardRateLimit)
Puts are delayed 1ms at a time when any level has a compaction score that
exceeds hard_rate_limit.
|
java.lang.Object |
setInplaceUpdateSupport(boolean inplaceUpdateSupport)
Allows thread-safe inplace updates.
|
java.lang.Object |
setLevelCompactionDynamicLevelBytes(boolean enableLevelCompactionDynamicLevelBytes)
If
true , RocksDB will pick target size of each level
dynamically. |
java.lang.Object |
setLevelZeroFileNumCompactionTrigger(int numFiles)
Number of files to trigger level-0 compaction.
|
java.lang.Object |
setLevelZeroSlowdownWritesTrigger(int numFiles)
Soft limit on number of level-0 files.
|
java.lang.Object |
setLevelZeroStopWritesTrigger(int numFiles)
Maximum number of level-0 files.
|
java.lang.Object |
setMaxBytesForLevelMultiplier(double multiplier)
The ratio between the total size of level-(L+1) files and the total
size of level-L files for all L.
|
java.lang.Object |
setMaxCompactionBytes(long maxCompactionBytes)
Maximum size of each compaction (not guarantee)
|
java.lang.Object |
setMaxMemCompactionLevel(int maxMemCompactionLevel)
Deprecated.
|
java.lang.Object |
setMaxTableFilesSizeFIFO(long maxTableFilesSize)
FIFO compaction option.
|
java.lang.Object |
setMemTableConfig(MemTableConfig config)
Set the config for mem-table.
|
java.lang.Object |
setMergeOperator(MergeOperator mergeOperator)
Set the merge operator to be used for merging two different key/value
pairs that share the same key.
|
java.lang.Object |
setMergeOperatorName(java.lang.String name)
Set the merge operator to be used for merging two merge operands
of the same key.
|
java.lang.Object |
setMinPartialMergeOperands(int minPartialMergeOperands)
The number of partial merge operands to accumulate before partial
merge will be performed.
|
java.lang.Object |
setMinWriteBufferNumberToMerge(int minWriteBufferNumberToMerge)
The minimum number of write buffers that will be merged together
before writing to storage.
|
java.lang.Object |
setNumLevels(int numLevels)
Set the number of levels for this database
If level-styled compaction is used, then this number determines
the total number of levels.
|
java.lang.Object |
setOptimizeFiltersForHits(boolean optimizeFiltersForHits)
This flag specifies that the implementation should optimize the filters
mainly for cases where keys are found rather than also optimize for keys
missed.
|
java.lang.Object |
setPurgeRedundantKvsWhileFlush(boolean purgeRedundantKvsWhileFlush)
Purge duplicate/deleted keys when a memtable is flushed to storage.
|
java.lang.Object |
setRateLimitDelayMaxMilliseconds(int rateLimitDelayMaxMilliseconds)
The maximum time interval a put will be stalled when hard_rate_limit
is enforced.
|
java.lang.Object |
setSoftRateLimit(double softRateLimit)
Puts are delayed 0-1 ms when any level has a compaction score that exceeds
soft_rate_limit.
|
java.lang.Object |
setTableFormatConfig(TableFormatConfig config)
Set the config for table format.
|
double |
softRateLimit()
Puts are delayed 0-1 ms when any level has a compaction score that exceeds
soft_rate_limit.
|
java.lang.String |
tableFactoryName() |
java.lang.Object |
useCappedPrefixExtractor(int n)
Same as fixed length prefix extractor, except that when slice is
shorter than the fixed length, it will use the full key.
|
java.lang.Object |
useFixedLengthPrefixExtractor(int n)
This prefix-extractor uses the first n bytes of a key as its prefix.
|
static final long DEFAULT_COMPACTION_MEMTABLE_MEMORY_BUDGET
ColumnFamilyOptionsInterface optimizeForPointLookup(long blockCacheSizeMb)
blockCacheSizeMb
- Block cache size in MBColumnFamilyOptionsInterface optimizeLevelStyleCompaction()
Default values for some parameters in ColumnFamilyOptions are not optimized for heavy workloads and big datasets, which means you might observe write stalls under some conditions. As a starting point for tuning RocksDB options, use the following for level style compaction.
Make sure to also call IncreaseParallelism(), which will provide the biggest performance gains.
Note: we might use more memory than memtable_memory_budget during high write rate period
java.lang.Object optimizeLevelStyleCompaction(long memtableMemoryBudget)
Default values for some parameters in ColumnFamilyOptions are not optimized for heavy workloads and big datasets, which means you might observe write stalls under some conditions. As a starting point for tuning RocksDB options, use the following for level style compaction.
Make sure to also call IncreaseParallelism(), which will provide the biggest performance gains.
Note: we might use more memory than memtable_memory_budget during high write rate period
memtableMemoryBudget
- memory budget in bytesjava.lang.Object optimizeUniversalStyleCompaction()
Default values for some parameters in ColumnFamilyOptions are not optimized for heavy workloads and big datasets, which means you might observe write stalls under some conditions. As a starting point for tuning RocksDB options, use the following for universal style compaction.
Universal style compaction is focused on reducing Write Amplification Factor for big data sets, but increases Space Amplification.
Make sure to also call IncreaseParallelism(), which will provide the biggest performance gains.
Note: we might use more memory than memtable_memory_budget during high write rate period
java.lang.Object optimizeUniversalStyleCompaction(long memtableMemoryBudget)
Default values for some parameters in ColumnFamilyOptions are not optimized for heavy workloads and big datasets, which means you might observe write stalls under some conditions. As a starting point for tuning RocksDB options, use the following for universal style compaction.
Universal style compaction is focused on reducing Write Amplification Factor for big data sets, but increases Space Amplification.
Make sure to also call IncreaseParallelism(), which will provide the biggest performance gains.
Note: we might use more memory than memtable_memory_budget during high write rate period
memtableMemoryBudget
- memory budget in bytesjava.lang.Object setComparator(BuiltinComparator builtinComparator)
BuiltinComparator
to be used with RocksDB.
Note: Comparator can be set once upon database creation.
Default: BytewiseComparator.builtinComparator
- a BuiltinComparator
type.java.lang.Object setComparator(AbstractComparator<? extends AbstractSlice<?>> comparator)
comparator
- java instance.java.lang.Object setMergeOperatorName(java.lang.String name)
Set the merge operator to be used for merging two merge operands of the same key. The merge function is invoked during compaction and at lookup time, if multiple key/value pairs belonging to the same key are found in the database.
name
- the name of the merge function, as defined by
the MergeOperators factory (see utilities/MergeOperators.h)
The merge function is specified by name and must be one of the
standard merge operators provided by RocksDB. The available
operators are "put", "uint64add", "stringappend" and "stringappendtest".java.lang.Object setMergeOperator(MergeOperator mergeOperator)
Set the merge operator to be used for merging two different key/value pairs that share the same key. The merge function is invoked during compaction and at lookup time, if multiple key/value pairs belonging to the same key are found in the database.
mergeOperator
- MergeOperator
instance.java.lang.Object setMinWriteBufferNumberToMerge(int minWriteBufferNumberToMerge)
minWriteBufferNumberToMerge
- the minimum number of write buffers
that will be merged together.int minWriteBufferNumberToMerge()
java.lang.Object useFixedLengthPrefixExtractor(int n)
n
- use the first n bytes of a key as its prefix.java.lang.Object useCappedPrefixExtractor(int n)
n
- use the first n bytes of a key as its prefix.java.lang.Object setCompressionType(CompressionType compressionType)
compressionType
- Compression Type.CompressionType compressionType()
java.lang.Object setCompressionPerLevel(java.util.List<CompressionType> compressionLevels)
Different levels can have different compression policies. There are cases where most lower levels would like to use quick compression algorithms while the higher levels (which have more data) use compression algorithms that have better compression but could be slower. This array, if non-empty, should have an entry for each level of the database; these override the value specified in the previous field 'compression'.
NOTICEIf level_compaction_dynamic_level_bytes=true
,
compression_per_level[0]
still determines L0
,
but other elements of the array are based on base level
(the level L0
files are merged to), and may not
match the level users see from info log for metadata.
If L0
files are merged to level - n
,
then, for i>0
, compression_per_level[i]
determines compaction type for level n+i-1
.
For example, if we have 5 levels, and we determine to
merge L0
data to L4
(which means L1..L3
will be empty), then the new files go to L4
uses
compression type compression_per_level[1]
.
If now L0
is merged to L2
. Data goes to
L2
will be compressed according to
compression_per_level[1]
, L3
using
compression_per_level[2]
and L4
using
compression_per_level[3]
. Compaction for each
level can change when data grows.
Default: empty
compressionLevels
- list of
CompressionType
instances.java.util.List<CompressionType> compressionPerLevel()
Return the currently set CompressionType
per instances.
CompressionType
instances.java.lang.Object setNumLevels(int numLevels)
numLevels
- the number of levels.int numLevels()
java.lang.Object setLevelZeroFileNumCompactionTrigger(int numFiles)
numFiles
- the number of files in level-0 to trigger compaction.int levelZeroFileNumCompactionTrigger()
java.lang.Object setLevelZeroSlowdownWritesTrigger(int numFiles)
numFiles
- soft limit on number of level-0 files.int levelZeroSlowdownWritesTrigger()
java.lang.Object setLevelZeroStopWritesTrigger(int numFiles)
numFiles
- the hard limit of the number of level-0 files.int levelZeroStopWritesTrigger()
@Deprecated java.lang.Object setMaxMemCompactionLevel(int maxMemCompactionLevel)
maxMemCompactionLevel
- Unused.@Deprecated int maxMemCompactionLevel()
java.lang.Object setLevelCompactionDynamicLevelBytes(boolean enableLevelCompactionDynamicLevelBytes)
If true
, RocksDB will pick target size of each level
dynamically. We will pick a base level b >= 1. L0 will be
directly merged into level b, instead of always into level 1.
Level 1 to b-1 need to be empty. We try to pick b and its target
size so that
At the same time max_bytes_for_level_multiplier and max_bytes_for_level_multiplier_additional are still satisfied.
With this option on, from an empty DB, we make last level the base
level, which means merging L0 data into the last level, until it exceeds
max_bytes_for_level_base. And then we make the second last level to be
base level, to start to merge L0 data to second last level, with its
target size to be 1/max_bytes_for_level_multiplier
of the last
levels extra size. After the data accumulates more so that we need to
move the base level to the third last one, and so on.
For example, assume max_bytes_for_level_multiplier=10
,
num_levels=6
, and max_bytes_for_level_base=10MB
.
Target sizes of level 1 to 5 starts with:
[- - - - 10MB]
with base level is level. Target sizes of level 1 to 4 are not applicable because they will not be used. Until the size of Level 5 grows to more than 10MB, say 11MB, we make base target to level 4 and now the targets looks like:
[- - - 1.1MB 11MB]
While data are accumulated, size targets are tuned based on actual data of level 5. When level 5 has 50MB of data, the target is like:
[- - - 5MB 50MB]
Until level 5's actual size is more than 100MB, say 101MB. Now if we keep level 4 to be the base level, its target size needs to be 10.1MB, which doesn't satisfy the target size range. So now we make level 3 the target size and the target sizes of the levels look like:
[- - 1.01MB 10.1MB 101MB]
In the same way, while level 5 further grows, all levels' targets grow, like
[- - 5MB 50MB 500MB]
Until level 5 exceeds 1000MB and becomes 1001MB, we make level 2 the base level and make levels' target sizes like this:
[- 1.001MB 10.01MB 100.1MB 1001MB]
and go on...
By doing it, we give max_bytes_for_level_multiplier
a priority
against max_bytes_for_level_base
, for a more predictable LSM tree
shape. It is useful to limit worse case space amplification.
max_bytes_for_level_multiplier_additional
is ignored with
this flag on.
Turning this feature on or off for an existing DB can cause unexpected LSM tree structure so it's not recommended.
Caution: this option is experimental
Default: false
enableLevelCompactionDynamicLevelBytes
- boolean value indicating
if LevelCompactionDynamicLevelBytes
shall be enabled.boolean levelCompactionDynamicLevelBytes()
Return if LevelCompactionDynamicLevelBytes
is enabled.
For further information see
setLevelCompactionDynamicLevelBytes(boolean)
levelCompactionDynamicLevelBytes
is enabled.java.lang.Object setMaxBytesForLevelMultiplier(double multiplier)
multiplier
- the ratio between the total size of level-(L+1)
files and the total size of level-L files for all L.double maxBytesForLevelMultiplier()
java.lang.Object setMaxCompactionBytes(long maxCompactionBytes)
maxCompactionBytes
- the compaction size limitlong maxCompactionBytes()
java.lang.Object setSoftRateLimit(double softRateLimit)
softRateLimit
- the soft-rate-limit of a compaction score
for put delay.double softRateLimit()
java.lang.Object setHardRateLimit(double hardRateLimit)
hardRateLimit
- the hard-rate-limit of a compaction score for put
delay.double hardRateLimit()
java.lang.Object setRateLimitDelayMaxMilliseconds(int rateLimitDelayMaxMilliseconds)
rateLimitDelayMaxMilliseconds
- the maximum time interval a put
will be stalled.int rateLimitDelayMaxMilliseconds()
java.lang.Object setPurgeRedundantKvsWhileFlush(boolean purgeRedundantKvsWhileFlush)
purgeRedundantKvsWhileFlush
- true if purging keys is disabled.boolean purgeRedundantKvsWhileFlush()
java.lang.Object setCompactionStyle(CompactionStyle compactionStyle)
compactionStyle
- Compaction style.CompactionStyle compactionStyle()
java.lang.Object setMaxTableFilesSizeFIFO(long maxTableFilesSize)
maxTableFilesSize
- the size limit of the total sum of table files.long maxTableFilesSizeFIFO()
java.lang.Object setMemTableConfig(MemTableConfig config)
config
- the mem-table config.java.lang.IllegalArgumentException
- thrown on 32-Bit platforms
while overflowing the underlying platform specific value.java.lang.String memTableFactoryName()
setTableFormatConfig(org.rocksdb.TableFormatConfig)
java.lang.Object setTableFormatConfig(TableFormatConfig config)
config
- the table format config.java.lang.String tableFactoryName()
java.lang.Object setInplaceUpdateSupport(boolean inplaceUpdateSupport)
inplaceUpdateSupport
- true if thread-safe inplace updates
are allowed.boolean inplaceUpdateSupport()
java.lang.Object setBloomLocality(int bloomLocality)
bloomLocality
- the level of locality of bloom-filter probes.int bloomLocality()
setBloomLocality(int)
java.lang.Object setMinPartialMergeOperands(int minPartialMergeOperands)
minPartialMergeOperands
- min partial merge operandsint minPartialMergeOperands()
java.lang.Object setOptimizeFiltersForHits(boolean optimizeFiltersForHits)
This flag specifies that the implementation should optimize the filters mainly for cases where keys are found rather than also optimize for keys missed. This would be used in cases where the application knows that there are very few misses or the performance in the case of misses is not important.
For now, this flag allows us to not store filters for the last level i.e the largest level which contains data of the LSM store. For keys which are hits, the filters in this level are not useful because we will search for the data anyway.
NOTE: the filters in other levels are still useful even for key hit because they tell us whether to look in that level or go to the higher level.
Default: false
optimizeFiltersForHits
- boolean value indicating if this flag is set.boolean optimizeFiltersForHits()
Returns the current state of the optimize_filters_for_hits
setting.
optimize_filters_for_hits
was set.