org.apache.cassandra.io.sstable.format

Class SSTableReader

java.lang.Object

org.apache.cassandra.io.sstable.SSTable

org.apache.cassandra.io.sstable.format.SSTableReader

All Implemented Interfaces:

RefCounted<SSTableReader>, SelfRefCounted<SSTableReader>

Direct Known Subclasses:

BigTableReader

public abstract class SSTableReader
extends SSTable
implements SelfRefCounted<SSTableReader>

An SSTableReader can be constructed in a number of places, but typically is either read from disk at startup, or constructed from a flushed memtable, or after compaction to replace some existing sstables. However once created, an sstablereader may also be modified. A reader's OpenReason describes its current stage in its lifecycle, as follows:

 
 NORMAL
 From:       None        => Reader has been read from disk, either at startup or from a flushed memtable
             EARLY       => Reader is the final result of a compaction
             MOVED_START => Reader WAS being compacted, but this failed and it has been restored to NORMAL status

 EARLY
 From:       None        => Reader is a compaction replacement that is either incomplete and has been opened
                            to represent its partial result status, or has been finished but the compaction
                            it is a part of has not yet completed fully
             EARLY       => Same as from None, only it is not the first time it has been

 MOVED_START
 From:       NORMAL      => Reader is being compacted. This compaction has not finished, but the compaction result
                            is either partially or fully opened, to either partially or fully replace this reader.
                            This reader's start key has been updated to represent this, so that reads only hit
                            one or the other reader.

 METADATA_CHANGE
 From:       NORMAL      => Reader has seen low traffic and the amount of memory available for index summaries is
                            constrained, so its index summary has been downsampled.
         METADATA_CHANGE => Same
  

Note that in parallel to this, there are two different Descriptor types; TMPLINK and FINAL; the latter corresponds to NORMAL state readers and all readers that replace a NORMAL one. TMPLINK is used for EARLY state readers and no others. When a reader is being compacted, if the result is large its replacement may be opened as EARLY before compaction completes in order to present the result to consumers earlier. In this case the reader will itself be changed to a MOVED_START state, where its start no longer represents its on-disk minimum key. This is to permit reads to be directed to only one reader when the two represent the same data. The EARLY file can represent a compaction result that is either partially complete and still in-progress, or a complete and immutable sstable that is part of a larger macro compaction action that has not yet fully completed. Currently ALL compaction results at least briefly go through an EARLY open state prior to completion, regardless of if early opening is enabled. Since a reader can be created multiple times over the same shared underlying resources, and the exact resources it shares between each instance differ subtly, we track the lifetime of any underlying resource with its own reference count, which each instance takes a Ref to. Each instance then tracks references to itself, and once these all expire it releases its Refs to these underlying resources. There is some shared cleanup behaviour needed only once all sstablereaders in a certain stage of their lifecycle (i.e. EARLY or NORMAL opening), and some that must only occur once all readers of any kind over a single logical sstable have expired. These are managed by the TypeTidy and GlobalTidy classes at the bottom, and are effectively managed as another resource each instance tracks its own Ref instance to, to ensure all of these resources are cleaned up safely and can be debugged otherwise. TODO: fill in details about Tracker and lifecycle interactions for tools, and for compaction strategies

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SSTableReader.Factory
static class	SSTableReader.OpenReason
static class	SSTableReader.Operator TODO: Move someplace reusable
static class	SSTableReader.SizeComparator
static class	SSTableReader.UniqueIdentifier

Nested classes/interfaces inherited from interface org.apache.cassandra.utils.concurrent.RefCounted

RefCounted.Tidy

Field Summary

Fields

Modifier and Type	Field and Description
protected IFilter	bf
protected BloomFilterTracker	bloomFilterTracker
protected SegmentedFile	dfile
SerializationHeader	header
protected SegmentedFile	ifile
protected IndexSummary	indexSummary
SSTableReader.UniqueIdentifier	instanceId
protected java.util.concurrent.atomic.AtomicBoolean	isSuspect
protected InstrumentingCache<KeyCacheKey,RowIndexEntry>	keyCache
protected java.util.concurrent.atomic.AtomicLong	keyCacheHit
protected java.util.concurrent.atomic.AtomicLong	keyCacheRequest
long	maxDataAge maxDataAge is a timestamp in local server time (e.g.
static java.util.Comparator<SSTableReader>	maxTimestampComparator
SSTableReader.OpenReason	openReason
protected RowIndexEntry.IndexSerializer	rowIndexEntrySerializer
static java.util.Comparator<SSTableReader>	sstableComparator
protected StatsMetadata	sstableMetadata
static com.google.common.collect.Ordering<SSTableReader>	sstableOrdering

Fields inherited from class org.apache.cassandra.io.sstable.SSTable

components, compression, descriptor, first, last, metadata, TOMBSTONE_HISTOGRAM_BIN_SIZE, TOMBSTONE_HISTOGRAM_SPOOL_SIZE, TOMBSTONE_HISTOGRAM_TTL_ROUND_SECONDS

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	SSTableReader(Descriptor desc, java.util.Set<Component> components, CFMetaData metadata, long maxDataAge, StatsMetadata sstableMetadata, SSTableReader.OpenReason openReason, SerializationHeader header)

Method Summary

Modifier and Type	Method and Description
void	addTo(Ref.IdentityCollection identities)
void	cacheKey(DecoratedKey key, RowIndexEntry info)
SSTableReader	cloneWithNewStart(DecoratedKey newStart, java.lang.Runnable runOnClose)
SSTableReader	cloneWithNewSummarySamplingLevel(ColumnFamilyStore parent, int samplingLevel) Returns a new SSTableReader with the same properties as this SSTableReader except that a new IndexSummary will be built at the target samplingLevel.
SSTableReader	cloneWithRestoredStart(DecoratedKey restoredStart)
void	createLinks(java.lang.String snapshotDirectoryPath)
boolean	equals(java.lang.Object that)
static double	estimateCompactionGain(java.util.Set<SSTableReader> overlapping) Estimates how much of the keys we would keep if the sstables were compacted together
long	estimatedKeys()
long	estimatedKeysForRanges(java.util.Collection<Range<Token>> ranges)
DecoratedKey	firstKeyBeyond(PartitionPosition token) Finds and returns the first key beyond a given token in this SSTable or null if no such key exists.
void	forceFilterFailures() For testing purposes only.
static long	getApproximateKeyCount(java.lang.Iterable<SSTableReader> sstables) Calculate approximate key count.
int	getAvgColumnSetPerRow()
IFilter	getBloomFilter()
long	getBloomFilterFalsePositiveCount()
long	getBloomFilterOffHeapSize() Returns the amount of memory in bytes used off heap by the bloom filter.
long	getBloomFilterSerializedSize()
long	getBloomFilterTruePositiveCount()
RowIndexEntry	getCachedPosition(DecoratedKey key, boolean updateStats)
protected RowIndexEntry	getCachedPosition(KeyCacheKey unifiedKey, boolean updateStats)
KeyCacheKey	getCacheKey(DecoratedKey key)
CompressionMetadata	getCompressionMetadata() Returns the compression metadata for this sstable.
long	getCompressionMetadataOffHeapSize() Returns the amount of memory in bytes used off heap by the compression meta-data.
double	getCompressionRatio()
double	getCrcCheckChance()
long	getCreationTimeFor(Component component)
ChannelProxy	getDataChannel()
double	getDroppableTombstonesBefore(int gcBefore)
double	getEffectiveIndexInterval()
EstimatedHistogram	getEstimatedColumnCount()
double	getEstimatedDroppableTombstoneRatio(int gcBefore)
EstimatedHistogram	getEstimatedPartitionSize()
FileDataInput	getFileDataInput(long position)
java.lang.String	getFilename()
ChannelProxy	getIndexChannel()
long	getIndexScanPosition(PartitionPosition key) Gets the position in the index file to start scanning to find the given key (at most indexInterval keys away, modulo downsampling of the index summary).
static long	getIndexScanPositionFromBinarySearchResult(int binarySearchResult, IndexSummary referencedIndexSummary)
static int	getIndexSummaryIndexFromBinarySearchResult(int binarySearchResult)
byte[]	getIndexSummaryKey(int index) Returns the key for the index summary entry at `index`.
long	getIndexSummaryOffHeapSize()
int	getIndexSummarySamplingLevel()
int	getIndexSummarySize() Returns the number of entries in the IndexSummary.
InstrumentingCache<KeyCacheKey,RowIndexEntry>	getKeyCache()
long	getKeyCacheHit()
long	getKeyCacheRequest()
java.lang.Iterable<DecoratedKey>	getKeySamples(Range<Token> range)
int	getMaxIndexSummarySize() Returns the approximate number of entries the IndexSummary would contain if it were at full sampling.
int	getMaxLocalDeletionTime()
long	getMaxTimestamp()
int	getMaxTTL()
int	getMinIndexInterval()
int	getMinLocalDeletionTime()
long	getMinTimestamp()
int	getMinTTL()
RowIndexEntry	getPosition(PartitionPosition key, SSTableReader.Operator op) Get position updating key cache and stats.
RowIndexEntry	getPosition(PartitionPosition key, SSTableReader.Operator op, boolean updateCacheAndStats)
protected abstract RowIndexEntry	getPosition(PartitionPosition key, SSTableReader.Operator op, boolean updateCacheAndStats, boolean permitMatchPastLast)
java.util.List<Pair<java.lang.Long,java.lang.Long>>	getPositionsForRanges(java.util.Collection<Range<Token>> ranges) Determine the minimal set of sections that can be extracted from this SSTable to cover the given ranges.
RestorableMeter	getReadMeter()
long	getRecentBloomFilterFalsePositiveCount()
long	getRecentBloomFilterTruePositiveCount()
ISSTableScanner	getScanner() I/O SSTableScanner
abstract ISSTableScanner	getScanner(java.util.Collection<Range<Token>> ranges, com.google.common.util.concurrent.RateLimiter limiter) Direct I/O SSTableScanner over a defined collection of ranges of tokens.
ISSTableScanner	getScanner(ColumnFilter columns, DataRange dataRange, boolean isForThrift)
abstract ISSTableScanner	getScanner(ColumnFilter columns, DataRange dataRange, com.google.common.util.concurrent.RateLimiter limiter, boolean isForThrift)
abstract ISSTableScanner	getScanner(java.util.Iterator<AbstractBounds<PartitionPosition>> rangeIterator) Direct I/O SSTableScanner over an iterator of bounds.
ISSTableScanner	getScanner(Range<Token> range, com.google.common.util.concurrent.RateLimiter limiter) Direct I/O SSTableScanner over a defined range of tokens.
abstract ISSTableScanner	getScanner(com.google.common.util.concurrent.RateLimiter limiter) Direct I/O SSTableScanner over the entirety of the sstable..
int	getSSTableLevel()
StatsMetadata	getSSTableMetadata()
static long	getTotalBytes(java.lang.Iterable<SSTableReader> sstables)
long	getTotalColumnsSet()
long	getTotalRows()
static long	getTotalUncompressedBytes(java.lang.Iterable<SSTableReader> sstables)
int	hashCode()
boolean	hasTombstones() sstable contains no tombstones if minLocalDeletionTime == Integer.MAX_VALUE
void	incrementReadCount() Increment the total row read count and read rate for this SSTable.
static SSTableReader	internalOpen(Descriptor desc, java.util.Set<Component> components, CFMetaData metadata, SegmentedFile ifile, SegmentedFile dfile, IndexSummary isummary, IFilter bf, long maxDataAge, StatsMetadata sstableMetadata, SSTableReader.OpenReason openReason, SerializationHeader header) Open a RowIndexedReader which already has its state initialized (by SSTableWriter).
boolean	isKeyCacheSetup()
boolean	isMarkedCompacted()
boolean	isMarkedSuspect()
boolean	isRepaired()
boolean	isReplaced()
abstract SliceableUnfilteredRowIterator	iterator(DecoratedKey key, ColumnFilter selectedColumns, boolean reversed, boolean isForThrift)
abstract SliceableUnfilteredRowIterator	iterator(FileDataInput file, DecoratedKey key, RowIndexEntry indexEntry, ColumnFilter selectedColumns, boolean reversed, boolean isForThrift)
boolean	loadSummary(SegmentedFile.Builder ibuilder, SegmentedFile.Builder dbuilder) Load index summary from Summary.db file if it exists.
static void	logOpenException(Descriptor descriptor, java.io.IOException e)
void	markObsolete(java.lang.Runnable tidier) Mark the sstable as obsolete, i.e., compacted into newer sstables.
void	markSuspect()
boolean	mayOverlapsWith(SSTableReader other) Checks if this sstable can overlap with another one based on the min/man clustering values.
boolean	newSince(long age) Tests if the sstable contains data newer than the given age param (in localhost currentMilli time).
long	onDiskLength()
static SSTableReader	open(Descriptor descriptor)
static SSTableReader	open(Descriptor desc, CFMetaData metadata)
static SSTableReader	open(Descriptor descriptor, java.util.Set<Component> components, CFMetaData metadata)
static SSTableReader	open(Descriptor descriptor, java.util.Set<Component> components, CFMetaData metadata, boolean validate, boolean trackHotness)
static java.util.Collection<SSTableReader>	openAll(java.util.Set<java.util.Map.Entry<Descriptor,java.util.Set<Component>>> entries, CFMetaData metadata)
RandomAccessReader	openDataReader()
RandomAccessReader	openDataReader(com.google.common.util.concurrent.RateLimiter limiter)
static SSTableReader	openForBatch(Descriptor descriptor, java.util.Set<Component> components, CFMetaData metadata) Open SSTable reader to be used in batch mode(such as sstableloader).
RandomAccessReader	openIndexReader()
static SSTableReader	openNoValidation(Descriptor descriptor, CFMetaData metadata)
static SSTableReader	openNoValidation(Descriptor descriptor, java.util.Set<Component> components, ColumnFamilyStore cfs)
void	overrideReadMeter(RestorableMeter readMeter)
Ref<SSTableReader>	ref()
void	releaseSummary()
void	reloadSSTableMetadata() Reloads the sstable metadata from disk.
static void	resetTidying()
void	runOnClose(java.lang.Runnable runOnClose)
static void	saveSummary(Descriptor descriptor, DecoratedKey first, DecoratedKey last, SegmentedFile.Builder ibuilder, SegmentedFile.Builder dbuilder, IndexSummary summary) Save index summary to Summary.db file.
void	saveSummary(SegmentedFile.Builder ibuilder, SegmentedFile.Builder dbuilder) Save index summary to Summary.db file.
Ref<SSTableReader>	selfRef()
void	setCrcCheckChance(double crcCheckChance) Set the value of CRC check chance.
void	setReplaced()
void	setupOnline()
Ref<SSTableReader>	tryRef()
long	uncompressedLength()

Methods inherited from class org.apache.cassandra.io.sstable.SSTable

addComponents, appendTOC, bytesOnDisk, componentsFor, decorateKey, delete, discoverComponentsFor, estimateRowsFromIndex, getAllFilePaths, getColumnFamilyName, getIndexFilename, getKeyspaceName, getMinimalKey, getPartitioner, readTOC, toString, tryComponentFromFilename

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

maxTimestampComparator

public static final java.util.Comparator<SSTableReader> maxTimestampComparator

sstableComparator

public static final java.util.Comparator<SSTableReader> sstableComparator

sstableOrdering

public static final com.google.common.collect.Ordering<SSTableReader> sstableOrdering

maxDataAge

public final long maxDataAge

maxDataAge is a timestamp in local server time (e.g. System.currentTimeMilli) which represents an upper bound to the newest piece of data stored in the sstable. In other words, this sstable does not contain items created later than maxDataAge. The field is not serialized to disk, so relying on it for more than what truncate does is not advised. When a new sstable is flushed, maxDataAge is set to the time of creation. When a sstable is created from compaction, maxDataAge is set to max of all merged sstables. The age is in milliseconds since epoc and is local to this host.

openReason

public final SSTableReader.OpenReason openReason

instanceId

public final SSTableReader.UniqueIdentifier instanceId

ifile

protected SegmentedFile ifile

dfile

protected SegmentedFile dfile

indexSummary

protected IndexSummary indexSummary

bf

protected IFilter bf

rowIndexEntrySerializer

protected final RowIndexEntry.IndexSerializer rowIndexEntrySerializer

keyCache

protected InstrumentingCache<KeyCacheKey,RowIndexEntry> keyCache

bloomFilterTracker

protected final BloomFilterTracker bloomFilterTracker

isSuspect

protected final java.util.concurrent.atomic.AtomicBoolean isSuspect

sstableMetadata

protected volatile StatsMetadata sstableMetadata

header

public final SerializationHeader header

keyCacheHit

protected final java.util.concurrent.atomic.AtomicLong keyCacheHit

keyCacheRequest

protected final java.util.concurrent.atomic.AtomicLong keyCacheRequest

Constructor Detail

SSTableReader

protected SSTableReader(Descriptor desc,
                        java.util.Set<Component> components,
                        CFMetaData metadata,
                        long maxDataAge,
                        StatsMetadata sstableMetadata,
                        SSTableReader.OpenReason openReason,
                        SerializationHeader header)

Method Detail

getApproximateKeyCount

public static long getApproximateKeyCount(java.lang.Iterable<SSTableReader> sstables)

Calculate approximate key count. If cardinality estimator is available on all given sstables, then this method use them to estimate key count. If not, then this uses index summaries.

Parameters:

sstables - SSTables to calculate key count

Returns:

estimated key count

estimateCompactionGain

public static double estimateCompactionGain(java.util.Set<SSTableReader> overlapping)

Estimates how much of the keys we would keep if the sstables were compacted together

open

public static SSTableReader open(Descriptor descriptor)
                          throws java.io.IOException

Throws:

java.io.IOException

open

public static SSTableReader open(Descriptor desc,
                                 CFMetaData metadata)
                          throws java.io.IOException

Throws:

java.io.IOException

open

public static SSTableReader open(Descriptor descriptor,
                                 java.util.Set<Component> components,
                                 CFMetaData metadata)
                          throws java.io.IOException

Throws:

java.io.IOException

openNoValidation

public static SSTableReader openNoValidation(Descriptor descriptor,
                                             java.util.Set<Component> components,
                                             ColumnFamilyStore cfs)
                                      throws java.io.IOException

Throws:

java.io.IOException

openNoValidation

public static SSTableReader openNoValidation(Descriptor descriptor,
                                             CFMetaData metadata)
                                      throws java.io.IOException

Throws:

java.io.IOException

openForBatch

public static SSTableReader openForBatch(Descriptor descriptor,
                                         java.util.Set<Component> components,
                                         CFMetaData metadata)
                                  throws java.io.IOException

Open SSTable reader to be used in batch mode(such as sstableloader).

Parameters:

descriptor -

components -

metadata -

Returns:

opened SSTableReader

Throws:

java.io.IOException

open

public static SSTableReader open(Descriptor descriptor,
                                 java.util.Set<Component> components,
                                 CFMetaData metadata,
                                 boolean validate,
                                 boolean trackHotness)
                          throws java.io.IOException

Throws:

java.io.IOException

logOpenException

public static void logOpenException(Descriptor descriptor,
                                    java.io.IOException e)

openAll

public static java.util.Collection<SSTableReader> openAll(java.util.Set<java.util.Map.Entry<Descriptor,java.util.Set<Component>>> entries,
                                                          CFMetaData metadata)

internalOpen

public static SSTableReader internalOpen(Descriptor desc,
                                         java.util.Set<Component> components,
                                         CFMetaData metadata,
                                         SegmentedFile ifile,
                                         SegmentedFile dfile,
                                         IndexSummary isummary,
                                         IFilter bf,
                                         long maxDataAge,
                                         StatsMetadata sstableMetadata,
                                         SSTableReader.OpenReason openReason,
                                         SerializationHeader header)

Open a RowIndexedReader which already has its state initialized (by SSTableWriter).

getTotalBytes

public static long getTotalBytes(java.lang.Iterable<SSTableReader> sstables)

getTotalUncompressedBytes

public static long getTotalUncompressedBytes(java.lang.Iterable<SSTableReader> sstables)

equals

public boolean equals(java.lang.Object that)

Overrides:

equals in class java.lang.Object

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object

getFilename

public java.lang.String getFilename()

Overrides:

getFilename in class SSTable

setupOnline

public void setupOnline()

isKeyCacheSetup

public boolean isKeyCacheSetup()

loadSummary

public boolean loadSummary(SegmentedFile.Builder ibuilder,
                           SegmentedFile.Builder dbuilder)

Load index summary from Summary.db file if it exists. if loaded index summary has different index interval from current value stored in schema, then Summary.db file will be deleted and this returns false to rebuild summary.

Parameters:

ibuilder -

dbuilder -

Returns:

true if index summary is loaded successfully from Summary.db file.

saveSummary

public void saveSummary(SegmentedFile.Builder ibuilder,
                        SegmentedFile.Builder dbuilder)

Save index summary to Summary.db file.

Parameters:

ibuilder -

dbuilder -

saveSummary

public static void saveSummary(Descriptor descriptor,
                               DecoratedKey first,
                               DecoratedKey last,
                               SegmentedFile.Builder ibuilder,
                               SegmentedFile.Builder dbuilder,
                               IndexSummary summary)

Save index summary to Summary.db file.

setReplaced

public void setReplaced()

isReplaced

public boolean isReplaced()

runOnClose

public void runOnClose(java.lang.Runnable runOnClose)

cloneWithRestoredStart

public SSTableReader cloneWithRestoredStart(DecoratedKey restoredStart)

cloneWithNewStart

public SSTableReader cloneWithNewStart(DecoratedKey newStart,
                                       java.lang.Runnable runOnClose)

cloneWithNewSummarySamplingLevel

public SSTableReader cloneWithNewSummarySamplingLevel(ColumnFamilyStore parent,
                                                      int samplingLevel)
                                               throws java.io.IOException

Returns a new SSTableReader with the same properties as this SSTableReader except that a new IndexSummary will be built at the target samplingLevel. This (original) SSTableReader instance will be marked as replaced, have its DeletingTask removed, and have its periodic read-meter sync task cancelled.

Parameters:

samplingLevel - the desired sampling level for the index summary on the new SSTableReader

Returns:

a new SSTableReader

Throws:

java.io.IOException

getReadMeter

public RestorableMeter getReadMeter()

getIndexSummarySamplingLevel

public int getIndexSummarySamplingLevel()

getIndexSummaryOffHeapSize

public long getIndexSummaryOffHeapSize()

getMinIndexInterval

public int getMinIndexInterval()

getEffectiveIndexInterval

public double getEffectiveIndexInterval()

releaseSummary

public void releaseSummary()

getIndexScanPosition

public long getIndexScanPosition(PartitionPosition key)

Gets the position in the index file to start scanning to find the given key (at most indexInterval keys away, modulo downsampling of the index summary). Always returns a value >= 0

getIndexScanPositionFromBinarySearchResult

public static long getIndexScanPositionFromBinarySearchResult(int binarySearchResult,
                                                              IndexSummary referencedIndexSummary)

getIndexSummaryIndexFromBinarySearchResult

public static int getIndexSummaryIndexFromBinarySearchResult(int binarySearchResult)

getCompressionMetadata

public CompressionMetadata getCompressionMetadata()

Returns the compression metadata for this sstable.

Throws:

java.lang.IllegalStateException - if the sstable is not compressed

getCompressionMetadataOffHeapSize

public long getCompressionMetadataOffHeapSize()

Returns the amount of memory in bytes used off heap by the compression meta-data.

Returns:

the amount of memory in bytes used off heap by the compression meta-data

forceFilterFailures

public void forceFilterFailures()

For testing purposes only.

getBloomFilter

public IFilter getBloomFilter()

getBloomFilterSerializedSize

public long getBloomFilterSerializedSize()

getBloomFilterOffHeapSize

public long getBloomFilterOffHeapSize()

Returns the amount of memory in bytes used off heap by the bloom filter.

Returns:

the amount of memory in bytes used off heap by the bloom filter

estimatedKeys

public long estimatedKeys()

Returns:

An estimate of the number of keys in this SSTable based on the index summary.

estimatedKeysForRanges

public long estimatedKeysForRanges(java.util.Collection<Range<Token>> ranges)

Parameters:

ranges -

Returns:

An estimate of the number of keys for given ranges in this SSTable.

getIndexSummarySize

public int getIndexSummarySize()

Returns the number of entries in the IndexSummary. At full sampling, this is approximately 1/INDEX_INTERVALth of the keys in this SSTable.

getMaxIndexSummarySize

public int getMaxIndexSummarySize()

Returns the approximate number of entries the IndexSummary would contain if it were at full sampling.

getIndexSummaryKey

public byte[] getIndexSummaryKey(int index)

Returns the key for the index summary entry at `index`.

getKeySamples

public java.lang.Iterable<DecoratedKey> getKeySamples(Range<Token> range)

getPositionsForRanges

public java.util.List<Pair<java.lang.Long,java.lang.Long>> getPositionsForRanges(java.util.Collection<Range<Token>> ranges)

Determine the minimal set of sections that can be extracted from this SSTable to cover the given ranges.

Returns:

A sorted list of (offset,end) pairs that cover the given ranges in the datafile for this SSTable.

getCacheKey

public KeyCacheKey getCacheKey(DecoratedKey key)

cacheKey

public void cacheKey(DecoratedKey key,
                     RowIndexEntry info)

getCachedPosition

public RowIndexEntry getCachedPosition(DecoratedKey key,
                                       boolean updateStats)

getCachedPosition

protected RowIndexEntry getCachedPosition(KeyCacheKey unifiedKey,
                                          boolean updateStats)

getPosition

public RowIndexEntry getPosition(PartitionPosition key,
                                 SSTableReader.Operator op)

Get position updating key cache and stats.

See Also:

getPosition(PartitionPosition, SSTableReader.Operator, boolean)

getPosition

public RowIndexEntry getPosition(PartitionPosition key,
                                 SSTableReader.Operator op,
                                 boolean updateCacheAndStats)

getPosition

protected abstract RowIndexEntry getPosition(PartitionPosition key,
                                             SSTableReader.Operator op,
                                             boolean updateCacheAndStats,
                                             boolean permitMatchPastLast)

Parameters:

key - The key to apply as the rhs to the given Operator. A 'fake' key is allowed to allow key selection by token bounds but only if op != * EQ

op - The Operator defining matching keys: the nearest key to the target matching the operator wins.

updateCacheAndStats - true if updating stats and cache

Returns:

The index entry corresponding to the key, or null if the key is not present

iterator

public abstract SliceableUnfilteredRowIterator iterator(DecoratedKey key,
                                                        ColumnFilter selectedColumns,
                                                        boolean reversed,
                                                        boolean isForThrift)

iterator

public abstract SliceableUnfilteredRowIterator iterator(FileDataInput file,
                                                        DecoratedKey key,
                                                        RowIndexEntry indexEntry,
                                                        ColumnFilter selectedColumns,
                                                        boolean reversed,
                                                        boolean isForThrift)

firstKeyBeyond

public DecoratedKey firstKeyBeyond(PartitionPosition token)

Finds and returns the first key beyond a given token in this SSTable or null if no such key exists.

uncompressedLength

public long uncompressedLength()

Returns:

The length in bytes of the data for this SSTable. For compressed files, this is not the same thing as the on disk size (see onDiskLength())

onDiskLength

public long onDiskLength()

Returns:

The length in bytes of the on disk size for this SSTable. For compressed files, this is not the same thing as the data length (see length())

getCrcCheckChance

public double getCrcCheckChance()

setCrcCheckChance

public void setCrcCheckChance(double crcCheckChance)

Set the value of CRC check chance. The argument supplied is obtained from the the property of the owning CFS. Called when either the SSTR is initialized, or the CFS's property is updated via JMX

Parameters:

crcCheckChance -

markObsolete

public void markObsolete(java.lang.Runnable tidier)

Mark the sstable as obsolete, i.e., compacted into newer sstables. When calling this function, the caller must ensure that the SSTableReader is not referenced anywhere except for threads holding a reference.

isMarkedCompacted

public boolean isMarkedCompacted()

markSuspect

public void markSuspect()

isMarkedSuspect

public boolean isMarkedSuspect()

getScanner

public ISSTableScanner getScanner()

I/O SSTableScanner

Returns:

A Scanner for seeking over the rows of the SSTable.

getScanner

public ISSTableScanner getScanner(ColumnFilter columns,
                                  DataRange dataRange,
                                  boolean isForThrift)

Parameters:

columns - the columns to return.

dataRange - filter to use when reading the columns

Returns:

A Scanner for seeking over the rows of the SSTable.

getScanner

public ISSTableScanner getScanner(Range<Token> range,
                                  com.google.common.util.concurrent.RateLimiter limiter)

Direct I/O SSTableScanner over a defined range of tokens.

Parameters:

range - the range of keys to cover

Returns:

A Scanner for seeking over the rows of the SSTable.

getScanner

public abstract ISSTableScanner getScanner(com.google.common.util.concurrent.RateLimiter limiter)

Direct I/O SSTableScanner over the entirety of the sstable..

Returns:

A Scanner over the full content of the SSTable.

getScanner

public abstract ISSTableScanner getScanner(java.util.Collection<Range<Token>> ranges,
                                           com.google.common.util.concurrent.RateLimiter limiter)

Direct I/O SSTableScanner over a defined collection of ranges of tokens.

Parameters:

ranges - the range of keys to cover

Returns:

A Scanner for seeking over the rows of the SSTable.

getScanner

public abstract ISSTableScanner getScanner(java.util.Iterator<AbstractBounds<PartitionPosition>> rangeIterator)

Direct I/O SSTableScanner over an iterator of bounds.

Parameters:

bounds - the keys to cover

Returns:

A Scanner for seeking over the rows of the SSTable.

getScanner

public abstract ISSTableScanner getScanner(ColumnFilter columns,
                                           DataRange dataRange,
                                           com.google.common.util.concurrent.RateLimiter limiter,
                                           boolean isForThrift)

Parameters:

columns - the columns to return.

dataRange - filter to use when reading the columns

Returns:

A Scanner for seeking over the rows of the SSTable.

getFileDataInput

public FileDataInput getFileDataInput(long position)

newSince

public boolean newSince(long age)

Tests if the sstable contains data newer than the given age param (in localhost currentMilli time). This works in conjunction with maxDataAge which is an upper bound on the create of data in this sstable.

Parameters:

age - The age to compare the maxDataAre of this sstable. Measured in millisec since epoc on this host

Returns:

True iff this sstable contains data that's newer than the given age parameter.

createLinks

public void createLinks(java.lang.String snapshotDirectoryPath)

isRepaired

public boolean isRepaired()

getBloomFilterFalsePositiveCount

public long getBloomFilterFalsePositiveCount()

getRecentBloomFilterFalsePositiveCount

public long getRecentBloomFilterFalsePositiveCount()

getBloomFilterTruePositiveCount

public long getBloomFilterTruePositiveCount()

getRecentBloomFilterTruePositiveCount

public long getRecentBloomFilterTruePositiveCount()

getKeyCache

public InstrumentingCache<KeyCacheKey,RowIndexEntry> getKeyCache()

getEstimatedPartitionSize

public EstimatedHistogram getEstimatedPartitionSize()

getEstimatedColumnCount

public EstimatedHistogram getEstimatedColumnCount()

getEstimatedDroppableTombstoneRatio

public double getEstimatedDroppableTombstoneRatio(int gcBefore)

getDroppableTombstonesBefore

public double getDroppableTombstonesBefore(int gcBefore)

getCompressionRatio

public double getCompressionRatio()

getMinTimestamp

public long getMinTimestamp()

getMaxTimestamp

public long getMaxTimestamp()

getMinLocalDeletionTime

public int getMinLocalDeletionTime()

getMaxLocalDeletionTime

public int getMaxLocalDeletionTime()

hasTombstones

public boolean hasTombstones()

sstable contains no tombstones if minLocalDeletionTime == Integer.MAX_VALUE

getMinTTL

public int getMinTTL()

getMaxTTL

public int getMaxTTL()

getTotalColumnsSet

public long getTotalColumnsSet()

getTotalRows

public long getTotalRows()

getAvgColumnSetPerRow

public int getAvgColumnSetPerRow()

getSSTableLevel

public int getSSTableLevel()

reloadSSTableMetadata

public void reloadSSTableMetadata()
                           throws java.io.IOException

Reloads the sstable metadata from disk. Called after level is changed on sstable, for example if the sstable is dropped to L0 Might be possible to remove in future versions

Throws:

java.io.IOException

getSSTableMetadata

public StatsMetadata getSSTableMetadata()

openDataReader

public RandomAccessReader openDataReader(com.google.common.util.concurrent.RateLimiter limiter)

openDataReader

public RandomAccessReader openDataReader()

openIndexReader

public RandomAccessReader openIndexReader()

getDataChannel

public ChannelProxy getDataChannel()

getIndexChannel

public ChannelProxy getIndexChannel()

getCreationTimeFor

public long getCreationTimeFor(Component component)

Parameters:

component - component to get timestamp.

Returns:

last modified time for given component. 0 if given component does not exist or IO error occurs.

getKeyCacheHit

public long getKeyCacheHit()

Returns:

Number of key cache hit

getKeyCacheRequest

public long getKeyCacheRequest()

Returns:

Number of key cache request

incrementReadCount

public void incrementReadCount()

Increment the total row read count and read rate for this SSTable. This should not be incremented for range slice queries, row cache hits, or non-query reads, like compaction.

mayOverlapsWith

public boolean mayOverlapsWith(SSTableReader other)

Checks if this sstable can overlap with another one based on the min/man clustering values. If this methods return false, we're guarantee that this and other have no overlapping data, i.e. no cells to reconcile.

tryRef

public Ref<SSTableReader> tryRef()

Specified by:

tryRef in interface RefCounted<SSTableReader>

Returns:

the a new Ref() to the managed object, incrementing its refcount, or null if it is already released

selfRef

public Ref<SSTableReader> selfRef()

Specified by:

selfRef in interface SelfRefCounted<SSTableReader>

ref

public Ref<SSTableReader> ref()

Specified by:

ref in interface RefCounted<SSTableReader>

overrideReadMeter

public void overrideReadMeter(RestorableMeter readMeter)

addTo

public void addTo(Ref.IdentityCollection identities)

resetTidying

public static void resetTidying()