org.apache.cassandra.io.sstable

Class SSTableReader

java.lang.Object

org.apache.cassandra.io.sstable.SSTable

org.apache.cassandra.io.sstable.SSTableReader

public class SSTableReader
extends SSTable

SSTableReaders are open()ed by Keyspace.onStart; after that they are created by SSTableWriter.renameAndOpen. Do not re-call open() on existing SSTable files; use the references kept by ColumnFamilyStore post-start instead.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	SSTableReader.EmptyCompactionScanner
static class	SSTableReader.OpenReason
static class	SSTableReader.Operator TODO: Move someplace reusable
static class	SSTableReader.SizeComparator

Field Summary

Fields

Modifier and Type	Field and Description
long	maxDataAge maxDataAge is a timestamp in local server time (e.g.
static java.util.Comparator<SSTableReader>	maxTimestampComparator
SSTableReader.OpenReason	openReason
RestorableMeter	readMeter
static java.util.Comparator<SSTableReader>	sstableComparator
static com.google.common.collect.Ordering<SSTableReader>	sstableOrdering

Fields inherited from class org.apache.cassandra.io.sstable.SSTable

components, compression, descriptor, first, last, metadata, partitioner, TOMBSTONE_HISTOGRAM_BIN_SIZE

Method Summary

Methods

Modifier and Type	Method and Description
boolean	acquireReference()
static boolean	acquireReferences(java.lang.Iterable<SSTableReader> sstables)
void	cacheKey(DecoratedKey key, RowIndexEntry info)
SSTableReader	cloneWithNewStart(DecoratedKey newStart, java.lang.Runnable runOnClose)
SSTableReader	cloneWithNewSummarySamplingLevel(ColumnFamilyStore parent, int samplingLevel) Returns a new SSTableReader with the same properties as this SSTableReader except that a new IndexSummary will be built at the target samplingLevel.
void	createLinks(java.lang.String snapshotDirectoryPath)
boolean	equals(java.lang.Object that)
long	estimatedKeys()
long	estimatedKeysForRanges(java.util.Collection<Range<Token>> ranges)
DecoratedKey	firstKeyBeyond(RowPosition token) Finds and returns the first key beyond a given token in this SSTable or null if no such key exists.
void	forceFilterFailures() For testing purposes only.
java.util.Set<java.lang.Integer>	getAncestors()
static long	getApproximateKeyCount(java.util.Collection<SSTableReader> sstables) Calculate approximate key count.
IFilter	getBloomFilter()
long	getBloomFilterFalsePositiveCount()
long	getBloomFilterSerializedSize()
long	getBloomFilterTruePositiveCount()
RowIndexEntry	getCachedPosition(DecoratedKey key, boolean updateStats)
CompressionMetadata	getCompressionMetadata() Returns the compression metadata for this sstable.
double	getCompressionRatio()
long	getCreationTimeFor(Component component)
SSTableReader	getCurrentReplacement()
double	getDroppableTombstonesBefore(int gcBefore)
double	getEffectiveIndexInterval()
EstimatedHistogram	getEstimatedColumnCount()
double	getEstimatedDroppableTombstoneRatio(int gcBefore)
EstimatedHistogram	getEstimatedRowSize()
FileDataInput	getFileDataInput(long position)
java.lang.String	getFilename()
java.lang.String	getIndexFilename()
long	getIndexScanPosition(RowPosition key) Gets the position in the index file to start scanning to find the given key (at most indexInterval keys away, modulo downsampling of the index summary).
byte[]	getIndexSummaryKey(int index) Returns the key for the index summary entry at `index`.
long	getIndexSummaryOffHeapSize()
int	getIndexSummarySamplingLevel()
int	getIndexSummarySize() Returns the number of entries in the IndexSummary.
InstrumentingCache<KeyCacheKey,RowIndexEntry>	getKeyCache()
long	getKeyCacheHit()
long	getKeyCacheRequest()
java.lang.Iterable<DecoratedKey>	getKeySamples(Range<Token> range)
int	getMaxIndexSummarySize() Returns the approximate number of entries the IndexSummary would contain if it were at full sampling.
long	getMaxTimestamp()
int	getMinIndexInterval()
long	getMinTimestamp()
RowIndexEntry	getPosition(RowPosition key, SSTableReader.Operator op) Get position updating key cache and stats.
RowIndexEntry	getPosition(RowPosition key, SSTableReader.Operator op, boolean updateCacheAndStats)
java.util.List<Pair<java.lang.Long,java.lang.Long>>	getPositionsForRanges(java.util.Collection<Range<Token>> ranges) Determine the minimal set of sections that can be extracted from this SSTable to cover the given ranges.
long	getRecentBloomFilterFalsePositiveCount()
long	getRecentBloomFilterTruePositiveCount()
ReplayPosition	getReplayPosition()
SSTableScanner	getScanner() I/O SSTableScanner
ICompactionScanner	getScanner(java.util.Collection<Range<Token>> ranges, com.google.common.util.concurrent.RateLimiter limiter) Direct I/O SSTableScanner over a defined collection of ranges of tokens.
SSTableScanner	getScanner(DataRange dataRange)
ICompactionScanner	getScanner(Range<Token> range, com.google.common.util.concurrent.RateLimiter limiter) Direct I/O SSTableScanner over a defined range of tokens.
SSTableScanner	getScanner(com.google.common.util.concurrent.RateLimiter limiter)
int	getSSTableLevel()
StatsMetadata	getSSTableMetadata()
static long	getTotalBytes(java.lang.Iterable<SSTableReader> sstables)
int	hashCode()
void	incrementReadCount() Increment the total row read count and read rate for this SSTable.
void	invalidateCacheKey(DecoratedKey key)
boolean	isMarkedCompacted()
boolean	isMarkedSuspect()
boolean	isRepaired()
boolean	loadSummary(SegmentedFile.Builder ibuilder, SegmentedFile.Builder dbuilder) Load index summary from Summary.db file if it exists.
static void	logOpenException(Descriptor descriptor, java.io.IOException e)
boolean	markObsolete() Mark the sstable as obsolete, i.e., compacted into newer sstables.
void	markSuspect()
boolean	newSince(long age) Tests if the sstable contains data newer than the given age param (in localhost currentMilli time).
long	onDiskLength()
static SSTableReader	open(Descriptor descriptor)
static SSTableReader	open(Descriptor desc, CFMetaData metadata)
static SSTableReader	open(Descriptor descriptor, java.util.Set<Component> components, CFMetaData metadata, IPartitioner partitioner)
static java.util.Collection<SSTableReader>	openAll(java.util.Set<java.util.Map.Entry<Descriptor,java.util.Set<Component>>> entries, CFMetaData metadata, IPartitioner partitioner)
RandomAccessReader	openDataReader()
RandomAccessReader	openDataReader(com.google.common.util.concurrent.RateLimiter limiter)
static SSTableReader	openForBatch(Descriptor descriptor, java.util.Set<Component> components, CFMetaData metadata, IPartitioner partitioner) Open SSTable reader to be used in batch mode(such as sstableloader).
RandomAccessReader	openIndexReader()
static SSTableReader	openNoValidation(Descriptor descriptor, java.util.Set<Component> components, CFMetaData metadata)
void	releaseReference() Release reference to this SSTableReader.
static void	releaseReferences(java.lang.Iterable<SSTableReader> sstables)
void	releaseSummary()
void	reloadSSTableMetadata() Reloads the sstable metadata from disk.
void	saveSummary(SegmentedFile.Builder ibuilder, SegmentedFile.Builder dbuilder) Save index summary to Summary.db file.
void	setReplacedBy(SSTableReader replacement)
void	setTrackedBy(DataTracker tracker)
long	uncompressedLength()

Methods inherited from class org.apache.cassandra.io.sstable.SSTable

addComponents, appendTOC, bytesOnDisk, componentsFor, delete, getColumnFamilyName, getKeyspaceName, getMinimalKey, readTOC, toString, tryComponentFromFilename

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

maxTimestampComparator

public static final java.util.Comparator<SSTableReader> maxTimestampComparator

sstableComparator

public static final java.util.Comparator<SSTableReader> sstableComparator

sstableOrdering

public static final com.google.common.collect.Ordering<SSTableReader> sstableOrdering

maxDataAge

public final long maxDataAge

maxDataAge is a timestamp in local server time (e.g. System.currentTimeMilli) which represents an upper bound to the newest piece of data stored in the sstable. In other words, this sstable does not contain items created later than maxDataAge. The field is not serialized to disk, so relying on it for more than what truncate does is not advised. When a new sstable is flushed, maxDataAge is set to the time of creation. When a sstable is created from compaction, maxDataAge is set to max of all merged sstables. The age is in milliseconds since epoc and is local to this host.

openReason

public final SSTableReader.OpenReason openReason

readMeter

public RestorableMeter readMeter

Method Detail

getApproximateKeyCount

public static long getApproximateKeyCount(java.util.Collection<SSTableReader> sstables)

Calculate approximate key count. If cardinality estimator is available on all given sstables, then this method use them to estimate key count. If not, then this uses index summaries.

Parameters:

sstables - SSTables to calculate key count

Returns:

estimated key count

open

public static SSTableReader open(Descriptor descriptor)
                          throws java.io.IOException

Throws:

java.io.IOException

open

public static SSTableReader open(Descriptor desc,
                 CFMetaData metadata)
                          throws java.io.IOException

Throws:

java.io.IOException

open

public static SSTableReader open(Descriptor descriptor,
                 java.util.Set<Component> components,
                 CFMetaData metadata,
                 IPartitioner partitioner)
                          throws java.io.IOException

Throws:

java.io.IOException

openNoValidation

public static SSTableReader openNoValidation(Descriptor descriptor,
                             java.util.Set<Component> components,
                             CFMetaData metadata)
                                      throws java.io.IOException

Throws:

java.io.IOException

openForBatch

public static SSTableReader openForBatch(Descriptor descriptor,
                         java.util.Set<Component> components,
                         CFMetaData metadata,
                         IPartitioner partitioner)
                                  throws java.io.IOException

Open SSTable reader to be used in batch mode(such as sstableloader).

Parameters:

descriptor -

components -

metadata -

partitioner -

Returns:

opened SSTableReader

Throws:

java.io.IOException

logOpenException

public static void logOpenException(Descriptor descriptor,
                    java.io.IOException e)

openAll

public static java.util.Collection<SSTableReader> openAll(java.util.Set<java.util.Map.Entry<Descriptor,java.util.Set<Component>>> entries,
                                          CFMetaData metadata,
                                          IPartitioner partitioner)

getTotalBytes

public static long getTotalBytes(java.lang.Iterable<SSTableReader> sstables)

equals

public boolean equals(java.lang.Object that)

Overrides:

equals in class java.lang.Object

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object

getFilename

public java.lang.String getFilename()

Overrides:

getFilename in class SSTable

getIndexFilename

public java.lang.String getIndexFilename()

Overrides:

getIndexFilename in class SSTable

setTrackedBy

public void setTrackedBy(DataTracker tracker)

loadSummary

public boolean loadSummary(SegmentedFile.Builder ibuilder,
                  SegmentedFile.Builder dbuilder)

Load index summary from Summary.db file if it exists. if loaded index summary has different index interval from current value stored in schema, then Summary.db file will be deleted and this returns false to rebuild summary.

Parameters:

ibuilder -

dbuilder -

Returns:

true if index summary is loaded successfully from Summary.db file.

saveSummary

public void saveSummary(SegmentedFile.Builder ibuilder,
               SegmentedFile.Builder dbuilder)

Save index summary to Summary.db file.

Parameters:

ibuilder -

dbuilder -

setReplacedBy

public void setReplacedBy(SSTableReader replacement)

cloneWithNewStart

public SSTableReader cloneWithNewStart(DecoratedKey newStart,
                              java.lang.Runnable runOnClose)

cloneWithNewSummarySamplingLevel

public SSTableReader cloneWithNewSummarySamplingLevel(ColumnFamilyStore parent,
                                             int samplingLevel)
                                               throws java.io.IOException

Returns a new SSTableReader with the same properties as this SSTableReader except that a new IndexSummary will be built at the target samplingLevel. This (original) SSTableReader instance will be marked as replaced, have its DeletingTask removed, and have its periodic read-meter sync task cancelled.

Parameters:

samplingLevel - the desired sampling level for the index summary on the new SSTableReader

Returns:

a new SSTableReader

Throws:

java.io.IOException

getIndexSummarySamplingLevel

public int getIndexSummarySamplingLevel()

getIndexSummaryOffHeapSize

public long getIndexSummaryOffHeapSize()

getMinIndexInterval

public int getMinIndexInterval()

getEffectiveIndexInterval

public double getEffectiveIndexInterval()

releaseSummary

public void releaseSummary()
                    throws java.io.IOException

Throws:

java.io.IOException

getIndexScanPosition

public long getIndexScanPosition(RowPosition key)

Gets the position in the index file to start scanning to find the given key (at most indexInterval keys away, modulo downsampling of the index summary).

getCompressionMetadata

public CompressionMetadata getCompressionMetadata()

Returns the compression metadata for this sstable.

Throws:

java.lang.IllegalStateException - if the sstable is not compressed

forceFilterFailures

public void forceFilterFailures()

For testing purposes only.

getBloomFilter

public IFilter getBloomFilter()

getBloomFilterSerializedSize

public long getBloomFilterSerializedSize()

estimatedKeys

public long estimatedKeys()

Returns:

An estimate of the number of keys in this SSTable based on the index summary.

estimatedKeysForRanges

public long estimatedKeysForRanges(java.util.Collection<Range<Token>> ranges)

Parameters:

ranges -

Returns:

An estimate of the number of keys for given ranges in this SSTable.

getIndexSummarySize

public int getIndexSummarySize()

Returns the number of entries in the IndexSummary. At full sampling, this is approximately 1/INDEX_INTERVALth of the keys in this SSTable.

getMaxIndexSummarySize

public int getMaxIndexSummarySize()

Returns the approximate number of entries the IndexSummary would contain if it were at full sampling.

getIndexSummaryKey

public byte[] getIndexSummaryKey(int index)

Returns the key for the index summary entry at `index`.

getKeySamples

public java.lang.Iterable<DecoratedKey> getKeySamples(Range<Token> range)

getPositionsForRanges

public java.util.List<Pair<java.lang.Long,java.lang.Long>> getPositionsForRanges(java.util.Collection<Range<Token>> ranges)

Determine the minimal set of sections that can be extracted from this SSTable to cover the given ranges.

Returns:

A sorted list of (offset,end) pairs that cover the given ranges in the datafile for this SSTable.

invalidateCacheKey

public void invalidateCacheKey(DecoratedKey key)

cacheKey

public void cacheKey(DecoratedKey key,
            RowIndexEntry info)

getCachedPosition

public RowIndexEntry getCachedPosition(DecoratedKey key,
                              boolean updateStats)

getPosition

public RowIndexEntry getPosition(RowPosition key,
                        SSTableReader.Operator op)

Get position updating key cache and stats.

See Also:

getPosition(org.apache.cassandra.db.RowPosition, org.apache.cassandra.io.sstable.SSTableReader.Operator, boolean)

getPosition

public RowIndexEntry getPosition(RowPosition key,
                        SSTableReader.Operator op,
                        boolean updateCacheAndStats)

Parameters:

key - The key to apply as the rhs to the given Operator. A 'fake' key is allowed to allow key selection by token bounds but only if op != * EQ

op - The Operator defining matching keys: the nearest key to the target matching the operator wins.

updateCacheAndStats - true if updating stats and cache

Returns:

The index entry corresponding to the key, or null if the key is not present

firstKeyBeyond

public DecoratedKey firstKeyBeyond(RowPosition token)

Finds and returns the first key beyond a given token in this SSTable or null if no such key exists.

uncompressedLength

public long uncompressedLength()

Returns:

The length in bytes of the data for this SSTable. For compressed files, this is not the same thing as the on disk size (see onDiskLength())

onDiskLength

public long onDiskLength()

Returns:

The length in bytes of the on disk size for this SSTable. For compressed files, this is not the same thing as the data length (see length())

acquireReference

public boolean acquireReference()

releaseReference

public void releaseReference()

Release reference to this SSTableReader. If there is no one referring to this SSTable, and is marked as compacted, all resources are cleaned up and files are deleted eventually.

markObsolete

public boolean markObsolete()

Mark the sstable as obsolete, i.e., compacted into newer sstables. When calling this function, the caller must ensure that the SSTableReader is not referenced anywhere except for threads holding a reference.

Returns:

true if the this is the first time the file was marked obsolete. Calling this multiple times is usually buggy (see exceptions in DataTracker.unmarkCompacting and removeOldSSTablesSize).

isMarkedCompacted

public boolean isMarkedCompacted()

markSuspect

public void markSuspect()

isMarkedSuspect

public boolean isMarkedSuspect()

getScanner

public SSTableScanner getScanner(DataRange dataRange)

Parameters:

dataRange - filter to use when reading the columns

Returns:

A Scanner for seeking over the rows of the SSTable.

getScanner

public SSTableScanner getScanner()

I/O SSTableScanner

Returns:

A Scanner for seeking over the rows of the SSTable.

getScanner

public SSTableScanner getScanner(com.google.common.util.concurrent.RateLimiter limiter)

getScanner

public ICompactionScanner getScanner(Range<Token> range,
                            com.google.common.util.concurrent.RateLimiter limiter)

Direct I/O SSTableScanner over a defined range of tokens.

Parameters:

range - the range of keys to cover

Returns:

A Scanner for seeking over the rows of the SSTable.

getScanner

public ICompactionScanner getScanner(java.util.Collection<Range<Token>> ranges,
                            com.google.common.util.concurrent.RateLimiter limiter)

Direct I/O SSTableScanner over a defined collection of ranges of tokens.

Parameters:

ranges - the range of keys to cover

Returns:

A Scanner for seeking over the rows of the SSTable.

getFileDataInput

public FileDataInput getFileDataInput(long position)

newSince

public boolean newSince(long age)

Tests if the sstable contains data newer than the given age param (in localhost currentMilli time). This works in conjunction with maxDataAge which is an upper bound on the create of data in this sstable.

Parameters:

age - The age to compare the maxDataAre of this sstable. Measured in millisec since epoc on this host

Returns:

True iff this sstable contains data that's newer than the given age parameter.

createLinks

public void createLinks(java.lang.String snapshotDirectoryPath)

isRepaired

public boolean isRepaired()

getCurrentReplacement

public SSTableReader getCurrentReplacement()

getBloomFilterFalsePositiveCount

public long getBloomFilterFalsePositiveCount()

getRecentBloomFilterFalsePositiveCount

public long getRecentBloomFilterFalsePositiveCount()

getBloomFilterTruePositiveCount

public long getBloomFilterTruePositiveCount()

getRecentBloomFilterTruePositiveCount

public long getRecentBloomFilterTruePositiveCount()

getKeyCache

public InstrumentingCache<KeyCacheKey,RowIndexEntry> getKeyCache()

getEstimatedRowSize

public EstimatedHistogram getEstimatedRowSize()

getEstimatedColumnCount

public EstimatedHistogram getEstimatedColumnCount()

getEstimatedDroppableTombstoneRatio

public double getEstimatedDroppableTombstoneRatio(int gcBefore)

getDroppableTombstonesBefore

public double getDroppableTombstonesBefore(int gcBefore)

getCompressionRatio

public double getCompressionRatio()

getReplayPosition

public ReplayPosition getReplayPosition()

getMinTimestamp

public long getMinTimestamp()

getMaxTimestamp

public long getMaxTimestamp()

getAncestors

public java.util.Set<java.lang.Integer> getAncestors()

getSSTableLevel

public int getSSTableLevel()

reloadSSTableMetadata

public void reloadSSTableMetadata()
                           throws java.io.IOException

Reloads the sstable metadata from disk. Called after level is changed on sstable, for example if the sstable is dropped to L0 Might be possible to remove in future versions

Throws:

java.io.IOException

getSSTableMetadata

public StatsMetadata getSSTableMetadata()

openDataReader

public RandomAccessReader openDataReader(com.google.common.util.concurrent.RateLimiter limiter)

openDataReader

public RandomAccessReader openDataReader()

openIndexReader

public RandomAccessReader openIndexReader()

getCreationTimeFor

public long getCreationTimeFor(Component component)

Parameters:

component - component to get timestamp.

Returns:

last modified time for given component. 0 if given component does not exist or IO error occurs.

getKeyCacheHit

public long getKeyCacheHit()

Returns:

Number of key cache hit

getKeyCacheRequest

public long getKeyCacheRequest()

Returns:

Number of key cache request

acquireReferences

public static boolean acquireReferences(java.lang.Iterable<SSTableReader> sstables)

Parameters:

sstables -

Returns:

true if all desired references were acquired. Otherwise, it will unreference any partial acquisition, and return false.

releaseReferences

public static void releaseReferences(java.lang.Iterable<SSTableReader> sstables)

incrementReadCount

public void incrementReadCount()

Increment the total row read count and read rate for this SSTable. This should not be incremented for range slice queries, row cache hits, or non-query reads, like compaction.