org.apache.hadoop.hbase.regionserver
Interface Store

All Superinterfaces:

HeapSize, StoreConfigInformation

All Known Implementing Classes:

HStore

@InterfaceAudience.Private
@InterfaceStability.Evolving
public interface Store
extends HeapSize, StoreConfigInformation

Interface for objects that hold a column family in a Region. Its a memstore and a set of zero or more StoreFiles, which stretch backwards over time.

Field Summary

static int	NO_PRIORITY
static int	PRIORITY_USER

Method Summary

long	add(KeyValue kv) Adds a value to the memstore
void	addChangedReaderObserver(ChangedReadersObserver o)
boolean	areWritesEnabled()
void	assertBulkLoadHFileOk(org.apache.hadoop.fs.Path srcPath) This throws a WrongRegionException if the HFile does not fit in this region, or an InvalidHFileException if the HFile is not valid.
void	bulkLoadHFile(String srcPathStr, long sequenceId) This method should only be called from HRegion.
void	cancelRequestedCompaction(CompactionContext compaction)
boolean	canSplit()
Collection<StoreFile>	close() Close all the readers We don't need to worry about subsequent requests because the HRegion holds a write lock that will prevent any more reads or writes.
List<StoreFile>	compact(CompactionContext compaction, CompactionThroughputController throughputController) Deprecated. see compact(CompactionContext, CompactionThroughputController, User)
List<StoreFile>	compact(CompactionContext compaction, CompactionThroughputController throughputController, User user)
void	completeCompactionMarker(org.apache.hadoop.hbase.protobuf.generated.WALProtos.CompactionDescriptor compaction) Call to complete a compaction.
org.apache.hadoop.hbase.regionserver.StoreFlushContext	createFlushContext(long cacheFlushId)
StoreFile.Writer	createWriterInTmp(long maxKeyCount, Compression.Algorithm compression, boolean isCompaction, boolean includeMVCCReadpoint, boolean includesTags)
StoreFile.Writer	createWriterInTmp(long maxKeyCount, Compression.Algorithm compression, boolean isCompaction, boolean includeMVCCReadpoint, boolean includesTags, boolean shouldDropBehind)
void	deleteChangedReaderObserver(ChangedReadersObserver o)
long	getAvgStoreFileAge()
CacheConfig	getCacheConfig() Used for tests.
String	getColumnFamilyName()
long	getCompactedCellsCount()
long	getCompactedCellsSize()
double	getCompactionPressure() This value can represent the degree of emergency of compaction for this store.
CompactionProgress	getCompactionProgress() getter for CompactionProgress object
int	getCompactPriority()
KeyValue.KVComparator	getComparator()
RegionCoprocessorHost	getCoprocessorHost()
HFileDataBlockEncoder	getDataBlockEncoder()
HColumnDescriptor	getFamily()
org.apache.hadoop.fs.FileSystem	getFileSystem()
long	getFlushableSize()
long	getFlushedCellsCount()
long	getFlushedCellsSize()
long	getLastCompactSize()
long	getMajorCompactedCellsCount()
long	getMajorCompactedCellsSize()
long	getMaxMemstoreTS()
long	getMaxStoreFileAge()
long	getMemStoreSize()
long	getMinStoreFileAge()
long	getNumHFiles()
long	getNumReferenceFiles()
HRegionInfo	getRegionInfo()
KeyValue	getRowKeyAtOrBefore(byte[] row) Find the key that matches row exactly, or the one that immediately precedes it.
ScanInfo	getScanInfo()
KeyValueScanner	getScanner(Scan scan, NavigableSet<byte[]> targetCols, long readPt) Return a scanner for both the memstore and the HStore files.
List<KeyValueScanner>	getScanners(boolean cacheBlocks, boolean isGet, boolean usePread, boolean isCompaction, ScanQueryMatcher matcher, byte[] startRow, byte[] stopRow, long readPt) Get all scanners with no filtering based on TTL (that happens further down the line).
long	getSize()
long	getSmallestReadPoint()
byte[]	getSplitPoint() Determines if Store should be split
Collection<StoreFile>	getStorefiles()
int	getStorefilesCount()
long	getStorefilesIndexSize()
long	getStorefilesSize()
long	getStoreSizeUncompressed()
TableName	getTableName()
long	getTotalStaticBloomSize() Returns the total byte size of all Bloom filter bit arrays.
long	getTotalStaticIndexSize() Returns the total size of all index blocks in the data block indexes, including the root level, intermediate levels, and the leaf level for multi-level indexes, or just the root level for single-level indexes.
boolean	hasReferences()
boolean	hasTooManyStoreFiles()
boolean	isMajorCompaction()
boolean	needsCompaction() See if there's too much store files in this store
CompactionContext	requestCompaction()
CompactionContext	requestCompaction(int priority, CompactionRequest baseRequest) Deprecated. see requestCompaction(int, CompactionRequest, User)
CompactionContext	requestCompaction(int priority, CompactionRequest baseRequest, User user)
void	rollback(KeyValue kv) Removes a kv from the memstore.
boolean	throttleCompaction(long compactionSize)
long	timeOfOldestEdit() When was the last edit done in the memstore
void	triggerMajorCompaction()
long	upsert(Iterable<Cell> cells, long readpoint) Adds or replaces the specified KeyValues.

Methods inherited from interface org.apache.hadoop.hbase.io.HeapSize

heapSize

Methods inherited from interface org.apache.hadoop.hbase.regionserver.StoreConfigInformation

getBlockingFileCount, getCompactionCheckMultiplier, getMemstoreFlushSize, getStoreFileTtl

Field Detail

PRIORITY_USER

static final int PRIORITY_USER

See Also:

Constant Field Values

NO_PRIORITY

static final int NO_PRIORITY

See Also:

Constant Field Values

Method Detail

getComparator

KeyValue.KVComparator getComparator()

getStorefiles

Collection<StoreFile> getStorefiles()

close

Collection<StoreFile> close()
                            throws IOException

Close all the readers We don't need to worry about subsequent requests because the HRegion holds a write lock that will prevent any more reads or writes.

Returns:

the StoreFiles that were previously being used.

Throws:

IOException - on failure

getScanner

KeyValueScanner getScanner(Scan scan,
                           NavigableSet<byte[]> targetCols,
                           long readPt)
                           throws IOException

Return a scanner for both the memstore and the HStore files. Assumes we are not in a compaction.

Parameters:

scan - Scan to apply when scanning the stores

targetCols - columns to scan

Returns:

a scanner over the current key values

Throws:

IOException - on failure

getScanners

List<KeyValueScanner> getScanners(boolean cacheBlocks,
                                  boolean isGet,
                                  boolean usePread,
                                  boolean isCompaction,
                                  ScanQueryMatcher matcher,
                                  byte[] startRow,
                                  byte[] stopRow,
                                  long readPt)
                                  throws IOException

Get all scanners with no filtering based on TTL (that happens further down the line).

Parameters:

cacheBlocks -

isGet -

usePread -

isCompaction -

matcher -

startRow -

stopRow -

readPt -

Returns:

all scanners for this store

Throws:

IOException

getScanInfo

ScanInfo getScanInfo()

upsert

long upsert(Iterable<Cell> cells,
            long readpoint)
            throws IOException

Adds or replaces the specified KeyValues.

For each KeyValue specified, if a cell with the same row, family, and qualifier exists in MemStore, it will be replaced. Otherwise, it will just be inserted to MemStore.

This operation is atomic on each KeyValue (row/family/qualifier) but not necessarily atomic across all of them.

Parameters:

cells -

readpoint - readpoint below which we can safely remove duplicate KVs

Returns:

memstore size delta

Throws:

IOException

add

long add(KeyValue kv)

Adds a value to the memstore

Parameters:

kv -

Returns:

memstore size delta

timeOfOldestEdit

long timeOfOldestEdit()

When was the last edit done in the memstore

rollback

void rollback(KeyValue kv)

Removes a kv from the memstore. The KeyValue is removed only if its key & memstoreTS match the key & memstoreTS value of the kv parameter.

Parameters:

kv -

getRowKeyAtOrBefore

KeyValue getRowKeyAtOrBefore(byte[] row)
                             throws IOException

Find the key that matches row exactly, or the one that immediately precedes it. WARNING: Only use this method on a table where writes occur with strictly increasing timestamps. This method assumes this pattern of writes in order to make it reasonably performant. Also our search is dependent on the axiom that deletes are for cells that are in the container that follows whether a memstore snapshot or a storefile, not for the current container: i.e. we'll see deletes before we come across cells we are to delete. Presumption is that the memstore#kvset is processed before memstore#snapshot and so on.

Parameters:

row - The row key of the targeted row.

Returns:

Found keyvalue or null if none found.

Throws:

IOException

getFileSystem

org.apache.hadoop.fs.FileSystem getFileSystem()

createWriterInTmp

StoreFile.Writer createWriterInTmp(long maxKeyCount,
                                   Compression.Algorithm compression,
                                   boolean isCompaction,
                                   boolean includeMVCCReadpoint,
                                   boolean includesTags)
                                   throws IOException

Parameters:

maxKeyCount -

compression - Compression algorithm to use

isCompaction - whether we are creating a new file in a compaction

includeMVCCReadpoint - whether we should out the MVCC readpoint

Returns:

Writer for a new StoreFile in the tmp dir.

Throws:

IOException

createWriterInTmp

StoreFile.Writer createWriterInTmp(long maxKeyCount,
                                   Compression.Algorithm compression,
                                   boolean isCompaction,
                                   boolean includeMVCCReadpoint,
                                   boolean includesTags,
                                   boolean shouldDropBehind)
                                   throws IOException

Parameters:

maxKeyCount -

compression - Compression algorithm to use

isCompaction - whether we are creating a new file in a compaction

includeMVCCReadpoint - whether we should out the MVCC readpoint

shouldDropBehind - should the writer drop caches behind writes

Returns:

Writer for a new StoreFile in the tmp dir.

Throws:

IOException

throttleCompaction

boolean throttleCompaction(long compactionSize)

getCompactionProgress

CompactionProgress getCompactionProgress()

getter for CompactionProgress object

Returns:

CompactionProgress object; can be null

requestCompaction

CompactionContext requestCompaction()
                                    throws IOException

Throws:

IOException

requestCompaction

@Deprecated
CompactionContext requestCompaction(int priority,
                                               CompactionRequest baseRequest)
                                    throws IOException

Deprecated. see requestCompaction(int, CompactionRequest, User)

Throws:

IOException

requestCompaction

CompactionContext requestCompaction(int priority,
                                    CompactionRequest baseRequest,
                                    User user)
                                    throws IOException

Throws:

IOException

cancelRequestedCompaction

void cancelRequestedCompaction(CompactionContext compaction)

compact

@Deprecated
List<StoreFile> compact(CompactionContext compaction,
                                   CompactionThroughputController throughputController)
                        throws IOException

Deprecated. see compact(CompactionContext, CompactionThroughputController, User)

Throws:

IOException

compact

List<StoreFile> compact(CompactionContext compaction,
                        CompactionThroughputController throughputController,
                        User user)
                        throws IOException

Throws:

IOException

isMajorCompaction

boolean isMajorCompaction()
                          throws IOException

Returns:

true if we should run a major compaction.

Throws:

IOException

triggerMajorCompaction

void triggerMajorCompaction()

needsCompaction

boolean needsCompaction()

See if there's too much store files in this store

Returns:

true if number of store files is greater than the number defined in minFilesToCompact

getCompactPriority

int getCompactPriority()

createFlushContext

org.apache.hadoop.hbase.regionserver.StoreFlushContext createFlushContext(long cacheFlushId)

completeCompactionMarker

void completeCompactionMarker(org.apache.hadoop.hbase.protobuf.generated.WALProtos.CompactionDescriptor compaction)
                              throws IOException

Call to complete a compaction. Its for the case where we find in the WAL a compaction that was not finished. We could find one recovering a WAL after a regionserver crash. See HBASE-2331.

Parameters:

compaction -

Throws:

IOException

canSplit

boolean canSplit()

getSplitPoint

byte[] getSplitPoint()

Determines if Store should be split

Returns:

byte[] if store should be split, null otherwise.

assertBulkLoadHFileOk

void assertBulkLoadHFileOk(org.apache.hadoop.fs.Path srcPath)
                           throws IOException

This throws a WrongRegionException if the HFile does not fit in this region, or an InvalidHFileException if the HFile is not valid.

Throws:

IOException

bulkLoadHFile

void bulkLoadHFile(String srcPathStr,
                   long sequenceId)
                   throws IOException

This method should only be called from HRegion. It is assumed that the ranges of values in the HFile fit within the stores assigned region. (assertBulkLoadHFileOk checks this)

Parameters:

srcPathStr -

sequenceId - sequence Id associated with the HFile

Throws:

IOException

hasReferences

boolean hasReferences()

Returns:

true if the store has any underlying reference files to older HFiles

getMemStoreSize

long getMemStoreSize()

Returns:

The size of this store's memstore, in bytes

getFlushableSize

long getFlushableSize()

Returns:

The amount of memory we could flush from this memstore; usually this is equal to getMemStoreSize() unless we are carrying snapshots and then it will be the size of outstanding snapshots.

getFamily

HColumnDescriptor getFamily()

getMaxMemstoreTS

long getMaxMemstoreTS()

Returns:

The maximum memstoreTS in all store files.

getDataBlockEncoder

HFileDataBlockEncoder getDataBlockEncoder()

Returns:

the data block encoder

getLastCompactSize

long getLastCompactSize()

Returns:

aggregate size of all HStores used in the last compaction

getSize

long getSize()

Returns:

aggregate size of HStore

getStorefilesCount

int getStorefilesCount()

Returns:

Count of store files

getMaxStoreFileAge

long getMaxStoreFileAge()

Returns:

Max age of store files in this store

getMinStoreFileAge

long getMinStoreFileAge()

Returns:

Min age of store files in this store

getAvgStoreFileAge

long getAvgStoreFileAge()

Returns:

Average age of store files in this store, 0 if no store files

getNumReferenceFiles

long getNumReferenceFiles()

Returns:

Number of reference files in this store

getNumHFiles

long getNumHFiles()

Returns:

Number of HFiles in this store

getStoreSizeUncompressed

long getStoreSizeUncompressed()

Returns:

The size of the store files, in bytes, uncompressed.

getStorefilesSize

long getStorefilesSize()

Returns:

The size of the store files, in bytes.

getStorefilesIndexSize

long getStorefilesIndexSize()

Returns:

The size of the store file indexes, in bytes.

getTotalStaticIndexSize

long getTotalStaticIndexSize()

Returns the total size of all index blocks in the data block indexes, including the root level, intermediate levels, and the leaf level for multi-level indexes, or just the root level for single-level indexes.

Returns:

the total size of block indexes in the store

getTotalStaticBloomSize

long getTotalStaticBloomSize()

Returns the total byte size of all Bloom filter bit arrays. For compound Bloom filters even the Bloom blocks currently not loaded into the block cache are counted.

Returns:

the total size of all Bloom filters in the store

getCacheConfig

CacheConfig getCacheConfig()

Used for tests.

Returns:

cache configuration for this Store.

getRegionInfo

HRegionInfo getRegionInfo()

Returns:

the parent region info hosting this store

getCoprocessorHost

RegionCoprocessorHost getCoprocessorHost()

areWritesEnabled

boolean areWritesEnabled()

getSmallestReadPoint

long getSmallestReadPoint()

Returns:

The smallest mvcc readPoint across all the scanners in this region. Writes older than this readPoint, are included in every read operation.

getColumnFamilyName

String getColumnFamilyName()

getTableName

TableName getTableName()

getFlushedCellsCount

long getFlushedCellsCount()

Returns:

The number of cells flushed to disk

getFlushedCellsSize

long getFlushedCellsSize()

Returns:

The total size of data flushed to disk, in bytes

getCompactedCellsCount

long getCompactedCellsCount()

Returns:

The number of cells processed during minor compactions

getCompactedCellsSize

long getCompactedCellsSize()

Returns:

The total amount of data processed during minor compactions, in bytes

getMajorCompactedCellsCount

long getMajorCompactedCellsCount()

Returns:

The number of cells processed during major compactions

getMajorCompactedCellsSize

long getMajorCompactedCellsSize()

Returns:

The total amount of data processed during major compactions, in bytes

addChangedReaderObserver

void addChangedReaderObserver(ChangedReadersObserver o)

deleteChangedReaderObserver

void deleteChangedReaderObserver(ChangedReadersObserver o)

hasTooManyStoreFiles

boolean hasTooManyStoreFiles()

Returns:

Whether this store has too many store files.

getCompactionPressure

double getCompactionPressure()

This value can represent the degree of emergency of compaction for this store. It should be greater than or equal to 0.0, any value greater than 1.0 means we have too many store files.

• if getStorefilesCount <= getMinFilesToCompact, return 0.0
• return (getStorefilesCount - getMinFilesToCompact) / (blockingFileCount - getMinFilesToCompact)

And for striped stores, we should calculate this value by the files in each stripe separately and return the maximum value.

It is similar to getCompactPriority() except that it is more suitable to use in a linear formula.