@InterfaceAudience.Private public class HRegion extends Object implements HeapSize, PropagatingConfigurationObserver
We maintain multiple HStores for a single HRegion.
An Store is a set of rows with some column data; together, they make up all the data for the rows.
Each HRegion has a 'startKey' and 'endKey'.
The first is inclusive, the second is exclusive (except for the final region) The endKey of region 0 is the same as startKey for region 1 (if it exists). The startKey for the first region is null. The endKey for the final region is null.
Locking at the HRegion level serves only one purpose: preventing the region from being closed (and consequently split) while other operations are ongoing. Each row level operation obtains both a row lock and a region read lock for the duration of the operation. While a scanner is being constructed, getScanner holds a read lock. If the scanner is successfully constructed, it holds a read lock until it is closed. A close takes out a write lock and consequently will block for ongoing operations and will block new operations from starting while the close is in progress.
An HRegion is defined by its table and its key extent.
It consists of at least one Store. The number of Stores should be configurable, so that data which is accessed together is stored in the same Store. Right now, we approximate that by building a single Store for each column family. (This config info will be communicated via the tabledesc.)
The HTableDescriptor contains metainfo about the HRegion's table. regionName is a unique identifier for this HRegion. (startKey, endKey] defines the keyspace for this HRegion.
Modifier and Type | Class and Description |
---|---|
static interface |
HRegion.BulkLoadListener
Listener class to enable callers of
bulkLoadHFile() to perform any necessary
pre/post processing of a given bulkload call
|
static class |
HRegion.FlushResult
Objects from this class are created when flushing to describe all the different states that
that method ends up in.
|
static class |
HRegion.Operation
Operation enum is used in
startRegionOperation() to provide operation context for
startRegionOperation to possibly invoke different checks before any region operations. |
static class |
HRegion.RowLock
Row lock held by a given thread.
|
Modifier and Type | Field and Description |
---|---|
protected org.apache.hadoop.conf.Configuration |
conf |
static long |
DEEP_OVERHEAD |
static int |
DEFAULT_CACHE_FLUSH_INTERVAL
Default interval for the memstore flush
|
static long |
DEFAULT_FLUSH_PER_CHANGES |
static long |
FIXED_OVERHEAD |
protected long |
lastFlushSeqId
The sequence id of the last flush on this region.
|
static String |
LOAD_CFS_ON_DEMAND_CONFIG_KEY |
static org.apache.commons.logging.Log |
LOG |
static long |
MAX_FLUSH_PER_CHANGES
The following MAX_FLUSH_PER_CHANGES is large enough because each KeyValue has 20+ bytes
overhead.
|
static String |
MEMSTORE_FLUSH_PER_CHANGES
Conf key to force a flush if there are already enough changes for one region in memstore
|
static String |
MEMSTORE_PERIODIC_FLUSH_INTERVAL
Conf key for the periodic flush interval
|
AtomicLong |
memstoreSize |
protected Map<byte[],Store> |
stores |
Constructor and Description |
---|
HRegion(HRegionFileSystem fs,
WAL wal,
org.apache.hadoop.conf.Configuration confParam,
HTableDescriptor htd,
RegionServerServices rsServices)
HRegion constructor.
|
HRegion(org.apache.hadoop.fs.Path tableDir,
WAL wal,
org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.conf.Configuration confParam,
HRegionInfo regionInfo,
HTableDescriptor htd,
RegionServerServices rsServices)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
long |
addAndGetGlobalMemstoreSize(long memStoreSize)
Increase the size of mem store in this region and the size of global mem
store
|
static void |
addRegionToMETA(HRegion meta,
HRegion r)
Inserts a new region's meta information into the passed
meta region. |
void |
addRegionToSnapshot(HBaseProtos.SnapshotDescription desc,
ForeignExceptionSnare exnSnare)
Complete taking the snapshot on the region.
|
Result |
append(Append append) |
Result |
append(Append append,
long nonceGroup,
long nonce)
Perform one or more append operations on a row.
|
boolean |
areWritesEnabled() |
OperationStatus[] |
batchMutate(Mutation[] mutations) |
OperationStatus[] |
batchMutate(Mutation[] mutations,
long nonceGroup,
long nonce)
Perform a batch of mutations.
|
OperationStatus[] |
batchReplay(WALSplitter.MutationReplay[] mutations,
long replaySeqId)
Replay a batch of mutations.
|
boolean |
bulkLoadHFiles(List<Pair<byte[],String>> familyPaths,
boolean assignSeqId) |
boolean |
bulkLoadHFiles(List<Pair<byte[],String>> familyPaths,
boolean assignSeqId,
HRegion.BulkLoadListener bulkLoadListener)
Attempts to atomically load a group of hfiles.
|
boolean |
checkAndMutate(byte[] row,
byte[] family,
byte[] qualifier,
CompareFilter.CompareOp compareOp,
ByteArrayComparable comparator,
Mutation w,
boolean writeToWAL) |
boolean |
checkAndRowMutate(byte[] row,
byte[] family,
byte[] qualifier,
CompareFilter.CompareOp compareOp,
ByteArrayComparable comparator,
RowMutations rm,
boolean writeToWAL) |
protected void |
checkReadOnly() |
protected void |
checkReadsEnabled() |
byte[] |
checkSplit()
Return the splitpoint.
|
Map<byte[],List<StoreFile>> |
close()
Close down this HRegion.
|
Map<byte[],List<StoreFile>> |
close(boolean abort)
Close down this HRegion.
|
static void |
closeHRegion(HRegion r)
This will do the necessary cleanup a call to
createHRegion(HRegionInfo, Path, Configuration, HTableDescriptor)
requires. |
void |
closeRegionOperation()
Closes the lock.
|
void |
closeRegionOperation(HRegion.Operation operation)
Closes the lock.
|
boolean |
compact(CompactionContext compaction,
Store store) |
void |
compactStores()
This is a helper function that compact all the stores synchronously
It is used by utilities and testing
|
void |
compactStores(boolean majorCompaction)
This is a helper function that compact all the stores synchronously
It is used by utilities and testing
|
static HDFSBlocksDistribution |
computeHDFSBlocksDistribution(org.apache.hadoop.conf.Configuration conf,
HTableDescriptor tableDescriptor,
HRegionInfo regionInfo)
This is a helper function to compute HDFS block distribution on demand
|
static HDFSBlocksDistribution |
computeHDFSBlocksDistribution(org.apache.hadoop.conf.Configuration conf,
HTableDescriptor tableDescriptor,
HRegionInfo regionInfo,
org.apache.hadoop.fs.Path tablePath)
This is a helper function to compute HDFS block distribution on demand
|
static HRegion |
createHRegion(HRegionInfo info,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.conf.Configuration conf,
HTableDescriptor hTableDescriptor)
Convenience method creating new HRegions.
|
static HRegion |
createHRegion(HRegionInfo info,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.conf.Configuration conf,
HTableDescriptor hTableDescriptor,
WAL wal) |
static HRegion |
createHRegion(HRegionInfo info,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.conf.Configuration conf,
HTableDescriptor hTableDescriptor,
WAL wal,
boolean initialize)
Convenience method creating new HRegions.
|
static HRegion |
createHRegion(HRegionInfo info,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.conf.Configuration conf,
HTableDescriptor hTableDescriptor,
WAL wal,
boolean initialize,
boolean ignoreWAL)
Convenience method creating new HRegions.
|
static HRegion |
createHRegion(HRegionInfo info,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.fs.Path tableDir,
org.apache.hadoop.conf.Configuration conf,
HTableDescriptor hTableDescriptor,
WAL wal,
boolean initialize,
boolean ignoreWAL)
Convenience method creating new HRegions.
|
void |
delete(Delete delete) |
void |
deregisterChildren(ConfigurationManager manager)
Needs to be called to deregister the children from the manager.
|
protected void |
doRegionCompactionPrep() |
boolean |
equals(Object o) |
Message |
execService(RpcController controller,
ClientProtos.CoprocessorServiceCall call)
Executes a single protocol buffer coprocessor endpoint
Service method using
the registered protocol handlers. |
HRegion.FlushResult |
flushcache()
Flush the cache.
|
Result |
get(Get get) |
List<Cell> |
get(Get get,
boolean withCoprocessor) |
long |
getBlockedRequestsCount() |
Result |
getClosestRowBefore(byte[] row,
byte[] family)
Return all the data for the row that matches row exactly,
or the one that immediately precedes it, at or immediately before
ts.
|
AdminProtos.GetRegionInfoResponse.CompactionState |
getCompactionState() |
int |
getCompactPriority() |
KeyValue.KVComparator |
getComparator() |
RegionCoprocessorHost |
getCoprocessorHost() |
protected Durability |
getEffectiveDurability(Durability d)
Returns effective durability from the passed durability and
the table descriptor.
|
byte[] |
getEndKey() |
org.apache.hadoop.fs.FileSystem |
getFilesystem() |
HDFSBlocksDistribution |
getHDFSBlocksDistribution()
This function will return the HDFS blocks distribution based on the data
captured when HFile is created
|
long |
getLargestHStoreSize() |
long |
getLastFlushTime() |
Map<byte[],Long> |
getMaxStoreSeqIdForLogReplay()
Gets max sequence ids of stores that was read from storage when this region was opened.
|
AtomicLong |
getMemstoreSize() |
MetricsRegion |
getMetrics() |
MultiVersionConsistencyControl |
getMVCC() |
long |
getOpenSeqNum()
Gets the latest sequence number that was read from storage when this region was opened.
|
long |
getReadpoint(IsolationLevel isolationLevel) |
static org.apache.hadoop.fs.Path |
getRegionDir(org.apache.hadoop.fs.Path rootdir,
HRegionInfo info)
Deprecated.
|
static org.apache.hadoop.fs.Path |
getRegionDir(org.apache.hadoop.fs.Path tabledir,
String name)
Deprecated.
|
HRegionFileSystem |
getRegionFileSystem() |
long |
getRegionId() |
HRegionInfo |
getRegionInfo() |
byte[] |
getRegionName() |
String |
getRegionNameAsString() |
ClientProtos.RegionLoadStats |
getRegionStats() |
HRegion.RowLock |
getRowLock(byte[] row)
Acquires a lock on the given row.
|
HRegion.RowLock |
getRowLock(byte[] row,
boolean waitForLock)
Tries to acquire a lock on the given row.
|
protected HRegion.RowLock |
getRowLockInternal(byte[] row,
boolean waitForLock)
A version of getRowLock(byte[], boolean) to use when a region operation has already been
started (the calling thread has already acquired the region-close-guard lock).
|
RegionScanner |
getScanner(Scan scan)
Return an iterator that scans over the HRegion, returning the indicated
columns and rows specified by the
Scan . |
protected RegionScanner |
getScanner(Scan scan,
List<KeyValueScanner> additionalScanners) |
AtomicLong |
getSequenceId()
Do not change this sequence id.
|
long |
getSmallestReadPoint() |
RegionSplitPolicy |
getSplitPolicy() |
byte[] |
getStartKey() |
Store |
getStore(byte[] column)
Return HStore instance.
|
List<String> |
getStoreFileList(byte[][] columns)
Return list of storeFiles for the set of CFs.
|
protected ThreadPoolExecutor |
getStoreFileOpenAndCloseThreadPool(String threadNamePrefix) |
protected ThreadPoolExecutor |
getStoreOpenAndCloseThreadPool(String threadNamePrefix) |
Map<byte[],Store> |
getStores() |
HTableDescriptor |
getTableDesc() |
WAL |
getWAL() |
int |
hashCode() |
boolean |
hasReferences() |
long |
heapSize() |
Result |
increment(Increment increment) |
Result |
increment(Increment increment,
long nonceGroup,
long nonce)
Perform one or more increment operations on a row.
|
long |
initialize()
Deprecated.
use HRegion.createHRegion() or HRegion.openHRegion()
|
protected HStore |
instantiateHStore(HColumnDescriptor family) |
protected RegionScanner |
instantiateRegionScanner(Scan scan,
List<KeyValueScanner> additionalScanners) |
protected HRegion.FlushResult |
internalFlushcache(MonitoredTask status)
Flush the memstore.
|
protected HRegion.FlushResult |
internalFlushcache(WAL wal,
long myseqid,
MonitoredTask status) |
boolean |
isAvailable() |
boolean |
isClosed() |
boolean |
isClosing() |
boolean |
isLoadingCfsOnDemandDefault() |
boolean |
isMergeable() |
boolean |
isRecovering() |
boolean |
isSplittable() |
static void |
main(String[] args)
Facility for dumping and compacting catalog tables.
|
static HRegion |
merge(HRegion a,
HRegion b)
Merge two regions whether they are adjacent or not.
|
static HRegion |
mergeAdjacent(HRegion srcA,
HRegion srcB)
Merge two HRegions.
|
void |
mutateRow(RowMutations rm) |
void |
mutateRowsWithLocks(Collection<Mutation> mutations,
Collection<byte[]> rowsToLock)
Perform atomic mutations within the region w/o nonces.
|
void |
mutateRowsWithLocks(Collection<Mutation> mutations,
Collection<byte[]> rowsToLock,
long nonceGroup,
long nonce)
Perform atomic mutations within the region.
|
void |
onConfigurationChange(org.apache.hadoop.conf.Configuration conf)
This method would be called by the
ConfigurationManager
object when the Configuration object is reloaded from disk. |
protected HRegion |
openHRegion(CancelableProgressable reporter)
Open HRegion.
|
static HRegion |
openHRegion(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path rootDir,
HRegionInfo info,
HTableDescriptor htd,
WAL wal)
Open a Region.
|
static HRegion |
openHRegion(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path rootDir,
HRegionInfo info,
HTableDescriptor htd,
WAL wal,
RegionServerServices rsServices,
CancelableProgressable reporter)
Open a Region.
|
static HRegion |
openHRegion(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.fs.Path tableDir,
HRegionInfo info,
HTableDescriptor htd,
WAL wal,
RegionServerServices rsServices,
CancelableProgressable reporter)
Open a Region.
|
static HRegion |
openHRegion(HRegion other,
CancelableProgressable reporter)
Useful when reopening a closed region (normally for unit tests)
|
static HRegion |
openHRegion(HRegionInfo info,
HTableDescriptor htd,
WAL wal,
org.apache.hadoop.conf.Configuration conf)
Open a Region.
|
static HRegion |
openHRegion(HRegionInfo info,
HTableDescriptor htd,
WAL wal,
org.apache.hadoop.conf.Configuration conf,
RegionServerServices rsServices,
CancelableProgressable reporter)
Open a Region.
|
static HRegion |
openHRegion(org.apache.hadoop.fs.Path rootDir,
HRegionInfo info,
HTableDescriptor htd,
WAL wal,
org.apache.hadoop.conf.Configuration conf)
Open a Region.
|
static HRegion |
openHRegion(org.apache.hadoop.fs.Path rootDir,
HRegionInfo info,
HTableDescriptor htd,
WAL wal,
org.apache.hadoop.conf.Configuration conf,
RegionServerServices rsServices,
CancelableProgressable reporter)
Open a Region.
|
protected void |
prepareToSplit()
Give the region a chance to prepare before it is split.
|
void |
processRowsWithLocks(RowProcessor<?,?> processor,
long nonceGroup,
long nonce)
Performs atomic multiple reads and writes on a given row.
|
void |
processRowsWithLocks(RowProcessor<?,?> processor,
long timeout,
long nonceGroup,
long nonce)
Performs atomic multiple reads and writes on a given row.
|
void |
put(Put put) |
void |
registerChildren(ConfigurationManager manager)
Needs to be called to register the children to the manager.
|
boolean |
registerService(Service instance)
Registers a new protocol buffer
Service subclass as a coprocessor endpoint to
be available for handling
execService(com.google.protobuf.RpcController,
org.apache.hadoop.hbase.protobuf.generated.ClientProtos.CoprocessorServiceCall) } calls. |
void |
releaseRowLocks(List<HRegion.RowLock> rowLocks)
If the given list of row locks is not null, releases all locks.
|
protected long |
replayRecoveredEditsIfAny(org.apache.hadoop.fs.Path regiondir,
Map<byte[],Long> maxSeqIdInStores,
CancelableProgressable reporter,
MonitoredTask status)
Read the edits put under this region by wal splitting process.
|
void |
reportCompactionRequestEnd(boolean isMajor,
int numFiles,
long filesSizeCompacted) |
void |
reportCompactionRequestStart(boolean isMajor) |
protected boolean |
restoreEdit(Store s,
Cell cell)
Used by tests
|
static boolean |
rowIsInRange(HRegionInfo info,
byte[] row)
Determines if the specified row is within the row range specified by the
specified HRegionInfo
|
void |
setCoprocessorHost(RegionCoprocessorHost coprocessorHost) |
void |
setReadsEnabled(boolean readsEnabled) |
void |
setRecovering(boolean newState)
Reset recovering state of current region
|
void |
startRegionOperation()
This method needs to be called before any public call that reads or
modifies data.
|
protected void |
startRegionOperation(HRegion.Operation op) |
void |
syncWal()
Explictly sync wal
|
String |
toString() |
void |
waitForFlushesAndCompactions()
Wait for all current flushes and compactions of the region to complete.
|
public static final org.apache.commons.logging.Log LOG
public static final String LOAD_CFS_ON_DEMAND_CONFIG_KEY
protected volatile long lastFlushSeqId
public final AtomicLong memstoreSize
protected final org.apache.hadoop.conf.Configuration conf
public static final String MEMSTORE_PERIODIC_FLUSH_INTERVAL
public static final int DEFAULT_CACHE_FLUSH_INTERVAL
public static final String MEMSTORE_FLUSH_PER_CHANGES
public static final long DEFAULT_FLUSH_PER_CHANGES
public static final long MAX_FLUSH_PER_CHANGES
public static final long FIXED_OVERHEAD
public static final long DEEP_OVERHEAD
@Deprecated public HRegion(org.apache.hadoop.fs.Path tableDir, WAL wal, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration confParam, HRegionInfo regionInfo, HTableDescriptor htd, RegionServerServices rsServices)
createHRegion(org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, org.apache.hadoop.hbase.HTableDescriptor)
or openHRegion(org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.hbase.HTableDescriptor, org.apache.hadoop.hbase.wal.WAL, org.apache.hadoop.conf.Configuration)
method.tableDir
- qualified path of directory where region should be located,
usually the table directory.wal
- The WAL is the outbound log for any updates to the HRegion
The wal file is a logfile from the previous execution that's
custom-computed for this HRegion. The HRegionServer computes and sorts the
appropriate wal info for this HRegion. If there is a previous wal file
(implying that the HRegion has been written-to before), then read it from
the supplied path.fs
- is the filesystem.confParam
- is global configuration settings.regionInfo
- - HRegionInfo that describes the region
is new), then read them from the supplied path.htd
- the table descriptorrsServices
- reference to RegionServerServices
or nullpublic HRegion(HRegionFileSystem fs, WAL wal, org.apache.hadoop.conf.Configuration confParam, HTableDescriptor htd, RegionServerServices rsServices)
createHRegion(org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, org.apache.hadoop.hbase.HTableDescriptor)
or openHRegion(org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.hbase.HTableDescriptor, org.apache.hadoop.hbase.wal.WAL, org.apache.hadoop.conf.Configuration)
method.fs
- is the filesystem.wal
- The WAL is the outbound log for any updates to the HRegion
The wal file is a logfile from the previous execution that's
custom-computed for this HRegion. The HRegionServer computes and sorts the
appropriate wal info for this HRegion. If there is a previous wal file
(implying that the HRegion has been written-to before), then read it from
the supplied path.confParam
- is global configuration settings.htd
- the table descriptorrsServices
- reference to RegionServerServices
or nullpublic long getBlockedRequestsCount()
public long getSmallestReadPoint()
@Deprecated public long initialize() throws IOException
IOException
- epublic boolean hasReferences()
public HDFSBlocksDistribution getHDFSBlocksDistribution()
public static HDFSBlocksDistribution computeHDFSBlocksDistribution(org.apache.hadoop.conf.Configuration conf, HTableDescriptor tableDescriptor, HRegionInfo regionInfo) throws IOException
conf
- configurationtableDescriptor
- HTableDescriptor of the tableregionInfo
- encoded name of the regionIOException
public static HDFSBlocksDistribution computeHDFSBlocksDistribution(org.apache.hadoop.conf.Configuration conf, HTableDescriptor tableDescriptor, HRegionInfo regionInfo, org.apache.hadoop.fs.Path tablePath) throws IOException
conf
- configurationtableDescriptor
- HTableDescriptor of the tableregionInfo
- encoded name of the regiontablePath
- the table directoryIOException
public AtomicLong getMemstoreSize()
public long addAndGetGlobalMemstoreSize(long memStoreSize)
public HRegionInfo getRegionInfo()
public MetricsRegion getMetrics()
public boolean isClosed()
public boolean isClosing()
public void setRecovering(boolean newState)
public boolean isRecovering()
public boolean isAvailable()
public boolean isSplittable()
public boolean isMergeable()
public boolean areWritesEnabled()
public MultiVersionConsistencyControl getMVCC()
public long getReadpoint(IsolationLevel isolationLevel)
public boolean isLoadingCfsOnDemandDefault()
public Map<byte[],List<StoreFile>> close() throws IOException
This method could take some time to execute, so don't call it from a time-sensitive thread.
IOException
- epublic Map<byte[],List<StoreFile>> close(boolean abort) throws IOException
abort
- true if server is aborting (only during testing)IOException
- epublic void waitForFlushesAndCompactions()
Exposed for TESTING.
protected ThreadPoolExecutor getStoreOpenAndCloseThreadPool(String threadNamePrefix)
protected ThreadPoolExecutor getStoreFileOpenAndCloseThreadPool(String threadNamePrefix)
public byte[] getStartKey()
public byte[] getEndKey()
public long getRegionId()
public byte[] getRegionName()
public String getRegionNameAsString()
public HTableDescriptor getTableDesc()
public WAL getWAL()
public org.apache.hadoop.fs.FileSystem getFilesystem()
FileSystem
being used by this regionpublic HRegionFileSystem getRegionFileSystem()
HRegionFileSystem
used by this regionpublic long getLastFlushTime()
public long getLargestHStoreSize()
public KeyValue.KVComparator getComparator()
protected void doRegionCompactionPrep() throws IOException
IOException
public void compactStores(boolean majorCompaction) throws IOException
majorCompaction
- True to force a major compaction regardless of thresholdsIOException
- epublic void compactStores() throws IOException
IOException
- epublic boolean compact(CompactionContext compaction, Store store) throws IOException
IOException
public HRegion.FlushResult flushcache() throws IOException
This method may block for some time, so it should not be called from a time-sensitive thread.
IOException
- general io exceptionsDroppedSnapshotException
- Thrown when replay of wal is required
because a Snapshot was not properly persisted.protected HRegion.FlushResult internalFlushcache(MonitoredTask status) throws IOException
This method may block for some time. Every time you call it, we up the regions sequence id even if we don't flush; i.e. the returned region id will be at least one larger than the last edit applied to this region. The returned id does not refer to an actual edit. The returned id can be used for say installing a bulk loaded file just ahead of the last hfile that was the result of this flush, etc.
IOException
- general io exceptionsDroppedSnapshotException
- Thrown when replay of wal is required
because a Snapshot was not properly persisted.protected HRegion.FlushResult internalFlushcache(WAL wal, long myseqid, MonitoredTask status) throws IOException
wal
- Null if we're NOT to go via wal.myseqid
- The seqid to use if wal
is null writing out flush file.IOException
internalFlushcache(MonitoredTask)
public Result getClosestRowBefore(byte[] row, byte[] family) throws IOException
row
- row keyfamily
- column family to find onIOException
- read exceptionspublic RegionScanner getScanner(Scan scan) throws IOException
Scan
.
This Iterator must be closed by the caller.
scan
- configured Scan
IOException
- read exceptionsprotected RegionScanner getScanner(Scan scan, List<KeyValueScanner> additionalScanners) throws IOException
IOException
protected RegionScanner instantiateRegionScanner(Scan scan, List<KeyValueScanner> additionalScanners) throws IOException
IOException
public void delete(Delete delete) throws IOException
delete
- delete objectIOException
- read exceptionspublic void put(Put put) throws IOException
IOException
public OperationStatus[] batchMutate(Mutation[] mutations, long nonceGroup, long nonce) throws IOException
mutations
- the list of mutationsIOException
public OperationStatus[] batchMutate(Mutation[] mutations) throws IOException
IOException
public OperationStatus[] batchReplay(WALSplitter.MutationReplay[] mutations, long replaySeqId) throws IOException
mutations
- mutations to replay.replaySeqId
- SeqId for current mutationsIOException
protected Durability getEffectiveDurability(Durability d)
public boolean checkAndMutate(byte[] row, byte[] family, byte[] qualifier, CompareFilter.CompareOp compareOp, ByteArrayComparable comparator, Mutation w, boolean writeToWAL) throws IOException
IOException
public boolean checkAndRowMutate(byte[] row, byte[] family, byte[] qualifier, CompareFilter.CompareOp compareOp, ByteArrayComparable comparator, RowMutations rm, boolean writeToWAL) throws IOException
IOException
public void addRegionToSnapshot(HBaseProtos.SnapshotDescription desc, ForeignExceptionSnare exnSnare) throws IOException
ForeignExceptionSnare
arg. (In the future other cancellable HRegion methods could eventually add a
ForeignExceptionSnare
, or we could do something fancier).desc
- snapshot description objectexnSnare
- ForeignExceptionSnare that captures external exceptions in case we need to
bail out. This is allowed to be null and will just be ignored in that case.IOException
- if there is an external or internal error causing the snapshot to failprotected void checkReadOnly() throws IOException
IOException
- Throws exception if region is in read-only mode.protected void checkReadsEnabled() throws IOException
IOException
public void setReadsEnabled(boolean readsEnabled)
protected long replayRecoveredEditsIfAny(org.apache.hadoop.fs.Path regiondir, Map<byte[],Long> maxSeqIdInStores, CancelableProgressable reporter, MonitoredTask status) throws UnsupportedEncodingException, IOException
We can ignore any wal message that has a sequence ID that's equal to or lower than minSeqId. (Because we know such messages are already reflected in the HFiles.)
While this is running we are putting pressure on memory yet we are outside of our usual accounting because we are not yet an onlined region (this stuff is being run as part of Region initialization). This means that if we're up against global memory limits, we'll not be flagged to flush because we are not online. We can't be flushed by usual mechanisms anyways; we're not yet online so our relative sequenceids are not yet aligned with WAL sequenceids -- not till we come up online, post processing of split edits.
But to help relieve memory pressure, at least manage our own heap size flushing if are in excess of per-region limits. Flushing, though, we have to be careful and avoid using the regionserver/wal sequenceid. Its running on a different line to whats going on in here in this region context so if we crashed replaying these edits, but in the midst had a flush that used the regionserver wal with a sequenceid in excess of whats going on in here in this region and with its split editlogs, then we could miss edits the next time we go to recover. So, we have to flush inline, using seqids that make sense in a this single region context only -- until we online.
maxSeqIdInStores
- Any edit found in split editlogs needs to be in excess of
the maxSeqId for the store to be applied, else its skipped.minSeqId
if nothing added from editlogs.UnsupportedEncodingException
IOException
protected boolean restoreEdit(Store s, Cell cell)
s
- Store to add edit too.cell
- Cell to add.protected HStore instantiateHStore(HColumnDescriptor family) throws IOException
IOException
public Store getStore(byte[] column)
column
- Name of column family hosted by this region.column
.
TODO: Make this lookup faster.public List<String> getStoreFileList(byte[][] columns) throws IllegalArgumentException
IllegalArgumentException
public HRegion.RowLock getRowLock(byte[] row, boolean waitForLock) throws IOException
waitForLock
- if true, will block until the lock is available.
Otherwise, just tries to obtain the lock and returns
false if unavailable.IOException
- if waitForLock was true and the lock could not be acquired after waitingprotected HRegion.RowLock getRowLockInternal(byte[] row, boolean waitForLock) throws IOException
IOException
public HRegion.RowLock getRowLock(byte[] row) throws IOException
IOException
- if the lock could not be acquired after waitingpublic void releaseRowLocks(List<HRegion.RowLock> rowLocks)
public boolean bulkLoadHFiles(List<Pair<byte[],String>> familyPaths, boolean assignSeqId) throws IOException
IOException
public boolean bulkLoadHFiles(List<Pair<byte[],String>> familyPaths, boolean assignSeqId, HRegion.BulkLoadListener bulkLoadListener) throws IOException
familyPaths
- List of PairbulkLoadListener
- Internal hooks enabling massaging/preparation of a
file about to be bulk loadedIOException
- if failed unrecoverably.public static HRegion createHRegion(HRegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, HTableDescriptor hTableDescriptor) throws IOException
WAL
for the created region. It
needs to be closed explicitly. Use getWAL()
to get
access. When done with a region created using this method, you will
need to explicitly close the WAL
it created too; it will not be
done for you. Not closing the wal will leave at least a daemon thread
running. Call closeHRegion(HRegion)
and it will do
necessary cleanup for you.info
- Info for region to create.rootDir
- Root directory for HBase instanceIOException
public static void closeHRegion(HRegion r) throws IOException
createHRegion(HRegionInfo, Path, Configuration, HTableDescriptor)
requires. This method will close the region and then close its
associated WAL
file. You can still use it if you call the other createHRegion,
the one that takes an WAL
instance but don't be surprised by the
call to the WAL.close()
on the WAL
the
HRegion was carrying.IOException
public static HRegion createHRegion(HRegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, HTableDescriptor hTableDescriptor, WAL wal, boolean initialize) throws IOException
WAL
for the created region needs to be closed explicitly.
Use getWAL()
to get access.info
- Info for region to create.rootDir
- Root directory for HBase instancewal
- shared WALinitialize
- - true to initialize the regionIOException
public static HRegion createHRegion(HRegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, HTableDescriptor hTableDescriptor, WAL wal, boolean initialize, boolean ignoreWAL) throws IOException
WAL
for the created region needs to be closed
explicitly, if it is not null.
Use getWAL()
to get access.info
- Info for region to create.rootDir
- Root directory for HBase instancewal
- shared WALinitialize
- - true to initialize the regionignoreWAL
- - true to skip generate new wal if it is null, mostly for createTableIOException
public static HRegion createHRegion(HRegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path tableDir, org.apache.hadoop.conf.Configuration conf, HTableDescriptor hTableDescriptor, WAL wal, boolean initialize, boolean ignoreWAL) throws IOException
WAL
for the created region needs to be closed
explicitly, if it is not null.
Use getWAL()
to get access.info
- Info for region to create.rootDir
- Root directory for HBase instancetableDir
- table directorywal
- shared WALinitialize
- - true to initialize the regionignoreWAL
- - true to skip generate new wal if it is null, mostly for createTableIOException
public static HRegion createHRegion(HRegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, HTableDescriptor hTableDescriptor, WAL wal) throws IOException
IOException
public static HRegion openHRegion(HRegionInfo info, HTableDescriptor htd, WAL wal, org.apache.hadoop.conf.Configuration conf) throws IOException
info
- Info for region to be opened.wal
- WAL for region to use. This method will call
WAL#setSequenceNumber(long) passing the result of the call to
HRegion#getMinSequenceId() to ensure the wal id is properly kept
up. HRegionStore does this every time it opens a new region.IOException
public static HRegion openHRegion(HRegionInfo info, HTableDescriptor htd, WAL wal, org.apache.hadoop.conf.Configuration conf, RegionServerServices rsServices, CancelableProgressable reporter) throws IOException
info
- Info for region to be openedhtd
- the table descriptorwal
- WAL for region to use. This method will call
WAL#setSequenceNumber(long) passing the result of the call to
HRegion#getMinSequenceId() to ensure the wal id is properly kept
up. HRegionStore does this every time it opens a new region.conf
- The Configuration object to use.rsServices
- An interface we can request flushes against.reporter
- An interface we can report progress against.IOException
public static HRegion openHRegion(org.apache.hadoop.fs.Path rootDir, HRegionInfo info, HTableDescriptor htd, WAL wal, org.apache.hadoop.conf.Configuration conf) throws IOException
rootDir
- Root directory for HBase instanceinfo
- Info for region to be opened.htd
- the table descriptorwal
- WAL for region to use. This method will call
WAL#setSequenceNumber(long) passing the result of the call to
HRegion#getMinSequenceId() to ensure the wal id is properly kept
up. HRegionStore does this every time it opens a new region.conf
- The Configuration object to use.IOException
public static HRegion openHRegion(org.apache.hadoop.fs.Path rootDir, HRegionInfo info, HTableDescriptor htd, WAL wal, org.apache.hadoop.conf.Configuration conf, RegionServerServices rsServices, CancelableProgressable reporter) throws IOException
rootDir
- Root directory for HBase instanceinfo
- Info for region to be opened.htd
- the table descriptorwal
- WAL for region to use. This method will call
WAL#setSequenceNumber(long) passing the result of the call to
HRegion#getMinSequenceId() to ensure the wal id is properly kept
up. HRegionStore does this every time it opens a new region.conf
- The Configuration object to use.rsServices
- An interface we can request flushes against.reporter
- An interface we can report progress against.IOException
public static HRegion openHRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootDir, HRegionInfo info, HTableDescriptor htd, WAL wal) throws IOException
conf
- The Configuration object to use.fs
- Filesystem to userootDir
- Root directory for HBase instanceinfo
- Info for region to be opened.htd
- the table descriptorwal
- WAL for region to use. This method will call
WAL#setSequenceNumber(long) passing the result of the call to
HRegion#getMinSequenceId() to ensure the wal id is properly kept
up. HRegionStore does this every time it opens a new region.IOException
public static HRegion openHRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootDir, HRegionInfo info, HTableDescriptor htd, WAL wal, RegionServerServices rsServices, CancelableProgressable reporter) throws IOException
conf
- The Configuration object to use.fs
- Filesystem to userootDir
- Root directory for HBase instanceinfo
- Info for region to be opened.htd
- the table descriptorwal
- WAL for region to use. This method will call
WAL#setSequenceNumber(long) passing the result of the call to
HRegion#getMinSequenceId() to ensure the wal id is properly kept
up. HRegionStore does this every time it opens a new region.rsServices
- An interface we can request flushes against.reporter
- An interface we can report progress against.IOException
public static HRegion openHRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path tableDir, HRegionInfo info, HTableDescriptor htd, WAL wal, RegionServerServices rsServices, CancelableProgressable reporter) throws IOException
conf
- The Configuration object to use.fs
- Filesystem to userootDir
- Root directory for HBase instanceinfo
- Info for region to be opened.htd
- the table descriptorwal
- WAL for region to use. This method will call
WAL#setSequenceNumber(long) passing the result of the call to
HRegion#getMinSequenceId() to ensure the wal id is properly kept
up. HRegionStore does this every time it opens a new region.rsServices
- An interface we can request flushes against.reporter
- An interface we can report progress against.IOException
public static HRegion openHRegion(HRegion other, CancelableProgressable reporter) throws IOException
other
- original objectreporter
- An interface we can report progress against.IOException
protected HRegion openHRegion(CancelableProgressable reporter) throws IOException
this
IOException
public static void addRegionToMETA(HRegion meta, HRegion r) throws IOException
meta
region. Used by the HMaster bootstrap code adding
new table to hbase:meta table.meta
- hbase:meta HRegion to be updatedr
- HRegion to add to meta
IOException
@Deprecated public static org.apache.hadoop.fs.Path getRegionDir(org.apache.hadoop.fs.Path tabledir, String name)
tabledir
- qualified path for tablename
- ENCODED region name@Deprecated public static org.apache.hadoop.fs.Path getRegionDir(org.apache.hadoop.fs.Path rootdir, HRegionInfo info)
rootdir
- qualified path of HBase root directoryinfo
- HRegionInfo for the regionpublic static boolean rowIsInRange(HRegionInfo info, byte[] row)
info
- HRegionInfo that specifies the row rangerow
- row to be checkedpublic static HRegion mergeAdjacent(HRegion srcA, HRegion srcB) throws IOException
IOException
public static HRegion merge(HRegion a, HRegion b) throws IOException
a
- region ab
- region bIOException
public Result get(Get get) throws IOException
get
- get objectIOException
- read exceptionspublic List<Cell> get(Get get, boolean withCoprocessor) throws IOException
IOException
public void mutateRow(RowMutations rm) throws IOException
IOException
public void mutateRowsWithLocks(Collection<Mutation> mutations, Collection<byte[]> rowsToLock) throws IOException
mutateRowsWithLocks(Collection, Collection, long, long)
IOException
public void mutateRowsWithLocks(Collection<Mutation> mutations, Collection<byte[]> rowsToLock, long nonceGroup, long nonce) throws IOException
mutations
- The list of mutations to perform.
mutations
can contain operations for multiple rows.
Caller has to ensure that all rows are contained in this region.rowsToLock
- Rows to locknonceGroup
- Optional nonce group of the operation (client Id)nonce
- Optional nonce of the operation (unique random id to ensure "more idempotence")
If multiple rows are locked care should be taken that
rowsToLock
is sorted in order to avoid deadlocks.IOException
public ClientProtos.RegionLoadStats getRegionStats()
public void processRowsWithLocks(RowProcessor<?,?> processor, long nonceGroup, long nonce) throws IOException
processor
- The object defines the reads and writes to a row.nonceGroup
- Optional nonce group of the operation (client Id)nonce
- Optional nonce of the operation (unique random id to ensure "more idempotence")IOException
public void processRowsWithLocks(RowProcessor<?,?> processor, long timeout, long nonceGroup, long nonce) throws IOException
processor
- The object defines the reads and writes to a row.timeout
- The timeout of the processor.process() execution
Use a negative number to switch off the time boundnonceGroup
- Optional nonce group of the operation (client Id)nonce
- Optional nonce of the operation (unique random id to ensure "more idempotence")IOException
public Result append(Append append) throws IOException
IOException
public Result append(Append append, long nonceGroup, long nonce) throws IOException
IOException
public Result increment(Increment increment) throws IOException
IOException
public Result increment(Increment increment, long nonceGroup, long nonce) throws IOException
IOException
public boolean registerService(Service instance)
Service
subclass as a coprocessor endpoint to
be available for handling
execService(com.google.protobuf.RpcController,
org.apache.hadoop.hbase.protobuf.generated.ClientProtos.CoprocessorServiceCall)
} calls.
Only a single instance may be registered per region for a given Service
subclass (the
instances are keyed on Descriptors.ServiceDescriptor.getFullName()
.
After the first registration, subsequent calls with the same service name will fail with
a return value of false
.
instance
- the Service
subclass instance to expose as a coprocessor endpointtrue
if the registration was successful, false
otherwisepublic Message execService(RpcController controller, ClientProtos.CoprocessorServiceCall call) throws IOException
Service
method using
the registered protocol handlers. Service
implementations must be registered via the
registerService(com.google.protobuf.Service)
method before they are available.controller
- an RpcController
implementation to pass to the invoked servicecall
- a CoprocessorServiceCall
instance identifying the service, method,
and parameters for the method invocationMessage
instance containing the method's resultIOException
- if no registered service handler is found or an error
occurs during the invocationregisterService(com.google.protobuf.Service)
protected void prepareToSplit()
public byte[] checkSplit()
public int getCompactPriority()
public RegionCoprocessorHost getCoprocessorHost()
public void setCoprocessorHost(RegionCoprocessorHost coprocessorHost)
coprocessorHost
- the new coprocessor hostpublic void startRegionOperation() throws IOException
IOException
protected void startRegionOperation(HRegion.Operation op) throws IOException
op
- The operation is about to be taken on the regionIOException
public void closeRegionOperation() throws IOException
IOException
public void closeRegionOperation(HRegion.Operation operation) throws IOException
startRegionOperation(Operation)
IOException
public static void main(String[] args) throws IOException
./bin/hbase org.apache.hadoop.hbase.regionserver.HRegion
IOException
public long getOpenSeqNum()
public Map<byte[],Long> getMaxStoreSeqIdForLogReplay()
public AdminProtos.GetRegionInfoResponse.CompactionState getCompactionState()
public void reportCompactionRequestStart(boolean isMajor)
public void reportCompactionRequestEnd(boolean isMajor, int numFiles, long filesSizeCompacted)
public AtomicLong getSequenceId()
sequenceId
comment.public void syncWal() throws IOException
IOException
public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)
ConfigurationManager
object when the Configuration
object is reloaded from disk.onConfigurationChange
in interface ConfigurationObserver
public void registerChildren(ConfigurationManager manager)
registerChildren
in interface PropagatingConfigurationObserver
manager
- : to register topublic void deregisterChildren(ConfigurationManager manager)
deregisterChildren
in interface PropagatingConfigurationObserver
manager
- : to deregister frompublic RegionSplitPolicy getSplitPolicy()
Copyright © 2015 The Apache Software Foundation. All Rights Reserved.