org.apache.hadoop.hbase.master.procedure

Class MasterProcedureScheduler

java.lang.Object

org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler

org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler

All Implemented Interfaces:

ProcedureScheduler

@InterfaceAudience.Private
public class MasterProcedureScheduler
extends AbstractProcedureScheduler

ProcedureScheduler for the Master Procedures. This ProcedureScheduler tries to provide to the ProcedureExecutor procedures that can be executed without having to wait on a lock. Most of the master operations can be executed concurrently, if they are operating on different tables (e.g. two create table procedures can be performed at the same time) or against two different servers; say two servers that crashed at about the same time.

Each procedure should implement an Interface providing information for this queue. For example table related procedures should implement TableProcedureInterface. Each procedure will be pushed in its own queue, and based on the operation type we may make smarter decisions: e.g. we can abort all the operations preceding a delete table, or similar.

Concurrency control

Concurrent access to member variables (tableRunQueue, serverRunQueue, locking, tableMap, serverBuckets) is controlled by schedLock(). This mainly includes:

• AbstractProcedureScheduler.push(Procedure, boolean, boolean): A push will add a Queue back to run-queue when:
  1. Queue was empty before push (so must have been out of run-queue)
  2. Child procedure is added (which means parent procedure holds exclusive lock, and it must have moved Queue out of run-queue)
• AbstractProcedureScheduler.poll(long): A poll will remove a Queue from run-queue when:
  1. Queue becomes empty after poll
  2. Exclusive lock is requested by polled procedure and lock is available (returns the procedure)
  3. Exclusive lock is requested but lock is not available (returns null)
  4. Polled procedure is child of parent holding exclusive lock and the next procedure is not a child
• Namespace/table/region locks: Queue is added back to run-queue when lock being released is:
  1. Exclusive lock
  2. Last shared lock (in case queue was removed because next procedure in queue required exclusive lock)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MasterProcedureScheduler.ServerQueue
static class	MasterProcedureScheduler.TableQueue

Constructor Summary

Constructors

Constructor and Description
MasterProcedureScheduler(org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method and Description
void	clear()
protected void	clearQueue()
void	completionCleanup(Procedure proc)
protected Procedure	dequeue()
String	dumpLocks() For debugging.
protected void	enqueue(Procedure proc, boolean addFront)
LockedResource	getLockResource(LockedResourceType resourceType, String resourceName)
List<LockedResource>	getLocks()
protected void	logLockedResource(LockedResourceType resourceType, String resourceName) Get lock info for a resource of specified type and name and log details
protected boolean	markTableAsDeleted(TableName table, Procedure procedure) Tries to remove the queue and the table-lock of the specified table.
protected boolean	queueHasRunnables()
protected int	queueSize()
boolean	waitNamespaceExclusiveLock(Procedure procedure, String namespace) Suspend the procedure if the specified namespace is already locked.
boolean	waitRegion(Procedure procedure, HRegionInfo regionInfo) Suspend the procedure if the specified region is already locked.
boolean	waitRegions(Procedure procedure, TableName table, HRegionInfo... regionInfo) Suspend the procedure if the specified set of regions are already locked.
boolean	waitServerExclusiveLock(Procedure procedure, ServerName serverName) Try to acquire the exclusive lock on the specified server.
boolean	waitTableExclusiveLock(Procedure procedure, TableName table) Suspend the procedure if the specified table is already locked.
boolean	waitTableSharedLock(Procedure procedure, TableName table) Suspend the procedure if the specified table is already locked.
void	wakeNamespaceExclusiveLock(Procedure procedure, String namespace) Wake the procedures waiting for the specified namespace
void	wakeRegion(Procedure procedure, HRegionInfo regionInfo) Wake the procedures waiting for the specified region
void	wakeRegions(Procedure procedure, TableName table, HRegionInfo... regionInfo) Wake the procedures waiting for the specified regions
void	wakeServerExclusiveLock(Procedure procedure, ServerName serverName) Wake the procedures waiting for the specified server
void	wakeTableExclusiveLock(Procedure procedure, TableName table) Wake the procedures waiting for the specified table
void	wakeTableSharedLock(Procedure procedure, TableName table) Wake the procedures waiting for the specified table
void	yield(Procedure proc)

Methods inherited from class org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler

addBack, addFront, getNullPollCalls, getPollCalls, hasRunnables, poll, poll, poll, push, schedLock, schedUnlock, signalAll, size, start, stop, suspendEvent, waitEvent, waitProcedure, wakeEvent, wakeEvents, wakePollIfNeeded, wakeProcedure, wakeWaitingProcedures

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MasterProcedureScheduler

public MasterProcedureScheduler(org.apache.hadoop.conf.Configuration conf)

Method Detail

yield

public void yield(Procedure proc)

enqueue

protected void enqueue(Procedure proc,
                       boolean addFront)

Specified by:

enqueue in class AbstractProcedureScheduler

queueHasRunnables

protected boolean queueHasRunnables()

Specified by:

queueHasRunnables in class AbstractProcedureScheduler

dequeue

protected Procedure dequeue()

Specified by:

dequeue in class AbstractProcedureScheduler

getLocks

public List<LockedResource> getLocks()

getLockResource

public LockedResource getLockResource(LockedResourceType resourceType,
                                      String resourceName)

clear

public void clear()

clearQueue

protected void clearQueue()

queueSize

protected int queueSize()

Specified by:

queueSize in class AbstractProcedureScheduler

completionCleanup

public void completionCleanup(Procedure proc)

logLockedResource

protected void logLockedResource(LockedResourceType resourceType,
                                 String resourceName)

Get lock info for a resource of specified type and name and log details

waitTableExclusiveLock

public boolean waitTableExclusiveLock(Procedure procedure,
                                      TableName table)

Suspend the procedure if the specified table is already locked. Other operations in the table-queue will be executed after the lock is released.

Parameters:

procedure - the procedure trying to acquire the lock

table - Table to lock

Returns:

true if the procedure has to wait for the table to be available

wakeTableExclusiveLock

public void wakeTableExclusiveLock(Procedure procedure,
                                   TableName table)

Wake the procedures waiting for the specified table

Parameters:

procedure - the procedure releasing the lock

table - the name of the table that has the exclusive lock

waitTableSharedLock

public boolean waitTableSharedLock(Procedure procedure,
                                   TableName table)

Suspend the procedure if the specified table is already locked. other "read" operations in the table-queue may be executed concurrently,

Parameters:

procedure - the procedure trying to acquire the lock

table - Table to lock

Returns:

true if the procedure has to wait for the table to be available

wakeTableSharedLock

public void wakeTableSharedLock(Procedure procedure,
                                TableName table)

Wake the procedures waiting for the specified table

Parameters:

procedure - the procedure releasing the lock

table - the name of the table that has the shared lock

markTableAsDeleted

protected boolean markTableAsDeleted(TableName table,
                                     Procedure procedure)

Tries to remove the queue and the table-lock of the specified table. If there are new operations pending (e.g. a new create), the remove will not be performed.

Parameters:

table - the name of the table that should be marked as deleted

procedure - the procedure that is removing the table

Returns:

true if deletion succeeded, false otherwise meaning that there are other new operations pending for that table (e.g. a new create).

waitRegion

public boolean waitRegion(Procedure procedure,
                          HRegionInfo regionInfo)

Suspend the procedure if the specified region is already locked.

Parameters:

procedure - the procedure trying to acquire the lock on the region

regionInfo - the region we are trying to lock

Returns:

true if the procedure has to wait for the regions to be available

waitRegions

public boolean waitRegions(Procedure procedure,
                           TableName table,
                           HRegionInfo... regionInfo)

Suspend the procedure if the specified set of regions are already locked.

Parameters:

procedure - the procedure trying to acquire the lock on the regions

table - the table name of the regions we are trying to lock

regionInfo - the list of regions we are trying to lock

Returns:

true if the procedure has to wait for the regions to be available

wakeRegion

public void wakeRegion(Procedure procedure,
                       HRegionInfo regionInfo)

Wake the procedures waiting for the specified region

Parameters:

procedure - the procedure that was holding the region

regionInfo - the region the procedure was holding

wakeRegions

public void wakeRegions(Procedure procedure,
                        TableName table,
                        HRegionInfo... regionInfo)

Wake the procedures waiting for the specified regions

Parameters:

procedure - the procedure that was holding the regions

regionInfo - the list of regions the procedure was holding

waitNamespaceExclusiveLock

public boolean waitNamespaceExclusiveLock(Procedure procedure,
                                          String namespace)

Suspend the procedure if the specified namespace is already locked.

Parameters:

procedure - the procedure trying to acquire the lock

namespace - Namespace to lock

Returns:

true if the procedure has to wait for the namespace to be available

See Also:

wakeNamespaceExclusiveLock(Procedure,String)

wakeNamespaceExclusiveLock

public void wakeNamespaceExclusiveLock(Procedure procedure,
                                       String namespace)

Wake the procedures waiting for the specified namespace

Parameters:

procedure - the procedure releasing the lock

namespace - the namespace that has the exclusive lock

See Also:

waitNamespaceExclusiveLock(Procedure,String)

waitServerExclusiveLock

public boolean waitServerExclusiveLock(Procedure procedure,
                                       ServerName serverName)

Try to acquire the exclusive lock on the specified server.

Parameters:

procedure - the procedure trying to acquire the lock

serverName - Server to lock

Returns:

true if the procedure has to wait for the server to be available

See Also:

wakeServerExclusiveLock(Procedure,ServerName)

wakeServerExclusiveLock

public void wakeServerExclusiveLock(Procedure procedure,
                                    ServerName serverName)

Wake the procedures waiting for the specified server

Parameters:

procedure - the procedure releasing the lock

serverName - the server that has the exclusive lock

See Also:

waitServerExclusiveLock(Procedure,ServerName)

dumpLocks

public String dumpLocks()
                 throws IOException

For debugging. Expensive.

Throws:

IOException