org.apache.hadoop.hbase.master.procedure

Class MasterProcedureScheduler

java.lang.Object

org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler

org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler

All Implemented Interfaces:

ProcedureScheduler

@InterfaceAudience.Private
public class MasterProcedureScheduler
extends AbstractProcedureScheduler

ProcedureScheduler for the Master Procedures. This ProcedureScheduler tries to provide to the ProcedureExecutor procedures that can be executed without having to wait on a lock. Most of the master operations can be executed concurrently, if they are operating on different tables (e.g. two create table procedures can be performed at the same time) or against two different servers; say two servers that crashed at about the same time.

Each procedure should implement an Interface providing information for this queue. For example table related procedures should implement TableProcedureInterface. Each procedure will be pushed in its own queue, and based on the operation type we may make smarter decisions: e.g. we can abort all the operations preceding a delete table, or similar.

Concurrency control

Concurrent access to member variables (tableRunQueue, serverRunQueue, locking, tableMap, serverBuckets) is controlled by schedLock(). This mainly includes:

• AbstractProcedureScheduler.push(Procedure, boolean, boolean): A push will add a Queue back to run-queue when:
  1. Queue was empty before push (so must have been out of run-queue)
  2. Child procedure is added (which means parent procedure holds exclusive lock, and it must have moved Queue out of run-queue)
• AbstractProcedureScheduler.poll(long): A poll will remove a Queue from run-queue when:
  1. Queue becomes empty after poll
  2. Exclusive lock is requested by polled procedure and lock is available (returns the procedure)
  3. Exclusive lock is requested but lock is not available (returns null)
  4. Polled procedure is child of parent holding exclusive lock and the next procedure is not a child
• Namespace/table/region locks: Queue is added back to run-queue when lock being released is:
  1. Exclusive lock
  2. Last shared lock (in case queue was removed because next procedure in queue required exclusive lock)

Constructor Summary

Constructors

Constructor and Description
MasterProcedureScheduler(Function<Long,Procedure<?>> procedureRetriever)

Method Summary

Modifier and Type	Method and Description
void	clear()
void	completionCleanup(Procedure proc)
protected Procedure	dequeue(boolean onlyUrgent)
String	dumpLocks() For debugging.
protected void	enqueue(Procedure proc, boolean addFront)
LockedResource	getLockResource(LockedResourceType resourceType, String resourceName)
List<LockedResource>	getLocks()
protected boolean	queueHasRunnables()
protected int	queueSize()
boolean	waitMetaExclusiveLock(Procedure<?> procedure) Deprecated. only used for RecoverMetaProcedure. Should be removed along with RecoverMetaProcedure.
boolean	waitNamespaceExclusiveLock(Procedure<?> procedure, String namespace) Suspend the procedure if the specified namespace is already locked.
boolean	waitPeerExclusiveLock(Procedure<?> procedure, String peerId) Try to acquire the exclusive lock on the specified peer.
boolean	waitRegion(Procedure<?> procedure, RegionInfo regionInfo) Suspend the procedure if the specified region is already locked.
boolean	waitRegions(Procedure<?> procedure, TableName table, RegionInfo... regionInfo) Suspend the procedure if the specified set of regions are already locked.
boolean	waitServerExclusiveLock(Procedure<?> procedure, ServerName serverName) Try to acquire the exclusive lock on the specified server.
boolean	waitTableExclusiveLock(Procedure<?> procedure, TableName table) Suspend the procedure if the specified table is already locked.
boolean	waitTableSharedLock(Procedure<?> procedure, TableName table) Suspend the procedure if the specified table is already locked.
void	wakeMetaExclusiveLock(Procedure<?> procedure) Deprecated. only used for RecoverMetaProcedure. Should be removed along with RecoverMetaProcedure.
void	wakeNamespaceExclusiveLock(Procedure<?> procedure, String namespace) Wake the procedures waiting for the specified namespace
void	wakePeerExclusiveLock(Procedure<?> procedure, String peerId) Wake the procedures waiting for the specified peer
void	wakeRegion(Procedure<?> procedure, RegionInfo regionInfo) Wake the procedures waiting for the specified region
void	wakeRegions(Procedure<?> procedure, TableName table, RegionInfo... regionInfo) Wake the procedures waiting for the specified regions
void	wakeServerExclusiveLock(Procedure<?> procedure, ServerName serverName) Wake the procedures waiting for the specified server
void	wakeTableExclusiveLock(Procedure<?> procedure, TableName table) Wake the procedures waiting for the specified table
void	wakeTableSharedLock(Procedure<?> procedure, TableName table) Wake the procedures waiting for the specified table
void	yield(Procedure proc)

Methods inherited from class org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler

addBack, addBack, addFront, addFront, addFront, dequeue, getNullPollCalls, getPollCalls, hasRunnables, poll, poll, poll, poll, poll, poll, push, schedLock, schedUnlock, signalAll, size, start, stop, waitProcedure, wakePollIfNeeded, wakeProcedure, wakeWaitingProcedures

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MasterProcedureScheduler

public MasterProcedureScheduler(Function<Long,Procedure<?>> procedureRetriever)

Method Detail

yield

public void yield(Procedure proc)

enqueue

protected void enqueue(Procedure proc,
                       boolean addFront)

Specified by:

enqueue in class AbstractProcedureScheduler

queueHasRunnables

protected boolean queueHasRunnables()

Specified by:

queueHasRunnables in class AbstractProcedureScheduler

dequeue

protected Procedure dequeue(boolean onlyUrgent)

Specified by:

dequeue in class AbstractProcedureScheduler

getLocks

public List<LockedResource> getLocks()

getLockResource

public LockedResource getLockResource(LockedResourceType resourceType,
                                      String resourceName)

clear

public void clear()

queueSize

protected int queueSize()

Specified by:

queueSize in class AbstractProcedureScheduler

completionCleanup

public void completionCleanup(Procedure proc)

waitTableExclusiveLock

public boolean waitTableExclusiveLock(Procedure<?> procedure,
                                      TableName table)

Suspend the procedure if the specified table is already locked. Other operations in the table-queue will be executed after the lock is released.

Parameters:

procedure - the procedure trying to acquire the lock

table - Table to lock

Returns:

true if the procedure has to wait for the table to be available

wakeTableExclusiveLock

public void wakeTableExclusiveLock(Procedure<?> procedure,
                                   TableName table)

Wake the procedures waiting for the specified table

Parameters:

procedure - the procedure releasing the lock

table - the name of the table that has the exclusive lock

waitTableSharedLock

public boolean waitTableSharedLock(Procedure<?> procedure,
                                   TableName table)

Suspend the procedure if the specified table is already locked. other "read" operations in the table-queue may be executed concurrently,

Parameters:

procedure - the procedure trying to acquire the lock

table - Table to lock

Returns:

true if the procedure has to wait for the table to be available

wakeTableSharedLock

public void wakeTableSharedLock(Procedure<?> procedure,
                                TableName table)

Wake the procedures waiting for the specified table

Parameters:

procedure - the procedure releasing the lock

table - the name of the table that has the shared lock

waitRegion

public boolean waitRegion(Procedure<?> procedure,
                          RegionInfo regionInfo)

Suspend the procedure if the specified region is already locked.

Parameters:

procedure - the procedure trying to acquire the lock on the region

regionInfo - the region we are trying to lock

Returns:

true if the procedure has to wait for the regions to be available

waitRegions

public boolean waitRegions(Procedure<?> procedure,
                           TableName table,
                           RegionInfo... regionInfo)

Suspend the procedure if the specified set of regions are already locked.

Parameters:

procedure - the procedure trying to acquire the lock on the regions

table - the table name of the regions we are trying to lock

regionInfo - the list of regions we are trying to lock

Returns:

true if the procedure has to wait for the regions to be available

wakeRegion

public void wakeRegion(Procedure<?> procedure,
                       RegionInfo regionInfo)

Wake the procedures waiting for the specified region

Parameters:

procedure - the procedure that was holding the region

regionInfo - the region the procedure was holding

wakeRegions

public void wakeRegions(Procedure<?> procedure,
                        TableName table,
                        RegionInfo... regionInfo)

Wake the procedures waiting for the specified regions

Parameters:

procedure - the procedure that was holding the regions

regionInfo - the list of regions the procedure was holding

waitNamespaceExclusiveLock

public boolean waitNamespaceExclusiveLock(Procedure<?> procedure,
                                          String namespace)

Suspend the procedure if the specified namespace is already locked.

Parameters:

procedure - the procedure trying to acquire the lock

namespace - Namespace to lock

Returns:

true if the procedure has to wait for the namespace to be available

See Also:

wakeNamespaceExclusiveLock(Procedure,String)

wakeNamespaceExclusiveLock

public void wakeNamespaceExclusiveLock(Procedure<?> procedure,
                                       String namespace)

Wake the procedures waiting for the specified namespace

Parameters:

procedure - the procedure releasing the lock

namespace - the namespace that has the exclusive lock

See Also:

waitNamespaceExclusiveLock(Procedure,String)

waitServerExclusiveLock

public boolean waitServerExclusiveLock(Procedure<?> procedure,
                                       ServerName serverName)

Try to acquire the exclusive lock on the specified server.

Parameters:

procedure - the procedure trying to acquire the lock

serverName - Server to lock

Returns:

true if the procedure has to wait for the server to be available

See Also:

wakeServerExclusiveLock(Procedure,ServerName)

wakeServerExclusiveLock

public void wakeServerExclusiveLock(Procedure<?> procedure,
                                    ServerName serverName)

Wake the procedures waiting for the specified server

Parameters:

procedure - the procedure releasing the lock

serverName - the server that has the exclusive lock

See Also:

waitServerExclusiveLock(Procedure,ServerName)

waitPeerExclusiveLock

public boolean waitPeerExclusiveLock(Procedure<?> procedure,
                                     String peerId)

Try to acquire the exclusive lock on the specified peer.

Parameters:

procedure - the procedure trying to acquire the lock

peerId - peer to lock

Returns:

true if the procedure has to wait for the peer to be available

See Also:

wakePeerExclusiveLock(Procedure, String)

wakePeerExclusiveLock

public void wakePeerExclusiveLock(Procedure<?> procedure,
                                  String peerId)

Wake the procedures waiting for the specified peer

Parameters:

procedure - the procedure releasing the lock

peerId - the peer that has the exclusive lock

See Also:

waitPeerExclusiveLock(Procedure, String)

waitMetaExclusiveLock

@Deprecated
public boolean waitMetaExclusiveLock(Procedure<?> procedure)

Deprecated. only used for RecoverMetaProcedure. Should be removed along with RecoverMetaProcedure.

Try to acquire the exclusive lock on meta.

Parameters:

procedure - the procedure trying to acquire the lock

Returns:

true if the procedure has to wait for meta to be available

See Also:

wakeMetaExclusiveLock(Procedure)

wakeMetaExclusiveLock

@Deprecated
public void wakeMetaExclusiveLock(Procedure<?> procedure)

Deprecated. only used for RecoverMetaProcedure. Should be removed along with RecoverMetaProcedure.

Wake the procedures waiting for meta.

Parameters:

procedure - the procedure releasing the lock

See Also:

waitMetaExclusiveLock(Procedure)

dumpLocks

public String dumpLocks()
                 throws IOException

For debugging. Expensive.

Throws:

IOException