Package com.github.toolarium.processing.unit.runtime.runnable.parallelization

Class ParallelProcessingUnit

java.lang.Object

com.github.toolarium.processing.unit.base.AbstractProcessingUnitImpl

com.github.toolarium.processing.unit.base.AbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer>

com.github.toolarium.processing.unit.runtime.runnable.parallelization.ParallelProcessingUnit

All Implemented Interfaces:

IProcessingUnitThrottlingSupport, IProcessingUnit, IParallelProcessingUnit, ParallelProcessingUnitParameters, java.lang.Thread.UncaughtExceptionHandler

public class ParallelProcessingUnit
extends AbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer>
implements ParallelProcessingUnitParameters, IParallelProcessingUnit, IProcessingUnitThrottlingSupport, java.lang.Thread.UncaughtExceptionHandler

Implements a parallel processing unit which let run a processing unit with multiple threads. A processing unit has just to implement the interface IParallelProcessingUnit which acts as marker interface. In case the processing unit needs an IObjectLockManager it can additional implement the interface IProcessingUnitObjectLockManagerSupport.

Field Summary

Fields inherited from interface com.github.toolarium.processing.unit.runtime.runnable.parallelization.ParallelProcessingUnitParameters

AGGREGATE_STATUS_PAUSE_TIME, LOCK_SIZE, MAX_NUMBER_OF_NO_PROGRESS_BEFORE_ABORT, NO_PROGRESS_PAUSE_TIME, NUMBER_OF_THREAD_PARAMETER, STARTUP_PHASED_SLEEP_TIME, UNLOCK_TIMEOUT

Constructor Summary

Constructors

Constructor	Description
ParallelProcessingUnit(java.lang.String id, java.lang.String name, java.lang.Class<? extends IProcessingUnit> processingUnitClass)	Constructor for ParallelProcessingUnit

Method Summary

Modifier and Type	Method	Description
protected ProcessingUnitStatusBuilder	aggregateProcessingUnitStatus(ProcessingUnitStatusBuilder processingUnitStatusBuilder)	Aggregate all processing unit status
protected void	checkExceptionInThreads()	Check for exceptions in runner threads
long	estimateNumberOfUnitsToProcess()	Estimate the number of units to process.
com.github.toolarium.common.bandwidth.IBandwidthThrottling	getBandwidthProcessingUnitThrottling()	Get the bandwidth processing unit throttling
IEmptyProcessingUnitHandler	getEmptyProcessingUnitHandler()	Get the empty processing unit handler
java.util.List<ParameterDefinition>	getParameterDefinition()	Gets the parameter definition list of the processing.
void	initialize(java.util.List<Parameter> parameterList, IProcessingUnitContext processingUnitContext)	Initializes the processing unit.
protected void	logObjectLockStatistic()	Log the object lock statistic
protected ParallelProcessingUnitPersistenceContainer	newPersistenceInstance()	Get a new persistence instance
void	onAborting()	This method is called in case of an abort (status aborting) before the resources are released.
void	onEnding()	This method is called on ending after a successful end (status ending) before the resources are released.
IProcessingUnitStatus	processUnit(ProcessingUnitStatusBuilder processingUnitStatusBuilder)	Process unit: This method will be called until the IProcessingUnitStatus.hasNext() returns false.
void	releaseResource()	Release resources will be called to release all internal referenced resources after a processing success, error or by a suspendProcessing (see method below).
void	resumeProcessing(IProcessingUnitProgress processingUnitProgress, IProcessingUnitPersistence processingPersistence)	After suspending a processing unit can be resumed.
void	setMaxNumberOfProcessingUnitCallsPerSecond(java.lang.String id, java.lang.String name, java.lang.Long maxNumberOfProcessingUnitCallsPerSecond)	Set the max number of processing unit calls per seconds
void	setObjectLockManager(com.github.toolarium.common.object.IObjectLockManager objectLockManager)	Sets the object lock manager
protected void	startThreadPool()	Start thread pool
IProcessingUnitPersistence	suspendProcessing()	Suspends the processing: The processing is able to persist its state with the help of the IProcessingUnitPersistence object.
void	uncaughtException(java.lang.Thread t, java.lang.Throwable e)
protected void	waitForThreadPoolTerminated()	Wait for thread pool terminated.

Methods inherited from class com.github.toolarium.processing.unit.base.AbstractProcessingUnitPersistenceImpl

getProcessingPersistence, removePersistenceInstance

Methods inherited from class com.github.toolarium.processing.unit.base.AbstractProcessingUnitImpl

getObjectLockManager, getParameterRuntime, getProcessingUnitContext, getProcessingUnitProgress, initializeParameterDefinition, processUnit, validateParameterList

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.github.toolarium.processing.unit.IProcessingUnit

processUnit, validateParameterList

Constructor Detail

ParallelProcessingUnit

public ParallelProcessingUnit(java.lang.String id,
                              java.lang.String name,
                              java.lang.Class<? extends IProcessingUnit> processingUnitClass)

Constructor for ParallelProcessingUnit

Parameters:

id - the unique id of the processing

name - the name of the processing

processingUnitClass - the processing unit class

Method Detail

getParameterDefinition

public java.util.List<ParameterDefinition> getParameterDefinition()

Description copied from interface: IProcessingUnit

Gets the parameter definition list of the processing.

Specified by:

getParameterDefinition in interface IProcessingUnit

Overrides:

getParameterDefinition in class AbstractProcessingUnitImpl

Returns:

the parameter definition list or null in case of no parameters

See Also:

AbstractProcessingUnitImpl.getParameterDefinition()

initialize

public void initialize(java.util.List<Parameter> parameterList,
                       IProcessingUnitContext processingUnitContext)
                throws ValidationException

Description copied from interface: IProcessingUnit

Initializes the processing unit. This will called as first to initialize the processing unit. It is also used when processing is interrupted and resumed.

Specified by:

initialize in interface IProcessingUnit

Overrides:

initialize in class AbstractProcessingUnitImpl

Parameters:

parameterList - the parameter list to run the processing.

processingUnitContext - the processing context.

Throws:

ValidationException - This will be throw in case the consistency check failures.

See Also:

AbstractProcessingUnitImpl.initialize(java.util.List, com.github.toolarium.processing.unit.IProcessingUnitContext)

estimateNumberOfUnitsToProcess

public long estimateNumberOfUnitsToProcess()
                                    throws ProcessingException

Description copied from interface: IProcessingUnit

Estimate the number of units to process. It will be called once after IProcessingUnit.initialize(List, IProcessingUnitContext). It set the the number of units to process in the object IProcessingUnitProgress. In case there are more elements to process than estimated, the progress will adapted. In case of a IProcessingUnit.resumeProcessing(IProcessingUnitProgress, IProcessingUnitPersistence) it will not be called again.

Specified by:

estimateNumberOfUnitsToProcess in interface IProcessingUnit

Overrides:

estimateNumberOfUnitsToProcess in class AbstractProcessingUnitImpl

Returns:

returns the number of units to process

Throws:

ProcessingException - Throws this exception in case of initialization failures.

See Also:

AbstractProcessingUnitImpl.estimateNumberOfUnitsToProcess()

processUnit

public IProcessingUnitStatus processUnit(ProcessingUnitStatusBuilder processingUnitStatusBuilder)
                                  throws ProcessingException

Description copied from class: AbstractProcessingUnitImpl

Process unit: This method will be called until the IProcessingUnitStatus.hasNext() returns false. Important: this method have to process the sequential or in a small block size.

Specified by:

processUnit in class AbstractProcessingUnitImpl

Parameters:

processingUnitStatusBuilder - the processing unit status builder

Returns:

the process unit status

Throws:

ProcessingException - In case of any failures occurs.

See Also:

AbstractProcessingUnitImpl.processUnit(com.github.toolarium.processing.unit.ProcessingUnitStatusBuilder)

onEnding

public void onEnding()

Description copied from interface: IProcessingUnit

This method is called on ending after a successful end (status ending) before the resources are released.

Specified by:

onEnding in interface IProcessingUnit

Overrides:

onEnding in class AbstractProcessingUnitImpl

See Also:

AbstractProcessingUnitImpl.onEnding()

onAborting

public void onAborting()

Description copied from interface: IProcessingUnit

This method is called in case of an abort (status aborting) before the resources are released.

Specified by:

onAborting in interface IProcessingUnit

Overrides:

onAborting in class AbstractProcessingUnitImpl

See Also:

AbstractProcessingUnitImpl.onAborting()

releaseResource

public void releaseResource()
                     throws ProcessingException

Description copied from interface: IProcessingUnit

Release resources will be called to release all internal referenced resources after a processing success, error or by a suspendProcessing (see method below). It will be called after IProcessingUnit.onEnding() or IProcessingUnit.onAborting().

Specified by:

releaseResource in interface IProcessingUnit

Overrides:

releaseResource in class AbstractProcessingUnitImpl

Throws:

ProcessingException - Throws this exception in case of releasing failure.

See Also:

AbstractProcessingUnitImpl.releaseResource()

suspendProcessing

public IProcessingUnitPersistence suspendProcessing()
                                             throws ProcessingException

Description copied from interface: IProcessingUnit

Suspends the processing: The processing is able to persist its state with the help of the IProcessingUnitPersistence object. On a resume this instance of the IProcessingUnitPersistence will be returned (see method below). The IProcessingUnitPersistence don't need to contain any parameters or additional statuses. They are covered by the framework.

Specified by:

suspendProcessing in interface IProcessingUnit

Overrides:

suspendProcessing in class AbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer>

Returns:

the processing persistence which contains all information to resume processing later (see resumeProcessing).

Throws:

ProcessingException - Throws this exception in case of while suspend the processing any failures occurs.

See Also:

AbstractProcessingUnitImpl.suspendProcessing()

resumeProcessing

public void resumeProcessing(IProcessingUnitProgress processingUnitProgress,
                             IProcessingUnitPersistence processingPersistence)
                      throws ProcessingException

Description copied from interface: IProcessingUnit

After suspending a processing unit can be resumed. The parameter list of the initialization is passed as well the IProcessingUnitPersistence which was returned by the suspendProcessing method. The main purpose of this call is to set the processing persistence.

Specified by:

resumeProcessing in interface IProcessingUnit

Overrides:

resumeProcessing in class AbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer>

Parameters:

processingUnitProgress - the processing unit progress of the processing unit before suspending.

processingPersistence - the processing persistence to resume after suspending.

Throws:

ProcessingException - Throws this exception in case of while resume the processing any failures occurs.

See Also:

AbstractProcessingUnitImpl.resumeProcessing(com.github.toolarium.processing.unit.IProcessingUnitProgress, com.github.toolarium.processing.unit.IProcessingUnitPersistence)

getEmptyProcessingUnitHandler

public IEmptyProcessingUnitHandler getEmptyProcessingUnitHandler()

Get the empty processing unit handler

Returns:

the empty processing unit handler

setObjectLockManager

public void setObjectLockManager(com.github.toolarium.common.object.IObjectLockManager objectLockManager)

Description copied from class: AbstractProcessingUnitImpl

Sets the object lock manager

Overrides:

setObjectLockManager in class AbstractProcessingUnitImpl

Parameters:

objectLockManager - the object lock manager

See Also:

AbstractProcessingUnitImpl.setObjectLockManager(com.github.toolarium.common.object.IObjectLockManager)

setMaxNumberOfProcessingUnitCallsPerSecond

public void setMaxNumberOfProcessingUnitCallsPerSecond(java.lang.String id,
                                                       java.lang.String name,
                                                       java.lang.Long maxNumberOfProcessingUnitCallsPerSecond)

Description copied from interface: IProcessingUnitThrottlingSupport

Set the max number of processing unit calls per seconds

Specified by:

setMaxNumberOfProcessingUnitCallsPerSecond in interface IProcessingUnitThrottlingSupport

Parameters:

id - the unique id of the processing

name - the name of the processing

maxNumberOfProcessingUnitCallsPerSecond - The max number of processing unit calls per seconds

See Also:

IProcessingUnitThrottlingSupport.setMaxNumberOfProcessingUnitCallsPerSecond(java.lang.String, java.lang.String, java.lang.Long)

getBandwidthProcessingUnitThrottling

public com.github.toolarium.common.bandwidth.IBandwidthThrottling getBandwidthProcessingUnitThrottling()

Description copied from interface: IProcessingUnitThrottlingSupport

Get the bandwidth processing unit throttling

Specified by:

getBandwidthProcessingUnitThrottling in interface IProcessingUnitThrottlingSupport

Returns:

the bandwidth processing unit throttling

See Also:

IProcessingUnitThrottlingSupport.getBandwidthProcessingUnitThrottling()

uncaughtException

public void uncaughtException(java.lang.Thread t,
                              java.lang.Throwable e)

Specified by:

uncaughtException in interface java.lang.Thread.UncaughtExceptionHandler

See Also:

Thread.UncaughtExceptionHandler.uncaughtException(java.lang.Thread, java.lang.Throwable)

newPersistenceInstance

protected ParallelProcessingUnitPersistenceContainer newPersistenceInstance()

Description copied from class: AbstractProcessingUnitPersistenceImpl

Get a new persistence instance

Specified by:

newPersistenceInstance in class AbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer>

Returns:

the persistence instance

See Also:

AbstractProcessingUnitPersistenceImpl.newPersistenceInstance()

aggregateProcessingUnitStatus

protected ProcessingUnitStatusBuilder aggregateProcessingUnitStatus(ProcessingUnitStatusBuilder processingUnitStatusBuilder)

Aggregate all processing unit status

Parameters:

processingUnitStatusBuilder - the processing status builder

Returns:

the processing status builder

checkExceptionInThreads

protected void checkExceptionInThreads()
                                throws ProcessingException

Check for exceptions in runner threads

Throws:

ProcessingException - in case of an error

startThreadPool

protected void startThreadPool()

Start thread pool

waitForThreadPoolTerminated

protected void waitForThreadPoolTerminated()

Wait for thread pool terminated.

logObjectLockStatistic

protected void logObjectLockStatistic()

Log the object lock statistic