Class ParallelProcessingUnit
- java.lang.Object
-
- com.github.toolarium.processing.unit.base.AbstractProcessingUnitImpl
-
- com.github.toolarium.processing.unit.base.AbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer>
-
- com.github.toolarium.processing.unit.runtime.runnable.parallelization.ParallelProcessingUnit
-
- All Implemented Interfaces:
IProcessingUnitThrottlingSupport,IProcessingUnit,IParallelProcessingUnit,ParallelProcessingUnitParameters,java.lang.Thread.UncaughtExceptionHandler
public class ParallelProcessingUnit extends AbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer> implements ParallelProcessingUnitParameters, IParallelProcessingUnit, IProcessingUnitThrottlingSupport, java.lang.Thread.UncaughtExceptionHandler
Implements a parallel processing unit which let run a processing unit with multiple threads. A processing unit has just to implement the interfaceIParallelProcessingUnitwhich acts as marker interface. In case the processing unit needs anIObjectLockManagerit can additional implement the interfaceIProcessingUnitObjectLockManagerSupport.
-
-
Field Summary
-
Fields inherited from interface com.github.toolarium.processing.unit.runtime.runnable.parallelization.ParallelProcessingUnitParameters
AGGREGATE_STATUS_PAUSE_TIME, LOCK_SIZE, MAX_NUMBER_OF_NO_PROGRESS_BEFORE_ABORT, NO_PROGRESS_PAUSE_TIME, NUMBER_OF_THREAD_PARAMETER, STARTUP_PHASED_SLEEP_TIME, UNLOCK_TIMEOUT
-
-
Constructor Summary
Constructors Constructor Description ParallelProcessingUnit(java.lang.String id, java.lang.String name, java.lang.Class<? extends IProcessingUnit> processingUnitClass)Constructor for ParallelProcessingUnit
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected ProcessingUnitStatusBuilderaggregateProcessingUnitStatus(ProcessingUnitStatusBuilder processingUnitStatusBuilder)Aggregate all processing unit statusprotected voidcheckExceptionInThreads()Check for exceptions in runner threadslongestimateNumberOfUnitsToProcess()Estimate the number of units to process.com.github.toolarium.common.bandwidth.IBandwidthThrottlinggetBandwidthProcessingUnitThrottling()Get the bandwidth processing unit throttlingIEmptyProcessingUnitHandlergetEmptyProcessingUnitHandler()Get the empty processing unit handlerjava.util.List<ParameterDefinition>getParameterDefinition()Gets the parameter definition list of the processing.voidinitialize(java.util.List<Parameter> parameterList, IProcessingUnitContext processingUnitContext)Initializes the processing unit.protected voidlogObjectLockStatistic()Log the object lock statisticprotected ParallelProcessingUnitPersistenceContainernewPersistenceInstance()Get a new persistence instancevoidonAborting()This method is called in case of an abort (status aborting) before the resources are released.voidonEnding()This method is called on ending after a successful end (status ending) before the resources are released.IProcessingUnitStatusprocessUnit(ProcessingUnitStatusBuilder processingUnitStatusBuilder)Process unit: This method will be called until theIProcessingUnitStatus.hasNext()returns false.voidreleaseResource()Release resources will be called to release all internal referenced resources after a processing success, error or by asuspendProcessing(see method below).voidresumeProcessing(IProcessingUnitProgress processingUnitProgress, IProcessingUnitPersistence processingPersistence)After suspending a processing unit can be resumed.voidsetMaxNumberOfProcessingUnitCallsPerSecond(java.lang.String id, java.lang.String name, java.lang.Long maxNumberOfProcessingUnitCallsPerSecond)Set the max number of processing unit calls per secondsvoidsetObjectLockManager(com.github.toolarium.common.object.IObjectLockManager objectLockManager)Sets the object lock managerprotected voidstartThreadPool()Start thread poolIProcessingUnitPersistencesuspendProcessing()Suspends the processing: The processing is able to persist its state with the help of theIProcessingUnitPersistenceobject.voiduncaughtException(java.lang.Thread t, java.lang.Throwable e)protected voidwaitForThreadPoolTerminated()Wait for thread pool terminated.-
Methods inherited from class com.github.toolarium.processing.unit.base.AbstractProcessingUnitPersistenceImpl
getProcessingPersistence, removePersistenceInstance
-
Methods inherited from class com.github.toolarium.processing.unit.base.AbstractProcessingUnitImpl
getObjectLockManager, getParameterRuntime, getProcessingUnitContext, getProcessingUnitProgress, initializeParameterDefinition, processUnit, validateParameterList
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface com.github.toolarium.processing.unit.IProcessingUnit
processUnit, validateParameterList
-
-
-
-
Constructor Detail
-
ParallelProcessingUnit
public ParallelProcessingUnit(java.lang.String id, java.lang.String name, java.lang.Class<? extends IProcessingUnit> processingUnitClass)Constructor for ParallelProcessingUnit- Parameters:
id- the unique id of the processingname- the name of the processingprocessingUnitClass- the processing unit class
-
-
Method Detail
-
getParameterDefinition
public java.util.List<ParameterDefinition> getParameterDefinition()
Description copied from interface:IProcessingUnitGets the parameter definition list of the processing.- Specified by:
getParameterDefinitionin interfaceIProcessingUnit- Overrides:
getParameterDefinitionin classAbstractProcessingUnitImpl- Returns:
- the parameter definition list or null in case of no parameters
- See Also:
AbstractProcessingUnitImpl.getParameterDefinition()
-
initialize
public void initialize(java.util.List<Parameter> parameterList, IProcessingUnitContext processingUnitContext) throws ValidationException
Description copied from interface:IProcessingUnitInitializes the processing unit. This will called as first to initialize the processing unit. It is also used when processing is interrupted and resumed.- Specified by:
initializein interfaceIProcessingUnit- Overrides:
initializein classAbstractProcessingUnitImpl- Parameters:
parameterList- the parameter list to run the processing.processingUnitContext- the processing context.- Throws:
ValidationException- This will be throw in case the consistency check failures.- See Also:
AbstractProcessingUnitImpl.initialize(java.util.List, com.github.toolarium.processing.unit.IProcessingUnitContext)
-
estimateNumberOfUnitsToProcess
public long estimateNumberOfUnitsToProcess() throws ProcessingExceptionDescription copied from interface:IProcessingUnitEstimate the number of units to process. It will be called once afterIProcessingUnit.initialize(List, IProcessingUnitContext). It set the the number of units to process in the objectIProcessingUnitProgress. In case there are more elements to process than estimated, the progress will adapted. In case of aIProcessingUnit.resumeProcessing(IProcessingUnitProgress, IProcessingUnitPersistence)it will not be called again.- Specified by:
estimateNumberOfUnitsToProcessin interfaceIProcessingUnit- Overrides:
estimateNumberOfUnitsToProcessin classAbstractProcessingUnitImpl- Returns:
- returns the number of units to process
- Throws:
ProcessingException- Throws this exception in case of initialization failures.- See Also:
AbstractProcessingUnitImpl.estimateNumberOfUnitsToProcess()
-
processUnit
public IProcessingUnitStatus processUnit(ProcessingUnitStatusBuilder processingUnitStatusBuilder) throws ProcessingException
Description copied from class:AbstractProcessingUnitImplProcess unit: This method will be called until theIProcessingUnitStatus.hasNext()returns false. Important: this method have to process the sequential or in a small block size.- Specified by:
processUnitin classAbstractProcessingUnitImpl- Parameters:
processingUnitStatusBuilder- the processing unit status builder- Returns:
- the process unit status
- Throws:
ProcessingException- In case of any failures occurs.- See Also:
AbstractProcessingUnitImpl.processUnit(com.github.toolarium.processing.unit.ProcessingUnitStatusBuilder)
-
onEnding
public void onEnding()
Description copied from interface:IProcessingUnitThis method is called on ending after a successful end (status ending) before the resources are released.- Specified by:
onEndingin interfaceIProcessingUnit- Overrides:
onEndingin classAbstractProcessingUnitImpl- See Also:
AbstractProcessingUnitImpl.onEnding()
-
onAborting
public void onAborting()
Description copied from interface:IProcessingUnitThis method is called in case of an abort (status aborting) before the resources are released.- Specified by:
onAbortingin interfaceIProcessingUnit- Overrides:
onAbortingin classAbstractProcessingUnitImpl- See Also:
AbstractProcessingUnitImpl.onAborting()
-
releaseResource
public void releaseResource() throws ProcessingExceptionDescription copied from interface:IProcessingUnitRelease resources will be called to release all internal referenced resources after a processing success, error or by asuspendProcessing(see method below). It will be called afterIProcessingUnit.onEnding()orIProcessingUnit.onAborting().- Specified by:
releaseResourcein interfaceIProcessingUnit- Overrides:
releaseResourcein classAbstractProcessingUnitImpl- Throws:
ProcessingException- Throws this exception in case of releasing failure.- See Also:
AbstractProcessingUnitImpl.releaseResource()
-
suspendProcessing
public IProcessingUnitPersistence suspendProcessing() throws ProcessingException
Description copied from interface:IProcessingUnitSuspends the processing: The processing is able to persist its state with the help of theIProcessingUnitPersistenceobject. On a resume this instance of theIProcessingUnitPersistencewill be returned (see method below). TheIProcessingUnitPersistencedon't need to contain any parameters or additional statuses. They are covered by the framework.- Specified by:
suspendProcessingin interfaceIProcessingUnit- Overrides:
suspendProcessingin classAbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer>- Returns:
- the processing persistence which contains all information to resume processing later (see resumeProcessing).
- Throws:
ProcessingException- Throws this exception in case of while suspend the processing any failures occurs.- See Also:
AbstractProcessingUnitImpl.suspendProcessing()
-
resumeProcessing
public void resumeProcessing(IProcessingUnitProgress processingUnitProgress, IProcessingUnitPersistence processingPersistence) throws ProcessingException
Description copied from interface:IProcessingUnitAfter suspending a processing unit can be resumed. The parameter list of the initialization is passed as well theIProcessingUnitPersistencewhich was returned by the suspendProcessing method. The main purpose of this call is to set the processing persistence.- Specified by:
resumeProcessingin interfaceIProcessingUnit- Overrides:
resumeProcessingin classAbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer>- Parameters:
processingUnitProgress- the processing unit progress of the processing unit before suspending.processingPersistence- the processing persistence to resume after suspending.- Throws:
ProcessingException- Throws this exception in case of while resume the processing any failures occurs.- See Also:
AbstractProcessingUnitImpl.resumeProcessing(com.github.toolarium.processing.unit.IProcessingUnitProgress, com.github.toolarium.processing.unit.IProcessingUnitPersistence)
-
getEmptyProcessingUnitHandler
public IEmptyProcessingUnitHandler getEmptyProcessingUnitHandler()
Get the empty processing unit handler- Returns:
- the empty processing unit handler
-
setObjectLockManager
public void setObjectLockManager(com.github.toolarium.common.object.IObjectLockManager objectLockManager)
Description copied from class:AbstractProcessingUnitImplSets the object lock manager- Overrides:
setObjectLockManagerin classAbstractProcessingUnitImpl- Parameters:
objectLockManager- the object lock manager- See Also:
AbstractProcessingUnitImpl.setObjectLockManager(com.github.toolarium.common.object.IObjectLockManager)
-
setMaxNumberOfProcessingUnitCallsPerSecond
public void setMaxNumberOfProcessingUnitCallsPerSecond(java.lang.String id, java.lang.String name, java.lang.Long maxNumberOfProcessingUnitCallsPerSecond)Description copied from interface:IProcessingUnitThrottlingSupportSet the max number of processing unit calls per seconds- Specified by:
setMaxNumberOfProcessingUnitCallsPerSecondin interfaceIProcessingUnitThrottlingSupport- Parameters:
id- the unique id of the processingname- the name of the processingmaxNumberOfProcessingUnitCallsPerSecond- The max number of processing unit calls per seconds- See Also:
IProcessingUnitThrottlingSupport.setMaxNumberOfProcessingUnitCallsPerSecond(java.lang.String, java.lang.String, java.lang.Long)
-
getBandwidthProcessingUnitThrottling
public com.github.toolarium.common.bandwidth.IBandwidthThrottling getBandwidthProcessingUnitThrottling()
Description copied from interface:IProcessingUnitThrottlingSupportGet the bandwidth processing unit throttling- Specified by:
getBandwidthProcessingUnitThrottlingin interfaceIProcessingUnitThrottlingSupport- Returns:
- the bandwidth processing unit throttling
- See Also:
IProcessingUnitThrottlingSupport.getBandwidthProcessingUnitThrottling()
-
uncaughtException
public void uncaughtException(java.lang.Thread t, java.lang.Throwable e)- Specified by:
uncaughtExceptionin interfacejava.lang.Thread.UncaughtExceptionHandler- See Also:
Thread.UncaughtExceptionHandler.uncaughtException(java.lang.Thread, java.lang.Throwable)
-
newPersistenceInstance
protected ParallelProcessingUnitPersistenceContainer newPersistenceInstance()
Description copied from class:AbstractProcessingUnitPersistenceImplGet a new persistence instance- Specified by:
newPersistenceInstancein classAbstractProcessingUnitPersistenceImpl<ParallelProcessingUnitPersistenceContainer>- Returns:
- the persistence instance
- See Also:
AbstractProcessingUnitPersistenceImpl.newPersistenceInstance()
-
aggregateProcessingUnitStatus
protected ProcessingUnitStatusBuilder aggregateProcessingUnitStatus(ProcessingUnitStatusBuilder processingUnitStatusBuilder)
Aggregate all processing unit status- Parameters:
processingUnitStatusBuilder- the processing status builder- Returns:
- the processing status builder
-
checkExceptionInThreads
protected void checkExceptionInThreads() throws ProcessingExceptionCheck for exceptions in runner threads- Throws:
ProcessingException- in case of an error
-
startThreadPool
protected void startThreadPool()
Start thread pool
-
waitForThreadPoolTerminated
protected void waitForThreadPoolTerminated()
Wait for thread pool terminated.
-
logObjectLockStatistic
protected void logObjectLockStatistic()
Log the object lock statistic
-
-