software.amazon.kinesis.coordinator

Class Scheduler

java.lang.Object

software.amazon.kinesis.coordinator.Scheduler

All Implemented Interfaces:

Runnable

public class Scheduler
extends Object
implements Runnable

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	gracefuleShutdownStarted

Constructor Summary

Constructors

Modifier	Constructor and Description
	Scheduler(CheckpointConfig checkpointConfig, CoordinatorConfig coordinatorConfig, LeaseManagementConfig leaseManagementConfig, LifecycleConfig lifecycleConfig, MetricsConfig metricsConfig, ProcessorConfig processorConfig, RetrievalConfig retrievalConfig)
protected	Scheduler(CheckpointConfig checkpointConfig, CoordinatorConfig coordinatorConfig, LeaseManagementConfig leaseManagementConfig, LifecycleConfig lifecycleConfig, MetricsConfig metricsConfig, ProcessorConfig processorConfig, RetrievalConfig retrievalConfig, software.amazon.kinesis.coordinator.DiagnosticEventFactory diagnosticEventFactory) Customers do not currently have the ability to customize the DiagnosticEventFactory, but this visibility is desired for testing.

Method Summary

Modifier and Type	Method and Description
protected ShardConsumer	buildConsumer(ShardInfo shardInfo, ShardRecordProcessorFactory shardRecordProcessorFactory, LeaseCleanupManager leaseCleanupManager)
Callable<Boolean>	createGracefulShutdownCallable() Creates a callable that will execute the graceful shutdown process.
boolean	hasGracefulShutdownStarted()
Future<Void>	requestShutdown() Deprecated.
void	run() Start consuming data from the stream, and pass it to the application record processors.
void	shutdown() Signals worker to shutdown.
Future<Boolean>	startGracefulShutdown() Requests a graceful shutdown of the worker, notifying record processors, that implement ShutdownNotificationAware, of the impending shutdown.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

gracefuleShutdownStarted

protected boolean gracefuleShutdownStarted

Constructor Detail

Scheduler

public Scheduler(@NonNull
                 CheckpointConfig checkpointConfig,
                 @NonNull
                 CoordinatorConfig coordinatorConfig,
                 @NonNull
                 LeaseManagementConfig leaseManagementConfig,
                 @NonNull
                 LifecycleConfig lifecycleConfig,
                 @NonNull
                 MetricsConfig metricsConfig,
                 @NonNull
                 ProcessorConfig processorConfig,
                 @NonNull
                 RetrievalConfig retrievalConfig)

Scheduler

protected Scheduler(@NonNull
                    CheckpointConfig checkpointConfig,
                    @NonNull
                    CoordinatorConfig coordinatorConfig,
                    @NonNull
                    LeaseManagementConfig leaseManagementConfig,
                    @NonNull
                    LifecycleConfig lifecycleConfig,
                    @NonNull
                    MetricsConfig metricsConfig,
                    @NonNull
                    ProcessorConfig processorConfig,
                    @NonNull
                    RetrievalConfig retrievalConfig,
                    @NonNull
                    software.amazon.kinesis.coordinator.DiagnosticEventFactory diagnosticEventFactory)

Customers do not currently have the ability to customize the DiagnosticEventFactory, but this visibility is desired for testing. This constructor is only used for testing to provide a mock DiagnosticEventFactory.

Method Detail

run

public void run()

Start consuming data from the stream, and pass it to the application record processors.

Specified by:

run in interface Runnable

startGracefulShutdown

public Future<Boolean> startGracefulShutdown()

Requests a graceful shutdown of the worker, notifying record processors, that implement ShutdownNotificationAware, of the impending shutdown. This gives the record processor a final chance to checkpoint. This will only create a single shutdown future. Additional attempts to start a graceful shutdown will return the previous future. It's possible that a record processor won't be notify before being shutdown. This can occur if the lease is lost after requesting shutdown, but before the notification is dispatched.

Requested Shutdown Process

When a shutdown process is requested it operates slightly differently to allow the record processors a chance to checkpoint a final time.

1. Call to request shutdown invoked.
2. Worker stops attempting to acquire new leases
3. Record Processor Shutdown Begins
   1. Record processor is notified of the impending shutdown, and given a final chance to checkpoint
   2. The lease for the record processor is then dropped.
   3. The record processor enters into an idle state waiting for the worker to complete final termination
   4. The worker will detect a record processor that has lost it's lease, and will terminate the record processor with ShutdownReason.LEASE_LOST
4. The worker will shutdown all record processors.
5. Once all record processors have been terminated, the worker will terminate all owned resources.
6. Once the worker shutdown is complete, the returned future is completed.

Returns:

a future that will be set once the shutdown has completed. True indicates that the graceful shutdown completed successfully. A false value indicates that a non-exception case caused the shutdown process to terminate early.

createGracefulShutdownCallable

public Callable<Boolean> createGracefulShutdownCallable()

Creates a callable that will execute the graceful shutdown process. This callable can be used to execute graceful shutdowns in your own executor, or execute the shutdown synchronously.

Returns:

a callable that run the graceful shutdown process. This may return a callable that return true if the graceful shutdown has already been completed.

Throws:

IllegalStateException - thrown by the callable if another callable has already started the shutdown process.

hasGracefulShutdownStarted

public boolean hasGracefulShutdownStarted()

shutdown

public void shutdown()

Signals worker to shutdown. Worker will try initiating shutdown of all record processors. Note that if executor services were passed to the worker by the user, worker will not attempt to shutdown those resources.

Shutdown Process

When called this will start shutdown of the record processor, and eventually shutdown the worker itself.

1. Call to start shutdown invoked
2. Lease coordinator told to stop taking leases, and to drop existing leases.
3. Worker discovers record processors that no longer have leases.
4. Worker triggers shutdown with state ShutdownReason.LEASE_LOST.
5. Once all record processors are shutdown, worker terminates owned resources.
6. Shutdown complete.

buildConsumer

protected ShardConsumer buildConsumer(@NonNull
                                      ShardInfo shardInfo,
                                      @NonNull
                                      ShardRecordProcessorFactory shardRecordProcessorFactory,
                                      @NonNull
                                      LeaseCleanupManager leaseCleanupManager)

requestShutdown

@Deprecated
public Future<Void> requestShutdown()

Deprecated.