software.amazon.kinesis.leases.dynamodb

Class DynamoDBLeaseManagementFactory

java.lang.Object

software.amazon.kinesis.leases.dynamodb.DynamoDBLeaseManagementFactory

All Implemented Interfaces:

LeaseManagementFactory

public class DynamoDBLeaseManagementFactory
extends Object
implements LeaseManagementFactory

Constructor Summary

Constructors

Constructor and Description
DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, String tableName, String workerIdentifier, ExecutorService executorService, long failoverTimeMillis, long epsilonMillis, int maxLeasesForWorker, int maxLeasesToStealAtOneTime, int maxLeaseRenewalThreads, boolean cleanupLeasesUponShardCompletion, boolean ignoreUnexpectedChildShards, long shardSyncIntervalMillis, boolean consistentReads, long listShardsBackoffTimeMillis, int maxListShardsRetryAttempts, int maxCacheMissesBeforeReload, long listShardsCacheAllowedAgeInSeconds, int cacheMissWarningModulus, long initialLeaseTableReadCapacity, long initialLeaseTableWriteCapacity, HierarchicalShardSyncer deprecatedHierarchicalShardSyncer, TableCreatorCallback tableCreatorCallback, Duration dynamoDbRequestTimeout, software.amazon.awssdk.services.dynamodb.model.BillingMode billingMode, LeaseSerializer leaseSerializer, Function<StreamConfig,ShardDetector> customShardDetectorProvider, boolean isMultiStreamMode, LeaseCleanupConfig leaseCleanupConfig) Constructor.
DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient, String streamName, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, String tableName, String workerIdentifier, ExecutorService executorService, InitialPositionInStreamExtended initialPositionInStream, long failoverTimeMillis, long epsilonMillis, int maxLeasesForWorker, int maxLeasesToStealAtOneTime, int maxLeaseRenewalThreads, boolean cleanupLeasesUponShardCompletion, boolean ignoreUnexpectedChildShards, long shardSyncIntervalMillis, boolean consistentReads, long listShardsBackoffTimeMillis, int maxListShardsRetryAttempts, int maxCacheMissesBeforeReload, long listShardsCacheAllowedAgeInSeconds, int cacheMissWarningModulus) Deprecated.
DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient, String streamName, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, String tableName, String workerIdentifier, ExecutorService executorService, InitialPositionInStreamExtended initialPositionInStream, long failoverTimeMillis, long epsilonMillis, int maxLeasesForWorker, int maxLeasesToStealAtOneTime, int maxLeaseRenewalThreads, boolean cleanupLeasesUponShardCompletion, boolean ignoreUnexpectedChildShards, long shardSyncIntervalMillis, boolean consistentReads, long listShardsBackoffTimeMillis, int maxListShardsRetryAttempts, int maxCacheMissesBeforeReload, long listShardsCacheAllowedAgeInSeconds, int cacheMissWarningModulus, long initialLeaseTableReadCapacity, long initialLeaseTableWriteCapacity) Deprecated.
DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient, String streamName, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, String tableName, String workerIdentifier, ExecutorService executorService, InitialPositionInStreamExtended initialPositionInStream, long failoverTimeMillis, long epsilonMillis, int maxLeasesForWorker, int maxLeasesToStealAtOneTime, int maxLeaseRenewalThreads, boolean cleanupLeasesUponShardCompletion, boolean ignoreUnexpectedChildShards, long shardSyncIntervalMillis, boolean consistentReads, long listShardsBackoffTimeMillis, int maxListShardsRetryAttempts, int maxCacheMissesBeforeReload, long listShardsCacheAllowedAgeInSeconds, int cacheMissWarningModulus, long initialLeaseTableReadCapacity, long initialLeaseTableWriteCapacity, HierarchicalShardSyncer hierarchicalShardSyncer, TableCreatorCallback tableCreatorCallback) Deprecated.
DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient, String streamName, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, String tableName, String workerIdentifier, ExecutorService executorService, InitialPositionInStreamExtended initialPositionInStream, long failoverTimeMillis, long epsilonMillis, int maxLeasesForWorker, int maxLeasesToStealAtOneTime, int maxLeaseRenewalThreads, boolean cleanupLeasesUponShardCompletion, boolean ignoreUnexpectedChildShards, long shardSyncIntervalMillis, boolean consistentReads, long listShardsBackoffTimeMillis, int maxListShardsRetryAttempts, int maxCacheMissesBeforeReload, long listShardsCacheAllowedAgeInSeconds, int cacheMissWarningModulus, long initialLeaseTableReadCapacity, long initialLeaseTableWriteCapacity, HierarchicalShardSyncer hierarchicalShardSyncer, TableCreatorCallback tableCreatorCallback, Duration dynamoDbRequestTimeout) Deprecated.
DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient, String streamName, software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient, String tableName, String workerIdentifier, ExecutorService executorService, InitialPositionInStreamExtended initialPositionInStream, long failoverTimeMillis, long epsilonMillis, int maxLeasesForWorker, int maxLeasesToStealAtOneTime, int maxLeaseRenewalThreads, boolean cleanupLeasesUponShardCompletion, boolean ignoreUnexpectedChildShards, long shardSyncIntervalMillis, boolean consistentReads, long listShardsBackoffTimeMillis, int maxListShardsRetryAttempts, int maxCacheMissesBeforeReload, long listShardsCacheAllowedAgeInSeconds, int cacheMissWarningModulus, long initialLeaseTableReadCapacity, long initialLeaseTableWriteCapacity, HierarchicalShardSyncer hierarchicalShardSyncer, TableCreatorCallback tableCreatorCallback, Duration dynamoDbRequestTimeout, software.amazon.awssdk.services.dynamodb.model.BillingMode billingMode) Deprecated.

Method Summary

Modifier and Type	Method and Description
LeaseCleanupManager	createLeaseCleanupManager(MetricsFactory metricsFactory) LeaseCleanupManager cleans up leases in the lease table for shards which have either expired past the stream's retention period or have been completely processed.
LeaseCoordinator	createLeaseCoordinator(@NonNull MetricsFactory metricsFactory)
DynamoDBLeaseRefresher	createLeaseRefresher()
ShardDetector	createShardDetector() Deprecated.
ShardDetector	createShardDetector(StreamConfig streamConfig) KinesisShardDetector supports reading from service only using streamName.
ShardSyncTaskManager	createShardSyncTaskManager(@NonNull MetricsFactory metricsFactory) Deprecated.
ShardSyncTaskManager	createShardSyncTaskManager(MetricsFactory metricsFactory, StreamConfig streamConfig) Create ShardSyncTaskManager from the streamConfig passed

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamoDBLeaseManagementFactory

@Deprecated
public DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient,
                                                  String streamName,
                                                  software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                                  String tableName,
                                                  String workerIdentifier,
                                                  ExecutorService executorService,
                                                  InitialPositionInStreamExtended initialPositionInStream,
                                                  long failoverTimeMillis,
                                                  long epsilonMillis,
                                                  int maxLeasesForWorker,
                                                  int maxLeasesToStealAtOneTime,
                                                  int maxLeaseRenewalThreads,
                                                  boolean cleanupLeasesUponShardCompletion,
                                                  boolean ignoreUnexpectedChildShards,
                                                  long shardSyncIntervalMillis,
                                                  boolean consistentReads,
                                                  long listShardsBackoffTimeMillis,
                                                  int maxListShardsRetryAttempts,
                                                  int maxCacheMissesBeforeReload,
                                                  long listShardsCacheAllowedAgeInSeconds,
                                                  int cacheMissWarningModulus)

Deprecated.

Constructor.

NOTE: This constructor is deprecated and will be removed in a future release.

Parameters:

kinesisClient -

streamName -

dynamoDBClient -

tableName -

workerIdentifier -

executorService -

initialPositionInStream -

failoverTimeMillis -

epsilonMillis -

maxLeasesForWorker -

maxLeasesToStealAtOneTime -

maxLeaseRenewalThreads -

cleanupLeasesUponShardCompletion -

ignoreUnexpectedChildShards -

shardSyncIntervalMillis -

consistentReads -

listShardsBackoffTimeMillis -

maxListShardsRetryAttempts -

maxCacheMissesBeforeReload -

listShardsCacheAllowedAgeInSeconds -

cacheMissWarningModulus -

DynamoDBLeaseManagementFactory

@Deprecated
public DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient,
                                                  String streamName,
                                                  software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                                  String tableName,
                                                  String workerIdentifier,
                                                  ExecutorService executorService,
                                                  InitialPositionInStreamExtended initialPositionInStream,
                                                  long failoverTimeMillis,
                                                  long epsilonMillis,
                                                  int maxLeasesForWorker,
                                                  int maxLeasesToStealAtOneTime,
                                                  int maxLeaseRenewalThreads,
                                                  boolean cleanupLeasesUponShardCompletion,
                                                  boolean ignoreUnexpectedChildShards,
                                                  long shardSyncIntervalMillis,
                                                  boolean consistentReads,
                                                  long listShardsBackoffTimeMillis,
                                                  int maxListShardsRetryAttempts,
                                                  int maxCacheMissesBeforeReload,
                                                  long listShardsCacheAllowedAgeInSeconds,
                                                  int cacheMissWarningModulus,
                                                  long initialLeaseTableReadCapacity,
                                                  long initialLeaseTableWriteCapacity)

Deprecated.

Constructor.

NOTE: This constructor is deprecated and will be removed in a future release.

Parameters:

kinesisClient -

streamName -

dynamoDBClient -

tableName -

workerIdentifier -

executorService -

initialPositionInStream -

failoverTimeMillis -

epsilonMillis -

maxLeasesForWorker -

maxLeasesToStealAtOneTime -

maxLeaseRenewalThreads -

cleanupLeasesUponShardCompletion -

ignoreUnexpectedChildShards -

shardSyncIntervalMillis -

consistentReads -

listShardsBackoffTimeMillis -

maxListShardsRetryAttempts -

maxCacheMissesBeforeReload -

listShardsCacheAllowedAgeInSeconds -

cacheMissWarningModulus -

initialLeaseTableReadCapacity -

initialLeaseTableWriteCapacity -

DynamoDBLeaseManagementFactory

@Deprecated
public DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient,
                                                  String streamName,
                                                  software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                                  String tableName,
                                                  String workerIdentifier,
                                                  ExecutorService executorService,
                                                  InitialPositionInStreamExtended initialPositionInStream,
                                                  long failoverTimeMillis,
                                                  long epsilonMillis,
                                                  int maxLeasesForWorker,
                                                  int maxLeasesToStealAtOneTime,
                                                  int maxLeaseRenewalThreads,
                                                  boolean cleanupLeasesUponShardCompletion,
                                                  boolean ignoreUnexpectedChildShards,
                                                  long shardSyncIntervalMillis,
                                                  boolean consistentReads,
                                                  long listShardsBackoffTimeMillis,
                                                  int maxListShardsRetryAttempts,
                                                  int maxCacheMissesBeforeReload,
                                                  long listShardsCacheAllowedAgeInSeconds,
                                                  int cacheMissWarningModulus,
                                                  long initialLeaseTableReadCapacity,
                                                  long initialLeaseTableWriteCapacity,
                                                  HierarchicalShardSyncer hierarchicalShardSyncer,
                                                  TableCreatorCallback tableCreatorCallback)

Deprecated.

Constructor.

Parameters:

kinesisClient -

streamName -

dynamoDBClient -

tableName -

workerIdentifier -

executorService -

initialPositionInStream -

failoverTimeMillis -

epsilonMillis -

maxLeasesForWorker -

maxLeasesToStealAtOneTime -

maxLeaseRenewalThreads -

cleanupLeasesUponShardCompletion -

ignoreUnexpectedChildShards -

shardSyncIntervalMillis -

consistentReads -

listShardsBackoffTimeMillis -

maxListShardsRetryAttempts -

maxCacheMissesBeforeReload -

listShardsCacheAllowedAgeInSeconds -

cacheMissWarningModulus -

initialLeaseTableReadCapacity -

initialLeaseTableWriteCapacity -

hierarchicalShardSyncer -

tableCreatorCallback -

DynamoDBLeaseManagementFactory

@Deprecated
public DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient,
                                                  String streamName,
                                                  software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                                  String tableName,
                                                  String workerIdentifier,
                                                  ExecutorService executorService,
                                                  InitialPositionInStreamExtended initialPositionInStream,
                                                  long failoverTimeMillis,
                                                  long epsilonMillis,
                                                  int maxLeasesForWorker,
                                                  int maxLeasesToStealAtOneTime,
                                                  int maxLeaseRenewalThreads,
                                                  boolean cleanupLeasesUponShardCompletion,
                                                  boolean ignoreUnexpectedChildShards,
                                                  long shardSyncIntervalMillis,
                                                  boolean consistentReads,
                                                  long listShardsBackoffTimeMillis,
                                                  int maxListShardsRetryAttempts,
                                                  int maxCacheMissesBeforeReload,
                                                  long listShardsCacheAllowedAgeInSeconds,
                                                  int cacheMissWarningModulus,
                                                  long initialLeaseTableReadCapacity,
                                                  long initialLeaseTableWriteCapacity,
                                                  HierarchicalShardSyncer hierarchicalShardSyncer,
                                                  TableCreatorCallback tableCreatorCallback,
                                                  Duration dynamoDbRequestTimeout)

Deprecated.

Constructor.

Parameters:

kinesisClient -

streamName -

dynamoDBClient -

tableName -

workerIdentifier -

executorService -

initialPositionInStream -

failoverTimeMillis -

epsilonMillis -

maxLeasesForWorker -

maxLeasesToStealAtOneTime -

maxLeaseRenewalThreads -

cleanupLeasesUponShardCompletion -

ignoreUnexpectedChildShards -

shardSyncIntervalMillis -

consistentReads -

listShardsBackoffTimeMillis -

maxListShardsRetryAttempts -

maxCacheMissesBeforeReload -

listShardsCacheAllowedAgeInSeconds -

cacheMissWarningModulus -

initialLeaseTableReadCapacity -

initialLeaseTableWriteCapacity -

hierarchicalShardSyncer -

tableCreatorCallback -

dynamoDbRequestTimeout -

DynamoDBLeaseManagementFactory

@Deprecated
public DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient,
                                                  String streamName,
                                                  software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                                  String tableName,
                                                  String workerIdentifier,
                                                  ExecutorService executorService,
                                                  InitialPositionInStreamExtended initialPositionInStream,
                                                  long failoverTimeMillis,
                                                  long epsilonMillis,
                                                  int maxLeasesForWorker,
                                                  int maxLeasesToStealAtOneTime,
                                                  int maxLeaseRenewalThreads,
                                                  boolean cleanupLeasesUponShardCompletion,
                                                  boolean ignoreUnexpectedChildShards,
                                                  long shardSyncIntervalMillis,
                                                  boolean consistentReads,
                                                  long listShardsBackoffTimeMillis,
                                                  int maxListShardsRetryAttempts,
                                                  int maxCacheMissesBeforeReload,
                                                  long listShardsCacheAllowedAgeInSeconds,
                                                  int cacheMissWarningModulus,
                                                  long initialLeaseTableReadCapacity,
                                                  long initialLeaseTableWriteCapacity,
                                                  HierarchicalShardSyncer hierarchicalShardSyncer,
                                                  TableCreatorCallback tableCreatorCallback,
                                                  Duration dynamoDbRequestTimeout,
                                                  software.amazon.awssdk.services.dynamodb.model.BillingMode billingMode)

Deprecated.

Constructor.

Parameters:

kinesisClient -

streamName -

dynamoDBClient -

tableName -

workerIdentifier -

executorService -

initialPositionInStream -

failoverTimeMillis -

epsilonMillis -

maxLeasesForWorker -

maxLeasesToStealAtOneTime -

maxLeaseRenewalThreads -

cleanupLeasesUponShardCompletion -

ignoreUnexpectedChildShards -

shardSyncIntervalMillis -

consistentReads -

listShardsBackoffTimeMillis -

maxListShardsRetryAttempts -

maxCacheMissesBeforeReload -

listShardsCacheAllowedAgeInSeconds -

cacheMissWarningModulus -

initialLeaseTableReadCapacity -

initialLeaseTableWriteCapacity -

hierarchicalShardSyncer -

tableCreatorCallback -

dynamoDbRequestTimeout -

billingMode -

DynamoDBLeaseManagementFactory

public DynamoDBLeaseManagementFactory(software.amazon.awssdk.services.kinesis.KinesisAsyncClient kinesisClient,
                                      software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient dynamoDBClient,
                                      String tableName,
                                      String workerIdentifier,
                                      ExecutorService executorService,
                                      long failoverTimeMillis,
                                      long epsilonMillis,
                                      int maxLeasesForWorker,
                                      int maxLeasesToStealAtOneTime,
                                      int maxLeaseRenewalThreads,
                                      boolean cleanupLeasesUponShardCompletion,
                                      boolean ignoreUnexpectedChildShards,
                                      long shardSyncIntervalMillis,
                                      boolean consistentReads,
                                      long listShardsBackoffTimeMillis,
                                      int maxListShardsRetryAttempts,
                                      int maxCacheMissesBeforeReload,
                                      long listShardsCacheAllowedAgeInSeconds,
                                      int cacheMissWarningModulus,
                                      long initialLeaseTableReadCapacity,
                                      long initialLeaseTableWriteCapacity,
                                      HierarchicalShardSyncer deprecatedHierarchicalShardSyncer,
                                      TableCreatorCallback tableCreatorCallback,
                                      Duration dynamoDbRequestTimeout,
                                      software.amazon.awssdk.services.dynamodb.model.BillingMode billingMode,
                                      LeaseSerializer leaseSerializer,
                                      Function<StreamConfig,ShardDetector> customShardDetectorProvider,
                                      boolean isMultiStreamMode,
                                      LeaseCleanupConfig leaseCleanupConfig)

Constructor.

Parameters:

kinesisClient -

dynamoDBClient -

tableName -

workerIdentifier -

executorService -

failoverTimeMillis -

epsilonMillis -

maxLeasesForWorker -

maxLeasesToStealAtOneTime -

maxLeaseRenewalThreads -

cleanupLeasesUponShardCompletion -

ignoreUnexpectedChildShards -

shardSyncIntervalMillis -

consistentReads -

listShardsBackoffTimeMillis -

maxListShardsRetryAttempts -

maxCacheMissesBeforeReload -

listShardsCacheAllowedAgeInSeconds -

cacheMissWarningModulus -

initialLeaseTableReadCapacity -

initialLeaseTableWriteCapacity -

deprecatedHierarchicalShardSyncer -

tableCreatorCallback -

dynamoDbRequestTimeout -

billingMode -

leaseSerializer -

customShardDetectorProvider -

isMultiStreamMode -

leaseCleanupConfig -

Method Detail

createLeaseCoordinator

public LeaseCoordinator createLeaseCoordinator(@NonNull
                                               @NonNull MetricsFactory metricsFactory)

Specified by:

createLeaseCoordinator in interface LeaseManagementFactory

createShardSyncTaskManager

@Deprecated
public ShardSyncTaskManager createShardSyncTaskManager(@NonNull
                                                                   @NonNull MetricsFactory metricsFactory)

Deprecated.

Specified by:

createShardSyncTaskManager in interface LeaseManagementFactory

createShardSyncTaskManager

public ShardSyncTaskManager createShardSyncTaskManager(MetricsFactory metricsFactory,
                                                       StreamConfig streamConfig)

Create ShardSyncTaskManager from the streamConfig passed

Specified by:

createShardSyncTaskManager in interface LeaseManagementFactory

Parameters:

metricsFactory -

streamConfig -

Returns:

ShardSyncTaskManager

createLeaseRefresher

public DynamoDBLeaseRefresher createLeaseRefresher()

Specified by:

createLeaseRefresher in interface LeaseManagementFactory

createShardDetector

@Deprecated
public ShardDetector createShardDetector()

Deprecated.

Specified by:

createShardDetector in interface LeaseManagementFactory

createShardDetector

public ShardDetector createShardDetector(StreamConfig streamConfig)

KinesisShardDetector supports reading from service only using streamName. Support for accountId and stream creation epoch is yet to be provided.

Specified by:

createShardDetector in interface LeaseManagementFactory

Parameters:

streamConfig -

Returns:

ShardDetector

createLeaseCleanupManager

public LeaseCleanupManager createLeaseCleanupManager(MetricsFactory metricsFactory)

LeaseCleanupManager cleans up leases in the lease table for shards which have either expired past the stream's retention period or have been completely processed.

Specified by:

createLeaseCleanupManager in interface LeaseManagementFactory

Parameters:

metricsFactory -

Returns:

LeaseCleanupManager