Class AbstractIncrementalSnapshotContext<T>
java.lang.Object
io.debezium.pipeline.source.snapshot.incremental.AbstractIncrementalSnapshotContext<T>
- All Implemented Interfaces:
IncrementalSnapshotContext<T>
- Direct Known Subclasses:
SignalBasedIncrementalSnapshotContext
@NotThreadSafe
public class AbstractIncrementalSnapshotContext<T>
extends Object
implements IncrementalSnapshotContext<T>
A class describing current state of incremental snapshot
- Author:
- Jiri Pechanec
-
Field Summary
Modifier and TypeFieldDescriptionprivate Object[]
The last primary key in chunk that is now in process.private String
static final String
static final String
static final String
private final Queue<DataCollection<T>>
static final String
static final String
private Object[]
The PK of the last record that was passed to Kafka Connect.private static final org.slf4j.Logger
private com.fasterxml.jackson.databind.ObjectMapper
private com.fasterxml.jackson.core.type.TypeReference<List<LinkedHashMap<String,
String>>> private Object[]
The largest PK in the table at the start of snapshot.private AtomicBoolean
Determines if the incremental snapshot was paused or not.private Table
private boolean
static final String
private final boolean
protected boolean
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionaddDataCollectionNamesToSnapshot
(List<String> dataCollectionIds, Optional<String> additionalCondition) private void
addTablesIdsToSnapshot
(List<DataCollection<T>> dataCollectionIds) private String
arrayToSerializedString
(Object[] array) Object[]
boolean
closeWindow
(String id) int
private String
boolean
protected static <U> IncrementalSnapshotContext<U>
init
(AbstractIncrementalSnapshotContext<U> context, Map<String, ?> offsets) boolean
boolean
boolean
void
maximumKey
(Object[] key) void
nextChunkPosition
(Object[] end) private boolean
The snapshotting process can receive out-of-order windowing signals after connector restart as depending on committed offset position some signals can be replayed.boolean
openWindow
(String id) void
boolean
removeDataCollectionFromSnapshot
(String dataCollectionId) private void
void
void
void
private Object[]
serializedStringToArray
(String field, String serialized) void
void
setSchemaVerificationPassed
(boolean schemaVerificationPassed) boolean
void
void
private List<DataCollection<T>>
stringToDataCollections
(String dataCollectionsStr) toString()
-
Field Details
-
LOGGER
private static final org.slf4j.Logger LOGGER -
INCREMENTAL_SNAPSHOT_KEY
- See Also:
-
DATA_COLLECTIONS_TO_SNAPSHOT_KEY
- See Also:
-
DATA_COLLECTIONS_TO_SNAPSHOT_KEY_ID
- See Also:
-
DATA_COLLECTIONS_TO_SNAPSHOT_KEY_ADDITIONAL_CONDITION
- See Also:
-
EVENT_PRIMARY_KEY
- See Also:
-
TABLE_MAXIMUM_KEY
- See Also:
-
windowOpened
protected boolean windowOpened -
chunkEndPosition
The last primary key in chunk that is now in process. -
dataCollectionsToSnapshot
-
useCatalogBeforeSchema
private final boolean useCatalogBeforeSchema -
lastEventKeySent
The PK of the last record that was passed to Kafka Connect. In case of connector restart the start of the first chunk will be populated from it. -
currentChunkId
-
maximumKey
The largest PK in the table at the start of snapshot. -
schema
-
schemaVerificationPassed
private boolean schemaVerificationPassed -
paused
Determines if the incremental snapshot was paused or not. -
mapper
private com.fasterxml.jackson.databind.ObjectMapper mapper -
mapperTypeRef
private com.fasterxml.jackson.core.type.TypeReference<List<LinkedHashMap<String,String>>> mapperTypeRef
-
-
Constructor Details
-
AbstractIncrementalSnapshotContext
public AbstractIncrementalSnapshotContext(boolean useCatalogBeforeSchema)
-
-
Method Details
-
openWindow
- Specified by:
openWindow
in interfaceIncrementalSnapshotContext<T>
-
closeWindow
- Specified by:
closeWindow
in interfaceIncrementalSnapshotContext<T>
-
pauseSnapshot
public void pauseSnapshot()- Specified by:
pauseSnapshot
in interfaceIncrementalSnapshotContext<T>
-
resumeSnapshot
public void resumeSnapshot()- Specified by:
resumeSnapshot
in interfaceIncrementalSnapshotContext<T>
-
isSnapshotPaused
public boolean isSnapshotPaused()- Specified by:
isSnapshotPaused
in interfaceIncrementalSnapshotContext<T>
-
notExpectedChunk
The snapshotting process can receive out-of-order windowing signals after connector restart as depending on committed offset position some signals can be replayed. In extreme case a signal can be received even when the incremental snapshot was completed just before the restart. Such windowing signals are ignored. -
deduplicationNeeded
public boolean deduplicationNeeded()- Specified by:
deduplicationNeeded
in interfaceIncrementalSnapshotContext<T>
-
arrayToSerializedString
-
serializedStringToArray
-
dataCollectionsToSnapshotAsString
-
stringToDataCollections
-
snapshotRunning
public boolean snapshotRunning()- Specified by:
snapshotRunning
in interfaceIncrementalSnapshotContext<T>
-
store
- Specified by:
store
in interfaceIncrementalSnapshotContext<T>
-
addTablesIdsToSnapshot
-
addDataCollectionNamesToSnapshot
public List<DataCollection<T>> addDataCollectionNamesToSnapshot(List<String> dataCollectionIds, Optional<String> additionalCondition) - Specified by:
addDataCollectionNamesToSnapshot
in interfaceIncrementalSnapshotContext<T>
-
stopSnapshot
public void stopSnapshot()- Specified by:
stopSnapshot
in interfaceIncrementalSnapshotContext<T>
-
removeDataCollectionFromSnapshot
- Specified by:
removeDataCollectionFromSnapshot
in interfaceIncrementalSnapshotContext<T>
-
init
protected static <U> IncrementalSnapshotContext<U> init(AbstractIncrementalSnapshotContext<U> context, Map<String, ?> offsets) -
sendEvent
- Specified by:
sendEvent
in interfaceIncrementalSnapshotContext<T>
-
currentDataCollectionId
- Specified by:
currentDataCollectionId
in interfaceIncrementalSnapshotContext<T>
-
dataCollectionsToBeSnapshottedCount
public int dataCollectionsToBeSnapshottedCount()- Specified by:
dataCollectionsToBeSnapshottedCount
in interfaceIncrementalSnapshotContext<T>
-
nextChunkPosition
- Specified by:
nextChunkPosition
in interfaceIncrementalSnapshotContext<T>
-
chunkEndPosititon
- Specified by:
chunkEndPosititon
in interfaceIncrementalSnapshotContext<T>
-
resetChunk
private void resetChunk() -
revertChunk
public void revertChunk()- Specified by:
revertChunk
in interfaceIncrementalSnapshotContext<T>
-
isNonInitialChunk
public boolean isNonInitialChunk()- Specified by:
isNonInitialChunk
in interfaceIncrementalSnapshotContext<T>
-
nextDataCollection
- Specified by:
nextDataCollection
in interfaceIncrementalSnapshotContext<T>
-
startNewChunk
public void startNewChunk()- Specified by:
startNewChunk
in interfaceIncrementalSnapshotContext<T>
-
currentChunkId
- Specified by:
currentChunkId
in interfaceIncrementalSnapshotContext<T>
-
maximumKey
- Specified by:
maximumKey
in interfaceIncrementalSnapshotContext<T>
-
maximumKey
- Specified by:
maximumKey
in interfaceIncrementalSnapshotContext<T>
-
getSchema
- Specified by:
getSchema
in interfaceIncrementalSnapshotContext<T>
-
setSchema
- Specified by:
setSchema
in interfaceIncrementalSnapshotContext<T>
-
isSchemaVerificationPassed
public boolean isSchemaVerificationPassed()- Specified by:
isSchemaVerificationPassed
in interfaceIncrementalSnapshotContext<T>
-
setSchemaVerificationPassed
public void setSchemaVerificationPassed(boolean schemaVerificationPassed) - Specified by:
setSchemaVerificationPassed
in interfaceIncrementalSnapshotContext<T>
-
toString
-