Class AbstractIncrementalSnapshotContext<T>
java.lang.Object
io.debezium.pipeline.source.snapshot.incremental.AbstractIncrementalSnapshotContext<T>
- All Implemented Interfaces:
IncrementalSnapshotContext<T>
- Direct Known Subclasses:
SignalBasedIncrementalSnapshotContext
@NotThreadSafe
public class AbstractIncrementalSnapshotContext<T>
extends Object
implements IncrementalSnapshotContext<T>
A class describing current state of incremental snapshot
- Author:
- Jiri Pechanec
-
Nested Class Summary
Modifier and TypeClassDescriptionprivate static class
-
Field Summary
Modifier and TypeFieldDescriptionprivate Object[]
The last primary key in chunk that is now in process.static final String
private String
private String
static final String
static final String
private Object[]
The PK of the last record that was passed to Kafka Connect.private static final org.slf4j.Logger
private Object[]
The largest PK in the table at the start of snapshot.private final AtomicBoolean
Determines if the incremental snapshot was paused or not.private Table
private boolean
private final AbstractIncrementalSnapshotContext.SnapshotDataCollection<T>
static final String
private final boolean
protected boolean
true
if window is opened and deduplication should be executed -
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionaddDataCollectionNamesToSnapshot
(String correlationId, List<String> dataCollectionIds, List<AdditionalCondition> additionalCondition, String surrogateKey) private void
addTablesIdsToSnapshot
(List<DataCollection<T>> dataCollectionIds) private String
arrayToSerializedString
(Object[] array) private Function<String,
DataCollection<T>> buildDataCollection
(List<AdditionalCondition> additionalCondition, String surrogateKey) Object[]
boolean
closeWindow
(String id) int
boolean
protected static <U> IncrementalSnapshotContext<U>
init
(AbstractIncrementalSnapshotContext<U> context, Map<String, ?> offsets) boolean
boolean
boolean
void
maximumKey
(Object[] key) void
nextChunkPosition
(Object[] end) private boolean
The snapshotting process can receive out-of-order windowing signals after connector restart as depending on committed offset position some signals can be replayed.boolean
openWindow
(String id) void
boolean
removeDataCollectionFromSnapshot
(String dataCollectionId) private void
void
void
void
private Object[]
serializedStringToArray
(String field, String serialized) void
void
setSchemaVerificationPassed
(boolean schemaVerificationPassed) boolean
void
void
toString()
void
-
Field Details
-
LOGGER
private static final org.slf4j.Logger LOGGER -
INCREMENTAL_SNAPSHOT_KEY
- See Also:
-
EVENT_PRIMARY_KEY
- See Also:
-
TABLE_MAXIMUM_KEY
- See Also:
-
CORRELATION_ID
- See Also:
-
snapshotDataCollection
-
windowOpened
protected boolean windowOpenedtrue
if window is opened and deduplication should be executed -
chunkEndPosition
The last primary key in chunk that is now in process. -
useCatalogBeforeSchema
private final boolean useCatalogBeforeSchema -
lastEventKeySent
The PK of the last record that was passed to Kafka Connect. In case of connector restart the start of the first chunk will be populated from it. -
currentChunkId
-
maximumKey
The largest PK in the table at the start of snapshot. -
schema
-
schemaVerificationPassed
private boolean schemaVerificationPassed -
correlationId
-
paused
Determines if the incremental snapshot was paused or not.
-
-
Constructor Details
-
AbstractIncrementalSnapshotContext
public AbstractIncrementalSnapshotContext(boolean useCatalogBeforeSchema)
-
-
Method Details
-
openWindow
- Specified by:
openWindow
in interfaceIncrementalSnapshotContext<T>
-
closeWindow
- Specified by:
closeWindow
in interfaceIncrementalSnapshotContext<T>
-
pauseSnapshot
public void pauseSnapshot()- Specified by:
pauseSnapshot
in interfaceIncrementalSnapshotContext<T>
-
resumeSnapshot
public void resumeSnapshot()- Specified by:
resumeSnapshot
in interfaceIncrementalSnapshotContext<T>
-
isSnapshotPaused
public boolean isSnapshotPaused()- Specified by:
isSnapshotPaused
in interfaceIncrementalSnapshotContext<T>
-
notExpectedChunk
The snapshotting process can receive out-of-order windowing signals after connector restart as depending on committed offset position some signals can be replayed. In extreme case a signal can be received even when the incremental snapshot was completed just before the restart. Such windowing signals are ignored. -
deduplicationNeeded
public boolean deduplicationNeeded()- Specified by:
deduplicationNeeded
in interfaceIncrementalSnapshotContext<T>
-
arrayToSerializedString
-
serializedStringToArray
-
snapshotRunning
public boolean snapshotRunning()- Specified by:
snapshotRunning
in interfaceIncrementalSnapshotContext<T>
-
store
- Specified by:
store
in interfaceIncrementalSnapshotContext<T>
-
addTablesIdsToSnapshot
-
addDataCollectionNamesToSnapshot
public List<DataCollection<T>> addDataCollectionNamesToSnapshot(String correlationId, List<String> dataCollectionIds, List<AdditionalCondition> additionalCondition, String surrogateKey) - Specified by:
addDataCollectionNamesToSnapshot
in interfaceIncrementalSnapshotContext<T>
-
buildDataCollection
private Function<String,DataCollection<T>> buildDataCollection(List<AdditionalCondition> additionalCondition, String surrogateKey) -
stopSnapshot
public void stopSnapshot()- Specified by:
stopSnapshot
in interfaceIncrementalSnapshotContext<T>
-
removeDataCollectionFromSnapshot
- Specified by:
removeDataCollectionFromSnapshot
in interfaceIncrementalSnapshotContext<T>
-
getDataCollections
- Specified by:
getDataCollections
in interfaceIncrementalSnapshotContext<T>
-
unsetCorrelationId
public void unsetCorrelationId()- Specified by:
unsetCorrelationId
in interfaceIncrementalSnapshotContext<T>
-
getCorrelationId
- Specified by:
getCorrelationId
in interfaceIncrementalSnapshotContext<T>
-
init
protected static <U> IncrementalSnapshotContext<U> init(AbstractIncrementalSnapshotContext<U> context, Map<String, ?> offsets) -
sendEvent
- Specified by:
sendEvent
in interfaceIncrementalSnapshotContext<T>
-
currentDataCollectionId
- Specified by:
currentDataCollectionId
in interfaceIncrementalSnapshotContext<T>
-
dataCollectionsToBeSnapshottedCount
public int dataCollectionsToBeSnapshottedCount()- Specified by:
dataCollectionsToBeSnapshottedCount
in interfaceIncrementalSnapshotContext<T>
-
nextChunkPosition
- Specified by:
nextChunkPosition
in interfaceIncrementalSnapshotContext<T>
-
chunkEndPosititon
- Specified by:
chunkEndPosititon
in interfaceIncrementalSnapshotContext<T>
-
resetChunk
private void resetChunk() -
revertChunk
public void revertChunk()- Specified by:
revertChunk
in interfaceIncrementalSnapshotContext<T>
-
isNonInitialChunk
public boolean isNonInitialChunk()- Specified by:
isNonInitialChunk
in interfaceIncrementalSnapshotContext<T>
-
nextDataCollection
- Specified by:
nextDataCollection
in interfaceIncrementalSnapshotContext<T>
-
startNewChunk
public void startNewChunk()- Specified by:
startNewChunk
in interfaceIncrementalSnapshotContext<T>
-
currentChunkId
- Specified by:
currentChunkId
in interfaceIncrementalSnapshotContext<T>
-
maximumKey
- Specified by:
maximumKey
in interfaceIncrementalSnapshotContext<T>
-
maximumKey
- Specified by:
maximumKey
in interfaceIncrementalSnapshotContext<T>
-
getSchema
- Specified by:
getSchema
in interfaceIncrementalSnapshotContext<T>
-
setSchema
- Specified by:
setSchema
in interfaceIncrementalSnapshotContext<T>
-
isSchemaVerificationPassed
public boolean isSchemaVerificationPassed()- Specified by:
isSchemaVerificationPassed
in interfaceIncrementalSnapshotContext<T>
-
setSchemaVerificationPassed
public void setSchemaVerificationPassed(boolean schemaVerificationPassed) - Specified by:
setSchemaVerificationPassed
in interfaceIncrementalSnapshotContext<T>
-
toString
-