Package io.debezium.pipeline.source
Class AbstractSnapshotChangeEventSource<P extends Partition,O extends OffsetContext>
java.lang.Object
io.debezium.pipeline.source.AbstractSnapshotChangeEventSource<P,O>
- All Implemented Interfaces:
ChangeEventSource
,SnapshotChangeEventSource<P,
,O> AutoCloseable
- Direct Known Subclasses:
RelationalSnapshotChangeEventSource
public abstract class AbstractSnapshotChangeEventSource<P extends Partition,O extends OffsetContext>
extends Object
implements SnapshotChangeEventSource<P,O>, AutoCloseable
An abstract implementation of
SnapshotChangeEventSource
that all implementations should extend
to inherit common functionality.- Author:
- Chris Cranford
-
Nested Class Summary
Modifier and TypeClassDescriptionstatic class
AbstractSnapshotChangeEventSource.SnapshotContext<P extends Partition,
O extends OffsetContext> Mutable context which is populated in the course of snapshottingstatic class
A configuration describing the task to be performed during snapshotting.Nested classes/interfaces inherited from interface io.debezium.pipeline.source.spi.ChangeEventSource
ChangeEventSource.ChangeEventSourceContext
-
Field Summary
Modifier and TypeFieldDescriptionprivate final CommonConnectorConfig
static final Duration
Interval for showing a log statement with the progress while scanning a single table.private static final org.slf4j.Logger
private final SnapshotProgressListener<P>
-
Constructor Summary
ConstructorDescriptionAbstractSnapshotChangeEventSource
(CommonConnectorConfig connectorConfig, SnapshotProgressListener<P> snapshotProgressListener) -
Method Summary
Modifier and TypeMethodDescriptionprotected void
aborted
(AbstractSnapshotChangeEventSource.SnapshotContext<P, O> snapshotContext) Completes the snapshot, doing any required clean-up (resource disposal etc.).void
close()
Completes the snapshot, doing any required clean-up (resource disposal etc.).protected void
completed
(AbstractSnapshotChangeEventSource.SnapshotContext<P, O> snapshotContext) Completes the snapshot, doing any required clean-up (resource disposal etc.).protected void
Delays snapshot execution as per theCommonConnectorConfig.SNAPSHOT_DELAY_MS
parameter.protected <T extends DataCollectionId>
Stream<T>determineDataCollectionsToBeSnapshotted
(Collection<T> allDataCollections) protected abstract SnapshotResult<O>
doExecute
(ChangeEventSource.ChangeEventSourceContext context, O previousOffset, AbstractSnapshotChangeEventSource.SnapshotContext<P, O> snapshotContext, AbstractSnapshotChangeEventSource.SnapshottingTask snapshottingTask) Executes this source.execute
(ChangeEventSource.ChangeEventSourceContext context, P partition, O previousOffset) Executes this source.protected abstract AbstractSnapshotChangeEventSource.SnapshottingTask
getSnapshottingTask
(P partition, O previousOffset) Returns the snapshotting task based on the previous offset (if available) and the connector's snapshotting mode.protected abstract AbstractSnapshotChangeEventSource.SnapshotContext<P,
O> Prepares the taking of a snapshot and returns an initialAbstractSnapshotChangeEventSource.SnapshotContext
.
-
Field Details
-
LOGGER
private static final org.slf4j.Logger LOGGER -
LOG_INTERVAL
Interval for showing a log statement with the progress while scanning a single table. -
connectorConfig
-
snapshotProgressListener
-
-
Constructor Details
-
AbstractSnapshotChangeEventSource
public AbstractSnapshotChangeEventSource(CommonConnectorConfig connectorConfig, SnapshotProgressListener<P> snapshotProgressListener)
-
-
Method Details
-
execute
public SnapshotResult<O> execute(ChangeEventSource.ChangeEventSourceContext context, P partition, O previousOffset) throws InterruptedException Description copied from interface:SnapshotChangeEventSource
Executes this source. Implementations should regularly check via the given context if they should stop. If that's the case, they should abort their processing and perform any clean-up needed, such as rolling back pending transactions, releasing locks etc.- Specified by:
execute
in interfaceSnapshotChangeEventSource<P extends Partition,
O extends OffsetContext> - Parameters:
context
- contextual information for this source's executionpartition
- the source partition from which the snapshot should be takenpreviousOffset
- previous offset restored from Kafka- Returns:
- an indicator to the position at which the snapshot was taken
- Throws:
InterruptedException
- in case the snapshot was aborted before completion
-
determineDataCollectionsToBeSnapshotted
protected <T extends DataCollectionId> Stream<T> determineDataCollectionsToBeSnapshotted(Collection<T> allDataCollections) -
delaySnapshotIfNeeded
protected void delaySnapshotIfNeeded(ChangeEventSource.ChangeEventSourceContext context) throws InterruptedException Delays snapshot execution as per theCommonConnectorConfig.SNAPSHOT_DELAY_MS
parameter.- Throws:
InterruptedException
-
doExecute
protected abstract SnapshotResult<O> doExecute(ChangeEventSource.ChangeEventSourceContext context, O previousOffset, AbstractSnapshotChangeEventSource.SnapshotContext<P, O> snapshotContext, AbstractSnapshotChangeEventSource.SnapshottingTask snapshottingTask) throws ExceptionExecutes this source. Implementations should regularly check via the given context if they should stop. If that's the case, they should abort their processing and perform any clean-up needed, such as rolling back pending transactions, releasing locks, etc.- Parameters:
context
- contextual information for this source's executionpreviousOffset
- previous offset restored from KafkasnapshotContext
- mutable context information populated throughout the snapshot processsnapshottingTask
- immutable information about what tasks should be performed during snapshot- Returns:
- an indicator to the position at which the snapshot was taken
- Throws:
Exception
-
getSnapshottingTask
protected abstract AbstractSnapshotChangeEventSource.SnapshottingTask getSnapshottingTask(P partition, O previousOffset) Returns the snapshotting task based on the previous offset (if available) and the connector's snapshotting mode. -
prepare
protected abstract AbstractSnapshotChangeEventSource.SnapshotContext<P,O> prepare(P partition) throws Exception Prepares the taking of a snapshot and returns an initialAbstractSnapshotChangeEventSource.SnapshotContext
.- Throws:
Exception
-
close
public void close()Completes the snapshot, doing any required clean-up (resource disposal etc.). The snapshot may have run successfully or have been aborted at this point.- Specified by:
close
in interfaceAutoCloseable
-
completed
Completes the snapshot, doing any required clean-up (resource disposal etc.). The snapshot have run successfully at this point.- Parameters:
snapshotContext
- snapshot context
-
aborted
Completes the snapshot, doing any required clean-up (resource disposal etc.). The snapshot is aborted at this point.- Parameters:
snapshotContext
- snapshot context
-