Package io.debezium.pipeline.source

Class AbstractSnapshotChangeEventSource<P extends Partition,O extends OffsetContext>

java.lang.Object

io.debezium.pipeline.source.AbstractSnapshotChangeEventSource<P,O>

All Implemented Interfaces:

ChangeEventSource, SnapshotChangeEventSource<P,O>

Direct Known Subclasses:

RelationalSnapshotChangeEventSource

public abstract class AbstractSnapshotChangeEventSource<P extends Partition,O extends OffsetContext>
extends Object
implements SnapshotChangeEventSource<P,O>

An abstract implementation of SnapshotChangeEventSource that all implementations should extend to inherit common functionality.

Author:

Chris Cranford

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AbstractSnapshotChangeEventSource.SnapshotContext<P extends Partition,O extends OffsetContext>	Mutable context which is populated in the course of snapshotting
static class	AbstractSnapshotChangeEventSource.SnapshottingTask	A configuration describing the task to be performed during snapshotting.

Nested classes/interfaces inherited from interface io.debezium.pipeline.source.spi.ChangeEventSource

ChangeEventSource.ChangeEventSourceContext

Field Summary

Fields

Modifier and Type	Field	Description
private CommonConnectorConfig	connectorConfig
static Duration	LOG_INTERVAL	Interval for showing a log statement with the progress while scanning a single table.
private static org.slf4j.Logger	LOGGER
private SnapshotProgressListener	snapshotProgressListener

Constructor Summary

Constructors

Constructor	Description
AbstractSnapshotChangeEventSource(CommonConnectorConfig connectorConfig, SnapshotProgressListener snapshotProgressListener)

Method Summary

Modifier and Type	Method	Description
protected void	complete(AbstractSnapshotChangeEventSource.SnapshotContext<P,O> snapshotContext)	Completes the snapshot, doing any required clean-up (resource disposal etc.).
protected void	delaySnapshotIfNeeded(ChangeEventSource.ChangeEventSourceContext context)	Delays snapshot execution as per the CommonConnectorConfig.SNAPSHOT_DELAY_MS parameter.
protected <T extends DataCollectionId> Stream<T>	determineDataCollectionsToBeSnapshotted(Collection<T> allDataCollections)
protected abstract SnapshotResult<O>	doExecute(ChangeEventSource.ChangeEventSourceContext context, O previousOffset, AbstractSnapshotChangeEventSource.SnapshotContext<P,O> snapshotContext, AbstractSnapshotChangeEventSource.SnapshottingTask snapshottingTask)	Executes this source.
SnapshotResult<O>	execute(ChangeEventSource.ChangeEventSourceContext context, P partition, O previousOffset)	Executes this source.
protected abstract AbstractSnapshotChangeEventSource.SnapshottingTask	getSnapshottingTask(O previousOffset)	Returns the snapshotting task based on the previous offset (if available) and the connector's snapshotting mode.
protected abstract AbstractSnapshotChangeEventSource.SnapshotContext<P,O>	prepare(P partition)	Prepares the taking of a snapshot and returns an initial AbstractSnapshotChangeEventSource.SnapshotContext.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

LOG_INTERVAL

public static final Duration LOG_INTERVAL

Interval for showing a log statement with the progress while scanning a single table.

connectorConfig

private final CommonConnectorConfig connectorConfig

snapshotProgressListener

private final SnapshotProgressListener snapshotProgressListener

Constructor Detail

AbstractSnapshotChangeEventSource

public AbstractSnapshotChangeEventSource(CommonConnectorConfig connectorConfig,
                                         SnapshotProgressListener snapshotProgressListener)

Method Detail

execute

public SnapshotResult<O> execute(ChangeEventSource.ChangeEventSourceContext context,
                                 P partition,
                                 O previousOffset)
                          throws InterruptedException

Description copied from interface: SnapshotChangeEventSource

Executes this source. Implementations should regularly check via the given context if they should stop. If that's the case, they should abort their processing and perform any clean-up needed, such as rolling back pending transactions, releasing locks etc.

Specified by:

execute in interface SnapshotChangeEventSource<P extends Partition,O extends OffsetContext>

Parameters:

context - contextual information for this source's execution

partition - the source partition from which the snapshot should be taken

previousOffset - previous offset restored from Kafka

Returns:

an indicator to the position at which the snapshot was taken

Throws:

InterruptedException - in case the snapshot was aborted before completion

determineDataCollectionsToBeSnapshotted

protected <T extends DataCollectionId> Stream<T> determineDataCollectionsToBeSnapshotted(Collection<T> allDataCollections)

delaySnapshotIfNeeded

protected void delaySnapshotIfNeeded(ChangeEventSource.ChangeEventSourceContext context)
                              throws InterruptedException

Delays snapshot execution as per the CommonConnectorConfig.SNAPSHOT_DELAY_MS parameter.

Throws:

InterruptedException

doExecute

protected abstract SnapshotResult<O> doExecute(ChangeEventSource.ChangeEventSourceContext context,
                                               O previousOffset,
                                               AbstractSnapshotChangeEventSource.SnapshotContext<P,O> snapshotContext,
                                               AbstractSnapshotChangeEventSource.SnapshottingTask snapshottingTask)
                                        throws Exception

Executes this source. Implementations should regularly check via the given context if they should stop. If that's the case, they should abort their processing and perform any clean-up needed, such as rolling back pending transactions, releasing locks, etc.

Parameters:

context - contextual information for this source's execution

previousOffset - previous offset restored from Kafka

snapshotContext - mutable context information populated throughout the snapshot process

snapshottingTask - immutable information about what tasks should be performed during snapshot

Returns:

an indicator to the position at which the snapshot was taken

Throws:

Exception

getSnapshottingTask

protected abstract AbstractSnapshotChangeEventSource.SnapshottingTask getSnapshottingTask(O previousOffset)

Returns the snapshotting task based on the previous offset (if available) and the connector's snapshotting mode.

prepare

protected abstract AbstractSnapshotChangeEventSource.SnapshotContext<P,O> prepare(P partition)
                                                                                 throws Exception

Prepares the taking of a snapshot and returns an initial AbstractSnapshotChangeEventSource.SnapshotContext.

Throws:

Exception

complete

protected void complete(AbstractSnapshotChangeEventSource.SnapshotContext<P,O> snapshotContext)

Completes the snapshot, doing any required clean-up (resource disposal etc.). The snapshot may have run successfully or have been aborted at this point.

Parameters:

snapshotContext - snapshot context