Package io.debezium.connector.common
Class BaseSourceTask<P extends Partition,O extends OffsetContext>
java.lang.Object
org.apache.kafka.connect.source.SourceTask
io.debezium.connector.common.BaseSourceTask<P,O>
- All Implemented Interfaces:
org.apache.kafka.connect.connector.Task
public abstract class BaseSourceTask<P extends Partition,O extends OffsetContext>
extends org.apache.kafka.connect.source.SourceTask
Base class for Debezium's CDC
SourceTask
implementations. Provides functionality common to all connectors,
such as validation of the configuration.- Author:
- Gunnar Morling
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.kafka.connect.source.SourceTask
org.apache.kafka.connect.source.SourceTask.TransactionBoundary
-
Field Summary
Modifier and TypeFieldDescriptionprivate final Clock
private ChangeEventSourceCoordinator<P,
O> The change event source coordinator for those connectors adhering to the new framework structure,null
for legacy-style connectors.private static final Duration
The latest offsets that have been acknowledged by the Kafka producer.private static final org.slf4j.Logger
private static final Duration
private final ElapsedTimeStrategy
private int
private Instant
Raw connector properties, kept here so they can be passed again in case of a restart.private ElapsedTimeStrategy
private Duration
private final AtomicReference<BaseSourceTask.State>
private final ReentrantLock
Used to ensure that start(), stop() and commitRecord() calls are serialized.Fields inherited from class org.apache.kafka.connect.source.SourceTask
context, TRANSACTION_BOUNDARY_CONFIG
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionvoid
commit()
void
commitRecord
(org.apache.kafka.connect.source.SourceRecord record) protected abstract List<org.apache.kafka.connect.source.SourceRecord>
doPoll()
Returns the next batch of source records, if any are available.protected abstract void
doStop()
Returns all configurationField
supported by this source task.getPreviousOffsets
(Partition.Provider<P> provider, OffsetContext.Loader<O> loader) Loads the connector's persistent offsets (if present) via the given loader.protected void
logStatistics
(List<org.apache.kafka.connect.source.SourceRecord> records) final List<org.apache.kafka.connect.source.SourceRecord>
poll()
protected abstract ChangeEventSourceCoordinator<P,
O> start
(Configuration config) Called once when starting this source task.final void
private boolean
Starts this connector in case it has been stopped after a retriable error, and the backoff period has passed.final void
stop()
private void
stop
(boolean restart) private void
updateLastOffset
(Map<String, ?> partition, Map<String, ?> lastOffset) protected Configuration
Methods inherited from class org.apache.kafka.connect.source.SourceTask
commitRecord, initialize
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.kafka.connect.connector.Task
version
-
Field Details
-
LOGGER
private static final org.slf4j.Logger LOGGER -
INITIAL_POLL_PERIOD_IN_MILLIS
-
MAX_POLL_PERIOD_IN_MILLIS
-
state
-
stateLock
Used to ensure that start(), stop() and commitRecord() calls are serialized. -
restartDelay
-
props
Raw connector properties, kept here so they can be passed again in case of a restart. -
coordinator
The change event source coordinator for those connectors adhering to the new framework structure,null
for legacy-style connectors. -
lastOffsets
The latest offsets that have been acknowledged by the Kafka producer. Will be acknowledged with the source database incommit()
(which may be a no-op depending on the connector). -
retriableRestartWait
-
pollOutputDelay
-
clock
-
previousOutputInstant
-
previousOutputBatchSize
private int previousOutputBatchSize
-
-
Constructor Details
-
BaseSourceTask
protected BaseSourceTask()
-
-
Method Details
-
start
- Specified by:
start
in interfaceorg.apache.kafka.connect.connector.Task
- Specified by:
start
in classorg.apache.kafka.connect.source.SourceTask
-
withMaskedSensitiveOptions
-
start
Called once when starting this source task.- Parameters:
config
- the task configuration; implementations should wrap it in a dedicated implementation ofCommonConnectorConfig
and work with typed access to configuration properties that way
-
poll
- Specified by:
poll
in classorg.apache.kafka.connect.source.SourceTask
- Throws:
InterruptedException
-
logStatistics
-
updateLastOffset
-
doPoll
protected abstract List<org.apache.kafka.connect.source.SourceRecord> doPoll() throws InterruptedExceptionReturns the next batch of source records, if any are available.- Throws:
InterruptedException
-
startIfNeededAndPossible
private boolean startIfNeededAndPossible()Starts this connector in case it has been stopped after a retriable error, and the backoff period has passed. -
stop
public final void stop()- Specified by:
stop
in interfaceorg.apache.kafka.connect.connector.Task
- Specified by:
stop
in classorg.apache.kafka.connect.source.SourceTask
-
stop
private void stop(boolean restart) -
doStop
protected abstract void doStop() -
commitRecord
public void commitRecord(org.apache.kafka.connect.source.SourceRecord record) throws InterruptedException - Overrides:
commitRecord
in classorg.apache.kafka.connect.source.SourceTask
- Throws:
InterruptedException
-
commit
- Overrides:
commit
in classorg.apache.kafka.connect.source.SourceTask
- Throws:
InterruptedException
-
getAllConfigurationFields
Returns all configurationField
supported by this source task. -
getPreviousOffsets
protected Offsets<P,O> getPreviousOffsets(Partition.Provider<P> provider, OffsetContext.Loader<O> loader) Loads the connector's persistent offsets (if present) via the given loader.
-