Package io.debezium.connector.common
Class BaseSourceTask<P extends Partition,O extends OffsetContext>
- java.lang.Object
-
- org.apache.kafka.connect.source.SourceTask
-
- io.debezium.connector.common.BaseSourceTask<P,O>
-
- All Implemented Interfaces:
org.apache.kafka.connect.connector.Task
public abstract class BaseSourceTask<P extends Partition,O extends OffsetContext> extends org.apache.kafka.connect.source.SourceTask
Base class for Debezium's CDCSourceTask
implementations. Provides functionality common to all connectors, such as validation of the configuration.- Author:
- Gunnar Morling
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static class
BaseSourceTask.State
-
Field Summary
Fields Modifier and Type Field Description private Clock
clock
private ChangeEventSourceCoordinator<P,O>
coordinator
The change event source coordinator for those connectors adhering to the new framework structure,null
for legacy-style connectors.private static long
INITIAL_POLL_PERIOD_IN_MILLIS
private Map<String,?>
lastOffset
The latest offset that has been acknowledged by the Kafka producer.private static org.slf4j.Logger
LOGGER
private static long
MAX_POLL_PERIOD_IN_MILLIS
private ElapsedTimeStrategy
pollOutputDelay
private Instant
previousOutputInstant
private Map<String,String>
props
Raw connector properties, kept here so they can be passed again in case of a restart.private long
recordCounter
private ElapsedTimeStrategy
restartDelay
private Duration
retriableRestartWait
private AtomicReference<BaseSourceTask.State>
state
private ReentrantLock
stateLock
Used to ensure that start(), stop() and commitRecord() calls are serialized.
-
Constructor Summary
Constructors Modifier Constructor Description protected
BaseSourceTask()
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description void
commit()
void
commitRecord(org.apache.kafka.connect.source.SourceRecord record)
protected abstract List<org.apache.kafka.connect.source.SourceRecord>
doPoll()
Returns the next batch of source records, if any are available.protected abstract void
doStop()
protected abstract Iterable<Field>
getAllConfigurationFields()
Returns all configurationField
supported by this source task.protected Offsets<P,O>
getPreviousOffsets(Partition.Provider<P> provider, OffsetContext.Loader<O> loader)
Loads the connector's persistent offsets (if present) via the given loader.(package private) void
logStatistics(List<org.apache.kafka.connect.source.SourceRecord> records)
List<org.apache.kafka.connect.source.SourceRecord>
poll()
protected abstract ChangeEventSourceCoordinator<P,O>
start(Configuration config)
Called once when starting this source task.void
start(Map<String,String> props)
private boolean
startIfNeededAndPossible()
Starts this connector in case it has been stopped after a retriable error, and the backoff period has passed.void
stop()
private void
stop(boolean restart)
-
-
-
Field Detail
-
LOGGER
private static final org.slf4j.Logger LOGGER
-
INITIAL_POLL_PERIOD_IN_MILLIS
private static final long INITIAL_POLL_PERIOD_IN_MILLIS
-
MAX_POLL_PERIOD_IN_MILLIS
private static final long MAX_POLL_PERIOD_IN_MILLIS
-
state
private final AtomicReference<BaseSourceTask.State> state
-
stateLock
private final ReentrantLock stateLock
Used to ensure that start(), stop() and commitRecord() calls are serialized.
-
restartDelay
private volatile ElapsedTimeStrategy restartDelay
-
props
private volatile Map<String,String> props
Raw connector properties, kept here so they can be passed again in case of a restart.
-
coordinator
private ChangeEventSourceCoordinator<P extends Partition,O extends OffsetContext> coordinator
The change event source coordinator for those connectors adhering to the new framework structure,null
for legacy-style connectors.
-
lastOffset
private volatile Map<String,?> lastOffset
The latest offset that has been acknowledged by the Kafka producer. Will be acknowledged with the source database incommit()
(which may be a no-op depending on the connector).
-
retriableRestartWait
private Duration retriableRestartWait
-
pollOutputDelay
private final ElapsedTimeStrategy pollOutputDelay
-
clock
private final Clock clock
-
recordCounter
private long recordCounter
-
previousOutputInstant
private Instant previousOutputInstant
-
-
Method Detail
-
start
public final void start(Map<String,String> props)
- Specified by:
start
in interfaceorg.apache.kafka.connect.connector.Task
- Specified by:
start
in classorg.apache.kafka.connect.source.SourceTask
-
start
protected abstract ChangeEventSourceCoordinator<P,O> start(Configuration config)
Called once when starting this source task.- Parameters:
config
- the task configuration; implementations should wrap it in a dedicated implementation ofCommonConnectorConfig
and work with typed access to configuration properties that way
-
poll
public final List<org.apache.kafka.connect.source.SourceRecord> poll() throws InterruptedException
- Specified by:
poll
in classorg.apache.kafka.connect.source.SourceTask
- Throws:
InterruptedException
-
logStatistics
void logStatistics(List<org.apache.kafka.connect.source.SourceRecord> records)
-
doPoll
protected abstract List<org.apache.kafka.connect.source.SourceRecord> doPoll() throws InterruptedException
Returns the next batch of source records, if any are available.- Throws:
InterruptedException
-
startIfNeededAndPossible
private boolean startIfNeededAndPossible()
Starts this connector in case it has been stopped after a retriable error, and the backoff period has passed.
-
stop
public final void stop()
- Specified by:
stop
in interfaceorg.apache.kafka.connect.connector.Task
- Specified by:
stop
in classorg.apache.kafka.connect.source.SourceTask
-
stop
private void stop(boolean restart)
-
doStop
protected abstract void doStop()
-
commitRecord
public void commitRecord(org.apache.kafka.connect.source.SourceRecord record) throws InterruptedException
- Overrides:
commitRecord
in classorg.apache.kafka.connect.source.SourceTask
- Throws:
InterruptedException
-
commit
public void commit() throws InterruptedException
- Overrides:
commit
in classorg.apache.kafka.connect.source.SourceTask
- Throws:
InterruptedException
-
getAllConfigurationFields
protected abstract Iterable<Field> getAllConfigurationFields()
Returns all configurationField
supported by this source task.
-
getPreviousOffsets
protected Offsets<P,O> getPreviousOffsets(Partition.Provider<P> provider, OffsetContext.Loader<O> loader)
Loads the connector's persistent offsets (if present) via the given loader.
-
-