Package io.debezium.connector.mysql
Class MySqlOffsetContext
- java.lang.Object
-
- io.debezium.connector.mysql.MySqlOffsetContext
-
- All Implemented Interfaces:
OffsetContext
public class MySqlOffsetContext extends Object implements OffsetContext
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
MySqlOffsetContext.Loader
-
Field Summary
Fields Modifier and Type Field Description private long
currentEventLengthInBytes
private String
currentGtidSet
static String
EVENTS_TO_SKIP_OFFSET_KEY
static String
GTID_SET_KEY
private IncrementalSnapshotContext<TableId>
incrementalSnapshotContext
private boolean
inTransaction
static String
NON_GTID_TRANSACTION_ID_FORMAT
private Map<String,String>
partition
private String
restartBinlogFilename
private long
restartBinlogPosition
private long
restartEventsToSkip
private String
restartGtidSet
private int
restartRowsToSkip
private static String
SERVER_PARTITION_KEY
private static String
SNAPSHOT_COMPLETED_KEY
private boolean
snapshotCompleted
private SourceInfo
sourceInfo
private org.apache.kafka.connect.data.Schema
sourceInfoSchema
static String
TIMESTAMP_KEY
private TransactionContext
transactionContext
private String
transactionId
-
Constructor Summary
Constructors Constructor Description MySqlOffsetContext(MySqlConnectorConfig connectorConfig, boolean snapshot, boolean snapshotCompleted, SourceInfo sourceInfo)
MySqlOffsetContext(MySqlConnectorConfig connectorConfig, boolean snapshot, boolean snapshotCompleted, TransactionContext transactionContext, IncrementalSnapshotContext<TableId> incrementalSnapshotContext, SourceInfo sourceInfo)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
changeEventCompleted()
void
commitTransaction()
void
completeEvent()
Capture that we're starting a new event.void
databaseEvent(String database, Instant timestamp)
void
event(DataCollectionId tableId, Instant timestamp)
long
eventsToSkipUponRestart()
Get the number of events after the last transaction BEGIN that we've already processed.IncrementalSnapshotContext<?>
getIncrementalSnapshotContext()
Map<String,?>
getOffset()
Map<String,?>
getPartition()
Get the Kafka Connect detail about the source "partition", which describes the portion of the source that we are consuming.SourceInfo
getSource()
org.apache.kafka.connect.data.Struct
getSourceInfo()
org.apache.kafka.connect.data.Schema
getSourceInfoSchema()
TransactionContext
getTransactionContext()
String
getTransactionId()
String
gtidSet()
Get the string representation of the GTID range for the MySQL binary log file.void
incrementalSnapshotEvents()
static MySqlOffsetContext
initial(MySqlConnectorConfig config)
boolean
isSnapshotCompleted()
boolean
isSnapshotRunning()
void
markLastSnapshotRecord()
private Map<String,Object>
offsetUsingPosition(long rowsToSkip)
void
postSnapshotCompletion()
void
preSnapshotCompletion()
void
preSnapshotStart()
private void
resetTransactionId()
int
rowsToSkipUponRestart()
Get the number of rows beyond thelast completely processed event
to be skipped upon restart.void
setBinlogStartPoint(String binlogFilename, long positionOfFirstEvent)
Set the position in the MySQL binlog where we will start reading.void
setBinlogThread(long threadId)
void
setCompletedGtidSet(String gtidSet)
Set the GTID set that captures all of the GTID transactions that have been completely processed.void
setEventPosition(long positionOfCurrentEvent, long eventSizeInBytes)
Set the position within the MySQL binary log file of the current event.void
setInitialSkips(long restartEventsToSkip, int restartRowsToSkip)
void
setQuery(String query)
Set the original SQL query.void
setRowNumber(int eventRowNumber, int totalNumberOfRows)
Given the row number within a binlog event and the total number of rows in that event, compute the Kafka Connect offset that is be included in the produced change event describing the row.private void
setTransactionId()
void
startGtid(String gtid, String gtidSet)
Record that a new GTID transaction has been started and has been included in the set of GTIDs known to the MySQL server.void
startNextTransaction()
void
tableEvent(String database, Set<TableId> tableIds, Instant timestamp)
String
toString()
-
-
-
Field Detail
-
SERVER_PARTITION_KEY
private static final String SERVER_PARTITION_KEY
- See Also:
- Constant Field Values
-
SNAPSHOT_COMPLETED_KEY
private static final String SNAPSHOT_COMPLETED_KEY
- See Also:
- Constant Field Values
-
EVENTS_TO_SKIP_OFFSET_KEY
public static final String EVENTS_TO_SKIP_OFFSET_KEY
- See Also:
- Constant Field Values
-
TIMESTAMP_KEY
public static final String TIMESTAMP_KEY
- See Also:
- Constant Field Values
-
GTID_SET_KEY
public static final String GTID_SET_KEY
- See Also:
- Constant Field Values
-
NON_GTID_TRANSACTION_ID_FORMAT
public static final String NON_GTID_TRANSACTION_ID_FORMAT
- See Also:
- Constant Field Values
-
sourceInfoSchema
private final org.apache.kafka.connect.data.Schema sourceInfoSchema
-
sourceInfo
private final SourceInfo sourceInfo
-
snapshotCompleted
private boolean snapshotCompleted
-
transactionContext
private final TransactionContext transactionContext
-
incrementalSnapshotContext
private final IncrementalSnapshotContext<TableId> incrementalSnapshotContext
-
restartGtidSet
private String restartGtidSet
-
currentGtidSet
private String currentGtidSet
-
restartBinlogFilename
private String restartBinlogFilename
-
restartBinlogPosition
private long restartBinlogPosition
-
restartRowsToSkip
private int restartRowsToSkip
-
restartEventsToSkip
private long restartEventsToSkip
-
currentEventLengthInBytes
private long currentEventLengthInBytes
-
inTransaction
private boolean inTransaction
-
transactionId
private String transactionId
-
-
Constructor Detail
-
MySqlOffsetContext
public MySqlOffsetContext(MySqlConnectorConfig connectorConfig, boolean snapshot, boolean snapshotCompleted, TransactionContext transactionContext, IncrementalSnapshotContext<TableId> incrementalSnapshotContext, SourceInfo sourceInfo)
-
MySqlOffsetContext
public MySqlOffsetContext(MySqlConnectorConfig connectorConfig, boolean snapshot, boolean snapshotCompleted, SourceInfo sourceInfo)
-
-
Method Detail
-
getPartition
public Map<String,?> getPartition()
Get the Kafka Connect detail about the source "partition", which describes the portion of the source that we are consuming. Since we're reading the binary log for a single database, the source partition specifies thedatabase server
.The resulting map is mutable for efficiency reasons (this information rarely changes), but should not be mutated.
- Specified by:
getPartition
in interfaceOffsetContext
- Returns:
- the source partition information; never null
-
getOffset
public Map<String,?> getOffset()
- Specified by:
getOffset
in interfaceOffsetContext
-
getSourceInfoSchema
public org.apache.kafka.connect.data.Schema getSourceInfoSchema()
- Specified by:
getSourceInfoSchema
in interfaceOffsetContext
-
getSourceInfo
public org.apache.kafka.connect.data.Struct getSourceInfo()
- Specified by:
getSourceInfo
in interfaceOffsetContext
-
isSnapshotRunning
public boolean isSnapshotRunning()
- Specified by:
isSnapshotRunning
in interfaceOffsetContext
-
isSnapshotCompleted
public boolean isSnapshotCompleted()
-
preSnapshotStart
public void preSnapshotStart()
- Specified by:
preSnapshotStart
in interfaceOffsetContext
-
preSnapshotCompletion
public void preSnapshotCompletion()
- Specified by:
preSnapshotCompletion
in interfaceOffsetContext
-
postSnapshotCompletion
public void postSnapshotCompletion()
- Specified by:
postSnapshotCompletion
in interfaceOffsetContext
-
setTransactionId
private void setTransactionId()
-
resetTransactionId
private void resetTransactionId()
-
getTransactionId
public String getTransactionId()
-
setInitialSkips
public void setInitialSkips(long restartEventsToSkip, int restartRowsToSkip)
-
initial
public static MySqlOffsetContext initial(MySqlConnectorConfig config)
-
markLastSnapshotRecord
public void markLastSnapshotRecord()
- Specified by:
markLastSnapshotRecord
in interfaceOffsetContext
-
event
public void event(DataCollectionId tableId, Instant timestamp)
- Specified by:
event
in interfaceOffsetContext
-
getTransactionContext
public TransactionContext getTransactionContext()
- Specified by:
getTransactionContext
in interfaceOffsetContext
-
incrementalSnapshotEvents
public void incrementalSnapshotEvents()
- Specified by:
incrementalSnapshotEvents
in interfaceOffsetContext
-
getIncrementalSnapshotContext
public IncrementalSnapshotContext<?> getIncrementalSnapshotContext()
- Specified by:
getIncrementalSnapshotContext
in interfaceOffsetContext
-
setBinlogStartPoint
public void setBinlogStartPoint(String binlogFilename, long positionOfFirstEvent)
Set the position in the MySQL binlog where we will start reading.- Parameters:
binlogFilename
- the name of the binary log file; may not be nullpositionOfFirstEvent
- the position in the binary log file to begin processing
-
setCompletedGtidSet
public void setCompletedGtidSet(String gtidSet)
Set the GTID set that captures all of the GTID transactions that have been completely processed.- Parameters:
gtidSet
- the string representation of the GTID set; may not be null, but may be an empty string if no GTIDs have been previously processed
-
gtidSet
public String gtidSet()
Get the string representation of the GTID range for the MySQL binary log file.- Returns:
- the string representation of the binlog GTID ranges; may be null
-
startGtid
public void startGtid(String gtid, String gtidSet)
Record that a new GTID transaction has been started and has been included in the set of GTIDs known to the MySQL server.- Parameters:
gtid
- the string representation of a specific GTID that has been begun; may not be nullgtidSet
- the string representation of GTID set that includes the newly begun GTID; may not be null
-
getSource
public SourceInfo getSource()
-
startNextTransaction
public void startNextTransaction()
-
commitTransaction
public void commitTransaction()
-
completeEvent
public void completeEvent()
Capture that we're starting a new event.
-
setEventPosition
public void setEventPosition(long positionOfCurrentEvent, long eventSizeInBytes)
Set the position within the MySQL binary log file of the current event.- Parameters:
positionOfCurrentEvent
- the position within the binary log file of the current eventeventSizeInBytes
- the size in bytes of this event
-
setQuery
public void setQuery(String query)
Set the original SQL query.- Parameters:
query
- the original SQL query that generated the event.
-
changeEventCompleted
public void changeEventCompleted()
-
eventsToSkipUponRestart
public long eventsToSkipUponRestart()
Get the number of events after the last transaction BEGIN that we've already processed.- Returns:
- the number of events in the transaction that have been processed completely
- See Also:
completeEvent()
,startNextTransaction()
-
rowsToSkipUponRestart
public int rowsToSkipUponRestart()
Get the number of rows beyond thelast completely processed event
to be skipped upon restart.- Returns:
- the number of rows to be skipped
-
setRowNumber
public void setRowNumber(int eventRowNumber, int totalNumberOfRows)
Given the row number within a binlog event and the total number of rows in that event, compute the Kafka Connect offset that is be included in the produced change event describing the row.This method should always be called before
#struct()
.- Parameters:
eventRowNumber
- the 0-based row number within the event for which the offset is to be producedtotalNumberOfRows
- the total number of rows within the event being processed- See Also:
#struct()
-
setBinlogThread
public void setBinlogThread(long threadId)
-
-