Class SourceInfo
- java.lang.Object
-
- io.debezium.connector.AbstractSourceInfo
-
- io.debezium.connector.common.BaseSourceInfo
-
- io.debezium.connector.mysql.SourceInfo
-
@NotThreadSafe final class SourceInfo extends BaseSourceInfo
Information about the source of information, which includes the position in the source binary log we have previously processed.The
source partition
information describes the database whose log is being consumed. Typically, the database is identified by the host address port number of the MySQL server and the name of the database. Here's a JSON-like representation of an example database:{ "server" : "production-server" }
The
source offset
information is included in each event and captures where the connector should restart if this event's offset is the last one recorded. The offset includes thebinlog filename
, theposition of the first event
in the binlog, thenumber of events to skip
, and thenumber of rows to also skip
.Here's a JSON-like representation of an example offset:
{ "server_id": 112233, "ts_sec": 1465937, "gtid": "db58b0ae-2c10-11e6-b284-0242ac110002:199", "file": "mysql-bin.000003", "pos" = 990, "event" = 0, "row": 0, "snapshot": true }
The "
gtids
" field only appears in offsets produced when GTIDs are enabled. The "snapshot
" field only appears in offsets produced when the connector is in the middle of a snapshot. And finally, the "ts
" field contains the seconds since Unix epoch (since Jan 1, 1970) of the MySQL event; the messageenvelopes
also have a timestamp, but that timestamp is the milliseconds since since Jan 1, 1970.Each change event envelope also includes the
source
struct that contains MySQL information about that particular event, including a mixture the fields from thepartition
(which is renamed in the source to make more sense), the binlog filename and position where the event can be found, and when GTIDs are enabled the GTID of the transaction in which the event occurs. Like with the offset, the "snapshot
" field only appears for events produced when the connector is in the middle of a snapshot. Note that this information is likely different than the offset information, since the connector may need to restart from either just after the most recently completed transaction or the beginning of the most recently started transaction (whichever appears later in the binlog).Here's a JSON-like representation of the source for the metadata for an event that corresponds to the above partition and offset:
{ "name": "production-server", "server_id": 112233, "ts_sec": 1465937, "gtid": "db58b0ae-2c10-11e6-b284-0242ac110002:199", "file": "mysql-bin.000003", "pos" = 1081, "row": 0, "snapshot": true, "thread" : 1, "db" : "inventory", "table" : "products" }
- Author:
- Randall Hauch
-
-
Field Summary
Fields Modifier and Type Field Description static String
BINLOG_FILENAME_OFFSET_KEY
static String
BINLOG_POSITION_OFFSET_KEY
static String
BINLOG_ROW_IN_EVENT_OFFSET_KEY
private String
currentBinlogFilename
private long
currentBinlogPosition
private String
currentGtid
private String
currentQuery
private int
currentRowNumber
private String
databaseName
static String
GTID_KEY
static String
QUERY_KEY
static String
SERVER_ID_KEY
private long
serverId
private Instant
sourceTime
private Set<TableId>
tableIds
static String
THREAD_KEY
private long
threadId
-
Fields inherited from class io.debezium.connector.common.BaseSourceInfo
snapshotRecord
-
Fields inherited from class io.debezium.connector.AbstractSourceInfo
COLLECTION_NAME_KEY, DATABASE_NAME_KEY, DEBEZIUM_CONNECTOR_KEY, DEBEZIUM_VERSION_KEY, SCHEMA_NAME_KEY, SEQUENCE_KEY, SERVER_NAME_KEY, SNAPSHOT_KEY, TABLE_NAME_KEY, TIMESTAMP_KEY
-
-
Constructor Summary
Constructors Constructor Description SourceInfo(MySqlConnectorConfig connectorConfig)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description String
binlogFilename()
Get the name of the MySQL binary log file that has last been processed.long
binlogPosition()
Get the position within the MySQL binary log file of the next event to be processed.static Document
createDocumentFromOffset(Map<String,?> offset)
Create aDocument
from the given offset.protected String
database()
void
databaseEvent(String databaseName)
(package private) long
getBinlogTimestampSeconds()
(package private) String
getCurrentBinlogFilename()
(package private) long
getCurrentBinlogPosition()
(package private) String
getCurrentGtid()
(package private) int
getCurrentRowNumber()
String
getQuery()
(package private) long
getServerId()
(package private) long
getThreadId()
boolean
isLastSnapshot()
Deprecated.void
setBinlogPosition(String binlogFilename, long positionOfFirstEvent)
Set the position in the MySQL binlog where we will start reading.void
setBinlogServerId(long serverId)
Set the server ID as found within the MySQL binary log file.void
setBinlogThread(long threadId)
Set the identifier of the MySQL thread that generated the most recent event.void
setBinlogTimestampSeconds(long timestampInSeconds)
Set the number of seconds since Unix epoch (January 1, 1970) as found within the MySQL binary log file.void
setEventPosition(long positionOfCurrentEvent)
Set the position within the MySQL binary log file of the current event.void
setQuery(String query)
Set the original SQL query.void
setRowNumber(int eventRowNumber)
Given the row number within a binlog event and the total number of rows in that event, compute the Kafka Connect offset that is be included in the produced change event describing the row.void
setSourceTime(Instant timestamp)
void
startGtid(String gtid)
(package private) String
table()
Returns a string representation of the table(s) affected by the current event.void
tableEvent(TableId tableId)
void
tableEvent(Set<TableId> tableIds)
protected Instant
timestamp()
String
toString()
-
Methods inherited from class io.debezium.connector.common.BaseSourceInfo
isSnapshot, setSnapshot, snapshot
-
Methods inherited from class io.debezium.connector.AbstractSourceInfo
schema, sequence, serverName, struct, structMaker
-
-
-
-
Field Detail
-
SERVER_ID_KEY
public static final String SERVER_ID_KEY
- See Also:
- Constant Field Values
-
GTID_KEY
public static final String GTID_KEY
- See Also:
- Constant Field Values
-
BINLOG_FILENAME_OFFSET_KEY
public static final String BINLOG_FILENAME_OFFSET_KEY
- See Also:
- Constant Field Values
-
BINLOG_POSITION_OFFSET_KEY
public static final String BINLOG_POSITION_OFFSET_KEY
- See Also:
- Constant Field Values
-
BINLOG_ROW_IN_EVENT_OFFSET_KEY
public static final String BINLOG_ROW_IN_EVENT_OFFSET_KEY
- See Also:
- Constant Field Values
-
THREAD_KEY
public static final String THREAD_KEY
- See Also:
- Constant Field Values
-
QUERY_KEY
public static final String QUERY_KEY
- See Also:
- Constant Field Values
-
currentGtid
private String currentGtid
-
currentBinlogFilename
private String currentBinlogFilename
-
currentBinlogPosition
private long currentBinlogPosition
-
currentRowNumber
private int currentRowNumber
-
serverId
private long serverId
-
sourceTime
private Instant sourceTime
-
threadId
private long threadId
-
currentQuery
private String currentQuery
-
databaseName
private String databaseName
-
-
Constructor Detail
-
SourceInfo
public SourceInfo(MySqlConnectorConfig connectorConfig)
-
-
Method Detail
-
setQuery
public void setQuery(String query)
Set the original SQL query.- Parameters:
query
- the original SQL query that generated the event.
-
getQuery
public String getQuery()
- Returns:
- the original SQL query that generated the event. NULL if no such query is associated.
-
setBinlogPosition
public void setBinlogPosition(String binlogFilename, long positionOfFirstEvent)
Set the position in the MySQL binlog where we will start reading.- Parameters:
binlogFilename
- the name of the binary log file; may not be nullpositionOfFirstEvent
- the position in the binary log file to begin processing
-
setEventPosition
public void setEventPosition(long positionOfCurrentEvent)
Set the position within the MySQL binary log file of the current event.- Parameters:
positionOfCurrentEvent
- the position within the binary log file of the current event
-
setRowNumber
public void setRowNumber(int eventRowNumber)
Given the row number within a binlog event and the total number of rows in that event, compute the Kafka Connect offset that is be included in the produced change event describing the row.This method should always be called before
AbstractSourceInfo.struct()
.- Parameters:
eventRowNumber
- the 0-based row number within the event for which the offset is to be produced- See Also:
AbstractSourceInfo.struct()
-
databaseEvent
public void databaseEvent(String databaseName)
-
tableEvent
public void tableEvent(TableId tableId)
-
startGtid
public void startGtid(String gtid)
-
setBinlogServerId
public void setBinlogServerId(long serverId)
Set the server ID as found within the MySQL binary log file.- Parameters:
serverId
- the server ID found within the binary log file
-
setBinlogTimestampSeconds
public void setBinlogTimestampSeconds(long timestampInSeconds)
Set the number of seconds since Unix epoch (January 1, 1970) as found within the MySQL binary log file. Note that the value in the binlog events is in seconds, but the library we use returns the value in milliseconds (with only second precision and therefore all fractions of a second are zero). We capture this as seconds since that is the precision that MySQL uses.- Parameters:
timestampInSeconds
- the timestamp in seconds found within the binary log file
-
setSourceTime
public void setSourceTime(Instant timestamp)
-
setBinlogThread
public void setBinlogThread(long threadId)
Set the identifier of the MySQL thread that generated the most recent event.- Parameters:
threadId
- the thread identifier; may be negative if not known
-
binlogFilename
public String binlogFilename()
Get the name of the MySQL binary log file that has last been processed.- Returns:
- the name of the binary log file; null if it has not been
set
-
binlogPosition
public long binlogPosition()
Get the position within the MySQL binary log file of the next event to be processed.- Returns:
- the position within the binary log file; null if it has not been
set
-
getServerId
long getServerId()
-
getThreadId
long getThreadId()
-
table
String table()
Returns a string representation of the table(s) affected by the current event. Will only represent more than a single table for events in the user-facing schema history topic for certain types of DDL. Will benull
for DDL events not applying to tables (CREATE DATABASE etc.).
-
getCurrentGtid
String getCurrentGtid()
-
getCurrentBinlogFilename
String getCurrentBinlogFilename()
-
getCurrentBinlogPosition
long getCurrentBinlogPosition()
-
getBinlogTimestampSeconds
long getBinlogTimestampSeconds()
-
getCurrentRowNumber
int getCurrentRowNumber()
-
createDocumentFromOffset
public static Document createDocumentFromOffset(Map<String,?> offset)
Create aDocument
from the given offset.- Parameters:
offset
- the offset to create the document from.- Returns:
- a
Document
with the offset data.
-
timestamp
protected Instant timestamp()
- Specified by:
timestamp
in classAbstractSourceInfo
-
database
protected String database()
- Specified by:
database
in classAbstractSourceInfo
-
isLastSnapshot
@Deprecated public boolean isLastSnapshot()
Deprecated.
-
-