Class SourceInfo
The source partition
information describes the database whose log is being consumed. Typically, the
database is identified by the host address port number of the MySQL server and the name of the database. Here's a JSON-like
representation of an example database:
{ "server" : "production-server" }
The source offset
information is included in each event and captures where the connector should restart
if this event's offset is the last one recorded. The offset includes the binlog filename
,
the position of the first event
in the binlog, the
number of events to skip
, and the
number of rows to also skip
.
Here's a JSON-like representation of an example offset:
{ "server_id": 112233, "ts_sec": 1465937, "gtid": "db58b0ae-2c10-11e6-b284-0242ac110002:199", "file": "mysql-bin.000003", "pos" = 990, "event" = 0, "row": 0, "snapshot": true }
The "gtids
" field only appears in offsets produced when GTIDs are enabled. The "snapshot
" field only appears in
offsets produced when the connector is in the middle of a snapshot. And finally, the "ts
" field contains the
seconds since Unix epoch (since Jan 1, 1970) of the MySQL event; the message envelopes
also have a
timestamp, but that timestamp is the milliseconds since since Jan 1, 1970.
Each change event envelope also includes the source
struct that contains MySQL information about that
particular event, including a mixture the fields from the partition
(which is renamed in the source to
make more sense), the binlog filename and position where the event can be found, and when GTIDs are enabled the GTID of the
transaction in which the event occurs. Like with the offset, the "snapshot
" field only appears for events produced
when the connector is in the middle of a snapshot. Note that this information is likely different than the offset information,
since the connector may need to restart from either just after the most recently completed transaction or the beginning
of the most recently started transaction (whichever appears later in the binlog).
Here's a JSON-like representation of the source for the metadata for an event that corresponds to the above partition and offset:
{ "name": "production-server", "server_id": 112233, "ts_sec": 1465937, "gtid": "db58b0ae-2c10-11e6-b284-0242ac110002:199", "file": "mysql-bin.000003", "pos" = 1081, "row": 0, "snapshot": true, "thread" : 1, "db" : "inventory", "table" : "products" }
- Author:
- Randall Hauch
-
Field Summary
Modifier and TypeFieldDescriptionstatic final String
static final String
static final String
private String
private long
private String
private String
private int
private String
static final String
static final String
static final String
private long
private Instant
static final String
private long
Fields inherited from class io.debezium.connector.common.BaseSourceInfo
snapshotRecord
Fields inherited from class io.debezium.connector.AbstractSourceInfo
COLLECTION_NAME_KEY, DATABASE_NAME_KEY, DEBEZIUM_CONNECTOR_KEY, DEBEZIUM_VERSION_KEY, SCHEMA_NAME_KEY, SEQUENCE_KEY, SERVER_NAME_KEY, SNAPSHOT_KEY, TABLE_NAME_KEY, TIMESTAMP_KEY
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionGet the name of the MySQL binary log file that has last been processed.long
Get the position within the MySQL binary log file of the next event to be processed.static Document
createDocumentFromOffset
(Map<String, ?> offset) Create aDocument
from the given offset.protected String
database()
void
databaseEvent
(String databaseName) (package private) long
(package private) String
(package private) long
(package private) String
(package private) int
getQuery()
(package private) long
(package private) long
boolean
Deprecated.void
setBinlogPosition
(String binlogFilename, long positionOfFirstEvent) Set the position in the MySQL binlog where we will start reading.void
setBinlogServerId
(long serverId) Set the server ID as found within the MySQL binary log file.void
setBinlogThread
(long threadId) Set the identifier of the MySQL thread that generated the most recent event.void
setBinlogTimestampSeconds
(long timestampInSeconds) Set the number of seconds since Unix epoch (January 1, 1970) as found within the MySQL binary log file.void
setEventPosition
(long positionOfCurrentEvent) Set the position within the MySQL binary log file of the current event.void
Set the original SQL query.void
setRowNumber
(int eventRowNumber) Given the row number within a binlog event and the total number of rows in that event, compute the Kafka Connect offset that is be included in the produced change event describing the row.void
setSourceTime
(Instant timestamp) void
(package private) String
table()
Returns a string representation of the table(s) affected by the current event.void
tableEvent
(TableId tableId) void
tableEvent
(Set<TableId> tableIds) protected Instant
toString()
Methods inherited from class io.debezium.connector.common.BaseSourceInfo
isSnapshot, setSnapshot, snapshot
Methods inherited from class io.debezium.connector.AbstractSourceInfo
schema, sequence, serverName, struct, structMaker
-
Field Details
-
SERVER_ID_KEY
- See Also:
-
GTID_KEY
- See Also:
-
BINLOG_FILENAME_OFFSET_KEY
- See Also:
-
BINLOG_POSITION_OFFSET_KEY
- See Also:
-
BINLOG_ROW_IN_EVENT_OFFSET_KEY
- See Also:
-
THREAD_KEY
- See Also:
-
QUERY_KEY
- See Also:
-
currentGtid
-
currentBinlogFilename
-
currentBinlogPosition
private long currentBinlogPosition -
currentRowNumber
private int currentRowNumber -
serverId
private long serverId -
sourceTime
-
threadId
private long threadId -
currentQuery
-
tableIds
-
databaseName
-
-
Constructor Details
-
SourceInfo
-
-
Method Details
-
setQuery
Set the original SQL query.- Parameters:
query
- the original SQL query that generated the event.
-
getQuery
- Returns:
- the original SQL query that generated the event. NULL if no such query is associated.
-
setBinlogPosition
Set the position in the MySQL binlog where we will start reading.- Parameters:
binlogFilename
- the name of the binary log file; may not be nullpositionOfFirstEvent
- the position in the binary log file to begin processing
-
setEventPosition
public void setEventPosition(long positionOfCurrentEvent) Set the position within the MySQL binary log file of the current event.- Parameters:
positionOfCurrentEvent
- the position within the binary log file of the current event
-
setRowNumber
public void setRowNumber(int eventRowNumber) Given the row number within a binlog event and the total number of rows in that event, compute the Kafka Connect offset that is be included in the produced change event describing the row.This method should always be called before
AbstractSourceInfo.struct()
.- Parameters:
eventRowNumber
- the 0-based row number within the event for which the offset is to be produced- See Also:
-
databaseEvent
-
tableEvent
-
tableEvent
-
startGtid
-
setBinlogServerId
public void setBinlogServerId(long serverId) Set the server ID as found within the MySQL binary log file.- Parameters:
serverId
- the server ID found within the binary log file
-
setBinlogTimestampSeconds
public void setBinlogTimestampSeconds(long timestampInSeconds) Set the number of seconds since Unix epoch (January 1, 1970) as found within the MySQL binary log file. Note that the value in the binlog events is in seconds, but the library we use returns the value in milliseconds (with only second precision and therefore all fractions of a second are zero). We capture this as seconds since that is the precision that MySQL uses.- Parameters:
timestampInSeconds
- the timestamp in seconds found within the binary log file
-
setSourceTime
-
setBinlogThread
public void setBinlogThread(long threadId) Set the identifier of the MySQL thread that generated the most recent event.- Parameters:
threadId
- the thread identifier; may be negative if not known
-
binlogFilename
Get the name of the MySQL binary log file that has last been processed.- Returns:
- the name of the binary log file; null if it has not been
set
-
binlogPosition
public long binlogPosition()Get the position within the MySQL binary log file of the next event to be processed.- Returns:
- the position within the binary log file; null if it has not been
set
-
getServerId
long getServerId() -
getThreadId
long getThreadId() -
table
String table()Returns a string representation of the table(s) affected by the current event. Will only represent more than a single table for events in the user-facing schema history topic for certain types of DDL. Will benull
for DDL events not applying to tables (CREATE DATABASE etc.). -
getCurrentGtid
String getCurrentGtid() -
getCurrentBinlogFilename
String getCurrentBinlogFilename() -
getCurrentBinlogPosition
long getCurrentBinlogPosition() -
getBinlogTimestampSeconds
long getBinlogTimestampSeconds() -
getCurrentRowNumber
int getCurrentRowNumber() -
toString
-
createDocumentFromOffset
Create aDocument
from the given offset.- Parameters:
offset
- the offset to create the document from.- Returns:
- a
Document
with the offset data.
-
timestamp
- Specified by:
timestamp
in classAbstractSourceInfo
-
database
- Specified by:
database
in classAbstractSourceInfo
-
isLastSnapshot
Deprecated.
-