Package io.debezium.connector.mysql

Class MySqlStreamingChangeEventSource

java.lang.Object
io.debezium.connector.mysql.MySqlStreamingChangeEventSource
All Implemented Interfaces:
ChangeEventSource, StreamingChangeEventSource<MySqlPartition,MySqlOffsetContext>
public class MySqlStreamingChangeEventSource extends Object implements StreamingChangeEventSource<MySqlPartition,MySqlOffsetContext>
Author:
Jiri Pechanec

  • Field Details

  • Constructor Details

  • Method Details

    • onEvent

      protected void onEvent(MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)

    • getEventTimestamp

      private Instant getEventTimestamp(com.github.shyiko.mysql.binlog.event.Event event, long eventTs)

    • ignoreEvent

      protected void ignoreEvent(MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)

    • handleEvent

      protected void handleEvent(MySqlPartition partition, MySqlOffsetContext offsetContext, ChangeEventSource.ChangeEventSourceContext context, com.github.shyiko.mysql.binlog.event.Event event)

    • unwrapData

      protected <T extends com.github.shyiko.mysql.binlog.event.EventData> T unwrapData(com.github.shyiko.mysql.binlog.event.Event event)

    • handleServerStop

      protected void handleServerStop(MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)
      Handle the supplied event that signals that mysqld has stopped.
      Parameters:
      event - the server stopped event to be processed; may not be null

    • handleServerHeartbeat

      protected void handleServerHeartbeat(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Handle the supplied event that is sent by a primary to a replica to let the replica know that the primary is still alive. Not written to a binary log.
      Parameters:
      event - the server stopped event to be processed; may not be null
      Throws:
      InterruptedException

    • handleServerIncident

      protected void handleServerIncident(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)
      Handle the supplied event that signals that an out of the ordinary event that occurred on the master. It notifies the replica that something happened on the primary that might cause data to be in an inconsistent state.
      Parameters:
      event - the server stopped event to be processed; may not be null

    • handleRotateLogsEvent

      protected void handleRotateLogsEvent(MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)
      Handle the supplied event with a RotateEventData that signals the logs are being rotated. This means that either the server was restarted, or the binlog has transitioned to a new file. In either case, subsequent table numbers will be different than those seen to this point.
      Parameters:
      event - the database change data event to be processed; may not be null

    • handleGtidEvent

      protected void handleGtidEvent(MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)
      Handle the supplied event with a GtidEventData that signals the beginning of a GTID transaction. We don't yet know whether this transaction contains any events we're interested in, but we have to record it so that we know the position of this event and know we've processed the binlog to this point.

      Note that this captures the current GTID and complete GTID set, regardless of whether the connector is filtering the GTID set upon connection. We do this because we actually want to capture all GTID set values found in the binlog, whether or not we process them. However, only when we connect do we actually want to pass to MySQL only those GTID ranges that are applicable per the configuration.

      Parameters:
      event - the GTID event to be processed; may not be null

    • handleMariaDbGtidEvent

      protected void handleMariaDbGtidEvent(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Throws:
      InterruptedException

    • handleRecordingQuery

      protected void handleRecordingQuery(MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)
      Handle the supplied event with an RowsQueryEventData or AnnotateRowsEventData by recording the original SQL query that generated the event.
      Parameters:
      event - the database change data event to be processed; may not be null

    • handleQueryEvent

      protected void handleQueryEvent(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Handle the supplied event with an QueryEventData by possibly recording the DDL statements as changes in the MySQL schemas.
      Parameters:
      partition - the partition in which the even occurred
      event - the database change data event to be processed; may not be null
      Throws:
      InterruptedException - if this thread is interrupted while recording the DDL statements

    • handleTransactionBegin

      private void handleTransactionBegin(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event, Long threadId) throws InterruptedException
      Throws:
      InterruptedException

    • handleTransactionCompletion

      private void handleTransactionCompletion(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Throws:
      InterruptedException

    • handleUpdateTableMetadata

      protected void handleUpdateTableMetadata(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Handle a change in the table metadata.

      This method should be called whenever we consume a TABLE_MAP event, and every transaction in the log should include one of these for each table affected by the transaction. Each table map event includes a monotonically-increasing numeric identifier, and this identifier is used within subsequent events within the same transaction. This table identifier can change when:

      1. the table structure is modified (e.g., via an ALTER TABLE ... command); or
      2. MySQL rotates to a new binary log file, even if the table structure does not change.
      Parameters:
      event - the update event; never null
      Throws:
      InterruptedException

    • handleTransactionPayload

      protected void handleTransactionPayload(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Handle an event of type TRANSACTION_PAYLOAD_EVENT

      This method should be called whenever a transaction payload event is encountered by the mysql binlog connector. A Transaction payload event is propagated from the binlog connector when compression is turned on over binlog. This method loops over the individual events in the compressed binlog and calls the respective atomic event handlers.

      Throws:
      InterruptedException

    • informAboutUnknownTableIfRequired

      private void informAboutUnknownTableIfRequired(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event, TableId tableId, Envelope.Operation operation) throws InterruptedException
      If we receive an event for a table that is monitored but whose metadata we don't know, either ignore that event or raise a warning or error as per the MySqlConnectorConfig.INCONSISTENT_SCHEMA_HANDLING_MODE configuration.
      Throws:
      InterruptedException

    • informAboutUnknownTableIfRequired

      private void informAboutUnknownTableIfRequired(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event, TableId tableId) throws InterruptedException
      Throws:
      InterruptedException

    • validateChangeEventWithTable

      private void validateChangeEventWithTable(Table table, Object[] before, Object[] after)

    • handleInsert

      protected void handleInsert(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Generate source records for the supplied event with an WriteRowsEventData.
      Parameters:
      partition - the partition in which the even occurred
      event - the database change data event to be processed; may not be null
      Throws:
      InterruptedException - if this thread is interrupted while blocking

    • handleUpdate

      protected void handleUpdate(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Generate source records for the supplied event with an UpdateRowsEventData.
      Parameters:
      partition - the partition in which the even occurred
      event - the database change data event to be processed; may not be null
      Throws:
      InterruptedException - if this thread is interrupted while blocking

    • handleDelete

      protected void handleDelete(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Generate source records for the supplied event with an DeleteRowsEventData.
      Parameters:
      partition - the partition in which the even occurred
      event - the database change data event to be processed; may not be null
      Throws:
      InterruptedException - if this thread is interrupted while blocking

    • handleChange

      private <T extends com.github.shyiko.mysql.binlog.event.EventData, U> void handleChange(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event, Envelope.Operation operation, Class<T> eventDataClass, MySqlStreamingChangeEventSource.TableIdProvider<T> tableIdProvider, MySqlStreamingChangeEventSource.RowsProvider<T,U> rowsProvider, MySqlStreamingChangeEventSource.BinlogChangeEmitter<U> changeEmitter, MySqlStreamingChangeEventSource.ChangeEventValidator<U> changeEventValidator) throws InterruptedException
      Throws:
      InterruptedException

    • viewChange

      protected void viewChange(MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Handle a EventType.VIEW_CHANGE event.
      Parameters:
      event - the database change data event to be processed; may not be null
      Throws:
      InterruptedException - if this thread is interrupted while blocking

    • prepareTransaction

      protected void prepareTransaction(MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) throws InterruptedException
      Handle a EventType.XA_PREPARE event.
      Parameters:
      event - the database change data event to be processed; may not be null
      Throws:
      InterruptedException - if this thread is interrupted while blocking

    • init

      public void init(MySqlOffsetContext offsetContext)
      Specified by:
      init in interface StreamingChangeEventSource<MySqlPartition,MySqlOffsetContext>

    • execute

      public void execute(ChangeEventSource.ChangeEventSourceContext context, MySqlPartition partition, MySqlOffsetContext offsetContext) throws InterruptedException
      Specified by:
      execute in interface StreamingChangeEventSource<MySqlPartition,MySqlOffsetContext>
      Throws:
      InterruptedException

    • waitWhenStreamingPaused

      private void waitWhenStreamingPaused(ChangeEventSource.ChangeEventSourceContext context) throws InterruptedException
      Throws:
      InterruptedException

    • getOffsetContext

      public MySqlOffsetContext getOffsetContext()
      Specified by:
      getOffsetContext in interface StreamingChangeEventSource<MySqlPartition,MySqlOffsetContext>

    • logStreamingSourceState

      private void logStreamingSourceState()

    • logEvent

      protected void logEvent(MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)

    • logStreamingSourceState

      private void logStreamingSourceState(org.slf4j.event.Level severity)

    • getMetrics

      MySqlStreamingChangeEventSourceMetrics getMetrics()

    • rewindBinaryLogClient

      void rewindBinaryLogClient(ChangeEventSource.ChangeEventSourceContext context, MySqlStreamingChangeEventSource.BinlogPosition position)

    • getCurrentBinlogPosition

      MySqlStreamingChangeEventSource.BinlogPosition getCurrentBinlogPosition()

    • wrap

      protected io.debezium.DebeziumException wrap(Throwable error)
      Wraps the specified exception in a DebeziumException, ensuring that all useful state is captured inside the new exception's message.
      Parameters:
      error - the exception; may not be null
      Returns:
      the wrapped Kafka Connect exception