Package io.debezium.connector.mysql
Class EventBuffer
java.lang.Object
io.debezium.connector.mysql.EventBuffer
This class represents a look-ahead buffer that allows Debezium to accumulate binlog events and decide
if the last event in transaction is either
ROLLBACK
or COMMIT
. The incoming events are either
supposed to be in transaction or out-of-transaction. When out-of-transaction they are sent directly into
the destination handler. When in transaction the event goes through the buffering.
The reason for the buffering is that the binlog contains rolled back transactions in some cases. E.g. that's the case when a temporary table is dropped (see DBZ-390). For rolled back transactions we may not propagate any of the contained events, hence the buffering is applied.
The transaction start is identified by a BEGIN
event. Transaction is ended either by COMMIT
event or by XID
an event.
If there are more events that can fit to the buffer then:
- Binlog position is recorded for the first event not fitting into the buffer
- Binlog position is recorded for the commit event
- Buffer content is sent to the final handler
- Binlog position is rewound and all events between the above recorded positions are sent to the final handler
- Author:
- Jiri Pechanec
-
Field Summary
Modifier and TypeFieldDescriptionprivate final Queue<com.github.shyiko.mysql.binlog.event.Event>
private final int
private final ChangeEventSource.ChangeEventSourceContext
Contains the position of the last event belonging to the transaction that has not fit into the buffer.Contains the position of the first event that has not fit into the buffer.private static final org.slf4j.Logger
private final MySqlStreamingChangeEventSource
private boolean
-
Constructor Summary
ConstructorDescriptionEventBuffer
(int capacity, MySqlStreamingChangeEventSource streamingChangeEventSource, ChangeEventSource.ChangeEventSourceContext changeEventSourceContext) -
Method Summary
Modifier and TypeMethodDescriptionvoid
add
(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) An entry point to the buffer that should be used by BinlogReader to push events.private void
addToBuffer
(com.github.shyiko.mysql.binlog.event.Event event) Adds an event to the buffer if there is a space available.private void
beginTransaction
(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) private void
clear()
Cleans-up the buffer after the transaction is either thrown away or streamed into a Kafka topicprivate void
completeTransaction
(MySqlPartition partition, MySqlOffsetContext offsetContext, boolean wellFormed, com.github.shyiko.mysql.binlog.event.Event event) Sends all events from the buffer int a final handler.private void
consumeEvent
(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) private boolean
private boolean
Whether we are replaying TX events from binlog that have not fit into the buffer beforeprivate void
private void
-
Field Details
-
LOGGER
private static final org.slf4j.Logger LOGGER -
capacity
private final int capacity -
buffer
-
streamingChangeEventSource
-
txStarted
private boolean txStarted -
changeEventSourceContext
-
largeTxNotBufferedPosition
Contains the position of the first event that has not fit into the buffer. -
forwardTillPosition
Contains the position of the last event belonging to the transaction that has not fit into the buffer.
-
-
Constructor Details
-
EventBuffer
EventBuffer(int capacity, MySqlStreamingChangeEventSource streamingChangeEventSource, ChangeEventSource.ChangeEventSourceContext changeEventSourceContext)
-
-
Method Details
-
add
public void add(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) An entry point to the buffer that should be used by BinlogReader to push events.- Parameters:
partition
- the partition to which the event belongsevent
- to be stored in the buffer
-
isReplayingEventsBeyondBufferCapacity
private boolean isReplayingEventsBeyondBufferCapacity()Whether we are replaying TX events from binlog that have not fit into the buffer before -
addToBuffer
private void addToBuffer(com.github.shyiko.mysql.binlog.event.Event event) Adds an event to the buffer if there is a space available. Records binlog position for the first event that does not fit for later replay.- Parameters:
event
-
-
switchToBufferFullMode
private void switchToBufferFullMode() -
isInBufferFullMode
private boolean isInBufferFullMode() -
consumeEvent
private void consumeEvent(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) -
beginTransaction
private void beginTransaction(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event) -
completeTransaction
private void completeTransaction(MySqlPartition partition, MySqlOffsetContext offsetContext, boolean wellFormed, com.github.shyiko.mysql.binlog.event.Event event) Sends all events from the buffer int a final handler. For large transactions it executes rewind of binlog reader back to the first event that was not stored in the buffer.- Parameters:
partition
- the partition where the transaction was committedwellFormed
-event
-
-
rollbackTransaction
private void rollbackTransaction() -
clear
private void clear()Cleans-up the buffer after the transaction is either thrown away or streamed into a Kafka topic
-