Package io.debezium.connector.mysql
Class EventBuffer
- java.lang.Object
-
- io.debezium.connector.mysql.EventBuffer
-
class EventBuffer extends Object
This class represents a look-ahead buffer that allows Debezium to accumulate binlog events and decide if the last event in transaction is eitherROLLBACK
orCOMMIT
. The incoming events are either supposed to be in transaction or out-of-transaction. When out-of-transaction they are sent directly into the destination handler. When in transaction the event goes through the buffering.The reason for the buffering is that the binlog contains rolled back transactions in some cases. E.g. that's the case when a temporary table is dropped (see DBZ-390). For rolled back transactions we may not propagate any of the contained events, hence the buffering is applied.
The transaction start is identified by a
BEGIN
event. Transaction is ended either byCOMMIT
event or byXID
an event.If there are more events that can fit to the buffer then:
- Binlog position is recorded for the first event not fitting into the buffer
- Binlog position is recorded for the commit event
- Buffer content is sent to the final handler
- Binlog position is rewound and all events between the above recorded positions are sent to the final handler
- Author:
- Jiri Pechanec
-
-
Field Summary
Fields Modifier and Type Field Description private Queue<com.github.shyiko.mysql.binlog.event.Event>
buffer
private int
capacity
private ChangeEventSource.ChangeEventSourceContext
changeEventSourceContext
private MySqlStreamingChangeEventSource.BinlogPosition
forwardTillPosition
Contains the position of the last event belonging to the transaction that has not fit into the buffer.private MySqlStreamingChangeEventSource.BinlogPosition
largeTxNotBufferedPosition
Contains the position of the first event that has not fit into the buffer.private static org.slf4j.Logger
LOGGER
private MySqlStreamingChangeEventSource
streamingChangeEventSource
private boolean
txStarted
-
Constructor Summary
Constructors Constructor Description EventBuffer(int capacity, MySqlStreamingChangeEventSource streamingChangeEventSource, ChangeEventSource.ChangeEventSourceContext changeEventSourceContext)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
add(com.github.shyiko.mysql.binlog.event.Event event)
An entry point to the buffer that should be used by BinlogReader to push events.private void
addToBuffer(com.github.shyiko.mysql.binlog.event.Event event)
Adds an event to the buffer if there is a space available.private void
beginTransaction(com.github.shyiko.mysql.binlog.event.Event event)
private void
clear()
Cleans-up the buffer after the transaction is either thrown away or streamed into a Kafka topicprivate void
completeTransaction(boolean wellFormed, com.github.shyiko.mysql.binlog.event.Event event)
Sends all events from the buffer int a final handler.private void
consumeEvent(com.github.shyiko.mysql.binlog.event.Event event)
private boolean
isInBufferFullMode()
private boolean
isReplayingEventsBeyondBufferCapacity()
Whether we are replaying TX events from binlog that have not fit into the buffer beforeprivate void
rollbackTransaction()
private void
switchToBufferFullMode()
-
-
-
Field Detail
-
LOGGER
private static final org.slf4j.Logger LOGGER
-
capacity
private final int capacity
-
buffer
private final Queue<com.github.shyiko.mysql.binlog.event.Event> buffer
-
streamingChangeEventSource
private final MySqlStreamingChangeEventSource streamingChangeEventSource
-
txStarted
private boolean txStarted
-
changeEventSourceContext
private final ChangeEventSource.ChangeEventSourceContext changeEventSourceContext
-
largeTxNotBufferedPosition
private MySqlStreamingChangeEventSource.BinlogPosition largeTxNotBufferedPosition
Contains the position of the first event that has not fit into the buffer.
-
forwardTillPosition
private MySqlStreamingChangeEventSource.BinlogPosition forwardTillPosition
Contains the position of the last event belonging to the transaction that has not fit into the buffer.
-
-
Constructor Detail
-
EventBuffer
public EventBuffer(int capacity, MySqlStreamingChangeEventSource streamingChangeEventSource, ChangeEventSource.ChangeEventSourceContext changeEventSourceContext)
-
-
Method Detail
-
add
public void add(com.github.shyiko.mysql.binlog.event.Event event)
An entry point to the buffer that should be used by BinlogReader to push events.- Parameters:
event
- to be stored in the buffer
-
isReplayingEventsBeyondBufferCapacity
private boolean isReplayingEventsBeyondBufferCapacity()
Whether we are replaying TX events from binlog that have not fit into the buffer before
-
addToBuffer
private void addToBuffer(com.github.shyiko.mysql.binlog.event.Event event)
Adds an event to the buffer if there is a space available. Records binlog position for the first event that does not fit for later replay.- Parameters:
event
-
-
switchToBufferFullMode
private void switchToBufferFullMode()
-
isInBufferFullMode
private boolean isInBufferFullMode()
-
consumeEvent
private void consumeEvent(com.github.shyiko.mysql.binlog.event.Event event)
-
beginTransaction
private void beginTransaction(com.github.shyiko.mysql.binlog.event.Event event)
-
completeTransaction
private void completeTransaction(boolean wellFormed, com.github.shyiko.mysql.binlog.event.Event event)
Sends all events from the buffer int a final handler. For large transactions it executes rewind of binlog reader back to the first event that was not stored in the buffer.- Parameters:
wellFormed
-event
-
-
rollbackTransaction
private void rollbackTransaction()
-
clear
private void clear()
Cleans-up the buffer after the transaction is either thrown away or streamed into a Kafka topic
-
-