Package io.debezium.pipeline.txmetadata
Class TransactionMonitor
java.lang.Object
io.debezium.pipeline.txmetadata.TransactionMonitor
The class has externalized its state in
TransactionContext
context class so it can be stored in and recovered from offsets.
The class receives all processed events and keeps the transaction tracking depending on transaction id.
Upon transaction change the metadata events are delivered to a dedicated topic informing about START/END
of the transaction,
including transaction id and in case of END
event the amount of events generated by the transaction.
Every event seen has its source
block enriched to contain
- transaction id
- the total event order in the transaction
- the order of event per table/collection source in the transaction
- Author:
- Jiri Pechanec
-
Field Summary
Modifier and TypeFieldDescriptionprivate final CommonConnectorConfig
static final String
static final String
static final String
static final String
static final String
static final String
static final String
static final String
static final String
private static final org.apache.kafka.connect.data.Schema
private final EventMetadataProvider
private static final org.slf4j.Logger
private final BlockingConsumer<org.apache.kafka.connect.source.SourceRecord>
private final String
static final org.apache.kafka.connect.data.Schema
protected final org.apache.kafka.connect.data.Schema
protected final org.apache.kafka.connect.data.Schema
-
Constructor Summary
ConstructorDescriptionTransactionMonitor
(CommonConnectorConfig connectorConfig, EventMetadataProvider eventMetadataProvider, SchemaNameAdjuster schemaNameAdjuster, BlockingConsumer<org.apache.kafka.connect.source.SourceRecord> sender, String topicName) -
Method Summary
Modifier and TypeMethodDescriptionprivate void
beginTransaction
(Partition partition, OffsetContext offsetContext, Instant timestamp) void
dataEvent
(Partition partition, DataCollectionId source, OffsetContext offset, Object key, org.apache.kafka.connect.data.Struct value) private void
endTransaction
(Partition partition, OffsetContext offsetContext, Instant timestamp) protected org.apache.kafka.connect.data.Struct
prepareTxBeginValue
(OffsetContext offsetContext, Instant timestamp) protected org.apache.kafka.connect.data.Struct
prepareTxEndValue
(OffsetContext offsetContext, Instant timestamp) protected org.apache.kafka.connect.data.Struct
prepareTxKey
(OffsetContext offsetContext) protected org.apache.kafka.connect.data.Struct
prepareTxStruct
(OffsetContext offsetContext, long dataCollectionEventOrder, org.apache.kafka.connect.data.Struct value) void
transactionComittedEvent
(Partition partition, OffsetContext offset, Instant timestamp) private void
transactionEvent
(OffsetContext offsetContext, DataCollectionId source, org.apache.kafka.connect.data.Struct value) void
transactionStartedEvent
(Partition partition, String transactionId, OffsetContext offset, Instant timestamp)
-
Field Details
-
LOGGER
private static final org.slf4j.Logger LOGGER -
DEBEZIUM_TRANSACTION_KEY
- See Also:
-
DEBEZIUM_TRANSACTION_ID_KEY
- See Also:
-
DEBEZIUM_TRANSACTION_TOTAL_ORDER_KEY
- See Also:
-
DEBEZIUM_TRANSACTION_DATA_COLLECTION_ORDER_KEY
- See Also:
-
DEBEZIUM_TRANSACTION_STATUS_KEY
- See Also:
-
DEBEZIUM_TRANSACTION_EVENT_COUNT_KEY
- See Also:
-
DEBEZIUM_TRANSACTION_COLLECTION_KEY
- See Also:
-
DEBEZIUM_TRANSACTION_DATA_COLLECTIONS_KEY
- See Also:
-
DEBEZIUM_TRANSACTION_TS_MS
- See Also:
-
TRANSACTION_BLOCK_SCHEMA
public static final org.apache.kafka.connect.data.Schema TRANSACTION_BLOCK_SCHEMA -
EVENT_COUNT_PER_DATA_COLLECTION_SCHEMA
private static final org.apache.kafka.connect.data.Schema EVENT_COUNT_PER_DATA_COLLECTION_SCHEMA -
transactionKeySchema
protected final org.apache.kafka.connect.data.Schema transactionKeySchema -
transactionValueSchema
protected final org.apache.kafka.connect.data.Schema transactionValueSchema -
eventMetadataProvider
-
topicName
-
sender
-
connectorConfig
-
-
Constructor Details
-
TransactionMonitor
public TransactionMonitor(CommonConnectorConfig connectorConfig, EventMetadataProvider eventMetadataProvider, SchemaNameAdjuster schemaNameAdjuster, BlockingConsumer<org.apache.kafka.connect.source.SourceRecord> sender, String topicName)
-
-
Method Details
-
dataEvent
public void dataEvent(Partition partition, DataCollectionId source, OffsetContext offset, Object key, org.apache.kafka.connect.data.Struct value) throws InterruptedException - Throws:
InterruptedException
-
transactionComittedEvent
public void transactionComittedEvent(Partition partition, OffsetContext offset, Instant timestamp) throws InterruptedException - Throws:
InterruptedException
-
transactionStartedEvent
public void transactionStartedEvent(Partition partition, String transactionId, OffsetContext offset, Instant timestamp) throws InterruptedException - Throws:
InterruptedException
-
prepareTxKey
-
prepareTxBeginValue
protected org.apache.kafka.connect.data.Struct prepareTxBeginValue(OffsetContext offsetContext, Instant timestamp) -
prepareTxEndValue
protected org.apache.kafka.connect.data.Struct prepareTxEndValue(OffsetContext offsetContext, Instant timestamp) -
prepareTxStruct
protected org.apache.kafka.connect.data.Struct prepareTxStruct(OffsetContext offsetContext, long dataCollectionEventOrder, org.apache.kafka.connect.data.Struct value) -
transactionEvent
private void transactionEvent(OffsetContext offsetContext, DataCollectionId source, org.apache.kafka.connect.data.Struct value) -
beginTransaction
private void beginTransaction(Partition partition, OffsetContext offsetContext, Instant timestamp) throws InterruptedException - Throws:
InterruptedException
-
endTransaction
private void endTransaction(Partition partition, OffsetContext offsetContext, Instant timestamp) throws InterruptedException - Throws:
InterruptedException
-