Package io.debezium.pipeline
Class EventDispatcher<P extends Partition,T extends DataCollectionId>
java.lang.Object
io.debezium.pipeline.EventDispatcher<P,T>
- All Implemented Interfaces:
AutoCloseable
public class EventDispatcher<P extends Partition,T extends DataCollectionId>
extends Object
implements AutoCloseable
Central dispatcher for data change and schema change events. The former will be routed to the change event queue, the
latter will be routed to the
DatabaseSchema
. But based on the applying include/exclude list configuration,
events may be not be dispatched at all.
This router is also in charge of emitting heartbeat messages, exposing of metrics via JMX etc.
- Author:
- Gunnar Morling
-
Nested Class Summary
Modifier and TypeClassDescriptionprivate static final class
private final class
static interface
EventDispatcher.InconsistentSchemaHandler<P extends Partition,
T extends DataCollectionId> Reaction to an incoming change event for which schema is not foundprivate final class
private final class
static interface
EventDispatcher.SnapshotReceiver<P extends Partition>
Change record receiver used during snapshotting.private final class
-
Field Summary
Modifier and TypeFieldDescriptionprivate final ChangeEventCreator
private final CommonConnectorConfig
private final boolean
private DataChangeEventListener<P>
private final DataCollectionFilters.DataCollectionFilter<T>
private final Heartbeat
private final HistorizedDatabaseSchema<T>
private final EventDispatcher.InconsistentSchemaHandler<P,
T> private IncrementalSnapshotChangeEventSource<P,
T> private static final org.slf4j.Logger
private final boolean
private final ChangeEventQueue<DataChangeEvent>
private final DatabaseSchema<T>
private final org.apache.kafka.connect.data.Schema
private final org.apache.kafka.connect.data.Schema
private final SignalProcessor<P,
?> private final EnumSet<Envelope.Operation>
private final SourceSignalChannel
private final EventDispatcher<P,
T>.StreamingChangeRecordReceiver Change event receiver for events dispatched from a streaming change event source.private final ConnectTableChangeSerializer
private final TopicNamingStrategy<T>
private final TransactionMonitor
-
Constructor Summary
ConstructorDescriptionEventDispatcher
(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventDispatcher.InconsistentSchemaHandler<P, T> inconsistentSchemaHandler, Heartbeat heartbeat, SchemaNameAdjuster schemaNameAdjuster, TransactionMonitor transactionMonitor, SignalProcessor<P, ?> signalProcessor) EventDispatcher
(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventDispatcher.InconsistentSchemaHandler<P, T> inconsistentSchemaHandler, EventMetadataProvider metadataProvider, Heartbeat heartbeat, SchemaNameAdjuster schemaNameAdjuster, SignalProcessor<P, ?> signalProcessor) EventDispatcher
(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventMetadataProvider metadataProvider, Heartbeat heartbeat, SchemaNameAdjuster schemaNameAdjuster) EventDispatcher
(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventMetadataProvider metadataProvider, Heartbeat heartbeat, SchemaNameAdjuster schemaNameAdjuster, SignalProcessor<P, ?> signalProcessor) EventDispatcher
(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventMetadataProvider metadataProvider, SchemaNameAdjuster schemaNameAdjuster) EventDispatcher
(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventMetadataProvider metadataProvider, SchemaNameAdjuster schemaNameAdjuster, SignalProcessor<P, ?> signalProcessor) -
Method Summary
Modifier and TypeMethodDescriptionvoid
alwaysDispatchHeartbeatEvent
(P partition, OffsetContext offset) void
close()
void
dispatchConnectorEvent
(P partition, ConnectorEvent event) boolean
dispatchDataChangeEvent
(P partition, T dataCollectionId, ChangeRecordEmitter<P> changeRecordEmitter) Dispatches one or moreDataChangeEvent
s.void
dispatchFilteredEvent
(P partition, OffsetContext offset) void
dispatchHeartbeatEvent
(P partition, OffsetContext offset) void
dispatchSchemaChangeEvent
(Collection<T> dataCollectionIds, SchemaChangeEventEmitter schemaChangeEventEmitter) void
dispatchSchemaChangeEvent
(P partition, OffsetContext offsetContext, T dataCollectionId, SchemaChangeEventEmitter schemaChangeEventEmitter) void
dispatchServerHeartbeatEvent
(P partition, OffsetContext offset) void
dispatchSnapshotEvent
(P partition, T dataCollectionId, ChangeRecordEmitter<P> changeRecordEmitter, EventDispatcher.SnapshotReceiver<P> receiver) void
dispatchTransactionCommittedEvent
(P partition, OffsetContext offset, Instant timestamp) void
dispatchTransactionStartedEvent
(P partition, String transactionId, OffsetContext offset, Instant timestamp) private void
enqueueHeartbeat
(org.apache.kafka.connect.source.SourceRecord record) void
enqueueNotification
(org.apache.kafka.connect.source.SourceRecord record) private void
enqueueSchemaChangeMessage
(org.apache.kafka.connect.source.SourceRecord record) private void
enqueueTransactionMessage
(org.apache.kafka.connect.source.SourceRecord record) errorOnMissingSchema
(P partition, T dataCollectionId, ChangeRecordEmitter<P> changeRecordEmitter) getIncrementalSnapshotChangeEventReceiver
(DataChangeEventListener<P> dataListener) boolean
ignoreMissingSchema
(T dataCollectionId, ChangeRecordEmitter<P> changeRecordEmitter) void
setEventListener
(DataChangeEventListener<P> eventListener) Provide a listener that is invoked for every incoming event to be processed.void
setIncrementalSnapshotChangeEventSource
(Optional<IncrementalSnapshotChangeEventSource<P, ? extends DataCollectionId>> incrementalSnapshotChangeEventSource) Enable support for incremental snapshotting.
-
Field Details
-
LOGGER
private static final org.slf4j.Logger LOGGER -
topicNamingStrategy
-
schema
-
historizedSchema
-
queue
-
filter
-
changeEventCreator
-
heartbeat
-
eventListener
-
emitTombstonesOnDelete
private final boolean emitTombstonesOnDelete -
inconsistentSchemaHandler
private final EventDispatcher.InconsistentSchemaHandler<P extends Partition,T extends DataCollectionId> inconsistentSchemaHandler -
transactionMonitor
-
connectorConfig
-
skippedOperations
-
neverSkip
private final boolean neverSkip -
schemaChangeKeySchema
private final org.apache.kafka.connect.data.Schema schemaChangeKeySchema -
schemaChangeValueSchema
private final org.apache.kafka.connect.data.Schema schemaChangeValueSchema -
tableChangesSerializer
-
sourceSignalChannel
-
incrementalSnapshotChangeEventSource
private IncrementalSnapshotChangeEventSource<P extends Partition,T extends DataCollectionId> incrementalSnapshotChangeEventSource -
streamingReceiver
private final EventDispatcher<P extends Partition,T extends DataCollectionId>.StreamingChangeRecordReceiver streamingReceiverChange event receiver for events dispatched from a streaming change event source. -
signalProcessor
-
-
Constructor Details
-
EventDispatcher
public EventDispatcher(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventMetadataProvider metadataProvider, SchemaNameAdjuster schemaNameAdjuster, SignalProcessor<P, ?> signalProcessor) -
EventDispatcher
public EventDispatcher(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventMetadataProvider metadataProvider, Heartbeat heartbeat, SchemaNameAdjuster schemaNameAdjuster, SignalProcessor<P, ?> signalProcessor) -
EventDispatcher
public EventDispatcher(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventMetadataProvider metadataProvider, Heartbeat heartbeat, SchemaNameAdjuster schemaNameAdjuster) -
EventDispatcher
public EventDispatcher(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventMetadataProvider metadataProvider, SchemaNameAdjuster schemaNameAdjuster) -
EventDispatcher
public EventDispatcher(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventDispatcher.InconsistentSchemaHandler<P, T> inconsistentSchemaHandler, EventMetadataProvider metadataProvider, Heartbeat heartbeat, SchemaNameAdjuster schemaNameAdjuster, SignalProcessor<P, ?> signalProcessor) -
EventDispatcher
public EventDispatcher(CommonConnectorConfig connectorConfig, TopicNamingStrategy<T> topicNamingStrategy, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventDispatcher.InconsistentSchemaHandler<P, T> inconsistentSchemaHandler, Heartbeat heartbeat, SchemaNameAdjuster schemaNameAdjuster, TransactionMonitor transactionMonitor, SignalProcessor<P, ?> signalProcessor)
-
-
Method Details
-
dispatchSnapshotEvent
public void dispatchSnapshotEvent(P partition, T dataCollectionId, ChangeRecordEmitter<P> changeRecordEmitter, EventDispatcher.SnapshotReceiver<P> receiver) throws InterruptedException - Throws:
InterruptedException
-
getSnapshotChangeEventReceiver
-
getIncrementalSnapshotChangeEventReceiver
public EventDispatcher.SnapshotReceiver<P> getIncrementalSnapshotChangeEventReceiver(DataChangeEventListener<P> dataListener) -
dispatchDataChangeEvent
public boolean dispatchDataChangeEvent(P partition, T dataCollectionId, ChangeRecordEmitter<P> changeRecordEmitter) throws InterruptedException Dispatches one or moreDataChangeEvent
s. If the given data collection is included in the currently captured set of collections, the given emitter will be invoked, so it can emit one or more events (in the common case, one event will be emitted, but e.g. in case of PK updates, it may be a deletion and a creation event). The receiving coordinator createsSourceRecord
s for all emitted events and passes them to this dispatcher'sChangeEventCreator
for converting them into data change events.- Returns:
true
if an event was dispatched (i.e. sent to the message broker),false
otherwise.- Throws:
InterruptedException
-
dispatchFilteredEvent
- Throws:
InterruptedException
-
dispatchTransactionCommittedEvent
public void dispatchTransactionCommittedEvent(P partition, OffsetContext offset, Instant timestamp) throws InterruptedException - Throws:
InterruptedException
-
dispatchTransactionStartedEvent
public void dispatchTransactionStartedEvent(P partition, String transactionId, OffsetContext offset, Instant timestamp) throws InterruptedException - Throws:
InterruptedException
-
dispatchConnectorEvent
-
errorOnMissingSchema
public Optional<DataCollectionSchema> errorOnMissingSchema(P partition, T dataCollectionId, ChangeRecordEmitter<P> changeRecordEmitter) -
ignoreMissingSchema
public Optional<DataCollectionSchema> ignoreMissingSchema(T dataCollectionId, ChangeRecordEmitter<P> changeRecordEmitter) -
dispatchSchemaChangeEvent
public void dispatchSchemaChangeEvent(P partition, OffsetContext offsetContext, T dataCollectionId, SchemaChangeEventEmitter schemaChangeEventEmitter) throws InterruptedException - Throws:
InterruptedException
-
dispatchSchemaChangeEvent
public void dispatchSchemaChangeEvent(Collection<T> dataCollectionIds, SchemaChangeEventEmitter schemaChangeEventEmitter) throws InterruptedException - Throws:
InterruptedException
-
alwaysDispatchHeartbeatEvent
public void alwaysDispatchHeartbeatEvent(P partition, OffsetContext offset) throws InterruptedException - Throws:
InterruptedException
-
dispatchHeartbeatEvent
- Throws:
InterruptedException
-
heartbeatsEnabled
public boolean heartbeatsEnabled() -
enqueueHeartbeat
private void enqueueHeartbeat(org.apache.kafka.connect.source.SourceRecord record) throws InterruptedException - Throws:
InterruptedException
-
enqueueTransactionMessage
private void enqueueTransactionMessage(org.apache.kafka.connect.source.SourceRecord record) throws InterruptedException - Throws:
InterruptedException
-
enqueueSchemaChangeMessage
private void enqueueSchemaChangeMessage(org.apache.kafka.connect.source.SourceRecord record) throws InterruptedException - Throws:
InterruptedException
-
dispatchServerHeartbeatEvent
public void dispatchServerHeartbeatEvent(P partition, OffsetContext offset) throws InterruptedException - Throws:
InterruptedException
-
enqueueNotification
public void enqueueNotification(org.apache.kafka.connect.source.SourceRecord record) throws InterruptedException - Throws:
InterruptedException
-
setEventListener
Provide a listener that is invoked for every incoming event to be processed.- Parameters:
eventListener
-
-
setIncrementalSnapshotChangeEventSource
public void setIncrementalSnapshotChangeEventSource(Optional<IncrementalSnapshotChangeEventSource<P, ? extends DataCollectionId>> incrementalSnapshotChangeEventSource) Enable support for incremental snapshotting. -
getSchema
-
getHistorizedSchema
-
getIncrementalSnapshotChangeEventSource
-
close
public void close()- Specified by:
close
in interfaceAutoCloseable
-