Package io.debezium.relational

Class HistorizedRelationalDatabaseConnectorConfig

java.lang.Object

io.debezium.config.CommonConnectorConfig

io.debezium.relational.RelationalDatabaseConnectorConfig

io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig

public abstract class HistorizedRelationalDatabaseConnectorConfig
extends RelationalDatabaseConnectorConfig

Configuration options shared across the relational CDC connectors which use a persistent database schema history.

Author:

Gunnar Morling

Nested Class Summary

Nested classes/interfaces inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig

RelationalDatabaseConnectorConfig.DecimalHandlingMode

Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig

CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.Version

Field Summary

Fields

Modifier and Type	Field	Description
protected static ConfigDefinition	CONFIG_DEFINITION
private Class<? extends org.apache.kafka.connect.source.SourceConnector>	connectorClass
static Field	DATABASE_HISTORY	The database history class is hidden in the #configDef() since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.
protected static int	DEFAULT_SNAPSHOT_FETCH_SIZE
private String	logicalName
private boolean	useCatalogBeforeSchema

Fields inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig

COLUMN_BLACKLIST, COLUMN_EXCLUDE_LIST, COLUMN_INCLUDE_LIST, COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, COLUMN_WHITELIST, COLUMN_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_BLACKLIST, DATABASE_BLACKLIST_NAME, DATABASE_EXCLUDE_LIST, DATABASE_EXCLUDE_LIST_NAME, DATABASE_INCLUDE_LIST, DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_INCLUDE_LIST_NAME, DATABASE_NAME, DATABASE_WHITELIST, DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_WHITELIST_NAME, DECIMAL_HANDLING_MODE, DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS, DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER, HOSTNAME, INCLUDE_SCHEMA_CHANGES, INCLUDE_SCHEMA_COMMENTS, MASK_COLUMN, MASK_COLUMN_WITH_HASH, MSG_KEY_COLUMNS, PASSWORD, PORT, PROPAGATE_COLUMN_SOURCE_TYPE, PROPAGATE_DATATYPE_SOURCE_TYPE, SCHEMA_BLACKLIST, SCHEMA_EXCLUDE_LIST, SCHEMA_EXCLUDE_LIST_NAME, SCHEMA_INCLUDE_LIST, SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, SCHEMA_INCLUDE_LIST_NAME, SCHEMA_WHITELIST, SCHEMA_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, SERVER_NAME, SERVER_NAME_PATTERN, SNAPSHOT_FULL_COLUMN_SCAN_FORCE, SNAPSHOT_LOCK_TIMEOUT_MS, SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE, TABLE_BLACKLIST, TABLE_BLACKLIST_NAME, TABLE_EXCLUDE_LIST, TABLE_EXCLUDE_LIST_NAME, TABLE_IGNORE_BUILTIN, TABLE_INCLUDE_LIST, TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_INCLUDE_LIST_NAME, TABLE_WHITELIST, TABLE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_WHITELIST_NAME, TIME_PRECISION_MODE, TRUNCATE_COLUMN, UNAVAILABLE_VALUE_PLACEHOLDER, USER

Fields inherited from class io.debezium.config.CommonConnectorConfig

BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, EVENT_PROCESSING_FAILURE_HANDLING_MODE, INCREMENTAL_SNAPSHOT_ALLOW_SCHEMA_CHANGES, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, SOURCE_STRUCT_MAKER_VERSION, TOMBSTONES_ON_DELETE, TRANSACTION_TOPIC

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass, Configuration config, String logicalName, Tables.TableFilter systemTablesFilter, boolean useCatalogBeforeSchema, int defaultSnapshotFetchSize, ColumnFilterMode columnFilterMode)
protected	HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass, Configuration config, String logicalName, Tables.TableFilter systemTablesFilter, boolean useCatalogBeforeSchema, ColumnFilterMode columnFilterMode)
protected	HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass, Configuration config, String logicalName, Tables.TableFilter systemTablesFilter, Selectors.TableIdToStringMapper tableIdMapper, boolean useCatalogBeforeSchema, ColumnFilterMode columnFilterMode)

Method Summary

Modifier and Type	Method	Description
DatabaseHistory	getDatabaseHistory()	Returns a configured (but not yet started) instance of the database history.
protected abstract HistoryRecordComparator	getHistoryRecordComparator()	Returns a comparator to be used when recovering records from the schema history, making sure no history entries newer than the offset we resume from are recovered (which could happen when restarting a connector after history records have been persisted but no new offset has been committed yet).
boolean	useCatalogBeforeSchema()

Methods inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig

getColumnFilter, getDecimalMode, getHeartbeatActionQuery, getJdbcConfig, getKeyMapper, getSnapshotSelectOverridesByTable, getTableFilters, getTableIdMapper, getTemporalPrecisionMode, getUnavailableValuePlaceholder, isFullColummnScanRequired, isSchemaChangesHistoryEnabled, isSchemaCommentsHistoryEnabled, schemaExcludeList, schemaIncludeList, snapshotLockTimeout, tableExcludeList, tableIncludeList

Methods inherited from class io.debezium.config.CommonConnectorConfig

binaryHandlingMode, customConverterRegistry, getConfig, getConnectorName, getContextName, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSnapshotMaxThreads, getSourceInfoStructMaker, getSourceInfoStructMaker, getTransactionTopic, isEmitTombstoneOnDelete, isIncrementalSnapshotSchemaChangesEnabled, legacyGetDataCollectionsToBeSnapshotted, shouldProvideTransactionMetadata, supportsOperationFiltering, supportsSchemaChangesDuringIncrementalSnapshot, validate, validateAndRecord, validateServerNameIsDifferentFromHistoryTopicName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_SNAPSHOT_FETCH_SIZE

protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE

See Also:

Constant Field Values

useCatalogBeforeSchema

private boolean useCatalogBeforeSchema

logicalName

private final String logicalName

connectorClass

private final Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass

DATABASE_HISTORY

public static final Field DATABASE_HISTORY

The database history class is hidden in the #configDef() since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.

CONFIG_DEFINITION

protected static final ConfigDefinition CONFIG_DEFINITION

Constructor Detail

HistorizedRelationalDatabaseConnectorConfig

protected HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass,
                                                      Configuration config,
                                                      String logicalName,
                                                      Tables.TableFilter systemTablesFilter,
                                                      boolean useCatalogBeforeSchema,
                                                      int defaultSnapshotFetchSize,
                                                      ColumnFilterMode columnFilterMode)

HistorizedRelationalDatabaseConnectorConfig

protected HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass,
                                                      Configuration config,
                                                      String logicalName,
                                                      Tables.TableFilter systemTablesFilter,
                                                      boolean useCatalogBeforeSchema,
                                                      ColumnFilterMode columnFilterMode)

HistorizedRelationalDatabaseConnectorConfig

protected HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass,
                                                      Configuration config,
                                                      String logicalName,
                                                      Tables.TableFilter systemTablesFilter,
                                                      Selectors.TableIdToStringMapper tableIdMapper,
                                                      boolean useCatalogBeforeSchema,
                                                      ColumnFilterMode columnFilterMode)

Method Detail

getDatabaseHistory

public DatabaseHistory getDatabaseHistory()

Returns a configured (but not yet started) instance of the database history.

useCatalogBeforeSchema

public boolean useCatalogBeforeSchema()

getHistoryRecordComparator

protected abstract HistoryRecordComparator getHistoryRecordComparator()

Returns a comparator to be used when recovering records from the schema history, making sure no history entries newer than the offset we resume from are recovered (which could happen when restarting a connector after history records have been persisted but no new offset has been committed yet).