Package io.debezium.relational

Class HistorizedRelationalDatabaseConnectorConfig

java.lang.Object

io.debezium.config.CommonConnectorConfig

io.debezium.relational.RelationalDatabaseConnectorConfig

io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig

public abstract class HistorizedRelationalDatabaseConnectorConfig
extends RelationalDatabaseConnectorConfig

Configuration options shared across the relational CDC connectors which use a persistent database schema history.

Author:

Gunnar Morling

Nested Class Summary

Nested classes/interfaces inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig

RelationalDatabaseConnectorConfig.DecimalHandlingMode

Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig

CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.SchemaNameAdjustmentMode, CommonConnectorConfig.Version

Field Summary

Fields

Modifier and Type	Field	Description
protected static final ConfigDefinition	CONFIG_DEFINITION
private final Class<? extends org.apache.kafka.connect.source.SourceConnector>	connectorClass
private static final String	DEFAULT_SCHEMA_HISTORY
protected static final int	DEFAULT_SNAPSHOT_FETCH_SIZE
private final boolean	multiPartitionMode
static final Field	SCHEMA_HISTORY	The database schema history class is hidden in the #configDef() since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.
private boolean	useCatalogBeforeSchema

Fields inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig

COLUMN_EXCLUDE_LIST, COLUMN_INCLUDE_LIST, COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_EXCLUDE_LIST, DATABASE_EXCLUDE_LIST_NAME, DATABASE_INCLUDE_LIST, DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_INCLUDE_LIST_NAME, DATABASE_NAME, DECIMAL_HANDLING_MODE, DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS, DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER, HOSTNAME, INCLUDE_SCHEMA_CHANGES, INCLUDE_SCHEMA_COMMENTS, MASK_COLUMN, MASK_COLUMN_WITH_HASH, MSG_KEY_COLUMNS, PASSWORD, PORT, PROPAGATE_COLUMN_SOURCE_TYPE, PROPAGATE_DATATYPE_SOURCE_TYPE, SCHEMA_EXCLUDE_LIST, SCHEMA_EXCLUDE_LIST_NAME, SCHEMA_INCLUDE_LIST, SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, SCHEMA_INCLUDE_LIST_NAME, SERVER_NAME_PATTERN, SNAPSHOT_FULL_COLUMN_SCAN_FORCE, SNAPSHOT_LOCK_TIMEOUT_MS, SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE, TABLE_EXCLUDE_LIST, TABLE_EXCLUDE_LIST_NAME, TABLE_IGNORE_BUILTIN, TABLE_INCLUDE_LIST, TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_INCLUDE_LIST_NAME, TIME_PRECISION_MODE, TRUNCATE_COLUMN, UNAVAILABLE_VALUE_PLACEHOLDER, USER

Fields inherited from class io.debezium.config.CommonConnectorConfig

BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, CUSTOM_RETRIABLE_EXCEPTION, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, DRIVER_CONFIG_PREFIX, EVENT_PROCESSING_FAILURE_HANDLING_MODE, INCREMENTAL_SNAPSHOT_ALLOW_SCHEMA_CHANGES, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, logicalName, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, MULTI_PARTITION_MODE, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SCHEMA_NAME_ADJUSTMENT_MODE, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, TASK_ID, TOMBSTONES_ON_DELETE, TOPIC_NAME_PATTERN, TOPIC_NAMING_STRATEGY, TOPIC_PREFIX

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass, Configuration config, Tables.TableFilter systemTablesFilter, boolean useCatalogBeforeSchema, int defaultSnapshotFetchSize, ColumnFilterMode columnFilterMode, boolean multiPartitionMode)
protected	HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass, Configuration config, Tables.TableFilter systemTablesFilter, Selectors.TableIdToStringMapper tableIdMapper, boolean useCatalogBeforeSchema, ColumnFilterMode columnFilterMode, boolean multiPartitionMode)

Method Summary

Modifier and Type	Method	Description
protected abstract HistoryRecordComparator	getHistoryRecordComparator()	Returns a comparator to be used when recovering records from the schema history, making sure no history entries newer than the offset we resume from are recovered (which could happen when restarting a connector after history records have been persisted but no new offset has been committed yet).
SchemaHistory	getSchemaHistory()	Returns a configured (but not yet started) instance of the database schema history.
boolean	multiPartitionMode()
boolean	useCatalogBeforeSchema()

Methods inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig

createHeartbeat, getColumnFilter, getDecimalMode, getHeartbeatActionQuery, getJdbcConfig, getKeyMapper, getSnapshotSelectOverridesByTable, getTableFilters, getTableIdMapper, getTemporalPrecisionMode, getUnavailableValuePlaceholder, isFullColummnScanRequired, isSchemaChangesHistoryEnabled, isSchemaCommentsHistoryEnabled, schemaExcludeList, schemaIncludeList, snapshotLockTimeout, tableExcludeList, tableIncludeList

Methods inherited from class io.debezium.config.CommonConnectorConfig

binaryHandlingMode, customConverterRegistry, customRetriableException, getConfig, getConnectorName, getContextName, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSnapshotMaxThreads, getSourceInfoStructMaker, getSourceInfoStructMaker, getTaskId, getTopicNamingStrategy, getTopicNamingStrategy, isEmitTombstoneOnDelete, isIncrementalSnapshotSchemaChangesEnabled, isSignalDataCollection, legacyGetDataCollectionsToBeSnapshotted, parseSignallingMessage, schemaNameAdjustmentMode, shouldProvideTransactionMetadata, supportsOperationFiltering, supportsSchemaChangesDuringIncrementalSnapshot, validate, validateAndRecord, validateSkippedOperation, validateTopicName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DEFAULT_SNAPSHOT_FETCH_SIZE

protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE

See Also:

• Constant Field Values

DEFAULT_SCHEMA_HISTORY

private static final String DEFAULT_SCHEMA_HISTORY

See Also:

• Constant Field Values

useCatalogBeforeSchema

private boolean useCatalogBeforeSchema

connectorClass

private final Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass

multiPartitionMode

private final boolean multiPartitionMode

SCHEMA_HISTORY

public static final Field SCHEMA_HISTORY

The database schema history class is hidden in the #configDef() since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.

CONFIG_DEFINITION

protected static final ConfigDefinition CONFIG_DEFINITION

Constructor Details

HistorizedRelationalDatabaseConnectorConfig

protected HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass,
 Configuration config,
 Tables.TableFilter systemTablesFilter,
 boolean useCatalogBeforeSchema,
 int defaultSnapshotFetchSize,
 ColumnFilterMode columnFilterMode,
 boolean multiPartitionMode)

HistorizedRelationalDatabaseConnectorConfig

protected HistorizedRelationalDatabaseConnectorConfig(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass,
 Configuration config,
 Tables.TableFilter systemTablesFilter,
 Selectors.TableIdToStringMapper tableIdMapper,
 boolean useCatalogBeforeSchema,
 ColumnFilterMode columnFilterMode,
 boolean multiPartitionMode)

Method Details

getSchemaHistory

public SchemaHistory getSchemaHistory()

Returns a configured (but not yet started) instance of the database schema history.

useCatalogBeforeSchema

public boolean useCatalogBeforeSchema()

multiPartitionMode

public boolean multiPartitionMode()

getHistoryRecordComparator

protected abstract HistoryRecordComparator getHistoryRecordComparator()

Returns a comparator to be used when recovering records from the schema history, making sure no history entries newer than the offset we resume from are recovered (which could happen when restarting a connector after history records have been persisted but no new offset has been committed yet).