Package io.debezium.relational

Class RelationalDatabaseConnectorConfig

java.lang.Object

io.debezium.config.CommonConnectorConfig

io.debezium.relational.RelationalDatabaseConnectorConfig

Direct Known Subclasses:

HistorizedRelationalDatabaseConnectorConfig

public abstract class RelationalDatabaseConnectorConfig
extends CommonConnectorConfig

Configuration options shared across the relational CDC connectors.

Author:

Gunnar Morling

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	RelationalDatabaseConnectorConfig.DecimalHandlingMode	The set of predefined DecimalHandlingMode options or aliases.

Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig

CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.SchemaNameAdjustmentMode, CommonConnectorConfig.Version

Field Summary

Fields

Modifier and Type	Field	Description
static final Field	COLUMN_EXCLUDE_LIST	A comma-separated list of regular expressions that match fully-qualified names of columns to be excluded from monitoring and change messages.
static final Field	COLUMN_INCLUDE_LIST	A comma-separated list of regular expressions that match fully-qualified names of columns to be excluded from monitoring and change messages.
static final String	COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
private final Tables.ColumnNameFilter	columnFilter
protected static final ConfigDefinition	CONFIG_DEFINITION
static final Field	DATABASE_EXCLUDE_LIST	A comma-separated list of regular expressions that match database names to be excluded from monitoring.
protected static final String	DATABASE_EXCLUDE_LIST_NAME
static final Field	DATABASE_INCLUDE_LIST	A comma-separated list of regular expressions that match database names to be monitored.
static final String	DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
protected static final String	DATABASE_INCLUDE_LIST_NAME
static final Field	DATABASE_NAME
static final Field	DECIMAL_HANDLING_MODE
static final long	DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS
static final String	DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER
private final String	heartbeatActionQuery
static final Field	HOSTNAME
static final Field	INCLUDE_SCHEMA_CHANGES
static final Field	INCLUDE_SCHEMA_COMMENTS
private final JdbcConfiguration	jdbcConfig
private final Key.KeyMapper	keyMapper
static final Field	MASK_COLUMN
static final Field	MASK_COLUMN_WITH_HASH
static final Field	MSG_KEY_COLUMNS
static final Field	PASSWORD
static final Field	PORT
static final Field	PROPAGATE_COLUMN_SOURCE_TYPE
static final Field	PROPAGATE_DATATYPE_SOURCE_TYPE
static final Field	SCHEMA_EXCLUDE_LIST	A comma-separated list of regular expressions that match schema names to be excluded from monitoring.
protected static final String	SCHEMA_EXCLUDE_LIST_NAME
static final Field	SCHEMA_INCLUDE_LIST	A comma-separated list of regular expressions that match schema names to be monitored.
static final String	SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
protected static final String	SCHEMA_INCLUDE_LIST_NAME
protected static final Pattern	SERVER_NAME_PATTERN
static final Field	SNAPSHOT_FULL_COLUMN_SCAN_FORCE
static final Field	SNAPSHOT_LOCK_TIMEOUT_MS
static final Field	SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE
static final Field	TABLE_EXCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified names of tables to be excluded from monitoring.
protected static final String	TABLE_EXCLUDE_LIST_NAME
static final Field	TABLE_IGNORE_BUILTIN
static final Field	TABLE_INCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified names of tables to be monitored.
static final String	TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
protected static final String	TABLE_INCLUDE_LIST_NAME
private final RelationalTableFilters	tableFilters
private final Selectors.TableIdToStringMapper	tableIdMapper
private final TemporalPrecisionMode	temporalPrecisionMode
static final Field	TIME_PRECISION_MODE
static final Field	TRUNCATE_COLUMN
static final Field	UNAVAILABLE_VALUE_PLACEHOLDER
static final Field	USER

Fields inherited from class io.debezium.config.CommonConnectorConfig

BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, CUSTOM_RETRIABLE_EXCEPTION, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, DRIVER_CONFIG_PREFIX, EVENT_PROCESSING_FAILURE_HANDLING_MODE, INCREMENTAL_SNAPSHOT_ALLOW_SCHEMA_CHANGES, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, logicalName, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, MULTI_PARTITION_MODE, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SCHEMA_NAME_ADJUSTMENT_MODE, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, TASK_ID, TOMBSTONES_ON_DELETE, TOPIC_NAME_PATTERN, TOPIC_NAMING_STRATEGY, TOPIC_PREFIX

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	RelationalDatabaseConnectorConfig(Configuration config, Tables.TableFilter systemTablesFilter, Selectors.TableIdToStringMapper tableIdMapper, int defaultSnapshotFetchSize, ColumnFilterMode columnFilterMode, boolean useCatalogBeforeSchema)

Method Summary

Modifier and Type	Method	Description
Heartbeat	createHeartbeat(TopicNamingStrategy topicNamingStrategy, SchemaNameAdjuster schemaNameAdjuster, HeartbeatConnectionProvider connectionProvider, HeartbeatErrorHandler errorHandler)
Tables.ColumnNameFilter	getColumnFilter()
JdbcValueConverters.DecimalMode	getDecimalMode()	Returns the Decimal mode Enum for decimal.handling.mode configuration.
String	getHeartbeatActionQuery()
JdbcConfiguration	getJdbcConfig()	Returns a "raw" configuration object exposing all the database driver related settings, without the "database." prefix.
Key.KeyMapper	getKeyMapper()
Map<TableId,String>	getSnapshotSelectOverridesByTable()	Returns any SELECT overrides, if present.
RelationalTableFilters	getTableFilters()
Selectors.TableIdToStringMapper	getTableIdMapper()
TemporalPrecisionMode	getTemporalPrecisionMode()	Returns the temporal precision mode mode Enum for time.precision.mode configuration.
byte[]	getUnavailableValuePlaceholder()
Boolean	isFullColummnScanRequired()
boolean	isSchemaChangesHistoryEnabled()
boolean	isSchemaCommentsHistoryEnabled()
String	schemaExcludeList()
String	schemaIncludeList()
Duration	snapshotLockTimeout()
String	tableExcludeList()
String	tableIncludeList()
private static int	validateColumnExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateDatabaseExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateMessageKeyColumnsField(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateSchemaExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateTableBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateTableExcludeList(Configuration config, Field field, Field.ValidationOutput problems)

Methods inherited from class io.debezium.config.CommonConnectorConfig

binaryHandlingMode, customConverterRegistry, customRetriableException, getConfig, getConnectorName, getContextName, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSnapshotMaxThreads, getSourceInfoStructMaker, getSourceInfoStructMaker, getTaskId, getTopicNamingStrategy, getTopicNamingStrategy, isEmitTombstoneOnDelete, isIncrementalSnapshotSchemaChangesEnabled, isSignalDataCollection, legacyGetDataCollectionsToBeSnapshotted, parseSignallingMessage, schemaNameAdjustmentMode, shouldProvideTransactionMetadata, supportsOperationFiltering, supportsSchemaChangesDuringIncrementalSnapshot, validate, validateAndRecord, validateSkippedOperation, validateTopicName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

SCHEMA_INCLUDE_LIST_NAME

protected static final String SCHEMA_INCLUDE_LIST_NAME

See Also:

• Constant Field Values

SCHEMA_EXCLUDE_LIST_NAME

protected static final String SCHEMA_EXCLUDE_LIST_NAME

See Also:

• Constant Field Values

DATABASE_INCLUDE_LIST_NAME

protected static final String DATABASE_INCLUDE_LIST_NAME

See Also:

• Constant Field Values

DATABASE_EXCLUDE_LIST_NAME

protected static final String DATABASE_EXCLUDE_LIST_NAME

See Also:

• Constant Field Values

TABLE_EXCLUDE_LIST_NAME

protected static final String TABLE_EXCLUDE_LIST_NAME

See Also:

• Constant Field Values

TABLE_INCLUDE_LIST_NAME

protected static final String TABLE_INCLUDE_LIST_NAME

See Also:

• Constant Field Values

SERVER_NAME_PATTERN

protected static final Pattern SERVER_NAME_PATTERN

TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

• Constant Field Values

COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

• Constant Field Values

SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

• Constant Field Values

DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

• Constant Field Values

DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS

public static final long DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS

DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER

public static final String DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER

See Also:

• Constant Field Values

HOSTNAME

public static final Field HOSTNAME

PORT

public static final Field PORT

USER

public static final Field USER

PASSWORD

public static final Field PASSWORD

DATABASE_NAME

public static final Field DATABASE_NAME

TABLE_INCLUDE_LIST

public static final Field TABLE_INCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified names of tables to be monitored. Fully-qualified names for tables are of the form <databaseName>.<tableName> or <databaseName>.<schemaName>.<tableName>. Must not be used with TABLE_EXCLUDE_LIST, and superseded by database inclusions/exclusions.

TABLE_EXCLUDE_LIST

public static final Field TABLE_EXCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified names of tables to be excluded from monitoring. Fully-qualified names for tables are of the form <databaseName>.<tableName> or <databaseName>.<schemaName>.<tableName>. Must not be used with TABLE_INCLUDE_LIST.

TABLE_IGNORE_BUILTIN

public static final Field TABLE_IGNORE_BUILTIN

COLUMN_EXCLUDE_LIST

public static final Field COLUMN_EXCLUDE_LIST

A comma-separated list of regular expressions that match fully-qualified names of columns to be excluded from monitoring and change messages. The exact form of fully qualified names for columns might vary between connector types. For instance, they could be of the form <databaseName>.<tableName>.<columnName> or <schemaName>.<tableName>.<columnName> or <databaseName>.<schemaName>.<tableName>.<columnName>.

COLUMN_INCLUDE_LIST

public static final Field COLUMN_INCLUDE_LIST

A comma-separated list of regular expressions that match fully-qualified names of columns to be excluded from monitoring and change messages. The exact form of fully qualified names for columns might vary between connector types. For instance, they could be of the form <databaseName>.<tableName>.<columnName> or <schemaName>.<tableName>.<columnName> or <databaseName>.<schemaName>.<tableName>.<columnName>.

MSG_KEY_COLUMNS

public static final Field MSG_KEY_COLUMNS

DECIMAL_HANDLING_MODE

public static final Field DECIMAL_HANDLING_MODE

SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE

public static final Field SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE

SCHEMA_INCLUDE_LIST

public static final Field SCHEMA_INCLUDE_LIST

A comma-separated list of regular expressions that match schema names to be monitored. Must not be used with SCHEMA_EXCLUDE_LIST.

SCHEMA_EXCLUDE_LIST

public static final Field SCHEMA_EXCLUDE_LIST

A comma-separated list of regular expressions that match schema names to be excluded from monitoring. Must not be used with SCHEMA_INCLUDE_LIST.

DATABASE_INCLUDE_LIST

public static final Field DATABASE_INCLUDE_LIST

A comma-separated list of regular expressions that match database names to be monitored. Must not be used with DATABASE_EXCLUDE_LIST.

DATABASE_EXCLUDE_LIST

public static final Field DATABASE_EXCLUDE_LIST

A comma-separated list of regular expressions that match database names to be excluded from monitoring. Must not be used with DATABASE_INCLUDE_LIST.

TIME_PRECISION_MODE

public static final Field TIME_PRECISION_MODE

SNAPSHOT_LOCK_TIMEOUT_MS

public static final Field SNAPSHOT_LOCK_TIMEOUT_MS

INCLUDE_SCHEMA_CHANGES

public static final Field INCLUDE_SCHEMA_CHANGES

INCLUDE_SCHEMA_COMMENTS

public static final Field INCLUDE_SCHEMA_COMMENTS

MASK_COLUMN_WITH_HASH

public static final Field MASK_COLUMN_WITH_HASH

MASK_COLUMN

public static final Field MASK_COLUMN

TRUNCATE_COLUMN

public static final Field TRUNCATE_COLUMN

PROPAGATE_COLUMN_SOURCE_TYPE

public static final Field PROPAGATE_COLUMN_SOURCE_TYPE

PROPAGATE_DATATYPE_SOURCE_TYPE

public static final Field PROPAGATE_DATATYPE_SOURCE_TYPE

SNAPSHOT_FULL_COLUMN_SCAN_FORCE

public static final Field SNAPSHOT_FULL_COLUMN_SCAN_FORCE

UNAVAILABLE_VALUE_PLACEHOLDER

public static final Field UNAVAILABLE_VALUE_PLACEHOLDER

CONFIG_DEFINITION

protected static final ConfigDefinition CONFIG_DEFINITION

tableFilters

private final RelationalTableFilters tableFilters

columnFilter

private final Tables.ColumnNameFilter columnFilter

temporalPrecisionMode

private final TemporalPrecisionMode temporalPrecisionMode

keyMapper

private final Key.KeyMapper keyMapper

tableIdMapper

private final Selectors.TableIdToStringMapper tableIdMapper

jdbcConfig

private final JdbcConfiguration jdbcConfig

heartbeatActionQuery

private final String heartbeatActionQuery

Constructor Details

RelationalDatabaseConnectorConfig

protected RelationalDatabaseConnectorConfig(Configuration config,
 Tables.TableFilter systemTablesFilter,
 Selectors.TableIdToStringMapper tableIdMapper,
 int defaultSnapshotFetchSize,
 ColumnFilterMode columnFilterMode,
 boolean useCatalogBeforeSchema)

Method Details

getTableFilters

public RelationalTableFilters getTableFilters()

getDecimalMode

public JdbcValueConverters.DecimalMode getDecimalMode()

Returns the Decimal mode Enum for decimal.handling.mode configuration. This defaults to precise if nothing is provided.

getTemporalPrecisionMode

public TemporalPrecisionMode getTemporalPrecisionMode()

Returns the temporal precision mode mode Enum for time.precision.mode configuration. This defaults to adaptive if nothing is provided.

getKeyMapper

public Key.KeyMapper getKeyMapper()

getJdbcConfig

public JdbcConfiguration getJdbcConfig()

Returns a "raw" configuration object exposing all the database driver related settings, without the "database." prefix. Typically used for passing through driver settings.

getHeartbeatActionQuery

public String getHeartbeatActionQuery()

getUnavailableValuePlaceholder

public byte[] getUnavailableValuePlaceholder()

snapshotLockTimeout

public Duration snapshotLockTimeout()

schemaExcludeList

public String schemaExcludeList()

schemaIncludeList

public String schemaIncludeList()

tableExcludeList

public String tableExcludeList()

tableIncludeList

public String tableIncludeList()

getColumnFilter

public Tables.ColumnNameFilter getColumnFilter()

isFullColummnScanRequired

public Boolean isFullColummnScanRequired()

validateColumnExcludeList

private static int validateColumnExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

isSchemaChangesHistoryEnabled

public boolean isSchemaChangesHistoryEnabled()

Overrides:

isSchemaChangesHistoryEnabled in class CommonConnectorConfig

Returns:

true if the connector should emit messages about schema changes into a public facing topic.

isSchemaCommentsHistoryEnabled

public boolean isSchemaCommentsHistoryEnabled()

Overrides:

isSchemaCommentsHistoryEnabled in class CommonConnectorConfig

Returns:

true if the connector should emit messages which include table and column comments.

getTableIdMapper

public Selectors.TableIdToStringMapper getTableIdMapper()

validateTableBlacklist

private static int validateTableBlacklist(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateTableExcludeList

private static int validateTableExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

getSnapshotSelectOverridesByTable

public Map<TableId,String> getSnapshotSelectOverridesByTable()

Returns any SELECT overrides, if present.

createHeartbeat

public Heartbeat createHeartbeat(TopicNamingStrategy topicNamingStrategy,
 SchemaNameAdjuster schemaNameAdjuster,
 HeartbeatConnectionProvider connectionProvider,
 HeartbeatErrorHandler errorHandler)

Overrides:

createHeartbeat in class CommonConnectorConfig

validateSchemaExcludeList

private static int validateSchemaExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateDatabaseExcludeList

private static int validateDatabaseExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateMessageKeyColumnsField

private static int validateMessageKeyColumnsField(Configuration config,
 Field field,
 Field.ValidationOutput problems)