Package io.debezium.relational

Class RelationalDatabaseConnectorConfig

java.lang.Object

io.debezium.config.CommonConnectorConfig

io.debezium.relational.RelationalDatabaseConnectorConfig

Direct Known Subclasses:

HistorizedRelationalDatabaseConnectorConfig

public abstract class RelationalDatabaseConnectorConfig
extends CommonConnectorConfig

Configuration options shared across the relational CDC connectors.

Author:

Gunnar Morling

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	RelationalDatabaseConnectorConfig.DecimalHandlingMode	The set of predefined DecimalHandlingMode options or aliases.

Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig

CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.Version

Field Summary

Fields

Modifier and Type	Field	Description
static Field	COLUMN_BLACKLIST	Deprecated.
static Field	COLUMN_EXCLUDE_LIST	A comma-separated list of regular expressions that match fully-qualified names of columns to be excluded from monitoring and change messages.
static Field	COLUMN_INCLUDE_LIST	A comma-separated list of regular expressions that match fully-qualified names of columns to be excluded from monitoring and change messages.
static String	COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
static Field	COLUMN_WHITELIST	Deprecated.
static String	COLUMN_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG
private Tables.ColumnNameFilter	columnFilter
protected static ConfigDefinition	CONFIG_DEFINITION
static Field	DATABASE_BLACKLIST	Deprecated.
protected static String	DATABASE_BLACKLIST_NAME
static Field	DATABASE_EXCLUDE_LIST	A comma-separated list of regular expressions that match database names to be excluded from monitoring.
protected static String	DATABASE_EXCLUDE_LIST_NAME
static Field	DATABASE_INCLUDE_LIST	A comma-separated list of regular expressions that match database names to be monitored.
static String	DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
protected static String	DATABASE_INCLUDE_LIST_NAME
static Field	DATABASE_NAME
static Field	DATABASE_WHITELIST	Deprecated.
static String	DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG
protected static String	DATABASE_WHITELIST_NAME
static Field	DECIMAL_HANDLING_MODE
static long	DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS
static String	DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER
private String	heartbeatActionQuery
static Field	HOSTNAME
static Field	INCLUDE_SCHEMA_CHANGES
static Field	INCLUDE_SCHEMA_COMMENTS
private Configuration	jdbcConfig
private Key.KeyMapper	keyMapper
static Field	MASK_COLUMN
static Field	MASK_COLUMN_WITH_HASH
static Field	MSG_KEY_COLUMNS
static Field	PASSWORD
static Field	PORT
static Field	PROPAGATE_COLUMN_SOURCE_TYPE
static Field	PROPAGATE_DATATYPE_SOURCE_TYPE
static Field	SCHEMA_BLACKLIST	Deprecated.
static Field	SCHEMA_EXCLUDE_LIST	A comma-separated list of regular expressions that match schema names to be excluded from monitoring.
protected static String	SCHEMA_EXCLUDE_LIST_NAME
static Field	SCHEMA_INCLUDE_LIST	A comma-separated list of regular expressions that match schema names to be monitored.
static String	SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
protected static String	SCHEMA_INCLUDE_LIST_NAME
static Field	SCHEMA_WHITELIST	Deprecated.
static String	SCHEMA_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG
static Field	SERVER_NAME
protected static Pattern	SERVER_NAME_PATTERN
static Field	SNAPSHOT_FULL_COLUMN_SCAN_FORCE
static Field	SNAPSHOT_LOCK_TIMEOUT_MS
static Field	SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE
static Field	TABLE_BLACKLIST	Deprecated.
protected static String	TABLE_BLACKLIST_NAME
static Field	TABLE_EXCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified names of tables to be excluded from monitoring.
protected static String	TABLE_EXCLUDE_LIST_NAME
static Field	TABLE_IGNORE_BUILTIN
static Field	TABLE_INCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified names of tables to be monitored.
static String	TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
protected static String	TABLE_INCLUDE_LIST_NAME
static Field	TABLE_WHITELIST	Deprecated.
static String	TABLE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG
protected static String	TABLE_WHITELIST_NAME
private RelationalTableFilters	tableFilters
private Selectors.TableIdToStringMapper	tableIdMapper
private TemporalPrecisionMode	temporalPrecisionMode
static Field	TIME_PRECISION_MODE
static Field	TRUNCATE_COLUMN
static Field	UNAVAILABLE_VALUE_PLACEHOLDER
static Field	USER

Fields inherited from class io.debezium.config.CommonConnectorConfig

BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, EVENT_PROCESSING_FAILURE_HANDLING_MODE, INCREMENTAL_SNAPSHOT_ALLOW_SCHEMA_CHANGES, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, SOURCE_STRUCT_MAKER_VERSION, TOMBSTONES_ON_DELETE, TRANSACTION_TOPIC

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	RelationalDatabaseConnectorConfig(Configuration config, String logicalName, Tables.TableFilter systemTablesFilter, Selectors.TableIdToStringMapper tableIdMapper, int defaultSnapshotFetchSize, ColumnFilterMode columnFilterMode)

Method Summary

Modifier and Type	Method	Description
Tables.ColumnNameFilter	getColumnFilter()
JdbcValueConverters.DecimalMode	getDecimalMode()	Returns the Decimal mode Enum for decimal.handling.mode configuration.
String	getHeartbeatActionQuery()
Configuration	getJdbcConfig()	Returns a "raw" configuration object exposing all the database driver related settings, without the "database." prefix.
Key.KeyMapper	getKeyMapper()
Map<TableId,String>	getSnapshotSelectOverridesByTable()	Returns any SELECT overrides, if present.
RelationalTableFilters	getTableFilters()
Selectors.TableIdToStringMapper	getTableIdMapper()
TemporalPrecisionMode	getTemporalPrecisionMode()	Returns the temporal precision mode mode Enum for time.precision.mode configuration.
byte[]	getUnavailableValuePlaceholder()
Boolean	isFullColummnScanRequired()
boolean	isSchemaChangesHistoryEnabled()
boolean	isSchemaCommentsHistoryEnabled()
String	schemaExcludeList()
String	schemaIncludeList()
Duration	snapshotLockTimeout()
String	tableExcludeList()
String	tableIncludeList()
private static int	validateColumnBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateColumnExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateDatabaseBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateDatabaseExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateMessageKeyColumnsField(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateSchemaBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateSchemaExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateServerName(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateTableBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateTableExcludeList(Configuration config, Field field, Field.ValidationOutput problems)

Methods inherited from class io.debezium.config.CommonConnectorConfig

binaryHandlingMode, customConverterRegistry, getConfig, getConnectorName, getContextName, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSnapshotMaxThreads, getSourceInfoStructMaker, getSourceInfoStructMaker, getTransactionTopic, isEmitTombstoneOnDelete, isIncrementalSnapshotSchemaChangesEnabled, legacyGetDataCollectionsToBeSnapshotted, shouldProvideTransactionMetadata, supportsOperationFiltering, supportsSchemaChangesDuringIncrementalSnapshot, validate, validateAndRecord, validateServerNameIsDifferentFromHistoryTopicName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SCHEMA_INCLUDE_LIST_NAME

protected static final String SCHEMA_INCLUDE_LIST_NAME

See Also:

Constant Field Values

SCHEMA_EXCLUDE_LIST_NAME

protected static final String SCHEMA_EXCLUDE_LIST_NAME

See Also:

Constant Field Values

DATABASE_WHITELIST_NAME

protected static final String DATABASE_WHITELIST_NAME

See Also:

Constant Field Values

DATABASE_INCLUDE_LIST_NAME

protected static final String DATABASE_INCLUDE_LIST_NAME

See Also:

Constant Field Values

DATABASE_BLACKLIST_NAME

protected static final String DATABASE_BLACKLIST_NAME

See Also:

Constant Field Values

DATABASE_EXCLUDE_LIST_NAME

protected static final String DATABASE_EXCLUDE_LIST_NAME

See Also:

Constant Field Values

TABLE_BLACKLIST_NAME

protected static final String TABLE_BLACKLIST_NAME

See Also:

Constant Field Values

TABLE_EXCLUDE_LIST_NAME

protected static final String TABLE_EXCLUDE_LIST_NAME

See Also:

Constant Field Values

TABLE_WHITELIST_NAME

protected static final String TABLE_WHITELIST_NAME

See Also:

Constant Field Values

TABLE_INCLUDE_LIST_NAME

protected static final String TABLE_INCLUDE_LIST_NAME

See Also:

Constant Field Values

SERVER_NAME_PATTERN

protected static final Pattern SERVER_NAME_PATTERN

TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

TABLE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String TABLE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

COLUMN_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String COLUMN_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

SCHEMA_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String SCHEMA_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

public static final String DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS

public static final long DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS

DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER

public static final String DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER

See Also:

Constant Field Values

HOSTNAME

public static final Field HOSTNAME

PORT

public static final Field PORT

USER

public static final Field USER

PASSWORD

public static final Field PASSWORD

DATABASE_NAME

public static final Field DATABASE_NAME

SERVER_NAME

public static final Field SERVER_NAME

TABLE_INCLUDE_LIST

public static final Field TABLE_INCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified names of tables to be monitored. Fully-qualified names for tables are of the form <databaseName>.<tableName> or <databaseName>.<schemaName>.<tableName>. Must not be used with TABLE_EXCLUDE_LIST, and superseded by database inclusions/exclusions.

TABLE_WHITELIST

@Deprecated
public static final Field TABLE_WHITELIST

Deprecated.

Old, backwards-compatible "whitelist" property.

TABLE_EXCLUDE_LIST

public static final Field TABLE_EXCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified names of tables to be excluded from monitoring. Fully-qualified names for tables are of the form <databaseName>.<tableName> or <databaseName>.<schemaName>.<tableName>. Must not be used with TABLE_INCLUDE_LIST.

TABLE_BLACKLIST

@Deprecated
public static final Field TABLE_BLACKLIST

Deprecated.

Old, backwards-compatible "blacklist" property.

TABLE_IGNORE_BUILTIN

public static final Field TABLE_IGNORE_BUILTIN

COLUMN_EXCLUDE_LIST

public static final Field COLUMN_EXCLUDE_LIST

A comma-separated list of regular expressions that match fully-qualified names of columns to be excluded from monitoring and change messages. The exact form of fully qualified names for columns might vary between connector types. For instance, they could be of the form <databaseName>.<tableName>.<columnName> or <schemaName>.<tableName>.<columnName> or <databaseName>.<schemaName>.<tableName>.<columnName>.

COLUMN_BLACKLIST

@Deprecated
public static final Field COLUMN_BLACKLIST

Deprecated.

Old, backwards-compatible "blacklist" property.

COLUMN_INCLUDE_LIST

public static final Field COLUMN_INCLUDE_LIST

A comma-separated list of regular expressions that match fully-qualified names of columns to be excluded from monitoring and change messages. The exact form of fully qualified names for columns might vary between connector types. For instance, they could be of the form <databaseName>.<tableName>.<columnName> or <schemaName>.<tableName>.<columnName> or <databaseName>.<schemaName>.<tableName>.<columnName>.

COLUMN_WHITELIST

@Deprecated
public static final Field COLUMN_WHITELIST

Deprecated.

Old, backwards-compatible "whitelist" property.

MSG_KEY_COLUMNS

public static final Field MSG_KEY_COLUMNS

DECIMAL_HANDLING_MODE

public static final Field DECIMAL_HANDLING_MODE

SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE

public static final Field SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE

SCHEMA_INCLUDE_LIST

public static final Field SCHEMA_INCLUDE_LIST

A comma-separated list of regular expressions that match schema names to be monitored. Must not be used with SCHEMA_EXCLUDE_LIST.

SCHEMA_WHITELIST

@Deprecated
public static final Field SCHEMA_WHITELIST

Deprecated.

Old, backwards-compatible "whitelist" property.

SCHEMA_EXCLUDE_LIST

public static final Field SCHEMA_EXCLUDE_LIST

A comma-separated list of regular expressions that match schema names to be excluded from monitoring. Must not be used with SCHEMA_INCLUDE_LIST.

SCHEMA_BLACKLIST

@Deprecated
public static final Field SCHEMA_BLACKLIST

Deprecated.

Old, backwards-compatible "blacklist" property.

DATABASE_INCLUDE_LIST

public static final Field DATABASE_INCLUDE_LIST

A comma-separated list of regular expressions that match database names to be monitored. Must not be used with DATABASE_BLACKLIST.

DATABASE_WHITELIST

@Deprecated
public static final Field DATABASE_WHITELIST

Deprecated.

DATABASE_EXCLUDE_LIST

public static final Field DATABASE_EXCLUDE_LIST

A comma-separated list of regular expressions that match database names to be excluded from monitoring. Must not be used with DATABASE_INCLUDE_LIST.

DATABASE_BLACKLIST

@Deprecated
public static final Field DATABASE_BLACKLIST

Deprecated.

TIME_PRECISION_MODE

public static final Field TIME_PRECISION_MODE

SNAPSHOT_LOCK_TIMEOUT_MS

public static final Field SNAPSHOT_LOCK_TIMEOUT_MS

INCLUDE_SCHEMA_CHANGES

public static final Field INCLUDE_SCHEMA_CHANGES

INCLUDE_SCHEMA_COMMENTS

public static final Field INCLUDE_SCHEMA_COMMENTS

MASK_COLUMN_WITH_HASH

public static final Field MASK_COLUMN_WITH_HASH

MASK_COLUMN

public static final Field MASK_COLUMN

TRUNCATE_COLUMN

public static final Field TRUNCATE_COLUMN

PROPAGATE_COLUMN_SOURCE_TYPE

public static final Field PROPAGATE_COLUMN_SOURCE_TYPE

PROPAGATE_DATATYPE_SOURCE_TYPE

public static final Field PROPAGATE_DATATYPE_SOURCE_TYPE

SNAPSHOT_FULL_COLUMN_SCAN_FORCE

public static final Field SNAPSHOT_FULL_COLUMN_SCAN_FORCE

UNAVAILABLE_VALUE_PLACEHOLDER

public static final Field UNAVAILABLE_VALUE_PLACEHOLDER

CONFIG_DEFINITION

protected static final ConfigDefinition CONFIG_DEFINITION

tableFilters

private final RelationalTableFilters tableFilters

columnFilter

private final Tables.ColumnNameFilter columnFilter

temporalPrecisionMode

private final TemporalPrecisionMode temporalPrecisionMode

keyMapper

private final Key.KeyMapper keyMapper

tableIdMapper

private final Selectors.TableIdToStringMapper tableIdMapper

jdbcConfig

private final Configuration jdbcConfig

heartbeatActionQuery

private final String heartbeatActionQuery

Constructor Detail

RelationalDatabaseConnectorConfig

protected RelationalDatabaseConnectorConfig(Configuration config,
                                            String logicalName,
                                            Tables.TableFilter systemTablesFilter,
                                            Selectors.TableIdToStringMapper tableIdMapper,
                                            int defaultSnapshotFetchSize,
                                            ColumnFilterMode columnFilterMode)

Method Detail

getTableFilters

public RelationalTableFilters getTableFilters()

getDecimalMode

public JdbcValueConverters.DecimalMode getDecimalMode()

Returns the Decimal mode Enum for decimal.handling.mode configuration. This defaults to precise if nothing is provided.

getTemporalPrecisionMode

public TemporalPrecisionMode getTemporalPrecisionMode()

Returns the temporal precision mode mode Enum for time.precision.mode configuration. This defaults to adaptive if nothing is provided.

getKeyMapper

public Key.KeyMapper getKeyMapper()

getJdbcConfig

public Configuration getJdbcConfig()

Returns a "raw" configuration object exposing all the database driver related settings, without the "database." prefix. Typically used for passing through driver settings.

getHeartbeatActionQuery

public String getHeartbeatActionQuery()

getUnavailableValuePlaceholder

public byte[] getUnavailableValuePlaceholder()

snapshotLockTimeout

public Duration snapshotLockTimeout()

schemaExcludeList

public String schemaExcludeList()

schemaIncludeList

public String schemaIncludeList()

tableExcludeList

public String tableExcludeList()

tableIncludeList

public String tableIncludeList()

getColumnFilter

public Tables.ColumnNameFilter getColumnFilter()

isFullColummnScanRequired

public Boolean isFullColummnScanRequired()

validateColumnBlacklist

private static int validateColumnBlacklist(Configuration config,
                                           Field field,
                                           Field.ValidationOutput problems)

validateColumnExcludeList

private static int validateColumnExcludeList(Configuration config,
                                             Field field,
                                             Field.ValidationOutput problems)

isSchemaChangesHistoryEnabled

public boolean isSchemaChangesHistoryEnabled()

Overrides:

isSchemaChangesHistoryEnabled in class CommonConnectorConfig

Returns:

true if the connector should emit messages about schema changes into a public facing topic.

isSchemaCommentsHistoryEnabled

public boolean isSchemaCommentsHistoryEnabled()

Overrides:

isSchemaCommentsHistoryEnabled in class CommonConnectorConfig

Returns:

true if the connector should emit messages which include table and column comments.

getTableIdMapper

public Selectors.TableIdToStringMapper getTableIdMapper()

validateTableBlacklist

private static int validateTableBlacklist(Configuration config,
                                          Field field,
                                          Field.ValidationOutput problems)

validateTableExcludeList

private static int validateTableExcludeList(Configuration config,
                                            Field field,
                                            Field.ValidationOutput problems)

getSnapshotSelectOverridesByTable

public Map<TableId,String> getSnapshotSelectOverridesByTable()

Returns any SELECT overrides, if present.

validateSchemaBlacklist

private static int validateSchemaBlacklist(Configuration config,
                                           Field field,
                                           Field.ValidationOutput problems)

validateSchemaExcludeList

private static int validateSchemaExcludeList(Configuration config,
                                             Field field,
                                             Field.ValidationOutput problems)

validateDatabaseExcludeList

private static int validateDatabaseExcludeList(Configuration config,
                                               Field field,
                                               Field.ValidationOutput problems)

validateDatabaseBlacklist

private static int validateDatabaseBlacklist(Configuration config,
                                             Field field,
                                             Field.ValidationOutput problems)

validateMessageKeyColumnsField

private static int validateMessageKeyColumnsField(Configuration config,
                                                  Field field,
                                                  Field.ValidationOutput problems)

validateServerName

private static int validateServerName(Configuration config,
                                      Field field,
                                      Field.ValidationOutput problems)