io.debezium.connector.mysql

Class MySqlConnectorConfig

java.lang.Object

io.debezium.config.CommonConnectorConfig

io.debezium.relational.RelationalDatabaseConnectorConfig

io.debezium.connector.mysql.MySqlConnectorConfig

public class MySqlConnectorConfig
extends RelationalDatabaseConnectorConfig

The configuration properties.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MySqlConnectorConfig.BigIntUnsignedHandlingMode The set of predefined BigIntUnsignedHandlingMode options or aliases.
static class	MySqlConnectorConfig.GtidNewChannelPosition The set of predefined Gtid New Channel Position options.
static class	MySqlConnectorConfig.SecureConnectionMode The set of predefined SecureConnectionMode options or aliases.
static class	MySqlConnectorConfig.SnapshotLockingMode The set of predefined Snapshot Locking Mode options.
static class	MySqlConnectorConfig.SnapshotMode The set of predefined SnapshotMode options or aliases.
static class	MySqlConnectorConfig.SnapshotNewTables

Nested classes/interfaces inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig

RelationalDatabaseConnectorConfig.DecimalHandlingMode

Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig

CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.Version

Field Summary

Fields

Modifier and Type	Field and Description
static Field.Set	ALL_FIELDS The set of Fields defined as part of this configuration.
static Field	BIGINT_UNSIGNED_HANDLING_MODE
static Field	BUFFER_SIZE_FOR_BINLOG_READER
static Field	CONNECTION_TIMEOUT_MS
private Duration	connectionTimeout
static Field	DATABASE_BLACKLIST A comma-separated list of regular expressions that match database names to be excluded from monitoring.
private static String	DATABASE_BLACKLIST_NAME
static Field	DATABASE_HISTORY The database history class is hidden in the configDef() since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.
static Field	DATABASE_WHITELIST A comma-separated list of regular expressions that match database names to be monitored.
private static String	DATABASE_WHITELIST_NAME
private static int	DEFAULT_BINLOG_BUFFER_SIZE Default size of the binlog buffer used for examining transactions and deciding whether to propagate them or not.
protected static int	DEFAULT_SNAPSHOT_FETCH_SIZE Minimum value used for fetch size hint.
static Field	ENABLE_TIME_ADJUSTER
static Field	EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE
protected static Field.Set	EXPOSED_FIELDS The set of Fields that are included in the configuration definition.
private MySqlConnectorConfig.GtidNewChannelPosition	gitIdNewChannelPosition
static Field	GTID_NEW_CHANNEL_POSITION If set to 'latest', connector when encountering new GTID channel after job restart will start reading it from the latest executed position (default).
static Field	GTID_SOURCE_EXCLUDES A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.
static Field	GTID_SOURCE_FILTER_DML_EVENTS If set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by the GTID_SOURCE_INCLUDES or GTID_SOURCE_EXCLUDES configuration options, if they are specified.
static Field	GTID_SOURCE_INCLUDES A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.
static Field	HOSTNAME
static Field	INCLUDE_SQL_QUERY
static Field	INCONSISTENT_SCHEMA_HANDLING_MODE
static Field	JDBC_DRIVER
static Field	KEEP_ALIVE
static Field	KEEP_ALIVE_INTERVAL_MS
private static org.slf4j.Logger	LOGGER
static Field	ON_CONNECT_STATEMENTS
static Field	PASSWORD
static Field	PORT
static Field	ROW_COUNT_FOR_STREAMING_RESULT_SETS
static Field	SERVER_ID
static Field	SERVER_ID_OFFSET
static Field	SERVER_NAME
static Field	SNAPSHOT_LOCKING_MODE
static Field	SNAPSHOT_MODE
static Field	SNAPSHOT_NEW_TABLES
private MySqlConnectorConfig.SnapshotLockingMode	snapshotLockingMode
private MySqlConnectorConfig.SnapshotNewTables	snapshotNewTables
static Field	SSL_KEYSTORE
static Field	SSL_KEYSTORE_PASSWORD
static Field	SSL_MODE
static Field	SSL_TRUSTSTORE
static Field	SSL_TRUSTSTORE_PASSWORD
static Field	TABLES_IGNORE_BUILTIN
private TemporalPrecisionMode	temporalPrecisionMode
static Field	TIME_PRECISION_MODE
static Field	USER

Fields inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig

COLUMN_BLACKLIST, COLUMN_WHITELIST, CONFIG_DEFINITION, DECIMAL_HANDLING_MODE, DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS, INCLUDE_SCHEMA_CHANGES, MASK_COLUMN, MASK_COLUMN_WITH_HASH, MSG_KEY_COLUMNS, SCHEMA_BLACKLIST, SCHEMA_WHITELIST, SNAPSHOT_LOCK_TIMEOUT_MS, SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE, TABLE_BLACKLIST, TABLE_BLACKLIST_NAME, TABLE_IGNORE_BUILTIN, TABLE_WHITELIST, TABLE_WHITELIST_NAME, TRUNCATE_COLUMN

Fields inherited from class io.debezium.config.CommonConnectorConfig

BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_POLL_INTERVAL_MILLIS, EVENT_PROCESSING_FAILURE_HANDLING_MODE, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, SANITIZE_FIELD_NAMES, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SOURCE_STRUCT_MAKER_VERSION, TOMBSTONES_ON_DELETE

Constructor Summary

Constructors

Constructor and Description
MySqlConnectorConfig(Configuration config)

Method Summary

Modifier and Type	Method and Description
protected static org.apache.kafka.common.config.ConfigDef	configDef()
Duration	getConnectionTimeout()
String	getConnectorName()
String	getContextName()
MySqlConnectorConfig.SnapshotLockingMode	getSnapshotLockingMode()
MySqlConnectorConfig.SnapshotNewTables	getSnapshotNewTables()
protected SourceInfoStructMaker<? extends AbstractSourceInfo>	getSourceInfoStructMaker(CommonConnectorConfig.Version version)
TemporalPrecisionMode	getTemporalPrecisionMode()
MySqlConnectorConfig.GtidNewChannelPosition	gtidNewChannelPosition()
private static int	randomServerId()
private static int	validateDatabaseBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateEventDeserializationFailureHandlingModeNotSet(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateGtidNewChannelPositionNotSet(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateGtidSetExcludes(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateInconsistentSchemaHandlingModeNotIgnore(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateSnapshotLockingMode(Configuration config, Field field, Field.ValidationOutput problems) Validate the new snapshot.locking.mode configuration, which replaces snapshot.minimal.locking.
private static int	validateTableBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateTimePrecisionMode(Configuration config, Field field, Field.ValidationOutput problems) Validate the time.precision.mode configuration.

Methods inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig

getDecimalMode, getKeyMapper, getSnapshotSelectOverridesByTable, getTableFilters, getTableIdMapper, isSchemaChangesHistoryEnabled, snapshotLockTimeout

Methods inherited from class io.debezium.config.CommonConnectorConfig

binaryHandlingMode, customConverterRegistry, getConfig, getEventProcessingFailureHandlingMode, getHeartbeatTopicsPrefix, getLogicalName, getMaxBatchSize, getMaxQueueSize, getPollInterval, getSanitizeFieldNames, getSkippedOps, getSnapshotDelay, getSnapshotFetchSize, getSourceInfoStructMaker, isEmitTombstoneOnDelete, shouldProvideTransactionMetadata, validateServerNameIsDifferentFromHistoryTopicName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

DEFAULT_SNAPSHOT_FETCH_SIZE

protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE

Minimum value used for fetch size hint. See DBZ-94 for details.

See Also:

Constant Field Values

DATABASE_WHITELIST_NAME

private static final String DATABASE_WHITELIST_NAME

See Also:

Constant Field Values

DATABASE_BLACKLIST_NAME

private static final String DATABASE_BLACKLIST_NAME

See Also:

Constant Field Values

DEFAULT_BINLOG_BUFFER_SIZE

private static final int DEFAULT_BINLOG_BUFFER_SIZE

Default size of the binlog buffer used for examining transactions and deciding whether to propagate them or not. A size of 0 disables the buffer, all events will be passed on directly as they are passed by the binlog client.

See Also:

Constant Field Values

HOSTNAME

public static final Field HOSTNAME

PORT

public static final Field PORT

USER

public static final Field USER

PASSWORD

public static final Field PASSWORD

ON_CONNECT_STATEMENTS

public static final Field ON_CONNECT_STATEMENTS

SERVER_NAME

public static final Field SERVER_NAME

SERVER_ID

public static final Field SERVER_ID

SERVER_ID_OFFSET

public static final Field SERVER_ID_OFFSET

SSL_MODE

public static final Field SSL_MODE

SSL_KEYSTORE

public static final Field SSL_KEYSTORE

SSL_KEYSTORE_PASSWORD

public static final Field SSL_KEYSTORE_PASSWORD

SSL_TRUSTSTORE

public static final Field SSL_TRUSTSTORE

SSL_TRUSTSTORE_PASSWORD

public static final Field SSL_TRUSTSTORE_PASSWORD

TABLES_IGNORE_BUILTIN

public static final Field TABLES_IGNORE_BUILTIN

JDBC_DRIVER

public static final Field JDBC_DRIVER

DATABASE_WHITELIST

public static final Field DATABASE_WHITELIST

A comma-separated list of regular expressions that match database names to be monitored. May not be used with DATABASE_BLACKLIST.

DATABASE_BLACKLIST

public static final Field DATABASE_BLACKLIST

A comma-separated list of regular expressions that match database names to be excluded from monitoring. May not be used with DATABASE_WHITELIST.

GTID_SOURCE_INCLUDES

public static final Field GTID_SOURCE_INCLUDES

A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching one of these include patterns will be used. May not be used with GTID_SOURCE_EXCLUDES.

GTID_SOURCE_EXCLUDES

public static final Field GTID_SOURCE_EXCLUDES

A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching none of these exclude patterns will be used. May not be used with GTID_SOURCE_INCLUDES.

GTID_SOURCE_FILTER_DML_EVENTS

public static final Field GTID_SOURCE_FILTER_DML_EVENTS

If set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by the GTID_SOURCE_INCLUDES or GTID_SOURCE_EXCLUDES configuration options, if they are specified. Defaults to true. When true, either GTID_SOURCE_INCLUDES or GTID_SOURCE_EXCLUDES must be set.

GTID_NEW_CHANNEL_POSITION

public static final Field GTID_NEW_CHANNEL_POSITION

If set to 'latest', connector when encountering new GTID channel after job restart will start reading it from the latest executed position (default). When set to 'earliest' the connector will start reading new GTID channels from the first available position. This is useful when in active-passive mysql setup during failover new GTID channel starts receiving writes, see DBZ-923. Defaults to latest.

CONNECTION_TIMEOUT_MS

public static final Field CONNECTION_TIMEOUT_MS

KEEP_ALIVE

public static final Field KEEP_ALIVE

KEEP_ALIVE_INTERVAL_MS

public static final Field KEEP_ALIVE_INTERVAL_MS

ROW_COUNT_FOR_STREAMING_RESULT_SETS

public static final Field ROW_COUNT_FOR_STREAMING_RESULT_SETS

BUFFER_SIZE_FOR_BINLOG_READER

public static final Field BUFFER_SIZE_FOR_BINLOG_READER

DATABASE_HISTORY

public static final Field DATABASE_HISTORY

The database history class is hidden in the configDef() since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.

INCLUDE_SQL_QUERY

public static final Field INCLUDE_SQL_QUERY

SNAPSHOT_MODE

public static final Field SNAPSHOT_MODE

SNAPSHOT_LOCKING_MODE

public static final Field SNAPSHOT_LOCKING_MODE

SNAPSHOT_NEW_TABLES

public static final Field SNAPSHOT_NEW_TABLES

TIME_PRECISION_MODE

public static final Field TIME_PRECISION_MODE

BIGINT_UNSIGNED_HANDLING_MODE

public static final Field BIGINT_UNSIGNED_HANDLING_MODE

EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE

public static final Field EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE

INCONSISTENT_SCHEMA_HANDLING_MODE

public static final Field INCONSISTENT_SCHEMA_HANDLING_MODE

ENABLE_TIME_ADJUSTER

public static final Field ENABLE_TIME_ADJUSTER

ALL_FIELDS

public static Field.Set ALL_FIELDS

The set of Fields defined as part of this configuration.

EXPOSED_FIELDS

protected static Field.Set EXPOSED_FIELDS

The set of Fields that are included in the configuration definition. This includes all fields defined in this class (though some are always invisible since they are not to be exposed to the user interface) plus several that are specific to the KafkaDatabaseHistory class, since history is always stored in Kafka when run via the user interface.

snapshotLockingMode

private final MySqlConnectorConfig.SnapshotLockingMode snapshotLockingMode

gitIdNewChannelPosition

private final MySqlConnectorConfig.GtidNewChannelPosition gitIdNewChannelPosition

snapshotNewTables

private final MySqlConnectorConfig.SnapshotNewTables snapshotNewTables

temporalPrecisionMode

private final TemporalPrecisionMode temporalPrecisionMode

connectionTimeout

private final Duration connectionTimeout

Constructor Detail

MySqlConnectorConfig

public MySqlConnectorConfig(Configuration config)

Method Detail

getSnapshotLockingMode

public MySqlConnectorConfig.SnapshotLockingMode getSnapshotLockingMode()

gtidNewChannelPosition

public MySqlConnectorConfig.GtidNewChannelPosition gtidNewChannelPosition()

getSnapshotNewTables

public MySqlConnectorConfig.SnapshotNewTables getSnapshotNewTables()

configDef

protected static org.apache.kafka.common.config.ConfigDef configDef()

validateGtidNewChannelPositionNotSet

private static int validateGtidNewChannelPositionNotSet(Configuration config,
                                                        Field field,
                                                        Field.ValidationOutput problems)

validateEventDeserializationFailureHandlingModeNotSet

private static int validateEventDeserializationFailureHandlingModeNotSet(Configuration config,
                                                                         Field field,
                                                                         Field.ValidationOutput problems)

validateInconsistentSchemaHandlingModeNotIgnore

private static int validateInconsistentSchemaHandlingModeNotIgnore(Configuration config,
                                                                   Field field,
                                                                   Field.ValidationOutput problems)

validateDatabaseBlacklist

private static int validateDatabaseBlacklist(Configuration config,
                                             Field field,
                                             Field.ValidationOutput problems)

validateTableBlacklist

private static int validateTableBlacklist(Configuration config,
                                          Field field,
                                          Field.ValidationOutput problems)

validateGtidSetExcludes

private static int validateGtidSetExcludes(Configuration config,
                                           Field field,
                                           Field.ValidationOutput problems)

validateSnapshotLockingMode

private static int validateSnapshotLockingMode(Configuration config,
                                               Field field,
                                               Field.ValidationOutput problems)

Validate the new snapshot.locking.mode configuration, which replaces snapshot.minimal.locking. If minimal.locking is explicitly defined and locking.mode is NOT explicitly defined: - coerce minimal.locking into the new snap.locking.mode property. If minimal.locking is NOT explicitly defined and locking.mode IS explicitly defined: - use new locking.mode property. If BOTH minimal.locking and locking.mode ARE defined: - Throw a validation error.

validateTimePrecisionMode

private static int validateTimePrecisionMode(Configuration config,
                                             Field field,
                                             Field.ValidationOutput problems)

Validate the time.precision.mode configuration. If adaptive is specified, this option has the potential to cause overflow which is why the option was deprecated and no longer supported for this connector.

randomServerId

private static int randomServerId()

getSourceInfoStructMaker

protected SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker(CommonConnectorConfig.Version version)

Specified by:

getSourceInfoStructMaker in class CommonConnectorConfig

getContextName

public String getContextName()

Specified by:

getContextName in class CommonConnectorConfig

getConnectorName

public String getConnectorName()

Specified by:

getConnectorName in class CommonConnectorConfig

getTemporalPrecisionMode

public TemporalPrecisionMode getTemporalPrecisionMode()

Overrides:

getTemporalPrecisionMode in class RelationalDatabaseConnectorConfig

getConnectionTimeout

public Duration getConnectionTimeout()