Package io.debezium.connector.mysql
Class MySqlConnectorConfig
- java.lang.Object
-
- io.debezium.config.CommonConnectorConfig
-
- io.debezium.relational.RelationalDatabaseConnectorConfig
-
- io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
-
- io.debezium.connector.mysql.MySqlConnectorConfig
-
public class MySqlConnectorConfig extends HistorizedRelationalDatabaseConnectorConfig
The configuration properties.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
MySqlConnectorConfig.BigIntUnsignedHandlingMode
The set of predefined BigIntUnsignedHandlingMode options or aliases.static class
MySqlConnectorConfig.GtidNewChannelPosition
The set of predefined Gtid New Channel Position options.static class
MySqlConnectorConfig.SecureConnectionMode
The set of predefined SecureConnectionMode options or aliases.static class
MySqlConnectorConfig.SnapshotLockingMode
The set of predefined Snapshot Locking Mode options.static class
MySqlConnectorConfig.SnapshotMode
The set of predefined SnapshotMode options or aliases.static class
MySqlConnectorConfig.SnapshotNewTables
-
Nested classes/interfaces inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
RelationalDatabaseConnectorConfig.DecimalHandlingMode
-
Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig
CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.Version
-
-
Field Summary
Fields Modifier and Type Field Description static Field.Set
ALL_FIELDS
The set ofField
s defined as part of this configuration.static Field
BIGINT_UNSIGNED_HANDLING_MODE
static Field
BUFFER_SIZE_FOR_BINLOG_READER
protected static Set<String>
BUILT_IN_DB_NAMES
private Configuration
config
private static ConfigDefinition
CONFIG_DEFINITION
static Field
CONNECTION_TIMEOUT_MS
private Duration
connectionTimeout
static Field
DATABASE_HISTORY
The database history class is hidden in theconfigDef()
since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.private Predicate<String>
ddlFilter
private static int
DEFAULT_BINLOG_BUFFER_SIZE
Default size of the binlog buffer used for examining transactions and deciding whether to propagate them or not.protected static int
DEFAULT_PORT
protected static int
DEFAULT_SNAPSHOT_FETCH_SIZE
Minimum value
used for fetch size hint.static Field
ENABLE_TIME_ADJUSTER
static Field
EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE
protected static Field.Set
EXPOSED_FIELDS
private MySqlConnectorConfig.GtidNewChannelPosition
gitIdNewChannelPosition
static Field
GTID_NEW_CHANNEL_POSITION
If set to 'latest', connector when encountering new GTID channel after job restart will start reading it from the latest executed position (default).static Field
GTID_SOURCE_EXCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.static Field
GTID_SOURCE_FILTER_DML_EVENTS
If set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by theGTID_SOURCE_INCLUDES
orGTID_SOURCE_EXCLUDES
configuration options, if they are specified.static Field
GTID_SOURCE_INCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.private Predicate<String>
gtidSourceFilter
static Field
INCLUDE_SQL_QUERY
static Field
INCONSISTENT_SCHEMA_HANDLING_MODE
private CommonConnectorConfig.EventProcessingFailureHandlingMode
inconsistentSchemaFailureHandlingMode
static Field
JDBC_DRIVER
static Field
KEEP_ALIVE
static Field
KEEP_ALIVE_INTERVAL_MS
private boolean
legacy
private static org.slf4j.Logger
LOGGER
static Field
ON_CONNECT_STATEMENTS
static Field
PORT
static Field
ROW_COUNT_FOR_STREAMING_RESULT_SETS
static Field
SERVER_ID
static Field
SERVER_ID_OFFSET
static Field
SERVER_NAME
static Field
SNAPSHOT_LOCKING_MODE
static Field
SNAPSHOT_MODE
static Field
SNAPSHOT_NEW_TABLES
private MySqlConnectorConfig.SnapshotLockingMode
snapshotLockingMode
private MySqlConnectorConfig.SnapshotMode
snapshotMode
private MySqlConnectorConfig.SnapshotNewTables
snapshotNewTables
private SourceInfoStructMaker<? extends AbstractSourceInfo>
sourceInfoStructMaker
static Field
SSL_KEYSTORE
static Field
SSL_KEYSTORE_PASSWORD
static Field
SSL_MODE
static Field
SSL_TRUSTSTORE
static Field
SSL_TRUSTSTORE_PASSWORD
static Field
TABLES_IGNORE_BUILTIN
private TemporalPrecisionMode
temporalPrecisionMode
(package private) static String
TEST_DISABLE_GLOBAL_LOCKING
It is not possible to test disabled global locking locally as regular MySQL build always provides global locking.static Field
TIME_PRECISION_MODE
-
Fields inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
COLUMN_BLACKLIST, COLUMN_EXCLUDE_LIST, COLUMN_INCLUDE_LIST, COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, COLUMN_WHITELIST, COLUMN_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_BLACKLIST, DATABASE_BLACKLIST_NAME, DATABASE_EXCLUDE_LIST, DATABASE_EXCLUDE_LIST_NAME, DATABASE_INCLUDE_LIST, DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_INCLUDE_LIST_NAME, DATABASE_NAME, DATABASE_WHITELIST, DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_WHITELIST_NAME, DECIMAL_HANDLING_MODE, DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS, HOSTNAME, INCLUDE_SCHEMA_CHANGES, MASK_COLUMN, MASK_COLUMN_WITH_HASH, MSG_KEY_COLUMNS, PASSWORD, PROPAGATE_COLUMN_SOURCE_TYPE, PROPAGATE_DATATYPE_SOURCE_TYPE, SCHEMA_BLACKLIST, SCHEMA_EXCLUDE_LIST, SCHEMA_EXCLUDE_LIST_NAME, SCHEMA_INCLUDE_LIST, SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, SCHEMA_INCLUDE_LIST_NAME, SCHEMA_WHITELIST, SCHEMA_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, SERVER_NAME_PATTERN, SNAPSHOT_FULL_COLUMN_SCAN_FORCE, SNAPSHOT_LOCK_TIMEOUT_MS, SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE, TABLE_BLACKLIST, TABLE_BLACKLIST_NAME, TABLE_EXCLUDE_LIST, TABLE_EXCLUDE_LIST_NAME, TABLE_IGNORE_BUILTIN, TABLE_INCLUDE_LIST, TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_INCLUDE_LIST_NAME, TABLE_WHITELIST, TABLE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_WHITELIST_NAME, TRUNCATE_COLUMN, USER
-
Fields inherited from class io.debezium.config.CommonConnectorConfig
BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, EVENT_PROCESSING_FAILURE_HANDLING_MODE, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, SOURCE_STRUCT_MAKER_VERSION, TOMBSTONES_ON_DELETE
-
-
Constructor Summary
Constructors Constructor Description MySqlConnectorConfig(Configuration config)
-
Method Summary
-
Methods inherited from class io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
getDatabaseHistory, useCatalogBeforeSchema
-
Methods inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
getColumnFilter, getDecimalMode, getKeyMapper, getSnapshotSelectOverridesByTable, getTableFilters, getTableIdMapper, isFullColummnScanRequired, isSchemaChangesHistoryEnabled, schemaExcludeList, schemaIncludeList, snapshotLockTimeout, tableExcludeList, tableIncludeList
-
Methods inherited from class io.debezium.config.CommonConnectorConfig
binaryHandlingMode, customConverterRegistry, getConfig, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSnapshotMaxThreads, isEmitTombstoneOnDelete, shouldProvideTransactionMetadata, validate, validateAndRecord, validateServerNameIsDifferentFromHistoryTopicName
-
-
-
-
Field Detail
-
LOGGER
private static final org.slf4j.Logger LOGGER
-
TEST_DISABLE_GLOBAL_LOCKING
static final String TEST_DISABLE_GLOBAL_LOCKING
It is not possible to test disabled global locking locally as regular MySQL build always provides global locking. So to bypass this limitation it is necessary to provide a backdoor to connector to disable it on its own.- See Also:
- Constant Field Values
-
DEFAULT_SNAPSHOT_FETCH_SIZE
protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE
Minimum value
used for fetch size hint. See DBZ-94 for details.- See Also:
- Constant Field Values
-
DEFAULT_PORT
protected static final int DEFAULT_PORT
- See Also:
- Constant Field Values
-
DEFAULT_BINLOG_BUFFER_SIZE
private static final int DEFAULT_BINLOG_BUFFER_SIZE
Default size of the binlog buffer used for examining transactions and deciding whether to propagate them or not. A size of 0 disables the buffer, all events will be passed on directly as they are passed by the binlog client.- See Also:
- Constant Field Values
-
PORT
public static final Field PORT
-
ON_CONNECT_STATEMENTS
public static final Field ON_CONNECT_STATEMENTS
-
SERVER_NAME
public static final Field SERVER_NAME
-
SERVER_ID
public static final Field SERVER_ID
-
SERVER_ID_OFFSET
public static final Field SERVER_ID_OFFSET
-
SSL_MODE
public static final Field SSL_MODE
-
SSL_KEYSTORE
public static final Field SSL_KEYSTORE
-
SSL_KEYSTORE_PASSWORD
public static final Field SSL_KEYSTORE_PASSWORD
-
SSL_TRUSTSTORE
public static final Field SSL_TRUSTSTORE
-
SSL_TRUSTSTORE_PASSWORD
public static final Field SSL_TRUSTSTORE_PASSWORD
-
TABLES_IGNORE_BUILTIN
public static final Field TABLES_IGNORE_BUILTIN
-
JDBC_DRIVER
public static final Field JDBC_DRIVER
-
GTID_SOURCE_INCLUDES
public static final Field GTID_SOURCE_INCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching one of these include patterns will be used. Must not be used withGTID_SOURCE_EXCLUDES
.
-
GTID_SOURCE_EXCLUDES
public static final Field GTID_SOURCE_EXCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching none of these exclude patterns will be used. Must not be used withGTID_SOURCE_INCLUDES
.
-
GTID_SOURCE_FILTER_DML_EVENTS
public static final Field GTID_SOURCE_FILTER_DML_EVENTS
If set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by theGTID_SOURCE_INCLUDES
orGTID_SOURCE_EXCLUDES
configuration options, if they are specified. Defaults to true. When true, eitherGTID_SOURCE_INCLUDES
orGTID_SOURCE_EXCLUDES
must be set.
-
GTID_NEW_CHANNEL_POSITION
public static final Field GTID_NEW_CHANNEL_POSITION
If set to 'latest', connector when encountering new GTID channel after job restart will start reading it from the latest executed position (default). When set to 'earliest' the connector will start reading new GTID channels from the first available position. This is useful when in active-passive mysql setup during failover new GTID channel starts receiving writes, see DBZ-923. Defaults to latest.
-
CONNECTION_TIMEOUT_MS
public static final Field CONNECTION_TIMEOUT_MS
-
KEEP_ALIVE
public static final Field KEEP_ALIVE
-
KEEP_ALIVE_INTERVAL_MS
public static final Field KEEP_ALIVE_INTERVAL_MS
-
ROW_COUNT_FOR_STREAMING_RESULT_SETS
public static final Field ROW_COUNT_FOR_STREAMING_RESULT_SETS
-
BUFFER_SIZE_FOR_BINLOG_READER
public static final Field BUFFER_SIZE_FOR_BINLOG_READER
-
DATABASE_HISTORY
public static final Field DATABASE_HISTORY
The database history class is hidden in theconfigDef()
since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.
-
INCLUDE_SQL_QUERY
public static final Field INCLUDE_SQL_QUERY
-
SNAPSHOT_MODE
public static final Field SNAPSHOT_MODE
-
SNAPSHOT_LOCKING_MODE
public static final Field SNAPSHOT_LOCKING_MODE
-
SNAPSHOT_NEW_TABLES
public static final Field SNAPSHOT_NEW_TABLES
-
TIME_PRECISION_MODE
public static final Field TIME_PRECISION_MODE
-
BIGINT_UNSIGNED_HANDLING_MODE
public static final Field BIGINT_UNSIGNED_HANDLING_MODE
-
EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE
public static final Field EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE
-
INCONSISTENT_SCHEMA_HANDLING_MODE
public static final Field INCONSISTENT_SCHEMA_HANDLING_MODE
-
ENABLE_TIME_ADJUSTER
public static final Field ENABLE_TIME_ADJUSTER
-
CONFIG_DEFINITION
private static final ConfigDefinition CONFIG_DEFINITION
-
ALL_FIELDS
public static Field.Set ALL_FIELDS
The set ofField
s defined as part of this configuration.
-
EXPOSED_FIELDS
protected static Field.Set EXPOSED_FIELDS
-
config
private final Configuration config
-
snapshotMode
private final MySqlConnectorConfig.SnapshotMode snapshotMode
-
snapshotLockingMode
private final MySqlConnectorConfig.SnapshotLockingMode snapshotLockingMode
-
gitIdNewChannelPosition
private final MySqlConnectorConfig.GtidNewChannelPosition gitIdNewChannelPosition
-
snapshotNewTables
private final MySqlConnectorConfig.SnapshotNewTables snapshotNewTables
-
temporalPrecisionMode
private final TemporalPrecisionMode temporalPrecisionMode
-
connectionTimeout
private final Duration connectionTimeout
-
inconsistentSchemaFailureHandlingMode
private final CommonConnectorConfig.EventProcessingFailureHandlingMode inconsistentSchemaFailureHandlingMode
-
legacy
private final boolean legacy
-
sourceInfoStructMaker
private final SourceInfoStructMaker<? extends AbstractSourceInfo> sourceInfoStructMaker
-
-
Constructor Detail
-
MySqlConnectorConfig
public MySqlConnectorConfig(Configuration config)
-
-
Method Detail
-
configDef
protected static org.apache.kafka.common.config.ConfigDef configDef()
-
supportsOperationFiltering
public boolean supportsOperationFiltering()
- Overrides:
supportsOperationFiltering
in classCommonConnectorConfig
-
useCursorFetch
public boolean useCursorFetch()
-
getSnapshotLockingMode
public MySqlConnectorConfig.SnapshotLockingMode getSnapshotLockingMode()
-
gtidNewChannelPosition
public MySqlConnectorConfig.GtidNewChannelPosition gtidNewChannelPosition()
-
getSnapshotNewTables
public MySqlConnectorConfig.SnapshotNewTables getSnapshotNewTables()
-
validateGtidNewChannelPositionNotSet
private static int validateGtidNewChannelPositionNotSet(Configuration config, Field field, Field.ValidationOutput problems)
-
validateEventDeserializationFailureHandlingModeNotSet
private static int validateEventDeserializationFailureHandlingModeNotSet(Configuration config, Field field, Field.ValidationOutput problems)
-
validateInconsistentSchemaHandlingModeNotIgnore
private static int validateInconsistentSchemaHandlingModeNotIgnore(Configuration config, Field field, Field.ValidationOutput problems)
-
validateGtidSetExcludes
private static int validateGtidSetExcludes(Configuration config, Field field, Field.ValidationOutput problems)
-
validateSnapshotLockingMode
private static int validateSnapshotLockingMode(Configuration config, Field field, Field.ValidationOutput problems)
Validate the new snapshot.locking.mode configuration, which replaces snapshot.minimal.locking. If minimal.locking is explicitly defined and locking.mode is NOT explicitly defined: - coerce minimal.locking into the new snap.locking.mode property. If minimal.locking is NOT explicitly defined and locking.mode IS explicitly defined: - use new locking.mode property. If BOTH minimal.locking and locking.mode ARE defined: - Throw a validation error.
-
validateTimePrecisionMode
private static int validateTimePrecisionMode(Configuration config, Field field, Field.ValidationOutput problems)
Validate the time.precision.mode configuration. Ifadaptive
is specified, this option has the potential to cause overflow which is why the option was deprecated and no longer supported for this connector.
-
randomServerId
private static int randomServerId()
-
getSourceInfoStructMaker
protected SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker(CommonConnectorConfig.Version version)
- Specified by:
getSourceInfoStructMaker
in classCommonConnectorConfig
-
getContextName
public String getContextName()
- Specified by:
getContextName
in classCommonConnectorConfig
-
getConnectorName
public String getConnectorName()
- Specified by:
getConnectorName
in classCommonConnectorConfig
-
getTemporalPrecisionMode
public TemporalPrecisionMode getTemporalPrecisionMode()
- Overrides:
getTemporalPrecisionMode
in classRelationalDatabaseConnectorConfig
-
getSnapshotMode
public MySqlConnectorConfig.SnapshotMode getSnapshotMode()
-
getConnectionTimeout
public Duration getConnectionTimeout()
-
inconsistentSchemaFailureHandlingMode
public CommonConnectorConfig.EventProcessingFailureHandlingMode inconsistentSchemaFailureHandlingMode()
-
hostname
public String hostname()
-
port
public int port()
-
username
public String username()
-
password
public String password()
-
serverId
public long serverId()
-
sslMode
public MySqlConnectorConfig.SecureConnectionMode sslMode()
-
sslModeEnabled
public boolean sslModeEnabled()
-
bufferSizeForStreamingChangeEventSource
public int bufferSizeForStreamingChangeEventSource()
-
gtidSourceFilter
public Predicate<String> gtidSourceFilter()
Get the predicate function that will returntrue
if a GTID source is to be included, orfalse
if a GTID source is to be excluded.- Returns:
- the GTID source predicate function; never null
-
includeSchemaChangeRecords
public boolean includeSchemaChangeRecords()
-
includeSqlQuery
public boolean includeSqlQuery()
-
rowCountForLargeTable
public long rowCountForLargeTable()
-
getHistoryRecordComparator
protected HistoryRecordComparator getHistoryRecordComparator()
- Specified by:
getHistoryRecordComparator
in classHistorizedRelationalDatabaseConnectorConfig
-
getColumnExcludeNameFilter
private static Tables.ColumnNameFilter getColumnExcludeNameFilter(String excludedColumnPatterns)
-
getColumnIncludeNameFilter
private static Tables.ColumnNameFilter getColumnIncludeNameFilter(String excludedColumnPatterns)
-
isBuiltInDatabase
public static boolean isBuiltInDatabase(String databaseName)
-
isNotBuiltInTable
public static boolean isNotBuiltInTable(TableId id)
-
legacy
boolean legacy()
-
useGlobalLock
boolean useGlobalLock()
Intended for testing only
-
getSourceInfoStructMaker
public SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker()
- Overrides:
getSourceInfoStructMaker
in classCommonConnectorConfig
-
-