Package io.debezium.connector.mysql
Class MySqlConnectorConfig
java.lang.Object
io.debezium.config.CommonConnectorConfig
io.debezium.relational.RelationalDatabaseConnectorConfig
io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
io.debezium.connector.mysql.MySqlConnectorConfig
The configuration properties.
-
Nested Class Summary
Modifier and TypeClassDescriptionstatic enum
The set of predefined BigIntUnsignedHandlingMode options or aliases.static enum
The set of predefined Gtid New Channel Position options.static enum
The set of predefined SecureConnectionMode options or aliases.static enum
The set of predefined Snapshot Locking Mode options.static enum
The set of predefined SnapshotMode options or aliases.static enum
Nested classes/interfaces inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
RelationalDatabaseConnectorConfig.DecimalHandlingMode
Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig
CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.SchemaNameAdjustmentMode, CommonConnectorConfig.Version
-
Field Summary
Modifier and TypeFieldDescriptionstatic Field.Set
The set ofField
s defined as part of this configuration.static final Field
static final Field
private final Configuration
private static final ConfigDefinition
static final Field
private final Duration
static final Field
The database history class is hidden in theconfigDef()
since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.private static final int
Default size of the binlog buffer used for examining transactions and deciding whether to propagate them or not.protected static final int
protected static final int
Minimum value
used for fetch size hint.static final Field
static final Field
protected static Field.Set
private final MySqlConnectorConfig.GtidNewChannelPosition
static final Field
Deprecated.static final Field
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.static final Field
If set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by theGTID_SOURCE_INCLUDES
orGTID_SOURCE_EXCLUDES
configuration options, if they are specified.static final Field
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.static final Field
static final Field
private final CommonConnectorConfig.EventProcessingFailureHandlingMode
static final Field
static final Field
static final Field
private final boolean
private static final org.slf4j.Logger
static final Field
static final Field
static final Field
private final boolean
static final Field
static final Field
static final Field
static final Field
static final Field
static final Field
static final Field
private final MySqlConnectorConfig.SnapshotLockingMode
private final MySqlConnectorConfig.SnapshotMode
private final MySqlConnectorConfig.SnapshotNewTables
private final SourceInfoStructMaker<? extends AbstractSourceInfo>
static final Field
static final Field
static final Field
static final Field
static final Field
static final Field
private final TemporalPrecisionMode
(package private) static final String
It is not possible to test disabled global locking locally as regular MySQL build always provides global locking.static final Field
Fields inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
COLUMN_BLACKLIST, COLUMN_EXCLUDE_LIST, COLUMN_INCLUDE_LIST, COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, COLUMN_WHITELIST, COLUMN_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_BLACKLIST, DATABASE_BLACKLIST_NAME, DATABASE_EXCLUDE_LIST, DATABASE_EXCLUDE_LIST_NAME, DATABASE_INCLUDE_LIST, DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_INCLUDE_LIST_NAME, DATABASE_NAME, DATABASE_WHITELIST, DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_WHITELIST_NAME, DECIMAL_HANDLING_MODE, DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS, DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER, HOSTNAME, INCLUDE_SCHEMA_CHANGES, INCLUDE_SCHEMA_COMMENTS, MASK_COLUMN, MASK_COLUMN_WITH_HASH, MSG_KEY_COLUMNS, PASSWORD, PROPAGATE_COLUMN_SOURCE_TYPE, PROPAGATE_DATATYPE_SOURCE_TYPE, SCHEMA_BLACKLIST, SCHEMA_EXCLUDE_LIST, SCHEMA_EXCLUDE_LIST_NAME, SCHEMA_INCLUDE_LIST, SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, SCHEMA_INCLUDE_LIST_NAME, SCHEMA_WHITELIST, SCHEMA_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, SERVER_NAME_PATTERN, SNAPSHOT_FULL_COLUMN_SCAN_FORCE, SNAPSHOT_LOCK_TIMEOUT_MS, SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE, TABLE_BLACKLIST, TABLE_BLACKLIST_NAME, TABLE_EXCLUDE_LIST, TABLE_EXCLUDE_LIST_NAME, TABLE_IGNORE_BUILTIN, TABLE_INCLUDE_LIST, TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_INCLUDE_LIST_NAME, TABLE_WHITELIST, TABLE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_WHITELIST_NAME, TRUNCATE_COLUMN, UNAVAILABLE_VALUE_PLACEHOLDER, USER
Fields inherited from class io.debezium.config.CommonConnectorConfig
BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, CUSTOM_RETRIABLE_EXCEPTION, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, EVENT_PROCESSING_FAILURE_HANDLING_MODE, INCREMENTAL_SNAPSHOT_ALLOW_SCHEMA_CHANGES, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SCHEMA_NAME_ADJUSTMENT_MODE, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, SOURCE_STRUCT_MAKER_VERSION, TASK_ID, TOMBSTONES_ON_DELETE, TRANSACTION_TOPIC
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionint
protected static org.apache.kafka.common.config.ConfigDef
protected HistoryRecordComparator
SourceInfoStructMaker<? extends AbstractSourceInfo>
protected SourceInfoStructMaker<? extends AbstractSourceInfo>
Get the predicate function that will returntrue
if a GTID source is to be included, orfalse
if a GTID source is to be excluded.hostname()
boolean
boolean
static boolean
isBuiltInDatabase
(String databaseName) static boolean
boolean
(package private) boolean
legacy()
password()
int
port()
private static int
long
long
serverId()
sslMode()
boolean
boolean
protected boolean
boolean
(package private) boolean
Intended for testing onlyusername()
private static int
validateEventDeserializationFailureHandlingModeNotSet
(Configuration config, Field field, Field.ValidationOutput problems) private static int
validateGtidNewChannelPositionNotSet
(Configuration config, Field field, Field.ValidationOutput problems) private static int
validateGtidSetExcludes
(Configuration config, Field field, Field.ValidationOutput problems) private static int
validateInconsistentSchemaHandlingModeNotIgnore
(Configuration config, Field field, Field.ValidationOutput problems) private static int
validateSnapshotLockingMode
(Configuration config, Field field, Field.ValidationOutput problems) Validate the new snapshot.locking.mode configuration, which replaces snapshot.minimal.locking.private static int
validateTimePrecisionMode
(Configuration config, Field field, Field.ValidationOutput problems) Validate the time.precision.mode configuration.Methods inherited from class io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
getDatabaseHistory, useCatalogBeforeSchema
Methods inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
getColumnFilter, getDecimalMode, getHeartbeatActionQuery, getJdbcConfig, getKeyMapper, getSnapshotSelectOverridesByTable, getTableFilters, getTableIdMapper, getUnavailableValuePlaceholder, isFullColummnScanRequired, isSchemaChangesHistoryEnabled, isSchemaCommentsHistoryEnabled, schemaExcludeList, schemaIncludeList, snapshotLockTimeout, tableExcludeList, tableIncludeList
Methods inherited from class io.debezium.config.CommonConnectorConfig
binaryHandlingMode, customConverterRegistry, customRetriableException, getConfig, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSnapshotMaxThreads, getTaskId, getTransactionTopic, isEmitTombstoneOnDelete, isIncrementalSnapshotSchemaChangesEnabled, isSignalDataCollection, legacyGetDataCollectionsToBeSnapshotted, parseSignallingMessage, schemaNameAdjustmentMode, shouldProvideTransactionMetadata, validate, validateAndRecord, validateServerNameIsDifferentFromHistoryTopicName, validateSkippedOperation
-
Field Details
-
LOGGER
private static final org.slf4j.Logger LOGGER -
TEST_DISABLE_GLOBAL_LOCKING
It is not possible to test disabled global locking locally as regular MySQL build always provides global locking. So to bypass this limitation it is necessary to provide a backdoor to connector to disable it on its own.- See Also:
-
DEFAULT_SNAPSHOT_FETCH_SIZE
protected static final int DEFAULT_SNAPSHOT_FETCH_SIZEMinimum value
used for fetch size hint. See DBZ-94 for details.- See Also:
-
DEFAULT_PORT
protected static final int DEFAULT_PORT- See Also:
-
DEFAULT_BINLOG_BUFFER_SIZE
private static final int DEFAULT_BINLOG_BUFFER_SIZEDefault size of the binlog buffer used for examining transactions and deciding whether to propagate them or not. A size of 0 disables the buffer, all events will be passed on directly as they are passed by the binlog client.- See Also:
-
PORT
-
ON_CONNECT_STATEMENTS
-
SERVER_NAME
-
SERVER_ID
-
SERVER_ID_OFFSET
-
SSL_MODE
-
SSL_KEYSTORE
-
SSL_KEYSTORE_PASSWORD
-
SSL_TRUSTSTORE
-
SSL_TRUSTSTORE_PASSWORD
-
TABLES_IGNORE_BUILTIN
-
JDBC_DRIVER
-
GTID_SOURCE_INCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching one of these include patterns will be used. Must not be used withGTID_SOURCE_EXCLUDES
. -
GTID_SOURCE_EXCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching none of these exclude patterns will be used. Must not be used withGTID_SOURCE_INCLUDES
. -
GTID_SOURCE_FILTER_DML_EVENTS
If set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by theGTID_SOURCE_INCLUDES
orGTID_SOURCE_EXCLUDES
configuration options, if they are specified. Defaults to true. When true, eitherGTID_SOURCE_INCLUDES
orGTID_SOURCE_EXCLUDES
must be set. -
GTID_NEW_CHANNEL_POSITION
Deprecated.If set to 'latest', connector when encountering new GTID channel after job restart will start reading it from the latest executed position (default). When set to 'earliest' the connector will start reading new GTID channels from the first available position. This is useful when in active-passive mysql setup during failover new GTID channel starts receiving writes, see DBZ-923. Defaults to latest. -
CONNECTION_TIMEOUT_MS
-
KEEP_ALIVE
-
KEEP_ALIVE_INTERVAL_MS
-
ROW_COUNT_FOR_STREAMING_RESULT_SETS
-
BUFFER_SIZE_FOR_BINLOG_READER
-
DATABASE_HISTORY
The database history class is hidden in theconfigDef()
since that is designed to work with a user interface, and in these situations using Kafka is the only way to go. -
INCLUDE_SQL_QUERY
-
SNAPSHOT_MODE
-
SNAPSHOT_LOCKING_MODE
-
SNAPSHOT_NEW_TABLES
-
TIME_PRECISION_MODE
-
BIGINT_UNSIGNED_HANDLING_MODE
-
EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE
-
INCONSISTENT_SCHEMA_HANDLING_MODE
-
ENABLE_TIME_ADJUSTER
-
READ_ONLY_CONNECTION
-
CONFIG_DEFINITION
-
ALL_FIELDS
The set ofField
s defined as part of this configuration. -
EXPOSED_FIELDS
-
BUILT_IN_DB_NAMES
-
config
-
snapshotMode
-
snapshotLockingMode
-
gitIdNewChannelPosition
-
snapshotNewTables
-
temporalPrecisionMode
-
connectionTimeout
-
gtidSourceFilter
-
inconsistentSchemaFailureHandlingMode
private final CommonConnectorConfig.EventProcessingFailureHandlingMode inconsistentSchemaFailureHandlingMode -
ddlFilter
-
legacy
private final boolean legacy -
sourceInfoStructMaker
-
readOnlyConnection
private final boolean readOnlyConnection
-
-
Constructor Details
-
MySqlConnectorConfig
-
-
Method Details
-
configDef
protected static org.apache.kafka.common.config.ConfigDef configDef() -
supportsOperationFiltering
public boolean supportsOperationFiltering()- Overrides:
supportsOperationFiltering
in classCommonConnectorConfig
-
supportsSchemaChangesDuringIncrementalSnapshot
protected boolean supportsSchemaChangesDuringIncrementalSnapshot()- Overrides:
supportsSchemaChangesDuringIncrementalSnapshot
in classCommonConnectorConfig
-
useCursorFetch
public boolean useCursorFetch() -
getSnapshotLockingMode
-
gtidNewChannelPosition
-
getSnapshotNewTables
-
validateGtidNewChannelPositionNotSet
private static int validateGtidNewChannelPositionNotSet(Configuration config, Field field, Field.ValidationOutput problems) -
validateEventDeserializationFailureHandlingModeNotSet
private static int validateEventDeserializationFailureHandlingModeNotSet(Configuration config, Field field, Field.ValidationOutput problems) -
validateInconsistentSchemaHandlingModeNotIgnore
private static int validateInconsistentSchemaHandlingModeNotIgnore(Configuration config, Field field, Field.ValidationOutput problems) -
validateGtidSetExcludes
private static int validateGtidSetExcludes(Configuration config, Field field, Field.ValidationOutput problems) -
validateSnapshotLockingMode
private static int validateSnapshotLockingMode(Configuration config, Field field, Field.ValidationOutput problems) Validate the new snapshot.locking.mode configuration, which replaces snapshot.minimal.locking. If minimal.locking is explicitly defined and locking.mode is NOT explicitly defined: - coerce minimal.locking into the new snap.locking.mode property. If minimal.locking is NOT explicitly defined and locking.mode IS explicitly defined: - use new locking.mode property. If BOTH minimal.locking and locking.mode ARE defined: - Throw a validation error. -
validateTimePrecisionMode
private static int validateTimePrecisionMode(Configuration config, Field field, Field.ValidationOutput problems) Validate the time.precision.mode configuration. Ifadaptive
is specified, this option has the potential to cause overflow which is why the option was deprecated and no longer supported for this connector. -
randomServerId
private static int randomServerId() -
getSourceInfoStructMaker
protected SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker(CommonConnectorConfig.Version version) - Specified by:
getSourceInfoStructMaker
in classCommonConnectorConfig
-
getContextName
- Specified by:
getContextName
in classCommonConnectorConfig
-
getConnectorName
- Specified by:
getConnectorName
in classCommonConnectorConfig
-
getTemporalPrecisionMode
- Overrides:
getTemporalPrecisionMode
in classRelationalDatabaseConnectorConfig
-
getSnapshotMode
-
getConnectionTimeout
-
inconsistentSchemaFailureHandlingMode
public CommonConnectorConfig.EventProcessingFailureHandlingMode inconsistentSchemaFailureHandlingMode() -
hostname
-
port
public int port() -
username
-
password
-
serverId
public long serverId() -
sslMode
-
sslModeEnabled
public boolean sslModeEnabled() -
bufferSizeForStreamingChangeEventSource
public int bufferSizeForStreamingChangeEventSource() -
gtidSourceFilter
Get the predicate function that will returntrue
if a GTID source is to be included, orfalse
if a GTID source is to be excluded.- Returns:
- the GTID source predicate function; never null
-
includeSchemaChangeRecords
public boolean includeSchemaChangeRecords() -
includeSqlQuery
public boolean includeSqlQuery() -
rowCountForLargeTable
public long rowCountForLargeTable() -
getHistoryRecordComparator
- Specified by:
getHistoryRecordComparator
in classHistorizedRelationalDatabaseConnectorConfig
-
isBuiltInDatabase
-
isNotBuiltInTable
-
getDdlFilter
-
legacy
boolean legacy() -
isReadOnlyConnection
public boolean isReadOnlyConnection() -
useGlobalLock
boolean useGlobalLock()Intended for testing only -
getSourceInfoStructMaker
- Overrides:
getSourceInfoStructMaker
in classCommonConnectorConfig
-