Package io.debezium.connector.mysql
Class MySqlDatabaseSchema
- java.lang.Object
-
- io.debezium.relational.RelationalDatabaseSchema
-
- io.debezium.relational.HistorizedRelationalDatabaseSchema
-
- io.debezium.connector.mysql.MySqlDatabaseSchema
-
- All Implemented Interfaces:
DatabaseSchema<TableId>
,HistorizedDatabaseSchema<TableId>
@NotThreadSafe public class MySqlDatabaseSchema extends HistorizedRelationalDatabaseSchema
Component that records the schema history for databases hosted by a MySQL database server. The schema information includes thetable definitions
and the Kafka ConnectSchema
s for each table, where theSchema
excludes any columns that have beenspecified
in the configuration.The history is changed by
applying DDL statements
, and every change ispersisted
as defined in the suppliedMySQL connector configuration
. This component can be reconstructed (e.g., on connector restart) and the historyloaded
from persisted storage.Note that when
applying DDL statements
, the caller is able to supply aconsumer function
that will be called with the DDL statements and the database to which they apply, grouped by database names. However, these will only be called based when the databases are included by the database filters defined in theMySQL connector configuration
.- Author:
- Randall Hauch
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface io.debezium.schema.HistorizedDatabaseSchema
HistorizedDatabaseSchema.SchemaChangeEventConsumer
-
-
Field Summary
Fields Modifier and Type Field Description private DdlChanges
ddlChanges
private DdlParser
ddlParser
private RelationalTableFilters
filters
private Set<String>
ignoredQueryStatements
private static org.slf4j.Logger
LOGGER
private boolean
storageInitialiationExecuted
private Map<Long,TableId>
tableIdsByTableNumber
-
Fields inherited from class io.debezium.relational.HistorizedRelationalDatabaseSchema
databaseHistory
-
Fields inherited from interface io.debezium.schema.DatabaseSchema
NO_CAPTURED_DATA_COLLECTIONS_WARNING
-
-
Constructor Summary
Constructors Constructor Description MySqlDatabaseSchema(MySqlConnectorConfig connectorConfig, MySqlValueConverters valueConverter, TopicSelector<TableId> topicSelector, SchemaNameAdjuster schemaNameAdjuster, boolean tableIdCaseInsensitive)
Create a schema component given the suppliedMySQL connector configuration
.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description private boolean
acceptableDatabase(String databaseName)
protected void
appendCreateTableStatement(StringBuilder sb, Table table)
protected void
appendDropTableStatement(StringBuilder sb, TableId tableId)
void
applySchemaChange(SchemaChangeEvent schemaChange)
boolean
assignTableNumber(long tableNumber, TableId id)
Assign the given table number to the table with the specifiedtable ID
.String[]
capturedTablesAsStringArray()
Get all table names for all databases that are monitored whose events are captured by Debeziumvoid
clearTableMappings()
Clear all of the table mappings.private void
emitChangeEvent(MySqlOffsetContext offset, List<SchemaChangeEvent> schemaChangeEvents, String sanitizedDbName, DdlParserListener.Event event, TableId tableId, SchemaChangeEvent.SchemaChangeEventType type, boolean snapshot)
protected DdlParser
getDdlParser()
TableId
getTableId(long tableNumber)
Return the table id associated with MySQL-specific table number.private TableId
getTableId(DdlParserListener.Event event)
boolean
historyExists()
Return true if the database history entity existsvoid
initializeStorage()
boolean
isGlobalSetVariableStatement(String ddl, String databaseName)
boolean
isStorageInitializationExecuted()
private List<SchemaChangeEvent>
parseDdl(String ddlStatements, String databaseName, MySqlOffsetContext offset, Instant sourceTime, boolean snapshot)
List<SchemaChangeEvent>
parseSnapshotDdl(String ddlStatements, String databaseName, MySqlOffsetContext offset, Instant sourceTime)
List<SchemaChangeEvent>
parseStreamingDdl(String ddlStatements, String databaseName, MySqlOffsetContext offset, Instant sourceTime)
protected void
refreshSchemas()
Discard any currently-cached schemas and rebuild them using the filters.void
setSystemVariables(Map<String,String> variables)
Set the system variables on the DDL parser.boolean
storeOnlyCapturedTables()
SystemVariables
systemVariables()
Get the system variables as known by the DDL parser.-
Methods inherited from class io.debezium.relational.HistorizedRelationalDatabaseSchema
close, record, recover, tableInformationComplete
-
Methods inherited from class io.debezium.relational.RelationalDatabaseSchema
assureNonEmptySchema, buildAndRegisterSchema, clearSchemas, getTableFilter, refresh, refreshSchema, removeSchema, schemaFor, tableFor, tableIds, tables
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface io.debezium.schema.DatabaseSchema
assureNonEmptySchema, schemaFor
-
-
-
-
Field Detail
-
LOGGER
private static final org.slf4j.Logger LOGGER
-
ddlParser
private final DdlParser ddlParser
-
filters
private final RelationalTableFilters filters
-
ddlChanges
private final DdlChanges ddlChanges
-
storageInitialiationExecuted
private boolean storageInitialiationExecuted
-
-
Constructor Detail
-
MySqlDatabaseSchema
public MySqlDatabaseSchema(MySqlConnectorConfig connectorConfig, MySqlValueConverters valueConverter, TopicSelector<TableId> topicSelector, SchemaNameAdjuster schemaNameAdjuster, boolean tableIdCaseInsensitive)
Create a schema component given the suppliedMySQL connector configuration
. The DDL statements passed to the schema are parsed and a logical model of the database schema is created.
-
-
Method Detail
-
capturedTablesAsStringArray
public String[] capturedTablesAsStringArray()
Get all table names for all databases that are monitored whose events are captured by Debezium- Returns:
- the array with the table names
-
setSystemVariables
public void setSystemVariables(Map<String,String> variables)
Set the system variables on the DDL parser.- Parameters:
variables
- the system variables; may not be null but may be empty
-
systemVariables
public SystemVariables systemVariables()
Get the system variables as known by the DDL parser.- Returns:
- the system variables; never null
-
appendDropTableStatement
protected void appendDropTableStatement(StringBuilder sb, TableId tableId)
-
appendCreateTableStatement
protected void appendCreateTableStatement(StringBuilder sb, Table table)
-
refreshSchemas
protected void refreshSchemas()
Discard any currently-cached schemas and rebuild them using the filters.
-
isGlobalSetVariableStatement
public boolean isGlobalSetVariableStatement(String ddl, String databaseName)
-
applySchemaChange
public void applySchemaChange(SchemaChangeEvent schemaChange)
-
parseSnapshotDdl
public List<SchemaChangeEvent> parseSnapshotDdl(String ddlStatements, String databaseName, MySqlOffsetContext offset, Instant sourceTime)
-
parseStreamingDdl
public List<SchemaChangeEvent> parseStreamingDdl(String ddlStatements, String databaseName, MySqlOffsetContext offset, Instant sourceTime)
-
parseDdl
private List<SchemaChangeEvent> parseDdl(String ddlStatements, String databaseName, MySqlOffsetContext offset, Instant sourceTime, boolean snapshot)
-
emitChangeEvent
private void emitChangeEvent(MySqlOffsetContext offset, List<SchemaChangeEvent> schemaChangeEvents, String sanitizedDbName, DdlParserListener.Event event, TableId tableId, SchemaChangeEvent.SchemaChangeEventType type, boolean snapshot)
-
acceptableDatabase
private boolean acceptableDatabase(String databaseName)
-
getTableId
private TableId getTableId(DdlParserListener.Event event)
-
getDdlParser
protected DdlParser getDdlParser()
- Specified by:
getDdlParser
in classHistorizedRelationalDatabaseSchema
-
historyExists
public boolean historyExists()
Return true if the database history entity exists
-
storeOnlyCapturedTables
public boolean storeOnlyCapturedTables()
- Specified by:
storeOnlyCapturedTables
in interfaceHistorizedDatabaseSchema<TableId>
- Overrides:
storeOnlyCapturedTables
in classHistorizedRelationalDatabaseSchema
-
assignTableNumber
public boolean assignTableNumber(long tableNumber, TableId id)
Assign the given table number to the table with the specifiedtable ID
.- Parameters:
tableNumber
- the table number found in binlog eventsid
- the identifier for the corresponding table- Returns:
true
if the assignment was successful, orfalse
if the table is currently excluded in the connector's configuration
-
getTableId
public TableId getTableId(long tableNumber)
Return the table id associated with MySQL-specific table number.- Parameters:
tableNumber
-- Returns:
- the table id or null if not known
-
clearTableMappings
public void clearTableMappings()
Clear all of the table mappings. This should be done when the logs are rotated, since in that a different table numbering scheme will be used by all subsequent TABLE_MAP binlog events.
-
initializeStorage
public void initializeStorage()
- Specified by:
initializeStorage
in interfaceHistorizedDatabaseSchema<TableId>
- Overrides:
initializeStorage
in classHistorizedRelationalDatabaseSchema
-
isStorageInitializationExecuted
public boolean isStorageInitializationExecuted()
-
-