Class MySqlSchema
- java.lang.Object
-
- io.debezium.relational.RelationalDatabaseSchema
-
- io.debezium.connector.mysql.legacy.MySqlSchema
-
- All Implemented Interfaces:
DatabaseSchema<TableId>
@NotThreadSafe public class MySqlSchema extends RelationalDatabaseSchema
Component that records the schema history for databases hosted by a MySQL database server. The schema information includes thetable definitions
and the Kafka ConnectSchema
s for each table, where theSchema
excludes any columns that have beenspecified
in the configuration.The history is changed by
applying DDL statements
, and every change ispersisted
as defined in the suppliedMySQL connector configuration
. This component can be reconstructed (e.g., on connector restart) and the historyloaded
from persisted storage.Note that when
applying DDL statements
, the caller is able to supply aconsumer function
that will be called with the DDL statements and the database to which they apply, grouped by database names. However, these will only be called based when the databases are included by the database filters defined in theMySQL connector configuration
.- Author:
- Randall Hauch
-
-
Field Summary
Fields Modifier and Type Field Description private DatabaseHistory
dbHistory
private DdlChanges
ddlChanges
private DdlParser
ddlParser
private Filters
filters
private HistoryRecordComparator
historyComparator
private Set<String>
ignoredQueryStatements
private static org.slf4j.Logger
logger
private boolean
recoveredTables
private boolean
skipUnparseableDDL
private boolean
storeOnlyCapturedTablesDdl
-
Fields inherited from interface io.debezium.schema.DatabaseSchema
NO_CAPTURED_DATA_COLLECTIONS_WARNING
-
-
Constructor Summary
Constructors Constructor Description MySqlSchema(MySqlConnectorConfig configuration, Predicate<String> gtidFilter, boolean tableIdCaseInsensitive, TopicSelector<TableId> topicSelector, Filters tableFilters)
Create a schema component given the suppliedMySQL connector configuration
.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
appendCreateTableStatement(StringBuilder sb, Table table)
protected void
appendDropTableStatement(StringBuilder sb, TableId tableId)
boolean
applyDdl(SourceInfo source, String databaseName, String ddlStatements, DdlChanges.DatabaseStatementStringConsumer statementConsumer)
Apply the supplied DDL statements to this database schema and record the history.String[]
capturedTablesAsStringArray()
Get all table names for all databases that are monitored whose events are captured by DebeziumFilters
filters()
Get thedatabase and table filters
defined by the configuration.private static MySqlValueConverters
getValueConverters(MySqlConnectorConfig configuration)
HistoryRecordComparator
historyComparator()
boolean
historyExists()
Return true if the database history entity existsString
historyLocation()
Get the information about where the DDL statement history is recorded.void
intializeHistoryStorage()
Initialize permanent storage for database historyboolean
isGlobalSetVariableStatement(String ddl, String databaseName)
boolean
isStoreOnlyCapturedTablesDdl()
boolean
isTableCaptured(TableId id)
Decide whether events should be captured for a given tablevoid
loadHistory(SourceInfo startingPoint)
Load the database schema information using the previously-recorded history, and stop reading the history when the the history reaches the supplied starting point.void
refreshSchemas()
Discard any currently-cached schemas and rebuild them using the filters.void
setSystemVariables(Map<String,String> variables)
Set the system variables on the DDL parser.void
shutdown()
Stop recording history and release any resources acquired sincestart()
.void
start()
Start by acquiring resources needed to persist the database historySystemVariables
systemVariables()
Get the system variables as known by the DDL parser.boolean
tableInformationComplete()
-
Methods inherited from class io.debezium.relational.RelationalDatabaseSchema
assureNonEmptySchema, buildAndRegisterSchema, clearSchemas, close, getTableFilter, isHistorized, refresh, refreshSchema, removeSchema, schemaFor, tableFor, tableIds, tables
-
-
-
-
Field Detail
-
logger
private static final org.slf4j.Logger logger
-
ddlParser
private final DdlParser ddlParser
-
filters
private final Filters filters
-
dbHistory
private final DatabaseHistory dbHistory
-
ddlChanges
private final DdlChanges ddlChanges
-
historyComparator
private final HistoryRecordComparator historyComparator
-
skipUnparseableDDL
private final boolean skipUnparseableDDL
-
storeOnlyCapturedTablesDdl
private final boolean storeOnlyCapturedTablesDdl
-
recoveredTables
private boolean recoveredTables
-
-
Constructor Detail
-
MySqlSchema
public MySqlSchema(MySqlConnectorConfig configuration, Predicate<String> gtidFilter, boolean tableIdCaseInsensitive, TopicSelector<TableId> topicSelector, Filters tableFilters)
Create a schema component given the suppliedMySQL connector configuration
.- Parameters:
configuration
- the connector configuration, which is presumed to be validgtidFilter
- the predicate function that should be applied to GTID sets in database history, and which returnstrue
if a GTID source is to be included, orfalse
if a GTID source is to be excluded; may be null if not neededtableIdCaseInsensitive
- true if table lookup ignores letter case
-
-
Method Detail
-
getValueConverters
private static MySqlValueConverters getValueConverters(MySqlConnectorConfig configuration)
-
historyComparator
public HistoryRecordComparator historyComparator()
-
start
public void start()
Start by acquiring resources needed to persist the database history
-
shutdown
public void shutdown()
Stop recording history and release any resources acquired sincestart()
.
-
filters
public Filters filters()
Get thedatabase and table filters
defined by the configuration.- Returns:
- the filters; never null
-
capturedTablesAsStringArray
public String[] capturedTablesAsStringArray()
Get all table names for all databases that are monitored whose events are captured by Debezium- Returns:
- the array with the table names
-
isTableCaptured
public boolean isTableCaptured(TableId id)
Decide whether events should be captured for a given table- Parameters:
id
- the fully-qualified table identifier; may be null- Returns:
- true if events from the table are captured
-
historyLocation
public String historyLocation()
Get the information about where the DDL statement history is recorded.- Returns:
- the history description; never null
-
setSystemVariables
public void setSystemVariables(Map<String,String> variables)
Set the system variables on the DDL parser.- Parameters:
variables
- the system variables; may not be null but may be empty
-
systemVariables
public SystemVariables systemVariables()
Get the system variables as known by the DDL parser.- Returns:
- the system variables; never null
-
appendDropTableStatement
protected void appendDropTableStatement(StringBuilder sb, TableId tableId)
-
appendCreateTableStatement
protected void appendCreateTableStatement(StringBuilder sb, Table table)
-
loadHistory
public void loadHistory(SourceInfo startingPoint)
Load the database schema information using the previously-recorded history, and stop reading the history when the the history reaches the supplied starting point.- Parameters:
startingPoint
- the source information with the currentSourceInfo.partition()
andoffset
at which the database schemas are to reflect; may not be null
-
historyExists
public boolean historyExists()
Return true if the database history entity exists
-
intializeHistoryStorage
public void intializeHistoryStorage()
Initialize permanent storage for database history
-
refreshSchemas
public void refreshSchemas()
Discard any currently-cached schemas and rebuild them using the filters.
-
applyDdl
public boolean applyDdl(SourceInfo source, String databaseName, String ddlStatements, DdlChanges.DatabaseStatementStringConsumer statementConsumer)
Apply the supplied DDL statements to this database schema and record the history. If astatementConsumer
is supplied, then call it for each sub-sequence of the DDL statements that all apply to the same database.Typically DDL statements are applied using a connection to a single database, and unless the statements use fully-qualified names, the DDL statements apply to this database.
- Parameters:
source
- the currentSourceInfo.partition()
andoffset
at which these changes are found; may not be nulldatabaseName
- the name of the default database under which these statements are applied; may not be nullddlStatements
- the;
-separated DDL statements; may be null or emptystatementConsumer
- the consumer that should be called with each sub-sequence of DDL statements that apply to a single database; may be null if no action is to be performed with the changes- Returns:
true
if changes were made to the database schema, orfalse
if the DDL statements had no effect on the database schema
-
isGlobalSetVariableStatement
public boolean isGlobalSetVariableStatement(String ddl, String databaseName)
-
isStoreOnlyCapturedTablesDdl
public boolean isStoreOnlyCapturedTablesDdl()
- Returns:
- true if only captured tables should be stored in database history, false if all tables should be stored
-
tableInformationComplete
public boolean tableInformationComplete()
- Specified by:
tableInformationComplete
in interfaceDatabaseSchema<TableId>
- Overrides:
tableInformationComplete
in classRelationalDatabaseSchema
-
-