Package io.debezium.connector.mysql

Class RowDeserializers

java.lang.Object
io.debezium.connector.mysql.RowDeserializers
public class RowDeserializers extends Object
Custom deserializers for the MySQL Binlog Client library.

A few of the MySQL Binlog Client row deserializers convert MySQL raw row data into Date, Time, and Timestamp values using Calendar instances (and thus implicitly use the local timezone to calculate the milliseconds past epoch. Rather than perform this conversion, we prefer to convert the raw MySQL row values directly into LocalDate, LocalTime, LocalDateTime, and OffsetDateTime.

Unfortunately, all of the methods used to deserialize individual values are defined on the AbstractRowsEventDataDeserializer base class, and inherited by the DeleteRowsEventDataDeserializer, UpdateRowsEventDataDeserializer, and WriteRowsEventDataDeserializer subclasses. Since we can't provide a new base class, the simplest way to override these methods is to subclass each of these 3 subclasses and override the methods on all 3 classes. It's ugly, but it works.

See the MySQL Date Time documentation.

Author:
Randall Hauch

  • Field Details

    • LOGGER

      private static final org.slf4j.Logger LOGGER

    • MASK_10_BITS

      private static final int MASK_10_BITS
      See Also:

    • MASK_6_BITS

      private static final int MASK_6_BITS
      See Also:

  • Constructor Details

    • RowDeserializers

      private RowDeserializers()

  • Method Details

    • deserializeString

      protected static Serializable deserializeString(int length, com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream) throws IOException
      Converts a MySQL string to a byte[].
      Parameters:
      length - the number of bytes used to store the length of the string
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the byte[] object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeVarString

      protected static Serializable deserializeVarString(int meta, com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream) throws IOException
      Converts a MySQL string to a byte[].
      Parameters:
      meta - the meta value containing the number of bytes in the length field
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the byte[] object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeDate

      protected static Serializable deserializeDate(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream, CommonConnectorConfig.EventProcessingFailureHandlingMode eventProcessingFailureHandlingMode) throws IOException
      Converts a MySQL DATE value to a LocalDate.

      This method treats all zero values for DATE columns as NULL, since they cannot be accurately represented as valid LocalDate objects.

      Parameters:
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the LocalDate object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeTime

      protected static Serializable deserializeTime(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream) throws IOException
      Converts a MySQL TIME value without fractional seconds to a Duration.
      Parameters:
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the LocalTime object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeTimeV2

      protected static Serializable deserializeTimeV2(int meta, com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream) throws IOException
      Converts a MySQL TIME value with fractional seconds to a Duration.
      Parameters:
      meta - the meta value containing the fractional second precision, or fsp
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the Duration object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeDatetime

      protected static Serializable deserializeDatetime(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream, CommonConnectorConfig.EventProcessingFailureHandlingMode eventProcessingFailureHandlingMode) throws IOException
      Converts a MySQL DATETIME value without fractional seconds to a LocalDateTime.

      This method treats all zero values for DATETIME columns as NULL, since they cannot be accurately represented as valid LocalDateTime objects.

      Parameters:
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the LocalDateTime object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeDatetimeV2

      protected static Serializable deserializeDatetimeV2(int meta, com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream, CommonConnectorConfig.EventProcessingFailureHandlingMode eventProcessingFailureHandlingMode) throws IOException
      Converts a MySQL DATETIME value with fractional seconds to a LocalDateTime.

      This method treats all zero values for DATETIME columns as NULL, since they cannot be accurately represented as valid LocalDateTime objects.

      Parameters:
      meta - the meta value containing the fractional second precision, or fsp
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the LocalDateTime object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeTimestamp

      protected static Serializable deserializeTimestamp(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream) throws IOException
      Converts a MySQL TIMESTAMP value without fractional seconds to a OffsetDateTime. MySQL stores the TIMESTAMP values as seconds past epoch in UTC, but the resulting OffsetDateTime will be in the local timezone.
      Parameters:
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the OffsetDateTime object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeTimestampV2

      protected static Serializable deserializeTimestampV2(int meta, com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream) throws IOException
      Converts a MySQL TIMESTAMP value with fractional seconds to a OffsetDateTime. MySQL stores the TIMESTAMP values as seconds + fractional seconds past epoch in UTC, but the resulting OffsetDateTime will be in the local timezone.
      Parameters:
      meta - the meta value containing the fractional second precision, or fsp
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the OffsetDateTime object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeYear

      protected static Serializable deserializeYear(com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream) throws IOException
      Converts a MySQL YEAR value to a Year object.
      Parameters:
      inputStream - the binary stream containing the raw binlog event data for the value
      Returns:
      the Year object
      Throws:
      IOException - if there is an error reading from the binlog event data

    • split

      protected static int[] split(long value, int divider, int length)
      Split the integer into multiple integers.

      We can't use/access the private split method in the AbstractRowsEventDataDeserializer class, so we replicate it here. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

      Parameters:
      value - the long value
      divider - the value used to separate the individual values (e.g., 10 to separate each digit into a separate value, 100 to separate each pair of digits into a separate value, 1000 to separate each 3 digits into a separate value, etc.)
      length - the expected length of the integer array
      Returns:
      the integer values

    • bigEndianLong

      protected static long bigEndianLong(byte[] bytes, int offset, int length)
      Read a big-endian long value.

      We can't use/access the private bigEndianLong method in the AbstractRowsEventDataDeserializer class, so we replicate it here. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

      Parameters:
      bytes - the bytes containing the big-endian representation of the value
      offset - the offset within the bytes byte array where the value starts
      length - the length of the byte representation within the bytes byte array
      Returns:
      the long value

    • bitSlice

      protected static int bitSlice(long value, int bitOffset, int numberOfBits, int payloadSize)
      Slice an integer out of a portion of long value.

      We can't use/access the private bitSlice method in the AbstractRowsEventDataDeserializer class, so we replicate it here. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

      Parameters:
      value - the long containing the integer encoded within it
      bitOffset - the number of bits where the integer value starts
      numberOfBits - the number of bits in the integer value
      payloadSize - the total number of bits used in the value
      Returns:
      the integer value

    • deserializeFractionalSecondsInNanos

      protected static int deserializeFractionalSecondsInNanos(int fsp, com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream) throws IOException
      Read the binary input stream to obtain the number of nanoseconds given the fractional seconds precision, or fsp.

      We can't use/access the deserializeFractionalSeconds method in the AbstractRowsEventDataDeserializer class, so we replicate it here with modifications to support nanoseconds rather than microseconds. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

      Parameters:
      fsp - the fractional seconds precision describing the number of digits precision used to store the fractional seconds (e.g., 1 for storing tenths of a second, 2 for storing hundredths, 3 for storing milliseconds, etc.)
      inputStream - the binary data stream
      Returns:
      the number of nanoseconds
      Throws:
      IOException - if there is an error reading from the binlog event data

    • deserializeFractionalSecondsInNanosNegative

      protected static int deserializeFractionalSecondsInNanosNegative(int fsp, com.github.shyiko.mysql.binlog.io.ByteArrayInputStream inputStream) throws IOException
      Read the binary input stream to obtain the number of nanoseconds given the fractional seconds precision, or fsp.

      We can't use/access the deserializeFractionalSeconds method in the AbstractRowsEventDataDeserializer class, so we replicate it here with modifications to support nanoseconds rather than microseconds and negative values. Note the original is licensed under the same Apache Software License 2.0 as Debezium.

      Parameters:
      fsp - the fractional seconds precision describing the number of digits precision used to store the fractional seconds (e.g., 1 for storing tenths of a second, 2 for storing hundredths, 3 for storing milliseconds, etc.)
      inputStream - the binary data stream
      Returns:
      the number of nanoseconds
      Throws:
      IOException - if there is an error reading from the binlog event data

    • handleException

      private static Serializable handleException(CommonConnectorConfig.EventProcessingFailureHandlingMode eventProcessingFailureHandlingMode, String columnType, Exception e, Serializable defaultValue)