Package io.debezium.connector.mysql.legacy

Class AbstractReader

  • All Implemented Interfaces:
    Reader
    Direct Known Subclasses:
    BinlogReader, SnapshotReader
    public abstract class AbstractReader
extends Object
implements Reader
    A component that performs a snapshot of a MySQL server, and records the schema changes in MySqlSchema.
    Author:
    Randall Hauch

    • Field Detail

      • logger

        protected final org.slf4j.Logger logger

      • name

        private final String name

      • records

        private final BlockingQueue<org.apache.kafka.connect.source.SourceRecord> records

      • failure

        private final AtomicReference<org.apache.kafka.connect.errors.ConnectException> failure

      • failureException

        private org.apache.kafka.connect.errors.ConnectException failureException

      • maxBatchSize

        private final int maxBatchSize

      • metronome

        private final Metronome metronome

      • pollInterval

        private final Duration pollInterval

    • Constructor Detail

      • AbstractReader

        public AbstractReader​(String name,
                      MySqlTaskContext context,
                      HaltingPredicate acceptAndContinue)
        Create a snapshot reader.
        Parameters:
        name - the name of the reader
        context - the task context in which this reader is running; may not be null
        acceptAndContinue - a predicate that returns true if the tested SourceRecord should be accepted and false if the record and all subsequent records should be ignored. The reader will stop accepting records once enqueueRecord(SourceRecord) is called with a record that tests as false. Can be null. If null, all records will be accepted.

    • Method Detail

      • name

        public String name()
        Description copied from interface: Reader
        Get the name of this reader.
        Specified by:
        name in interface Reader
        Returns:
        the reader's name; never null

      • uponCompletion

        public void uponCompletion​(Runnable handler)
        Description copied from interface: Reader
        Set the function that should be called when this reader transitions from the Reader.State.STOPPING to Reader.State.STOPPED state, which is after all generated records have been consumed via the poll method.

        This method should only be called while the reader is in the Reader.State.STOPPED state.

        Specified by:
        uponCompletion in interface Reader
        Parameters:
        handler - the function; may not be null

      • initialize

        public final void initialize()
        Description copied from interface: Reader
        Perform any initialization of the reader before being started. This method should be called exactly once before Reader.start() is called, and it should block until all initialization is completed.
        Specified by:
        initialize in interface Reader

      • destroy

        public final void destroy()
        Description copied from interface: Reader
        After the reader has stopped, there may still be some resources we want left available until the connector task is destroyed. This method is used to clean up those remaining resources upon shutdown. This method is effectively the opposite of Reader.initialize(), performing any de-initialization of the reader entity before shutdown. This method should be called exactly once after Reader.stop() is called, and it should block until all de-initialization is completed.
        Specified by:
        destroy in interface Reader

      • start

        public void start()
        Description copied from interface: Reader
        Start the reader and return immediately. Once started, the SourceRecords generated by the reader can be obtained by periodically calling Reader.poll() until that method returns null.

        This method does nothing if it is already running.

        Specified by:
        start in interface Reader

      • doInitialize

        protected void doInitialize()
        The reader has been requested to initialize resources prior to starting. This should only be called once before doStart().

      • doDestroy

        protected void doDestroy()
        The reader has been requested to de-initialize resources after stopping. This should only be called once after doStop().

      • doStart

        protected abstract void doStart()
        The reader has been requested to start, so initialize any un-initialized resources required by the reader.

      • doStop

        protected abstract void doStop()
        The reader has been requested to stop, so perform any work required to stop the reader's resources that were previously started.

        This method is always called when stop() is called, and the first time isRunning() will return true the first time and false for any subsequent calls.

      • doCleanup

        protected abstract void doCleanup()
        The reader has completed all processing and all enqueued records have been consumed, so this reader should clean up any resources that might remain.

      • completeSuccessfully

        protected void completeSuccessfully()
        Call this method only when the reader has successfully completed all of its work, signaling that subsequent calls to poll() should forever return null and that this reader should transition from Reader.State.STOPPING to Reader.State.STOPPED.

      • failed

        protected void failed​(Throwable error)
        Call this method only when the reader has failed, that a subsequent call to poll() should throw this error, and that doCleanup() can be called at any time.
        Parameters:
        error - the error that resulted in the failure; should not be null

      • failed

        protected void failed​(Throwable error,
                      String msg)
        Call this method only when the reader has failed, that a subsequent call to poll() should throw this error, and that doCleanup() can be called at any time.
        Parameters:
        error - the error that resulted in the failure; should not be null
        msg - the error message; may not be null

      • wrap

        protected org.apache.kafka.connect.errors.ConnectException wrap​(Throwable error)
        Wraps the specified exception in a ConnectException, ensuring that all useful state is captured inside the new exception's message.
        Parameters:
        error - the exception; may not be null
        Returns:
        the wrapped Kafka Connect exception

      • state

        public Reader.State state()
        Description copied from interface: Reader
        Get the current state of this reader.
        Specified by:
        state in interface Reader
        Returns:
        the state; never null

      • isRunning

        protected boolean isRunning()

      • poll

        public List<org.apache.kafka.connect.source.SourceRecord> poll()
                                                        throws InterruptedException
        Description copied from interface: Reader
        Poll for the next batch of source records. This method returns null only when all records generated by this reader have been processed, following the natural or explicit stopping of this reader. Note that this method may block if no additional records are available but the reader may produce more, thus callers should call this method continually until this method returns null.
        Specified by:
        poll in interface Reader
        Returns:
        the list of source records that may or may not be empty; or null when there will be no more records because the reader has completely Reader.State.STOPPED.
        Throws:
        InterruptedException - if this thread is interrupted while waiting for more records

      • cleanupResources

        protected void cleanupResources()
        This method is normally called by poll() when there this reader finishes normally and all generated records are consumed prior to being stopped. However, if this reader is explicitly stopped while still working, then subclasses should call this method when they have completed all of their shutdown work.

      • pollComplete

        protected void pollComplete​(List<org.apache.kafka.connect.source.SourceRecord> batch)
        Method called when poll() completes sending a non-zero-sized batch of records.
        Parameters:
        batch - the batch of records being recorded

      • enqueueRecord

        protected void enqueueRecord​(org.apache.kafka.connect.source.SourceRecord record)
                      throws InterruptedException
        Enqueue a record so that it can be obtained when this reader is polled. This method will block if the queue is full.
        Parameters:
        record - the record to be enqueued
        Throws:
        InterruptedException - if interrupted while waiting for the queue to have room for this record