Package io.smallrye.reactive.messaging.kafka.commit

Class KafkaThrottledLatestProcessedCommit

java.lang.Object
io.smallrye.reactive.messaging.kafka.commit.ContextHolder
io.smallrye.reactive.messaging.kafka.commit.KafkaThrottledLatestProcessedCommit
All Implemented Interfaces:
KafkaCommitHandler
public class KafkaThrottledLatestProcessedCommit extends ContextHolder implements KafkaCommitHandler
Will keep track of received messages and commit to the next offset after the latest ACKed message in sequence. Will commit periodically as defined by `auto.commit.interval.ms` (default: 5000)

This strategy mimics the behavior of the kafka consumer when `enable.auto.commit` is `true`.

The connector will be marked as unhealthy in the presence of any received record that has gone too long without being processed as defined by `throttled.unprocessed-record-max-age.ms` (default: 60000). If `throttled.unprocessed-record-max-age.ms` is set to less than or equal to 0 then will not perform any health check (this might lead to running out of memory).

This strategy guarantees at-least-once delivery even if the channel performs asynchronous processing.

To use set `commit-strategy` to `throttled`.

  • Method Details

    • partitionsAssigned

      public void partitionsAssigned(Collection<org.apache.kafka.common.TopicPartition> partitions)
      New partitions are assigned. This method is called from the Kafka poll thread.
      Specified by:
      partitionsAssigned in interface KafkaCommitHandler
      Parameters:
      partitions - the list of partitions that are now assigned to the consumer (may include partitions previously assigned to the consumer)

    • partitionsRevoked

      public void partitionsRevoked(Collection<org.apache.kafka.common.TopicPartition> partitions)
      Revoked partitions. This method is called from the Kafka pool thread.
      Specified by:
      partitionsRevoked in interface KafkaCommitHandler
      Parameters:
      partitions - The list of partitions that were assigned to the consumer and now need to be revoked (may not include all currently assigned partitions).

    • received

      public <K, V> io.smallrye.mutiny.Uni<IncomingKafkaRecord<K,V>> received(IncomingKafkaRecord<K,V> record)
      Received a new record from Kafka. This method is called from a Vert.x event loop.
      Specified by:
      received in interface KafkaCommitHandler
      Type Parameters:
      K - the key
      V - the value
      Parameters:
      record - the record
      Returns:
      the record emitted once everything has been done

    • handle

      public <K, V> io.smallrye.mutiny.Uni<Void> handle(IncomingKafkaRecord<K,V> record)
      A message has been acknowledged. This method is NOT necessarily called on an event loop.
      Specified by:
      handle in interface KafkaCommitHandler
      Type Parameters:
      K - the key
      V - the value
      Parameters:
      record - the record
      Returns:
      a completion stage indicating when the commit complete

    • terminate

      public void terminate(boolean graceful)
      Description copied from interface: KafkaCommitHandler
      Called on channel shutdown
      Specified by:
      terminate in interface KafkaCommitHandler
      Parameters:
      graceful - true if it is a graceful shutdown