Package io.smallrye.reactive.messaging.kafka.commit

Class KafkaCheckpointCommit

java.lang.Object

io.smallrye.reactive.messaging.kafka.commit.ContextHolder

io.smallrye.reactive.messaging.kafka.commit.KafkaCheckpointCommit

All Implemented Interfaces:

KafkaCommitHandler

Direct Known Subclasses:

KafkaFileCheckpointCommit

@Experimental("Experimental API")
public abstract class KafkaCheckpointCommit
extends ContextHolder
implements KafkaCommitHandler

Abstract commit handler for checkpointing processing state on a state store Instead of committing topic-partition offsets back to Kafka, checkpointing commit handlers persist and restore offsets on an external store. It associates a ProcessingState with a topic-partition offset, and lets the processing resume from the checkpointed state. This abstract implementation holds a local map of ProcessingState per topic-partition, and ensures it is accessed on the captured Vert.x context. Concrete implementations provide fetchProcessingState(TopicPartition) and persistProcessingState(TopicPartition, ProcessingState) depending on the state store.

Nested Class Summary

Nested classes/interfaces inherited from interface io.smallrye.reactive.messaging.kafka.commit.KafkaCommitHandler

KafkaCommitHandler.Factory, KafkaCommitHandler.Strategy

Field Summary

Fields

Modifier and Type	Field	Description
protected KafkaConnectorIncomingConfiguration	config
protected KafkaConsumer<?,?>	consumer
protected KafkaLogging	log
protected Map<org.apache.kafka.common.TopicPartition,ProcessingState<?>>	processingStateMap
protected BiConsumer<Throwable,Boolean>	reportFailure

Fields inherited from class io.smallrye.reactive.messaging.kafka.commit.ContextHolder

context, vertx

Constructor Summary

Constructors

Constructor	Description
KafkaCheckpointCommit(io.vertx.mutiny.core.Vertx vertx, KafkaConnectorIncomingConfiguration config, KafkaConsumer<?,?> consumer, BiConsumer<Throwable,Boolean> reportFailure, int defaultTimeout)

Method Summary

Modifier and Type	Method	Description
protected abstract io.smallrye.mutiny.Uni<ProcessingState<?>>	fetchProcessingState(org.apache.kafka.common.TopicPartition partition)
<K,V> io.smallrye.mutiny.Uni<Void>	handle(IncomingKafkaRecord<K,V> record)	Handle message acknowledgment
void	partitionsAssigned(Collection<org.apache.kafka.common.TopicPartition> partitions)	Called on partitions assigned on Kafka rebalance listener
void	partitionsRevoked(Collection<org.apache.kafka.common.TopicPartition> partitions)	Called on partitions revokd on Kafka rebalance listener
protected abstract io.smallrye.mutiny.Uni<Void>	persistProcessingState(org.apache.kafka.common.TopicPartition partition, ProcessingState<?> state)
<K,V> io.smallrye.mutiny.Uni<IncomingKafkaRecord<K,V>>	received(IncomingKafkaRecord<K,V> record)	Called on message received but before calling the processing function.
void	terminate(boolean graceful)	Called on channel shutdown

Methods inherited from class io.smallrye.reactive.messaging.kafka.commit.ContextHolder

capture, capture, getContext, getTimeoutInMillis, runOnContext, runOnContextAndAwait

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected KafkaLogging log

processingStateMap

protected final Map<org.apache.kafka.common.TopicPartition,ProcessingState<?>> processingStateMap

config

protected final KafkaConnectorIncomingConfiguration config

consumer

protected final KafkaConsumer<?,?> consumer

reportFailure

protected final BiConsumer<Throwable,Boolean> reportFailure

Constructor Detail

KafkaCheckpointCommit

public KafkaCheckpointCommit(io.vertx.mutiny.core.Vertx vertx,
                             KafkaConnectorIncomingConfiguration config,
                             KafkaConsumer<?,?> consumer,
                             BiConsumer<Throwable,Boolean> reportFailure,
                             int defaultTimeout)

Method Detail

received

public <K,V> io.smallrye.mutiny.Uni<IncomingKafkaRecord<K,V>> received(IncomingKafkaRecord<K,V> record)

Description copied from interface: KafkaCommitHandler

Called on message received but before calling the processing function. Returned Uni allows chaining asynchronous actions before message processing.

Specified by:

received in interface KafkaCommitHandler

Type Parameters:

K - type of record key

V - type of record value

Parameters:

record - incoming Kafka record

Returns:

the Uni yielding the received record

handle

public <K,V> io.smallrye.mutiny.Uni<Void> handle(IncomingKafkaRecord<K,V> record)

Description copied from interface: KafkaCommitHandler

Handle message acknowledgment

Specified by:

handle in interface KafkaCommitHandler

Type Parameters:

K - type of record key

V - type of record value

Parameters:

record - incoming Kafka record

Returns:

a completion stage completed when the message is acknowledged.

terminate

public void terminate(boolean graceful)

Description copied from interface: KafkaCommitHandler

Called on channel shutdown

Specified by:

terminate in interface KafkaCommitHandler

Parameters:

graceful - true if it is a graceful shutdown

partitionsAssigned

public void partitionsAssigned(Collection<org.apache.kafka.common.TopicPartition> partitions)

Description copied from interface: KafkaCommitHandler

Called on partitions assigned on Kafka rebalance listener

Specified by:

partitionsAssigned in interface KafkaCommitHandler

Parameters:

partitions - assigned partitions

partitionsRevoked

public void partitionsRevoked(Collection<org.apache.kafka.common.TopicPartition> partitions)

Description copied from interface: KafkaCommitHandler

Called on partitions revokd on Kafka rebalance listener

Specified by:

partitionsRevoked in interface KafkaCommitHandler

Parameters:

partitions - revoked partitions

fetchProcessingState

protected abstract io.smallrye.mutiny.Uni<ProcessingState<?>> fetchProcessingState(org.apache.kafka.common.TopicPartition partition)

persistProcessingState

protected abstract io.smallrye.mutiny.Uni<Void> persistProcessingState(org.apache.kafka.common.TopicPartition partition,
                                                                       ProcessingState<?> state)