Package org.sdase.commons.server.kafka.consumer.strategies

Class MessageListenerStrategy<K,V>

java.lang.Object
org.sdase.commons.server.kafka.consumer.strategies.MessageListenerStrategy<K,V>
Type Parameters:
K - key object type
V - value object type
Direct Known Subclasses:
AutocommitMLS, RetryProcessingErrorMLS
public abstract class MessageListenerStrategy<K,V> extends Object
Base for MessageListener strategies. A strategy defies the way how records are handled and how the consumer interacts with the kafka broker, i.e. the commit handling.

A strategy might support the usage of a @MessageHandler that encapsulates the business logic

  • Constructor Details

    • MessageListenerStrategy

      public MessageListenerStrategy()

  • Method Details

    • init

      public void init(Set<String> metadataFields)
      Parameters:
      metadataFields - the configured MetadataContext.metadataFields()

    • processRecords

      public abstract void processRecords(org.apache.kafka.clients.consumer.ConsumerRecords<K,V> records, org.apache.kafka.clients.consumer.KafkaConsumer<K,V> consumer)
      Implementation of processing and commit logic during poll loop of MessageListener.

      The strategy should mark each record that was successfully processed by calling addOffsetToCommitOnClose(org.apache.kafka.clients.consumer.ConsumerRecord<K, V>) to commit the current offset in case the application shuts down. 

       SimpleTimer timer = new SimpleTimer();
 handler.handle(record);
 addOffsetToCommitOnClose(record);

 // Prometheus
 double elapsedSeconds = timer.elapsedSeconds();
 consumerProcessedMsgHistogram.observe(elapsedSeconds, consumerName, record.topic());
      Parameters:
      records - consumer records that should be processed in this poll loop.
      consumer - the consumer to communicate with Kafka

    • messageHandlerContextFor

      protected MessageHandlerContextCloseable messageHandlerContextFor(org.apache.kafka.clients.consumer.ConsumerRecord<K,V> consumerRecord)
      Creates a context for the current Thread to handle a message with appropriate information in ThreadLocals.

      Example usage (notice the nested "try" to keep the context for error handling): 

         try (var ignored = messageHandlerContextFor(consumerRecord)) {
     try {
       messageHandler.handle(consumerRecord);
     } catch (Exception e) {
       errorHandler.handleError(consumerRecord, e, consumer);
     }
   }

      The context will be set up as follows:

      1. Sets the MetadataContext for the current thread to handle the given consumerContext with the desired context. The context information is based on the ConsumerRecord.headers() considering all configured metadata fields.

        This method should be used as try-with-resources around the message handler to automatically reset the context after processing is done.

      Parameters:
      consumerRecord - the consumer record that will be handled
      Returns:
      a Closeable that will reset the created contexts to the previous state on MessageHandlerContextCloseable.close()

    • resetOffsetsToCommitOnClose

      public void resetOffsetsToCommitOnClose()

    • addOffsetToCommitOnClose

      protected void addOffsetToCommitOnClose(org.apache.kafka.clients.consumer.ConsumerRecord<K,V> consumerRecord)

    • commitOnClose

      public void commitOnClose(org.apache.kafka.clients.consumer.KafkaConsumer<K,V> consumer)
      Invoked if the listener shuts down to eventually commit or not commit messages
      Parameters:
      consumer - the consumer to communicate with Kafka

    • verifyConsumerConfig

      public abstract void verifyConsumerConfig(Map<String,String> config)

    • forcedConfigToApply

      public Map<String,String> forcedConfigToApply()
      Configuration properties that are applied to the ConsumerConfig of a consumer using this strategy. If these properties are absent in the configuration or configured different, the properties returned by this method must be applied. verifyConsumerConfig(Map) may still verify that the forced config is correctly applied or may require further conditions the user has to set intentionally.
      Returns:
      a map with configuration properties that must be applied to ConsumerConfig.setConfig(Map) to make the strategy work as expected