Package com.markosindustries.distroboy.kafka

Class LatestKafkaOffsetSpec

java.lang.Object
com.markosindustries.distroboy.kafka.LatestKafkaOffsetSpec
All Implemented Interfaces:
KafkaOffsetSpec
public class LatestKafkaOffsetSpec
extends java.lang.Object
implements KafkaOffsetSpec
Load the latest offsets from Kafka for each partition. WARNING: for topics being actively written to, this may generate an inconsistent view from each node in the distroboy cluster. It is often better to use a timestamp in the past to ensure all nodes see the same time window of data.

  • Constructor Summary

    Constructors
    Constructor Description
    LatestKafkaOffsetSpec()  

  • Method Summary

    Modifier and Type Method Description
    <K,​ V> java.util.Map<org.apache.kafka.common.TopicPartition,​java.lang.Long> getOffsets​(org.apache.kafka.clients.consumer.Consumer<K,​V> kafkaConsumer, java.util.Collection<org.apache.kafka.common.TopicPartition> topicPartitions)
    Load the offsets for a given set of topicPartitions

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • LatestKafkaOffsetSpec

      public LatestKafkaOffsetSpec()

  • Method Details

    • getOffsets

      public <K,​ V> java.util.Map<org.apache.kafka.common.TopicPartition,​java.lang.Long> getOffsets​(org.apache.kafka.clients.consumer.Consumer<K,​V> kafkaConsumer, java.util.Collection<org.apache.kafka.common.TopicPartition> topicPartitions)
      Description copied from interface: KafkaOffsetSpec
      Load the offsets for a given set of topicPartitions
      Specified by:
      getOffsets in interface KafkaOffsetSpec
      Type Parameters:
      K - The type of keys the KafkaConsumer will deserialise
      V - The type of values the KafkaConsumer will deserialise
      Parameters:
      kafkaConsumer - A KafkaConsumer to communicate with Kafka via
      topicPartitions - The set of TopicPartitions to load offsets for
      Returns:
      A map from each TopicPartition specified to it's offset according to this Spec.