KafkaConsume

trait KafkaConsume[F <: ([_$1] =>> Any), K, V]
class Object
trait Matchable
class Any
class KafkaConsumer[F, K, V]

Value members

Methods

Alias for partitionedStream.parJoinUnbounded.
See partitionedStream for more information.
Note
you have to first use subscribe to subscribe the consumer
before using this Stream. If you forgot to subscribe, there
will be a NotSubscribedException raised in the Stream.
def partitionedStream: Stream[F, Stream[F, CommittableConsumerRecord[F, K, V]]]
Stream where the elements themselves are Streams which continually
request records for a single partition. These Streams will have to be
processed in parallel, using parJoin or parJoinUnbounded. Note that
when using parJoin(n) and n is smaller than the number of currently
assigned partitions, then there will be assigned partitions which won't
be processed. For that reason, prefer parJoinUnbounded and the actual
limit will be the number of assigned partitions.

If you do not want to process all partitions in parallel, then you
can use stream instead, where records for all partitions are in
a single Stream.
Note
you have to first use subscribe to subscribe the consumer
before using this Stream. If you forgot to subscribe, there
will be a NotSubscribedException raised in the Stream.
def partitionsMapStream: Stream[F, Map[TopicPartition, Stream[F, CommittableConsumerRecord[F, K, V]]]]
Stream where each element contains a current assignment. The current
assignment is the Map, where keys is a TopicPartition, and values are
streams with records for a particular TopicPartition.

New assignments will be received on each rebalance. On rebalance,
Kafka revoke all previously assigned partitions, and after that assigned
new partitions all at once. partitionsMapStream reflects this process
in a streaming manner.

Note, that partition streams for revoked partitions will
be closed after the new assignment comes.

This is the most generic Stream method. If you don't need such control,
consider using partitionedStream or stream methods.
They are both based on a partitionsMapStream.
See also
Note
you have to first use subscribe to subscribe the consumer
before using this Stream. If you forgot to subscribe, there
will be a NotSubscribedException raised in the Stream.
def stopConsuming: F[Unit]
Stops consuming new messages from Kafka.
This method could be used to implement a graceful shutdown.

This method has a few effects:
1. After this call no more data will be fetched from Kafka through the poll method.
2. All currently running streams will continue to run until all in-flight messages will be processed.
It means that streams will be completed when all fetched messages will be processed.

If some of the stream methods will be called after stopConsuming call,
these methods will return empty streams.

More than one call of stopConsuming will have no effect.