zio.kafka.consumer.fetch

Members list

Type members

Classlikes

A fetch strategy determined which stream are allowed to fetch data in the next poll.

A fetch strategy determined which stream are allowed to fetch data in the next poll.

WARNING: this is an EXPERIMENTAL API and may change in an incompatible way without notice in any zio-kafka version.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
final case class QueueSizeBasedFetchStrategy(maxPartitionQueueSize: Int) extends FetchStrategy

A fetch strategy that allows a stream to fetch data when its queue size is below maxPartitionQueueSize.

A fetch strategy that allows a stream to fetch data when its queue size is below maxPartitionQueueSize.

Value parameters

maxPartitionQueueSize

Maximum number of records to be buffered per partition. This buffer improves throughput and supports varying downstream message processing time, while maintaining some backpressure. Large values effectively disable backpressure at the cost of high memory usage, low values will effectively disable prefetching in favour of low memory consumption. The number of records that is fetched on every poll is controlled by the max.poll.records setting, the number of records fetched for every partition is somewhere between 0 and max.poll.records. A value that is a power of 2 offers somewhat better queueing performance. The default value for this parameter is 2 * the default max.poll.records of 500, rounded to the nearest power of 2.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all