zio.kafka.consumer.fetch

Members list

Type members

Classlikes

A fetch strategy determined which stream are allowed to fetch data in the next poll.

A fetch strategy determined which stream are allowed to fetch data in the next poll.

WARNING: this is an EXPERIMENTAL API and may change in an incompatible way without notice in any zio-kafka version.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
final case class QueueSizeBasedFetchStrategy(partitionPreFetchBufferLimit: Int) extends FetchStrategy

A fetch strategy that allows a stream to fetch data when its queue size is at or below partitionPreFetchBufferLimit.

A fetch strategy that allows a stream to fetch data when its queue size is at or below partitionPreFetchBufferLimit.

Value parameters

partitionPreFetchBufferLimit

The queue size at or below which more records are fetched and buffered (per partition). This buffer improves throughput and supports varying downstream message processing time, while maintaining some backpressure. Large values effectively disable backpressure at the cost of high memory usage, low values will effectively disable prefetching in favor of low memory consumption. The number of records that is fetched on every poll is controlled by the max.poll.records setting, the number of records fetched for every partition is somewhere between 0 and max.poll.records. The default value for this parameter is 2 * the default max.poll.records of 500, rounded to the nearest power of 2. The value 0 disables pre-fetching.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all