Producer

zio.kafka.producer.Producer
See theProducer companion object
trait Producer

Attributes

Companion
object
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Abstract methods

def flush: Task[Unit]

Flushes the producer's internal buffer. This will guarantee that all records currently buffered will be transmitted to the broker.

Flushes the producer's internal buffer. This will guarantee that all records currently buffered will be transmitted to the broker.

Attributes

Expose internal producer metrics

Expose internal producer metrics

Attributes

Get the partition metadata for the given topic

Get the partition metadata for the given topic

Attributes

Produces a single record and await broker acknowledgement.

Produces a single record and await broker acknowledgement.

Use produceAsync to avoid the round-trip-time penalty for each record.

Attributes

def produce[R, K, V](record: ProducerRecord[K, V], keySerializer: Serializer[R, K], valueSerializer: Serializer[R, V]): RIO[R, RecordMetadata]

Produces a single record and await broker acknowledgement.

Produces a single record and await broker acknowledgement.

Use produceAsync to avoid the round-trip-time penalty for each record.

Attributes

def produce[R, K, V](topic: String, key: K, value: V, keySerializer: Serializer[R, K], valueSerializer: Serializer[R, V]): RIO[R, RecordMetadata]

Produces a single record and await broker acknowledgement.

Produces a single record and await broker acknowledgement.

Use produceAsync to avoid the round-trip-time penalty for each record.

Attributes

Produces a single record. The effect returned from this method has two layers and describes the completion of two actions:

Produces a single record. The effect returned from this method has two layers and describes the completion of two actions:

  1. The outer layer describes the enqueueing of the record to the Producer's internal buffer.
  2. The inner layer describes receiving an acknowledgement from the broker for the transmission of the record.

It is usually recommended to not await the inner layer of every individual record, but enqueue a batch of records and await all of their acknowledgements at once. That amortizes the cost of sending requests to Kafka and increases throughput.

Attributes

def produceAsync[R, K, V](record: ProducerRecord[K, V], keySerializer: Serializer[R, K], valueSerializer: Serializer[R, V]): RIO[R, Task[RecordMetadata]]

Produces a single record. The effect returned from this method has two layers and describes the completion of two actions:

Produces a single record. The effect returned from this method has two layers and describes the completion of two actions:

  1. The outer layer describes the enqueueing of the record to the Producer's internal buffer.
  2. The inner layer describes receiving an acknowledgement from the broker for the transmission of the record.

It is usually recommended to not await the inner layer of every individual record, but enqueue a batch of records and await all of their acknowledgements at once. That amortizes the cost of sending requests to Kafka and increases throughput.

Attributes

def produceAsync[R, K, V](topic: String, key: K, value: V, keySerializer: Serializer[R, K], valueSerializer: Serializer[R, V]): RIO[R, Task[RecordMetadata]]

Produces a single record. The effect returned from this method has two layers and describes the completion of two actions:

Produces a single record. The effect returned from this method has two layers and describes the completion of two actions:

  1. The outer layer describes the enqueueing of the record to the Producer's internal buffer.
  2. The inner layer describes receiving an acknowledgement from the broker for the transmission of the record.

It is usually recommended to not await the inner layer of every individual record, but enqueue a batch of records and await all of their acknowledgements at once. That amortizes the cost of sending requests to Kafka and increases throughput.

Attributes

Produces a chunk of records.

Produces a chunk of records.

Use produceChunkAsync to avoid the round-trip-time penalty for each record.

When publishing any of the records fails, the whole batch fails even though some records might have been published. Use produceChunkAsyncWithFailures to get results per record.

Attributes

def produceChunk[R, K, V](records: Chunk[ProducerRecord[K, V]], keySerializer: Serializer[R, K], valueSerializer: Serializer[R, V]): RIO[R, Chunk[RecordMetadata]]

Produces a chunk of records.

Produces a chunk of records.

Use produceChunkAsync to avoid the round-trip-time penalty for each record.

When publishing any of the records fails, the whole batch fails even though some records might have been published. Use produceChunkAsyncWithFailures to get results per record.

Attributes

Produces a chunk of records. The effect returned from this method has two layers and describes the completion of two actions:

Produces a chunk of records. The effect returned from this method has two layers and describes the completion of two actions:

  1. The outer layer describes the enqueueing of all the records to the Producer's internal buffer.
  2. The inner layer describes receiving an acknowledgement from the broker for the transmission of the records.

It is possible that for chunks that exceed the producer's internal buffer size, the outer layer will also signal the transmission of part of the chunk. Regardless, awaiting the inner layer guarantees the transmission of the entire chunk.

When publishing any of the records fails, the whole batch fails even though some records might have been published. Use produceChunkAsyncWithFailures to get results per record.

Attributes

def produceChunkAsync[R, K, V](records: Chunk[ProducerRecord[K, V]], keySerializer: Serializer[R, K], valueSerializer: Serializer[R, V]): RIO[R, Task[Chunk[RecordMetadata]]]

Produces a chunk of records. The effect returned from this method has two layers and describes the completion of two actions:

Produces a chunk of records. The effect returned from this method has two layers and describes the completion of two actions:

  1. The outer layer describes the enqueueing of all the records to the Producer's internal buffer.
  2. The inner layer describes receiving an acknowledgement from the broker for the transmission of the records.

It is possible that for chunks that exceed the producer's internal buffer size, the outer layer will also signal the transmission of part of the chunk. Regardless, awaiting the inner layer guarantees the transmission of the entire chunk.

When publishing any of the records fails, the whole batch fails even though some records might have been published. Use produceChunkAsyncWithFailures to get results per record.

Attributes

Produces a chunk of records. The effect returned from this method has two layers and describes the completion of two actions:

Produces a chunk of records. The effect returned from this method has two layers and describes the completion of two actions:

  1. The outer layer describes the enqueueing of all the records to the Producer's internal buffer.
  2. The inner layer describes receiving an acknowledgement from the broker for the transmission of the records.

It is possible that for chunks that exceed the producer's internal buffer size, the outer layer will also signal the transmission of part of the chunk. Regardless, awaiting the inner layer guarantees the transmission of the entire chunk.

This variant of produceChunkAsync more accurately reflects that individual records within the Chunk can fail to publish, rather than the failure being at the level of the Chunk.

When attempt to send a record into buffer for publication fails, the following records in the chunk are not published. This is indicated with a Producer.PublishOmittedException.

This variant does not accept serializers as they may also fail independently of each record and this is not reflected in the return type.

Attributes

Concrete methods

final def produceAll[R, K, V](keySerializer: Serializer[R, K], valueSerializer: Serializer[R, V]): ZPipeline[R, Throwable, ProducerRecord[K, V], RecordMetadata]

A stream pipeline that produces all records from the stream.

A stream pipeline that produces all records from the stream.

Attributes