zio-kinesis-dynamic-consumer/nl.vroste.zio.kinesis.client.dynamicconsumer/DynamicConsumer/Checkpointer

Checkpointer

trait Checkpointer

Staging area for checkpoints

Guarantees that the last staged record is checkpointed upon stream shutdown / interruption

class Object
trait Matchable
class Any

Value members

Abstract methods

def checkpoint: ZIO[Any, Throwable, Unit]

Checkpoint the last staged checkpoint

Checkpoint the last staged checkpoint

Exceptions you should be prepared to handle:

  • software.amazon.kinesis.exceptions.ShutdownException when the lease for this shard has been lost, when another worker has stolen the lease (this can happen at any time).
  • software.amazon.kinesis.exceptions.ThrottlingException

See also software.amazon.kinesis.processor.RecordProcessorCheckpointer

def stage(r: Record[_]): UIO[Unit]

Stages a record for checkpointing

Stages a record for checkpointing

Checkpoints are not actually performed until checkpoint is called

Value parameters:
r

Record to checkpoint

Returns:

Effect that completes immediately

Concrete methods

def checkpointBatched[R](nr: Long, interval: Duration): ZStream[R, Throwable, Any] => ZStream[R & Any, Throwable, Unit]

Helper method to add batch checkpointing to a shard stream

Helper method to add batch checkpointing to a shard stream

Usage: shardStream.viaFunction(checkpointer.checkpointBatched(1000, 1.second))

Value parameters:
interval

Maximum interval before checkpointing

nr

Maximum number of records before checkpointing

Returns:

Function that results in a ZStream that produces Unit values for successful checkpoints, fails with an exception when checkpointing fails or becomes an empty stream when the lease for this shard is lost, thereby ending the stream.

def checkpointNow(r: Record[_]): ZIO[Any, Throwable, Unit]

Immediately checkpoint this record

Immediately checkpoint this record

def stageOnSuccess[R, E, A](effect: ZIO[R, E, A])(r: Record[_]): ZIO[R, E, A]

Helper method that ensures that a checkpoint is staged when 'effect' completes successfully, even when the fiber is interrupted. When 'effect' fails or is itself interrupted, the checkpoint is not staged.

Helper method that ensures that a checkpoint is staged when 'effect' completes successfully, even when the fiber is interrupted. When 'effect' fails or is itself interrupted, the checkpoint is not staged.

Value parameters:
effect

Effect to execute

r

Record to stage a checkpoint for

Returns:

Effect that completes with the result of 'effect'