KafkaActor
Related Doc:
package kafkaactors
Override this method in the implementing class to customize the consumer settings to your liking.
The ref for this actor that permits sending messages through the Kafka broker instead of directly to the actor itself.
The ref for this actor that permits sending messages through the Kafka broker instead of
directly to the actor itself. Override this implementation if you need to include custom
producing behavior in the KafkaActorRef.
A kind of LiftActor capable of consuming messages from a Kafka topic.
This actor imposes a few restrictions that normal LiftActors do not. Specifically:
userMessageHandlerinstead ofmessageHandler.InternalKafkaActorMessage.Other than the above, this Actor behaves very similarly to a normal
LiftActor. You can send messages directly to it, thus bypassing Kafka, by using its!orsendmethods.Configuration
For this actor to work correctly with Kafka, you'll ahve to provide it with some basic configuration. The required overrides are:
bootstrapServers: This needs to be the broker list for your Kafka clustergroupId: This is the groupId the actor should consume under. See Kafka docs for more details.kafkaTopic: The topic the actor should subscribe toThe Kafka consumer works by polling for a certain number of milliseconds, and then returning if no messages are retrieved. We abstract away that event loop behavior, but sometimes applications need to tweak how long the consumer will sleep in order to optimize performance. To change that you can also override the following:
pollTime: The amount of time, in milliseconds, the consumer should wait for recrods. Defaults to 500ms.If you need to tune more specific settings, you can provide a
consumerPropsCustomizerthat will get to alter thePropertiesobject before we pass it into theKafkaConsumerconstructor. This is what you'll need to implement if you want to provide custom settings for things like authentication, encryption, etc. By default, we provide the bootstrap servers, the group ID, we disable auto committing, and provide a key and value serializer implementation.Please be careful when overriding settings that were set by the
KafkaActoritself.Starting consumption
Once the actor is created, it'll behave like a normal
LiftActoruntil its told to connect up to Kafka and start consuming messages. To do that your code will need to transmit theStartConsumermessage to it like so:You can also stop consuming anytime you like by transmitting
StopConsumeror you can force committing offsets by transmittingCommitOffsetsto the actor if you need to do so for some reason, though as mentioned below those cases should be rare.Processing messages
When messages come from the topic, they will be parsed and extracted to case class objects using lift-json. The messages will then be put in the actor's normal mailbox using
!and be subjet to normal actor processing rules. Every time the actor consumes messages it'll also add aCommitOffsetsmessage onto the end of the message batch.Because of the way the actor mailbox works,
CommitOffsetswon't be processed until all of the messages in that batch have been processed. Thus, if you have a class of errors that may cause you to want to avoid checkpointing offsets to Kafka, you sould throw an exception of some sort in youruserMessageHandlerso things blow up.Sending messages
KafkaActorworks like a normal actor if you use the regular sending methods. However, you may find it useful to send messages to this actor _through_ Kafka even when the actor is running in the local process. To facilitate this, you can send messages through the includedreflike so:This will cause the message to be routed through a Kafka producer and then consumed by the actor using the normal consumption means.