Kafka configuration parameters to be used on executors.
Kafka configuration parameters to be used on executors. Requires "bootstrap.servers" to be set with Kafka broker(s) specified in host1:port1,host2:port2 form.
Must return a fully configured Kafka Consumer, including subscribed or assigned topics.
Must return a fully configured Kafka Consumer, including subscribed or assigned topics. See Kafka docs. This consumer will be used on the driver to query for offsets only, not messages. The consumer must be returned in a state that it is safe to call poll(0) on.
A map from TopicPartition to offset, indicating how far the driver has successfully read. Will be empty on initial start, possibly non-empty on restart from checkpoint.
:: Experimental :: Choice of how to create and configure underlying Kafka Consumers on driver and executors. See ConsumerStrategies to obtain instances. Kafka 0.10 consumers can require additional, sometimes complex, setup after object instantiation. This interface encapsulates that process, and allows it to be checkpointed.
type of Kafka message key
type of Kafka message value