KafkaConsumerConfig

Configuration for Kafka Consumer.

For the official documentation on the available configuration options, see Consumer Configs on kafka.apache.org.

groupId: is the group.id setting, a unique string that identifies the consumer group this consumer belongs to.
zookeeperConnect: is the zookeeper.connect setting, a list of host/port pairs to use for establishing the initial connection to the Zookeeper cluster.
consumerId: is the consumer.id setting, a unique string that identifies the consumer (will be autogenerated if not set).
socketTimeout: is the socket.timeout.ms setting, the socket timeout for network requests.
socketReceiveBufferInBytes: is the socket.receive.buffer.bytes setting, the size of the socket receive buffer for network requests.
fetchMessageMaxBytes: is the fetch.message.max.bytes setting, the maximum amount of data per-partition the server will return.
numConsumerFetchers: is the num.consumer.fetchers setting, the number of fetcher threads to spawn.
autoCommitEnable: is the auto.commit.enable setting. If true the consumer's offset will be periodically committed in the background.
autoCommitInterval: is the auto.commit.interval.ms setting, the frequency of autocommits.
queuedMaxMessageChunks: is the queued.max.message.chunks setting, the maximum number of message chunks that consumer may buffer.
rebalanceMaxRetries: is the rebalance.max.retries setting, the number of attempts to rebalance the consumer group when a new consumer joins.
fetchMinBytes: is the fetch.min.bytes setting, the minimum amount of data the server should return for a fetch request.
fetchWaitMaxTime: is the fetch.wait.max.ms setting, the maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.
rebalanceBackoffTime: is the rebalance.backoff.m setting. The amount of time to wait before attempting to rebalance the consumer group.
refreshLeaderBackoffTime: is the refresh.leader.backoff.ms setting. The amount of time to wait before trying to elect a new leader for a consumer group that has lost one.
autoOffsetReset: is the auto.offset.reset setting, specifying what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted).
consumerTimeout: is the consumer.timeout.ms setting, which specifies the amount of time to wait before throwing an exception when there's nothing to consume.
excludeInternalTopics: is the exclude.internal.topics setting. Whether records from internal topics (such as offsets) should be exposed to the consumer. If set to true the only way to receive records from an internal topic is subscribing to it.
partitionAssignmentStrategy: is the partition.assignment.strategy setting, which chooses how partitions will be assigned to consumer streams (range or roundrobin). Note that roundrobin strategy results in a more even load distribution, but will not work when consuming from multiple topics.
clientId: is the client.id setting, an id string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.
zookeeperSessionTimeout: is the zookeeper.session.timeout.ms setting, the maximum amount of time to wait for a heartbeat before initiating a rebalance.
zookeeperConnectionTimeout: is the zookeeper.connection.timeout.ms setting, the maximum amount of time the client will wait to establish a connection to ZooKeeper.
zookeeperSyncTime: is the zookeeper.sync.time.ms setting, the maximum lag allowed for ZK followers.
offsetsStorage: is the offsets.storage setting, that controls where offsets are stored (zookeeper or kafka).
offsetsChannelBackoffTime: is the offsets.channel.backoff.ms setting, the backoff period when reconnecting the offsets channel or retrying failed offset fetch/commit requests.
offsetsChannelSocketTimeout: is the offsets.channel.socket.timeout.ms setting. Socket timeout when reading responses for offset fetch/commit requests.
offsetsCommitMaxRetries: is the offsets.commit.max.retries setting, The maximum amount of retries for commiting the offset. This retry count only applies to offset commits during shut-down. It does not apply to commits originating from the auto-commit thread.
dualCommitEnabled: is the dual.commit.enabled setting, which can be used to dual commit offsets to ZooKeeper if using kafka as offsets.storage. This is required during migration from ZooKeeper-based offset storage to Kafka-based offset storage.

Linear Supertypes

Serializable, Serializable, Product, Equals, AnyRef, Any

Instance Constructors

new KafkaConsumerConfig(groupId: String, zookeeperConnect: String, consumerId: String, socketTimeout: FiniteDuration, socketReceiveBufferInBytes: Int, fetchMessageMaxBytes: Int, numConsumerFetchers: Int, autoCommitEnable: Boolean, autoCommitInterval: FiniteDuration, queuedMaxMessageChunks: Int, rebalanceMaxRetries: Int, fetchMinBytes: Int, fetchWaitMaxTime: FiniteDuration, rebalanceBackoffTime: FiniteDuration, refreshLeaderBackoffTime: FiniteDuration, autoOffsetReset: AutoOffsetReset, consumerTimeout: FiniteDuration, excludeInternalTopics: Boolean, partitionAssignmentStrategy: PartitionAssignmentStrategy, clientId: String, zookeeperSessionTimeout: FiniteDuration, zookeeperConnectionTimeout: FiniteDuration, zookeeperSyncTime: FiniteDuration, offsetsStorage: OffsetsStorage, offsetsChannelBackoffTime: FiniteDuration, offsetsChannelSocketTimeout: FiniteDuration, offsetsCommitMaxRetries: Int, dualCommitEnabled: Boolean)

groupId
is the group.id setting, a unique string that identifies the consumer group this consumer belongs to.
zookeeperConnect
is the zookeeper.connect setting, a list of host/port pairs to use for establishing the initial connection to the Zookeeper cluster.
consumerId
is the consumer.id setting, a unique string that identifies the consumer (will be autogenerated if not set).
socketTimeout
is the socket.timeout.ms setting, the socket timeout for network requests.
socketReceiveBufferInBytes
is the socket.receive.buffer.bytes setting, the size of the socket receive buffer for network requests.
fetchMessageMaxBytes
is the fetch.message.max.bytes setting, the maximum amount of data per-partition the server will return.
numConsumerFetchers
is the num.consumer.fetchers setting, the number of fetcher threads to spawn.
autoCommitEnable
is the auto.commit.enable setting. If true the consumer's offset will be periodically committed in the background.
autoCommitInterval
is the auto.commit.interval.ms setting, the frequency of autocommits.
queuedMaxMessageChunks
is the queued.max.message.chunks setting, the maximum number of message chunks that consumer may buffer.
rebalanceMaxRetries
is the rebalance.max.retries setting, the number of attempts to rebalance the consumer group when a new consumer joins.
fetchMinBytes
is the fetch.min.bytes setting, the minimum amount of data the server should return for a fetch request.
fetchWaitMaxTime
is the fetch.wait.max.ms setting, the maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.
rebalanceBackoffTime
is the rebalance.backoff.m setting. The amount of time to wait before attempting to rebalance the consumer group.
refreshLeaderBackoffTime
is the refresh.leader.backoff.ms setting. The amount of time to wait before trying to elect a new leader for a consumer group that has lost one.
autoOffsetReset
is the auto.offset.reset setting, specifying what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted).
consumerTimeout
is the consumer.timeout.ms setting, which specifies the amount of time to wait before throwing an exception when there's nothing to consume.
excludeInternalTopics
is the exclude.internal.topics setting. Whether records from internal topics (such as offsets) should be exposed to the consumer. If set to true the only way to receive records from an internal topic is subscribing to it.
partitionAssignmentStrategy
is the partition.assignment.strategy setting, which chooses how partitions will be assigned to consumer streams (range or roundrobin). Note that roundrobin strategy results in a more even load distribution, but will not work when consuming from multiple topics.
clientId
is the client.id setting, an id string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.
zookeeperSessionTimeout
is the zookeeper.session.timeout.ms setting, the maximum amount of time to wait for a heartbeat before initiating a rebalance.
zookeeperConnectionTimeout
is the zookeeper.connection.timeout.ms setting, the maximum amount of time the client will wait to establish a connection to ZooKeeper.
zookeeperSyncTime
is the zookeeper.sync.time.ms setting, the maximum lag allowed for ZK followers.
offsetsStorage
is the offsets.storage setting, that controls where offsets are stored (zookeeper or kafka).
offsetsChannelBackoffTime
is the offsets.channel.backoff.ms setting, the backoff period when reconnecting the offsets channel or retrying failed offset fetch/commit requests.
offsetsChannelSocketTimeout
is the offsets.channel.socket.timeout.ms setting. Socket timeout when reading responses for offset fetch/commit requests.
offsetsCommitMaxRetries
is the offsets.commit.max.retries setting, The maximum amount of retries for commiting the offset. This retry count only applies to offset commits during shut-down. It does not apply to commits originating from the auto-commit thread.
dualCommitEnabled
is the dual.commit.enabled setting, which can be used to dual commit offsets to ZooKeeper if using kafka as offsets.storage. This is required during migration from ZooKeeper-based offset storage to Kafka-based offset storage.

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
val autoCommitEnable: Boolean

is the auto.commit.enable setting.
is the auto.commit.enable setting. If true the consumer's offset will be periodically committed in the background.
val autoCommitInterval: FiniteDuration

is the auto.commit.interval.ms setting, the frequency of autocommits.
val autoOffsetReset: AutoOffsetReset

is the auto.offset.reset setting, specifying what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g.
is the auto.offset.reset setting, specifying what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted).
val clientId: String

is the client.id setting, an id string to pass to the server when making requests.
is the client.id setting, an id string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
val consumerId: String

is the consumer.id setting, a unique string that identifies the consumer (will be autogenerated if not set).
val consumerTimeout: FiniteDuration

is the consumer.timeout.ms setting, which specifies the amount of time to wait before throwing an exception when there's nothing to consume.
val dualCommitEnabled: Boolean

is the dual.commit.enabled setting, which can be used to dual commit offsets to ZooKeeper if using kafka as offsets.storage.
is the dual.commit.enabled setting, which can be used to dual commit offsets to ZooKeeper if using kafka as offsets.storage. This is required during migration from ZooKeeper-based offset storage to Kafka-based offset storage.
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
val excludeInternalTopics: Boolean

is the exclude.internal.topics setting.
is the exclude.internal.topics setting. Whether records from internal topics (such as offsets) should be exposed to the consumer. If set to true the only way to receive records from an internal topic is subscribing to it.
val fetchMessageMaxBytes: Int

is the fetch.message.max.bytes setting, the maximum amount of data per-partition the server will return.
val fetchMinBytes: Int

is the fetch.min.bytes setting, the minimum amount of data the server should return for a fetch request.
val fetchWaitMaxTime: FiniteDuration

is the fetch.wait.max.ms setting, the maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
val groupId: String

is the group.id setting, a unique string that identifies the consumer group this consumer belongs to.
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val numConsumerFetchers: Int

is the num.consumer.fetchers setting, the number of fetcher threads to spawn.
val offsetsChannelBackoffTime: FiniteDuration

is the offsets.channel.backoff.ms setting, the backoff period when reconnecting the offsets channel or retrying failed offset fetch/commit requests.
val offsetsChannelSocketTimeout: FiniteDuration

is the offsets.channel.socket.timeout.ms setting.
is the offsets.channel.socket.timeout.ms setting. Socket timeout when reading responses for offset fetch/commit requests.
val offsetsCommitMaxRetries: Int

is the offsets.commit.max.retries setting, The maximum amount of retries for commiting the offset.
is the offsets.commit.max.retries setting, The maximum amount of retries for commiting the offset. This retry count only applies to offset commits during shut-down. It does not apply to commits originating from the auto-commit thread.
val offsetsStorage: OffsetsStorage

is the offsets.storage setting, that controls where offsets are stored (zookeeper or kafka).
val partitionAssignmentStrategy: PartitionAssignmentStrategy

is the partition.assignment.strategy setting, which chooses how partitions will be assigned to consumer streams (range or roundrobin).
is the partition.assignment.strategy setting, which chooses how partitions will be assigned to consumer streams (range or roundrobin). Note that roundrobin strategy results in a more even load distribution, but will not work when consuming from multiple topics.
val queuedMaxMessageChunks: Int

is the queued.max.message.chunks setting, the maximum number of message chunks that consumer may buffer.
val rebalanceBackoffTime: FiniteDuration

is the rebalance.backoff.m setting.
is the rebalance.backoff.m setting. The amount of time to wait before attempting to rebalance the consumer group.
val rebalanceMaxRetries: Int

is the rebalance.max.retries setting, the number of attempts to rebalance the consumer group when a new consumer joins.
val refreshLeaderBackoffTime: FiniteDuration

is the refresh.leader.backoff.ms setting.
is the refresh.leader.backoff.ms setting. The amount of time to wait before trying to elect a new leader for a consumer group that has lost one.
val socketReceiveBufferInBytes: Int

is the socket.receive.buffer.bytes setting, the size of the socket receive buffer for network requests.
val socketTimeout: FiniteDuration

is the socket.timeout.ms setting, the socket timeout for network requests.
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toMap: Map[String, String]
def toProperties: Properties
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
val zookeeperConnect: String

is the zookeeper.connect setting, a list of host/port pairs to use for establishing the initial connection to the Zookeeper cluster.
val zookeeperConnectionTimeout: FiniteDuration

is the zookeeper.connection.timeout.ms setting, the maximum amount of time the client will wait to establish a connection to ZooKeeper.
val zookeeperSessionTimeout: FiniteDuration

is the zookeeper.session.timeout.ms setting, the maximum amount of time to wait for a heartbeat before initiating a rebalance.
val zookeeperSyncTime: FiniteDuration

is the zookeeper.sync.time.ms setting, the maximum lag allowed for ZK followers.

Related Docs: object KafkaConsumerConfig | package kafka

Instance Constructors

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

val autoCommitEnable: Boolean

val autoCommitInterval: FiniteDuration

val autoOffsetReset: AutoOffsetReset

val clientId: String

def clone(): AnyRef

val consumerId: String

val consumerTimeout: FiniteDuration

val dualCommitEnabled: Boolean

final def eq(arg0: AnyRef): Boolean

val excludeInternalTopics: Boolean

val fetchMessageMaxBytes: Int

val fetchMinBytes: Int

val fetchWaitMaxTime: FiniteDuration

def finalize(): Unit

final def getClass(): Class[_]

val groupId: String

final def isInstanceOf[T0]: Boolean

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

val numConsumerFetchers: Int

val offsetsChannelBackoffTime: FiniteDuration

val offsetsChannelSocketTimeout: FiniteDuration

val offsetsCommitMaxRetries: Int

val offsetsStorage: OffsetsStorage

val partitionAssignmentStrategy: PartitionAssignmentStrategy

val queuedMaxMessageChunks: Int

val rebalanceBackoffTime: FiniteDuration

val rebalanceMaxRetries: Int

val refreshLeaderBackoffTime: FiniteDuration

val socketReceiveBufferInBytes: Int

val socketTimeout: FiniteDuration

final def synchronized[T0](arg0: ⇒ T0): T0

def toMap: Map[String, String]

def toProperties: Properties

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

val zookeeperConnect: String

val zookeeperConnectionTimeout: FiniteDuration

val zookeeperSessionTimeout: FiniteDuration

val zookeeperSyncTime: FiniteDuration

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped