ConsumerSettings contain settings necessary to create a KafkaConsumer. At the very least, this includes key and value deserializers.
The following consumer configuration defaults are used.
- auto.offset.reset
is set to none
to avoid the surprise of the otherwise default latest
setting.
- enable.auto.commit
is set to false
since offset commits are managed manually.
Several convenience functions are provided so that you don't have to work with String
values and ConsumerConfig
for configuration. It's still possible to specify ConsumerConfig
values with functions like withProperty.
ConsumerSettings instances are immutable and all modification functions return a new ConsumerSettings instance.
Use ConsumerSettings#apply
to create a new instance.
Attributes
- Companion
- object
- Source
- ConsumerSettings.scala
- Graph
-
- Supertypes
Members list
Value members
Abstract methods
The time to wait for the Java KafkaConsumer
to shutdown.
The default value is 20 seconds.
The time to wait for the Java KafkaConsumer
to shutdown.
The default value is 20 seconds.
Attributes
- Source
- ConsumerSettings.scala
The CommitRecovery strategy for recovering from offset commit exceptions.
The default is CommitRecovery#Default.
The CommitRecovery strategy for recovering from offset commit exceptions.
The default is CommitRecovery#Default.
Attributes
- Source
- ConsumerSettings.scala
The time to wait for offset commits to complete.
The time to wait for offset commits to complete. If an offset commit doesn't complete within this time, a CommitTimeoutException will be raised instead.
The default value is 15 seconds.
Attributes
- Source
- ConsumerSettings.scala
A custom ExecutionContext
to use for blocking Kafka operations.
A custom ExecutionContext
to use for blocking Kafka operations. If not provided, a default single-threaded ExecutionContext
will be created when creating a KafkaConsumer
instance.
Attributes
- Source
- ConsumerSettings.scala
The Deserializer
to use for deserializing record keys.
The maximum number of record batches to prefetch per topic-partition.
The maximum number of record batches to prefetch per topic-partition. This means that, while records are being processed, there can be up to maxPrefetchBatches * max.poll.records
records per topic-partition that have already been fetched, and are waiting to be processed. You can use withMaxPollRecords to control the max.poll.records
setting.
This setting effectively controls backpressure, i.e. the maximum number of batches to prefetch per topic-parititon before starting to slow down (not fetching more records) until processing has caught-up.
Note that prefetching cannot be disabled and is generally preferred since it yields improved performance. The minimum value for this setting is 2
.
Attributes
- Source
- ConsumerSettings.scala
How often we should attempt to call poll
on the Java KafkaConsumer
.
The default value is 50 milliseconds.
How often we should attempt to call poll
on the Java KafkaConsumer
.
The default value is 50 milliseconds.
Attributes
- Source
- ConsumerSettings.scala
How long we should allow the poll
call to block for in the Java KafkaConsumer
.
The default value is 50 milliseconds.
How long we should allow the poll
call to block for in the Java KafkaConsumer
.
The default value is 50 milliseconds.
Attributes
- Source
- ConsumerSettings.scala
Properties which can be provided when creating a Java KafkaConsumer
instance.
Properties which can be provided when creating a Java KafkaConsumer
instance. Numerous functions in ConsumerSettings add properties here if the settings are used by the Java KafkaConsumer
.
Attributes
- Source
- ConsumerSettings.scala
The function used to specify metadata for records.
The function used to specify metadata for records. This metadata will be included in OffsetAndMetadata
in the CommittableOffsets, and can then be committed with the offsets.
By default, there will be no metadata, as determined by OffsetFetchResponse.NO_METADATA
.
Attributes
- Source
- ConsumerSettings.scala
The Deserializer
to use for deserializing record values.
Returns a new ConsumerSettings instance with the specified allow auto create topics.
Returns a new ConsumerSettings instance with the specified allow auto create topics. This is equivalent to setting the following property using the withProperty function, except you can specify it with a Boolean
instead of a String
.
ConsumerConfig.ALLOW_AUTO_CREATE_TOPICS_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified auto commit interval.
Returns a new ConsumerSettings instance with the specified auto commit interval. This is equivalent to setting the following property using the withProperty function, except you can specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified auto offset reset.
Returns a new ConsumerSettings instance with the specified auto offset reset. This is equivalent to setting the following property using the withProperty function, except you can specify it with AutoOffsetReset instead of a String
.
ConsumerConfig.AUTO_OFFSET_RESET_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified bootstrap servers.
Returns a new ConsumerSettings instance with the specified bootstrap servers. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified client id.
Returns a new ConsumerSettings instance with the specified client id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.CLIENT_ID_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified client rack.
Returns a new ConsumerSettings instance with the specified client rack. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.CLIENT_RACK_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified closeTimeout.
Creates a new ConsumerSettings with the specified closeTimeout.
Attributes
- Source
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified CommitRecovery as the commitRecovery to use.
Creates a new ConsumerSettings with the specified CommitRecovery as the commitRecovery to use.
Attributes
- Source
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified commitTimeout.
Creates a new ConsumerSettings with the specified commitTimeout.
Attributes
- Source
- ConsumerSettings.scala
Includes the credentials properties from the provided KafkaCredentialStore
Includes the credentials properties from the provided KafkaCredentialStore
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified ExecutionContext to use for blocking operations.
Returns a new ConsumerSettings instance with the specified ExecutionContext to use for blocking operations.
Because the underlying Java consumer is not thread-safe, the ExecutionContext *must* be single-threaded. If in doubt, leave this unset so that a default single-threaded blocker will be provided.
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified default api timeout.
Returns a new ConsumerSettings instance with the specified default api timeout. This is equivalent to setting the following property using the withProperty function, except you can specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.DEFAULT_API_TIMEOUT_MS_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Creates a new ConsumerSettings
instance that replaces the serializers with those provided.
Creates a new ConsumerSettings
instance that replaces the serializers with those provided. Note that this will remove any custom recordMetadata
configuration.
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified auto commit setting.
Returns a new ConsumerSettings instance with the specified auto commit setting. This is equivalent to setting the following property using the withProperty function, except you can specify it with a Boolean
instead of a String
.
ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG
Note that by default, this setting is set to false
.
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified group id.
Returns a new ConsumerSettings instance with the specified group id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.GROUP_ID_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified group instance id.
Returns a new ConsumerSettings instance with the specified group instance id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.GROUP_INSTANCE_ID_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified heartbeat interval.
Returns a new ConsumerSettings instance with the specified heartbeat interval. This is equivalent to setting the following property using the withProperty function, except you can specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.HEARTBEAT_INTERVAL_MS_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified isolation level.
Returns a new ConsumerSettings instance with the specified isolation level. This is equivalent to setting the following property using the withProperty function, except you can specify it with an IsolationLevel instead of a String
.
ConsumerConfig.ISOLATION_LEVEL_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified max poll interval.
Returns a new ConsumerSettings instance with the specified max poll interval. This is equivalent to setting the following property using the withProperty function, except you can specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified max poll records.
Returns a new ConsumerSettings instance with the specified max poll records. This is equivalent to setting the following property using the withProperty function, except you can specify it with an Int
instead of a String
.
ConsumerConfig.MAX_POLL_RECORDS_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified value for maxPrefetchBatches.
Creates a new ConsumerSettings with the specified value for maxPrefetchBatches. Note that if a value lower than the minimum 2
is specified, maxPrefetchBatches will instead be set to 2
and not the specified value.
Attributes
- Source
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified pollInterval.
Creates a new ConsumerSettings with the specified pollInterval.
Attributes
- Source
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified pollTimeout.
Creates a new ConsumerSettings with the specified pollTimeout.
Attributes
- Source
- ConsumerSettings.scala
Includes the specified keys and values as properties.
Includes the specified keys and values as properties. The keys should be part of the ConsumerConfig
keys, and the values should be valid choices for the keys.
Attributes
- Source
- ConsumerSettings.scala
Includes the specified keys and values as properties.
Includes the specified keys and values as properties. The keys should be part of the ConsumerConfig
keys, and the values should be valid choices for the keys.
Attributes
- Source
- ConsumerSettings.scala
Includes a property with the specified key
and value
.
Includes a property with the specified key
and value
. The key should be one of the keys in ConsumerConfig
, and the value should be a valid choice for the key.
Attributes
- Source
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified recordMetadata.
Creates a new ConsumerSettings with the specified recordMetadata. Note that replacing the serializers via withSerializers
will reset this to the default.
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified request timeout.
Returns a new ConsumerSettings instance with the specified request timeout. This is equivalent to setting the following property using the withProperty function, except you can specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG
Attributes
- Source
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified session timeout.
Returns a new ConsumerSettings instance with the specified session timeout. This is equivalent to setting the following property using the withProperty function, except you can specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG
Attributes
- Source
- ConsumerSettings.scala