ConsumerSettings contain settings necessary to create a
KafkaConsumer. At the very least, this includes key and
value deserializers.
The following consumer configuration defaults are used.
auto.offset.reset
is set tonone
to avoid the surprise of the otherwise defaultlatest
setting.enable.auto.commit
is set tofalse
since offset commits are managed manually.
Several convenience functions are provided so that you don't have to work with `String` values and `ConsumerConfig` for configuration. It's still possible to specify `ConsumerConfig` values with functions like [[withProperty]].
[[ConsumerSettings]] instances are immutable and all modification functions return a new [[ConsumerSettings]] instance.
Use `ConsumerSettings#apply` to create a new instance.
- Companion:
- object
- Source:
- ConsumerSettings.scala
Value members
Abstract methods
The time to wait for the Java KafkaConsumer
to shutdown.
The default value is 20 seconds.
The time to wait for the Java KafkaConsumer
to shutdown.
The default value is 20 seconds.
- Source:
- ConsumerSettings.scala
The CommitRecovery strategy for recovering from offset
commit exceptions.
The default is CommitRecovery#Default.
The CommitRecovery strategy for recovering from offset
commit exceptions.
The default is CommitRecovery#Default.
- Source:
- ConsumerSettings.scala
The time to wait for offset commits to complete. If an offset commit
doesn't complete within this time, a CommitTimeoutException will
be raised instead.
The default value is 15 seconds.
The time to wait for offset commits to complete. If an offset commit
doesn't complete within this time, a CommitTimeoutException will
be raised instead.
The default value is 15 seconds.
- Source:
- ConsumerSettings.scala
A custom ExecutionContext
to use for blocking Kafka operations. If not
provided, a default single-threaded ExecutionContext
will be created
when creating a KafkaConsumer
instance.
A custom ExecutionContext
to use for blocking Kafka operations. If not
provided, a default single-threaded ExecutionContext
will be created
when creating a KafkaConsumer
instance.
- Source:
- ConsumerSettings.scala
The Deserializer
to use for deserializing record keys.
The Deserializer
to use for deserializing record keys.
- Source:
- ConsumerSettings.scala
The maximum number of record batches to prefetch per topic-partition.
This means that, while records are being processed, there can be up
to maxPrefetchBatches * max.poll.records
records per topic-partition
that have already been fetched, and are waiting to be processed. You can
use withMaxPollRecords to control the max.poll.records
setting.
This setting effectively controls backpressure, i.e. the maximum number
of batches to prefetch per topic-parititon before starting to slow down
(not fetching more records) until processing has caught-up.
Note that prefetching cannot be disabled and is generally preferred since
it yields improved performance. The minimum value for this setting is 2
.
The maximum number of record batches to prefetch per topic-partition.
This means that, while records are being processed, there can be up
to maxPrefetchBatches * max.poll.records
records per topic-partition
that have already been fetched, and are waiting to be processed. You can
use withMaxPollRecords to control the max.poll.records
setting.
This setting effectively controls backpressure, i.e. the maximum number
of batches to prefetch per topic-parititon before starting to slow down
(not fetching more records) until processing has caught-up.
Note that prefetching cannot be disabled and is generally preferred since
it yields improved performance. The minimum value for this setting is 2
.
- Source:
- ConsumerSettings.scala
How often we should attempt to call poll
on the Java KafkaConsumer
.
The default value is 50 milliseconds.
How often we should attempt to call poll
on the Java KafkaConsumer
.
The default value is 50 milliseconds.
- Source:
- ConsumerSettings.scala
How long we should allow the poll
call to block for in the
Java KafkaConsumer
.
The default value is 50 milliseconds.
How long we should allow the poll
call to block for in the
Java KafkaConsumer
.
The default value is 50 milliseconds.
- Source:
- ConsumerSettings.scala
Properties which can be provided when creating a Java KafkaConsumer
instance. Numerous functions in ConsumerSettings add properties
here if the settings are used by the Java KafkaConsumer
.
Properties which can be provided when creating a Java KafkaConsumer
instance. Numerous functions in ConsumerSettings add properties
here if the settings are used by the Java KafkaConsumer
.
- Source:
- ConsumerSettings.scala
The function used to specify metadata for records. This
metadata will be included in OffsetAndMetadata
in the
CommittableOffsets, and can then be committed with
the offsets.
By default, there will be no metadata, as determined by
OffsetFetchResponse.NO_METADATA
.
The function used to specify metadata for records. This
metadata will be included in OffsetAndMetadata
in the
CommittableOffsets, and can then be committed with
the offsets.
By default, there will be no metadata, as determined by
OffsetFetchResponse.NO_METADATA
.
- Source:
- ConsumerSettings.scala
The Deserializer
to use for deserializing record values.
The Deserializer
to use for deserializing record values.
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
allow auto create topics. This is equivalent to setting the
following property using the withProperty function, except
you can specify it with a Boolean
instead of a String
.
Returns a new ConsumerSettings instance with the specified
allow auto create topics. This is equivalent to setting the
following property using the withProperty function, except
you can specify it with a Boolean
instead of a String
.
ConsumerConfig.ALLOW_AUTO_CREATE_TOPICS_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
auto commit interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
Returns a new ConsumerSettings instance with the specified
auto commit interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
auto offset reset. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with AutoOffsetReset instead of a String
.
Returns a new ConsumerSettings instance with the specified
auto offset reset. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with AutoOffsetReset instead of a String
.
ConsumerConfig.AUTO_OFFSET_RESET_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified bootstrap servers. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified bootstrap servers. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified client id. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified client id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.CLIENT_ID_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified client rack. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified client rack. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.CLIENT_RACK_CONFIG
- Source:
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified closeTimeout.
Creates a new ConsumerSettings with the specified closeTimeout.
- Source:
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified CommitRecovery as the commitRecovery to use.
Creates a new ConsumerSettings with the specified CommitRecovery as the commitRecovery to use.
- Source:
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified commitTimeout.
Creates a new ConsumerSettings with the specified commitTimeout.
- Source:
- ConsumerSettings.scala
Includes the credentials properties from the provided KafkaCredentialStore
Includes the credentials properties from the provided KafkaCredentialStore
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified ExecutionContext to use for blocking operations.
Returns a new ConsumerSettings instance with the specified ExecutionContext to use for blocking operations.
Because the underlying Java consumer is not thread-safe, the ExecutionContext must be single-threaded. If in doubt, leave this unset so that a default single-threaded blocker will be provided.
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
default api timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
Returns a new ConsumerSettings instance with the specified
default api timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.DEFAULT_API_TIMEOUT_MS_CONFIG
- Source:
- ConsumerSettings.scala
Creates a new ConsumerSettings
instance that replaces the serializers with those provided.
Note that this will remove any custom recordMetadata
configuration.
Creates a new ConsumerSettings
instance that replaces the serializers with those provided.
Note that this will remove any custom recordMetadata
configuration.
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
auto commit setting. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a Boolean
instead of a String
.
Returns a new ConsumerSettings instance with the specified
auto commit setting. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a Boolean
instead of a String
.
ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG
Note that by default, this setting is set to false
.
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified group id. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified group id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.GROUP_ID_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified group instance id. This is equivalent to setting the following property using the withProperty function.
Returns a new ConsumerSettings instance with the specified group instance id. This is equivalent to setting the following property using the withProperty function.
ConsumerConfig.GROUP_INSTANCE_ID_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
heartbeat interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
Returns a new ConsumerSettings instance with the specified
heartbeat interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.HEARTBEAT_INTERVAL_MS_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
isolation level. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with an IsolationLevel instead of a String
.
Returns a new ConsumerSettings instance with the specified
isolation level. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with an IsolationLevel instead of a String
.
ConsumerConfig.ISOLATION_LEVEL_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
max poll interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
Returns a new ConsumerSettings instance with the specified
max poll interval. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
max poll records. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with an Int
instead of a String
.
Returns a new ConsumerSettings instance with the specified
max poll records. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with an Int
instead of a String
.
ConsumerConfig.MAX_POLL_RECORDS_CONFIG
- Source:
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified value
for maxPrefetchBatches. Note that if a value lower than
the minimum 2
is specified, maxPrefetchBatches will
instead be set to 2
and not the specified value.
Creates a new ConsumerSettings with the specified value
for maxPrefetchBatches. Note that if a value lower than
the minimum 2
is specified, maxPrefetchBatches will
instead be set to 2
and not the specified value.
- Source:
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified pollInterval.
Creates a new ConsumerSettings with the specified pollInterval.
- Source:
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified pollTimeout.
Creates a new ConsumerSettings with the specified pollTimeout.
- Source:
- ConsumerSettings.scala
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig
keys, and
the values should be valid choices for the keys.
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig
keys, and
the values should be valid choices for the keys.
- Source:
- ConsumerSettings.scala
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig
keys, and
the values should be valid choices for the keys.
Includes the specified keys and values as properties. The
keys should be part of the ConsumerConfig
keys, and
the values should be valid choices for the keys.
- Source:
- ConsumerSettings.scala
Includes a property with the specified key
and value
.
The key should be one of the keys in ConsumerConfig
,
and the value should be a valid choice for the key.
Includes a property with the specified key
and value
.
The key should be one of the keys in ConsumerConfig
,
and the value should be a valid choice for the key.
- Source:
- ConsumerSettings.scala
Creates a new ConsumerSettings with the specified recordMetadata.
Note that replacing the serializers via withSerializers
will reset
this to the default.
Creates a new ConsumerSettings with the specified recordMetadata.
Note that replacing the serializers via withSerializers
will reset
this to the default.
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
request timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
Returns a new ConsumerSettings instance with the specified
request timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG
- Source:
- ConsumerSettings.scala
Returns a new ConsumerSettings instance with the specified
session timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
Returns a new ConsumerSettings instance with the specified
session timeout. This is equivalent to setting the following
property using the withProperty function, except you can
specify it with a FiniteDuration
instead of a String
.
ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG
- Source:
- ConsumerSettings.scala