public interface ConsumerBuilder<T> extends Cloneable
ConsumerBuilder
is used to configure and create instances of Consumer
.PulsarClient.newConsumer()
Modifier and Type | Method and Description |
---|---|
ConsumerBuilder<T> |
acknowledgmentGroupTime(long delay,
TimeUnit unit)
Group the consumer acknowledgments for the specified time.
|
ConsumerBuilder<T> |
ackTimeout(long ackTimeout,
TimeUnit timeUnit)
Set the timeout for unacked messages, truncated to the nearest millisecond.
|
ConsumerBuilder<T> |
ackTimeoutTickTime(long tickTime,
TimeUnit timeUnit)
Define the granularity of the ack-timeout redelivery.
|
ConsumerBuilder<T> |
autoAckOldestChunkedMessageOnQueueFull(boolean autoAckOldestChunkedMessageOnQueueFull)
Buffering large number of outstanding uncompleted chunked messages can create memory pressure and it can be
guarded by providing this @maxPendingChuckedMessage threshold.
|
ConsumerBuilder<T> |
autoUpdatePartitions(boolean autoUpdate)
If enabled, the consumer will auto subscribe for partitions increasement.
|
ConsumerBuilder<T> |
batchReceivePolicy(BatchReceivePolicy batchReceivePolicy)
Set batch receive policy
BatchReceivePolicy for consumer. |
ConsumerBuilder<T> |
clone()
Create a copy of the current consumer builder.
|
ConsumerBuilder<T> |
consumerEventListener(ConsumerEventListener consumerEventListener)
Sets a
ConsumerEventListener for the consumer. |
ConsumerBuilder<T> |
consumerName(String consumerName)
Set the consumer name.
|
ConsumerBuilder<T> |
cryptoFailureAction(ConsumerCryptoFailureAction action)
Sets the ConsumerCryptoFailureAction to the value specified.
|
ConsumerBuilder<T> |
cryptoKeyReader(CryptoKeyReader cryptoKeyReader)
Sets a
CryptoKeyReader . |
ConsumerBuilder<T> |
deadLetterPolicy(DeadLetterPolicy deadLetterPolicy)
Set dead letter policy for consumer.
|
ConsumerBuilder<T> |
enableBatchIndexAcknowledgment(boolean batchIndexAcknowledgmentEnabled)
Enable or disable the batch index acknowledgment.
|
ConsumerBuilder<T> |
enableRetry(boolean retryEnable)
If enabled, the consumer will auto retry message.
|
ConsumerBuilder<T> |
expireTimeOfIncompleteChunkedMessage(long duration,
TimeUnit unit)
If producer fails to publish all the chunks of a message then consumer can expire incomplete chunks if consumer
won't be able to receive all chunks in expire times (default 1 hour).
|
ConsumerBuilder<T> |
intercept(ConsumerInterceptor<T>... interceptors)
Intercept
Consumer . |
ConsumerBuilder<T> |
keySharedPolicy(KeySharedPolicy keySharedPolicy)
Set KeyShared subscription policy for consumer.
|
ConsumerBuilder<T> |
loadConf(Map<String,Object> config)
Load the configuration from provided config map.
|
ConsumerBuilder<T> |
maxPendingChuckedMessage(int maxPendingChuckedMessage)
Consumer buffers chunk messages into memory until it receives all the chunks of the original message.
|
ConsumerBuilder<T> |
maxTotalReceiverQueueSizeAcrossPartitions(int maxTotalReceiverQueueSizeAcrossPartitions)
Set the max total receiver queue size across partitons.
|
ConsumerBuilder<T> |
messageCrypto(MessageCrypto messageCrypto)
Sets a
MessageCrypto . |
ConsumerBuilder<T> |
messageListener(MessageListener<T> messageListener)
Sets a
MessageListener for the consumer |
ConsumerBuilder<T> |
negativeAckRedeliveryDelay(long redeliveryDelay,
TimeUnit timeUnit)
Set the delay to wait before re-delivering messages that have failed to be process.
|
ConsumerBuilder<T> |
patternAutoDiscoveryPeriod(int periodInMinutes)
Set topics auto discovery period when using a pattern for topics consumer.
|
ConsumerBuilder<T> |
patternAutoDiscoveryPeriod(int interval,
TimeUnit unit)
Set topics auto discovery period when using a pattern for topics consumer.
|
ConsumerBuilder<T> |
priorityLevel(int priorityLevel)
Shared subscription
Sets priority level for the shared subscription consumers to which broker gives more priority while dispatching
messages.
|
ConsumerBuilder<T> |
properties(Map<String,String> properties)
Add all the properties in the provided map to the consumer.
|
ConsumerBuilder<T> |
property(String key,
String value)
Set a name/value property with this consumer.
|
ConsumerBuilder<T> |
readCompacted(boolean readCompacted)
If enabled, the consumer will read messages from the compacted topic rather than reading the full message backlog
of the topic.
|
ConsumerBuilder<T> |
receiverQueueSize(int receiverQueueSize)
Sets the size of the consumer receive queue.
|
ConsumerBuilder<T> |
replicateSubscriptionState(boolean replicateSubscriptionState) |
ConsumerBuilder<T> |
startMessageIdInclusive()
Set the consumer to include the given position of any reset operation like
or
{@link Consumer#seek(MessageId)} . |
Consumer<T> |
subscribe()
Finalize the
Consumer creation by subscribing to the topic. |
CompletableFuture<Consumer<T>> |
subscribeAsync()
Finalize the
Consumer creation by subscribing to the topic in asynchronous mode. |
ConsumerBuilder<T> |
subscriptionInitialPosition(SubscriptionInitialPosition subscriptionInitialPosition)
Set the
SubscriptionInitialPosition for the consumer. |
ConsumerBuilder<T> |
subscriptionMode(SubscriptionMode subscriptionMode)
Select the subscription mode to be used when subscribing to the topic.
|
ConsumerBuilder<T> |
subscriptionName(String subscriptionName)
Specify the subscription name for this consumer.
|
ConsumerBuilder<T> |
subscriptionTopicsMode(RegexSubscriptionMode regexSubscriptionMode)
Determines to which topics this consumer should be subscribed to - Persistent, Non-Persistent, or both.
|
ConsumerBuilder<T> |
subscriptionType(SubscriptionType subscriptionType)
Select the subscription type to be used when subscribing to the topic.
|
ConsumerBuilder<T> |
topic(String... topicNames)
Specify the topics this consumer will subscribe on.
|
ConsumerBuilder<T> |
topics(List<String> topicNames)
Specify a list of topics that this consumer will subscribe on.
|
ConsumerBuilder<T> |
topicsPattern(Pattern topicsPattern)
Specify a pattern for topics that this consumer will subscribe on.
|
ConsumerBuilder<T> |
topicsPattern(String topicsPattern)
Specify a pattern for topics that this consumer will subscribe on.
|
ConsumerBuilder<T> clone()
Cloning the builder can be used to share an incomplete configuration and specialize it multiple times. For example:
ConsumerBuilder<String> builder = client.newConsumer(Schema.STRING)
.subscriptionName("my-subscription-name")
.subscriptionType(SubscriptionType.Shared)
.receiverQueueSize(10);
Consumer<String> consumer1 = builder.clone().topic("my-topic-1").subscribe();
Consumer<String> consumer2 = builder.clone().topic("my-topic-2").subscribe();
ConsumerBuilder<T> loadConf(Map<String,Object> config)
Example:
Map<String, Object> config = new HashMap<>();
config.put("ackTimeoutMillis", 1000);
config.put("receiverQueueSize", 2000);
Consumer<byte[]> builder = client.newConsumer()
.loadConf(config)
.subscribe();
Consumer<byte[]> consumer = builder.subscribe();
config
- configuration to loadConsumer<T> subscribe() throws PulsarClientException
Consumer
creation by subscribing to the topic.
If the subscription does not exist, a new subscription will be created. By default the subscription
will be created at the end of the topic. See subscriptionInitialPosition(SubscriptionInitialPosition)
to configure the initial position behavior.
Once a subscription is created, it will retain the data and the subscription cursor even if the consumer is not connected.
PulsarClientException
- if the the subscribe operation failsCompletableFuture<Consumer<T>> subscribeAsync()
Consumer
creation by subscribing to the topic in asynchronous mode.
If the subscription does not exist, a new subscription will be created. By default the subscription
will be created at the end of the topic. See subscriptionInitialPosition(SubscriptionInitialPosition)
to configure the initial position behavior.
Once a subscription is created, it will retain the data and the subscription cursor even if the consumer is not connected.
Consumer
instancePulsarClientException
- if the the subscribe operation failsConsumerBuilder<T> topic(String... topicNames)
topicNames
- a set of topic that the consumer will subscribe onConsumerBuilder<T> topics(List<String> topicNames)
topicNames
- a list of topic that the consumer will subscribe onConsumerBuilder<T> topicsPattern(Pattern topicsPattern)
The pattern will be applied to subscribe to all the topics, within a single namespace, that will match the pattern.
The consumer will automatically subscribe to topics created after itself.
topicsPattern
- a regular expression to select a list of topics to subscribe toConsumerBuilder<T> topicsPattern(String topicsPattern)
It accepts regular expression and will be compiled into a pattern internally. Eg. "persistent://public/default/pattern-topic-.*"
The pattern will be applied to subscribe to all the topics, within a single namespace, that will match the pattern.
The consumer will automatically subscribe to topics created after itself.
topicsPattern
- given regular expression for topics patternConsumerBuilder<T> subscriptionName(String subscriptionName)
This argument is required when constructing the consumer.
subscriptionName
- the name of the subscription that this consumer should attach toConsumerBuilder<T> ackTimeout(long ackTimeout, TimeUnit timeUnit)
By default, the acknowledge timeout is disabled and that means that messages delivered to a consumer will not be re-delivered unless the consumer crashes.
When enabling ack timeout, if a message is not acknowledged within the specified timeout it will be re-delivered to the consumer (possibly to a different consumer in case of a shared subscription).
ackTimeout
- for unacked messages.timeUnit
- unit in which the timeout is provided.ConsumerBuilder<T> ackTimeoutTickTime(long tickTime, TimeUnit timeUnit)
By default, the tick time is set to 1 second. Using an higher tick time will reduce the memory overhead to track messages when the ack-timeout is set to bigger values (eg: 1hour).
tickTime
- the min precision for the ack timeout messages trackertimeUnit
- unit in which the timeout is provided.ConsumerBuilder<T> negativeAckRedeliveryDelay(long redeliveryDelay, TimeUnit timeUnit)
When application uses Consumer.negativeAcknowledge(Message)
, the failed message
will be redelivered after a fixed timeout. The default is 1 min.
redeliveryDelay
- redelivery delay for failed messagestimeUnit
- unit in which the timeout is provided.Consumer.negativeAcknowledge(Message)
ConsumerBuilder<T> subscriptionType(SubscriptionType subscriptionType)
Options are:
subscriptionType
- the subscription type valueConsumerBuilder<T> subscriptionMode(SubscriptionMode subscriptionMode)
Options are:
subscriptionMode
- the subscription mode valueConsumerBuilder<T> messageListener(MessageListener<T> messageListener)
MessageListener
for the consumer
When a MessageListener
is set, application will receive messages through it. Calls to
Consumer.receive()
will not be allowed.
messageListener
- the listener objectConsumerBuilder<T> cryptoKeyReader(CryptoKeyReader cryptoKeyReader)
CryptoKeyReader
.
Configure the key reader to be used to decrypt the message payloads.
cryptoKeyReader
- CryptoKeyReader objectConsumerBuilder<T> messageCrypto(MessageCrypto messageCrypto)
MessageCrypto
.
Contains methods to encrypt/decrypt message for End to End Encryption.
messageCrypto
- MessageCrypto objectConsumerBuilder<T> cryptoFailureAction(ConsumerCryptoFailureAction action)
action
- the action the consumer will take in case of decryption failuresConsumerBuilder<T> receiverQueueSize(int receiverQueueSize)
The consumer receive queue controls how many messages can be accumulated by the Consumer
before the
application calls Consumer.receive()
. Using a higher value could potentially increase the consumer
throughput at the expense of bigger memory utilization.
Setting the consumer queue size as zero
Consumer.receive(int, TimeUnit)
nor Partitioned Topics can be used if the consumer queue
size is zero. Consumer.receive()
function call should not be interrupted when the consumer queue size is
zero.Consumer.receive()
call will remain blocked while Consumer.receiveAsync()
receives
exception in callback. consumer will not be able receive any further message unless batch-message in pipeline
is removed1000
messages and should be good for most use cases.receiverQueueSize
- the new receiver queue size valueConsumerBuilder<T> acknowledgmentGroupTime(long delay, TimeUnit unit)
By default, the consumer will use a 100 ms grouping time to send out the acknowledgments to the broker.
Setting a group time of 0, will send out the acknowledgments immediately. A longer ack group time will be more efficient at the expense of a slight increase in message re-deliveries after a failure.
delay
- the max amount of time an acknowledgemnt can be delayedunit
- the time unit for the delayConsumerBuilder<T> replicateSubscriptionState(boolean replicateSubscriptionState)
replicateSubscriptionState
- ConsumerBuilder<T> maxTotalReceiverQueueSizeAcrossPartitions(int maxTotalReceiverQueueSizeAcrossPartitions)
This setting will be used to reduce the receiver queue size for individual partitions
receiverQueueSize(int)
if the total exceeds this value (default: 50000).
The purpose of this setting is to have an upper-limit on the number
of messages that a consumer can be pushed at once from a broker, across all
the partitions.
maxTotalReceiverQueueSizeAcrossPartitions
- max pending messages across all the partitionsConsumerBuilder<T> consumerName(String consumerName)
Consumer name is informative and it can be used to indentify a particular consumer instance from the topic stats.
consumerName
- ConsumerBuilder<T> consumerEventListener(ConsumerEventListener consumerEventListener)
ConsumerEventListener
for the consumer.
The consumer group listener is used for receiving consumer state change in a consumer group for failover subscription. Application can then react to the consumer state changes.
consumerEventListener
- the consumer group listener objectConsumerBuilder<T> readCompacted(boolean readCompacted)
readCompacted can only be enabled subscriptions to persistent topics, which have a single active consumer (i.e. failure or exclusive subscriptions). Attempting to enable it on subscriptions to a non-persistent topics or on a shared subscription, will lead to the subscription call throwing a PulsarClientException.
readCompacted
- whether to read from the compacted topicConsumerBuilder<T> patternAutoDiscoveryPeriod(int periodInMinutes)
periodInMinutes
- number of minutes between checks for
new topics matching pattern set with topicsPattern(String)
ConsumerBuilder<T> patternAutoDiscoveryPeriod(int interval, TimeUnit unit)
interval
- the amount of delay between checks for
new topics matching pattern set with topicsPattern(String)
unit
- the unit of the topics auto discovery periodConsumerBuilder<T> priorityLevel(int priorityLevel)
In Shared subscription mode, broker will first dispatch messages to max priority-level consumers if they have permits, else broker will consider next priority level consumers.
If subscription has consumer-A with priorityLevel 0 and Consumer-B with priorityLevel 1 then broker will dispatch messages to only consumer-A until it runs out permit and then broker starts dispatching messages to Consumer-B.
Consumer PriorityLevel Permits C1 0 2 C2 0 1 C3 0 1 C4 1 2 C5 1 1 Order in which broker dispatches messages to consumers: C1, C2, C3, C1, C4, C5, C4
Failover subscription Broker selects active consumer for a failover-subscription based on consumer's priority-level and lexicographical sorting of a consumer name. eg:
1. Active consumer = C1 : Same priority-level and lexicographical sorting Consumer PriorityLevel Name C1 0 aaa C2 0 bbb 2. Active consumer = C2 : Consumer with highest priority Consumer PriorityLevel Name C1 1 aaa C2 0 bbb Partitioned-topics: Broker evenly assigns partitioned topics to highest priority consumers.
priorityLevel
- the priority of this consumerConsumerBuilder<T> property(String key, String value)
Properties are application defined metadata that can be attached to the consumer. When getting the topic stats, this metadata will be associated to the consumer stats for easier identification.
key
- the property keyvalue
- the property valuekey
- value
- ConsumerBuilder<T> properties(Map<String,String> properties)
Properties are application defined metadata that can be attached to the consumer. When getting the topic stats, this metadata will be associated to the consumer stats for easier identification.
properties
- the map with propertiesConsumerBuilder<T> subscriptionInitialPosition(SubscriptionInitialPosition subscriptionInitialPosition)
SubscriptionInitialPosition
for the consumer.subscriptionInitialPosition
- the position where to initialize a newly created subscriptionConsumerBuilder<T> subscriptionTopicsMode(RegexSubscriptionMode regexSubscriptionMode)
regexSubscriptionMode
- Pattern subscription modeConsumerBuilder<T> intercept(ConsumerInterceptor<T>... interceptors)
Consumer
.interceptors
- the list of interceptors to intercept the consumer created by this builder.ConsumerBuilder<T> deadLetterPolicy(DeadLetterPolicy deadLetterPolicy)
By default some message will redelivery so many times possible, even to the extent that it can be never stop. By using dead letter mechanism messages will has the max redelivery count, when message exceeding the maximum number of redeliveries, message will send to the Dead Letter Topic and acknowledged automatic.
You can enable the dead letter mechanism by setting dead letter policy. example:
client.newConsumer() .deadLetterPolicy(DeadLetterPolicy.builder().maxRedeliverCount(10).build()) .subscribe();Default dead letter topic name is {TopicName}-{Subscription}-DLQ. To setting a custom dead letter topic name
client.newConsumer() .deadLetterPolicy(DeadLetterPolicy .builder() .maxRedeliverCount(10) .deadLetterTopic("your-topic-name") .build()) .subscribe();When a dead letter policy is specified, and no ackTimeoutMillis is specified, then the ack timeout will be set to 30000 millisecond.
ConsumerBuilder<T> autoUpdatePartitions(boolean autoUpdate)
autoUpdate
- whether to auto update partition increasementConsumerBuilder<T> keySharedPolicy(KeySharedPolicy keySharedPolicy)
By default, KeyShared subscription use auto split hash range to maintain consumers. If you want to set a different KeyShared policy, you can set by following example:
client.newConsumer() .keySharedPolicy(KeySharedPolicy.stickyHashRange().ranges(Range.of(0, 10))) .subscribe();Details about sticky hash range policy, please see
KeySharedPolicy.KeySharedPolicySticky
.
Or
client.newConsumer() .keySharedPolicy(KeySharedPolicy.autoSplitHashRange()) .subscribe();Details about auto split hash range policy, please see
KeySharedPolicy.KeySharedPolicyAutoSplit
.keySharedPolicy
- The KeySharedPolicy
want to specifyConsumerBuilder<T> startMessageIdInclusive()
or
{@link Consumer#seek(MessageId)}
.ConsumerBuilder<T> batchReceivePolicy(BatchReceivePolicy batchReceivePolicy)
BatchReceivePolicy
for consumer.
By default, consumer will use BatchReceivePolicy.DEFAULT_POLICY
as batch receive policy.
Example:
client.newConsumer().batchReceivePolicy(BatchReceivePolicy.builder() .maxNumMessages(100) .maxNumBytes(5 * 1024 * 1024) .timeout(100, TimeUnit.MILLISECONDS) .build()).subscribe();
ConsumerBuilder<T> enableRetry(boolean retryEnable)
retryEnable
- whether to auto retry messageConsumerBuilder<T> enableBatchIndexAcknowledgment(boolean batchIndexAcknowledgmentEnabled)
ConsumerBuilder<T> maxPendingChuckedMessage(int maxPendingChuckedMessage)
eg: M1-C1, M2-C1, M1-C2, M2-C2 Here, Messages M1-C1 and M1-C2 belong to original message M1, M2-C1 and M2-C2 messages belong to M2 message.Buffering large number of outstanding uncompleted chunked messages can create memory pressure and it can be guarded by providing this @maxPendingChuckedMessage threshold. Once, consumer reaches this threshold, it drops the outstanding unchunked-messages by silently acking or asking broker to redeliver later by marking it unacked. This behavior can be controlled by configuration: @autoAckOldestChunkedMessageOnQueueFull
maxPendingChuckedMessage
- ConsumerBuilder<T> autoAckOldestChunkedMessageOnQueueFull(boolean autoAckOldestChunkedMessageOnQueueFull)
autoAckOldestChunkedMessageOnQueueFull
- ConsumerBuilder<T> expireTimeOfIncompleteChunkedMessage(long duration, TimeUnit unit)
duration
- unit
- Copyright © 2017–2020 Apache Software Foundation. All rights reserved.