Package io.confluent.parallelconsumer
Interface ParallelStreamProcessor<K,V>
- All Superinterfaces:
AutoCloseable
,Closeable
,DrainingCloseable
,ParallelConsumer<K,
V>
- All Known Implementing Classes:
JStreamParallelEoSStreamProcessor
,ParallelEoSStreamProcessor
Parallel message consumer which also can optionally produce 0 or many
ProducerRecord
results to be published
back to Kafka.- See Also:
-
pollAndProduce(java.util.function.Function<io.confluent.parallelconsumer.PollContext<K, V>, org.apache.kafka.clients.producer.ProducerRecord<K, V>>)
pollAndProduceMany(java.util.function.Function<io.confluent.parallelconsumer.PollContext<K, V>, java.util.List<org.apache.kafka.clients.producer.ProducerRecord<K, V>>>, java.util.function.Consumer<io.confluent.parallelconsumer.ParallelStreamProcessor.ConsumeProduceResult<K, V, K, V>>)
-
Nested Class Summary
Modifier and TypeInterfaceDescriptionstatic class
A simple triple structure to capture the set of coinciding data.Nested classes/interfaces inherited from interface io.confluent.parallelconsumer.internal.DrainingCloseable
DrainingCloseable.DrainingMode
Nested classes/interfaces inherited from interface io.confluent.parallelconsumer.ParallelConsumer
ParallelConsumer.Tuple<L,
R> -
Field Summary
Fields inherited from interface io.confluent.parallelconsumer.internal.DrainingCloseable
DEFAULT_TIMEOUT
-
Method Summary
Modifier and TypeMethodDescriptionstatic <KK,
VV> ParallelStreamProcessor<KK, VV> createEosStreamProcessor
(ParallelConsumerOptions<KK, VV> options) void
poll
(Consumer<PollContext<K, V>> usersVoidConsumptionFunction) Register a function to be applied in parallel to each received messagevoid
pollAndProduce
(Function<PollContext<K, V>, org.apache.kafka.clients.producer.ProducerRecord<K, V>> userFunction) Register a function to be applied in parallel to each received message, which in turn returns aProducerRecord
to be sent back to the broker.void
pollAndProduce
(Function<PollContext<K, V>, org.apache.kafka.clients.producer.ProducerRecord<K, V>> userFunction, Consumer<ParallelStreamProcessor.ConsumeProduceResult<K, V, K, V>> callback) Register a function to be applied in parallel to each received message, which in turn returns aProducerRecord
to be sent back to the broker.void
pollAndProduceMany
(Function<PollContext<K, V>, List<org.apache.kafka.clients.producer.ProducerRecord<K, V>>> userFunction) Register a function to be applied in parallel to each received message, which in turn returns one or manyProducerRecord
s to be sent back to the broker.void
pollAndProduceMany
(Function<PollContext<K, V>, List<org.apache.kafka.clients.producer.ProducerRecord<K, V>>> userFunction, Consumer<ParallelStreamProcessor.ConsumeProduceResult<K, V, K, V>> callback) Register a function to be applied in parallel to each received message, which in turn returns one or moreProducerRecord
s to be sent back to the broker.Methods inherited from interface io.confluent.parallelconsumer.internal.DrainingCloseable
close, close, closeDontDrainFirst, closeDontDrainFirst, closeDrainFirst, closeDrainFirst, workRemaining
Methods inherited from interface io.confluent.parallelconsumer.ParallelConsumer
pauseIfRunning, resumeIfPaused, subscribe, subscribe, subscribe, subscribe
-
Method Details
-
createEosStreamProcessor
static <KK,VV> ParallelStreamProcessor<KK,VV> createEosStreamProcessor(ParallelConsumerOptions<KK, VV> options) -
poll
Register a function to be applied in parallel to each received message- Parameters:
usersVoidConsumptionFunction
- the function
-
pollAndProduceMany
void pollAndProduceMany(Function<PollContext<K, V>, List<org.apache.kafka.clients.producer.ProducerRecord<K, V>>> userFunction, Consumer<ParallelStreamProcessor.ConsumeProduceResult<K, V, K, V>> callback) Register a function to be applied in parallel to each received message, which in turn returns one or moreProducerRecord
s to be sent back to the broker.- Parameters:
callback
- applied after the produced message is acknowledged by kafka
-
pollAndProduceMany
void pollAndProduceMany(Function<PollContext<K, V>, List<org.apache.kafka.clients.producer.ProducerRecord<K, V>>> userFunction) Register a function to be applied in parallel to each received message, which in turn returns one or manyProducerRecord
s to be sent back to the broker. -
pollAndProduce
void pollAndProduce(Function<PollContext<K, V>, org.apache.kafka.clients.producer.ProducerRecord<K, V>> userFunction) Register a function to be applied in parallel to each received message, which in turn returns aProducerRecord
to be sent back to the broker. -
pollAndProduce
void pollAndProduce(Function<PollContext<K, V>, org.apache.kafka.clients.producer.ProducerRecord<K, V>> userFunction, Consumer<ParallelStreamProcessor.ConsumeProduceResult<K, V, K, V>> callback) Register a function to be applied in parallel to each received message, which in turn returns aProducerRecord
to be sent back to the broker.- Parameters:
callback
- applied after the produced message is acknowledged by kafka
-