The AsyncFlushing trait allows outstanding futures to be tracked to completion when the flush() method is called
The AsyncTransformer trait allows async futures to be used to emit records downstreams
FlushingAwareServer must be mixed in to servers that rely on manually controlling when a flush/commit occurs.
FlushingAwareServer must be mixed in to servers that rely on manually controlling when a flush/commit occurs. As such, this trait will be needed when using the following classes, FlushingProcessor, FlushingTransformer, AsyncProcessor, AsyncTransformer, and FinatraTransformer
This trait sets 'kafka.commit.interval' to 'Duration.Top' to disable the normal Kafka Streams commit process. As such the only commits that will occur are triggered manually, thus allowing us to control when flush/commit occurs
The AsyncTransformer trait allows async futures to be used to emit records downstreams
See https://issues.apache.org/jira/browse/KAFKA-6989 for related ticket in Kafka Streams backlog See also: https://stackoverflow.com/questions/42049047/how-to-handle-error-and-dont-commit-when-use-kafka-streams-dsl/42056286#comment92197161_42056286 https://stackoverflow.com/questions/42064430/external-system-queries-during-kafka-stream-processing?noredirect=1&lq=1 https://issues.apache.org/jira/browse/KAFKA-7432
Note: Completed futures add output records to an outstandingResults set. Future's do not directly call context.forward on success since the Processor/Transformer classes have a defined lifecycle which revolve around 2 main processing methods (process/punctuate or transform/punctuate). Kafka Streams then ensures that these methods are never called from 2 threads at the same time. Kafka Streams assumes "forward" would only ever be called from the thread that calls process/transform/punctuate. As such, it could be dangerous to have a Finagle thread calling forward at any time
Note 2: throwIfAsyncFailure is used to fail the Kafka Streams service more quickly than waiting for an eventual failure to occur at the next commit interval. We try to fail fast and a future failure will result in your entire instance shutting down. This default behavior prevents data loss. If you want your service to handle failed futures please use handle/transform on your returned future