Package org.apache.druid.java.util.common.guava

Class ParallelMergeCombiningSequence<T>

java.lang.Object

org.apache.druid.java.util.common.guava.YieldingSequenceBase<T>

org.apache.druid.java.util.common.guava.ParallelMergeCombiningSequence<T>

All Implemented Interfaces:

Sequence<T>

public class ParallelMergeCombiningSequence<T>
extends YieldingSequenceBase<T>

Artisanal, locally-sourced, hand-crafted, gluten and GMO free, bespoke, free-range, organic, small-batch parallel merge combining sequence. See proposal: https://github.com/apache/druid/issues/8577 Functionally equivalent to wrapping CombiningSequence around a MergeSequence, but done in parallel on a ForkJoinPool running in 'async' mode.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ParallelMergeCombiningSequence.CancellationFuture	ListenableFuture that allows ParallelMergeCombiningSequence to be registered with QueryWatcher.registerQueryFuture(org.apache.druid.query.Query<?>, com.google.common.util.concurrent.ListenableFuture<?>) to participate in query cancellation or anything else that has a need to watch the activity on the merge pool.
static class	ParallelMergeCombiningSequence.CancellationGizmo	Token used to stop internal parallel processing across all tasks in the merge pool.
static class	ParallelMergeCombiningSequence.MergeCombineMetrics	Metrics for the execution of a ParallelMergeCombiningSequence on the ForkJoinPool

Field Summary

Fields

Modifier and Type	Field	Description
static int	DEFAULT_TASK_INITIAL_YIELD_NUM_ROWS
static int	DEFAULT_TASK_SMALL_BATCH_NUM_ROWS
static int	DEFAULT_TASK_TARGET_RUN_TIME_MILLIS

Constructor Summary

Constructors

Constructor	Description
ParallelMergeCombiningSequence(ForkJoinPool workerPool, List<Sequence<T>> inputSequences, com.google.common.collect.Ordering<T> orderingFn, BinaryOperator<T> combineFn, boolean hasTimeout, long timeoutMillis, int queryPriority, int parallelism, int yieldAfter, int batchSize, int targetTimeMillis, Consumer<ParallelMergeCombiningSequence.MergeCombineMetrics> reporter)

Method Summary

Modifier and Type	Method	Description
ParallelMergeCombiningSequence.CancellationFuture	getCancellationFuture()
<OutType> Yielder<OutType>	toYielder(OutType initValue, YieldingAccumulator<OutType,T> accumulator)	Return a Yielder for accumulated sequence.

Methods inherited from class org.apache.druid.java.util.common.guava.YieldingSequenceBase

accumulate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.druid.java.util.common.guava.Sequence

filter, flatMap, flatMerge, forEach, limit, map, skip, toList, withBaggage, withEffect

Field Detail

DEFAULT_TASK_TARGET_RUN_TIME_MILLIS

public static final int DEFAULT_TASK_TARGET_RUN_TIME_MILLIS

See Also:

Constant Field Values

DEFAULT_TASK_INITIAL_YIELD_NUM_ROWS

public static final int DEFAULT_TASK_INITIAL_YIELD_NUM_ROWS

See Also:

Constant Field Values

DEFAULT_TASK_SMALL_BATCH_NUM_ROWS

public static final int DEFAULT_TASK_SMALL_BATCH_NUM_ROWS

See Also:

Constant Field Values

Constructor Detail

ParallelMergeCombiningSequence

public ParallelMergeCombiningSequence(ForkJoinPool workerPool,
                                      List<Sequence<T>> inputSequences,
                                      com.google.common.collect.Ordering<T> orderingFn,
                                      BinaryOperator<T> combineFn,
                                      boolean hasTimeout,
                                      long timeoutMillis,
                                      int queryPriority,
                                      int parallelism,
                                      int yieldAfter,
                                      int batchSize,
                                      int targetTimeMillis,
                                      Consumer<ParallelMergeCombiningSequence.MergeCombineMetrics> reporter)

Method Detail

toYielder

public <OutType> Yielder<OutType> toYielder(OutType initValue,
                                            YieldingAccumulator<OutType,T> accumulator)

Description copied from interface: Sequence

Return a Yielder for accumulated sequence.

Type Parameters:

OutType - the type of accumulated value.

Parameters:

initValue - the initial value to pass along to start the accumulation.

accumulator - the accumulator which is responsible for accumulating input values.

Returns:

a Yielder for accumulated sequence.

See Also:

Yielder

getCancellationFuture

public ParallelMergeCombiningSequence.CancellationFuture getCancellationFuture()