org.apache.druid.java.util.common.guava

Class ParallelMergeCombiningSequence<T>

java.lang.Object

org.apache.druid.java.util.common.guava.YieldingSequenceBase<T>

org.apache.druid.java.util.common.guava.ParallelMergeCombiningSequence<T>

All Implemented Interfaces:

Sequence<T>

public class ParallelMergeCombiningSequence<T>
extends YieldingSequenceBase<T>

Artisanal, locally-sourced, hand-crafted, gluten and GMO free, bespoke, free-range, organic, small-batch parallel merge combining sequence. See proposal: https://github.com/apache/druid/issues/8577 Functionally equivalent to wrapping CombiningSequence around a MergeSequence, but done in parallel on a ForkJoinPool running in 'async' mode.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ParallelMergeCombiningSequence.MergeCombineMetrics Metrics for the execution of a ParallelMergeCombiningSequence on the ForkJoinPool

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_TASK_INITIAL_YIELD_NUM_ROWS
static int	DEFAULT_TASK_SMALL_BATCH_NUM_ROWS
static int	DEFAULT_TASK_TARGET_RUN_TIME_MILLIS

Constructor Summary

Constructors

Constructor and Description
ParallelMergeCombiningSequence(ForkJoinPool workerPool, List<Sequence<T>> inputSequences, com.google.common.collect.Ordering<T> orderingFn, BinaryOperator<T> combineFn, boolean hasTimeout, long timeoutMillis, int queryPriority, int parallelism, int yieldAfter, int batchSize, int targetTimeMillis, Consumer<ParallelMergeCombiningSequence.MergeCombineMetrics> reporter)

Method Summary

Modifier and Type	Method and Description
org.apache.druid.java.util.common.guava.ParallelMergeCombiningSequence.CancellationGizmo	getCancellationGizmo()
<OutType> Yielder<OutType>	toYielder(OutType initValue, YieldingAccumulator<OutType,T> accumulator) Return a Yielder for accumulated sequence.

Methods inherited from class org.apache.druid.java.util.common.guava.YieldingSequenceBase

accumulate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.druid.java.util.common.guava.Sequence

filter, flatMap, flatMerge, forEach, limit, map, skip, toList, withBaggage, withEffect

Field Detail

DEFAULT_TASK_TARGET_RUN_TIME_MILLIS

public static final int DEFAULT_TASK_TARGET_RUN_TIME_MILLIS

See Also:

Constant Field Values

DEFAULT_TASK_INITIAL_YIELD_NUM_ROWS

public static final int DEFAULT_TASK_INITIAL_YIELD_NUM_ROWS

See Also:

Constant Field Values

DEFAULT_TASK_SMALL_BATCH_NUM_ROWS

public static final int DEFAULT_TASK_SMALL_BATCH_NUM_ROWS

See Also:

Constant Field Values

Constructor Detail

ParallelMergeCombiningSequence

public ParallelMergeCombiningSequence(ForkJoinPool workerPool,
                                      List<Sequence<T>> inputSequences,
                                      com.google.common.collect.Ordering<T> orderingFn,
                                      BinaryOperator<T> combineFn,
                                      boolean hasTimeout,
                                      long timeoutMillis,
                                      int queryPriority,
                                      int parallelism,
                                      int yieldAfter,
                                      int batchSize,
                                      int targetTimeMillis,
                                      Consumer<ParallelMergeCombiningSequence.MergeCombineMetrics> reporter)

Method Detail

toYielder

public <OutType> Yielder<OutType> toYielder(OutType initValue,
                                            YieldingAccumulator<OutType,T> accumulator)

Description copied from interface: Sequence

Return a Yielder for accumulated sequence.

Type Parameters:

OutType - the type of accumulated value.

Parameters:

initValue - the initial value to pass along to start the accumulation.

accumulator - the accumulator which is responsible for accumulating input values.

Returns:

a Yielder for accumulated sequence.

See Also:

Yielder

getCancellationGizmo

public org.apache.druid.java.util.common.guava.ParallelMergeCombiningSequence.CancellationGizmo getCancellationGizmo()