org.apache.druid.query.topn

Class PooledTopNAlgorithm

java.lang.Object

org.apache.druid.query.topn.BaseTopNAlgorithm<int[],BufferAggregator[],PooledTopNAlgorithm.PooledTopNParams>

org.apache.druid.query.topn.PooledTopNAlgorithm

All Implemented Interfaces:

TopNAlgorithm<int[],PooledTopNAlgorithm.PooledTopNParams>

public class PooledTopNAlgorithm
extends BaseTopNAlgorithm<int[],BufferAggregator[],PooledTopNAlgorithm.PooledTopNParams>

This TopNAlgorithm is highly specialized for processing aggregates on string columns that are ColumnCapabilities.isDictionaryEncoded() and ColumnCapabilities.areDictionaryValuesUnique(). This algorithm is built around using a direct ByteBuffer from the 'processing pool' of intermediary results buffers, to aggregate using the dictionary id directly as the key, to defer looking up the value until is necessary. At runtime, this implementation is specialized with wizardry to optimize for processing common top-n query shapes, see computeSpecializedScanAndAggregateImplementations(), Generic1AggPooledTopNScanner and Generic1AggPooledTopNScannerPrototype, Generic2AggPooledTopNScanner and Generic2AggPooledTopNScannerPrototype, CalledFromHotLoop, HotLoopCallee, RuntimeShapeInspector for more details.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PooledTopNAlgorithm.PooledTopNParams

Nested classes/interfaces inherited from class org.apache.druid.query.topn.BaseTopNAlgorithm

BaseTopNAlgorithm.AggregatorArrayProvider, BaseTopNAlgorithm.BaseArrayProvider<T>

Field Summary

Fields inherited from class org.apache.druid.query.topn.BaseTopNAlgorithm

storageAdapter

Fields inherited from interface org.apache.druid.query.topn.TopNAlgorithm

EMPTY_ARRAY, INIT_POSITION_VALUE, SKIP_POSITION_VALUE

Constructor Summary

Constructors

Constructor and Description
PooledTopNAlgorithm(StorageAdapter storageAdapter, TopNQuery query, NonBlockingPool<ByteBuffer> bufferPool)

Method Summary

Modifier and Type	Method and Description
void	cleanup(PooledTopNAlgorithm.PooledTopNParams params)
protected void	closeAggregators(BufferAggregator[] bufferAggregators)
protected int	computeNewLength(int[] dimValSelector, int numProcessed, int numToProcess) Skip invalid value, calculate length to have enough valid value to process or hit the end.
protected BufferAggregator[]	makeDimValAggregateStore(PooledTopNAlgorithm.PooledTopNParams params)
protected int[]	makeDimValSelector(PooledTopNAlgorithm.PooledTopNParams params, int numProcessed, int numToProcess)
PooledTopNAlgorithm.PooledTopNParams	makeInitParams(ColumnSelectorPlus selectorPlus, Cursor cursor)
protected long	scanAndAggregate(PooledTopNAlgorithm.PooledTopNParams params, int[] positions, BufferAggregator[] theAggregators) Returns the number of processed rows (i.
protected int[]	updateDimValSelector(int[] dimValSelector, int numProcessed, int numToProcess)
protected void	updateResults(PooledTopNAlgorithm.PooledTopNParams params, int[] positions, BufferAggregator[] theAggregators, TopNResultBuilder resultBuilder)

Methods inherited from class org.apache.druid.query.topn.BaseTopNAlgorithm

makeAggregators, makeBufferAggregators, makeResultBuilder, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PooledTopNAlgorithm

public PooledTopNAlgorithm(StorageAdapter storageAdapter,
                           TopNQuery query,
                           NonBlockingPool<ByteBuffer> bufferPool)

Method Detail

makeInitParams

public PooledTopNAlgorithm.PooledTopNParams makeInitParams(ColumnSelectorPlus selectorPlus,
                                                           Cursor cursor)

makeDimValSelector

protected int[] makeDimValSelector(PooledTopNAlgorithm.PooledTopNParams params,
                                   int numProcessed,
                                   int numToProcess)

Specified by:

makeDimValSelector in class BaseTopNAlgorithm<int[],BufferAggregator[],PooledTopNAlgorithm.PooledTopNParams>

computeNewLength

protected int computeNewLength(int[] dimValSelector,
                               int numProcessed,
                               int numToProcess)

Description copied from class: BaseTopNAlgorithm

Skip invalid value, calculate length to have enough valid value to process or hit the end.

Overrides:

computeNewLength in class BaseTopNAlgorithm<int[],BufferAggregator[],PooledTopNAlgorithm.PooledTopNParams>

Parameters:

dimValSelector - the dim value selector which record value is valid or invalid.

numProcessed - the start position to process

numToProcess - the number of valid value to process

Returns:

the length between which have enough valid value to process or hit the end.

updateDimValSelector

protected int[] updateDimValSelector(int[] dimValSelector,
                                     int numProcessed,
                                     int numToProcess)

Specified by:

updateDimValSelector in class BaseTopNAlgorithm<int[],BufferAggregator[],PooledTopNAlgorithm.PooledTopNParams>

makeDimValAggregateStore

protected BufferAggregator[] makeDimValAggregateStore(PooledTopNAlgorithm.PooledTopNParams params)

Specified by:

makeDimValAggregateStore in class BaseTopNAlgorithm<int[],BufferAggregator[],PooledTopNAlgorithm.PooledTopNParams>

scanAndAggregate

protected long scanAndAggregate(PooledTopNAlgorithm.PooledTopNParams params,
                                int[] positions,
                                BufferAggregator[] theAggregators)

Description copied from class: BaseTopNAlgorithm

Returns the number of processed rows (i. e. after postFilters are applied inside the cursor being processed).

Specified by:

scanAndAggregate in class BaseTopNAlgorithm<int[],BufferAggregator[],PooledTopNAlgorithm.PooledTopNParams>

updateResults

protected void updateResults(PooledTopNAlgorithm.PooledTopNParams params,
                             int[] positions,
                             BufferAggregator[] theAggregators,
                             TopNResultBuilder resultBuilder)

Specified by:

updateResults in class BaseTopNAlgorithm<int[],BufferAggregator[],PooledTopNAlgorithm.PooledTopNParams>

closeAggregators

protected void closeAggregators(BufferAggregator[] bufferAggregators)

Specified by:

closeAggregators in class BaseTopNAlgorithm<int[],BufferAggregator[],PooledTopNAlgorithm.PooledTopNParams>

cleanup

public void cleanup(PooledTopNAlgorithm.PooledTopNParams params)