Class StreamingMergeSortedGrouper<KeyType>
- java.lang.Object
-
- org.apache.druid.query.groupby.epinephelinae.StreamingMergeSortedGrouper<KeyType>
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Grouper<KeyType>
public class StreamingMergeSortedGrouper<KeyType> extends Object implements Grouper<KeyType>
A streaming grouper which can aggregate sorted inputs. This grouper can aggregate while its iterator is being consumed. The aggregation thread and the iterating thread can be different. This grouper is backed by an off-heap circular array. The reading thread is able to read data from an array slot only when aggregation for the grouping key correspoing to that slot is finished. Since the reading and writing threads cannot access the same array slot at the same time, they can read/write data without contention. This class uses the spinlock for waiting for at least one slot to become available when the array is empty or full. If the array is empty, the reading thread waits for the aggregation for an array slot is finished. If the array is full, the writing thread waits for the reading thread to read at least one aggregate from the array.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.apache.druid.query.groupby.epinephelinae.Grouper
Grouper.BufferComparator, Grouper.Entry<T>, Grouper.KeySerde<T>, Grouper.KeySerdeFactory<T>
-
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description AggregateResult
aggregate(KeyType key)
Aggregate the current row with the provided key.AggregateResult
aggregate(KeyType key, int notUsed)
Aggregate the current row with the provided key.void
close()
Close the grouper and release associated resources.void
finish()
Signal that no more inputs are added.void
init()
Initialize the grouper.boolean
isInitialized()
Check this grouper is initialized or not.CloseableIterator<Grouper.Entry<KeyType>>
iterator()
Return a sorted iterator.CloseableIterator<Grouper.Entry<KeyType>>
iterator(boolean sorted)
Return a sorted iterator.static <KeyType> int
requiredBufferCapacity(Grouper.KeySerde<KeyType> keySerde, AggregatorFactory[] aggregatorFactories)
Returns the minimum buffer capacity required for this grouper.void
reset()
Reset the grouper to its initial state.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.druid.query.groupby.epinephelinae.Grouper
hashFunction
-
-
-
-
Method Detail
-
requiredBufferCapacity
public static <KeyType> int requiredBufferCapacity(Grouper.KeySerde<KeyType> keySerde, AggregatorFactory[] aggregatorFactories)
Returns the minimum buffer capacity required for this grouper. This grouper keeps track read/write indexes and they cannot point the same array slot at the same time. Since the read/write indexes move circularly, one extra slot is needed in addition to the read/write slots. Finally, the required minimum buffer capacity is 3 * record size.- Returns:
- required minimum buffer capacity
-
init
public void init()
Description copied from interface:Grouper
Initialize the grouper. This method needs to be called before callingGrouper.aggregate(Object)
andGrouper.aggregate(Object, int)
.
-
isInitialized
public boolean isInitialized()
Description copied from interface:Grouper
Check this grouper is initialized or not.- Specified by:
isInitialized
in interfaceGrouper<KeyType>
- Returns:
- true if the grouper is already initialized, otherwise false.
-
aggregate
public AggregateResult aggregate(KeyType key, int notUsed)
Description copied from interface:Grouper
Aggregate the current row with the provided key. Some implementations are thread-safe and some are not.- Specified by:
aggregate
in interfaceGrouper<KeyType>
- Parameters:
key
- key objectnotUsed
- result ofGrouper.hashFunction()
on the key- Returns:
- result that is ok if the row was aggregated, not ok if a resource limit was hit
-
aggregate
public AggregateResult aggregate(KeyType key)
Description copied from interface:Grouper
Aggregate the current row with the provided key. Some implementations are thread-safe and some are not.
-
reset
public void reset()
Description copied from interface:Grouper
Reset the grouper to its initial state.
-
close
public void close()
Description copied from interface:Grouper
Close the grouper and release associated resources.
-
finish
public void finish()
Signal that no more inputs are added. Must be called afteraggregate(Object)
is called for the last input.
-
iterator
public CloseableIterator<Grouper.Entry<KeyType>> iterator()
Return a sorted iterator. This method can be called safely while writing, and the iterating thread and the writing thread can be different. The result iterator always returns sorted results. This method should be called only one time per grouper.- Returns:
- a sorted iterator
-
iterator
public CloseableIterator<Grouper.Entry<KeyType>> iterator(boolean sorted)
Return a sorted iterator. This method can be called safely while writing and iterating thread and writing thread can be different. The result iterator always returns sorted results. This method should be called only one time per grouper.
-
-