Class SpillingGrouper<KeyType>
- java.lang.Object
-
- org.apache.druid.query.groupby.epinephelinae.SpillingGrouper<KeyType>
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Grouper<KeyType>
public class SpillingGrouper<KeyType> extends Object implements Grouper<KeyType>
Grouper based around a single underlyingBufferHashGrouper
. Not thread-safe. When the underlying grouper is full, its contents are sorted and written to temporary files using "spillMapper".
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.apache.druid.query.groupby.epinephelinae.Grouper
Grouper.BufferComparator, Grouper.Entry<T>, Grouper.KeySerde<T>, Grouper.KeySerdeFactory<T>
-
-
Constructor Summary
Constructors Constructor Description SpillingGrouper(com.google.common.base.Supplier<ByteBuffer> bufferSupplier, Grouper.KeySerdeFactory<KeyType> keySerdeFactory, ColumnSelectorFactory columnSelectorFactory, AggregatorFactory[] aggregatorFactories, int bufferGrouperMaxSize, float bufferGrouperMaxLoadFactor, int bufferGrouperInitialBuckets, LimitedTemporaryStorage temporaryStorage, com.fasterxml.jackson.databind.ObjectMapper spillMapper, boolean spillingAllowed, DefaultLimitSpec limitSpec, boolean sortHasNonGroupingFields, int mergeBufferSize)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description AggregateResult
aggregate(KeyType key, int keyHash)
Aggregate the current row with the provided key.void
close()
Close the grouper and release associated resources.void
init()
Initialize the grouper.boolean
isInitialized()
Check this grouper is initialized or not.boolean
isSpillingAllowed()
CloseableIterator<Grouper.Entry<KeyType>>
iterator(boolean sorted)
Iterate through entries.List<String>
mergeAndGetDictionary()
Returns a dictionary of string keys added to this grouper.void
reset()
Reset the grouper to its initial state.void
setSpillingAllowed(boolean spillingAllowed)
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.druid.query.groupby.epinephelinae.Grouper
aggregate, hashFunction
-
-
-
-
Constructor Detail
-
SpillingGrouper
public SpillingGrouper(com.google.common.base.Supplier<ByteBuffer> bufferSupplier, Grouper.KeySerdeFactory<KeyType> keySerdeFactory, ColumnSelectorFactory columnSelectorFactory, AggregatorFactory[] aggregatorFactories, int bufferGrouperMaxSize, float bufferGrouperMaxLoadFactor, int bufferGrouperInitialBuckets, LimitedTemporaryStorage temporaryStorage, com.fasterxml.jackson.databind.ObjectMapper spillMapper, boolean spillingAllowed, DefaultLimitSpec limitSpec, boolean sortHasNonGroupingFields, int mergeBufferSize)
-
-
Method Detail
-
init
public void init()
Description copied from interface:Grouper
Initialize the grouper. This method needs to be called before callingGrouper.aggregate(Object)
andGrouper.aggregate(Object, int)
.
-
isInitialized
public boolean isInitialized()
Description copied from interface:Grouper
Check this grouper is initialized or not.- Specified by:
isInitialized
in interfaceGrouper<KeyType>
- Returns:
- true if the grouper is already initialized, otherwise false.
-
aggregate
public AggregateResult aggregate(KeyType key, int keyHash)
Description copied from interface:Grouper
Aggregate the current row with the provided key. Some implementations are thread-safe and some are not.- Specified by:
aggregate
in interfaceGrouper<KeyType>
- Parameters:
key
- key objectkeyHash
- result ofGrouper.hashFunction()
on the key- Returns:
- result that is ok if the row was aggregated, not ok if a resource limit was hit
-
reset
public void reset()
Description copied from interface:Grouper
Reset the grouper to its initial state.
-
close
public void close()
Description copied from interface:Grouper
Close the grouper and release associated resources.
-
mergeAndGetDictionary
public List<String> mergeAndGetDictionary()
Returns a dictionary of string keys added to this grouper. Note that the dictionary of keySerde is spilled on local storage whenever the inner grouper is spilled. If there are spilled dictionaries, this method loads them from disk and returns a merged dictionary.- Returns:
- a dictionary which is a list of unique strings
-
isSpillingAllowed
public boolean isSpillingAllowed()
-
setSpillingAllowed
public void setSpillingAllowed(boolean spillingAllowed)
-
iterator
public CloseableIterator<Grouper.Entry<KeyType>> iterator(boolean sorted)
Description copied from interface:Grouper
Iterate through entries.Some implementations allow writes even after this method is called. After you are done with the iterator returned by this method, you should either call
Grouper.close()
(if you are done with the Grouper) orGrouper.reset()
(if you want to reuse it). Some implementations allow callingGrouper.iterator(boolean)
again if you want another iterator. But, this method must not be called by multiple threads concurrently.If "sorted" is true then the iterator will return sorted results. It will use KeyType's natural ordering on deserialized objects, and will use the
Grouper.KeySerde.bufferComparator()
on serialized objects. Woe be unto you if these comparators are not equivalent.Callers must process and discard the returned
Grouper.Entry
s immediately because some implementations can reuse the key objects.
-
-