org.apache.druid.data.input

Class SegmentsSplitHintSpec

java.lang.Object

org.apache.druid.data.input.SegmentsSplitHintSpec

All Implemented Interfaces:

SplitHintSpec

public class SegmentsSplitHintSpec
extends Object
implements SplitHintSpec

SplitHintSpec for IngestSegmentFirehoseFactory and DruidInputSource. In DruidInputSource, this spec is converted into MaxSizeSplitHintSpec. As a result, its split(java.util.Iterator<T>, java.util.function.Function<T, org.apache.druid.data.input.InputFileAttribute>) method is never called (IngestSegmentFirehoseFactory creates splits on its own instead of calling the split() method). This doesn't necessarily mean this class is deprecated in favor of the MaxSizeSplitHintSpec. We may want to create more optimized splits in the future. For example, segments can be split to maximize the rollup ratio if the segments have different sets of columns or even different value ranges of columns.

Field Summary

Fields

Modifier and Type	Field and Description
static String	TYPE

Constructor Summary

Constructors

Constructor and Description
SegmentsSplitHintSpec(HumanReadableBytes maxInputSegmentBytesPerTask, Integer maxNumSegments)

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object o)
HumanReadableBytes	getMaxInputSegmentBytesPerTask()
int	getMaxNumSegments()
int	hashCode()
<T> Iterator<List<T>>	split(Iterator<T> inputIterator, Function<T,InputFileAttribute> inputAttributeExtractor) Returns an iterator of splits.
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

TYPE

public static final String TYPE

See Also:

Constant Field Values

Constructor Detail

SegmentsSplitHintSpec

public SegmentsSplitHintSpec(@Nullable
                             HumanReadableBytes maxInputSegmentBytesPerTask,
                             @Nullable
                             Integer maxNumSegments)

Method Detail

getMaxInputSegmentBytesPerTask

public HumanReadableBytes getMaxInputSegmentBytesPerTask()

getMaxNumSegments

public int getMaxNumSegments()

split

public <T> Iterator<List<T>> split(Iterator<T> inputIterator,
                                   Function<T,InputFileAttribute> inputAttributeExtractor)

Description copied from interface: SplitHintSpec

Returns an iterator of splits. A split has a list of files of the type T.

Specified by:

split in interface SplitHintSpec

Parameters:

inputIterator - that returns input files.

inputAttributeExtractor - to create InputFileAttribute for each input file. This may involve a network call, so implementations of SplitHintSpec should use it only if needed, and reuse results if appropriate.

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object