Package org.apache.druid.segment.nested

Class GlobalDictionaryEncodedFieldColumnWriter<T>

java.lang.Object

org.apache.druid.segment.nested.GlobalDictionaryEncodedFieldColumnWriter<T>

Direct Known Subclasses:

ScalarDoubleFieldColumnWriter, ScalarLongFieldColumnWriter, ScalarStringFieldColumnWriter, VariantArrayFieldColumnWriter, VariantFieldColumnWriter

public abstract class GlobalDictionaryEncodedFieldColumnWriter<T>
extends Object

Base class for writer of global dictionary encoded nested field columns for NestedDataColumnSerializer. Nested columns are written in multiple passes. The first pass processes the 'raw' nested data with a StructuredDataProcessor which will call addValue(int, Object) for writers of each field which is present. For this type of writer, this entails building a local dictionary (localDictionary) to map into the global dictionary (globalDictionaryIdLookup) and writes this unsorted localId to an intermediate integer column, intermediateValueWriter.

When processing the 'raw' value column is complete, the writeTo(int, FileSmoosher) method will sort the local ids and write them out to a local sorted dictionary, iterate over intermediateValueWriter swapping the unsorted local ids with the sorted ids and writing to the compressed id column writer encodedValueSerializer, building the bitmap indexes along the way.

See Also:

- single type double columns, - single type long columns, - single type string columns, - single type array columns of double, long, or string, - mixed type columns of any combination

Field Summary

Fields

Modifier and Type	Field	Description
protected it.unimi.dsi.fastutil.ints.Int2ObjectRBTreeMap<MutableBitmap>	arrayElements
protected String	columnName
protected int	cursorPosition
protected SingleValueColumnarIntsSerializer	encodedValueSerializer
protected String	fieldName
protected Closer	fieldResourceCloser
protected int	flags
protected DictionaryIdLookup	globalDictionaryIdLookup
protected IndexSpec	indexSpec
protected FixedIndexedIntWriter	intermediateValueWriter
protected LocalDimensionDictionary	localDictionary
protected SegmentWriteOutMedium	segmentWriteOutMedium
protected DictionaryEncodedColumnPartSerde.VERSION	version

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	GlobalDictionaryEncodedFieldColumnWriter(String columnName, String fieldName, SegmentWriteOutMedium segmentWriteOutMedium, IndexSpec indexSpec, DictionaryIdLookup globalDictionaryIdLookup)

Method Summary

Modifier and Type	Method	Description
void	addValue(int row, Object val)	Add a value to the unsorted local dictionary and write to an intermediate column
void	open()	Open the writer so that addValue(int, Object) can be called
void	writeLongAndDoubleColumnLength(WritableByteChannel channel, int longLength, int doubleLength)
void	writeTo(int finalRowCount, FileSmoosher smoosher)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

segmentWriteOutMedium

protected final SegmentWriteOutMedium segmentWriteOutMedium

columnName

protected final String columnName

fieldName

protected final String fieldName

indexSpec

protected final IndexSpec indexSpec

globalDictionaryIdLookup

protected final DictionaryIdLookup globalDictionaryIdLookup

localDictionary

protected final LocalDimensionDictionary localDictionary

arrayElements

protected final it.unimi.dsi.fastutil.ints.Int2ObjectRBTreeMap<MutableBitmap> arrayElements

fieldResourceCloser

protected final Closer fieldResourceCloser

intermediateValueWriter

protected FixedIndexedIntWriter intermediateValueWriter

flags

protected int flags

version

protected DictionaryEncodedColumnPartSerde.VERSION version

encodedValueSerializer

protected SingleValueColumnarIntsSerializer encodedValueSerializer

cursorPosition

protected int cursorPosition

Constructor Detail

GlobalDictionaryEncodedFieldColumnWriter

protected GlobalDictionaryEncodedFieldColumnWriter(String columnName,
                                                   String fieldName,
                                                   SegmentWriteOutMedium segmentWriteOutMedium,
                                                   IndexSpec indexSpec,
                                                   DictionaryIdLookup globalDictionaryIdLookup)

Method Detail

open

public void open()
          throws IOException

Open the writer so that addValue(int, Object) can be called

Throws:

IOException

addValue

public void addValue(int row,
                     Object val)
              throws IOException

Add a value to the unsorted local dictionary and write to an intermediate column

Throws:

IOException

writeTo

public void writeTo(int finalRowCount,
                    FileSmoosher smoosher)
             throws IOException

Throws:

IOException

writeLongAndDoubleColumnLength

public void writeLongAndDoubleColumnLength(WritableByteChannel channel,
                                           int longLength,
                                           int doubleLength)
                                    throws IOException

Throws:

IOException