org.apache.druid.segment.nested

Class NestedCommonFormatColumnSerializer

java.lang.Object

org.apache.druid.segment.nested.NestedCommonFormatColumnSerializer

All Implemented Interfaces:

GenericColumnSerializer<StructuredData>, Serializer

Direct Known Subclasses:

NestedDataColumnSerializer, ScalarNestedCommonFormatColumnSerializer, VariantColumnSerializer

public abstract class NestedCommonFormatColumnSerializer
extends Object
implements GenericColumnSerializer<StructuredData>

Basic serializer implementation for the NestedCommonFormatColumn family of columns. The AutoTypeColumnIndexer catalogs the types and fields present in the data it processes using a StructuredDataProcessor. When persisting and merging segments, the AutoTypeColumnMerger will choose the most appropriate serializer based on the data which was processed as follows:

See Also:

- single type double columns, - single type long columns, - single type string columns, - single type array columns of string, long, double, and mixed type columns, - nested columns, - legacy nested column format created by {@link org.apache.druid.segment.NestedDataColumnIndexer} and {@link org.apache.druid.segment.NestedDataColumnMerger}

Field Summary

Fields

Modifier and Type	Field and Description
static String	ARRAY_DICTIONARY_FILE_NAME
static String	ARRAY_ELEMENT_BITMAP_INDEX_FILE_NAME
static String	ARRAY_ELEMENT_DICTIONARY_FILE_NAME
static String	BITMAP_INDEX_FILE_NAME
static String	DOUBLE_DICTIONARY_FILE_NAME
static String	DOUBLE_VALUE_COLUMN_FILE_NAME
static String	ENCODED_VALUE_COLUMN_FILE_NAME
static String	LONG_DICTIONARY_FILE_NAME
static String	LONG_VALUE_COLUMN_FILE_NAME
static String	NESTED_FIELD_PREFIX
static String	NULL_BITMAP_FILE_NAME
static String	RAW_FILE_NAME
static String	STRING_DICTIONARY_FILE_NAME
static byte	V0

Constructor Summary

Constructors

Constructor and Description
NestedCommonFormatColumnSerializer()

Method Summary

Modifier and Type	Method and Description
protected ByteBuffer	computeFilenameBytes()
abstract String	getColumnName()
abstract DictionaryIdLookup	getGlobalLookup()
static String	getInternalFileName(String fileNameBase, String field) Nested field columns are stored in separate
abstract boolean	hasNulls()
abstract void	openDictionaryWriter()
abstract void	serializeDictionaries(Iterable<String> strings, Iterable<Long> longs, Iterable<Double> doubles, Iterable<int[]> arrays)
void	serializeFields(SortedMap<String,FieldTypeInfo.MutableTypeSet> fields)
protected void	writeInternal(FileSmoosher smoosher, Serializer serializer, String fileName)
protected void	writeV0Header(WritableByteChannel channel, ByteBuffer columnNameBuffer)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.druid.segment.GenericColumnSerializer

open, serialize

Methods inherited from interface org.apache.druid.segment.serde.Serializer

getSerializedSize, writeTo

Field Detail

V0

public static final byte V0

See Also:

Constant Field Values

STRING_DICTIONARY_FILE_NAME

public static final String STRING_DICTIONARY_FILE_NAME

See Also:

Constant Field Values

LONG_DICTIONARY_FILE_NAME

public static final String LONG_DICTIONARY_FILE_NAME

See Also:

Constant Field Values

DOUBLE_DICTIONARY_FILE_NAME

public static final String DOUBLE_DICTIONARY_FILE_NAME

See Also:

Constant Field Values

ARRAY_DICTIONARY_FILE_NAME

public static final String ARRAY_DICTIONARY_FILE_NAME

See Also:

Constant Field Values

ARRAY_ELEMENT_DICTIONARY_FILE_NAME

public static final String ARRAY_ELEMENT_DICTIONARY_FILE_NAME

See Also:

Constant Field Values

ENCODED_VALUE_COLUMN_FILE_NAME

public static final String ENCODED_VALUE_COLUMN_FILE_NAME

See Also:

Constant Field Values

LONG_VALUE_COLUMN_FILE_NAME

public static final String LONG_VALUE_COLUMN_FILE_NAME

See Also:

Constant Field Values

DOUBLE_VALUE_COLUMN_FILE_NAME

public static final String DOUBLE_VALUE_COLUMN_FILE_NAME

See Also:

Constant Field Values

BITMAP_INDEX_FILE_NAME

public static final String BITMAP_INDEX_FILE_NAME

See Also:

Constant Field Values

ARRAY_ELEMENT_BITMAP_INDEX_FILE_NAME

public static final String ARRAY_ELEMENT_BITMAP_INDEX_FILE_NAME

See Also:

Constant Field Values

RAW_FILE_NAME

public static final String RAW_FILE_NAME

See Also:

Constant Field Values

NULL_BITMAP_FILE_NAME

public static final String NULL_BITMAP_FILE_NAME

See Also:

Constant Field Values

NESTED_FIELD_PREFIX

public static final String NESTED_FIELD_PREFIX

See Also:

Constant Field Values

Constructor Detail

NestedCommonFormatColumnSerializer

public NestedCommonFormatColumnSerializer()

Method Detail

openDictionaryWriter

public abstract void openDictionaryWriter()
                                   throws IOException

Throws:

IOException

serializeFields

public void serializeFields(SortedMap<String,FieldTypeInfo.MutableTypeSet> fields)
                     throws IOException

Throws:

IOException

serializeDictionaries

public abstract void serializeDictionaries(Iterable<String> strings,
                                           Iterable<Long> longs,
                                           Iterable<Double> doubles,
                                           Iterable<int[]> arrays)
                                    throws IOException

Throws:

IOException

getColumnName

public abstract String getColumnName()

getGlobalLookup

public abstract DictionaryIdLookup getGlobalLookup()

hasNulls

public abstract boolean hasNulls()

writeInternal

protected void writeInternal(FileSmoosher smoosher,
                             Serializer serializer,
                             String fileName)
                      throws IOException

Throws:

IOException

writeV0Header

protected void writeV0Header(WritableByteChannel channel,
                             ByteBuffer columnNameBuffer)
                      throws IOException

Throws:

IOException

computeFilenameBytes

protected ByteBuffer computeFilenameBytes()

getInternalFileName

public static String getInternalFileName(String fileNameBase,
                                         String field)

Nested field columns are stored in separate