Package org.apache.druid.segment.nested
Class NestedCommonFormatColumnSerializer
- java.lang.Object
-
- org.apache.druid.segment.nested.NestedCommonFormatColumnSerializer
-
- All Implemented Interfaces:
GenericColumnSerializer<StructuredData>
,Serializer
- Direct Known Subclasses:
NestedDataColumnSerializer
,ScalarNestedCommonFormatColumnSerializer
,VariantColumnSerializer
public abstract class NestedCommonFormatColumnSerializer extends Object implements GenericColumnSerializer<StructuredData>
Basic serializer implementation for theNestedCommonFormatColumn
family of columns. TheAutoTypeColumnIndexer
catalogs the types and fields present in the data it processes using aStructuredDataProcessor
. When persisting and merging segments, theAutoTypeColumnMerger
will choose the most appropriate serializer based on the data which was processed as follows:
-
-
Field Summary
Fields Modifier and Type Field Description static String
ARRAY_DICTIONARY_FILE_NAME
static String
ARRAY_ELEMENT_BITMAP_INDEX_FILE_NAME
static String
ARRAY_ELEMENT_DICTIONARY_FILE_NAME
static String
BITMAP_INDEX_FILE_NAME
static String
DOUBLE_DICTIONARY_FILE_NAME
static String
DOUBLE_VALUE_COLUMN_FILE_NAME
static String
ENCODED_VALUE_COLUMN_FILE_NAME
static String
LONG_DICTIONARY_FILE_NAME
static String
LONG_VALUE_COLUMN_FILE_NAME
static String
NESTED_FIELD_PREFIX
static String
NULL_BITMAP_FILE_NAME
static String
RAW_FILE_NAME
static String
STRING_DICTIONARY_FILE_NAME
static byte
V0
-
Constructor Summary
Constructors Constructor Description NestedCommonFormatColumnSerializer()
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected ByteBuffer
computeFilenameBytes()
abstract String
getColumnName()
abstract DictionaryIdLookup
getGlobalLookup()
static String
getInternalFileName(String fileNameBase, String field)
Nested field columns are stored in separateabstract boolean
hasNulls()
abstract void
openDictionaryWriter()
abstract void
serializeDictionaries(Iterable<String> strings, Iterable<Long> longs, Iterable<Double> doubles, Iterable<int[]> arrays)
void
serializeFields(SortedMap<String,FieldTypeInfo.MutableTypeSet> fields)
protected void
writeInternal(FileSmoosher smoosher, ByteBuffer buffer, String fileName)
protected void
writeInternal(FileSmoosher smoosher, Serializer serializer, String fileName)
protected void
writeV0Header(WritableByteChannel channel, ByteBuffer columnNameBuffer)
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.druid.segment.GenericColumnSerializer
open, serialize
-
Methods inherited from interface org.apache.druid.segment.serde.Serializer
getSerializedSize, writeTo
-
-
-
-
Field Detail
-
V0
public static final byte V0
- See Also:
- Constant Field Values
-
STRING_DICTIONARY_FILE_NAME
public static final String STRING_DICTIONARY_FILE_NAME
- See Also:
- Constant Field Values
-
LONG_DICTIONARY_FILE_NAME
public static final String LONG_DICTIONARY_FILE_NAME
- See Also:
- Constant Field Values
-
DOUBLE_DICTIONARY_FILE_NAME
public static final String DOUBLE_DICTIONARY_FILE_NAME
- See Also:
- Constant Field Values
-
ARRAY_DICTIONARY_FILE_NAME
public static final String ARRAY_DICTIONARY_FILE_NAME
- See Also:
- Constant Field Values
-
ARRAY_ELEMENT_DICTIONARY_FILE_NAME
public static final String ARRAY_ELEMENT_DICTIONARY_FILE_NAME
- See Also:
- Constant Field Values
-
ENCODED_VALUE_COLUMN_FILE_NAME
public static final String ENCODED_VALUE_COLUMN_FILE_NAME
- See Also:
- Constant Field Values
-
LONG_VALUE_COLUMN_FILE_NAME
public static final String LONG_VALUE_COLUMN_FILE_NAME
- See Also:
- Constant Field Values
-
DOUBLE_VALUE_COLUMN_FILE_NAME
public static final String DOUBLE_VALUE_COLUMN_FILE_NAME
- See Also:
- Constant Field Values
-
BITMAP_INDEX_FILE_NAME
public static final String BITMAP_INDEX_FILE_NAME
- See Also:
- Constant Field Values
-
ARRAY_ELEMENT_BITMAP_INDEX_FILE_NAME
public static final String ARRAY_ELEMENT_BITMAP_INDEX_FILE_NAME
- See Also:
- Constant Field Values
-
RAW_FILE_NAME
public static final String RAW_FILE_NAME
- See Also:
- Constant Field Values
-
NULL_BITMAP_FILE_NAME
public static final String NULL_BITMAP_FILE_NAME
- See Also:
- Constant Field Values
-
NESTED_FIELD_PREFIX
public static final String NESTED_FIELD_PREFIX
- See Also:
- Constant Field Values
-
-
Method Detail
-
openDictionaryWriter
public abstract void openDictionaryWriter() throws IOException
- Throws:
IOException
-
serializeFields
public void serializeFields(SortedMap<String,FieldTypeInfo.MutableTypeSet> fields) throws IOException
- Throws:
IOException
-
serializeDictionaries
public abstract void serializeDictionaries(Iterable<String> strings, Iterable<Long> longs, Iterable<Double> doubles, Iterable<int[]> arrays) throws IOException
- Throws:
IOException
-
getColumnName
public abstract String getColumnName()
-
getGlobalLookup
public abstract DictionaryIdLookup getGlobalLookup()
-
hasNulls
public abstract boolean hasNulls()
-
writeInternal
protected void writeInternal(FileSmoosher smoosher, Serializer serializer, String fileName) throws IOException
- Throws:
IOException
-
writeInternal
protected void writeInternal(FileSmoosher smoosher, ByteBuffer buffer, String fileName) throws IOException
- Throws:
IOException
-
writeV0Header
protected void writeV0Header(WritableByteChannel channel, ByteBuffer columnNameBuffer) throws IOException
- Throws:
IOException
-
computeFilenameBytes
protected ByteBuffer computeFilenameBytes()
-
-