Package org.apache.druid.segment.data
Class CompressionFactory
- java.lang.Object
-
- org.apache.druid.segment.data.CompressionFactory
-
public class CompressionFactory extends Object
Compression of metrics is done by using a combination ofCompressionStrategy
and Encoding(such asCompressionFactory.LongEncodingStrategy
for type Long). CompressionStrategy is unaware of the data type and is based on byte operations. It must compress and decompress in block of bytes. Encoding refers to compression method relies on data format, so a different set of Encodings exist for each data type.Storage Format : Byte 1 : version (currently 0x02) Byte 2 - 5 : number of values Byte 6 - 9 : size per block (even if block format isn't used, this is needed for backward compatibility) Byte 10 : compression strategy (contains a flag if there's an encoding byte, see below for how the flag is defined) Byte 11(optional) : encoding type
Encoding specific header (described below)
Block related header (if block compression is used, described in GenericIndexed)
Values
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
CompressionFactory.LongEncodingFormat
static interface
CompressionFactory.LongEncodingReader
static class
CompressionFactory.LongEncodingStrategy
The compression of decompression of encodings are separated into different enums.static interface
CompressionFactory.LongEncodingWriter
This writer output encoded values to the given ByteBuffer or OutputStream.
-
Field Summary
Fields Modifier and Type Field Description static CompressionFactory.LongEncodingStrategy
DEFAULT_LONG_ENCODING_STRATEGY
static byte
DELTA_ENCODING_VERSION
Delta Encoding Header v1: Byte 1 : version Byte 2 - 9 : base value Byte 10 - 13 : number of bits per valuestatic CompressionFactory.LongEncodingFormat
LEGACY_LONG_ENCODING_FORMAT
static int
MAX_TABLE_SIZE
static byte
TABLE_ENCODING_VERSION
Table Encoding Header v1 : Byte 1 : version Byte 2 - 5 : table size Byte 6 - (6 + 8 * table size - 1) : table of encoding, where the ith 8-byte value is encoded as i
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static byte
clearEncodingFlag(byte strategyId)
static ColumnarDoublesSerializer
getDoubleSerializer(String columnName, SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, ByteOrder byteOrder, CompressionStrategy compression)
static com.google.common.base.Supplier<ColumnarDoubles>
getDoubleSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder byteOrder, CompressionStrategy strategy)
static ColumnarFloatsSerializer
getFloatSerializer(String columnName, SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, ByteOrder order, CompressionStrategy compressionStrategy)
static com.google.common.base.Supplier<ColumnarFloats>
getFloatSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder order, CompressionStrategy strategy)
static ColumnarLongsSerializer
getLongSerializer(String columnName, SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, ByteOrder order, CompressionFactory.LongEncodingStrategy encodingStrategy, CompressionStrategy compressionStrategy)
static com.google.common.base.Supplier<ColumnarLongs>
getLongSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder order, CompressionFactory.LongEncodingFormat encodingFormat, CompressionStrategy strategy)
static boolean
hasEncodingFlag(byte strategyId)
static byte
setEncodingFlag(byte strategyId)
-
-
-
Field Detail
-
DEFAULT_LONG_ENCODING_STRATEGY
public static final CompressionFactory.LongEncodingStrategy DEFAULT_LONG_ENCODING_STRATEGY
-
LEGACY_LONG_ENCODING_FORMAT
public static final CompressionFactory.LongEncodingFormat LEGACY_LONG_ENCODING_FORMAT
-
DELTA_ENCODING_VERSION
public static final byte DELTA_ENCODING_VERSION
Delta Encoding Header v1: Byte 1 : version Byte 2 - 9 : base value Byte 10 - 13 : number of bits per value- See Also:
- Constant Field Values
-
TABLE_ENCODING_VERSION
public static final byte TABLE_ENCODING_VERSION
Table Encoding Header v1 : Byte 1 : version Byte 2 - 5 : table size Byte 6 - (6 + 8 * table size - 1) : table of encoding, where the ith 8-byte value is encoded as i- See Also:
- Constant Field Values
-
MAX_TABLE_SIZE
public static final int MAX_TABLE_SIZE
- See Also:
- Constant Field Values
-
-
Method Detail
-
hasEncodingFlag
public static boolean hasEncodingFlag(byte strategyId)
-
setEncodingFlag
public static byte setEncodingFlag(byte strategyId)
-
clearEncodingFlag
public static byte clearEncodingFlag(byte strategyId)
-
getLongSupplier
public static com.google.common.base.Supplier<ColumnarLongs> getLongSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder order, CompressionFactory.LongEncodingFormat encodingFormat, CompressionStrategy strategy)
-
getLongSerializer
public static ColumnarLongsSerializer getLongSerializer(String columnName, SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, ByteOrder order, CompressionFactory.LongEncodingStrategy encodingStrategy, CompressionStrategy compressionStrategy)
-
getFloatSupplier
public static com.google.common.base.Supplier<ColumnarFloats> getFloatSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder order, CompressionStrategy strategy)
-
getFloatSerializer
public static ColumnarFloatsSerializer getFloatSerializer(String columnName, SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, ByteOrder order, CompressionStrategy compressionStrategy)
-
getDoubleSupplier
public static com.google.common.base.Supplier<ColumnarDoubles> getDoubleSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder byteOrder, CompressionStrategy strategy)
-
getDoubleSerializer
public static ColumnarDoublesSerializer getDoubleSerializer(String columnName, SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, ByteOrder byteOrder, CompressionStrategy compression)
-
-