Package org.apache.druid.segment.data
Class FrontCodedIndexedWriter
- java.lang.Object
-
- org.apache.druid.segment.data.FrontCodedIndexedWriter
-
- All Implemented Interfaces:
DictionaryWriter<byte[]>
,Serializer
public class FrontCodedIndexedWriter extends Object implements DictionaryWriter<byte[]>
DictionaryWriter
for aFrontCodedIndexed
, written to aSegmentWriteOutMedium
. Values MUST be added to this dictionary writer in sorted order, which is enforced. Front coding is a type of delta encoding for byte arrays, where values are grouped into buckets. The first value of the bucket is written entirely, and remaining values are stored as pairs of an integer which indicates how much of the first byte array of the bucket to use as a prefix, (or the preceding value of the bucket if using 'incremental' buckets) followed by the remaining value bytes after the prefix. This writer is designed for use with UTF-8 encoded strings that are written in an order compatible withString.compareTo(String)
.- See Also:
for additional details.
-
-
Constructor Summary
Constructors Constructor Description FrontCodedIndexedWriter(SegmentWriteOutMedium segmentWriteOutMedium, ByteOrder byteOrder, int bucketSize, byte version)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description byte[]
get(int index)
int
getCardinality()
long
getSerializedSize()
Returns the number of bytes, that this Serializer will write to the output _channel_ (not smoosher) on aSerializer.writeTo(java.nio.channels.WritableByteChannel, org.apache.druid.java.util.common.io.smoosh.FileSmoosher)
call.boolean
isSorted()
void
open()
void
write(byte[] value)
static int
writeBucketV0(ByteBuffer buffer, byte[][] values, int numValues)
Write bucket of values to aByteBuffer
.static int
writeBucketV1(ByteBuffer buffer, byte[][] values, int numValues)
Write bucket of values to aByteBuffer
.void
writeTo(WritableByteChannel channel, FileSmoosher smoosher)
Writes serialized form of this object to the given channel.static int
writeValue(ByteBuffer buffer, byte[] bytes)
Write a variable length byte[] value to aByteBuffer
, storing the length as aVByte
encoded integer followed by the value itself.
-
-
-
Constructor Detail
-
FrontCodedIndexedWriter
public FrontCodedIndexedWriter(SegmentWriteOutMedium segmentWriteOutMedium, ByteOrder byteOrder, int bucketSize, byte version)
-
-
Method Detail
-
open
public void open() throws IOException
- Specified by:
open
in interfaceDictionaryWriter<byte[]>
- Throws:
IOException
-
write
public void write(@Nullable byte[] value) throws IOException
- Specified by:
write
in interfaceDictionaryWriter<byte[]>
- Throws:
IOException
-
getSerializedSize
public long getSerializedSize() throws IOException
Description copied from interface:Serializer
Returns the number of bytes, that this Serializer will write to the output _channel_ (not smoosher) on aSerializer.writeTo(java.nio.channels.WritableByteChannel, org.apache.druid.java.util.common.io.smoosh.FileSmoosher)
call.- Specified by:
getSerializedSize
in interfaceSerializer
- Throws:
IOException
-
writeTo
public void writeTo(WritableByteChannel channel, FileSmoosher smoosher) throws IOException
Description copied from interface:Serializer
Writes serialized form of this object to the given channel. If parallel data streams are needed, they could be created with the provided smoosher.- Specified by:
writeTo
in interfaceSerializer
- Throws:
IOException
-
isSorted
public boolean isSorted()
- Specified by:
isSorted
in interfaceDictionaryWriter<byte[]>
-
get
@Nullable public byte[] get(int index) throws IOException
- Specified by:
get
in interfaceDictionaryWriter<byte[]>
- Throws:
IOException
-
getCardinality
public int getCardinality()
- Specified by:
getCardinality
in interfaceDictionaryWriter<byte[]>
-
writeBucketV0
public static int writeBucketV0(ByteBuffer buffer, byte[][] values, int numValues)
Write bucket of values to aByteBuffer
. The first value is written completely, subsequent values are written with an integer to indicate how much of the first value in the bucket is a prefix of the value, followed by the remaining bytes of the value. UsesVByte
encoded integers to indicate prefix length and value length.
-
writeBucketV1
public static int writeBucketV1(ByteBuffer buffer, byte[][] values, int numValues)
Write bucket of values to aByteBuffer
. The first value is written completely, subsequent values are written with an integer to indicate how much of the preceding value in the bucket is a prefix of the value, followed by the remaining bytes of the value. UsesVByte
encoded integers to indicate prefix length and value length.
-
writeValue
public static int writeValue(ByteBuffer buffer, byte[] bytes)
Write a variable length byte[] value to aByteBuffer
, storing the length as aVByte
encoded integer followed by the value itself. Returns the number of bytes written to the buffer. This method returns a negative value if there is no room available in the buffer, so that it can be grown if needed.
-
-