Package org.apache.druid.frame.file
Class FrameFile
- java.lang.Object
-
- org.apache.druid.frame.file.FrameFile
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
public class FrameFile extends Object implements Closeable
A file containingFrame
data. Frame files are written byFrameFileWriter
. Frame files can optionally be partitioned, by providing partition numbers to theFrameFileWriter.writeFrame(org.apache.druid.frame.Frame, int)
method when creating the file. Partitions are contiguous within the frame file. Frame files can contain up toInteger.MAX_VALUE
frames. Generally, frames are on the order of 1 MB in size, so this allows well over a petabyte of data per file. Ought to be enough for anyone. Format: - 2 bytes:FrameFileWriter.MAGIC
- NNN bytes: sequence ofFrameFileWriter.MARKER_FRAME
followed by one compressed frame (seeFrame
) - 1 byte:FrameFileWriter.MARKER_NO_MORE_FRAMES
- 4 bytes * numPartitions: end frame number of each partition (exclusive), as little-endian ints. Note that partitions may be empty. In this case, certain adjacent values in this array will be equal. Only present if the file is partitioned. - 8 bytes * numFrames: end of each compressed frame (exclusive), relative to start of file, as little-endian longs - 4 bytes: number of frames, as little-endian int - 4 bytes: number of partitions, as little-endian int - 4 bytes: length of footer, fromFrameFileWriter.MARKER_NO_MORE_FRAMES
to EOF - 4 bytes: checksum of footer (xxhash64, truncated to 32 bits), not considering these final 4 bytes Instances of this class are not thread-safe. For sharing across threads, usenewReference()
to create an additional reference.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
FrameFile.Flag
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
File
file()
Returns the file that this instance is backed by.Frame
frame(int frameNumber)
Reads a frame from the file.int
getPartitionStartFrame(int partition)
First frame of a given partition.FrameFile
newReference()
Creates a new reference to this file.int
numFrames()
Number of frames in the file.int
numPartitions()
Number of partitions in the file, or zero if the file is unpartitioned.static FrameFile
open(File file, ByteTracker byteTracker, FrameFile.Flag... flags)
Open a frame file with certain optional flags.
-
-
-
Method Detail
-
open
public static FrameFile open(File file, @Nullable ByteTracker byteTracker, FrameFile.Flag... flags) throws IOException
Open a frame file with certain optional flags.- Parameters:
file
- ƒrame filebyteTracker
-flags
- optional flags- Throws:
IOException
-
numFrames
public int numFrames()
Number of frames in the file.
-
numPartitions
public int numPartitions()
Number of partitions in the file, or zero if the file is unpartitioned.
-
getPartitionStartFrame
public int getPartitionStartFrame(int partition)
First frame of a given partition. Partitions beyondnumPartitions()
are treated as empty: if provided, this method returnsnumFrames()
.
-
frame
public Frame frame(int frameNumber)
Reads a frame from the file.
-
newReference
public FrameFile newReference()
Creates a new reference to this file. Callingclose()
releases the reference. The original file is closed when it, and all additional references, are closed. The new FrameFile instance may be used concurrently with the original FrameFile instance.
-
file
public File file()
Returns the file that this instance is backed by.
-
close
public void close() throws IOException
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Throws:
IOException
-
-