org.apache.druid.frame.file

Class FrameFile

java.lang.Object

org.apache.druid.frame.file.FrameFile

All Implemented Interfaces:

Closeable, AutoCloseable

public class FrameFile
extends Object
implements Closeable

A file containing Frame data. Frame files are written by FrameFileWriter. Frame files can optionally be partitioned, by providing partition numbers to the FrameFileWriter.writeFrame(org.apache.druid.frame.Frame, int) method when creating the file. Partitions are contiguous within the frame file. Frame files can contain up to Integer.MAX_VALUE frames. Generally, frames are on the order of 1 MB in size, so this allows well over a petabyte of data per file. Ought to be enough for anyone. Format: - 2 bytes: FrameFileWriter.MAGIC - NNN bytes: sequence of FrameFileWriter.MARKER_FRAME followed by one compressed frame (see Frame) - 1 byte: FrameFileWriter.MARKER_NO_MORE_FRAMES - 4 bytes * numPartitions: end frame number of each partition (exclusive), as little-endian ints. Note that partitions may be empty. In this case, certain adjacent values in this array will be equal. Only present if the file is partitioned. - 8 bytes * numFrames: end of each compressed frame (exclusive), relative to start of file, as little-endian longs - 4 bytes: number of frames, as little-endian int - 4 bytes: number of partitions, as little-endian int - 4 bytes: length of footer, from FrameFileWriter.MARKER_NO_MORE_FRAMES to EOF - 4 bytes: checksum of footer (xxhash64, truncated to 32 bits), not considering these final 4 bytes Instances of this class are not thread-safe. For sharing across threads, use newReference() to create an additional reference.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FrameFile.Flag

Method Summary

Modifier and Type	Method and Description
void	close()
File	file() Returns the file that this instance is backed by.
Frame	frame(int frameNumber) Reads a frame from the file.
int	getPartitionStartFrame(int partition) First frame of a given partition.
FrameFile	newReference() Creates a new reference to this file.
int	numFrames() Number of frames in the file.
int	numPartitions() Number of partitions in the file, or zero if the file is unpartitioned.
static FrameFile	open(File file, ByteTracker byteTracker, FrameFile.Flag... flags) Open a frame file with certain optional flags.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

open

public static FrameFile open(File file,
                             @Nullable
                             ByteTracker byteTracker,
                             FrameFile.Flag... flags)
                      throws IOException

Open a frame file with certain optional flags.

Parameters:

file - ƒrame file

byteTracker -

flags - optional flags

Throws:

IOException

numFrames

public int numFrames()

Number of frames in the file.

numPartitions

public int numPartitions()

Number of partitions in the file, or zero if the file is unpartitioned.

getPartitionStartFrame

public int getPartitionStartFrame(int partition)

First frame of a given partition. Partitions beyond numPartitions() are treated as empty: if provided, this method returns numFrames().

frame

public Frame frame(int frameNumber)

Reads a frame from the file.

newReference

public FrameFile newReference()

Creates a new reference to this file. Calling close() releases the reference. The original file is closed when it, and all additional references, are closed. The new FrameFile instance may be used concurrently with the original FrameFile instance.

file

public File file()

Returns the file that this instance is backed by.

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException