org.apache.hadoop.hbase.io.hfile
Class HFileWriterV2

java.lang.Object
  org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter
      org.apache.hadoop.hbase.io.hfile.HFileWriterV2

All Implemented Interfaces:

Closeable, HFile.Writer

Direct Known Subclasses:

HFileWriterV3

@InterfaceAudience.Private
public class HFileWriterV2
extends AbstractHFileWriter

Writes HFile format version 2.

Field Summary

protected HFileBlock.Writer	fsBlockWriter Unified version 2 block writer
static int	KEY_VALUE_VER_WITH_MEMSTORE Version for KeyValue which includes memstore timestamp
static byte[]	KEY_VALUE_VERSION KeyValue version in FileInfo
protected long	lastDataBlockOffset The offset of the last data block or 0 if the file is empty.
static byte[]	MAX_MEMSTORE_TS_KEY Max memstore (mvcc) timestamp in FileInfo
protected long	maxMemstoreTS

Fields inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter

blockEncoder, cacheConf, closeOutputStream, comparator, entryCount, fileInfo, firstKeyInBlock, hFileContext, lastKeyBuffer, lastKeyLength, lastKeyOffset, metaData, metaNames, name, outputStream, path, totalKeyLength, totalUncompressedBytes, totalValueLength

Constructor Summary

HFileWriterV2(org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FSDataOutputStream ostream, KeyValue.KVComparator comparator, HFileContext context)
Constructor that takes a path, creates and closes the output stream.

Method Summary

void	addDeleteFamilyBloomFilter(BloomFilterWriter bfw) Store delete family Bloom filter in the file, which is only supported in HFile V2.
void	addGeneralBloomFilter(BloomFilterWriter bfw) Store general Bloom filter in the file.
void	addInlineBlockWriter(InlineBlockWriter ibw) Adds an inline block writer such as a multi-level block index writer or a compound Bloom filter writer.
void	append(byte[] key, byte[] value) Add key/value to file.
void	append(byte[] key, byte[] value, byte[] tag)
void	append(KeyValue kv) Add key/value to file.
protected void	append(long memstoreTS, byte[] key, int koffset, int klength, byte[] value, int voffset, int vlength) Add key/value to file.
void	appendMetaBlock(String metaBlockName, org.apache.hadoop.io.Writable content) Add a meta block to the end of the file.
protected void	checkBlockBoundary() At a block boundary, write all the inline blocks and opens new block.
void	close()
protected void	finishInit(org.apache.hadoop.conf.Configuration conf) Additional initialization steps
HFileContext	getFileContext() Return the file context for the HFile this writer belongs to
protected int	getMajorVersion()
protected int	getMinorVersion()
protected void	newBlock() Ready a new block for writing.

Methods inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter

appendFileInfo, checkKey, checkValue, compressionByName, createOutputStream, finishClose, finishFileInfo, getPath, toString, writeFileInfo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MAX_MEMSTORE_TS_KEY

public static final byte[] MAX_MEMSTORE_TS_KEY

Max memstore (mvcc) timestamp in FileInfo

KEY_VALUE_VERSION

public static final byte[] KEY_VALUE_VERSION

KeyValue version in FileInfo

KEY_VALUE_VER_WITH_MEMSTORE

public static final int KEY_VALUE_VER_WITH_MEMSTORE

Version for KeyValue which includes memstore timestamp

See Also:

Constant Field Values

fsBlockWriter

protected HFileBlock.Writer fsBlockWriter

Unified version 2 block writer

lastDataBlockOffset

protected long lastDataBlockOffset

The offset of the last data block or 0 if the file is empty.

maxMemstoreTS

protected long maxMemstoreTS

Constructor Detail

HFileWriterV2

public HFileWriterV2(org.apache.hadoop.conf.Configuration conf,
                     CacheConfig cacheConf,
                     org.apache.hadoop.fs.FileSystem fs,
                     org.apache.hadoop.fs.Path path,
                     org.apache.hadoop.fs.FSDataOutputStream ostream,
                     KeyValue.KVComparator comparator,
                     HFileContext context)
              throws IOException

Constructor that takes a path, creates and closes the output stream.

Throws:

IOException

Method Detail

finishInit

protected void finishInit(org.apache.hadoop.conf.Configuration conf)

Additional initialization steps

checkBlockBoundary

protected void checkBlockBoundary()
                           throws IOException

At a block boundary, write all the inline blocks and opens new block.

Throws:

IOException

newBlock

protected void newBlock()
                 throws IOException

Ready a new block for writing.

Throws:

IOException

appendMetaBlock

public void appendMetaBlock(String metaBlockName,
                            org.apache.hadoop.io.Writable content)

Add a meta block to the end of the file. Call before close(). Metadata blocks are expensive. Fill one with a bunch of serialized data rather than do a metadata block per metadata instance. If metadata is small, consider adding to file info using AbstractHFileWriter.appendFileInfo(byte[], byte[])

Parameters:

metaBlockName - name of the block

content - will call readFields to get data later (DO NOT REUSE)

append

public void append(KeyValue kv)
            throws IOException

Add key/value to file. Keys must be added in an order that agrees with the Comparator passed on construction.

Parameters:

kv - KeyValue to add. Cannot be empty nor null.

Throws:

IOException

append

public void append(byte[] key,
                   byte[] value)
            throws IOException

Add key/value to file. Keys must be added in an order that agrees with the Comparator passed on construction.

Parameters:

key - Key to add. Cannot be empty nor null.

value - Value to add. Cannot be empty nor null.

Throws:

IOException

append

protected void append(long memstoreTS,
                      byte[] key,
                      int koffset,
                      int klength,
                      byte[] value,
                      int voffset,
                      int vlength)
               throws IOException

Add key/value to file. Keys must be added in an order that agrees with the Comparator passed on construction.

Parameters:

key -

koffset -

klength -

value -

voffset -

vlength -

Throws:

IOException

close

public void close()
           throws IOException

Throws:

IOException

addInlineBlockWriter

public void addInlineBlockWriter(InlineBlockWriter ibw)

Description copied from interface: HFile.Writer

Adds an inline block writer such as a multi-level block index writer or a compound Bloom filter writer.

addGeneralBloomFilter

public void addGeneralBloomFilter(BloomFilterWriter bfw)

Description copied from interface: HFile.Writer

Store general Bloom filter in the file. This does not deal with Bloom filter internals but is necessary, since Bloom filters are stored differently in HFile version 1 and version 2.

addDeleteFamilyBloomFilter

public void addDeleteFamilyBloomFilter(BloomFilterWriter bfw)

Description copied from interface: HFile.Writer

Store delete family Bloom filter in the file, which is only supported in HFile V2.

append

public void append(byte[] key,
                   byte[] value,
                   byte[] tag)
            throws IOException

Throws:

IOException

getMajorVersion

protected int getMajorVersion()

getMinorVersion

protected int getMinorVersion()

getFileContext

public HFileContext getFileContext()

Description copied from interface: HFile.Writer

Return the file context for the HFile this writer belongs to