org.apache.hadoop.io

Class SequenceFile.Writer

java.lang.Object

org.apache.hadoop.io.SequenceFile.Writer

All Implemented Interfaces:

Closeable, AutoCloseable, Syncable

Enclosing class:

SequenceFile

public static class SequenceFile.Writer
extends Object
implements Closeable, Syncable

Write key/value pairs to a sequence-format file.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	SequenceFile.Writer.Option

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.io.serializer.Serializer	compressedValSerializer
protected org.apache.hadoop.io.serializer.Serializer	keySerializer
protected org.apache.hadoop.io.serializer.Serializer	uncompressedValSerializer

Constructor Summary

Constructors

Constructor and Description
SequenceFile.Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass) Deprecated. Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.
SequenceFile.Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, int bufferSize, short replication, long blockSize, Progressable progress, SequenceFile.Metadata metadata) Deprecated. Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.
SequenceFile.Writer(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, Progressable progress, SequenceFile.Metadata metadata) Deprecated. Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.

Method Summary

Methods

Modifier and Type	Method and Description
void	append(Object key, Object val) Append a key/value pair.
void	append(Writable key, Writable val) Append a key/value pair.
static SequenceFile.Writer.Option	appendIfExists(boolean value)
void	appendRaw(byte[] keyData, int keyOffset, int keyLength, SequenceFile.ValueBytes val)
static SequenceFile.Writer.Option	blockSize(long value)
static SequenceFile.Writer.Option	bufferSize(int value)
void	close() Close the file.
static SequenceFile.Writer.Option	compression(SequenceFile.CompressionType value)
static SequenceFile.Writer.Option	compression(SequenceFile.CompressionType value, CompressionCodec codec)
static SequenceFile.Writer.Option	file(Path value)
CompressionCodec	getCompressionCodec() Returns the compression codec of data in this file.
Class	getKeyClass() Returns the class of keys in this file.
long	getLength() Returns the current length of the output file.
Class	getValueClass() Returns the class of values in this file.
void	hflush() Flush out the data in client's user buffer.
void	hsync() Similar to posix fsync, flush out the data in client's user buffer all the way to the disk device (but the disk may have it in its cache).
static SequenceFile.Writer.Option	keyClass(Class<?> value)
static SequenceFile.Writer.Option	metadata(SequenceFile.Metadata value)
static SequenceFile.Writer.Option	progressable(Progressable value)
static SequenceFile.Writer.Option	replication(short value)
static SequenceFile.Writer.Option	stream(FSDataOutputStream value)
void	sync() create a sync point
void	syncFs() Deprecated. Use hsync() or hflush() instead
static SequenceFile.Writer.Option	valueClass(Class<?> value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

keySerializer

protected org.apache.hadoop.io.serializer.Serializer keySerializer

uncompressedValSerializer

protected org.apache.hadoop.io.serializer.Serializer uncompressedValSerializer

compressedValSerializer

protected org.apache.hadoop.io.serializer.Serializer compressedValSerializer

Constructor Detail

SequenceFile.Writer

@Deprecated
public SequenceFile.Writer(FileSystem fs,
                              Configuration conf,
                              Path name,
                              Class keyClass,
                              Class valClass)
                    throws IOException

Deprecated. Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.

Create the named file.

Throws:

IOException

SequenceFile.Writer

@Deprecated
public SequenceFile.Writer(FileSystem fs,
                              Configuration conf,
                              Path name,
                              Class keyClass,
                              Class valClass,
                              Progressable progress,
                              SequenceFile.Metadata metadata)
                    throws IOException

Deprecated. Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.

Create the named file with write-progress reporter.

Throws:

IOException

SequenceFile.Writer

@Deprecated
public SequenceFile.Writer(FileSystem fs,
                              Configuration conf,
                              Path name,
                              Class keyClass,
                              Class valClass,
                              int bufferSize,
                              short replication,
                              long blockSize,
                              Progressable progress,
                              SequenceFile.Metadata metadata)
                    throws IOException

Deprecated. Use SequenceFile.createWriter(Configuration, Writer.Option...) instead.

Create the named file with write-progress reporter.

Throws:

IOException

Method Detail

file

public static SequenceFile.Writer.Option file(Path value)

bufferSize

public static SequenceFile.Writer.Option bufferSize(int value)

stream

public static SequenceFile.Writer.Option stream(FSDataOutputStream value)

replication

public static SequenceFile.Writer.Option replication(short value)

appendIfExists

public static SequenceFile.Writer.Option appendIfExists(boolean value)

blockSize

public static SequenceFile.Writer.Option blockSize(long value)

progressable

public static SequenceFile.Writer.Option progressable(Progressable value)

keyClass

public static SequenceFile.Writer.Option keyClass(Class<?> value)

valueClass

public static SequenceFile.Writer.Option valueClass(Class<?> value)

metadata

public static SequenceFile.Writer.Option metadata(SequenceFile.Metadata value)

compression

public static SequenceFile.Writer.Option compression(SequenceFile.CompressionType value)

compression

public static SequenceFile.Writer.Option compression(SequenceFile.CompressionType value,
                                     CompressionCodec codec)

getKeyClass

public Class getKeyClass()

Returns the class of keys in this file.

getValueClass

public Class getValueClass()

Returns the class of values in this file.

getCompressionCodec

public CompressionCodec getCompressionCodec()

Returns the compression codec of data in this file.

sync

public void sync()
          throws IOException

create a sync point

Specified by:

sync in interface Syncable

Throws:

IOException

See Also:

Syncable.hflush()

syncFs

@Deprecated
public void syncFs()
            throws IOException

Deprecated. Use hsync() or hflush() instead

flush all currently written data to the file system

Throws:

IOException

hsync

public void hsync()
           throws IOException

Description copied from interface: Syncable

Similar to posix fsync, flush out the data in client's user buffer all the way to the disk device (but the disk may have it in its cache).

Specified by:

hsync in interface Syncable

Throws:

IOException - if error occurs

hflush

public void hflush()
            throws IOException

Description copied from interface: Syncable

Flush out the data in client's user buffer. After the return of this call, new readers will see the data.

Specified by:

hflush in interface Syncable

Throws:

IOException - if any error occurs

close

public void close()
           throws IOException

Close the file.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

append

public void append(Writable key,
          Writable val)
            throws IOException

Append a key/value pair.

Throws:

IOException

append

public void append(Object key,
          Object val)
            throws IOException

Append a key/value pair.

Throws:

IOException

appendRaw

public void appendRaw(byte[] keyData,
             int keyOffset,
             int keyLength,
             SequenceFile.ValueBytes val)
               throws IOException

Throws:

IOException

getLength

public long getLength()
               throws IOException

Returns the current length of the output file.

This always returns a synchronized position. In other words, immediately after calling SequenceFile.Reader.seek(long) with a position returned by this method, SequenceFile.Reader.next(Writable) may be called. However the key may be earlier in the file than key last written when this method was called (e.g., with block-compression, it may be the first key in the block that was being written when this method was called).

Throws:

IOException