Package org.apache.beam.sdk.io
Class FileBasedSink.Writer<DestinationT,OutputT>
- java.lang.Object
-
- org.apache.beam.sdk.io.FileBasedSink.Writer<DestinationT,OutputT>
-
- Type Parameters:
OutputT
- the type of values to write.
- Enclosing class:
- FileBasedSink<UserT,DestinationT,OutputT>
public abstract static class FileBasedSink.Writer<DestinationT,OutputT> extends java.lang.Object
Abstract writer that writes a bundle to aFileBasedSink
. Subclass implementations provide a method that can write a single value to aWritableByteChannel
.Subclass implementations may also override methods that write headers and footers before and after the values in a bundle, respectively, as well as provide a MIME type for the output channel.
Multiple
FileBasedSink.Writer
instances may be created on the same worker, and therefore any access to static members or methods should be thread safe.
-
-
Constructor Summary
Constructors Constructor Description Writer(FileBasedSink.WriteOperation<DestinationT,OutputT> writeOperation, java.lang.String mimeType)
Construct a newFileBasedSink.Writer
that will produce files of the given MIME type.
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description void
cleanup()
void
close()
Closes the channel and returns the bundle result.protected void
finishWrite()
DestinationT
getDestination()
Return the user destination object for this writer.ResourceId
getOutputFile()
FileBasedSink.WriteOperation<DestinationT,OutputT>
getWriteOperation()
Return the WriteOperation that this Writer belongs to.void
open(java.lang.String uId)
Opens a uniquely named temporary file and initializes the writer usingprepareWrite(java.nio.channels.WritableByteChannel)
.protected abstract void
prepareWrite(java.nio.channels.WritableByteChannel channel)
Called with the channel that a subclass will write its header, footer, and values to.abstract void
write(OutputT value)
Called for each value in the bundle.protected void
writeFooter()
Writes footer at the end of output files.protected void
writeHeader()
Writes header at the beginning of output files.
-
-
-
Constructor Detail
-
Writer
public Writer(FileBasedSink.WriteOperation<DestinationT,OutputT> writeOperation, java.lang.String mimeType)
Construct a newFileBasedSink.Writer
that will produce files of the given MIME type.
-
-
Method Detail
-
prepareWrite
protected abstract void prepareWrite(java.nio.channels.WritableByteChannel channel) throws java.lang.Exception
Called with the channel that a subclass will write its header, footer, and values to. Subclasses should either keep a reference to the channel provided or create and keep a reference to an appropriate object that they will use to write to it.Called before any subsequent calls to writeHeader, writeFooter, and write.
- Throws:
java.lang.Exception
-
writeHeader
protected void writeHeader() throws java.lang.Exception
Writes header at the beginning of output files. Nothing by default; subclasses may override.- Throws:
java.lang.Exception
-
writeFooter
protected void writeFooter() throws java.lang.Exception
Writes footer at the end of output files. Nothing by default; subclasses may override.- Throws:
java.lang.Exception
-
finishWrite
protected void finishWrite() throws java.lang.Exception
Called after all calls towriteHeader()
,write(OutputT)
andwriteFooter()
. If any resources opened in the write processes need to be flushed, flush them here.- Throws:
java.lang.Exception
-
open
public final void open(java.lang.String uId) throws java.lang.Exception
Opens a uniquely named temporary file and initializes the writer usingprepareWrite(java.nio.channels.WritableByteChannel)
.The unique id that is given to open should be used to ensure that the writer's output does not interfere with the output of other Writers, as a bundle may be executed many times for fault tolerance.
- Throws:
java.lang.Exception
-
write
public abstract void write(OutputT value) throws java.lang.Exception
Called for each value in the bundle.- Throws:
java.lang.Exception
-
getOutputFile
public ResourceId getOutputFile()
-
cleanup
public final void cleanup() throws java.lang.Exception
- Throws:
java.lang.Exception
-
close
public final void close() throws java.lang.Exception
Closes the channel and returns the bundle result.- Throws:
java.lang.Exception
-
getWriteOperation
public FileBasedSink.WriteOperation<DestinationT,OutputT> getWriteOperation()
Return the WriteOperation that this Writer belongs to.
-
getDestination
public DestinationT getDestination()
Return the user destination object for this writer.
-
-