Class FileBasedSink.Writer<DestinationT,​OutputT>

  • Type Parameters:
    OutputT - the type of values to write.
    Enclosing class:
    FileBasedSink<UserT,​DestinationT,​OutputT>

    public abstract static class FileBasedSink.Writer<DestinationT,​OutputT>
    extends java.lang.Object
    Abstract writer that writes a bundle to a FileBasedSink. Subclass implementations provide a method that can write a single value to a WritableByteChannel.

    Subclass implementations may also override methods that write headers and footers before and after the values in a bundle, respectively, as well as provide a MIME type for the output channel.

    Multiple FileBasedSink.Writer instances may be created on the same worker, and therefore any access to static members or methods should be thread safe.

    • Method Detail

      • prepareWrite

        protected abstract void prepareWrite​(java.nio.channels.WritableByteChannel channel)
                                      throws java.lang.Exception
        Called with the channel that a subclass will write its header, footer, and values to. Subclasses should either keep a reference to the channel provided or create and keep a reference to an appropriate object that they will use to write to it.

        Called before any subsequent calls to writeHeader, writeFooter, and write.

        Throws:
        java.lang.Exception
      • writeHeader

        protected void writeHeader()
                            throws java.lang.Exception
        Writes header at the beginning of output files. Nothing by default; subclasses may override.
        Throws:
        java.lang.Exception
      • writeFooter

        protected void writeFooter()
                            throws java.lang.Exception
        Writes footer at the end of output files. Nothing by default; subclasses may override.
        Throws:
        java.lang.Exception
      • finishWrite

        protected void finishWrite()
                            throws java.lang.Exception
        Called after all calls to writeHeader(), write(OutputT) and writeFooter(). If any resources opened in the write processes need to be flushed, flush them here.
        Throws:
        java.lang.Exception
      • open

        public final void open​(java.lang.String uId)
                        throws java.lang.Exception
        Opens a uniquely named temporary file and initializes the writer using prepareWrite(java.nio.channels.WritableByteChannel).

        The unique id that is given to open should be used to ensure that the writer's output does not interfere with the output of other Writers, as a bundle may be executed many times for fault tolerance.

        Throws:
        java.lang.Exception
      • write

        public abstract void write​(OutputT value)
                            throws java.lang.Exception
        Called for each value in the bundle.
        Throws:
        java.lang.Exception
      • getOutputFile

        public ResourceId getOutputFile()
      • cleanup

        public final void cleanup()
                           throws java.lang.Exception
        Throws:
        java.lang.Exception
      • close

        public final void close()
                         throws java.lang.Exception
        Closes the channel and returns the bundle result.
        Throws:
        java.lang.Exception
      • getDestination

        public DestinationT getDestination()
        Return the user destination object for this writer.