Package org.apache.beam.sdk.io

Class FileBasedSink.FilenamePolicy

java.lang.Object

org.apache.beam.sdk.io.FileBasedSink.FilenamePolicy

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

DefaultFilenamePolicy

Enclosing class:

FileBasedSink<UserT,DestinationT,OutputT>

@Experimental(FILESYSTEM)
public abstract static class FileBasedSink.FilenamePolicy
extends java.lang.Object
implements java.io.Serializable

A naming policy for output files.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor	Description
FilenamePolicy()

Method Summary

Modifier and Type	Method	Description
void	populateDisplayData(DisplayData.Builder builder)	Populates the display data.
abstract @Nullable ResourceId	unwindowedFilename(int shardNumber, int numShards, FileBasedSink.OutputFileHints outputFileHints)	When a sink has not requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a FileBasedSink.OutputFileHints containing information about the file, including a suggested (e.g.
abstract ResourceId	windowedFilename(int shardNumber, int numShards, BoundedWindow window, PaneInfo paneInfo, FileBasedSink.OutputFileHints outputFileHints)	When a sink has requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a FileBasedSink.OutputFileHints containing information about the file, including a suggested extension (e.g.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FilenamePolicy

public FilenamePolicy()

Method Detail

windowedFilename

@Experimental(FILESYSTEM)
public abstract ResourceId windowedFilename(int shardNumber,
                                            int numShards,
                                            BoundedWindow window,
                                            PaneInfo paneInfo,
                                            FileBasedSink.OutputFileHints outputFileHints)

When a sink has requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a FileBasedSink.OutputFileHints containing information about the file, including a suggested extension (e.g. coming from Compression).

The policy must return unique and consistent filenames for different windows and panes.

unwindowedFilename

@Experimental(FILESYSTEM)
public abstract @Nullable ResourceId unwindowedFilename(int shardNumber,
                                                        int numShards,
                                                        FileBasedSink.OutputFileHints outputFileHints)

When a sink has not requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a FileBasedSink.OutputFileHints containing information about the file, including a suggested (e.g. coming from Compression).

The shardNumber and numShards parameters, should be used by the policy to generate unique and consistent filenames.

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Populates the display data.