Package org.apache.beam.sdk.util

Class FilePatternMatchingShardedFile

java.lang.Object

org.apache.beam.sdk.util.FilePatternMatchingShardedFile

All Implemented Interfaces:

java.io.Serializable, ShardedFile

@Internal
public class FilePatternMatchingShardedFile
extends java.lang.Object
implements ShardedFile

A sharded file which matches a given file pattern. Note that the file pattern must match at least one file.

Note that file matching should only occur once the file system is in a stable state and guaranteed to provide a consistent result during file pattern matching.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor	Description
FilePatternMatchingShardedFile(java.lang.String filePattern)	Constructs an FilePatternMatchingShardedFile for the given file pattern.

Method Summary

Modifier and Type	Method	Description
java.util.List<java.lang.String>	readFilesWithRetries()	Discovers all shards of this file.
java.util.List<java.lang.String>	readFilesWithRetries(Sleeper sleeper, BackOff backOff)	Discovers all shards of this file using the provided Sleeper and BackOff.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FilePatternMatchingShardedFile

public FilePatternMatchingShardedFile(java.lang.String filePattern)

Constructs an FilePatternMatchingShardedFile for the given file pattern. Note that the file pattern must match at least one file.

Note that file matching should only occur once the file system is in a stable state and guaranteed to provide a consistent result during file pattern matching.

Method Detail

readFilesWithRetries

public java.util.List<java.lang.String> readFilesWithRetries(Sleeper sleeper,
                                                             BackOff backOff)
                                                      throws java.io.IOException,
                                                             java.lang.InterruptedException

Discovers all shards of this file using the provided Sleeper and BackOff.

Specified by:

readFilesWithRetries in interface ShardedFile

Throws:

java.io.IOException

java.lang.InterruptedException

readFilesWithRetries

public java.util.List<java.lang.String> readFilesWithRetries()
                                                      throws java.io.IOException,
                                                             java.lang.InterruptedException

Discovers all shards of this file.

Throws:

java.io.IOException

java.lang.InterruptedException

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object