Module java.xml.bind

Package org.apache.flink.fs.s3presto

Class FlinkS3PrestoFileSystem

java.lang.Object

org.apache.flink.core.fs.FileSystem

org.apache.flink.fs.s3presto.common.HadoopFileSystem

org.apache.flink.fs.s3.common.FlinkS3FileSystem

org.apache.flink.fs.s3presto.FlinkS3PrestoFileSystem

All Implemented Interfaces:

org.apache.flink.core.fs.EntropyInjectingFileSystem, org.apache.flink.core.fs.IFileSystem, org.apache.flink.core.fs.PathsCopyingFileSystem

public class FlinkS3PrestoFileSystem
extends org.apache.flink.fs.s3.common.FlinkS3FileSystem

FlinkS3PrestoFileSystem provides custom recursive deletion functionality to work around a bug in the internally used Presto file system.

https://github.com/prestodb/presto/issues/17416

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.fs.s3.common.FlinkS3FileSystem

org.apache.flink.fs.s3.common.FlinkS3FileSystem.S5CmdConfiguration

Nested classes/interfaces inherited from class org.apache.flink.core.fs.FileSystem

org.apache.flink.core.fs.FileSystem.FSKey, org.apache.flink.core.fs.FileSystem.WriteMode

Nested classes/interfaces inherited from interface org.apache.flink.core.fs.PathsCopyingFileSystem

org.apache.flink.core.fs.PathsCopyingFileSystem.CopyRequest

Field Summary

Fields inherited from class org.apache.flink.fs.s3.common.FlinkS3FileSystem

S3_MULTIPART_MIN_PART_SIZE

Constructor Summary

Constructors

Constructor	Description
FlinkS3PrestoFileSystem(org.apache.hadoop.fs.FileSystem hadoopS3FileSystem, org.apache.flink.fs.s3.common.FlinkS3FileSystem.S5CmdConfiguration s5CmdConfiguration, String localTmpDirectory, String entropyInjectionKey, int entropyLength, org.apache.flink.fs.s3.common.writer.S3AccessHelper s3UploadHelper, long s3uploadPartSize, int maxConcurrentUploadsPerStream)

Method Summary

Modifier and Type	Method	Description
boolean	delete(org.apache.flink.core.fs.Path path, boolean recursive)

Methods inherited from class org.apache.flink.fs.s3.common.FlinkS3FileSystem

canCopyPaths, copyFiles, createRecoverableWriter, generateEntropy, getEntropyInjectionKey, getLocalTmpDir

Methods inherited from class org.apache.flink.fs.s3presto.common.HadoopFileSystem

create, create, createRecoverableWriter, exists, getDefaultBlockSize, getFileBlockLocations, getFileStatus, getHadoopFileSystem, getHomeDirectory, getUri, getWorkingDirectory, isDistributedFS, listStatus, mkdirs, open, open, rename, toHadoopPath

Methods inherited from class org.apache.flink.core.fs.FileSystem

create, get, getDefaultFsUri, getLocalFileSystem, getUnguardedFileSystem, initialize, initialize, initOutPathDistFS, initOutPathLocalFS

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.core.fs.IFileSystem

create, createRecoverableWriter, exists, getFileBlockLocations, getFileStatus, getHomeDirectory, getUri, getWorkingDirectory, initOutPathDistFS, initOutPathLocalFS, isDistributedFS, listStatus, mkdirs, open, open, rename

Constructor Detail

FlinkS3PrestoFileSystem

public FlinkS3PrestoFileSystem(org.apache.hadoop.fs.FileSystem hadoopS3FileSystem,
                               @Nullable
                               org.apache.flink.fs.s3.common.FlinkS3FileSystem.S5CmdConfiguration s5CmdConfiguration,
                               String localTmpDirectory,
                               @Nullable
                               String entropyInjectionKey,
                               int entropyLength,
                               @Nullable
                               org.apache.flink.fs.s3.common.writer.S3AccessHelper s3UploadHelper,
                               long s3uploadPartSize,
                               int maxConcurrentUploadsPerStream)

Method Detail

delete

public boolean delete(org.apache.flink.core.fs.Path path,
                      boolean recursive)
               throws IOException

Specified by:

delete in interface org.apache.flink.core.fs.IFileSystem

Overrides:

delete in class org.apache.flink.fs.s3presto.common.HadoopFileSystem

Throws:

IOException