A B C D E F G H I L M N O P R S T V W

A

ACCESS_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants

AnonymousAWSCredentialsProvider - Class in org.apache.hadoop.fs.s3a

AnonymousAWSCredentialsProvider() - Constructor for class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider

append(Path, int, Progressable) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

This optional operation is not yet supported.

append(Path, int, Progressable) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Append to an existing file (optional operation).

append(Path, int, Progressable) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

This optional operation is not yet supported.

available() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream

B

BasicAWSCredentialsProvider - Class in org.apache.hadoop.fs.s3a

BasicAWSCredentialsProvider(String, String) - Constructor for class org.apache.hadoop.fs.s3a.BasicAWSCredentialsProvider

Block - Class in org.apache.hadoop.fs.s3

Holds metadata about a block of data being stored in a FileSystemStore.

Block(long, long) - Constructor for class org.apache.hadoop.fs.s3.Block

blockExists(long) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

BUFFER_DIR - Static variable in class org.apache.hadoop.fs.s3a.Constants

C

CANNED_ACL - Static variable in class org.apache.hadoop.fs.s3a.Constants

close() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream

close() - Method in class org.apache.hadoop.fs.s3a.S3AOutputStream

Constants - Class in org.apache.hadoop.fs.s3a

Constants() - Constructor for class org.apache.hadoop.fs.s3a.Constants

copyFromLocalFile(boolean, boolean, Path, Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

The src file is on the local disk.

create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Create an FSDataOutputStream at the indicated Path with write-progress reporting.

create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

D

DEFAULT_CANNED_ACL - Static variable in class org.apache.hadoop.fs.s3a.Constants

DEFAULT_MAX_ERROR_RETRIES - Static variable in class org.apache.hadoop.fs.s3a.Constants

DEFAULT_MAX_PAGING_KEYS - Static variable in class org.apache.hadoop.fs.s3a.Constants

DEFAULT_MAXIMUM_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants

DEFAULT_MIN_MULTIPART_THRESHOLD - Static variable in class org.apache.hadoop.fs.s3a.Constants

DEFAULT_MULTIPART_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants

DEFAULT_PURGE_EXISTING_MULTIPART - Static variable in class org.apache.hadoop.fs.s3a.Constants

DEFAULT_PURGE_EXISTING_MULTIPART_AGE - Static variable in class org.apache.hadoop.fs.s3a.Constants

DEFAULT_SECURE_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants

DEFAULT_SOCKET_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants

delete(Path, boolean) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

delete(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Delete a file.

delete(Path, boolean) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

deleteBlock(Block) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

deleteINode(Path) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

deserialize(InputStream) - Static method in class org.apache.hadoop.fs.s3.INode

DIRECTORY_INODE - Static variable in class org.apache.hadoop.fs.s3.INode

dump() - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

Diagnostic method to dump all INodes to the console.

E

equals(Object) - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus

Compare if this object is equal to another object

F

FILE_TYPES - Static variable in class org.apache.hadoop.fs.s3.INode

FileSystemStore - Interface in org.apache.hadoop.fs.s3

A facility for storing and retrieving INodes and Blocks.

finishedWrite(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

flush() - Method in class org.apache.hadoop.fs.s3a.S3AOutputStream

G

getAccessKey() - Method in class org.apache.hadoop.fs.s3.S3Credentials

getBlocks() - Method in class org.apache.hadoop.fs.s3.INode

getCanonicalServiceName() - Method in class org.apache.hadoop.fs.s3.S3FileSystem

getCanonicalServiceName() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Override getCononicalServiceName because we don't support token in S3A

getCanonicalServiceName() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

getCredentials() - Method in class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider

getCredentials() - Method in class org.apache.hadoop.fs.s3a.BasicAWSCredentialsProvider

getDefaultBlockSize() - Method in class org.apache.hadoop.fs.s3.S3FileSystem

getDefaultBlockSize() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Deprecated. use FileSystem.getDefaultBlockSize(Path) instead

getDefaultBlockSize() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

getFileStatus(Path) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

FileStatus for S3 file systems.

getFileStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Return a file status object that represents the path.

getFileStatus(Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

getFileType() - Method in class org.apache.hadoop.fs.s3.INode

getId() - Method in class org.apache.hadoop.fs.s3.Block

getLastBytesTransferred() - Method in class org.apache.hadoop.fs.s3a.S3AOutputStream.ProgressableProgressListener

getLength() - Method in class org.apache.hadoop.fs.s3.Block

getPos() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream

getScheme() - Method in class org.apache.hadoop.fs.s3.S3FileSystem

Return the protocol scheme for the FileSystem.

getScheme() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Return the protocol scheme for the FileSystem.

getScheme() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

Return the protocol scheme for the FileSystem.

getSecretAccessKey() - Method in class org.apache.hadoop.fs.s3.S3Credentials

getSerializedLength() - Method in class org.apache.hadoop.fs.s3.INode

getUri() - Method in class org.apache.hadoop.fs.s3.S3FileSystem

getUri() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Returns a URI whose scheme and authority identify this FileSystem.

getUri() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

getVersion() - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

getWorkingDirectory() - Method in class org.apache.hadoop.fs.s3.S3FileSystem

getWorkingDirectory() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Get the current working directory for the given file system

getWorkingDirectory() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

H

hashCode() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus

Returns a hash code value for the object, which is defined as the hash code of the path name.

I

initialize(URI, Configuration) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

initialize(URI) - Method in class org.apache.hadoop.fs.s3.MigrationTool

initialize(URI, Configuration) - Method in class org.apache.hadoop.fs.s3.S3Credentials

initialize(URI, Configuration) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

initialize(URI, Configuration) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Called after a new FileSystem instance is constructed.

initialize(URI, Configuration) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

INode - Class in org.apache.hadoop.fs.s3

Holds file metadata including type (regular file, or directory), and the list of blocks that are pointers to the data.

INode(INode.FileType, Block[]) - Constructor for class org.apache.hadoop.fs.s3.INode

inodeExists(Path) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

isDirectory() - Method in class org.apache.hadoop.fs.s3.INode

isEmptyDirectory() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus

isFile() - Method in class org.apache.hadoop.fs.s3.INode

isFile(Path) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

L

listDeepSubPaths(Path) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

listStatus(Path) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

listStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

List the statuses of the files/directories in the given path if the path is a directory.

listStatus(Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

If f is a file, this method will make a single call to S3.

listSubPaths(Path) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

LOG - Static variable in class org.apache.hadoop.fs.s3a.S3AFileSystem

LOG - Static variable in class org.apache.hadoop.fs.s3a.S3AInputStream

LOG - Static variable in class org.apache.hadoop.fs.s3a.S3AOutputStream

LOG - Static variable in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

M

main(String[]) - Static method in class org.apache.hadoop.fs.s3.MigrationTool

markSupported() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream

MAX_ERROR_RETRIES - Static variable in class org.apache.hadoop.fs.s3a.Constants

MAX_PAGING_KEYS - Static variable in class org.apache.hadoop.fs.s3a.Constants

MAXIMUM_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants

MigrationTool - Class in org.apache.hadoop.fs.s3

This class is a tool for migrating data from an older to a newer version of an S3 filesystem.

MigrationTool() - Constructor for class org.apache.hadoop.fs.s3.MigrationTool

MIN_MULTIPART_THRESHOLD - Static variable in class org.apache.hadoop.fs.s3a.Constants

mkdirs(Path, FsPermission) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

mkdirs(Path, FsPermission) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Make the given file and all non-existent parents into directories.

mkdirs(Path, FsPermission) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

MULTIPART_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants

N

NativeS3FileSystem - Class in org.apache.hadoop.fs.s3native

A FileSystem for reading and writing files stored on Amazon S3.

NativeS3FileSystem() - Constructor for class org.apache.hadoop.fs.s3native.NativeS3FileSystem

NativeS3FileSystem(NativeFileSystemStore) - Constructor for class org.apache.hadoop.fs.s3native.NativeS3FileSystem

O

open(Path, int) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

open(Path, int) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Opens an FSDataInputStream at the indicated Path.

open(Path, int) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

org.apache.hadoop.fs.s3 - package org.apache.hadoop.fs.s3

A distributed, block-based implementation of FileSystem that uses Amazon S3 as a backing store.

org.apache.hadoop.fs.s3a - package org.apache.hadoop.fs.s3a

org.apache.hadoop.fs.s3native - package org.apache.hadoop.fs.s3native

A distributed implementation of FileSystem for reading and writing files on Amazon S3.

P

progressChanged(ProgressEvent) - Method in class org.apache.hadoop.fs.s3a.S3AOutputStream.ProgressableProgressListener

purge() - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

Delete everything.

PURGE_EXISTING_MULTIPART - Static variable in class org.apache.hadoop.fs.s3a.Constants

PURGE_EXISTING_MULTIPART_AGE - Static variable in class org.apache.hadoop.fs.s3a.Constants

R

read() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream

read(byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream

refresh() - Method in class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider

refresh() - Method in class org.apache.hadoop.fs.s3a.BasicAWSCredentialsProvider

rename(Path, Path) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

rename(Path, Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Renames Path src to Path dst.

rename(Path, Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

retrieveBlock(Block, long) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

retrieveINode(Path) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

run(String[]) - Method in class org.apache.hadoop.fs.s3.MigrationTool

S

S3_BLOCK_SIZE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3_BLOCK_SIZE_KEY - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3_BYTES_PER_CHECKSUM_DEFAULT - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3_BYTES_PER_CHECKSUM_KEY - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3_CLIENT_WRITE_PACKET_SIZE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3_CLIENT_WRITE_PACKET_SIZE_KEY - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3_NATIVE_BLOCK_SIZE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_NATIVE_BLOCK_SIZE_KEY - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_NATIVE_BYTES_PER_CHECKSUM_DEFAULT - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_NATIVE_BYTES_PER_CHECKSUM_KEY - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_NATIVE_CLIENT_WRITE_PACKET_SIZE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_NATIVE_CLIENT_WRITE_PACKET_SIZE_KEY - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_NATIVE_REPLICATION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_NATIVE_REPLICATION_KEY - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_NATIVE_STREAM_BUFFER_SIZE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_NATIVE_STREAM_BUFFER_SIZE_KEY - Static variable in class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

S3_REPLICATION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3_REPLICATION_KEY - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3_STREAM_BUFFER_SIZE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3_STREAM_BUFFER_SIZE_KEY - Static variable in class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3AFileStatus - Class in org.apache.hadoop.fs.s3a

S3AFileStatus(boolean, boolean, Path) - Constructor for class org.apache.hadoop.fs.s3a.S3AFileStatus

S3AFileStatus(long, long, Path) - Constructor for class org.apache.hadoop.fs.s3a.S3AFileStatus

S3AFileSystem - Class in org.apache.hadoop.fs.s3a

S3AFileSystem() - Constructor for class org.apache.hadoop.fs.s3a.S3AFileSystem

S3AInputStream - Class in org.apache.hadoop.fs.s3a

S3AInputStream(String, String, long, AmazonS3Client, FileSystem.Statistics) - Constructor for class org.apache.hadoop.fs.s3a.S3AInputStream

S3AOutputStream - Class in org.apache.hadoop.fs.s3a

S3AOutputStream(Configuration, AmazonS3Client, S3AFileSystem, String, String, Progressable, CannedAccessControlList, FileSystem.Statistics, String) - Constructor for class org.apache.hadoop.fs.s3a.S3AOutputStream

S3AOutputStream.ProgressableProgressListener - Class in org.apache.hadoop.fs.s3a

S3AOutputStream.ProgressableProgressListener(Upload, Progressable, FileSystem.Statistics) - Constructor for class org.apache.hadoop.fs.s3a.S3AOutputStream.ProgressableProgressListener

S3Credentials - Class in org.apache.hadoop.fs.s3

Extracts AWS credentials from the filesystem URI or configuration.

S3Credentials() - Constructor for class org.apache.hadoop.fs.s3.S3Credentials

S3Exception - Exception in org.apache.hadoop.fs.s3

Thrown if there is a problem communicating with Amazon S3.

S3Exception(Throwable) - Constructor for exception org.apache.hadoop.fs.s3.S3Exception

S3FileSystem - Class in org.apache.hadoop.fs.s3

A block-based FileSystem backed by Amazon S3.

S3FileSystem() - Constructor for class org.apache.hadoop.fs.s3.S3FileSystem

S3FileSystem(FileSystemStore) - Constructor for class org.apache.hadoop.fs.s3.S3FileSystem

S3FileSystemConfigKeys - Class in org.apache.hadoop.fs.s3

This class contains constants for configuration keys used in the s3 file system.

S3FileSystemConfigKeys() - Constructor for class org.apache.hadoop.fs.s3.S3FileSystemConfigKeys

S3FileSystemException - Exception in org.apache.hadoop.fs.s3

Thrown when there is a fatal exception while using S3FileSystem.

S3FileSystemException(String) - Constructor for exception org.apache.hadoop.fs.s3.S3FileSystemException

S3N_FOLDER_SUFFIX - Static variable in class org.apache.hadoop.fs.s3a.Constants

S3NativeFileSystemConfigKeys - Class in org.apache.hadoop.fs.s3native

This class contains constants for configuration keys used in the s3 file system.

S3NativeFileSystemConfigKeys() - Constructor for class org.apache.hadoop.fs.s3native.S3NativeFileSystemConfigKeys

SECRET_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants

SECURE_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants

seek(long) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream

seekToNewSource(long) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream

serialize() - Method in class org.apache.hadoop.fs.s3.INode

SERVER_SIDE_ENCRYPTION_ALGORITHM - Static variable in class org.apache.hadoop.fs.s3a.Constants

setWorkingDirectory(Path) - Method in class org.apache.hadoop.fs.s3.S3FileSystem

setWorkingDirectory(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem

Set the current working directory for the given file system.

setWorkingDirectory(Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem

Set the working directory to the given directory.

SOCKET_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants

storeBlock(Block, File) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

storeINode(Path, INode) - Method in interface org.apache.hadoop.fs.s3.FileSystemStore

T

toString() - Method in class org.apache.hadoop.fs.s3.Block

toString() - Method in class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider

toString() - Method in class org.apache.hadoop.fs.s3a.BasicAWSCredentialsProvider

V

VersionMismatchException - Exception in org.apache.hadoop.fs.s3

Thrown when Hadoop cannot read the version of the data stored in S3FileSystem.

VersionMismatchException(String, String) - Constructor for exception org.apache.hadoop.fs.s3.VersionMismatchException

W

write(int) - Method in class org.apache.hadoop.fs.s3a.S3AOutputStream

write(byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.S3AOutputStream

A B C D E F G H I L M N O P R S T V W