org.apache.hadoop.fs.s3a

Interface S3AInternals

@InterfaceStability.Unstable
 @InterfaceAudience.LimitedPrivate(value="testing/diagnostics")
public interface S3AInternals

This is an unstable interface for access to S3A Internal state, S3 operations and the S3 client connector itself.

Method Summary

Modifier and Type	Method and Description
long	abortMultipartUploads(org.apache.hadoop.fs.Path path) Abort multipart uploads under a path.
software.amazon.awssdk.services.s3.S3Client	getAmazonS3Client(String reason) Returns the S3 client used by this filesystem.
String	getBucketLocation() Get the region of a bucket.
String	getBucketLocation(String bucketName) Get the region of a bucket; fixing up the region so it can be used in the builders of other AWS clients.
software.amazon.awssdk.services.s3.model.HeadBucketResponse	getBucketMetadata() Request bucket metadata.
software.amazon.awssdk.services.s3.model.HeadObjectResponse	getObjectMetadata(org.apache.hadoop.fs.Path path) Low-level call to get at the object metadata.
boolean	isMultipartCopyEnabled() Is multipart copy enabled?
AWSCredentialProviderList	shareCredentials(String purpose) Get a shared copy of the AWS credentials, with its reference counter updated.

Method Detail

getAmazonS3Client

software.amazon.awssdk.services.s3.S3Client getAmazonS3Client(String reason)

Returns the S3 client used by this filesystem. Will log at debug.

Warning This bypasses core S3A operations, including auditing. It is intended for use in testing, diagnostics and for accessing operations not available through the S3A connector itself.

Unless audit spans are created through the S3AFileSystem, make sure that fs.s3a.audit.reject.out.of.span.operations is set to false.

Mocking note: this is the same S3Client as is used by the owning filesystem; changes to this client will be reflected by changes in the behavior of that filesystem.

Parameters:

reason - a justification for requesting access.

Returns:

S3Client

getBucketLocation

@Retries.RetryTranslated
 @AuditEntryPoint
String getBucketLocation()
                                                             throws IOException

Get the region of a bucket. Invoked from StoreContext; consider an entry point.

Returns:

the region in which a bucket is located

Throws:

AccessDeniedException - if the caller lacks permission.

IOException - on any failure.

getBucketLocation

@AuditEntryPoint
 @Retries.RetryTranslated
String getBucketLocation(String bucketName)
                                                             throws IOException

Get the region of a bucket; fixing up the region so it can be used in the builders of other AWS clients. Requires the caller to have the AWS role permission s3:GetBucketLocation. Retry policy: retrying, translated.

Parameters:

bucketName - the name of the bucket

Returns:

the region in which a bucket is located

Throws:

AccessDeniedException - if the caller lacks permission.

IOException - on any failure.

getObjectMetadata

@AuditEntryPoint
 @Retries.RetryTranslated
software.amazon.awssdk.services.s3.model.HeadObjectResponse getObjectMetadata(org.apache.hadoop.fs.Path path)
                                                                                                                  throws IOException

Low-level call to get at the object metadata. Auditing: An audit entry point.

Parameters:

path - path to the object. This will be qualified.

Returns:

metadata

Throws:

IOException - IO and object access problems.

shareCredentials

AWSCredentialProviderList shareCredentials(String purpose)

Get a shared copy of the AWS credentials, with its reference counter updated. Caller is required to call close() on this after they have finished using it.

Parameters:

purpose - what is this for? This is for logging

Returns:

a reference to shared credentials.

getBucketMetadata

@AuditEntryPoint
 @Retries.RetryTranslated
software.amazon.awssdk.services.s3.model.HeadBucketResponse getBucketMetadata()
                                                                                                                  throws IOException

Request bucket metadata.

Returns:

the metadata

Throws:

UnknownStoreException - the bucket is absent

IOException - any other problem talking to S3

isMultipartCopyEnabled

boolean isMultipartCopyEnabled()

Is multipart copy enabled?

Returns:

true if the transfer manager is used to copy files.

abortMultipartUploads

@AuditEntryPoint
 @Retries.RetryTranslated
long abortMultipartUploads(org.apache.hadoop.fs.Path path)
                                                               throws IOException

Abort multipart uploads under a path.

Parameters:

path - path to abort uploads under.

Returns:

a count of aborts

Throws:

IOException - trouble; FileNotFoundExceptions are swallowed.