- abort() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object.InconsistentS3InputStream
-
- abort(Path, UploadHandle) - Method in class org.apache.hadoop.fs.s3a.S3AMultipartUploader
-
- abort() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
Aborts the IO.
- abortAllSinglePendingCommits(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Enumerate all pending files in a dir/tree, abort.
- aborted - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- abortJob(JobContext, JobStatus.State) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- abortJobInternal(JobContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
The internal job abort operation; can be overridden in tests.
- abortJobInternal(JobContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- abortMultipartCommit(String, String) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.CommitContext
-
- abortMultipartCommit(String, String) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Abort a multipart commit operation.
- abortMultipartUpload(String, String, Invoker.Retried) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Abort a multipart upload operation.
- abortMultipartUpload(MultipartUpload) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Abort a multipart commit operation.
- abortMultipartUploadsUnderPath(String) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Abort multipart uploads under a path: limited to the first
few hundred.
- abortOutstandingMultipartUploads(long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Abort all outstanding MPUs older than a given age.
- abortPendingUploads(JobContext, List<SinglePendingCommit>, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Abort all pending uploads in the list.
- abortPendingUploads(JobContext, AbstractS3ACommitter.ActiveCommit, boolean, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Abort all pending uploads in the list.
- abortPendingUploadsInCleanup(boolean) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Abort all pending uploads to the destination directory during
job cleanup operations.
- abortPendingUploadsInCleanup(boolean) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- abortPendingUploadsUnderPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Abort all pending uploads to the destination FS under a path.
- abortSingleCommit(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.CommitContext
-
- abortTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Abort a task.
- abortTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Abort the task.
- abortWith(Tasks.Task<I, ?>) - Method in class org.apache.hadoop.fs.s3a.commit.Tasks.Builder
-
- aboutToComplete(String, List<PartETag>, long) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
Complete operation: generate the final commit data, put it.
- aboutToComplete(String, List<PartETag>, long) - Method in class org.apache.hadoop.fs.s3a.commit.PutTracker
-
Callback when the upload is is about to complete.
- AbstractAWSCredentialProvider - Class in org.apache.hadoop.fs.s3a.auth
-
Base class for AWS credential providers which
take a URI and config in their constructor.
- AbstractAWSCredentialProvider(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.auth.AbstractAWSCredentialProvider
-
Construct from URI + configuration.
- AbstractDelegationTokenBinding - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
An AbstractDelegationTokenBinding implementation is a class which
handles the binding of its underlying authentication mechanism to the
Hadoop Delegation token mechanism.
- AbstractDelegationTokenBinding(String, Text) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Constructor.
- AbstractDelegationTokenBinding.TokenSecretManager - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
The secret manager always uses the same secret; the
factory for new identifiers is that of the token manager.
- AbstractDTService - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
This is the base class for both the delegation binding
code and the back end service created; allows for
shared methods across both.
- AbstractDTService(String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Protected constructor.
- AbstractS3ACommitter - Class in org.apache.hadoop.fs.s3a.commit
-
Abstract base class for S3A committers; allows for any commonality
between different architectures.
- AbstractS3ACommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Create a committer.
- AbstractS3ACommitter.ActiveCommit - Class in org.apache.hadoop.fs.s3a.commit
-
State of the active commit operation.
- AbstractS3ACommitterFactory - Class in org.apache.hadoop.fs.s3a.commit
-
Dynamically create the output committer based on subclass type and settings.
- AbstractS3ACommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
- AbstractS3ATokenIdentifier - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
An S3A Delegation Token Identifier: contains the information needed
to talk to S3A.
- AbstractS3ATokenIdentifier(Text, URI, Text, Text, String, EncryptionSecrets) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Constructor.
- AbstractS3ATokenIdentifier(Text, Text, Text, Text, URI) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Constructor.
- AbstractS3ATokenIdentifier(Text, Token<AbstractS3ATokenIdentifier>) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Build from a token.
- AbstractS3ATokenIdentifier(Text) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
For subclasses to use in their own empty-constructors.
- AbstractS3GuardDynamoDBDiagnostic - Class in org.apache.hadoop.fs.s3a.s3guard
-
Entry point for S3Guard diagnostics operations against DynamoDB tables.
- AbstractS3GuardDynamoDBDiagnostic(String) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
Constructor.
- AbstractS3GuardDynamoDBDiagnostic(String, S3AFileSystem, DynamoDBMetadataStore, URI) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
Constructor.
- AbstractSessionCredentialsProvider - Class in org.apache.hadoop.fs.s3a.auth
-
Base class for session credential support.
- AbstractSessionCredentialsProvider(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Constructor.
- AbstractSessionCredentialsProvider.NoCredentials - Class in org.apache.hadoop.fs.s3a.auth
-
A special set of null credentials which are not the anonymous class.
- AbstractStoreOperation - Class in org.apache.hadoop.fs.s3a.impl
-
Base class of operations in the store.
- AbstractStoreOperation(StoreContext) - Constructor for class org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation
-
constructor.
- accept(Path, S3ObjectSummary) - Method in class org.apache.hadoop.fs.s3a.Listing.AcceptAllButSelfAndS3nDirs
-
Reject a summary entry if the key path is the qualified Path, or
it ends with "_$folder$".
- accept(Path, String) - Method in class org.apache.hadoop.fs.s3a.Listing.AcceptAllButSelfAndS3nDirs
-
Accept all prefixes except the one for the base path, "self".
- accept(FileStatus) - Method in class org.apache.hadoop.fs.s3a.Listing.AcceptAllButSelfAndS3nDirs
-
- ACCEPT_ALL - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
A Path filter which accepts all filenames.
- AcceptAllButSelfAndS3nDirs(Path) - Constructor for class org.apache.hadoop.fs.s3a.Listing.AcceptAllButSelfAndS3nDirs
-
Constructor.
- ACCESS_DENIED - Static variable in class org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteSupport
-
This is the exception exit code if access was denied on a delete.
- ACCESS_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- ACCESS_POLICY - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
The access policies we want for operations.
- action - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- ActiveCommit(FileSystem, List<Path>) - Constructor for class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
Construct from a source FS and list of files.
- add(RoleModel.Policy) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Add the statements of another policy to this one.
- add(Collection<RoleModel.Statement>) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Add a collection of statements.
- add(RoleModel.Statement) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Add a single statement.
- add(AWSCredentialsProvider) - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Add a new provider.
- add(Path) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- add(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Add a commit.
- addActions(String...) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- addActions(Collection<String>) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- addAgeOptions() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- addAll(AWSCredentialProviderList) - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Add all providers from another list to this one.
- addAncestors(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
This adds all new ancestors of a path as directories.
- addAncestors(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- addAncestors(Path, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
This adds all new ancestors of a path as directories.
- addAncestors(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- addAncestors(MetadataStore, Path, ITtlTimeProvider, BulkOperationState) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
This adds all new ancestors of a path as directories.
- addDelegationTokens(Configuration, Credentials, String, String) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADtFetcher
-
Returns Token object via FileSystem, null if bad argument.
- addDiagnostic(String, String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Add a diagnostics entry.
- addFileSystemStatistics(Map<String, Long>) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Add the filesystem statistics to the map; overwriting anything
with the same name.
- addMoveAncestors(MetadataStore, Collection<Path>, Collection<PathMetadata>, Path, Path, Path, String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Helper method that records the move of all ancestors of a path.
- addMoveDir(MetadataStore, Collection<Path>, Collection<PathMetadata>, Path, Path, String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Helper function that records the move of directory paths, adding
resulting metadata to the supplied lists.
- addMoveFile(MetadataStore, Collection<Path>, Collection<PathMetadata>, Path, Path, long, long, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
- addNode(S3GuardFsck.DDBTreeNode) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.DDBTree
-
- addResources(String...) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- addResources(Collection<String>) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
Add a list of resources.
- addUUID(String, String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Insert the UUID to a path if it is not there already.
- addValueToQuantiles(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Add a value to a quantiles statistic.
- AGE_OPTIONS_USAGE - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- ALL - Static variable in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
All probes.
- ALL_DDB_TABLES - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All DynamoDB tables: "arn:aws:dynamodb:*".
- ALLOW_S3_AND_SGUARD - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Policy for all S3 and S3Guard operations, and SSE-KMS.
- allowAllDynamoDBOperations(String) - Static method in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- allowAuthoritative(Path) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Is the path for this instance considered authoritative on the client,
that is: will listing/status operations only be handled by the metastore,
with no fallback to S3.
- allowAuthoritative(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Is a path to be considered as authoritative?
True iff there is an authoritative metastore or if there
is a non-auth store with the supplied path under
one of the paths declared as authoritative.
- allowAuthoritative(Path, S3AFileSystem, boolean, Collection<String>) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Is the path for the given FS instance authoritative?
- allowS3GuardClientOperations(String) - Static method in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- allowS3Operations(String, boolean) - Static method in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
From an S3 bucket name, build an ARN to refer to it.
- AnonymousAWSCredentialsProvider - Class in org.apache.hadoop.fs.s3a
-
AnonymousAWSCredentialsProvider supports anonymous access to AWS services
through the AWS SDK.
- AnonymousAWSCredentialsProvider() - Constructor for class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider
-
- append(Path, int, Progressable) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Append to an existing file (optional operation).
- append(Path, int, Progressable) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- applyLocatedFiles(RemoteIterator<? extends LocatedFileStatus>, S3AUtils.CallOnLocatedFileStatus) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Apply an operation to every LocatedFileStatus in a remote
iterator.
- applyRevisionConstraint(GetObjectRequest, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Applies the given
revisionId
as a server-side qualification on the
GetObjectRequest.
- applyRevisionConstraint(CopyObjectRequest, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Applies the given
revisionId
as a server-side qualification on the
CopyObjectRequest.
- applyRevisionConstraint(GetObjectMetadataRequest, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Applies the given
revisionId
as a server-side qualification on the
GetObjectMetadataRequest.
- assertQualified(Path) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Assert that the path is qualified with a host and scheme.
- assertQualified(Path...) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Assert that all paths are valid.
- ASSUMED_ROLE_ARN - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
ARN of AWS Role to request: "fs.s3a.assumed.role.arn".
- ASSUMED_ROLE_CREDENTIALS_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- ASSUMED_ROLE_CREDENTIALS_PROVIDER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
List of providers to authenticate for the assumed role: "fs.s3a.assumed.role.credentials.provider".
- ASSUMED_ROLE_POLICY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
JSON policy containing the policy to apply to the role: "fs.s3a.assumed.role.policy".
- ASSUMED_ROLE_SESSION_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Duration of assumed roles before a refresh is attempted.
- ASSUMED_ROLE_SESSION_DURATION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default duration of an assumed role: "1h".
- ASSUMED_ROLE_SESSION_NAME - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Session name for the assumed role, must be valid characters according
to the AWS APIs: "fs.s3a.assumed.role.session.name".
- ASSUMED_ROLE_STS_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Security Token Service Endpoint: "fs.s3a.assumed.role.sts.endpoint".
- ASSUMED_ROLE_STS_ENDPOINT_REGION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Region for the STS endpoint; needed if the endpoint
is set to anything other then the central one.: "fs.s3a.assumed.role.sts.endpoint.region".
- ASSUMED_ROLE_STS_ENDPOINT_REGION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value for the STS endpoint region; needed for
v4 signing: "".
- AssumedRoleCredentialProvider - Class in org.apache.hadoop.fs.s3a.auth
-
Support IAM Assumed roles by instantiating an instance of
STSAssumeRoleSessionCredentialsProvider from configuration
properties, including wiring up the inner authenticator, and,
unless overridden, creating a session name from the current user.
- AssumedRoleCredentialProvider(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
Instantiate.
- audit(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.AuthoritativeAuditOperation
-
Audit the tree.
- AUTH_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- AuthDirContentMismatch(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.AuthDirContentMismatch
-
- AUTHORITATIVE_PATH - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Whether or not to allow MetadataStore to be source of truth for a path prefix
- AuthoritativeAuditOperation - Class in org.apache.hadoop.fs.s3a.s3guard
-
Audit a directory tree for being authoritative.
- AuthoritativeAuditOperation(StoreContext, DynamoDBMetadataStore, boolean, boolean) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.AuthoritativeAuditOperation
-
Constructor.
- AuthoritativeAuditOperation.NonAuthoritativeDirException - Exception in org.apache.hadoop.fs.s3a.s3guard
-
A directory was found which was non-authoritative.
- available() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object.InconsistentS3InputStream
-
- available() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- available() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- AWS_CREDENTIALS_PROVIDER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- AWS_SERVICE_IDENTIFIER_DDB - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- AWS_SERVICE_IDENTIFIER_S3 - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- AWS_SERVICE_IDENTIFIER_STS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- AWSBadRequestException - Exception in org.apache.hadoop.fs.s3a
-
A 400 "Bad Request" exception was received.
- AWSBadRequestException(String, AmazonServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSBadRequestException
-
Instantiate.
- AWSClientIOException - Exception in org.apache.hadoop.fs.s3a
-
IOException equivalent of an AmazonClientException.
- AWSClientIOException(String, SdkBaseException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSClientIOException
-
- AWSCredentialProviderList - Class in org.apache.hadoop.fs.s3a
-
A list of providers.
- AWSCredentialProviderList() - Constructor for class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Empty instance.
- AWSCredentialProviderList(Collection<AWSCredentialsProvider>) - Constructor for class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Create with an initial list of providers.
- AWSCredentialProviderList(String, AWSCredentialsProvider...) - Constructor for class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Create with an initial list of providers.
- AWSNoResponseException - Exception in org.apache.hadoop.fs.s3a
-
Status code 443, no response from server.
- AWSNoResponseException(String, AmazonServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSNoResponseException
-
- AWSPolicyProvider - Interface in org.apache.hadoop.fs.s3a.auth.delegation
-
Interface for providers of AWS policy for accessing data.
- AWSPolicyProvider.AccessLevel - Enum in org.apache.hadoop.fs.s3a.auth.delegation
-
Access levels.
- AWSRedirectException - Exception in org.apache.hadoop.fs.s3a
-
Request is redirected.
- AWSRedirectException(String, AmazonServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSRedirectException
-
Instantiate.
- AWSS3IOException - Exception in org.apache.hadoop.fs.s3a
-
Wrap a AmazonS3Exception as an IOE, relaying all
getters.
- AWSS3IOException(String, AmazonS3Exception) - Constructor for exception org.apache.hadoop.fs.s3a.AWSS3IOException
-
Instantiate.
- AWSServiceIOException - Exception in org.apache.hadoop.fs.s3a
-
A specific exception from AWS operations.
- AWSServiceIOException(String, AmazonServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
Instantiate.
- AWSServiceThrottledException - Exception in org.apache.hadoop.fs.s3a
-
Exception raised when a service was throttled.
- AWSServiceThrottledException(String, AmazonServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSServiceThrottledException
-
Instantiate.
- AwsSignerInitializer - Interface in org.apache.hadoop.fs.s3a.auth
-
Interface which can be implemented to allow initialization of any custom
signers which may be used by the
S3AFileSystem.
- AWSStatus500Exception - Exception in org.apache.hadoop.fs.s3a
-
A 500 response came back from a service.
- AWSStatus500Exception(String, AmazonServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSStatus500Exception
-
- call(LocatedFileStatus) - Method in interface org.apache.hadoop.fs.s3a.S3AUtils.CallOnLocatedFileStatus
-
- call(LocatedFileStatus) - Method in interface org.apache.hadoop.fs.s3a.S3AUtils.LocatedFileStatusMap
-
- CallableSupplier<T> - Class in org.apache.hadoop.fs.s3a.impl
-
A bridge from Callable to Supplier; catching exceptions
raised by the callable and wrapping them as appropriate.
- CallableSupplier(Callable<T>) - Constructor for class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
Create.
- CANNED_ACL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- canonicalizeUri(URI) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- canonicalizeUri(URI, int) - Static method in class org.apache.hadoop.fs.s3native.S3xLoginHelper
-
Canonicalize the given URI.
- castFrom(AmazonS3) - Static method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
Convenience function for test code to cast from supertype.
- CHANGE_DETECT_MODE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Mode to run change detection in.
- CHANGE_DETECT_MODE_CLIENT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Change is detected on the client side by comparing the returned id with the
expected id.
- CHANGE_DETECT_MODE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default change detection mode: server.
- CHANGE_DETECT_MODE_NONE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Change detection is turned off.
- CHANGE_DETECT_MODE_SERVER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Change is detected by passing the expected value in the GetObject request.
- CHANGE_DETECT_MODE_WARN - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Change is detected on the client side by comparing the returned id with the
expected id.
- CHANGE_DETECT_REQUIRE_VERSION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
If true, raises a
RemoteFileChangedException exception when S3
doesn't provide the attribute defined by fs.s3a.change.detection.source.
- CHANGE_DETECT_REQUIRE_VERSION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default change detection require version: true.
- CHANGE_DETECT_SOURCE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Where to get the value to use in change detection.
- CHANGE_DETECT_SOURCE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default change detection mechanism: eTag.
- CHANGE_DETECT_SOURCE_ETAG - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
eTag as the change detection mechanism.
- CHANGE_DETECT_SOURCE_VERSION_ID - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Object versionId as the change detection mechanism.
- CHANGE_DETECTED - Static variable in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- CHANGE_REPORTED_BY_S3 - Static variable in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- ChangeDetectionPolicy - Class in org.apache.hadoop.fs.s3a.impl
-
Object change detection policy.
- ChangeDetectionPolicy(ChangeDetectionPolicy.Mode, boolean) - Constructor for class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- ChangeDetectionPolicy.Mode - Enum in org.apache.hadoop.fs.s3a.impl
-
What to do when change is detected.
- ChangeDetectionPolicy.Source - Enum in org.apache.hadoop.fs.s3a.impl
-
The S3 object attribute used to detect change.
- ChangeTracker - Class in org.apache.hadoop.fs.s3a.impl
-
Change tracking for input streams: the version ID or etag of the object is
tracked and compared on open/re-open.
- ChangeTracker(String, ChangeDetectionPolicy, AtomicLong, S3ObjectAttributes) - Constructor for class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Create a change tracker.
- checkBucketNameOrDDBTableNameProvided(List<String>) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Check if bucket or DDB table name is set.
- checkDdbInternalConsistency(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck
-
Check the DynamoDB metadatastore internally for consistency.
- checkIfS3BucketIsGuarded(List<String>) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- checkNotEmpty() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Verify that the provider list is not empty.
- checkPath(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Check that a Path belongs to this FileSystem.
- checkPath(Configuration, URI, Path, int) - Static method in class org.apache.hadoop.fs.s3native.S3xLoginHelper
-
Check the path, ignoring authentication details.
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory".
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory".
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory".
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory".
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory".
- cleanup(JobContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Cleanup the job context, including aborting anything pending
and destroying the thread pool.
- cleanup(JobContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- cleanupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- cleanupStagingDirs() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Clean up any staging directories.
- cleanupStagingDirs() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Delete the magic directory.
- cleanupStagingDirs() - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- clearBucketOption(Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Clear a bucket-specific property.
- clearInconsistency(S3AFileSystem) - Static method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
Clear any accumulated inconsistency state.
- clearInconsistency() - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
Clear all oustanding inconsistent keys.
- clearTempFolderInfo(TaskAttemptID) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Remove all information held about task attempts.
- clone() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- close() - Method in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
Propagate the close() call to the inner stsProvider.
- close() - Method in class org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
-
- close() - Method in class org.apache.hadoop.fs.s3a.auth.SignerManager
-
- close() - Method in class org.apache.hadoop.fs.s3a.auth.STSClientFactory.STSClient
-
- close() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Close routine will close all providers in the list which implement
Closeable.
- close() - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.CommitContext
-
- close() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- close() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object.InconsistentS3InputStream
-
- close() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Close the filesystem.
- close() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Close the stream.
- close() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
- close() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
Close triggers the merge of statistics into the filesystem's
instrumentation instance.
- close() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.OutputStreamStatistics
-
Output stream has closed.
- close() - Method in class org.apache.hadoop.fs.s3a.s3guard.BulkOperationState
-
- close() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- close() - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- close() - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- close() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Close the FS and metastore.
- close() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- closeAll(Logger, Closeable...) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Deprecated.
- closeAutocloseables(Logger, AutoCloseable...) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Close the Closeable objects and ignore any Exception or
null pointers.
- closed - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- closeOperations - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- CMK_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- commit(SinglePendingCommit, String) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.CommitContext
-
Commit a single pending commit; exceptions are caught
and converted to an outcome.
- commitAborted() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.CommitterStatistics
-
A commit has been aborted.
- commitCompleted(long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.CommitterStatistics
-
A commit has been completed.
- CommitConstants - Class in org.apache.hadoop.fs.s3a.commit
-
Constants for working with committers.
- commitCreated() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.CommitterStatistics
-
A commit has been created.
- commitFailed() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.CommitterStatistics
-
- commitJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Commit work.
- commitJobInternal(JobContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Internal Job commit operation: where the S3 requests are made
(potentially in parallel).
- CommitOperations - Class in org.apache.hadoop.fs.s3a.commit
-
The implementation of the various actions a committer needs.
- CommitOperations(S3AFileSystem) - Constructor for class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Instantiate.
- CommitOperations.CommitContext - Class in org.apache.hadoop.fs.s3a.commit
-
Commit context.
- CommitOperations.MaybeIOE - Class in org.apache.hadoop.fs.s3a.commit
-
- commitOrFail(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.CommitContext
-
Commit the operation, throwing an exception on any failure.
- commitPendingUploads(JobContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Commit all the pending uploads.
- commitReverted() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.CommitterStatistics
-
- commitTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- commitTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- commitTaskInternal(TaskAttemptContext, List<? extends FileStatus>) - Method in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
- commitTaskInternal(TaskAttemptContext, List<? extends FileStatus>) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Commit the task by uploading all created files and then
writing a pending entry for them.
- COMMITTER_NAME_DIRECTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- COMMITTER_NAME_FILE - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- COMMITTER_NAME_MAGIC - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- COMMITTER_NAME_PARTITIONED - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- COMMITTER_NAME_STAGING - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
This is the staging committer base class; only used for testing.
- CommitterStatistics() - Constructor for class org.apache.hadoop.fs.s3a.S3AInstrumentation.CommitterStatistics
-
- commitUpload(String, String, List<PartETag>, long, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
This completes a multipart upload to the destination key via
finalizeMultipartUpload().
- commitUploaded(long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.CommitterStatistics
-
Data has been uploaded to be committed in a subsequent operation.
- commitUploaded(long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.OutputStreamStatistics
-
Data has been uploaded to be committed in a subsequent operation;
to be called at the end of the write.
- CommitUtils - Class in org.apache.hadoop.fs.s3a.commit
-
Static utility methods related to S3A commitment processing, both
staging and magic.
- CommitUtilsWithMR - Class in org.apache.hadoop.fs.s3a.commit
-
These are commit utility methods which import classes from
hadoop-mapreduce, and so only work when that module is on the
classpath.
- compareFileStatusToPathMetadata(S3AFileStatus, PathMetadata) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck
-
- compareS3DirContentToMs(S3AFileStatus, List<S3AFileStatus>) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck
-
Compares S3 directory content to the metadata store.
- compareS3ToMs(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck
-
Compares S3 to MS.
- complete(Path, Map<Integer, PartHandle>, UploadHandle) - Method in class org.apache.hadoop.fs.s3a.S3AMultipartUploader
-
- completeMPUwithRetries(String, String, List<PartETag>, long, AtomicInteger) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
This completes a multipart upload to the destination key via
finalizeMultipartUpload().
- completeMultipartUpload(CompleteMultipartUploadRequest) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- completeRename() - Method in class org.apache.hadoop.fs.s3a.s3guard.DelayedUpdateRenameTracker
-
- completeRename() - Method in class org.apache.hadoop.fs.s3a.s3guard.ProgressiveRenameTracker
-
- completeRename() - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
Complete the operation.
- COMPONENT - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
- COMPONENT - Static variable in class org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
-
- COMPRESSION_OPT_GZIP - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Gzipped.
- COMPRESSION_OPT_NONE - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
No compression.
- CONFLICT_MODE_APPEND - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Conflict mode: "append".
- CONFLICT_MODE_FAIL - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Conflict mode: "fail".
- CONFLICT_MODE_REPLACE - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Conflict mode: "replace".
- ConflictResolution - Enum in org.apache.hadoop.fs.s3a.commit.staging
-
Enum of conflict resolution algorithms.
- connectivityFailure - Variable in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Client connectivity: baseExponentialRetry without worrying about whether
or not the command is idempotent.
- Constants - Class in org.apache.hadoop.fs.s3a
-
- containsBasePath(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Does the list of magic elements contain a base path marker?
- containsViolation() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.ComparePair
-
- CONTEXT - Static variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
"s3aFileSystem" Currently all s3a metrics are placed in a single
"context".
- ContextAccessors - Interface in org.apache.hadoop.fs.s3a.impl
-
An interface to implement for providing accessors to
S3AFileSystem-level API calls.
- continueListObjects(S3ListRequest, S3ListResult) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
List the next set of objects.
- convertToIOException(Exception) - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
Convert a passed in exception (expected to be an IOE or AWS exception)
into an IOException.
- convertTokenIdentifier(AbstractS3ATokenIdentifier, Class<T>) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Verify that a token identifier is of a specific class.
- convertToPaths(Collection<DeleteObjectsRequest.KeyVersion>, Function<String, Path>) - Static method in class org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteSupport
-
Given a list of delete requests, convert them all to paths.
- copyFile(String, String, S3ObjectAttributes, S3AReadOpContext) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Copy a single object in the bucket via a COPY operation.
- copyFromLocalFile(boolean, boolean, Path, Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
The src file is on the local disk.
- CopyOutcome - Class in org.apache.hadoop.fs.s3a.impl
-
Extracts the outcome of a TransferManager-executed copy operation.
- CopyOutcome(CopyResult, InterruptedException, SdkBaseException) - Constructor for class org.apache.hadoop.fs.s3a.impl.CopyOutcome
-
- counter(String, String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a counter in the registry.
- counter(Statistic) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a counter in the registry.
- create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
- create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- CREATE_SUCCESSFUL_JOB_OUTPUT_DIR_MARKER - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to trigger creation of a marker file on job completion.
- createAwsConf(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
- createAwsConf(Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Create a new AWS ClientConfiguration.
- createAWSCredentialProviderSet(URI, Configuration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
- createClientConnection(AWSSecurityTokenService, Invoker) - Static method in class org.apache.hadoop.fs.s3a.auth.STSClientFactory
-
Create an STS Client instance.
- createCredentials(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Implementation point: whatever the subclass must do to load credentials.
- createCredentials(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialProvider
-
Perform the binding, looking up the DT and parsing it.
- createCredentials(Configuration) - Method in class org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
-
The credentials here must include a session token, else this operation
will raise an exception.
- createDefaultOriginMessage() - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Create the default origin text message with local hostname and
timestamp.
- createDelegationOperations() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create an instance of the delegation operations.
- createDelegationToken(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Create a delegation token for the user.
- createDelegationToken(EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Create a delegation token for the user.
- createDynamoDBClient(String, String, AWSCredentialsProvider) - Method in interface org.apache.hadoop.fs.s3a.s3guard.DynamoDBClientFactory
-
Create a DynamoDB client object from configuration.
- createDynamoDBClient(String, String, AWSCredentialsProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBClientFactory.DefaultDynamoDBClientFactory
-
- createEmptyIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
- createEmptyIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- createEmptyIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
- createEmptyIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- createExceptionMap() - Method in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Subclasses can override this like a constructor to change behavior: call
superclass method, then modify it as needed, and return it.
- createExceptionMap() - Method in class org.apache.hadoop.fs.s3a.S3GuardExistsRetryPolicy
-
- createFileStatus(Path, S3ObjectSummary, long, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Create a files status instance from a listing.
- createFileStatusListingIterator(Path, S3ListRequest, PathFilter, Listing.FileStatusAcceptor) - Method in class org.apache.hadoop.fs.s3a.Listing
-
Create a FileStatus iterator against a path, with a given list object
request.
- createFileStatusListingIterator(Path, S3ListRequest, PathFilter, Listing.FileStatusAcceptor, RemoteIterator<S3AFileStatus>) - Method in class org.apache.hadoop.fs.s3a.Listing
-
Create a FileStatus iterator against a path, with a given
list object request.
- createIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding.TokenSecretManager
-
- createListObjectsRequest(String, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create a
ListObjectsRequest request against this bucket,
with the maximum keys returned in a query set by
S3AFileSystem.maxKeys.
- createLocatedFileStatusIterator(RemoteIterator<S3AFileStatus>) - Method in class org.apache.hadoop.fs.s3a.Listing
-
Create a located status iterator over a file status iterator.
- createMultipartUploader(FileSystem, Configuration) - Method in class org.apache.hadoop.fs.s3a.S3AMultipartUploader.Factory
-
- createNonRecursive(Path, FsPermission, EnumSet<CreateFlag>, int, short, long, Progressable) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
- createObjectAttributes(Path, String, String, long) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Create the attributes of an object for subsequent use.
- createObjectAttributes(S3AFileStatus) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Create the attributes of an object for subsequent use.
- createOutputCommitter(Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
- createPassword(AbstractS3ATokenIdentifier) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding.TokenSecretManager
-
- createPolicy(ChangeDetectionPolicy.Mode, ChangeDetectionPolicy.Source, boolean) - Static method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Create a policy.
- createPutObjectRequest(String, InputStream, long) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Create a PutObjectRequest request against the specific key.
- createPutObjectRequest(String, File) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Create a PutObjectRequest request to upload a file.
- createReadContext(FileStatus) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Create the read context for reading from the referenced file,
using FS state as well as the status.
- createS3Client(URI, String, AWSCredentialsProvider, String) - Method in class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
- createS3Client(URI, String, AWSCredentialsProvider, String) - Method in interface org.apache.hadoop.fs.s3a.S3ClientFactory
-
Creates a new AmazonS3 client.
- createSecretMananger() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Create a secret manager.
- createSSEAwsKeyManagementParams(EncryptionSecrets) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecretOperations
-
Create SSE-KMS options for a request, iff the encryption is SSE-KMS.
- createSSECustomerKey(EncryptionSecrets) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecretOperations
-
Create SSE-C client side key encryption options on demand.
- createStorageStatistics() - Static method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create the storage statistics or bind to an existing one.
- createStoreContext() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Build an immutable store context.
- createSuccessMarker(Path, SuccessData, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Save the success data to the _SUCCESS file.
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
Implementation point: create a task committer for a specific filesystem.
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory
-
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory
-
Create a task committer.
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory
-
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory
-
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory
-
- createTempFile(String, long) - Method in interface org.apache.hadoop.fs.s3a.impl.ContextAccessors
-
Create a temporary file.
- createTempFile(String, long) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Create a temporary file somewhere.
- createThrottledExecutor(int) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Create a new executor service with a given capacity.
- createThrottledExecutor() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- createThrottleRetryPolicy(Configuration) - Method in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Create the throttling policy.
- createThrottleRetryPolicy(Configuration) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardDataAccessRetryPolicy
-
- createTokenIdentifier(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Create a token identifier with all the information needed
to be included in a delegation token.
- createTokenIdentifier(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
Create a new delegation token.
- createTokenIdentifier(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
Create the Token Identifier.
- createTokenIdentifier(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- createTracker(Path, String) - Method in class org.apache.hadoop.fs.s3a.commit.MagicCommitIntegration
-
Given a path and a key to that same path, create a tracker for it.
- createUploadFileStatus(Path, boolean, long, long, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Create a file status for object we just uploaded.
- createWrappedCommitter(JobContext, Configuration) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Create the wrapped committer.
- CredentialInitializationException - Exception in org.apache.hadoop.fs.s3a
-
Exception which Hadoop's AWSCredentialsProvider implementations should
throw when there is a problem with the credential setup.
- CredentialInitializationException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.CredentialInitializationException
-
- CredentialInitializationException(String) - Constructor for exception org.apache.hadoop.fs.s3a.CredentialInitializationException
-
- CREDENTIALS_CONVERTED_TO_DELEGATION_TOKEN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
A message added to the standard origin string when the DT is
built from session credentials passed in.
- CSV_HEADER_OPT_IGNORE - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Ignore the header.
- CSV_HEADER_OPT_NONE - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
No header: first row is data.
- CSV_HEADER_OPT_USE - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Use the header.
- CSV_INPUT_COMMENT_MARKER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
String which indicates the row is actually a comment.
- CSV_INPUT_COMMENT_MARKER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default marker.
- CSV_INPUT_FIELD_DELIMITER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default field delimiter.
- CSV_INPUT_HEADER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
How should headers be used?
Value: "fs.s3a.select.input.csv.header".
- CSV_INPUT_HEADER_OPT_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default header mode: "ignore".
- CSV_INPUT_INPUT_FIELD_DELIMITER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Field delimiter.
- CSV_INPUT_QUOTE_CHARACTER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Quote Character.
- CSV_INPUT_QUOTE_CHARACTER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default Quote Character.
- CSV_INPUT_QUOTE_ESCAPE_CHARACTER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Character to escape quotes.
- CSV_INPUT_QUOTE_ESCAPE_CHARACTER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default quote escape character.
- CSV_INPUT_RECORD_DELIMITER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Record delimiter.
- CSV_INPUT_RECORD_DELIMITER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default delimiter
Value: "\n".
- CSV_OUTPUT_FIELD_DELIMITER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Field delimiter.
- CSV_OUTPUT_FIELD_DELIMITER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default field delimiter.
- CSV_OUTPUT_QUOTE_CHARACTER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Quote Character.
- CSV_OUTPUT_QUOTE_CHARACTER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default Quote Character.
- CSV_OUTPUT_QUOTE_ESCAPE_CHARACTER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Character to escape quotes.
- CSV_OUTPUT_QUOTE_ESCAPE_CHARACTER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default quote escape character.
- CSV_OUTPUT_QUOTE_FIELDS - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Should CSV fields be quoted?
One of : ALWAYS, ASNEEDED
Value: "fs.s3a.select.output.csv.quote.fields".
- CSV_OUTPUT_QUOTE_FIELDS_ALWAYS - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Output quotation policy (default): "always".
- CSV_OUTPUT_QUOTE_FIELDS_AS_NEEEDED - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Output quotation policy: "asneeded".
- CSV_OUTPUT_RECORD_DELIMITER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Record delimiter.
- CSV_OUTPUT_RECORD_DELIMITER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Default delimiter
Value: "\n".
- CUSTOM_SIGNERS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
List of custom Signers.
- dateToLong(Date) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Date to long conversion.
- DAYS_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- DDB_ADMIN - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Operations needed for DDB/S3Guard Admin.
- DDB_ALL_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All DynamoDB operations: "dynamodb:*".
- DDB_BATCH_GET_ITEM - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Permission for DDB operation: "dynamodb:BatchGetItem".
- DDB_BATCH_WRITE_ITEM - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Batch write permission for DDB: "dynamodb:BatchWriteItem".
- DDB_DELETE_ITEM - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Permission for DDB delete operation: "dynamodb:DeleteItem".
- DDB_DESCRIBE_TABLE - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Permission for DDB describeTable() operation: "dynamodb:DescribeTable".
- DDB_GET_ITEM - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Permission for DDB operation to get a record: "dynamodb:GetItem".
- DDB_PUT_ITEM - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Permission for DDB write record operation: "dynamodb:PutItem".
- DDB_QUERY - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Permission to query the DDB table: "dynamodb:Query".
- DDB_UPDATE_ITEM - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Permission for DDB update single item operation: "dynamodb:UpdateItem".
- DDBPathMetadata - Class in org.apache.hadoop.fs.s3a.s3guard
-
- DDBPathMetadata(PathMetadata) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DDBPathMetadata
-
- DDBPathMetadata(S3AFileStatus) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DDBPathMetadata
-
- DDBPathMetadata(S3AFileStatus, Tristate, boolean, long) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DDBPathMetadata
-
- DDBPathMetadata(S3AFileStatus, Tristate, boolean, boolean, long) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DDBPathMetadata
-
- DDBTree() - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.DDBTree
-
- decrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Decrement a gauge by a specific value.
- decrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Decrement a gauge by a specific value.
- decrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Decrement a specific gauge.
- DEFAULT_ASSUMED_ROLE_STS_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default endpoint for session tokens: "".
- DEFAULT_AUTHORITATIVE_PATH - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_BLOCKSIZE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Default blocksize as used in blocksize and FS status queries: 33554432.
- DEFAULT_BLOCKSIZE - Static variable in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Default blocksize as used in blocksize and FS status queries.
- DEFAULT_CANNED_ACL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_COMMITTER_THREADS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- DEFAULT_CONFLICT_MODE - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Default conflict mode: "append".
- DEFAULT_CREATE_SUCCESSFUL_JOB_DIR_MARKER - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Default job marker option: true.
- DEFAULT_DELAY_KEY_MSEC - Static variable in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
How many seconds affected keys will have delayed visibility.
- DEFAULT_DELAY_KEY_PROBABILITY - Static variable in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- DEFAULT_DELAY_KEY_SUBSTRING - Static variable in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
Keys containing this substring will be subject to delayed visibility.
- DEFAULT_DELEGATION_TOKEN_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Default token binding "".
- DEFAULT_DELEGATION_TOKEN_DURATION - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Default duration of a delegation token: "1h".
- DEFAULT_DELEGATION_TOKEN_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Default endpoint for session tokens: "".
- DEFAULT_DELEGATION_TOKEN_REGION - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Region default: "".
- DEFAULT_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value of s3 endpoint.
- DEFAULT_ESTABLISH_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_EXECUTOR_CAPACITY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_FAST_BUFFER_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_FAST_UPLOAD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- DEFAULT_FAST_UPLOAD_ACTIVE_BLOCKS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Limit of queued block upload operations before writes
block.
- DEFAULT_FAST_UPLOAD_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default buffer option: "disk".
- DEFAULT_KEEPALIVE_TIME - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_LIST_VERSION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MAGIC_COMMITTER_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Is the committer enabled by default? No.
- DEFAULT_MAX_ERROR_RETRIES - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default number of times the AWS client library should retry errors before
escalating to the S3A code: 10.
- DEFAULT_MAX_PAGING_KEYS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MAX_THREADS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MAX_TOTAL_TASKS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MAXIMUM_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_METADATASTORE_AUTHORITATIVE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_METADATASTORE_METADATA_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default TTL in milliseconds: 15 minutes.
- DEFAULT_MIN_MULTIPART_THRESHOLD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MULTIPART_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_PURGE_EXISTING_MULTIPART - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_PURGE_EXISTING_MULTIPART_AGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_READAHEAD_RANGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_REQUEST_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_S3_CLIENT_FACTORY_IMPL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_S3GUARD_DISABLED_WARN_LEVEL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_S3GUARD_METASTORE_LOCAL_ENTRY_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_S3GUARD_METASTORE_LOCAL_MAX_RECORDS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_SECURE_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_SOCKET_RECV_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_SOCKET_SEND_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_SOCKET_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_SSL_CHANNEL_MODE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_STAGING_COMMITTER_UNIQUE_FILENAMES - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- DefaultDynamoDBClientFactory() - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DynamoDBClientFactory.DefaultDynamoDBClientFactory
-
- DefaultS3ClientFactory - Class in org.apache.hadoop.fs.s3a
-
- DefaultS3ClientFactory() - Constructor for class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
- DelayedUpdateRenameTracker - Class in org.apache.hadoop.fs.s3a.s3guard
-
This is the rename updating strategy originally used:
a collection of source paths and a list of destinations are created,
then updated at the end (possibly slow).
- DelayedUpdateRenameTracker(StoreContext, MetadataStore, Path, Path, BulkOperationState) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DelayedUpdateRenameTracker
-
- DELEGATION_TOKEN_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Property containing classname for token binding: "fs.s3a.delegation.token.binding".
- DELEGATION_TOKEN_CREDENTIALS_PROVIDER - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Key to list AWS credential providers for Session/role
credentials: "fs.s3a.aws.credentials.provider".
- DELEGATION_TOKEN_DURATION - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Duration of tokens in time: "fs.s3a.assumed.role.session.duration".
- DELEGATION_TOKEN_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Endpoint for session tokens, used when building delegation tokens:
"fs.s3a.assumed.role.sts.endpoint".
- DELEGATION_TOKEN_FULL_CREDENTIALS_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Token binding to pass full credentials: "org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding".
- DELEGATION_TOKEN_REGION - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Region for DT issuing; must be non-empty if the endpoint is set: "fs.s3a.assumed.role.sts.endpoint.region".
- DELEGATION_TOKEN_ROLE_ARN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
ARN of the delegation token: "fs.s3a.assumed.role.arn".
- DELEGATION_TOKEN_ROLE_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Role DTs: "org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding".
- DELEGATION_TOKEN_SESSION_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Session Token binding classname: "org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding".
- DelegationConstants - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
All the constants related to delegation tokens.
- DelegationOperations - Interface in org.apache.hadoop.fs.s3a.auth.delegation
-
All operations used for delegation which aren't in the store context.
- DelegationTokenIOException - Exception in org.apache.hadoop.fs.s3a.auth.delegation
-
General IOException for Delegation Token issues.
- DelegationTokenIOException(String) - Constructor for exception org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException
-
- DelegationTokenIOException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException
-
- DelegationTokenProvider - Interface in org.apache.hadoop.fs.s3a.auth.delegation
-
Interface for S3A Delegation Token access.
- delete(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Delete a Path.
- delete(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- delete(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- delete(Path, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Deletes exactly one path, leaving a tombstone to prevent lingering,
inconsistent copies of it from being listed.
- delete(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- delete(Path, boolean) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- DELETE_CONSIDERED_IDEMPOTENT - Static variable in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
This declared delete as idempotent.
- deleteDestinationPaths(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Delete the working paths of a job.
- deleteDirectoryTree(Path, String) - Method in class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
Delete a directory tree.
- deleteFailed(Exception, List<Path>, List<Path>) - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
The delete failed.
- deleteObject(DeleteObjectRequest) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- deleteObject(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Delete an object.
- deleteObjectAtPath(Path, String, boolean, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Delete an object, also updating the metastore.
- deleteObjects(DeleteObjectsRequest) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- DeleteOperation - Class in org.apache.hadoop.fs.s3a.impl
-
Implementation of the delete() operation.
- DeleteOperation(StoreContext, S3AFileStatus, boolean, OperationCallbacks, int) - Constructor for class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
Constructor.
- deletePaths(Collection<Path>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- deletePaths(Collection<Path>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- deletePaths(Collection<Path>, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Delete the paths.
- deletePaths(Collection<Path>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- deleteQuietly(FileSystem, Path, boolean) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Delete a path quietly: failures are logged at DEBUG.
- deleteSubtree(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- deleteSubtree(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- deleteSubtree(Path, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Deletes the entire sub-tree rooted at the given path, leaving tombstones
to prevent lingering, inconsistent copies of it from being listed.
- deleteSubtree(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- deleteSuccessMarker(Path) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Delete any existing _SUCCESS file.
- deleteTaskAttemptPathQuietly(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Delete the task attempt path without raising any errors.
- deleteTaskWorkingPathQuietly(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Delete the working path of a task; no-op if there is none, that
is: this is a job.
- deleteWithWarning(FileSystem, Path, boolean) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Delete a path: failures are logged at WARN.
- deployUnbonded() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Perform any actions when deploying unbonded, and return a list
of credential providers.
- deployUnbonded() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- deployUnbonded() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Return an unbonded provider chain.
- DescendantsIterator - Class in org.apache.hadoop.fs.s3a.s3guard
-
DescendantsIterator is a RemoteIterator that implements
pre-ordering breadth-first traversal (BFS) of a path and all of its
descendants recursively.
- DescendantsIterator(MetadataStore, PathMetadata) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DescendantsIterator
-
Creates a new DescendantsIterator.
- destinationPath() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Build the destination path of the object.
- destroy() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- destroy() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
- destroy() - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- destroy() - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Destroy all resources associated with the metadata store.
- destroy() - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- destroyThreadPool() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Destroy any thread pools; wait for that to finish,
but don't overreact if it doesn't finish in time.
- DIR_MARKER_ONLY - Static variable in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
We only want the dir marker.
- DIR_MERGE_UPDATES_ALL_RECORDS_NONAUTH - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Hard-coded policy : false.
- DIRECTORIES - Static variable in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
Skip the HEAD and only look for directories.
- directory(Path) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Given a directory path, return the S3 resource to it.
- DIRECTORY_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Directory committer factory: "org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory".
- directoryCreated() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A created a directory.
- directoryDeleted() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A just deleted a directory.
- directoryMarkedAuthoritative() - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentation
-
A directory has been tagged as authoritative.
- directoryMarkedAuthoritative() - Method in class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- directoryMarkerCopied(Path, Path, boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.DelayedUpdateRenameTracker
-
- directoryMarkerCopied(Path, Path, boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.ProgressiveRenameTracker
-
A directory marker has been added.
- directoryMarkerCopied(Path, Path, boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
A directory marker has been copied.
- DirectoryStagingCommitter - Class in org.apache.hadoop.fs.s3a.commit.staging
-
This commits to a directory.
- DirectoryStagingCommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter
-
- DirectoryStagingCommitterFactory - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Factory for the Directory committer.
- DirectoryStagingCommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory
-
- DirInS3FileInMs(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.DirInS3FileInMs
-
- DirListingMetadata - Class in org.apache.hadoop.fs.s3a.s3guard
-
DirListingMetadata models a directory listing stored in a
MetadataStore.
- DirListingMetadata(Path, Collection<PathMetadata>, boolean, long) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Create a directory listing metadata container.
- DirListingMetadata(Path, Collection<PathMetadata>, boolean) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
- DirListingMetadata(DirListingMetadata) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Copy constructor.
- dirListingUnion(MetadataStore, Path, List<S3AFileStatus>, DirListingMetadata, boolean, ITtlTimeProvider) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Given directory listing metadata from both the backing store and the
MetadataStore, merge the two sources of truth to create a consistent
view of the current directory contents, which can be returned to clients.
- dirMetaToStatuses(DirListingMetadata) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Convert the data of a directory listing to an array of FileStatus
entries.
- DISABLED_LOG_MSG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Format string to use when warning that S3Guard is disabled.
- doFix(S3GuardFsck.ComparePair) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler
-
- dstFileStatus - Variable in class org.apache.hadoop.fs.s3a.S3AOpContext
-
FileStatus for "destination" path being operated on.
- dump(String, String, String, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Dump all the metrics to a string.
- dumpDiagnostics(String, String, String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Dump the diagnostics (if any) to a string.
- dumpMetastore(DumpS3GuardDynamoTable.CsvFile, Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
list children under the metastore from a base path, through
a recursive query + walk strategy.
- dumpMetrics(String, String, String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Dump the metrics (if any) to a string.
- dumpRawS3ObjectStore(DumpS3GuardDynamoTable.CsvFile) - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Dump the raw S3 Object Store.
- DumpS3GuardDynamoTable - Class in org.apache.hadoop.fs.s3a.s3guard
-
This is a low-level diagnostics entry point which does a CVE/TSV dump of
the DDB state.
- DumpS3GuardDynamoTable(String) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Instantiate.
- DumpS3GuardDynamoTable() - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Instantiate with default name.
- DumpS3GuardDynamoTable(S3AFileSystem, DynamoDBMetadataStore, File, URI) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Bind to a specific FS + store.
- dumpStore(S3AFileSystem, DynamoDBMetadataStore, Configuration, File, URI) - Static method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Entry point to dump the metastore and s3 store world views
- DynamoDBClientFactory - Interface in org.apache.hadoop.fs.s3a.s3guard
-
Interface to create a DynamoDB client.
- DynamoDBClientFactory.DefaultDynamoDBClientFactory - Class in org.apache.hadoop.fs.s3a.s3guard
-
The default implementation for creating an AmazonDynamoDB.
- DynamoDBMetadataStore - Class in org.apache.hadoop.fs.s3a.s3guard
-
DynamoDBMetadataStore is a
MetadataStore that persists
file system metadata to DynamoDB.
- DynamoDBMetadataStore() - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- DynamoDBMetadataStoreTableManager - Class in org.apache.hadoop.fs.s3a.s3guard
-
Managing dynamo tables for S3Guard dynamodb based metadatastore.
- DynamoDBMetadataStoreTableManager(DynamoDB, String, String, AmazonDynamoDB, Configuration, Invoker, RetryPolicy) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
- E_BAD_PATH - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error message for bad path: "Path does not represent a magic-commit path".
- E_DELEGATION_TOKENS_DISABLED - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
- E_DEST_EXISTS - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error text when the destination path exists and the committer
must abort the job/task "Destination path exists and committer conflict resolution mode is \"fail\"".
- E_FORBIDDEN_AWS_PROVIDER - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
Error message when the AWS provider list built up contains a forbidden
entry.
- E_INCOMPATIBLE_ITEM_VERSION - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
Error: version mismatch.
- E_INCOMPATIBLE_TAG_VERSION - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
Error: version mismatch.
- E_NEGATIVE_READAHEAD_VALUE - Static variable in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- E_NO_ARN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
Error message when there is no Role ARN.
- E_NO_AWS_CREDENTIALS - Static variable in exception org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException
-
The default error message: "No AWS Credentials".
- E_NO_MAGIC_PATH_ELEMENT - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error message for a path without a magic element in the list: "No __magic element in path".
- E_NO_ROLE - Static variable in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
- E_NO_SESSION_TOKENS_FOR_ROLE_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
If the token binding auth chain is only session-level auth, you
can't use the role binding: "Cannot issue S3A Role Delegation Tokens without full AWS credentials".
- E_NO_SUCH_BUCKET - Static variable in class org.apache.hadoop.fs.s3a.impl.ErrorTranslation.AwsErrorCodes
-
The AWS S3 error code used to recognize when a 404 means the bucket is
unknown.
- E_NO_VERSION_MARKER_AND_NOT_EMPTY - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
Error: version marker not found in table but the table is not empty.
- E_NONAUTH - Static variable in class org.apache.hadoop.fs.s3a.s3guard.AuthoritativeAuditOperation
-
Exception error string: "Directory is not marked as authoritative in the S3Guard store".
- E_NORMAL_FS - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error message if filesystem isn't magic: "Filesystem does not have support for \'magic\' committer enabled in configuration option fs.s3a.committer.magic.enabled".
- E_ON_DEMAND_NO_SET_CAPACITY - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- E_WRONG_FS - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error message if the dest FS isn't S3A: "Output path is not on an S3A Filesystem".
- effect(boolean) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Map a bool to an effect.
- effect - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
Default effect is Deny; forces callers to switch on Allow.
- elementsToKey(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Take a list of elements and create an S3 key by joining them
with "/" between each one.
- empty() - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Return a set of empty credentials.
- empty() - Static method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
Get the empty entry.
- EMPTY - Static variable in class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
- EMPTY_DIR - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Convenience parameter for passing into constructor.
- ENABLE_MULTI_DELETE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- ENCRYPTION_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- EncryptionSecretOperations - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
- EncryptionSecrets - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Encryption options in a form which can serialized or marshalled as a hadoop
Writeable.
- EncryptionSecrets() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Empty constructor, for use in marshalling.
- EncryptionSecrets(S3AEncryptionMethods, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Create a pair of secrets.
- EncryptionSecrets(String, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Create a pair of secrets.
- ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- ensureOutputParameterInRange(String, long) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Ensure that the long value is in the range of an integer.
- entryAdded(long) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentation
-
An entry was added.
- entryAdded(long) - Method in class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- entryPoint(Statistic) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Entry point to an operation.
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Equality check is on superclass and UUID only.
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Compare if this object is equal to another object.
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.s3guard.DDBPathMetadata
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard.TtlTimeProvider
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
Equality test matches user and password.
- ERROR_ENTRY_NOT_AUTH_IN_DDB - Static variable in class org.apache.hadoop.fs.s3a.s3guard.AuthoritativeAuditOperation
-
Exception error code when a path is non-auth in the DB}.
- ERROR_PATH_NOT_AUTH_IN_FS - Static variable in class org.apache.hadoop.fs.s3a.s3guard.AuthoritativeAuditOperation
-
Exception error code when a path is not configured to be
auth in the S3A FS Config: 5.
- errorIgnored() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Note that an error was ignored.
- errorln() - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- errorln(String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- ErrorTranslation - Class in org.apache.hadoop.fs.s3a.impl
-
Translate from AWS SDK-wrapped exceptions into IOExceptions with
as much information as possible.
- ErrorTranslation.AwsErrorCodes - Class in org.apache.hadoop.fs.s3a.impl
-
AWS error codes explicitly recognized and processes specially;
kept in their own class for isolation.
- ESTABLISH_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- ETAG_CHECKSUM_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Should etags be exposed as checksums?
- ETAG_CHECKSUM_ENABLED_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value: false.
- EtagMismatch(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.EtagMismatch
-
- execute() - Method in class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
Delete a file or directory tree.
- execute() - Method in class org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation
-
Execute the operation.
- execute() - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
- execute() - Method in interface org.apache.hadoop.fs.s3a.Invoker.Operation
-
- execute() - Method in interface org.apache.hadoop.fs.s3a.Invoker.VoidOperation
-
- execute() - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Dump the filesystem and the metastore.
- execute() - Method in class org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable
-
Extract the host from the FS URI, then scan and
delete all entries from that bucket.
- executeOnlyOnce() - Method in class org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation
-
Check that the operation has not been invoked twice.
- executeWith(ExecutorService) - Method in class org.apache.hadoop.fs.s3a.commit.Tasks.Builder
-
Declare executor service: if null, the tasks are executed in a single
thread.
- ExecutingStoreOperation<T> - Class in org.apache.hadoop.fs.s3a.impl
-
- ExecutingStoreOperation(StoreContext) - Constructor for class org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation
-
constructor.
- EXECUTOR_CAPACITY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- exists(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override superclass so as to add statistic collection.
- exit(int, String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- exitException(int, String, Object...) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Build a exception to throw with a formatted message.
- EXPERIMENTAL_AWS_INTERNAL_THROTTLING - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Experimental/Unstable feature: should the AWS client library retry
throttle responses before escalating to the S3A code: "fs.s3a.experimental.aws.s3.throttling".
- EXPERIMENTAL_AWS_INTERNAL_THROTTLING_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- ExpirableMetadata - Class in org.apache.hadoop.fs.s3a.s3guard
-
Expirable Metadata abstract class is for storing the field needed for
metadata classes in S3Guard that could be expired with TTL.
- ExpirableMetadata() - Constructor for class org.apache.hadoop.fs.s3a.s3guard.ExpirableMetadata
-
- extractException(String, String, ExecutionException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Extract an exception from a failed future, and convert to an IOE.
- extractIdentifier(Token<? extends AbstractS3ATokenIdentifier>) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
From a token, get the session token identifier.
- extractInnerException(IllegalArgumentException) - Static method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
Take an IllegalArgumentException raised by a DDB operation
and if it contains an inner SDK exception, unwrap it.
- extractUndeletedPaths(MultiObjectDeleteException, Function<String, Path>) - Static method in class org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteSupport
-
Build a list of undeleted paths from a MultiObjectDeleteException.
- Factory() - Constructor for class org.apache.hadoop.fs.s3a.S3AMultipartUploader.Factory
-
- fail - Variable in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
No retry on network and tangible API issues.
- FAIL_INJECT_CLIENT_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FAIL_INJECT_INCONSISTENCY_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Inconsistency (visibility delay) injection settings.
- FAIL_INJECT_INCONSISTENCY_MSEC - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FAIL_INJECT_INCONSISTENCY_PROBABILITY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FAIL_INJECT_THROTTLE_PROBABILITY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FAIL_ON_METADATA_WRITE_ERROR - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Whether to fail when there is an error writing to the metadata store.
- FAIL_ON_METADATA_WRITE_ERROR_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value (true) for FAIL_ON_METADATA_WRITE_ERROR.
- failDestinationExists(Path, String) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Generate a PathExistsException because the destination exists.
- failure(String, Throwable) - Static method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
Generate a failure exception for throwing.
- failure(String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
Generate a failure exception for throwing.
- FailureInjectionPolicy - Class in org.apache.hadoop.fs.s3a
-
Simple object which stores current failure injection settings.
- FailureInjectionPolicy(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- fakeDirsDeleted(int) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that fake directory request was made.
- FAST_BUFFER_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- FAST_UPLOAD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- FAST_UPLOAD_ACTIVE_BLOCKS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Maximum Number of blocks a single output stream can have
active (uploading, or queued to the central FileSystem
instance's pool of queued operations.
- FAST_UPLOAD_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
What buffer to use.
- FAST_UPLOAD_BUFFER_ARRAY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Use an in-memory array.
- FAST_UPLOAD_BUFFER_DISK - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Buffer blocks to disk: "disk".
- FAST_UPLOAD_BYTEBUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Use a byte buffer.
- FILE_NEVER_FOUND - Static variable in exception org.apache.hadoop.fs.s3a.RemoteFileChangedException
-
While trying to get information on a file known to S3Guard, the
file never became visible in S3.
- FILE_NOT_FOUND_SINGLE_ATTEMPT - Static variable in exception org.apache.hadoop.fs.s3a.RemoteFileChangedException
-
The file wasn't found in rename after a single attempt -the unguarded
codepath.
- fileCopied(Path, S3ObjectAttributes, S3ObjectAttributes, Path, long, boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.DelayedUpdateRenameTracker
-
- fileCopied(Path, S3ObjectAttributes, S3ObjectAttributes, Path, long, boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.ProgressiveRenameTracker
-
When a file is copied, any ancestors
are calculated and then the store is updated with
the destination entries.
- fileCopied(Path, S3ObjectAttributes, S3ObjectAttributes, Path, long, boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
A file has been copied.
- fileCreated() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A created a file.
- fileDeleted(int) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A deleted one or more files.
- FileInS3DirInMs(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.FileInS3DirInMs
-
- filename(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Get the filename of a path: the last element.
- filesCopied(int, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A copied some files within the store.
- FILESYSTEM_TEMP_PATH - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterConstants
-
The temporary path for staging data, if not explicitly set.
- finalDestination(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Calculates the final destination of a file.
- finishRename(Path, Path) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
The rename has finished; perform any store cleanup operations
such as creating/deleting directory markers.
- fixBucketRegion(String) - Static method in class org.apache.hadoop.fs.s3a.impl.NetworkBinding
-
Given an S3 bucket region as returned by a bucket location query,
fix it into a form which can be used by other AWS commands.
- fixViolation(S3AFileSystem, DynamoDBMetadataStore) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.OrphanDDBEntry
-
- fixViolation(S3AFileSystem, DynamoDBMetadataStore) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ViolationHandler
-
- fixViolations(List<S3GuardFsck.ComparePair>) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck
-
Fix violations found during check.
- FLAT_CSV - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Suffix for the flat list: "-flat.csv".
- flatmapLocatedFiles(RemoteIterator<LocatedFileStatus>, S3AUtils.LocatedFileStatusMap<Optional<T>>) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Map an operation to every LocatedFileStatus in a remote
iterator, returning a list of the all results which were not empty.
- FORCE - Static variable in class org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable
-
Name of the force option.
- foreach(Iterable<I>) - Static method in class org.apache.hadoop.fs.s3a.commit.Tasks
-
- foreach(I[]) - Static method in class org.apache.hadoop.fs.s3a.commit.Tasks
-
- forgetMetadata(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- forgetMetadata(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- forgetMetadata(Path) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Removes the record of exactly one path.
- forgetMetadata(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- formatAppAttemptDir(int) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Format the application attempt directory.
- forwardSeekOperations - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- fromAWSCredentials(AWSSessionCredentials) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Create from a set of AWS credentials.
- fromBool(boolean) - Static method in enum org.apache.hadoop.fs.s3a.Tristate
-
- fromEnvironment(Map<String, String>) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Build a set of credentials from the environment.
- fromFileStatus(FileStatus, Tristate, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Convenience constructor for creating from a vanilla FileStatus plus
an isEmptyDirectory flag.
- fromFileSystem(URI, Configuration) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Loads the credentials from the owning S3A FS, including
from Hadoop credential providers.
- fromStatusList(FileSystem, List<? extends FileStatus>) - Static method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
Create an active commit of the given pending files.
- fromSTSCredentials(Credentials) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Create a set of marshalled credentials from a set of credentials
issued by an STS call.
- fromSymbol(String) - Static method in enum org.apache.hadoop.fs.s3a.Statistic
-
Get a statistic from a symbol.
- FS_S3A - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FS_S3A_BLOCK_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FS_S3A_BUCKET_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Prefix for S3A bucket-specific properties: "fs.s3a.bucket.".
- FS_S3A_COMMITTER_NAME - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Option to identify the S3A committer:
"fs.s3a.committer.name".
- FS_S3A_COMMITTER_STAGING_ABORT_PENDING_UPLOADS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Should the staging committers abort all pending uploads to the destination
directory? Default: true.
- FS_S3A_COMMITTER_STAGING_CONFLICT_MODE - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Staging committer conflict resolution policy: "fs.s3a.committer.staging.conflict-mode".
- FS_S3A_COMMITTER_STAGING_TMP_PATH - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Path in the cluster filesystem for temporary data: "fs.s3a.committer.staging.tmp.path".
- FS_S3A_COMMITTER_STAGING_UNIQUE_FILENAMES - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Option for final files to have a uniqueness name through job attempt info,
falling back to a new UUID if there is no job attempt information to use.
- FS_S3A_COMMITTER_STAGING_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
A unique identifier to use for this work: "fs.s3a.committer.staging.uuid".
- FS_S3A_COMMITTER_THREADS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Number of threads in committers for parallel operations on files
(upload, commit, abort, delete...): "fs.s3a.committer.threads".
- FS_S3A_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Prefix for all S3A properties: "fs.s3a.".
- FS_S3A_SELECT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
- FS_S3A_SELECT_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Flag: is S3 select enabled?
Value: "fs.s3a.select.enabled".
- FS_S3A_SELECT_INPUT_CSV - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Prefix for all CSV input options.
- FS_S3A_SELECT_OUTPUT_CSV - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Prefix for all CSV output options.
- FULL_TOKEN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- FULL_TOKEN_KIND - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
- FULL_TOKEN_NAME - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Name of full token: "S3ADelegationToken/Full".
- FullCredentialsTokenBinding - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Full credentials: they are simply passed as-is, rather than
converted to a session.
- FullCredentialsTokenBinding() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- FullCredentialsTokenIdentifier - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
The full credentials payload is the same of that for a session token, but
a different token kind is used.
- FullCredentialsTokenIdentifier() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenIdentifier
-
- FullCredentialsTokenIdentifier(URI, Text, Text, MarshalledCredentials, EncryptionSecrets, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenIdentifier
-
- fullKey(S3AFileStatus) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Build the full S3 key for a request from the status entry,
possibly adding a "/" if it represents directory and it does
not have a trailing slash already.
- gauge(String, String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a gauge in the registry.
- get() - Method in class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
- get(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Lookup entry within this directory listing.
- get(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- get(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- get(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- get(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- get(Path) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Gets metadata for a path.
- get(Path, boolean) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Gets metadata for a path.
- get(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- get(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- getAccessKey() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- getAdditionalDetails() - Method in exception org.apache.hadoop.fs.s3a.AWSS3IOException
-
- getAmazonDynamoDB() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- getAmazonS3ClientForTesting(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Returns the S3 client used by this filesystem.
- getAppAttemptId(JobContext) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Get the Application Attempt ID for this job.
- getArgumentList(int, int, String) - Method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
Get the list of arguments, after validating the list size.
- getArguments() - Method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
Get the argument list.
- getAuthoritativePaths(S3AFileSystem) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
- getAWSAccessKeyId() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider.NoCredentials
-
- getAWSAccessKeys(URI, Configuration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Return the access key and secret for S3 API use.
- getAwsException() - Method in class org.apache.hadoop.fs.s3a.impl.CopyOutcome
-
- getAWSSecretKey() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider.NoCredentials
-
- getBaseMagicTaskAttemptPath(TaskAttemptContext, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Get the base Magic attempt path, without any annotations to mark relative
references.
- getBaseTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Compute the base path where the output of a task attempt is written.
- getBaseTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- getBaseTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Return the local work path as the destination for writing work.
- getBatchWriteCapacityExceededCount() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- getBoundDT() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get any bound DT.
- getBoundOrNewDT(EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get any bound DT or create a new one.
- getBucket() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- getBucket() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getBucket() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getBucket() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the bucket of this filesystem.
- getBucket() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getBucketLocation() - Method in interface org.apache.hadoop.fs.s3a.impl.ContextAccessors
-
Get the region of a bucket.
- getBucketLocation() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the location of the bucket.
- getBucketLocation() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the region of a bucket.
- getBucketLocation(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the region of a bucket; fixing up the region so it can be used
in the builders of other AWS clients.
- getBucketName() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- getBucketOption(Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a bucket-specific property.
- getBytesPendingUpload() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.OutputStreamStatistics
-
Get the number of bytes pending upload.
- getBytesRead() - Method in class org.apache.hadoop.fs.s3a.select.SelectTool
-
- getCanonicalServiceName() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
The canonical name of the service.
- getCanonicalServiceName() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Return a service name iff delegation tokens are enabled and the
token binding is issuing delegation tokens.
- getCanonicalUri() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Get the canonical URI of the filesystem, which is what is
used to identify the tokens.
- getCanonicalUri() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the canonical URI.
- getCause() - Method in exception org.apache.hadoop.fs.s3a.AWSClientIOException
-
- getCause() - Method in exception org.apache.hadoop.fs.s3a.AWSS3IOException
-
- getCause() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- getChangeDetectionPolicy() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getChangeDetectionPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the change detection policy for this FS instance.
- getChangeDetectionPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
- getCommandFormat() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- getCommitOperations() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the commit actions instance.
- getCommits() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- getCommittedBytes() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getCommittedFileCount() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getCommittedObjects() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getCommittedTaskPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Compute the path where the output of a committed task is stored until
the entire job is committed.
- getCommittedTaskPath(int, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Compute the path where the output of a committed task is stored until the
entire job is committed for a specific application attempt.
- getCommitter() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getCommonPrefixes() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- getConf() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractAWSCredentialProvider
-
- getConf() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- getConf() - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Get the configuration of this instance; essentially the owning
filesystem configuration.
- getConfictModeOption(JobContext, Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the conflict mode option string.
- getConfiguration() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getConfiguration() - Method in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Get the configuration this policy was created from.
- getConfigurationOption(JobContext, Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Get a configuration option, with any value in the job configuration
taking priority over that in the filesystem.
- getConflictResolutionMode(JobContext, Configuration) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- getContentMap() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.DDBTree
-
- getContentRangeFinish() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- getContentRangeStart() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- getCopyResult() - Method in class org.apache.hadoop.fs.s3a.impl.CopyOutcome
-
- getCounterValue(Statistic) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Get the value of a counter.
- getCounterValue(String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Get the value of a counter.
- getCreated() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- getCreated() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
When was the upload created?
- getCreationCount() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
How many delegation tokens have been issued?
- getCredentialProviders() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get the AWS credential provider.
- getCredentials() - Method in class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider
-
- getCredentials() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Get the credentials.
- getCredentials() - Method in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
Get credentials.
- getCredentials() - Method in class org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
-
Ask for the credentials.
- getCredentials() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Iterate through the list of providers, to find one with credentials.
- getCredentials() - Method in class org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
-
- getDate() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Timestamp as date; no expectation of parseability.
- getDate() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getDecodedIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get any decoded identifier from the bound DT; empty if not bound.
- getDefaultBlockSize() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getDefaultPort() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getDelayKeyMsec() - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- getDelayKeyMsec() - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- getDelayKeyProbability() - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- getDelayKeySubstring() - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- getDelegationToken(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get a delegation token if the FS is set up for them.
- getDelegationTokens() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the delegation token support for this filesystem;
not null iff delegation support is enabled.
- getDescription() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Return a description.
- getDescription() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
- getDescription() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- getDescription() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getDescription() - Method in enum org.apache.hadoop.fs.s3a.Statistic
-
- getDest() - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
- getDestFS() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the destination FS, creating it on demand if needed.
- getDestinationFileSystem(Path, JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
Get the destination filesystem, returning null if there is none.
- getDestinationFS(Path, Configuration) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the destination filesystem from the output path and the configuration.
- getDestinationKey() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getDestKey() - Method in class org.apache.hadoop.fs.s3a.commit.PutTracker
-
get the destination key.
- getDestS3AFS() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the destination as an S3A Filesystem; casting it.
- getDiagnostics() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getDiagnostics() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- getDiagnostics() - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- getDiagnostics() - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Get any diagnostics information from a store, as a list of (key, value)
tuples for display.
- getDiagnostics() - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- getDirListingMeta() - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataEntry
-
- getDstFileStatus() - Method in class org.apache.hadoop.fs.s3a.S3AOpContext
-
- getDuration() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Duration of sessions.
- getEncryptionAlgorithm() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
- getEncryptionAlgorithm(String, Configuration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get the server-side encryption algorithm.
- getEncryptionKey() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
- getEncryptionMethod() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Get the encryption method.
- getEncryptionSecrets() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Get the encryption secrets.
- getEncryptionSecrets() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get the encryption secrets of the DT.
- getEncryptionSecrets() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the encryption secrets.
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.AuthDirContentMismatch
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.DirInS3FileInMs
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.EtagMismatch
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.FileInS3DirInMs
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.LengthMismatch
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ModTimeMismatch
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.NoEtag
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.NoLastUpdatedField
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.NoMetadataEntry
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.NoParentEntry
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.OrphanDDBEntry
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ParentIsAFile
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ParentTombstoned
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.TombstonedInMsNotDeletedInS3
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.VersionIdMismatch
-
- getError() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ViolationHandler
-
- getErrorCode() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- getErrorResponseXml() - Method in exception org.apache.hadoop.fs.s3a.AWSS3IOException
-
- getETag() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
- getETag() - Method in class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- getETag() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getEtags() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getException() - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.MaybeIOE
-
Get any exception.
- getExecutor() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getExitCode() - Method in exception org.apache.hadoop.fs.s3a.RenameFailedException
-
- getExitCode() - Method in exception org.apache.hadoop.fs.s3a.s3guard.AuthoritativeAuditOperation.NonAuthoritativeDirException
-
- getExpiration() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Expiration; will be 0 for none known.
- getExpirationDateTime() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Expiration date time as passed in from source.
- getExpirationDateTime() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Get a temporal representing the time of expiration, if there
is one.
- getExpiryTime() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Return the expiry time in seconds since 1970-01-01.
- getExpiryTime() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Return the expiry time in seconds since 1970-01-01.
- getExtendedRequestId() - Method in exception org.apache.hadoop.fs.s3a.AWSS3IOException
-
- getExtraData() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Any custom extra data committer subclasses may choose to add.
- getExtraFilesDeleted() - Method in class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
- getFailureLimit() - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- getFileChecksum(Path, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
When enabled, get the etag of a object at the path via HEAD request and
return it as a checksum object.
- getFileMeta() - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataEntry
-
- getFilename() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
This is the filename of the pending file itself.
- getFilenames() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getFilesDeleted() - Method in class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
- getFileStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Return a file status object that represents the path.
- getFileStatus() - Method in class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
- getFileStatus(Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- getFilesystem() - Method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
- getFilesystem() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- getFinalKey(String, JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Returns the final S3 key for a relative path.
- getFinalPath(String, JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Returns the final S3 location for a relative path as a Hadoop Path.
- getFsDelegationToken() - Method in interface org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenProvider
-
- getFsDelegationToken() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getFsURI() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getHandler() - Method in enum org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.Violation
-
- getHostname() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getInitializationException() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Get any IOE raised during initialization.
- getInputPolicy() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getInputPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the input policy for this FS instance.
- getInputPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get the IO policy.
- getInstrumentation() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getInstrumentation() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get S3A Instrumentation.
- getInstrumentation() - Method in class org.apache.hadoop.fs.s3a.S3AOpContext
-
- getInstrumentation() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- getInstrumentation() - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Get any S3GuardInstrumentation for this store...must not be null.
- getInterruptedException() - Method in class org.apache.hadoop.fs.s3a.impl.CopyOutcome
-
- getInvoker() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Get the invoker for STS calls.
- getInvoker() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getInvoker() - Method in class org.apache.hadoop.fs.s3a.S3AOpContext
-
- getInvoker() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Get the operation invoker for write operations.
- getJobAttemptFileSystem(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the filesystem for the job attempt.
- getJobAttemptPath(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Compute the path where the output of a given job attempt will be placed.
- getJobAttemptPath(int) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Compute the path where the output of a given job attempt will be placed.
- getJobAttemptPath(int) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Compute the path where the output of a given job attempt will be placed.
- getJobAttemptPath(JobContext, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Compute the path where the output of a given job attempt will be placed.
- getJobAttemptPath(int) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- getJobAttemptPath(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
For a job attempt path, the staging committer returns that of the
wrapped committer.
- getJobContext() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the job/task context this committer was instantiated with.
- getJobId() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getKey() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- getKey() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getKind() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Get the kind of the tokens managed here.
- getLastUpdated() - Method in class org.apache.hadoop.fs.s3a.s3guard.ExpirableMetadata
-
- getLen() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getLength() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Destination file size.
- getLength() - Method in class org.apache.hadoop.fs.s3a.UploadInfo
-
- getLinesRead() - Method in class org.apache.hadoop.fs.s3a.select.SelectTool
-
Number of lines read, when printing to the console.
- getListing() - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
- getListStatusCount() - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
- getLocalTaskAttemptTempDir(Configuration, String, TaskAttemptID) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Get the task attempt temporary directory in the local filesystem.
- getLogNoVersionSupport() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getLong(String) - Method in class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- getLongStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- getMagicJobAttemptPath(int, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Compute the "magic" path for a job attempt.
- getMagicJobAttemptsPath(Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Get the location of magic job attempts.
- getMagicTaskAttemptPath(TaskAttemptContext, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Compute the path where the output of a task attempt is stored until
that task is committed.
- getMagicTaskAttemptsPath(JobContext, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Compute the path where the output of magic task attempts are stored.
- getMarshalledCredentials() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Get the marshalled credentials.
- getMessage() - Method in exception org.apache.hadoop.fs.s3a.AWSClientIOException
-
- getMetadataStore() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getMetadataStore() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the metadata store.
- getMetadataStore() - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
Get the metadata store.
- getMetadataStore(FileSystem, ITtlTimeProvider) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Create a new instance of the configured MetadataStore.
- getMetadataTtl() - Method in interface org.apache.hadoop.fs.s3a.s3guard.ITtlTimeProvider
-
- getMetadataTtl() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard.TtlTimeProvider
-
- getMethod() - Method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
- getMethod(String) - Static method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
Get the encryption mechanism from the value provided.
- getMetrics() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getMetrics(MetricsCollector, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
- getMetricsSystem() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
- getMode() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getModificationTime() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Get the modification time of the file/directory.
- getMsDirListing() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.ComparePair
-
- getMsDirListing() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ViolationHandler
-
- getMsFileStatus() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ViolationHandler
-
- getMsPathMetadata() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.ComparePair
-
- getMultipartSizeProperty(Configuration, String, long) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
- getMultiPartThreshold() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the threshold for multipart files.
- getMultipartUploadCommitsDirectory(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Build a qualified temporary path for the multipart upload commit
information in the cluster filesystem.
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the name of this committer.
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter
-
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- getName() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- getName() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Return sub-command name.
- getName() - Method in class org.apache.hadoop.fs.s3a.select.SelectTool
-
- getNow() - Method in interface org.apache.hadoop.fs.s3a.s3guard.ITtlTimeProvider
-
- getNow() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard.TtlTimeProvider
-
- getObject(GetObjectRequest) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- getObject(String, String) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- getObjectContent() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- getObjectHeaders(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get all the headers of the object of a path, if the object exists.
- getObjectMetadata() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- getObjectMetadata(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Low-level call to get at the object metadata.
- getObjectMetadata(Path, ChangeTracker, Invoker, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Low-level call to get at the object metadata.
- getObjectMetadata(String, ChangeTracker, Invoker, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Request object metadata; increments counters in the process.
- getObjectSummaries() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- getOperation() - Method in class org.apache.hadoop.fs.s3a.s3guard.BulkOperationState
-
Get the operation type.
- getOperationState() - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
- getOrigin() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- getOutputPath() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Final path of output, in the destination FS.
- getOwner() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Get the owner of this Service.
- getOwner() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the owner of the filesystem.
- getOwner() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the owner of this FS: who created it?
- getOwner() - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
- getOwnerText() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Return the name of the owner to be used in tokens.
- getParent(String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Get the parent path of a string path: everything up to but excluding
the last "/" in the path.
- getPartCount() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Get the number of etags.
- getPartition(String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Returns the partition of a relative file path, or null if the path is a
file name with no relative directory.
- getPartitions(Path, List<? extends FileStatus>) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Get the set of partitions from the list of files being staged.
- getPartitionSize() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the partition size for multipart operations.
- getPassword() - Method in class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
- getPath() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get the path of this read.
- getPath() - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
- getPath() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.ComparePair
-
- getPath() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getPathMetadata() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ViolationHandler
-
- getPolicy(Configuration) - Static method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Reads the change detection policy from Configuration.
- getPolicy(String) - Static method in enum org.apache.hadoop.fs.s3a.S3AInputPolicy
-
Choose an FS access policy.
- getPolicyProvider() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
- getPos() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- getPos() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- getRawObjectStoreCount() - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
- getRawResponseContent() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- getReadahead() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Get the current readahead value.
- getReadahead() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get the readahead for this operation.
- getReadahead() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
Get the current readahead value.
- getReadInvoker() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get invoker to use for read operations.
- getReadThrottleEventCount() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Get the count of read throttle events.
- getRedirectLocation() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- getRefCount() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Get the current reference count.
- getRegistry() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Get the metrics registry.
- getRelativePath(Path, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Using URI#relativize(), build the relative path from the
base path to the full path.
- getRequestId() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- getRetryCallback() - Method in class org.apache.hadoop.fs.s3a.Invoker
-
- getRetryPolicy() - Method in class org.apache.hadoop.fs.s3a.Invoker
-
- getRevisionId(ObjectMetadata, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Pulls the attribute this policy uses to detect change out of the S3 object
metadata.
- getRevisionId(S3ObjectAttributes) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getRevisionId(CopyResult) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getRevisionId() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- getRole() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Used in logging and reporting to help disentangle messages.
- getRoleARN() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- getRoot() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.DDBTree
-
- getS3AFileSystem(Path, Configuration, boolean) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Get the S3A FS of a path.
- getS3AStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Access the input stream statistics.
- getS3AStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
Get the Stream statistics.
- getS3DirListing() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.ComparePair
-
- getS3DirListing() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ViolationHandler
-
- getS3FileStatus() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.ComparePair
-
- getS3FileStatus() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ViolationHandler
-
- getS3GuardInstrumentation() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a MetastoreInstrumentation instrumentation instance.
- getS3guardInvoker() - Method in class org.apache.hadoop.fs.s3a.S3AOpContext
-
- getSaved() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
When was the upload saved?
- getScanEntryResult() - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
- getScanThrottleEventCount() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Get the count of scan throttle events.
- getScheme() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Return the protocol scheme for the FileSystem.
- getScheme() - Method in class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- getScheme() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
Return the protocol scheme for the FileSystem.
- getSecondScanResult() - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
- getSecretKey() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- getSecretManagerPasssword() - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Get the password to use in secret managers.
- getSelectDuration() - Method in class org.apache.hadoop.fs.s3a.select.SelectTool
-
- getServerSideEncryptionAlgorithm() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the encryption algorithm of this endpoint.
- getServerSideEncryptionAlgorithm() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getServerSideEncryptionKey(String, Configuration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get any SSE key from a configuration/credential provider.
- getServerSideEncryptionKey() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getService() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get the service identifier of the owning FS.
- getServiceName() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADtFetcher
-
Returns the service name for HDFS, which is also a valid URL prefix.
- getServiceName() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- getSessionToken() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- getSeverity() - Method in enum org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.Violation
-
- getSource() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getSource() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- getSourceFiles() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getSourceFS() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getSourceRoot() - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
- getStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
- getStats() - Method in class org.apache.hadoop.fs.s3a.S3AOpContext
-
- getStatusCode() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- getStorageStatistics() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the storage statistics of this filesystem.
- getStorageStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the storage statistics of this filesystem.
- getStore() - Method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
- getStore() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- getStoreContext() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
- getStoreContext() - Method in class org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation
-
Get the store context.
- getSymbol() - Method in enum org.apache.hadoop.fs.s3a.Statistic
-
- getTable() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
- getTableArn() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
- getTableHandler() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- getTableName() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- getTableTagsFromConfig() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
Return tags from configuration and the version marker for adding to
dynamo table during creation.
- getTaggingCount() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- getTaskAttemptFilesystem(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the task attempt path filesystem.
- getTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Compute the path where the output of a task attempt is stored until
that task is committed.
- getTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Compute the path where the output of a task attempt is stored until
that task is committed.
- getTaskAttemptPath(TaskAttemptContext, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Compute the path where the output of a task attempt is stored until
that task is committed.
- getTaskId() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getTaskOutput(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Lists the output of a task under the task attempt path.
- getTempJobAttemptPath(int, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Compute a path for temporary data associated with a job.
- getTempTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get a temporary directory for data.
- getTempTaskAttemptPath(TaskAttemptContext, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtilsWithMR
-
Compute the path where the output of a given job attempt will be placed.
- getTempTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Get a temporary directory for data.
- getTempTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- getText() - Method in enum org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.CredentialTypeRequired
-
- getText() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Arbitrary notes.
- getThrottleProbability() - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- getTimeProvider() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the time provider.
- getTimestamp() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getTokenIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Token identifier bound to.
- getTokenIssuingPolicy() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Predicate: will this binding issue a DT?
That is: should the filesystem declare that it is issuing
delegation tokens? If true
- getTokenIssuingPolicy() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
- getTokenKind() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get the kind of the issued tokens.
- getTreewalkCount() - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
- getTtlTimeProvider() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getUpload() - Method in class org.apache.hadoop.fs.s3a.UploadInfo
-
- getUploadId() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getUploadUUID(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the UUID of an upload; may be the job ID.
- getUploadUUID(Configuration, JobID) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the UUID of a Job.
- getUri() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractAWSCredentialProvider
-
Get the binding URI: may be null.
- getUri() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- getUri() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getUri() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Returns a URI whose scheme and authority identify this FileSystem.
- getUri() - Method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
- getUri() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- getUriDefaultPort() - Method in class org.apache.hadoop.fs.s3a.S3A
-
- getUsage() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- getUsage() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- getUsage() - Method in class org.apache.hadoop.fs.s3a.select.SelectTool
-
- getUser() - Method in class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
- getUserAgentField() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Return a string for use in building up the User-Agent field, so
get into the S3 access logs.
- getUserAgentField() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Return a string for use in building up the User-Agent field, so
get into the S3 access logs.
- getUserAgentField() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
UA field contains the UUID of the token if present.
- getUsername() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getUsername() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the username of the FS.
- getUuid() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Get the UUID of this token identifier.
- getUUID() - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
A UUID for this upload, as calculated with.
- getV1() - Method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
- getV1() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- getV2() - Method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
- getV2() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- getVersion() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- getVersion() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getVersionId() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
- getVersionId() - Method in class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- getVersionId() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getVersionMarkerFromTags(Table, AmazonDynamoDB) - Static method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
- getVersionMarkerItem() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
Get the version mark item in the existing DynamoDB table.
- getVersionMismatchCounter() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
Get a reference to the version mismatch counter.
- getVersionMismatches() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- getViolations() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.ComparePair
-
- getWithTtl(MetadataStore, Path, ITtlTimeProvider, boolean, boolean) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Get a path entry provided it is not considered expired.
- getWorkingDirectory() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the current working directory for the given file system.
- getWorkingDirectory() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- getWorkPath() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
This is the critical method for FileOutputFormat; it declares
the path for work.
- getWriteOperationHelper() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get a WriteOperationHelper instance.
- getWriteThrottleEventCount() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Get the count of write throttle events.
- globStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- globStatus(Path, PathFilter) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override superclass so as to disable symlink resolution and so avoid
some calls to the FS which may have problems when the store is being
inconsistent.
- GUARDED_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- IAMInstanceCredentialsProvider - Class in org.apache.hadoop.fs.s3a.auth
-
This is an IAM credential provider which wraps
an EC2ContainerCredentialsProviderWrapper
to provide credentials when the S3A connector is instantiated on AWS EC2
or the AWS container services.
- IAMInstanceCredentialsProvider() - Constructor for class org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
-
- ignoreIOExceptions(Logger, String, String, Invoker.Operation<T>) - Static method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute an operation and ignore all raised IOExceptions; log at INFO;
full stack only at DEBUG.
- ignoreIOExceptions(Logger, String, String, Invoker.VoidOperation) - Static method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute an operation and ignore all raised IOExceptions; log at INFO;
full stack only at DEBUG.
- InconsistentAmazonS3Client - Class in org.apache.hadoop.fs.s3a
-
A wrapper around AmazonS3 that injects
inconsistency and/or errors.
- InconsistentAmazonS3Client(AWSCredentialsProvider, ClientConfiguration, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
Instantiate.
- InconsistentS3ClientFactory - Class in org.apache.hadoop.fs.s3a
-
S3 Client factory used for testing with eventual consistency fault injection.
- InconsistentS3ClientFactory() - Constructor for class org.apache.hadoop.fs.s3a.InconsistentS3ClientFactory
-
- InconsistentS3InputStream(S3ObjectInputStream) - Constructor for class org.apache.hadoop.fs.s3a.InconsistentS3Object.InconsistentS3InputStream
-
- InconsistentS3Object - Class in org.apache.hadoop.fs.s3a
-
Wrapper around S3Object so we can do failure injection on
getObjectContent() and S3ObjectInputStream.
- InconsistentS3Object(S3Object, FailureInjectionPolicy) - Constructor for class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- InconsistentS3Object.InconsistentS3InputStream - Class in org.apache.hadoop.fs.s3a
-
Wraps S3ObjectInputStream and implements failure injection.
- incrementCounter(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Increment a specific counter.
- incrementCounter(Statistic, AtomicLong) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Increment a specific counter.
- incrementCounter(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
Increment a specific counter.
- incrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Increment a gauge by a specific value.
- incrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment a gauge by a specific value.
- incrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Increment a specific gauge.
- incrementPutCompletedStatistics(boolean, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
At the end of a put/multipart upload operation, update the
relevant counters and gauges.
- incrementPutProgressStatistics(String, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Callback for use in progress callbacks from put/multipart upload events.
- incrementPutStartStatistics(long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
At the start of a put/multipart upload operation, update the
relevant counters.
- incrementReadOperations() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment read operations.
- incrementStatistic(Statistic) - Method in class org.apache.hadoop.fs.s3a.impl.BulkDeleteRetryHandler
-
Increment a statistic by 1.
- incrementStatistic(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.BulkDeleteRetryHandler
-
Increment a statistic by a specific value.
- incrementStatistic(Statistic) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Increment a statistic by 1.
- incrementStatistic(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Increment a statistic by a specific value.
- incrementStatistic(Statistic) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment a statistic by 1.
- incrementStatistic(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment a statistic by a specific value.
- incrementWriteOperations() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment the write operation counter.
- init() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
- initConnectionSettings(Configuration, ClientConfiguration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Initializes all AWS SDK settings related to connection management.
- initCustomSigners() - Method in class org.apache.hadoop.fs.s3a.auth.SignerManager
-
Initialize custom signers and register them with the AWS SDK.
- initFileOutputCommitterOptions(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Init the context config with everything needed for the file output
committer.
- initialize() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
Initialize the tracker.
- initialize() - Method in class org.apache.hadoop.fs.s3a.commit.PutTracker
-
Startup event.
- initialize(URI, Configuration) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Called after a new FileSystem instance is constructed.
- initialize(Path) - Method in class org.apache.hadoop.fs.s3a.S3AMultipartUploader
-
- initialize(FileSystem, ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Performs one-time initialization of the metadata store.
- initialize(Configuration, ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Performs one-time initialization of the metadata store via configuration.
- initialize(FileSystem, ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- initialize(Configuration, ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- initialize(FileSystem, ITtlTimeProvider) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Performs one-time initialization of the metadata store.
- initialize(Configuration, ITtlTimeProvider) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Performs one-time initialization of the metadata store via configuration.
- initialize(FileSystem, ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- initialize(Configuration, ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- initialize(URI, Configuration) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
Always fail to initialize.
- initialized() - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentation
-
Initialized event.
- initialized() - Method in class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- initiateBulkWrite(BulkOperationState.OperationType, Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- initiateBulkWrite(BulkOperationState.OperationType, Path) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Initiate a bulk update and create an operation state for it.
- initiateBulkWrite(MetadataStore, BulkOperationState.OperationType, Path) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Initiate a bulk write and create an operation state for it.
- initiateCommitOperation() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Start the final commit/abort commit operations.
- initiateCommitOperation(Path) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Begin the final commit.
- initiateCommitOperation(Path) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Initiate a commit operation through any metastore.
- initiateCopy(S3ALocatedFileStatus, String, Path, String, Path) - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
Initiate a copy operation in the executor.
- initiateMultipartUpload(InitiateMultipartUploadRequest) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- initiateMultiPartUpload(String) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Start the multipart upload process.
- initiateRenameOperation(StoreContext, Path, S3AFileStatus, Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Initiate the rename operation by creating the tracker for the filesystem
to keep up to date with state changes in the S3A bucket.
- initiateRenameOperation(StoreContext, Path, S3AFileStatus, Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- initiateRenameOperation(StoreContext, Path, S3AFileStatus, Path) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Start a rename operation.
- initiateRenameOperation(StoreContext, Path, S3AFileStatus, Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- initMetadataStore(boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Parse metadata store from command line option or HDFS configuration.
- initOutput(Path) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Init the output filesystem and path.
- initProxySupport(Configuration, String, ClientConfiguration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Initializes AWS SDK proxy support in the AWS client configuration
if the S3A settings enable it.
- initS3AFileSystem(String) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Create and initialize a new S3A FileSystem instance.
- innerListStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
List the statuses of the files/directories in the given path if the path is
a directory.
- INPUT_FADV_NORMAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
General input.
- INPUT_FADV_RANDOM - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Optimized purely for random seek+read/positionedRead operations;
The performance of sequential IO may be reduced in exchange for
more efficient seek() operations.
- INPUT_FADV_SEQUENTIAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Optimized for sequential access.
- INPUT_FADVISE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Which input strategy to use for buffering, seeking and similar when
reading data.
- inputPolicy - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- inputPolicySet(int) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
The input policy has been switched.
- InternalCommitterConstants - Class in org.apache.hadoop.fs.s3a.commit
-
These are internal constants not intended for public use.
- InternalConstants - Class in org.apache.hadoop.fs.s3a.impl
-
Internal constants private only to the S3A codebase.
- InternalSelectConstants - Class in org.apache.hadoop.fs.s3a.select
-
Constants for internal use in the org.apache.hadoop.fs.s3a module itself.
- intOption(Configuration, String, int, int) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a integer option >= the minimum allowed value.
- INVALID_CREDENTIALS - Static variable in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Error text on invalid non-empty credentials: "Invalid AWS credentials".
- invalidArgs(String, Object...) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Build the exception to raise on invalid arguments.
- Invoker - Class in org.apache.hadoop.fs.s3a
-
Class to provide lambda expression invocation of AWS operations.
- Invoker(RetryPolicy, Invoker.Retried) - Constructor for class org.apache.hadoop.fs.s3a.Invoker
-
Instantiate.
- Invoker.Operation<T> - Interface in org.apache.hadoop.fs.s3a
-
Arbitrary operation throwing an IOException.
- Invoker.Retried - Interface in org.apache.hadoop.fs.s3a
-
Callback for retry and notification operations.
- Invoker.VoidOperation - Interface in org.apache.hadoop.fs.s3a
-
Void operation which may raise an IOException.
- isAuthoritative() - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
- isAuthoritativeDir() - Method in class org.apache.hadoop.fs.s3a.s3guard.DDBPathMetadata
-
- isBoundToDT() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Predicate: is there a bound DT?
- isClosed() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Get the closed flag.
- isDeleted() - Method in class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
- isDirectory(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override superclass so as to add statistic collection.
- isEmpty() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Is this empty: does it contain any credentials at all?
This test returns true if either the access key or secret key is empty.
- isEmpty() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- isEmpty() - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Is the underlying directory known to be empty?
- isEmptyDirectory() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
- isEmptyDirectory() - Method in class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
Query if a directory is empty.
- isEnabled() - Method in class org.apache.hadoop.fs.s3a.select.SelectBinding
-
Is the service supported?
- isExpired(long, long) - Method in class org.apache.hadoop.fs.s3a.s3guard.ExpirableMetadata
-
- isFile(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override superclass so as to add statistic collection.
- isInitialized() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Has an attempt to initialize the credentials been attempted?
- isMagicCommitEnabled() - Method in class org.apache.hadoop.fs.s3a.commit.MagicCommitIntegration
-
Is magic commit enabled?
- isMagicCommitEnabled() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Is magic commit enabled?
- isMagicCommitPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.MagicCommitIntegration
-
Predicate: is a path a magic commit path?
- isMagicCommitPath(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Predicate: is a path a magic commit path?
True if magic commit is enabled and the path qualifies as special.
- isMagicPath(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Is a path in the magic tree?
- isMultiObjectDeleteEnabled() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- isNullMetadataStore(MetadataStore) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Although NullMetadataStore does nothing, callers may wish to avoid work
(fast path) when the NullMetadataStore is in use.
- isRecursivelyAuthoritative() - Method in class org.apache.hadoop.fs.s3a.s3guard.MetadataStoreListFilesIterator
-
- isRequesterCharged() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- isRequireVersion() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- isRetryable() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- isRetryable() - Method in exception org.apache.hadoop.fs.s3a.CredentialInitializationException
-
This exception is not going to go away if you try calling it again.
- isS3GuardEnabled() - Method in class org.apache.hadoop.fs.s3a.S3AOpContext
-
- isServerSide() - Method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
Flag to indicate this is a server-side encryption option.
- isThrottleException(Exception) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Is the exception an instance of a throttling exception.
- isTokenRequired() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADtFetcher
-
- isTracked(String) - Method in class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- isTruncated() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- isUnknownBucket(AmazonServiceException) - Static method in class org.apache.hadoop.fs.s3a.impl.ErrorTranslation
-
Does this exception indicate that the AWS Bucket was unknown.
- isUseListV1() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- isV1() - Method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
Is this a v1 API request or v2?
- isV1() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
Is this a v1 API result or v2?
- isValid(MarshalledCredentials.CredentialTypeRequired) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Is this a valid set of credentials tokens?
- iterator() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Iterate over the etags.
- iterator() - Method in class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- iteratorToStatuses(RemoteIterator<S3AFileStatus>, Set<Path>) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
- ITtlTimeProvider - Interface in org.apache.hadoop.fs.s3a.s3guard
-
This interface is defined for handling TTL expiry of metadata in S3Guard.
- lastElement(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Last element of a (non-empty) list.
- LengthMismatch(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.LengthMismatch
-
- LIST_ONLY - Static variable in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
We only want the dir marker.
- LIST_VERSION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 API level parameters.
- listAndFilter(FileSystem, Path, boolean, PathFilter) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
List located files and filter them as a classic listFiles(path, filter)
would do.
- listAWSPolicyRules(Set<AWSPolicyProvider.AccessLevel>) - Method in interface org.apache.hadoop.fs.s3a.auth.delegation.AWSPolicyProvider
-
Get the AWS policy statements required for accessing this service.
- listAWSPolicyRules(Set<AWSPolicyProvider.AccessLevel>) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Build the AWS policy for restricted access to the resources needed
by this bucket.
- listAWSPolicyRules(Set<AWSPolicyProvider.AccessLevel>) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
The administrative policy includes all DDB table operations;
application access is restricted to those operations S3Guard operations
require when working with data in a guarded bucket.
- listChildren(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- listChildren(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- listChildren(Path) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Lists metadata for all direct children of a path.
- listChildren(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- listChildrenWithTtl(MetadataStore, Path, ITtlTimeProvider, boolean) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
List children; mark the result as non-auth if the TTL has expired.
- listFiles(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
.
- listFilesAndEmptyDirectories(Path, S3AFileStatus, boolean, boolean) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Recursive list of files and empty directories.
- listFilesAndEmptyDirectories(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Recursive List of files and empty directories.
- listFilesAndEmptyDirectoriesForceNonAuth(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Recursive List of files and empty directories, force metadatastore
to act like it is non-authoritative.
- Listing - Class in org.apache.hadoop.fs.s3a
-
Place for the S3A listing classes; keeps all the small classes under control.
- Listing(S3AFileSystem) - Constructor for class org.apache.hadoop.fs.s3a.Listing
-
- Listing.AcceptAllButSelfAndS3nDirs - Class in org.apache.hadoop.fs.s3a
-
Accept all entries except the base path and those which map to S3N
pseudo directory markers.
- listLocatedStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override superclass so as to add statistic collection.
- listLocatedStatus(Path, PathFilter) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
.
- listMultipartUploads(ListMultipartUploadsRequest) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- listMultipartUploads(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Listing all multipart uploads; limited to the first few hundred.
- listObjects(Path, String) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Create an iterator over objects in S3 only; S3Guard
is not involved.
- listObjects(ListObjectsRequest) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- listObjects(S3ListRequest) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Initiate a listObjects operation, incrementing metrics
in the process.
- listObjectsV2(ListObjectsV2Request) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- listPendingUploads(JobContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the list of pending uploads for this job attempt.
- listPendingUploadsToAbort(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the list of pending uploads for this job attempt, swallowing
exceptions.
- listPendingUploadsToCommit(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the list of pending uploads for this job attempt.
- listPendingUploadsToCommit(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Get the list of pending uploads for this job attempt, by listing
all .pendingset files in the job attempt directory.
- listPendingUploadsToCommit(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the list of pending uploads for this job attempt.
- listPendingUploadsUnderPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
List all pending uploads to the destination FS under a path.
- listProviderNames() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
List all the providers' names.
- listStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
List the statuses of the files/directories in the given path if the path is
a directory.
- listStatus(Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- listStatusFilesystem(DumpS3GuardDynamoTable.CsvFile, Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Dump the filesystem via a recursive listStatus call.
- listTombstones() - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
- listTombstones() - Method in class org.apache.hadoop.fs.s3a.s3guard.MetadataStoreListFilesIterator
-
- listUploads(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
List any pending multipart uploads whose keys begin with prefix, using
an iterator that can handle an unlimited number of entries.
- load(FileSystem, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Load an instance from a file, then validate it.
- load(FileSystem, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Load an instance from a file, then validate it.
- load(FileSystem, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Load an instance from a file, then validate it.
- loadAWSProviderClasses(Configuration, String, Class<?>...) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Load list of AWS credential provider/credential provider factory classes.
- loadSinglePendingCommits(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Load all single pending commits in the directory.
- LocalMetadataEntry - Class in org.apache.hadoop.fs.s3a.s3guard
-
LocalMetadataEntry is used to store entries in the cache of
LocalMetadataStore.
- LocalMetadataStore - Class in org.apache.hadoop.fs.s3a.s3guard
-
This is a local, in-memory implementation of MetadataStore.
- LocalMetadataStore() - Constructor for class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- locateAllSinglePendingCommits(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Locate all files with the pending suffix under a directory.
- LOCATION_UNKNOWN - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
Output when the client cannot get the location of a bucket.
- LOG - Static variable in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- LOG - Static variable in interface org.apache.hadoop.fs.s3a.s3guard.DynamoDBClientFactory
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.MetadataStoreListFilesIterator
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
- LOG - Static variable in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- LOG_EVENT - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Log retries at debug.
- LOG_EVENT - Static variable in class org.apache.hadoop.fs.s3a.Invoker
-
Log retries at debug.
- logDnsLookup(Configuration) - Static method in class org.apache.hadoop.fs.s3a.impl.NetworkBinding
-
Log the dns address associated with s3 endpoint.
- logError(S3GuardFsck.ComparePair) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler
-
- LogExactlyOnce - Class in org.apache.hadoop.fs.s3a.impl
-
Log exactly once, even across threads.
- LogExactlyOnce(Logger) - Constructor for class org.apache.hadoop.fs.s3a.impl.LogExactlyOnce
-
- Login() - Constructor for class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
Create an instance with no login details.
- Login(String, String) - Constructor for class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
- LOGIN_WARNING - Static variable in class org.apache.hadoop.fs.s3native.S3xLoginHelper
-
- logS3GuardDisabled(Logger, String, String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Log that S3Guard is disabled -optionally raise an exception.
- longBytesOption(Configuration, String, long, long) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a long option >= the minimum allowed value, supporting memory
prefixes K,M,G,T,P.
- longOption(Configuration, String, long, long) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a long option >= the minimum allowed value.
- lookupGauge(String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Look up a gauge.
- lookupMetric(String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Look up a metric from both the registered set and the lighter weight
stream entries.
- lookupPassword(String, Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Deprecated.
- lookupPassword(String, Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a password from a configuration, including JCEKS files, handling both
the absolute key and bucket override.
- lookupPassword(String, Configuration, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a password from a configuration, including JCEKS files, handling both
the absolute key and bucket override.
- lookupQuantiles(String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Look up a quantiles.
- lookupS3ADelegationToken(Credentials, URI) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Look for any S3A token for the given FS service.
- lookupToken(Credentials, Text, Text) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Look up a token from the credentials, verify it is of the correct
kind.
- lookupToken(Credentials, Text) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Look up any token from the service; cast it to one of ours.
- ls(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
List files.
- MAGIC - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- MAGIC_COMMITTER_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to indicate whether support for the Magic committer is enabled
in the filesystem.
- MAGIC_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Magic committer factory: "org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory".
- MAGIC_COMMITTER_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to indicate whether support for the Magic committer is enabled
in the filesystem.
- MAGIC_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- MagicCommitIntegration - Class in org.apache.hadoop.fs.s3a.commit
-
Adds the code needed for S3A to support magic committers.
- MagicCommitIntegration(S3AFileSystem, boolean) - Constructor for class org.apache.hadoop.fs.s3a.commit.MagicCommitIntegration
-
Instantiate.
- MagicCommitPaths - Class in org.apache.hadoop.fs.s3a.commit
-
Operations on (magic) paths.
- MagicCommitTracker - Class in org.apache.hadoop.fs.s3a.commit.magic
-
Put tracker for Magic commits.
- MagicCommitTracker(Path, String, String, String, String, WriteOperationHelper) - Constructor for class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
Magic commit tracker.
- magicElementIndex(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Get the index of the magic path element.
- magicPathChildren(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Get the child path elements under the magic path.
- magicPathParents(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Get the parent path elements of the magic path.
- MagicS3GuardCommitter - Class in org.apache.hadoop.fs.s3a.commit.magic
-
This is a dedicated committer which requires the "magic" directory feature
of the S3A Filesystem to be enabled; it then uses paths for task and job
attempts in magic paths, so as to ensure that the final output goes direct
to the destination directory.
- MagicS3GuardCommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Create a task committer.
- MagicS3GuardCommitterFactory - Class in org.apache.hadoop.fs.s3a.commit.magic
-
Factory for the Magic committer.
- MagicS3GuardCommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory
-
- magicSubdir(Path) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Get the magic subdirectory of a destination directory.
- main(String[]) - Static method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
This is the JVM entry point for the service launcher.
- main(String[]) - Static method in class org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable
-
This is the Main entry point for the service launcher.
- main(String[]) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Main entry point.
- makeDirsOrdered(MetadataStore, List<Path>, String, boolean, ITtlTimeProvider) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
- makeIOE(String, Exception) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Convert any exception to an IOE, if needed.
- makeQualified(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- mapLocatedFiles(RemoteIterator<? extends LocatedFileStatus>, S3AUtils.LocatedFileStatusMap<T>) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Map an operation to every LocatedFileStatus in a remote
iterator, returning a list of the results.
- mark(int) - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- markAsAuthoritative(Path, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Mark the directories instantiated under the destination path
as authoritative.
- markAsAuthoritative(Path, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Mark all directories created/touched in an operation as authoritative.
- markDeleted(Path, long) - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Replace an entry with a tombstone.
- markSupported() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- markSupported() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- MarshalledCredentialBinding - Class in org.apache.hadoop.fs.s3a.auth
-
- MarshalledCredentialProvider - Class in org.apache.hadoop.fs.s3a.auth
-
AWS credential provider driven from marshalled session/full credentials
(full, simple session or role).
- MarshalledCredentialProvider(String, URI, Configuration, MarshalledCredentials, MarshalledCredentials.CredentialTypeRequired) - Constructor for class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialProvider
-
Constructor.
- MarshalledCredentials - Class in org.apache.hadoop.fs.s3a.auth
-
Stores the credentials for a session or for a full login.
- MarshalledCredentials() - Constructor for class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Constructor.
- MarshalledCredentials(String, String, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Create from a set of properties.
- MarshalledCredentials.CredentialTypeRequired - Enum in org.apache.hadoop.fs.s3a.auth
-
Enumeration of credential types for use in validation methods.
- MATCH_ALL_KEYS - Static variable in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
Special config value since we can't store empty strings in XML.
- MAX_ENTRIES_TO_DELETE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
The maximum number of entries that can be deleted in any bulk delete
call to S3: 1000.
- MAX_ERROR_RETRIES - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Number of times the AWS client library should retry errors before
escalating to the S3A code: "fs.s3a.attempts.maximum".
- MAX_MULTIPART_COUNT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Maximum number of partitions in a multipart upload: 10000.
- MAX_PAGING_KEYS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- MAX_READ_FAILURES - Static variable in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- MAX_SECRET_LENGTH - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
- MAX_TEXT_LENGTH - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
The maximum string length supported for text fields.
- MAX_THREADS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- MAX_TOTAL_TASKS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- MAXIMUM_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- maybe(boolean, T) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Convert a value into a non-empty Optional instance if
the value of include is true.
- maybeAddTrailingSlash(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Turns a path (relative or otherwise) into an S3 key, adding a trailing
"/" if the path is not the root and does not already have a "/"
at the end.
- maybeAddTrailingSlash(String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Turns a path (relative or otherwise) into an S3 key, adding a trailing
"/" if the path is not the root and does not already have a "/"
at the end.
- maybeApplyConstraint(GetObjectRequest) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Apply any revision control set by the policy if it is to be
enforced on the server.
- maybeApplyConstraint(CopyObjectRequest) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Apply any revision control set by the policy if it is to be
enforced on the server.
- maybeApplyConstraint(GetObjectMetadataRequest) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- maybeAwaitCompletion(CompletableFuture<Void>) - Static method in class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
Block awaiting completion for any non-null future passed in;
No-op if a null arg was supplied.
- maybeCreateSuccessMarker(JobContext, List<String>) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- maybeCreateSuccessMarkerFromCommits(JobContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- maybeIgnore(boolean, String, Invoker.VoidOperation) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Execute an operation; maybe suppress any raised IOException.
- maybeIgnore(boolean, String, IOException) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Log or rethrow a caught IOException.
- MaybeIOE(IOException) - Constructor for class org.apache.hadoop.fs.s3a.commit.CommitOperations.MaybeIOE
-
Construct with an exception.
- maybeRethrow() - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.MaybeIOE
-
Rethrow any exception.
- maybeRetry(boolean, String, String, boolean, Invoker.Retried, Invoker.VoidOperation) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a void operation with retry processing when doRetry=true, else
just once.
- maybeRetry(boolean, String, String, boolean, Invoker.VoidOperation) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a void operation with the default retry callback invoked when
doRetry=true, else just once.
- maybeRetry(boolean, String, String, boolean, Invoker.Retried, Invoker.Operation<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with retry processing when doRetry=true, else just once.
- META_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- MetadataPersistenceException - Exception in org.apache.hadoop.fs.s3a
-
Indicates the metadata associated with the given Path could not be persisted
to the metadata store (e.g.
- MetadataPersistenceException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.MetadataPersistenceException
-
Constructs a MetadataPersistenceException.
- MetadataStore - Interface in org.apache.hadoop.fs.s3a.s3guard
-
MetadataStore defines the set of operations that any metadata store
implementation must provide.
- MetadataStore.PruneMode - Enum in org.apache.hadoop.fs.s3a.s3guard
-
Modes of operation for prune.
- METADATASTORE_AUTHORITATIVE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Whether or not to allow MetadataStore to be source of truth.
- METADATASTORE_METADATA_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
How long a directory listing in the MS is considered as authoritative.
- MetadataStoreCapabilities - Class in org.apache.hadoop.fs.s3a.s3guard
-
All the capability constants used for the
MetadataStore implementations.
- MetadataStoreListFilesIterator - Class in org.apache.hadoop.fs.s3a.s3guard
-
MetadataStoreListFilesIterator is a RemoteIterator that
is similar to DescendantsIterator but does not return directories
that have (or may have) children, and will also provide access to the set of
tombstones to allow recently deleted S3 objects to be filtered out from a
corresponding request.
- MetadataStoreListFilesIterator(MetadataStore, PathMetadata, boolean) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.MetadataStoreListFilesIterator
-
- MetastoreInstrumentation - Interface in org.apache.hadoop.fs.s3a.s3guard
-
Instrumentation exported to S3Guard.
- MetastoreInstrumentationImpl - Class in org.apache.hadoop.fs.s3a.s3guard
-
- MetastoreInstrumentationImpl() - Constructor for class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- metastoreOperationRetried(Exception, int, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Callback from
Invoker when an operation against a metastore
is retried.
- METRIC_TAG_BUCKET - Static variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
"bucket" The name of a field added to metrics records
that indicates the hostname portion of the FS URL.
- METRIC_TAG_FILESYSTEM_ID - Static variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
"s3aFileSystemId" The name of a field added to metrics
records that uniquely identifies a specific FileSystem instance.
- METRICS_SYSTEM_NAME - Static variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
"s3a-file-system" The name of the s3a-specific metrics
system instance used for s3a metrics.
- MIN_MULTIPART_THRESHOLD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- MINUTES_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- mkdirs(Path, FsPermission) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Make the given path and all non-existent parents into
directories.
- mkdirs(Path, FsPermission) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- ModTimeMismatch(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ModTimeMismatch
-
- move(Collection<Path>, Collection<PathMetadata>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Record the effects of a FileSystem.rename(Path, Path) in the
MetadataStore.
- move(Collection<Path>, Collection<PathMetadata>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- move(Collection<Path>, Collection<PathMetadata>, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Record the effects of a FileSystem.rename(Path, Path) in the
MetadataStore.
- move(Collection<Path>, Collection<PathMetadata>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- moveSourceDirectory() - Method in class org.apache.hadoop.fs.s3a.s3guard.DelayedUpdateRenameTracker
-
- moveSourceDirectory() - Method in class org.apache.hadoop.fs.s3a.s3guard.ProgressiveRenameTracker
-
- moveSourceDirectory() - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
Top level directory move.
- MultiObjectDeleteSupport - Class in org.apache.hadoop.fs.s3a.impl
-
Support for Multi Object Deletion.
- MultiObjectDeleteSupport(StoreContext, BulkOperationState) - Constructor for class org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteSupport
-
Initiate with a store context.
- MULTIPART_MIN_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The minimum multipart size which S3 supports.
- MULTIPART_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- MultipartUtils - Class in org.apache.hadoop.fs.s3a
-
MultipartUtils upload-specific functions for use by S3AFileSystem and Hadoop
CLI.
- MultipartUtils.UploadIterator - Class in org.apache.hadoop.fs.s3a
-
Iterator over multipart uploads.
- ParentIsAFile(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ParentIsAFile
-
- ParentTombstoned(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ParentTombstoned
-
- parseArgs(String[]) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Parse CLI arguments and returns the position arguments.
- parseDynamoDBRegion(List<String>) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Parse DynamoDB region from either -m option or a S3 path.
- PARTITION_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Partitioned committer factory: "org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory".
- PartitionedStagingCommitter - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Partitioned committer.
- PartitionedStagingCommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
- PartitionedStagingCommitterFactory - Class in org.apache.hadoop.fs.s3a.commit.staging
-
- PartitionedStagingCommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory
-
- path(Path, String...) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Varags constructor of paths.
- PATH_STYLE_ACCESS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PathCommitException - Exception in org.apache.hadoop.fs.s3a.commit
-
Path exception to use for various commit issues.
- PathCommitException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.commit.PathCommitException
-
- PathCommitException(String, String) - Constructor for exception org.apache.hadoop.fs.s3a.commit.PathCommitException
-
- PathCommitException(Path, String) - Constructor for exception org.apache.hadoop.fs.s3a.commit.PathCommitException
-
- PathCommitException(String, String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.commit.PathCommitException
-
- PathMetadata - Class in org.apache.hadoop.fs.s3a.s3guard
-
- PathMetadata(S3AFileStatus) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
Creates a new PathMetadata containing given FileStatus.
- PathMetadata(S3AFileStatus, long) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
Creates a new PathMetadata containing given FileStatus.
- PathMetadata(S3AFileStatus, Tristate) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
Creates a new PathMetadata.
- PathMetadata(S3AFileStatus, Tristate, boolean) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
Creates a new PathMetadata.
- PathMetadata(S3AFileStatus, Tristate, boolean, long) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
Creates a new PathMetadata.
- PathMetadataDynamoDBTranslation - Class in org.apache.hadoop.fs.s3a.s3guard
-
Defines methods for translating between domain model objects and their
representations in the DynamoDB schema.
- Paths - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Path operations for the staging committers.
- pathToKey(Path) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
- pathToKey(Path) - Method in interface org.apache.hadoop.fs.s3a.impl.ContextAccessors
-
Turns a path (relative or otherwise) into an S3 key.
- pathToKey(Path) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Turns a path (relative or otherwise) into an S3 key.
- pathToKey(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Turns a path (relative or otherwise) into an S3 key.
- pathToParentKey(Path) - Static method in class org.apache.hadoop.fs.s3a.s3guard.PathMetadataDynamoDBTranslation
-
e.g.
- PENDING_FILTER - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Filter to find all {code .pending} files.
- PENDING_SUFFIX - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Suffix applied to pending commit metadata: ".pending".
- PendingSet - Class in org.apache.hadoop.fs.s3a.commit.files
-
Persistent format for multiple pending commits.
- PendingSet() - Constructor for class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- PendingSet(int) - Constructor for class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- PENDINGSET_FILTER - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Filter to find all {code .pendingset} files.
- PENDINGSET_SUFFIX - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Suffix applied to multiple pending commit metadata: ".pendingset".
- PersistentCommitData - Class in org.apache.hadoop.fs.s3a.commit.files
-
Class for single/multiple commit data structures.
- PersistentCommitData() - Constructor for class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
- PERSISTS_AUTHORITATIVE_BIT - Static variable in class org.apache.hadoop.fs.s3a.s3guard.MetadataStoreCapabilities
-
This capability tells if the metadata store supports authoritative
directories.
- policy(RoleModel.Statement...) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
From a set of statements, create a policy.
- policy(List<RoleModel.Statement>) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
From a set of statements, create a policy.
- Policy() - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Empty constructor: initializes the statements to an empty list.
- Policy(List<RoleModel.Statement>) - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
- Policy(RoleModel.Statement...) - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
- policySetCount - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- precommitCheckPendingFiles(JobContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Run a precommit check that all files are loadable.
- preCommitJob(JobContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Subclass-specific pre-Job-commit actions.
- preCommitJob(JobContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter
-
Pre-commit actions for a job.
- preCommitJob(JobContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
All
Job-side conflict resolution.
- preCommitJob(JobContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Pre-commit actions for a job.
- PRECONDITIONS_FAILED - Static variable in exception org.apache.hadoop.fs.s3a.RemoteFileChangedException
-
- prepareSTSClient() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Get the client to AWS STS.
- prettyPrint(StringBuilder) - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Log contents to supplied StringBuilder in a pretty fashion.
- prettyPrint() - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
- prettyPrint(StringBuilder) - Method in class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
Log contents to supplied StringBuilder in a pretty fashion.
- prettyPrint() - Method in class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
- println(PrintStream, String, Object...) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Print a formatted string followed by a newline to the output stream.
- printStoreDiagnostics(PrintStream, MetadataStore) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Retrieve and Print store diagnostics.
- processDeleteFailure(MultiObjectDeleteException, List<DeleteObjectsRequest.KeyVersion>) - Method in class org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteSupport
-
Process a delete failure by removing from the metastore all entries
which where deleted, as inferred from the delete failures exception
and the original list of files to delete declares to have been deleted.
- processDeleteFailureGenericException(Exception, List<DeleteObjectsRequest.KeyVersion>) - Method in class org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteSupport
-
A delete operation failed.
- processException(SdkBaseException, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Process an exception generated against the change policy.
- processMetadata(ObjectMetadata, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Process metadata response from server for validation against the change
policy.
- processResponse(S3Object, String, long) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Process the response from the server for validation against the
change policy.
- processResponse(CopyResult) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Process the response from the server for validation against the
change policy.
- ProgressableProgressListener - Class in org.apache.hadoop.fs.s3a
-
Listener to progress from AWS regarding transfers.
- ProgressableProgressListener(S3AFileSystem, String, Upload, Progressable) - Constructor for class org.apache.hadoop.fs.s3a.ProgressableProgressListener
-
Instantiate.
- progressChanged(ProgressEvent) - Method in class org.apache.hadoop.fs.s3a.ProgressableProgressListener
-
- ProgressiveRenameTracker - Class in org.apache.hadoop.fs.s3a.s3guard
-
This rename tracker progressively updates the metadata store
as it proceeds, during the parallelized copy operation.
- ProgressiveRenameTracker(StoreContext, MetadataStore, Path, Path, BulkOperationState) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.ProgressiveRenameTracker
-
- propagateBucketOptions(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Propagates bucket-specific settings into generic S3A configuration keys.
- PROXY_DOMAIN - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_HOST - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_PASSWORD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_PORT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_USERNAME - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_WORKSTATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- prune(MetadataStore.PruneMode, long) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- prune(MetadataStore.PruneMode, long, String) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Prune files, in batches.
- prune(MetadataStore.PruneMode, long) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- prune(MetadataStore.PruneMode, long, String) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- prune(MetadataStore.PruneMode, long) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
- prune(MetadataStore.PruneMode, long, String) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
- prune(MetadataStore.PruneMode, long) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- prune(MetadataStore.PruneMode, long, String) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- PURGE_EXISTING_MULTIPART - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PURGE_EXISTING_MULTIPART_AGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PurgeS3GuardDynamoTable - Class in org.apache.hadoop.fs.s3a.s3guard
-
Purge the S3Guard table of a FileSystem from all entries related to
that table.
- PurgeS3GuardDynamoTable(String) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable
-
- PurgeS3GuardDynamoTable() - Constructor for class org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable
-
- PurgeS3GuardDynamoTable(S3AFileSystem, DynamoDBMetadataStore, URI, boolean) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable
-
- purgeStore(S3AFileSystem, DynamoDBMetadataStore, Configuration, URI, boolean) - Static method in class org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable
-
API Entry point to dump the metastore and S3 store world views
- PURPOSE - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- PURPOSE - Static variable in class org.apache.hadoop.fs.s3a.select.SelectTool
-
- put(PathMetadata) - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Add an entry to the directory listing.
- put(PathMetadata) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- put(PathMetadata, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- put(Collection<? extends PathMetadata>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- put(DirListingMetadata, List<Path>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Save directory listing metadata.
- put(PathMetadata) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- put(PathMetadata, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- put(DirListingMetadata, List<Path>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- put(Collection<? extends PathMetadata>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- put(PathMetadata) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Saves metadata for exactly one path.
- put(PathMetadata, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Saves metadata for exactly one path, potentially
using any bulk operation state to eliminate duplicate work.
- put(Collection<? extends PathMetadata>, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Saves metadata for any number of paths.
- put(DirListingMetadata, List<Path>, BulkOperationState) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
Save directory listing metadata.
- put(PathMetadata) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- put(PathMetadata, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- put(Collection<? extends PathMetadata>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- put(DirListingMetadata, List<Path>, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- putAndReturn(MetadataStore, S3AFileStatus, ITtlTimeProvider) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Helper function which puts a given S3AFileStatus into the MetadataStore and
returns the same S3AFileStatus.
- putAndReturn(MetadataStore, S3AFileStatus, ITtlTimeProvider, BulkOperationState) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Helper function which puts a given S3AFileStatus into the MetadataStore and
returns the same S3AFileStatus.
- putAuthDirectoryMarker(MetadataStore, S3AFileStatus, ITtlTimeProvider, BulkOperationState) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Creates an authoritative directory marker for the store.
- putObject(PutObjectRequest) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- putObject(PutObjectRequest) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Start a transfer-manager managed async PUT of an object,
incrementing the put requests and put bytes
counters.
- putObject(PutObjectRequest) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
PUT an object directly (i.e.
- putPart(Path, InputStream, int, UploadHandle, long) - Method in class org.apache.hadoop.fs.s3a.S3AMultipartUploader
-
- PutTracker - Class in org.apache.hadoop.fs.s3a.commit
-
Multipart put tracker.
- PutTracker(String) - Constructor for class org.apache.hadoop.fs.s3a.commit.PutTracker
-
Instantiate.
- putWithTtl(MetadataStore, DirListingMetadata, List<Path>, ITtlTimeProvider, BulkOperationState) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Put a directory entry, setting the updated timestamp of the
directory and its children.
- putWithTtl(MetadataStore, PathMetadata, ITtlTimeProvider, BulkOperationState) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Put an entry, using the time provider to set its timestamp.
- putWithTtl(MetadataStore, Collection<? extends PathMetadata>, ITtlTimeProvider, BulkOperationState) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Put entries, using the time provider to set their timestamp.
- RAW_CSV - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Suffix for the raw S3 dump: "-s3.csv".
- read() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object.InconsistentS3InputStream
-
- read(byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object.InconsistentS3InputStream
-
- read() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- read(byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
This updates the statistics on read operations started and whether
or not the read operation "completed", that is: returned the exact
number of bytes requested.
- read() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
Read a byte.
- read(byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- read(long, byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
Read at a specific position.
- READ_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- READAHEAD_RANGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
read ahead buffer size to prevent connection re-establishments.
- readException() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
An ignored stream read exception was received.
- readExceptions - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- readFields(DataInput) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Read state.
- readFields(DataInput) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Read in from the writable stream.
- readFields(DataInput) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
- readFields(DataInput) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Read in the fields.
- readFully(long, byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Subclass readFully() operation which only seeks at the start
of the series of operations; seeking back at the end.
- readFullyOperations - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- readFullyOperationStarted(long, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
A PositionedRead.read(position, buffer, offset, length)
operation has just started.
- readOperationCompleted(int, int) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
A read operation has completed.
- readOperations - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- readOperationStarted(long, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
A read(byte[] buf, int off, int len) operation has started.
- readsIncomplete - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- recordsDeleted(int) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentation
-
Records have been deleted.
- recordsDeleted(int) - Method in class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- recordsRead(int) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentation
-
Records have been read.
- recordsRead(int) - Method in class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- recordsWritten(int) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentation
-
records have been written (including tombstones).
- recordsWritten(int) - Method in class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- recoverTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Task recovery considered unsupported: Warn and fail.
- recursiveDirectoryRename() - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
Execute a full recursive rename.
- refresh() - Method in class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider
-
- refresh() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractAWSCredentialProvider
-
Refresh is a no-op by default.
- refresh() - Method in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
- refresh() - Method in class org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
-
- refresh() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Refresh all child entries.
- refresh() - Method in class org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
-
- REGION_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- registerStore(String, Configuration, DelegationTokenProvider, UserGroupInformation) - Method in interface org.apache.hadoop.fs.s3a.auth.AwsSignerInitializer
-
Register a store instance.
- rejectSecretsInURIs(URI) - Static method in class org.apache.hadoop.fs.s3native.S3xLoginHelper
-
Extract the login details from a URI, raising an exception if
the URI contains them.
- remainingInCurrentRequest() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Bytes left in the current request.
- remainingInFile() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Bytes left in stream.
- RemoteFileChangedException - Exception in org.apache.hadoop.fs.s3a
-
Indicates the S3 object is out of sync with the expected version.
- RemoteFileChangedException(String, String, String) - Constructor for exception org.apache.hadoop.fs.s3a.RemoteFileChangedException
-
Constructs a RemoteFileChangedException.
- RemoteFileChangedException(String, String, String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.RemoteFileChangedException
-
Constructs a RemoteFileChangedException.
- remove(Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Remove entry from this directory.
- removeExpiredEntriesFromListing(long, long) - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Remove expired entries from the listing based on TTL.
- removeKeys(List<DeleteObjectsRequest.KeyVersion>, boolean, List<Path>, BulkOperationState, boolean) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Remove keys from the store, updating the metastore on a
partial delete represented as a MultiObjectDeleteException failure by
deleting all those entries successfully deleted and then rethrowing
the MultiObjectDeleteException.
- removeKeys(List<DeleteObjectsRequest.KeyVersion>, boolean, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- rename(Path, Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Renames Path src to Path dst.
- rename(Path, Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- RENAME_PARALLEL_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
This is an arbitrary value: 10.
- renameFailed(Exception) - Method in class org.apache.hadoop.fs.s3a.s3guard.DelayedUpdateRenameTracker
-
- renameFailed(Exception) - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
Rename has failed.
- RenameFailedException - Exception in org.apache.hadoop.fs.s3a
-
Error to indicate that a specific rename failed.
- RenameFailedException(String, String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.RenameFailedException
-
- RenameFailedException(String, String, String) - Constructor for exception org.apache.hadoop.fs.s3a.RenameFailedException
-
- RenameFailedException(Path, Path, String) - Constructor for exception org.apache.hadoop.fs.s3a.RenameFailedException
-
- renameFileToDest() - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
The source is a file: rename it to the destination.
- RenameOperation - Class in org.apache.hadoop.fs.s3a.impl
-
A parallelized rename operation which updates the metastore in the
process, through whichever
RenameTracker the store provides.
- RenameOperation(StoreContext, Path, String, S3AFileStatus, Path, String, S3AFileStatus, OperationCallbacks, int) - Constructor for class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
Initiate the rename.
- RenameTracker - Class in org.apache.hadoop.fs.s3a.s3guard
-
A class which manages updating the metastore with the rename process
as initiated in the S3AFilesystem rename.
- RenameTracker(String, StoreContext, MetadataStore, Path, Path, BulkOperationState) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
Constructor.
- REQUEST_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- requestRole(String, String, String, long, TimeUnit) - Method in class org.apache.hadoop.fs.s3a.auth.STSClientFactory.STSClient
-
Request a set of role credentials.
- requestSessionCredentials(AWSCredentialsProvider, ClientConfiguration, String, String, int, Invoker) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Request a set of credentials from an STS endpoint.
- requestSessionCredentials(long, TimeUnit) - Method in class org.apache.hadoop.fs.s3a.auth.STSClientFactory.STSClient
-
Request a set of session credentials.
- require(boolean, String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
Require a condition to hold, otherwise an exception is thrown.
- requiresDelayedCommitOutputInFileSystem() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Flag to indicate whether or not the destination filesystem needs
to be configured to support magic paths where the output isn't immediately
visible.
- requiresDelayedCommitOutputInFileSystem() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Require magic paths in the FS client.
- requireServiceStarted() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Require the service to be started.
- requireServiceState(Service.STATE) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Require that the service is in a given state.
- reset() - Method in class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- reset() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- resetConnection() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Forcibly reset the stream, by aborting the connection.
- resetTempFolderCache() - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Reset the temp folder cache; useful in tests.
- resource(String, String, boolean) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a resource.
- resource(Path, boolean, boolean) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Given a path, return the S3 resource to it.
- resource - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- Retries - Class in org.apache.hadoop.fs.s3a
-
Annotations to inform the caller of an annotated method whether
the method performs retries and/or exception translation internally.
- Retries() - Constructor for class org.apache.hadoop.fs.s3a.Retries
-
- Retries.OnceExceptionsSwallowed - Annotation Type in org.apache.hadoop.fs.s3a
-
One attempt, Exceptions are swallowed.
- Retries.OnceMixed - Annotation Type in org.apache.hadoop.fs.s3a
-
No retry, expect a bit of both.
- Retries.OnceRaw - Annotation Type in org.apache.hadoop.fs.s3a
-
No retry, exceptions are not translated.
- Retries.OnceTranslated - Annotation Type in org.apache.hadoop.fs.s3a
-
No retry, exceptions are translated.
- Retries.RetryExceptionsSwallowed - Annotation Type in org.apache.hadoop.fs.s3a
-
Retried, Exceptions are swallowed.
- Retries.RetryMixed - Annotation Type in org.apache.hadoop.fs.s3a
-
Retried, mixed translation.
- Retries.RetryRaw - Annotation Type in org.apache.hadoop.fs.s3a
-
Retried, no translation.
- Retries.RetryTranslated - Annotation Type in org.apache.hadoop.fs.s3a
-
Retried, exceptions are translated.
- retrievePassword(AbstractS3ATokenIdentifier) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding.TokenSecretManager
-
- retry(String, String, boolean, Invoker.Retried, Invoker.VoidOperation) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a void operation with retry processing.
- retry(String, String, boolean, Invoker.VoidOperation) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a void operation with the default retry callback invoked.
- retry(String, String, boolean, Invoker.Operation<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with the default retry callback invoked.
- retry(String, String, boolean, Invoker.Retried, Invoker.Operation<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with retry processing.
- retry(String, String, boolean, Invoker.Operation<T>) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Execute a function with retry processing.
- RETRY_INTERVAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Interval between retry attempts.: "fs.s3a.retry.interval".
- RETRY_INTERVAL_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default retry interval: "500ms".
- RETRY_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Number of times to retry any repeatable S3 client request on failure,
excluding throttling requests: "fs.s3a.retry.limit".
- RETRY_LIMIT_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default retry limit: 7.
- RETRY_THROTTLE_INTERVAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Interval between retry attempts on throttled requests: "fs.s3a.retry.throttle.interval".
- RETRY_THROTTLE_INTERVAL_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default throttled retry interval: "500ms".
- RETRY_THROTTLE_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Number of times to retry any throttled request: "fs.s3a.retry.throttle.limit".
- RETRY_THROTTLE_LIMIT_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default throttled retry limit: 20.
- retryIdempotentCalls - Variable in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Idempotent calls which raise IOEs are retried.
- retrying() - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentation
-
S3Guard is retrying after a (retryable) failure.
- retrying() - Method in class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- retryUntranslated(String, boolean, Invoker.Operation<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with retry processing and no translation.
- retryUntranslated(String, boolean, Invoker.Retried, Invoker.Operation<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with retry processing: AWS SDK Exceptions
are not translated.
- revertCommit(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.CommitContext
-
- revertCommit(SinglePendingCommit, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Revert a pending commit by deleting the destination.
- revertCommit(String, BulkOperationState) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Revert a commit by deleting the file.
- revertWith(Tasks.Task<I, ?>) - Method in class org.apache.hadoop.fs.s3a.commit.Tasks.Builder
-
- ROLE_TOKEN_KIND - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
- ROLE_TOKEN_NAME - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Name of role token: "S3ADelegationToken/Role".
- RoleElt() - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.RoleElt
-
- RoleModel - Class in org.apache.hadoop.fs.s3a.auth
-
Jackson Role Model for Role Properties, for API clients and tests.
- RoleModel() - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel
-
- RoleModel.Effects - Enum in org.apache.hadoop.fs.s3a.auth
-
Effect options.
- RoleModel.Policy - Class in org.apache.hadoop.fs.s3a.auth
-
A policy is one or more statements.
- RoleModel.RoleElt - Class in org.apache.hadoop.fs.s3a.auth
-
Any element in a role.
- RoleModel.Statement - Class in org.apache.hadoop.fs.s3a.auth
-
A single statement.
- RolePolicies - Class in org.apache.hadoop.fs.s3a.auth
-
Operations, statements and policies covering the operations
needed to work with S3 and S3Guard.
- RoleTokenBinding - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Role Token support requests an explicit role and automatically restricts
that role to the given policy of the binding.
- RoleTokenBinding() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
Constructor.
- RoleTokenIdentifier - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Role token identifier.
- RoleTokenIdentifier() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenIdentifier
-
- RoleTokenIdentifier(URI, Text, Text, MarshalledCredentials, EncryptionSecrets, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenIdentifier
-
- ROOT_PATH_STRING - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck
-
- run(Tasks.Task<I, E>) - Method in class org.apache.hadoop.fs.s3a.commit.Tasks.Builder
-
- run(I, Exception) - Method in interface org.apache.hadoop.fs.s3a.commit.Tasks.FailureTask
-
process a failure.
- run(I) - Method in interface org.apache.hadoop.fs.s3a.commit.Tasks.Task
-
- run(String[], PrintStream) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- run(String[]) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- run(String[], PrintStream) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Run the tool, capturing the output (if the tool supports that).
- run(Configuration, String...) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Execute the command with the given arguments.
- run(String[], PrintStream) - Method in class org.apache.hadoop.fs.s3a.select.SelectTool
-
Execute the select operation.
- S3_ABORT_MULTIPART_UPLOAD - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Abort multipart upload is needed for the S3A Commit protocols.
- S3_ALL_BUCKETS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All S3 buckets: "arn:aws:s3:::*".
- S3_ALL_DELETE - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All s3:Delete* operations.
- S3_ALL_GET - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All s3:Get* operations.
- S3_ALL_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All S3 operations: "s3:*".
- S3_ALL_PUT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
S3 Put*.
- S3_BUCKET_ALL_LIST - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_BUCKET_LIST_BUCKET - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
List the contents of a bucket.
- S3_BUCKET_LIST_MULTIPART_UPLOADS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
This is used by the abort operation in S3A commit work.
- S3_BUCKET_READ_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Policies which can be applied to bucket resources for read operations.
- S3_CLIENT_FACTORY_IMPL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- S3_DELETE_OBJECT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_DELETE_OBJECT_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_DELETE_OBJECT_VERSION - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_DELETE_OBJECT_VERSION_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_BUCKET_LOCATION - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_ACL - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_TORRENT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_VERSION - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_VERSION_ACL - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_VERSION_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_VERSION_TORRENT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_LIST_MULTIPART_UPLOAD_PARTS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
List multipart upload is needed for the S3A Commit protocols.
- S3_METADATA_STORE_IMPL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- S3_PATH_RW_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Actions needed to write data to an S3A Path.
- S3_PATH_WRITE_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Actions needed to write data to an S3A Path.
- S3_PUT_OBJECT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_PUT_OBJECT_ACL - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_PUT_OBJECT_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_PUT_OBJECT_VERSION_ACL - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_PUT_OBJECT_VERSION_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_RESTORE_OBJECT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_ROOT_READ_OPERATIONS_LIST - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_ROOT_RW_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Actions needed for R/W IO from the root of a bucket.
- S3_SELECT_CAPABILITY - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Does the FS Support S3 Select?
Value: "fs.s3a.capability.select.sql".
- S3A - Class in org.apache.hadoop.fs.s3a
-
S3A implementation of AbstractFileSystem.
- S3A(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.S3A
-
- S3A_BUCKET_PROBE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Bucket validation parameter which can be set by client.
- S3A_BUCKET_PROBE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value of bucket validation parameter.
- S3A_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
S3 Committer factory: "org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory".
- S3A_COMMITTER_FACTORY_KEY - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Key to set for the S3A schema to use the specific committer.
- S3A_DEFAULT_PORT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default port for this is 443: HTTPS.
- S3A_SECURITY_CREDENTIAL_PROVIDER_PATH - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Extra set of security credentials which will be prepended to that
set in "hadoop.security.credential.provider.path".
- S3ACommitterFactory - Class in org.apache.hadoop.fs.s3a.commit
-
- S3ACommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory
-
- S3ADelegationTokens - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Support for creating a DT from a filesystem.
- S3ADelegationTokens() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Instantiate.
- S3ADelegationTokens.TokenIssuingPolicy - Enum in org.apache.hadoop.fs.s3a.auth.delegation
-
- S3ADtFetcher - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
A DT fetcher for S3A.
- S3ADtFetcher() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.S3ADtFetcher
-
- S3AEncryptionMethods - Enum in org.apache.hadoop.fs.s3a
-
This enum is to centralize the encryption methods and
the value required in the configuration.
- S3AFileStatus - Class in org.apache.hadoop.fs.s3a
-
File status for an S3A "file".
- S3AFileStatus(boolean, Path, String) - Constructor for class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Create a directory status.
- S3AFileStatus(Tristate, Path, String) - Constructor for class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Create a directory status.
- S3AFileStatus(long, long, Path, long, String, String, String) - Constructor for class org.apache.hadoop.fs.s3a.S3AFileStatus
-
A simple file.
- S3AFileSystem - Class in org.apache.hadoop.fs.s3a
-
The core S3A Filesystem implementation.
- S3AFileSystem() - Constructor for class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- S3AInputPolicy - Enum in org.apache.hadoop.fs.s3a
-
Filesystem input policy.
- S3AInputStream - Class in org.apache.hadoop.fs.s3a
-
The input stream for an S3A object.
- S3AInputStream(S3AReadOpContext, S3ObjectAttributes, AmazonS3) - Constructor for class org.apache.hadoop.fs.s3a.S3AInputStream
-
Create the stream.
- S3AInstrumentation - Class in org.apache.hadoop.fs.s3a
-
Instrumentation of S3a.
- S3AInstrumentation(URI) - Constructor for class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
- S3AInstrumentation.CommitterStatistics - Class in org.apache.hadoop.fs.s3a
-
Instrumentation exported to S3Guard Committers.
- S3AInstrumentation.DelegationTokenStatistics - Class in org.apache.hadoop.fs.s3a
-
Instrumentation exported to S3A Delegation Token support.
- S3AInstrumentation.InputStreamStatistics - Class in org.apache.hadoop.fs.s3a
-
Statistics updated by an input stream during its actual operation.
- S3AInstrumentation.OutputStreamStatistics - Class in org.apache.hadoop.fs.s3a
-
Statistics updated by an output stream during its actual operation.
- S3ALocatedFileStatus - Class in org.apache.hadoop.fs.s3a
-
LocatedFileStatus extended to also carry ETag and object version ID.
- S3ALocatedFileStatus(S3AFileStatus, BlockLocation[]) - Constructor for class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- S3AMultipartUploader - Class in org.apache.hadoop.fs.s3a
-
MultipartUploader for S3AFileSystem.
- S3AMultipartUploader(FileSystem, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.S3AMultipartUploader
-
- S3AMultipartUploader.Factory - Class in org.apache.hadoop.fs.s3a
-
Factory for creating MultipartUploader objects for s3a:// FileSystems.
- S3AOpContext - Class in org.apache.hadoop.fs.s3a
-
Base class for operation context struct passed through codepaths for main
S3AFileSystem operations.
- S3AOpContext(boolean, Invoker, Invoker, FileSystem.Statistics, S3AInstrumentation, FileStatus) - Constructor for class org.apache.hadoop.fs.s3a.S3AOpContext
-
Alternate constructor that allows passing in two invokers, the common
one, and another with the S3Guard Retry Policy.
- S3AOpContext(boolean, Invoker, FileSystem.Statistics, S3AInstrumentation, FileStatus) - Constructor for class org.apache.hadoop.fs.s3a.S3AOpContext
-
Constructor using common invoker and retry policy.
- S3AReadOpContext - Class in org.apache.hadoop.fs.s3a
-
Read-specific operation context struct.
- S3AReadOpContext(Path, boolean, Invoker, Invoker, FileSystem.Statistics, S3AInstrumentation, FileStatus, S3AInputPolicy, ChangeDetectionPolicy, long) - Constructor for class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Instantiate.
- S3ARetryPolicy - Class in org.apache.hadoop.fs.s3a
-
The S3A request retry policy.
- S3ARetryPolicy(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Instantiate.
- S3AStorageStatistics - Class in org.apache.hadoop.fs.s3a
-
Storage statistics for S3A.
- S3AStorageStatistics() - Constructor for class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- S3AUtils - Class in org.apache.hadoop.fs.s3a
-
Utility methods for S3A code.
- S3AUtils.CallOnLocatedFileStatus - Interface in org.apache.hadoop.fs.s3a
-
An interface for use in lambda-expressions working with
directory tree listings.
- S3AUtils.LocatedFileStatusMap<T> - Interface in org.apache.hadoop.fs.s3a
-
An interface for use in lambda-expressions working with
directory tree listings.
- S3ClientFactory - Interface in org.apache.hadoop.fs.s3a
-
Factory for creation of AmazonS3 client instances.
- S3Guard - Class in org.apache.hadoop.fs.s3a.s3guard
-
Logic for integrating MetadataStore with S3A.
- S3Guard.DisabledWarnLevel - Enum in org.apache.hadoop.fs.s3a.s3guard
-
Warning levels to use when reporting S3Guard as disabled.
- S3Guard.TtlTimeProvider - Class in org.apache.hadoop.fs.s3a.s3guard
-
Runtime implementation for TTL Time Provider interface.
- S3GUARD_CLI_PRUNE_AGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Minimum period of time (in milliseconds) to keep metadata (may only be
applied when a prune command is manually run).
- S3GUARD_CONSISTENCY_RETRY_INTERVAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Initial retry interval: "fs.s3a.s3guard.consistency.retry.interval".
- S3GUARD_CONSISTENCY_RETRY_INTERVAL_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default initial retry interval: "2s".
- S3GUARD_CONSISTENCY_RETRY_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Number of times to retry any repeatable S3 client request on failure,
excluding throttling requests: "fs.s3a.s3guard.consistency.retry.limit".
- S3GUARD_CONSISTENCY_RETRY_LIMIT_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default retry limit: 7.
- S3GUARD_DDB_BACKGROUND_SLEEP_MSEC_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- S3GUARD_DDB_BACKGROUND_SLEEP_MSEC_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Period of time (in milliseconds) to sleep between batches of writes.
- S3GUARD_DDB_BATCH_WRITE_REQUEST_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The maximum put or delete requests per BatchWriteItem request.
- S3GUARD_DDB_CLIENT_FACTORY_IMPL - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
- S3GUARD_DDB_MAX_RETRIES - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- S3GUARD_DDB_MAX_RETRIES_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Max retries on batched/throttled DynamoDB operations before giving up and
throwing an IOException.
- S3GUARD_DDB_REGION_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The region of the DynamoDB service.
- S3GUARD_DDB_TABLE_CAPACITY_READ_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default read capacity when creating a table.
- S3GUARD_DDB_TABLE_CAPACITY_READ_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Read capacity when creating a table.
- S3GUARD_DDB_TABLE_CAPACITY_WRITE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default write capacity when creating a table.
- S3GUARD_DDB_TABLE_CAPACITY_WRITE_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Write capacity when creating a table.
- S3GUARD_DDB_TABLE_CREATE_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Whether to create the DynamoDB table if the table does not exist.
- S3GUARD_DDB_TABLE_NAME_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The DynamoDB table name to use.
- S3GUARD_DDB_TABLE_SSE_CMK - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The KMS Master Key (CMK) used for the KMS encryption on the table.
- S3GUARD_DDB_TABLE_SSE_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Whether server-side encryption (SSE) is enabled or disabled on the table.
- S3GUARD_DDB_TABLE_TAG - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
A prefix for adding tags to the DDB Table upon creation.
- S3GUARD_DDB_THROTTLE_RETRY_INTERVAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- S3GUARD_DDB_THROTTLE_RETRY_INTERVAL_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- S3GUARD_DISABLED_WARN_LEVEL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The warn level if S3Guard is disabled.
- S3GUARD_METASTORE_DYNAMO - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Use DynamoDB for the metadata: "org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore".
- S3GUARD_METASTORE_LOCAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Use Local memory for the metadata: "org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore".
- S3GUARD_METASTORE_LOCAL_ENTRY_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Time to live in milliseconds in LocalMetadataStore.
- S3GUARD_METASTORE_LOCAL_MAX_RECORDS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Maximum number of records in LocalMetadataStore.
- S3GUARD_METASTORE_NULL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The default "Null" metadata store: "org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore".
- S3GuardDataAccessRetryPolicy - Class in org.apache.hadoop.fs.s3a.s3guard
-
A Retry policy whose throttling comes from the S3Guard config options.
- S3GuardDataAccessRetryPolicy(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardDataAccessRetryPolicy
-
- S3GuardExistsRetryPolicy - Class in org.apache.hadoop.fs.s3a
-
Slightly-modified retry policy for cases when the file is present in the
MetadataStore, but may be still throwing FileNotFoundException from S3.
- S3GuardExistsRetryPolicy(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.S3GuardExistsRetryPolicy
-
Instantiate.
- S3GuardFsck - Class in org.apache.hadoop.fs.s3a.s3guard
-
Main class for the FSCK factored out from S3GuardTool
The implementation uses fixed DynamoDBMetadataStore as the backing store
for metadata.
- S3GuardFsck(S3AFileSystem, MetadataStore) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck
-
Creates an S3GuardFsck.
- S3GuardFsck.ComparePair - Class in org.apache.hadoop.fs.s3a.s3guard
-
A compare pair with the pair of metadata and the list of violations.
- S3GuardFsck.DDBTree - Class in org.apache.hadoop.fs.s3a.s3guard
-
DDBTree is the tree that represents the structure of items in the DynamoDB.
- S3GuardFsck.Violation - Enum in org.apache.hadoop.fs.s3a.s3guard
-
Violation with severity and the handler.
- S3GuardFsckViolationHandler - Class in org.apache.hadoop.fs.s3a.s3guard
-
Violation handler for the S3Guard's fsck.
- S3GuardFsckViolationHandler(S3AFileSystem, DynamoDBMetadataStore) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler
-
- S3GuardFsckViolationHandler.AuthDirContentMismatch - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's a directory listing content mismatch.
- S3GuardFsckViolationHandler.DirInS3FileInMs - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's a directory is a file metadata in MS.
- S3GuardFsckViolationHandler.EtagMismatch - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's an etag mismatch.
- S3GuardFsckViolationHandler.FileInS3DirInMs - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when a file metadata is a directory in MS.
- S3GuardFsckViolationHandler.HandleMode - Enum in org.apache.hadoop.fs.s3a.s3guard
-
- S3GuardFsckViolationHandler.LengthMismatch - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's a length mismatch.
- S3GuardFsckViolationHandler.ModTimeMismatch - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's a modtime mismatch.
- S3GuardFsckViolationHandler.NoEtag - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's no etag.
- S3GuardFsckViolationHandler.NoLastUpdatedField - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's no last updated field for the entry.
- S3GuardFsckViolationHandler.NoMetadataEntry - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's no matching metadata entry in the MS.
- S3GuardFsckViolationHandler.NoParentEntry - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's no parent entry.
- S3GuardFsckViolationHandler.OrphanDDBEntry - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler there's no parent in the MetadataStore.
- S3GuardFsckViolationHandler.ParentIsAFile - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when the parent of an entry is a file.
- S3GuardFsckViolationHandler.ParentTombstoned - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when the parent of an entry is tombstoned.
- S3GuardFsckViolationHandler.TombstonedInMsNotDeletedInS3 - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's a tombstoned entry in the ms is
present, but the object is not deleted in S3.
- S3GuardFsckViolationHandler.VersionIdMismatch - Class in org.apache.hadoop.fs.s3a.s3guard
-
The violation handler when there's a version id mismatch.
- S3GuardFsckViolationHandler.ViolationHandler - Class in org.apache.hadoop.fs.s3a.s3guard
-
Violation handler abstract class.
- S3GuardTool - Class in org.apache.hadoop.fs.s3a.s3guard
-
CLI to manage S3Guard Metadata Store.
- S3GuardTool(Configuration, String...) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Constructor a S3Guard tool with HDFS configuration.
- S3GuardTool.BucketInfo - Class in org.apache.hadoop.fs.s3a.s3guard
-
Get info about a bucket and its S3Guard integration status.
- S3ListRequest - Class in org.apache.hadoop.fs.s3a
-
API version-independent container for S3 List requests.
- S3ListRequest(ListObjectsRequest, ListObjectsV2Request) - Constructor for class org.apache.hadoop.fs.s3a.S3ListRequest
-
- S3ListResult - Class in org.apache.hadoop.fs.s3a
-
API version-independent container for S3 List responses.
- S3ListResult(ObjectListing, ListObjectsV2Result) - Constructor for class org.apache.hadoop.fs.s3a.S3ListResult
-
- S3N_FOLDER_SUFFIX - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- S3ObjectAttributes - Class in org.apache.hadoop.fs.s3a
-
This class holds attributed of an object independent of the
file status type.
- S3ObjectAttributes(String, Path, String, S3AEncryptionMethods, String, String, String, long) - Constructor for class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- S3ObjectAttributes(Path, CopyResult, S3AEncryptionMethods, String, long) - Constructor for class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
Construct from the result of a copy and those parameters
which aren't included in an AWS SDK response.
- S3xLoginHelper - Class in org.apache.hadoop.fs.s3native
-
Class to aid logging in to S3 endpoints.
- S3xLoginHelper.Login - Class in org.apache.hadoop.fs.s3native
-
Simple tuple of login details.
- save(FileSystem, Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- save(FileSystem, Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Save to a hadoop filesystem.
- save(FileSystem, Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- save(FileSystem, Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- SC_404 - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
404 error code.
- SC_PRECONDITION_FAILED - Static variable in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
412 Precondition Failed (HTTP/1.1 - RFC 2616)
- SCAN2_CSV - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Suffix for the second DDB scan: : "-scan-2.csv".
- SCAN_CSV - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Suffix for the DDB scan: "-scan.csv".
- SECONDS_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- SECRET_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SECURE_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- seek(long) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- seek(long) - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
Forward seeks are supported, but not backwards ones.
- SEEK_UNSUPPORTED - Static variable in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- seekBackwards(long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
Seek backwards, incrementing the seek and backward seek counters.
- seekForwards(long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
Record a forward seek, adding a seek operation, a forward
seek operation, and any bytes skipped.
- seekOperations - Variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
- seekToNewSource(long) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- seekToNewSource(long) - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- select(S3AReadOpContext, String, Configuration, Optional<SSECustomerKey>, S3ObjectAttributes) - Method in class org.apache.hadoop.fs.s3a.select.SelectBinding
-
Build and execute a select request.
- select(Path, SelectObjectContentRequest, String) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Execute an S3 Select operation.
- SELECT_ERRORS_INCLUDE_SQL - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Should Select errors include the SQL statement?
It is easier to debug but a security risk if the exceptions
ever get printed/logged and the query contains secrets.
- SELECT_FORMAT_CSV - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
CSV as an input or output format: "csv".
- SELECT_FORMAT_JSON - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
JSON as an input or output format: "json".
- SELECT_INPUT_COMPRESSION - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
How is the input compressed? This applies to all formats.
- SELECT_INPUT_FORMAT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Input format for data.
- SELECT_OPTIONS - Static variable in class org.apache.hadoop.fs.s3a.select.InternalSelectConstants
-
An unmodifiable set listing the options
supported in openFile().
- SELECT_OUTPUT_FORMAT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Output format for data -that is, what the results are generated
as.
- SELECT_SQL - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
This is the big SQL expression: "fs.s3a.select.sql".
- SELECT_UNSUPPORTED - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
- SelectBinding - Class in org.apache.hadoop.fs.s3a.select
-
Class to do the S3 select binding and build a select request from the
supplied arguments/configuration.
- SelectBinding(WriteOperationHelper) - Constructor for class org.apache.hadoop.fs.s3a.select.SelectBinding
-
Constructor.
- SelectConstants - Class in org.apache.hadoop.fs.s3a.select
-
Options related to S3 Select.
- SelectInputStream - Class in org.apache.hadoop.fs.s3a.select
-
An input stream for S3 Select return values.
- SelectInputStream(S3AReadOpContext, S3ObjectAttributes, SelectObjectContentResult) - Constructor for class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
Create the stream.
- selectTokenFromFSOwner() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Find a token for the FS user and canonical filesystem URI.
- SelectTool - Class in org.apache.hadoop.fs.s3a.select
-
This is a CLI tool for the select operation, which is available
through the S3Guard command.
- SelectTool(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.select.SelectTool
-
- serializer() - Static method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Get a JSON serializer for this class.
- serializer() - Static method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Get a JSON serializer for this class.
- SERVER_SIDE_ENCRYPTION_AES256 - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- SERVER_SIDE_ENCRYPTION_ALGORITHM - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SERVER_SIDE_ENCRYPTION_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- serviceInit(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
- serviceInit(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
- serviceInit(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Init the service.
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Service startup: create the secret manager.
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Service startup includes binding to any delegation token, and
deploying unbounded if there is none.
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Service start will read in all configuration options
then build that client.
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Bind to the argument list, including validating the CLI.
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.s3guard.PurgeS3GuardDynamoTable
-
Bind to the argument list, including validating the CLI.
- serviceStop() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Stop the token binding.
- serviceStop() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- SESSION_TOKEN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- SESSION_TOKEN - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
session token for when using TemporaryAWSCredentialsProvider: : "fs.s3a.session.token".
- SESSION_TOKEN_KIND - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
- SESSION_TOKEN_NAME - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Name of session token: "S3ADelegationToken/Session".
- SessionTokenBinding - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
The session token DT binding: creates an AWS session token
for the DT, extracts and serves it up afterwards.
- SessionTokenBinding() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Constructor for reflection.
- SessionTokenBinding(String, Text) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Constructor for subclasses.
- SessionTokenIdentifier - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
A token identifier which contains a set of AWS session credentials,
credentials which will be valid until they expire.
- SessionTokenIdentifier() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Constructor for service loader use.
- SessionTokenIdentifier(Text) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Constructor for subclasses.
- SessionTokenIdentifier(Text, Text, Text, URI, MarshalledCredentials, EncryptionSecrets, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Constructor.
- SessionTokenIdentifier(Text, Text, Text, Text, URI) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Constructor.
- setAccessKey(String) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setAllowed(boolean) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- setAmazonS3Client(AmazonS3) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Set the client -used in mocking tests to force in a different client.
- setAuthoritative(boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
Marks this directory listing as full and authoritative.
- setAuthoritativeDir(boolean) - Method in class org.apache.hadoop.fs.s3a.s3guard.DDBPathMetadata
-
- setBucket(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setBucket(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Set the bucket.
- setBucketName(String) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- setBucketOption(Configuration, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Set a bucket-specific property to a particular value.
- setCommits(List<SinglePendingCommit>) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- setCommitter(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setConf(Configuration) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- setCreated(long) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setDate(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setDate(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setDescription(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setDestFS(FileSystem) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Set the destination FS: the FS of the final output.
- setDestinationKey(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setDiagnostics(Map<String, String>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setDirListingMetadata(DirListingMetadata) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataEntry
-
- setEncryptionSecrets(EncryptionSecrets) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Set the encryption secrets for requests.
- setEtags(List<String>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setExpiration(long) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setExpirationDateTime(Optional<OffsetDateTime>) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- setExtraData(Map<String, String>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setFailureInjectionPolicy(S3AFileSystem, FailureInjectionPolicy) - Static method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
A way for tests to patch in a different fault injection policy at runtime.
- setFailureLimit(int) - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- setFailureLimit(int) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
Set the limit on failures before all operations pass through.
- setFilename(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setFilenames(List<String>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setHostname(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setIfDefined(Configuration, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Set a key if the value is non-empty.
- setInputPolicy(S3AInputPolicy) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Change the input policy for this FS.
- setIsEmptyDirectory(Tristate) - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Update isEmptyDirectory attribute.
- setJobId(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setKey(String) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- setLastUpdated(long) - Method in class org.apache.hadoop.fs.s3a.s3guard.ExpirableMetadata
-
- setLength(long) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setMetrics(Map<String, Long>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setName(String) - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Set the name; adds a ": " if needed.
- setName(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setObjectContent(S3ObjectInputStream) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- setObjectContent(InputStream) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- setObjectMetadata(ObjectMetadata) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- setOptionalMultipartUploadRequestParameters(InitiateMultipartUploadRequest) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Set the optional parameters when initiating the request (encryption,
headers, storage, etc).
- setOptionalUploadPartRequestParameters(UploadPartRequest) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Sets server side encryption parameters to the part upload
request when encryption is enabled.
- setOrigin(String) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- setOutputPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Set the output path.
- setPathMetadata(PathMetadata) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataEntry
-
- setReadahead(Long) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- setReadahead(Long) - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
Set the readahead.
- setRedirectLocation(String) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- setRequesterCharged(boolean) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- setRoleARN(String) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setRoot(S3GuardFsck.DDBTreeNode) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.DDBTree
-
- setSaved(long) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setSecretKey(String) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setSecretsInConfiguration(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Patch a configuration with the secrets.
- setSessionToken(String) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setStore(MetadataStore) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- setTaggingCount(Integer) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- setTaskId(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setText(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setThrottleProbability(float) - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
Set the probability of throttling a request.
- setThrottleProbability(float) - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
Set the probability of throttling a request.
- setTimestamp(long) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setTokenIdentifier(Optional<SessionTokenIdentifier>) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- setTtlTimeProvider(ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- setTtlTimeProvider(ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- setTtlTimeProvider(ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- setTtlTimeProvider(ITtlTimeProvider) - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetadataStore
-
The TtlTimeProvider has to be set during the initialization for the
metadatastore, but this method can be used for testing, and change the
instance during runtime.
- setTtlTimeProvider(ITtlTimeProvider) - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- setupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Base job setup (optionally) deletes the success marker and
always creates the destination directory.
- setupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- setupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter
-
- setupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Set up the job, including calling the same method on the
wrapped committer.
- setUploadId(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setupTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- setupTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- setupTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- setUri(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setUri(URI, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- setUri(URI) - Method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
- setUri(String) - Method in class org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardDynamoDBDiagnostic
-
Set the URI from a string; will add a "/" if needed.
- setVersion(int) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- setVersion(int) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setVersionId(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
set the S3 object versionId, else null.
- setWorkingDirectory(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Set the current working directory for the given file system.
- setWorkingDirectory(Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- setWorkPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Set the work path for this committer.
- share() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Get a reference to this object with an updated reference count.
- shareCredentials(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get a shared copy of the AWS credentials, with its reference
counter updated.
- SharedInstanceCredentialProvider - Class in org.apache.hadoop.fs.s3a
-
This credential provider has jittered between existing and non-existing,
but it turns up in documentation enough that it has been restored.
- SharedInstanceCredentialProvider() - Constructor for class org.apache.hadoop.fs.s3a.SharedInstanceCredentialProvider
-
- shouldDelay(String) - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
Should we delay listing visibility for this key?
- shouldRetry(Exception, int, int, boolean) - Method in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
- sid - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- SignerManager - Class in org.apache.hadoop.fs.s3a.auth
-
Class to handle custom signers.
- SignerManager(String, DelegationTokenProvider, Configuration, UserGroupInformation) - Constructor for class org.apache.hadoop.fs.s3a.auth.SignerManager
-
- signifiesConnectionBroken(SdkBaseException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Cue that an AWS exception is likely to be an EOF Exception based
on the message coming back from an XML/JSON parser.
- SIGNING_ALGORITHM - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
There's 3 parameters that can be used to specify a non-default signing
algorithm.
fs.s3a.signing-algorithm - This property has existed for the longest time.
- SIGNING_ALGORITHM_DDB - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SIGNING_ALGORITHM_S3 - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SIGNING_ALGORITHM_STS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SimpleAWSCredentialsProvider - Class in org.apache.hadoop.fs.s3a
-
Support simple credentials for authenticating with AWS.
- SimpleAWSCredentialsProvider(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
-
Build the credentials from a filesystem URI and configuration.
- singleCommitThreadPool() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the thread pool for executing the single file commit/revert
within the commit of all uploads of a single task.
- SinglePendingCommit - Class in org.apache.hadoop.fs.s3a.commit.files
-
This is the serialization format for uploads yet to be committerd.
- SinglePendingCommit() - Constructor for class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- size() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Get the size of this list.
- size() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- size() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Number of commits.
- skip(long) - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object.InconsistentS3InputStream
-
- skip(long) - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
- SOCKET_RECV_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SOCKET_SEND_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SOCKET_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- sourceObjectsDeleted(Collection<Path>) - Method in class org.apache.hadoop.fs.s3a.s3guard.DelayedUpdateRenameTracker
-
- sourceObjectsDeleted(Collection<Path>) - Method in class org.apache.hadoop.fs.s3a.s3guard.ProgressiveRenameTracker
-
As source objects are deleted, so is the list of entries.
- sourceObjectsDeleted(Collection<Path>) - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
Note that source objects have been deleted.
- SPARK_APP_ID - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterConstants
-
The App ID for jobs.
- SPARK_WRITE_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterConstants
-
The UUID for jobs: "spark.sql.sources.writeJobUUID".
- splitPathToElements(Path) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Take an absolute path, split it into a list of elements.
- splitUndeletedKeys(MultiObjectDeleteException, Collection<DeleteObjectsRequest.KeyVersion>) - Method in class org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteSupport
-
Process a multi object delete exception by building two paths from
the delete request: one of all deleted files, one of all undeleted values.
- SSE_C_NO_KEY_ERROR - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
Encryption SSE-C used but the config lacks an encryption key.
- SSE_DEFAULT_MASTER_KEY - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
The AWS managed CMK for DynamoDB server side encryption.
- SSE_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- SSE_S3_WITH_KEY_ERROR - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
Encryption SSE-S3 is used but the caller also set an encryption key.
- SSL_CHANNEL_MODE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- STAGING_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Directory committer factory: "org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory".
- STAGING_UPLOADS - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterConstants
-
Filename used under ~/${UUID} for the staging files.
- StagingCommitter - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Committer based on the contributed work of the
Netflix multipart committers.
The working directory of each task is actually under a temporary
path in the local filesystem; jobs write directly into it.
- StagingCommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Committer for a single task attempt.
- StagingCommitterConstants - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Internal staging committer constants.
- StagingCommitterFactory - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Factory for the staging committer.
- StagingCommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory
-
- STANDARD_AWS_PROVIDERS - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
The standard AWS provider list for AWS connections.
- STANDARD_OPENFILE_KEYS - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
The known keys used in a standard openFile call.
- statement - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
- statement(boolean, String, String...) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a statement.
- statement(boolean, String, Collection<String>) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a statement.
- statement(boolean, Path, boolean, boolean, String...) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a statement.
- statement(boolean, Path, boolean, boolean, Collection<String>) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a statement.
- Statement(RoleModel.Effects) - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- STATEMENT_ALL_DDB - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Statement to allow all DDB access.
- STATEMENT_ALL_S3 - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Allow all S3 Operations.
- STATEMENT_ALL_S3_GET_BUCKET_LOCATION - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
The s3:GetBucketLocation permission is for all buckets, not for
any named bucket, which complicates permissions.
- STATEMENT_ALLOW_SSE_KMS_READ - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Statement to allow read access to KMS keys, so the ability
to read SSE-KMS data,, but not decrypt it.
- STATEMENT_ALLOW_SSE_KMS_RW - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Statement to allow KMS R/W access access, so full use of
SSE-KMS.
- STATEMENT_S3GUARD_CLIENT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Statement to allow all client operations needed for S3Guard,
but none of the admin operations.
- Statistic - Enum in org.apache.hadoop.fs.s3a
-
Statistic which are collected in S3A.
- STATUS_CODE - Static variable in exception org.apache.hadoop.fs.s3a.AWSBadRequestException
-
HTTP status code which signals this failure mode was triggered: 400.
- STATUS_CODE - Static variable in exception org.apache.hadoop.fs.s3a.AWSServiceThrottledException
-
HTTP status code which signals this failure mode was triggered: 503.
- StatusProbeEnum - Enum in org.apache.hadoop.fs.s3a.impl
-
Enum of probes which can be made of S3.
- stopAbortsOnFailure() - Method in class org.apache.hadoop.fs.s3a.commit.Tasks.Builder
-
- stopAllServices() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Stop all services.
- stopOnFailure() - Method in class org.apache.hadoop.fs.s3a.commit.Tasks.Builder
-
- stopRevertsOnFailure() - Method in class org.apache.hadoop.fs.s3a.commit.Tasks.Builder
-
- STORE_CAPABILITY_MAGIC_COMMITTER - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to indicate that a store supports magic committers.
- STORE_CAPABILITY_MAGIC_COMMITTER_OLD - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Deprecated.
- STORE_CSV - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Suffix for a recursive treewalk through the metastore: "-store.csv".
- storeClosed() - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentation
-
Store has been closed.
- storeClosed() - Method in class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- StoreContext - Class in org.apache.hadoop.fs.s3a.impl
-
This class provides the core context of the S3A filesystem to subsidiary
components, without exposing the entire parent class.
- StoreContext(URI, String, Configuration, String, UserGroupInformation, ListeningExecutorService, int, Invoker, S3AInstrumentation, S3AStorageStatistics, S3AInputPolicy, ChangeDetectionPolicy, boolean, MetadataStore, boolean, ContextAccessors, ITtlTimeProvider) - Constructor for class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Instantiate.
- storeNotFound(FileNotFoundException) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Handle store not found by converting to an exit exception
with specific error code.
- STREAM_CAPABILITY_MAGIC_OUTPUT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to indicate whether a stream is a magic output stream;
returned in StreamCapabilities
Value: "fs.s3a.capability.magic.output.stream".
- STREAM_CAPABILITY_MAGIC_OUTPUT_OLD - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Deprecated.
- streamClose(boolean, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
The inner stream was closed.
- streamOpened() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
The inner stream was opened.
- stringify(AmazonServiceException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get low level details of an amazon exception for logging; multi-line.
- stringify(AmazonS3Exception) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get low level details of an amazon exception for logging; multi-line.
- stringify(S3ObjectSummary) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
String information about a summary entry for debug messages.
- STS_STANDARD - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
The standard STS server.
- STSClientFactory - Class in org.apache.hadoop.fs.s3a.auth
-
Factory for creating STS Clients.
- STSClientFactory() - Constructor for class org.apache.hadoop.fs.s3a.auth.STSClientFactory
-
- STSClientFactory.STSClient - Class in org.apache.hadoop.fs.s3a.auth
-
STS client connection with retries.
- submit(Executor, Callable<T>) - Static method in class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
Submit a callable into a completable future.
- SUCCESS_MARKER_FILE_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
The limit to the number of committed objects tracked during
job commits and saved to the _SUCCESS file.
- SuccessData - Class in org.apache.hadoop.fs.s3a.commit.files
-
Summary data saved into a _SUCCESS marker file.
- SuccessData() - Constructor for class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- suppressExceptions() - Method in class org.apache.hadoop.fs.s3a.commit.Tasks.Builder
-
- suppressExceptions(boolean) - Method in class org.apache.hadoop.fs.s3a.commit.Tasks.Builder
-
- TABLE_ROOT - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterConstants
-
Name of the root partition :"table_root".
- TableDeleteTimeoutException - Exception in org.apache.hadoop.fs.s3a.s3guard
-
An exception raised when a table being deleted is still present after
the wait time is exceeded.
- TAG_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- tagTableWithVersionMarker() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
- taskCompleted(boolean) - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Note that a task has completed.
- taskCompleted(boolean) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.CommitterStatistics
-
- Tasks - Class in org.apache.hadoop.fs.s3a.commit
-
Utility class for parallel execution, takes closures for the various
actions.
- Tasks.Builder<I> - Class in org.apache.hadoop.fs.s3a.commit
-
Builder for task execution.
- Tasks.FailureTask<I,E extends Exception> - Interface in org.apache.hadoop.fs.s3a.commit
-
Callback invoked on a failure.
- Tasks.Task<I,E extends Exception> - Interface in org.apache.hadoop.fs.s3a.commit
-
Callback invoked to process an item.
- TEMP_DATA - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Temp data which is not auto-committed: "__temp-data".
- tempDirForStaging(FileSystem, Configuration) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Try to come up with a good temp directory for different filesystems.
- TEMPORARY - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
This is the "Pending" directory of the FileOutputCommitter;
data written here is, in that algorithm, renamed into place.
- TemporaryAWSCredentialsProvider - Class in org.apache.hadoop.fs.s3a
-
Support session credentials for authenticating with AWS.
- TemporaryAWSCredentialsProvider(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
-
Construct from just a configuration.
- TemporaryAWSCredentialsProvider(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
-
Constructor: the URI will be null if the provider is inited unbonded
to a filesystem.
- THREAD_POOL_SHUTDOWN_DELAY_SECONDS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
How long to wait for the thread pool to terminate when cleaning up.
- THREAD_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- THROTTLE_LOG_NAME - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Name of the log for throttling events.
- throttled() - Method in interface org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentation
-
Throttled request.
- throttled() - Method in class org.apache.hadoop.fs.s3a.s3guard.MetastoreInstrumentationImpl
-
- throttlePolicy - Variable in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Policy for throttle requests, which are considered repeatable, even for
non-idempotent calls, as the service rejected the call entirely.
- toAWSCredentials(MarshalledCredentials, MarshalledCredentials.CredentialTypeRequired, String) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Create an AWS credential set from a set of marshalled credentials.
- toBytes() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- toBytes() - Method in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Serialize to JSON and then to a byte array, after performing a
preflight validation of the data to be saved.
- toBytes() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- toBytes() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- toJson(RoleModel.Policy) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
- TOKEN_MISMATCH - Static variable in exception org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException
-
The far end is expecting a different token kind than
that which the client created.
- TOKEN_NAME_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Prefix for token names: "S3ADelegationToken/".
- TOKEN_WRONG_CLASS - Static variable in exception org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException
-
Error: delegation token/token identifier class isn't the right one.
- tokenIssued() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.DelegationTokenStatistics
-
A token has been issued.
- TokenSecretManager() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding.TokenSecretManager
-
- toMap() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Copy all the metrics to a map of (name, long-value).
- tombstone(Path, long) - Static method in class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
Create a tombstone from the current time.
- TombstonedInMsNotDeletedInS3(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.TombstonedInMsNotDeletedInS3
-
- toPartEtags(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
Convert an ordered list of strings to a list of index etag parts.
- toS3AFileStatus() - Method in class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
Generate an S3AFileStatus instance, including etag and
version ID, if present.
- toString() - Method in class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider
-
- toString() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
- toString() - Method in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
- toString() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
- toString() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- toString() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
String function returns the encryption mode but not any other
secrets.
- toString() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
- toString() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Add the (sanitized) marshalled credentials to the string value.
- toString() - Method in enum org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.CredentialTypeRequired
-
- toString() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
String value MUST NOT include any secrets.
- toString() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
The string value is this class name and the string values of nested
providers.
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.CommitContext
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations.MaybeIOE
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.CommitOperations
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.PutTracker
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
- toString() - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- toString() - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- toString() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
String value for logging.
- toString() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- toString() - Method in class org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client
-
- toString() - Method in class org.apache.hadoop.fs.s3a.InconsistentS3Object
-
- toString() - Method in class org.apache.hadoop.fs.s3a.S3A
-
- toString() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
- toString() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- toString() - Method in enum org.apache.hadoop.fs.s3a.S3AInputPolicy
-
- toString() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
String value includes statistics as well as stream state.
- toString() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics
-
String operator describes all the current statistics.
- toString() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation.OutputStreamStatistics
-
- toString() - Method in class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- toString() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
- toString() - Method in exception org.apache.hadoop.fs.s3a.s3guard.AuthoritativeAuditOperation.NonAuthoritativeDirException
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.DDBPathMetadata
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.DirListingMetadata
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataEntry
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.PathMetadata
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.RenameTracker
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard.TtlTimeProvider
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.ComparePair
-
- toString() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.DDBTree
-
- toString() - Method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
- toString(SelectObjectContentRequest) - Static method in class org.apache.hadoop.fs.s3a.select.SelectBinding
-
Stringify the given SelectObjectContentRequest, as its
toString() operator doesn't.
- toString() - Method in class org.apache.hadoop.fs.s3a.select.SelectInputStream
-
String value includes statistics as well as stream state.
- toString() - Method in class org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
-
- toString() - Method in enum org.apache.hadoop.fs.s3a.Statistic
-
The string value is simply the symbol.
- toString() - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
The toString method is intended to be used in logging/toString calls.
- toString(URI) - Static method in class org.apache.hadoop.fs.s3native.S3xLoginHelper
-
Create a stripped down string value for error messages.
- touch(long) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Set the various timestamp fields to the supplied value.
- toUri(String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Convert a path to a URI, catching any URISyntaxException
and converting to an invalid args exception.
- translateDeleteException(String, MultiObjectDeleteException) - Static method in class org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteSupport
-
A MultiObjectDeleteException is raised if one or more
paths listed in a bulk DELETE operation failed.
- translateDynamoDBException(String, String, AmazonDynamoDBException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Translate a DynamoDB exception into an IOException.
- translateException(String, Path, AmazonClientException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Translate an exception raised in an operation into an IOException.
- translateException(String, String, SdkBaseException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Translate an exception raised in an operation into an IOException.
- TREE_CSV - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Suffix for the treewalk scan of the S3A Filesystem: "-tree.csv".
- treewalkFilesystem(DumpS3GuardDynamoTable.CsvFile, Path) - Method in class org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardDynamoTable
-
Dump the filesystem via a treewalk.
- Tristate - Enum in org.apache.hadoop.fs.s3a
-
Simple enum to express {true, false, don't know}.
- trueWithProbability(float) - Static method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- TtlTimeProvider(long) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3Guard.TtlTimeProvider
-
- TtlTimeProvider(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3Guard.TtlTimeProvider
-
- v1(ListObjectsRequest) - Static method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
Restricted constructors to ensure v1 or v2, not both.
- v1(ObjectListing) - Static method in class org.apache.hadoop.fs.s3a.S3ListResult
-
Restricted constructors to ensure v1 or v2, not both.
- v2(ListObjectsV2Request) - Static method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
Restricted constructors to ensure v1 or v2, not both.
- v2(ListObjectsV2Result) - Static method in class org.apache.hadoop.fs.s3a.S3ListResult
-
Restricted constructors to ensure v1 or v2, not both.
- validate() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Validate the token by looking at its fields.
- validate(String, MarshalledCredentials.CredentialTypeRequired) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Verify that a set of credentials is valid.
- validate() - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Validation includes validating all statements.
- validate() - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.RoleElt
-
validation operation.
- validate() - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- validate() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Validate the data: those fields which must be non empty, must be set.
- validate() - Method in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Validate the data: those fields which must be non empty, must be set.
- validate() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- validate() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- validateCollectionClass(Iterable, Class) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Verify that all instances in a collection are of the given class.
- validateReadahead(Long) - Static method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
from a possibly null Long value, return a valid
readahead.
- ValidationFailure - Exception in org.apache.hadoop.fs.s3a.commit
-
Exception raised on validation failures; kept as an IOException
for consistency with other failures.
- ValidationFailure(String, Object...) - Constructor for exception org.apache.hadoop.fs.s3a.commit.ValidationFailure
-
Create an instance with string formatting applied to the message
and arguments.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.auth.delegation.AWSPolicyProvider.AccessLevel
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.TokenIssuingPolicy
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.CredentialTypeRequired
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.auth.RoleModel.Effects
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.commit.staging.ConflictResolution
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy.Mode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy.Source
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.S3AInputPolicy
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.s3guard.BulkOperationState.OperationType
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.s3guard.MetadataStore.PruneMode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.s3guard.S3Guard.DisabledWarnLevel
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.Violation
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.HandleMode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.Statistic
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.Tristate
-
Returns the enum constant of this type with the specified name.
- values() - Static method in enum org.apache.hadoop.fs.s3a.auth.delegation.AWSPolicyProvider.AccessLevel
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.TokenIssuingPolicy
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.CredentialTypeRequired
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.auth.RoleModel.Effects
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.commit.staging.ConflictResolution
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy.Mode
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy.Source
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.S3AInputPolicy
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.s3guard.BulkOperationState.OperationType
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.s3guard.MetadataStore.PruneMode
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.s3guard.S3Guard.DisabledWarnLevel
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.Violation
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.HandleMode
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.Statistic
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.Tristate
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- VERBOSE - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- verify(boolean, String, Object...) - Static method in exception org.apache.hadoop.fs.s3a.commit.ValidationFailure
-
Verify that a condition holds.
- verifyBucketExists() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Verify that the bucket exists.
- verifyBucketExistsV2() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Verify that the bucket exists.
- verifyIsMagicCommitFS(S3AFileSystem) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Verify that an S3A FS instance is a magic commit FS.
- verifyIsMagicCommitPath(S3AFileSystem, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Verify that the path is a magic one.
- verifyIsS3AFS(FileSystem, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Verify that an FS is an S3A FS.
- verifyVersionCompatibility() - Method in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager
-
Verify that a table version is compatible with this S3Guard client.
- version - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
- VERSION - Static variable in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
- VERSION - Static variable in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Supported version value: 1.
- VERSION - Static variable in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Supported version value: 1.
- VERSION - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
Current version number.
- VERSION_MARKER_ITEM_NAME - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
parent/child name to use in the version marker.
- VERSION_MARKER_TAG_NAME - Static variable in class org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
-
parent/child name to use in the version marker.
- VersionIdMismatch(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.VersionIdMismatch
-
- ViolationHandler(S3GuardFsck.ComparePair) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardFsckViolationHandler.ViolationHandler
-