- abortAllSinglePendingCommits(Path, CommitContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Enumerate all pending files in a dir/tree, abort.
- aborted() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- abortJob(JobContext, JobStatus.State) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- abortJobInternal(CommitContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
The internal job abort operation; can be overridden in tests.
- abortJobInternal(CommitContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- abortMultipartCommit(String, String) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
- abortMultipartCommit(String, String) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Create an AbortMultipartUpload request and POST it to S3,
incrementing statistics afterwards.
- abortMultipartCommit(String, String) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Abort a multipart commit operation.
- abortMultipartCommit(String, String) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Abort a multipart commit operation.
- abortMultipartUpload(String, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Abort a multipart upload.
- abortMultipartUpload(MultipartUpload) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Abort a multipart upload.
- abortMultipartUpload(String, String, boolean, Invoker.Retried) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Abort a multipart upload operation.
- abortMultipartUpload(MultipartUpload) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Abort a multipart commit operation.
- abortMultipartUpload(String, String, boolean, Invoker.Retried) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Abort a multipart upload operation.
- abortMultipartUpload(MultipartUpload) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Abort a multipart commit operation.
- abortMultipartUploads(Path) - Method in interface org.apache.hadoop.fs.s3a.S3AInternals
-
Abort multipart uploads under a path.
- abortMultipartUploadsUnderPath(String) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Abort multipart uploads under a path: limited to the first
few hundred.
- abortMultipartUploadsUnderPath(String) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Abort multipart uploads under a path: limited to the first
few hundred.
- abortMultipartUploadsUnderPrefix(String) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Abort multipart uploads under a path; paged.
- abortOutstandingMultipartUploads(long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Abort all outstanding MPUs older than a given age.
- abortPendingUploads(CommitContext, List<SinglePendingCommit>, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Abort all pending uploads in the list.
- abortPendingUploads(CommitContext, AbstractS3ACommitter.ActiveCommit, boolean, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Abort all pending uploads in the list.
- abortPendingUploadsInCleanup(boolean, CommitContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Abort all pending uploads to the destination directory during
job cleanup operations.
- abortPendingUploadsUnderPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Abort all pending uploads to the destination FS under a path.
- abortSingleCommit(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
- abortSingleCommit(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Abort the multipart commit supplied.
- abortTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Abort a task.
- abortTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Abort the task.
- abortUploadsUnderPathInvoked() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext.EmptyMultipartUploaderStatistics
-
- abortUploadsUnderPathInvoked() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.S3AMultipartUploaderStatisticsImpl
-
- abortUploadsUnderPathInvoked() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AMultipartUploaderStatistics
-
- aboutToComplete(String, List<CompletedPart>, long, IOStatistics) - Method in class org.apache.hadoop.fs.s3a.commit.magic.InMemoryMagicCommitTracker
-
- aboutToComplete(String, List<CompletedPart>, long, IOStatistics) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
Complete operation: generate the final commit data, put it.
- aboutToComplete(String, List<CompletedPart>, long, IOStatistics) - Method in class org.apache.hadoop.fs.s3a.commit.magic.S3MagicCommitTracker
-
- aboutToComplete(String, List<CompletedPart>, long, IOStatistics) - Method in class org.apache.hadoop.fs.s3a.commit.PutTracker
-
Callback when the upload is is about to complete.
- ABSTRACT_PROVIDER - Static variable in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
- AbstractAuditSpanImpl - Class in org.apache.hadoop.fs.s3a.audit.impl
-
Base class for the audit spans implementations..
- AbstractAuditSpanImpl(String, String) - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.AbstractAuditSpanImpl
-
Constructor.
- AbstractAuditSpanImpl(String, long, String) - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.AbstractAuditSpanImpl
-
Constructor.
- AbstractAWSCredentialProvider - Class in org.apache.hadoop.fs.s3a.auth
-
Base class for AWS credential providers which
take a URI and config in their constructor.
- AbstractAWSCredentialProvider(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.auth.AbstractAWSCredentialProvider
-
Construct from URI + configuration.
- AbstractDelegationTokenBinding - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
An AbstractDelegationTokenBinding implementation is a class which
handles the binding of its underlying authentication mechanism to the
Hadoop Delegation token mechanism.
- AbstractDelegationTokenBinding(String, Text) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Constructor.
- AbstractDelegationTokenBinding.TokenSecretManager - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
The secret manager always uses the same secret; the
factory for new identifiers is that of the token manager.
- AbstractDTService - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
This is the base class for both the delegation binding
code and the back end service created; allows for
shared methods across both.
- AbstractDTService(String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Protected constructor.
- AbstractOperationAuditor - Class in org.apache.hadoop.fs.s3a.audit.impl
-
This is a long-lived service which is created in S3A FS initialize
(make it fast!) which provides context for tracking operations made to S3.
- AbstractOperationAuditor(String) - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.AbstractOperationAuditor
-
Construct.
- AbstractS3ACommitter - Class in org.apache.hadoop.fs.s3a.commit
-
Abstract base class for S3A committers; allows for any commonality
between different architectures.
- AbstractS3ACommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Create a committer.
- AbstractS3ACommitter.ActiveCommit - Class in org.apache.hadoop.fs.s3a.commit
-
State of the active commit operation.
- AbstractS3ACommitter.JobUUIDSource - Enum in org.apache.hadoop.fs.s3a.commit
-
Enumeration of Job UUID source.
- AbstractS3ACommitterFactory - Class in org.apache.hadoop.fs.s3a.commit
-
Dynamically create the output committer based on subclass type and settings.
- AbstractS3ACommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
- AbstractS3AStatisticsSource - Class in org.apache.hadoop.fs.s3a.statistics.impl
-
Base class for implementing IOStatistics sources in the S3 module.
- AbstractS3AStatisticsSource() - Constructor for class org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource
-
- AbstractS3ATokenIdentifier - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
An S3A Delegation Token Identifier: contains the information needed
to talk to S3A.
- AbstractS3ATokenIdentifier(Text, URI, Text, Text, String, EncryptionSecrets) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Constructor.
- AbstractS3ATokenIdentifier(Text, Text, Text, Text, URI) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Constructor.
- AbstractS3ATokenIdentifier(Text, Token<AbstractS3ATokenIdentifier>) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Build from a token.
- AbstractS3ATokenIdentifier(Text) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
For subclasses to use in their own empty-constructors.
- AbstractSessionCredentialsProvider - Class in org.apache.hadoop.fs.s3a.auth
-
Base class for session credential support.
- AbstractSessionCredentialsProvider(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Constructor.
- AbstractSessionCredentialsProvider.NoCredentials - Class in org.apache.hadoop.fs.s3a.auth
-
A special set of null credentials which are not the anonymous class.
- AbstractStoreOperation - Class in org.apache.hadoop.fs.s3a.impl
-
Base class of operations in the store.
- AbstractStoreOperation(StoreContext) - Constructor for class org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation
-
Constructor.
- AbstractStoreOperation(StoreContext, AuditSpan) - Constructor for class org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation
-
Constructor.
- accept(Path, S3Object) - Method in class org.apache.hadoop.fs.s3a.Listing.AcceptAllButSelfAndS3nDirs
-
Reject a s3Object entry if the key path is the qualified Path, or
it ends with "_$folder$".
- accept(Path, String) - Method in class org.apache.hadoop.fs.s3a.Listing.AcceptAllButSelfAndS3nDirs
-
Accept all prefixes except the one for the base path, "self".
- accept(FileStatus) - Method in class org.apache.hadoop.fs.s3a.Listing.AcceptAllButSelfAndS3nDirs
-
- ACCEPT_ALL - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
A Path filter which accepts all filenames.
- AcceptAllButSelfAndS3nDirs(Path) - Constructor for class org.apache.hadoop.fs.s3a.Listing.AcceptAllButSelfAndS3nDirs
-
Constructor.
- access(Path, FsAction) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Soft check of access by forwarding to the audit manager
and so on to the auditor.
- ACCESS_DENIED - Static variable in exception org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteException
-
This is the exception exit code if access was denied on a delete.
- ACCESS_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- ACCESS_POLICY - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
The access policies we want for operations.
- accessKeyId() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider.NoCredentials
-
- accessPointFromArn(String) - Static method in class org.apache.hadoop.fs.s3a.ArnResource
-
Parses the passed `arn` string into a full ArnResource.
- acquireReadCapacity(int) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Acquire read capacity for rate limiting Acquire read capacity for operations..
- acquireReadCapacity(int) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Acquire read capacity for operations.
- acquireWriteCapacity(int) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Acquire write capacity for rate limiting Acquire write capacity for operations..
- acquireWriteCapacity(int) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Acquire write capacity for operations.
- action - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- activate() - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractAuditSpanImpl
-
- activate(AuditSpanS3A) - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- activate() - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopSpan
-
- activate(AuditSpanS3A) - Method in interface org.apache.hadoop.fs.s3a.audit.impl.NoopSpan.SpanActivationCallbacks
-
Span was activated.
- activateAuditSpan() - Method in class org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation
-
Activate the audit span.
- ActiveAuditManagerS3A - Class in org.apache.hadoop.fs.s3a.audit.impl
-
Thread management for the active audit.
- ActiveAuditManagerS3A(IOStatisticsStore) - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
Instantiate.
- ActiveCommit(FileSystem, List<? extends FileStatus>) - Constructor for class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
Construct from a source FS and list of files.
- ActiveOperationContext - Class in org.apache.hadoop.fs.s3a.impl
-
Context for any active operation.
- ActiveOperationContext(long, S3AStatisticsContext) - Constructor for class org.apache.hadoop.fs.s3a.impl.ActiveOperationContext
-
- add(RoleModel.Policy) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Add the statements of another policy to this one.
- add(Collection<RoleModel.Statement>) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Add a collection of statements.
- add(RoleModel.Statement) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Add a single statement.
- add(AwsCredentialsProvider) - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Add a new SDK V2 provider.
- add(FileStatus) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- add(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Add a commit.
- addActions(String...) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- addActions(Collection<String>) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- addAgeOptions() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- addAll(AWSCredentialProviderList) - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Add all providers from another list to this one.
- addAttribute(String, String) - Method in class org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor
-
Add an attribute.
- addDelegationTokens(Configuration, Credentials, String, String) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADtFetcher
-
Returns Token object via FileSystem, null if bad argument.
- addDiagnostic(String, String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Add a diagnostics entry.
- addFileSystemStatistics(Map<String, Long>) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Add the filesystem statistics to the map; overwriting anything
with the same name.
- addResources(String...) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- addResources(Collection<String>) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
Add a list of resources.
- addUUID(String, String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Insert the UUID to a path if it is not there already.
- addValueToQuantiles(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Add a value to a quantiles statistic.
- addValueToQuantiles(Statistic, long) - Method in interface org.apache.hadoop.fs.s3a.statistics.CountersAndGauges
-
Add a value to a quantiles statistic.
- addValueToQuantiles(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
Add a value to a quantiles statistic.
- addValueToQuantiles(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- afterExecution(Context.AfterExecution, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
Forward to active span.
- afterMarshalling(Context.AfterMarshalling, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- afterTransmission(Context.AfterTransmission, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- afterUnmarshalling(Context.AfterUnmarshalling, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- AGE_OPTIONS_USAGE - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- ALL - Static variable in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
Look for files and directories.
- ALLOW_REQUESTER_PAYS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Configuration option for S3 Requester Pays feature: "fs.s3a.requester.pays.enabled".
- allowAuthoritative(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Is a path to be considered as authoritative?
is a store with the supplied path under
one of the paths declared as authoritative.
- allowAuthoritative(Path, S3AFileSystem, Collection<String>) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Is the path for the given FS instance authoritative?
- allowS3Operations(String, boolean) - Static method in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
From an S3 bucket name, build an ARN to refer to it.
- analyze(SdkRequest) - Method in class org.apache.hadoop.fs.s3a.audit.AWSRequestAnalyzer
-
Given an AWS request, try to analyze it to operation,
read/write and path.
- ANONYMOUS_CREDENTIALS_V1 - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
V1 credential provider: "com.amazonaws.auth.AnonymousAWSCredentials".
- AnonymousAWSCredentialsProvider - Class in org.apache.hadoop.fs.s3a
-
AnonymousAWSCredentialsProvider supports anonymous access to AWS services
through the AWS SDK.
- AnonymousAWSCredentialsProvider() - Constructor for class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider
-
- AP_INACCESSIBLE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Error message to indicate Access Points are not accessible or don't exist.
- AP_REQUIRED_EXCEPTION - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Error message to indicate Access Points are required to be used for S3 access.
- append(Path, int, Progressable) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Append to an existing file (optional operation).
- append(Path, int, Progressable) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- apply() - Method in class org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation
-
- apply() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
drain the stream.
- applyDefaultOptions(S3AReadOpContext) - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport
-
Propagate the default options to the operation context
being built up.
- applyLocatedFiles(RemoteIterator<? extends LocatedFileStatus>, S3AUtils.CallOnLocatedFileStatus) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Apply an operation to every LocatedFileStatus in a remote
iterator.
- applyOptions(S3AReadOpContext) - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
Propagate the options to the operation context
being built up.
- applyRevisionConstraint(GetObjectRequest.Builder, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Applies the given
revisionId
as a server-side qualification on the
GetObjectRequest.Builder.
- applyRevisionConstraint(CopyObjectRequest.Builder, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Applies the given
revisionId
as a server-side qualification on the
CopyObjectRequest.
- applyRevisionConstraint(HeadObjectRequest.Builder, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Applies the given
revisionId
as a server-side qualification on the
GetObjectMetadataRequest.
- ARCHIVE_STATUS - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Header describing what archive tier the object is in, if any.
- ARN_BUCKET_OPTION - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
AccessPoint ARN for the bucket.
- ArnResource - Class in org.apache.hadoop.fs.s3a
-
Represents an Arn Resource, this can be an accesspoint or bucket.
- ASSUMED_ROLE_ARN - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
ARN of AWS Role to request: "fs.s3a.assumed.role.arn".
- ASSUMED_ROLE_CREDENTIALS_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- ASSUMED_ROLE_CREDENTIALS_PROVIDER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
List of providers to authenticate for the assumed role: "fs.s3a.assumed.role.credentials.provider".
- ASSUMED_ROLE_EXTERNAL_ID - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
external id for assume role request: "fs.s3a.assumed.role.external.id".
- ASSUMED_ROLE_POLICY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
JSON policy containing the policy to apply to the role: "fs.s3a.assumed.role.policy".
- ASSUMED_ROLE_SESSION_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Duration of assumed roles before a refresh is attempted.
- ASSUMED_ROLE_SESSION_DURATION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default duration of an assumed role: "1h".
- ASSUMED_ROLE_SESSION_NAME - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Session name for the assumed role, must be valid characters according
to the AWS APIs: "fs.s3a.assumed.role.session.name".
- ASSUMED_ROLE_STS_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Security Token Service Endpoint: "fs.s3a.assumed.role.sts.endpoint".
- ASSUMED_ROLE_STS_ENDPOINT_REGION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Region for the STS endpoint; needed if the endpoint
is set to anything other then the central one.: "fs.s3a.assumed.role.sts.endpoint.region".
- ASSUMED_ROLE_STS_ENDPOINT_REGION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value for the STS endpoint region; needed for
v4 signing: "".
- AssumedRoleCredentialProvider - Class in org.apache.hadoop.fs.s3a.auth
-
Support IAM Assumed roles by instantiating an instance of
STSAssumeRoleSessionCredentialsProvider from configuration
properties, including wiring up the inner authenticator, and,
unless overridden, creating a session name from the current user.
- AssumedRoleCredentialProvider(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
Instantiate.
- ASYNC_DRAIN_THRESHOLD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The threshold at which drain operations switch
to being asynchronous with the schedule/wait overhead
compared to synchronous.
- attachSpanToRequest(ExecutionAttributes, AuditSpanS3A) - Static method in class org.apache.hadoop.fs.s3a.audit.AuditIntegration
-
Attach a span to the execution attributes.
- AUDIT - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Audit sub-command: "-audit".
- AUDIT_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Is auditing enabled?
Value: "fs.s3a.audit.enabled".
- AUDIT_ENABLED_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Default auditing flag.
- AUDIT_EXECUTION_INTERCEPTORS - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
List of extra AWS SDK execution interceptors: "fs.s3a.audit.execution.interceptors".
- AUDIT_MANAGER_OPERATION - Static variable in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- AUDIT_REQUEST_HANDLERS - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Deprecated list of extra AWS SDK request handlers: "fs.s3a.audit.request.handlers".
- AUDIT_SERVICE_CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Name of class used for audit logs: "fs.s3a.audit.service.classname".
- AUDIT_SPAN_EXECUTION_ATTRIBUTE - Static variable in class org.apache.hadoop.fs.s3a.audit.impl.S3AInternalAuditConstants
-
Exceution attribute for audit span callbacks.
- AuditContextUpdater - Class in org.apache.hadoop.fs.s3a.commit.impl
-
Class to track/update context information to set
in threads.
- AuditContextUpdater(JobContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.impl.AuditContextUpdater
-
Construct.
- AuditContextUpdater(String) - Constructor for class org.apache.hadoop.fs.s3a.commit.impl.AuditContextUpdater
-
- AuditFailureException - Exception in org.apache.hadoop.fs.s3a.audit
-
This is in the AWS exception tree so that exceptions raised in the
AWS SDK are correctly reported up.
- AuditFailureException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.audit.AuditFailureException
-
- AuditFailureException(String) - Constructor for exception org.apache.hadoop.fs.s3a.audit.AuditFailureException
-
- AuditIntegration - Class in org.apache.hadoop.fs.s3a.audit
-
Support for integrating auditing within the S3A code.
- AuditManagerS3A - Interface in org.apache.hadoop.fs.s3a.audit
-
Interface for Audit Managers auditing operations through the
AWS libraries.
- AuditOperationRejectedException - Exception in org.apache.hadoop.fs.s3a.audit
-
The auditor has rejected the operation as forbidden/unavailable.
- AuditOperationRejectedException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.audit.AuditOperationRejectedException
-
- AuditOperationRejectedException(String) - Constructor for exception org.apache.hadoop.fs.s3a.audit.AuditOperationRejectedException
-
- AuditSpanS3A - Interface in org.apache.hadoop.fs.s3a.audit
-
An Audit Span with S3A integration, specifically
callbacks from the AWS client SDK.
- AUTH_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "auth".
- AUTH_SCHEME_AWS_SIGV_4 - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
AWS V4 Auth Scheme to use when creating signers: "aws.auth#sigv4".
- AUTHORITATIVE_PATH - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Paths considered "authoritative".
- available() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Returns the number of bytes available for reading without blocking.
- available() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Returns the number of bytes that can read from this stream without blocking.
- available() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- availablePolicies() - Static method in class org.apache.hadoop.fs.s3a.impl.DirectoryPolicyImpl
-
Enumerate all available policies.
- AWS_AUTH_CLASS_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Prefix of auth classes in AWS SDK V1.
- AWS_CREDENTIALS_PROVIDER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- AWS_CREDENTIALS_PROVIDER_MAPPING - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
AWS credentials providers mapping with key/value pairs.
- AWS_LOG_REGEXP_GROUPS - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Ordered list of regular expression group names.
- AWS_REGION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
AWS S3 region for the bucket.
- AWS_REGION_SYSPROP - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
The system property used by the AWS SDK to identify the region.
- AWS_S3_ACCESSPOINT_REQUIRED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Require that all S3 access is made through Access Points.
- AWS_S3_CENTRAL_REGION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The special S3 region which can be used to talk to any bucket.
- AWS_S3_CLASSLOADER_ISOLATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Are extensions classes, such as
fs.s3a.aws.credentials.provider,
going to be loaded from the same classloader that loaded
the
S3AFileSystem?
It is useful to turn classloader isolation off for Apache Spark applications
that might load
S3AFileSystem from the Spark distribution (Launcher classloader)
while users might want to provide custom extensions (loaded by Spark MutableClassloader).
- AWS_S3_DEFAULT_REGION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The default S3 region when using cross region client.
- AWS_S3_VECTOR_ACTIVE_RANGE_READS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Maximum number of range reads a single input stream can have
active (downloading, or queued) to the central FileSystem
instance's pool of queued operations.
- AWS_S3_VECTOR_READS_MAX_MERGED_READ_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
What is the largest merged read size in bytes such
that we group ranges together during vectored read.
- AWS_S3_VECTOR_READS_MIN_SEEK_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
What is the smallest reasonable seek in bytes such
that we group ranges together during vectored read operation.
- AWS_SERVICE_IDENTIFIER_DDB - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- AWS_SERVICE_IDENTIFIER_S3 - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- AWS_SERVICE_IDENTIFIER_STS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- AWSApiCallTimeoutException - Exception in org.apache.hadoop.fs.s3a
-
IOException equivalent of an ApiCallTimeoutException.
- AWSApiCallTimeoutException(String, Exception) - Constructor for exception org.apache.hadoop.fs.s3a.AWSApiCallTimeoutException
-
Constructor.
- AWSAuditEventCallbacks - Interface in org.apache.hadoop.fs.s3a.audit
-
Callbacks for audit spans.
- AWSBadRequestException - Exception in org.apache.hadoop.fs.s3a
-
A 400 "Bad Request" exception was received.
- AWSBadRequestException(String, AwsServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSBadRequestException
-
Instantiate.
- AWSCannedACL - Enum in org.apache.hadoop.fs.s3a.impl
-
Enum to map AWS SDK V1 Acl values to SDK V2.
- AWSClientConfig - Class in org.apache.hadoop.fs.s3a.impl
-
Methods for configuring the S3 client.
- AWSClientIOException - Exception in org.apache.hadoop.fs.s3a
-
IOException equivalent of an SdkException.
- AWSClientIOException(String, SdkException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSClientIOException
-
- AWSCredentialProviderList - Class in org.apache.hadoop.fs.s3a
-
A list of providers.
- AWSCredentialProviderList() - Constructor for class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Empty instance.
- AWSCredentialProviderList(Collection<AwsCredentialsProvider>) - Constructor for class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Create with an initial list of providers.
- AWSCredentialProviderList(String, AwsCredentialsProvider...) - Constructor for class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Create with an initial list of SDK V2 credential providers.
- AWSERRORCODE_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "awserrorcode".
- awsErrorDetails() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- AWSHeaders - Interface in org.apache.hadoop.fs.s3a.impl
-
Common S3 HTTP header values used throughout the Amazon Web Services S3 Java client.
- AWSNoResponseException - Exception in org.apache.hadoop.fs.s3a
-
Status code 443, no response from server.
- AWSNoResponseException(String, AwsServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSNoResponseException
-
- AWSPolicyProvider - Interface in org.apache.hadoop.fs.s3a.auth.delegation
-
Interface for providers of AWS policy for accessing data.
- AWSPolicyProvider.AccessLevel - Enum in org.apache.hadoop.fs.s3a.auth.delegation
-
Access levels.
- AWSRedirectException - Exception in org.apache.hadoop.fs.s3a
-
Request is redirected.
- AWSRedirectException(String, AwsServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSRedirectException
-
Instantiate.
- AWSRequestAnalyzer - Class in org.apache.hadoop.fs.s3a.audit
-
Extract information from a request.
- AWSRequestAnalyzer() - Constructor for class org.apache.hadoop.fs.s3a.audit.AWSRequestAnalyzer
-
- AWSRequestAnalyzer.RequestInfo - Class in org.apache.hadoop.fs.s3a.audit
-
Info about a request.
- AWSS3IOException - Exception in org.apache.hadoop.fs.s3a
-
Wrap a S3Exception as an IOE, relaying all
getters.
- AWSS3IOException(String, S3Exception) - Constructor for exception org.apache.hadoop.fs.s3a.AWSS3IOException
-
Instantiate.
- AWSServiceIOException - Exception in org.apache.hadoop.fs.s3a
-
A specific exception from AWS operations.
- AWSServiceIOException(String, AwsServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
Instantiate.
- AWSServiceThrottledException - Exception in org.apache.hadoop.fs.s3a
-
Exception raised when a service was throttled.
- AWSServiceThrottledException(String, AwsServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSServiceThrottledException
-
Instantiate.
- AwsSignerInitializer - Interface in org.apache.hadoop.fs.s3a.auth
-
Interface which can be implemented to allow initialization of any custom
signers which may be used by the
S3AFileSystem.
- AwsStatisticsCollector - Class in org.apache.hadoop.fs.s3a.statistics.impl
-
Collect statistics from the AWS SDK and forward to an instance of
StatisticsFromAwsSdk and thence into the S3A statistics.
- AwsStatisticsCollector(StatisticsFromAwsSdk) - Constructor for class org.apache.hadoop.fs.s3a.statistics.impl.AwsStatisticsCollector
-
Instantiate.
- AWSStatus500Exception - Exception in org.apache.hadoop.fs.s3a
-
A 5xx response came back from a service.
- AWSStatus500Exception(String, AwsServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSStatus500Exception
-
- AWSUnsupportedFeatureException - Exception in org.apache.hadoop.fs.s3a
-
A store returned an error indicating that it does not support a
specific S3 feature such as the chosen ChangeDetectionPolicy or
other AWS-S3 feature that the third-party store does not support.
- AWSUnsupportedFeatureException(String, AwsServiceException) - Constructor for exception org.apache.hadoop.fs.s3a.AWSUnsupportedFeatureException
-
Instantiate.
- AwsV1BindingSupport - Class in org.apache.hadoop.fs.s3a.adapter
-
Binding support; the sole way which the rest of the code should instantiate v1 SDK libraries.
- CACHE_CONTROL - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- call(LocatedFileStatus) - Method in interface org.apache.hadoop.fs.s3a.S3AUtils.CallOnLocatedFileStatus
-
- call(LocatedFileStatus) - Method in interface org.apache.hadoop.fs.s3a.S3AUtils.LocatedFileStatusMap
-
- CallableSupplier<T> - Class in org.apache.hadoop.fs.s3a.impl
-
A bridge from Callable to Supplier; catching exceptions
raised by the callable and wrapping them as appropriate.
- CallableSupplier(Callable<T>) - Constructor for class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
Create.
- CallableSupplier(AuditSpan, Callable<T>) - Constructor for class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
Create.
- cancelDeleteOnExit(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Cancel the scheduled deletion of the path when the FileSystem is closed.
- CANNED_ACL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- canonicalizeUri(URI) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- canonicalizeUri(URI, int) - Static method in class org.apache.hadoop.fs.s3native.S3xLoginHelper
-
Canonicalize the given URI.
- CAPABILITY_FORMAT - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- CENTRAL_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The central endpoint :"s3.amazonaws.com".
- CHANGE_DETECT_MODE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Mode to run change detection in.
- CHANGE_DETECT_MODE_CLIENT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Change is detected on the client side by comparing the returned id with the
expected id.
- CHANGE_DETECT_MODE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default change detection mode: server.
- CHANGE_DETECT_MODE_NONE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Change detection is turned off.
- CHANGE_DETECT_MODE_SERVER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Change is detected by passing the expected value in the GetObject request.
- CHANGE_DETECT_MODE_WARN - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Change is detected on the client side by comparing the returned id with the
expected id.
- CHANGE_DETECT_REQUIRE_VERSION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
If true, raises a
RemoteFileChangedException exception when S3
doesn't provide the attribute defined by fs.s3a.change.detection.source.
- CHANGE_DETECT_REQUIRE_VERSION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default change detection require version: true.
- CHANGE_DETECT_SOURCE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Where to get the value to use in change detection.
- CHANGE_DETECT_SOURCE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default change detection mechanism: eTag.
- CHANGE_DETECT_SOURCE_ETAG - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
eTag as the change detection mechanism.
- CHANGE_DETECT_SOURCE_VERSION_ID - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Object versionId as the change detection mechanism.
- CHANGE_DETECTED - Static variable in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- CHANGE_REPORTED_BY_S3 - Static variable in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- ChangeDetectionPolicy - Class in org.apache.hadoop.fs.s3a.impl
-
Object change detection policy.
- ChangeDetectionPolicy(ChangeDetectionPolicy.Mode, boolean) - Constructor for class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- ChangeDetectionPolicy.Mode - Enum in org.apache.hadoop.fs.s3a.impl
-
What to do when change is detected.
- ChangeDetectionPolicy.Source - Enum in org.apache.hadoop.fs.s3a.impl
-
The S3 object attribute used to detect change.
- ChangeTracker - Class in org.apache.hadoop.fs.s3a.impl
-
Change tracking for input streams: the version ID or etag of the object is
tracked and compared on open/re-open.
- ChangeTracker(String, ChangeDetectionPolicy, ChangeTrackerStatistics, S3ObjectAttributes) - Constructor for class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Create a change tracker.
- ChangeTrackerStatistics - Interface in org.apache.hadoop.fs.s3a.statistics
-
Interface for change tracking statistics.
- checkAccess(Path, S3AFileStatus, FsAction) - Method in interface org.apache.hadoop.fs.s3a.audit.AuditManagerS3A
-
Check for permission to access a path.
- checkAccess(Path, S3AFileStatus, FsAction) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- checkAccess(Path, S3AFileStatus, FsAction) - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
Forward to the auditor.
- checkAccess(Path, S3AFileStatus, FsAction) - Method in interface org.apache.hadoop.fs.s3a.audit.OperationAuditor
-
Check for permission to access a path.
- checkDiskBuffer(Configuration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Check whether the configuration for S3ABlockOutputStream is
consistent or not.
- checkNoS3Guard(URI, Configuration) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
Assert that the FS is not configured to use an unsupported S3Guard
option.
- checkNotEmpty() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Verify that the provider list is not empty.
- checkPath(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Check that a Path belongs to this FileSystem.
- checkPath(Configuration, URI, Path, int) - Static method in class org.apache.hadoop.fs.s3native.S3xLoginHelper
-
Check the path, ignoring authentication details.
- CHECKSUM_VALIDATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Should checksums be validated on download?
This is slower and not needed on TLS connections.
- CHECKSUM_VALIDATION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory".
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory".
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory".
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory".
- CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory
-
Name of this class: "org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory".
- CLEAN - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Clean Sub-command: "-clean".
- cleanup(CommitContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Cleanup the job context, including aborting anything pending
and destroying the thread pool.
- cleanup(CommitContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Staging committer cleanup includes calling wrapped committer's
cleanup method, and removing staging uploads path and all
destination paths in the final filesystem.
- cleanupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- cleanupStagingDirs() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Clean up any staging directories.
- cleanupStagingDirs() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Delete the magic directory.
- cleanupStagingDirs() - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- clearBucketOption(Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Clear a bucket-specific property.
- clearTempFolderInfo(TaskAttemptID) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Remove all information held about task attempts.
- ClientManager - Interface in org.apache.hadoop.fs.s3a.impl
-
Interface for on-demand/async creation of AWS clients
and extension services.
- clientManager() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Get the client manager.
- clientManager() - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
- ClientManagerImpl - Class in org.apache.hadoop.fs.s3a.impl
-
- ClientManagerImpl(S3ClientFactory, S3ClientFactory.S3ClientCreationParameters, DurationTrackerFactory) - Constructor for class org.apache.hadoop.fs.s3a.impl.ClientManagerImpl
-
Constructor.
- cloneObjectMetadata(HeadObjectResponse, Map<String, String>, CopyObjectRequest.Builder) - Static method in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
Creates a copy of the passed metadata.
- close() - Method in class org.apache.hadoop.fs.s3a.adapter.V1ToV2AwsCredentialProviderAdapter
-
Close the wrapped provider if it implements Closeable/AutoCloseable.
- close() - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractAuditSpanImpl
-
Invoke AuditSpan.deactivate().
- close() - Method in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
Propagate the close() call to the inner stsProvider.
- close() - Method in class org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
-
- close() - Method in class org.apache.hadoop.fs.s3a.auth.SignerManager
-
- close() - Method in class org.apache.hadoop.fs.s3a.auth.STSClientFactory.STSClient
-
- close() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Close routine will close all providers in the list which implement
Closeable.
- close() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
- close() - Method in class org.apache.hadoop.fs.s3a.impl.BulkDeleteOperation
-
- close() - Method in interface org.apache.hadoop.fs.s3a.impl.ClientManager
-
Close operation is required to not raise exceptions.
- close() - Method in class org.apache.hadoop.fs.s3a.impl.ClientManagerImpl
-
Close() is synchronized to avoid race conditions between
slow client creation and this close operation.
- close() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- close() - Method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders.BaseContentProvider
-
Close the current stream.
- close() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ABlockManager
-
- close() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ACachingBlockManager
-
- close() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ACachingInputStream
-
- close() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Closes this stream and releases all acquired resources.
- close() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Closes this stream and releases all acquired resources.
- close() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObjectReader
-
- close() - Method in class org.apache.hadoop.fs.s3a.S3ADataBlocks.BlockFactory
-
Implement any close/cleanup operation.
- close() - Method in class org.apache.hadoop.fs.s3a.S3ADataBlocks.BlockUploadData
-
Close: closes any upload stream provided in the constructor.
- close() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Close the filesystem.
- close() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Close the stream.
- close() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
- close() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Close the FS.
- close() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AwsStatisticsCollector
-
- close() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext.EmptyMultipartUploaderStatistics
-
- close() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.S3AMultipartUploaderStatisticsImpl
-
- close() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- close() - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Deactivate the audit span.
- closeAll(Logger, Closeable...) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Deprecated.
- closeAutocloseables(Logger, AutoCloseable...) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Close the Closeable objects and ignore any Exception or
null pointers.
- commit(SinglePendingCommit, String) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Commit a single pending commit; exceptions are caught
and converted to an outcome.
- commit(SinglePendingCommit, String) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Commit a single pending commit; exceptions are caught
and converted to an outcome.
- commitAborted() - Method in interface org.apache.hadoop.fs.s3a.statistics.CommitterStatistics
-
A commit has been aborted.
- commitCompleted(long) - Method in interface org.apache.hadoop.fs.s3a.statistics.CommitterStatistics
-
A commit has been completed.
- CommitConstants - Class in org.apache.hadoop.fs.s3a.commit
-
Constants for working with committers.
- CommitContext - Class in org.apache.hadoop.fs.s3a.commit.impl
-
Commit context.
- CommitContext(CommitOperations, JobContext, int, IOStatisticsContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Create.
- CommitContext(CommitOperations, Configuration, String, int, IOStatisticsContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Create for testing.
- commitCreated() - Method in interface org.apache.hadoop.fs.s3a.statistics.CommitterStatistics
-
A commit has been created.
- commitFailed() - Method in interface org.apache.hadoop.fs.s3a.statistics.CommitterStatistics
-
A commit failed.
- commitJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Commit work.
- commitJobInternal(CommitContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Internal Job commit operation: where the S3 requests are made
(potentially in parallel).
- CommitOperations - Class in org.apache.hadoop.fs.s3a.commit.impl
-
The implementation of the various actions a committer needs.
- CommitOperations(S3AFileSystem) - Constructor for class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Instantiate.
- CommitOperations(S3AFileSystem, CommitterStatistics, String) - Constructor for class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Instantiate.
- CommitOperations.MaybeIOE - Class in org.apache.hadoop.fs.s3a.commit.impl
-
- commitOrFail(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Commit the operation, throwing an exception on any failure.
- commitOrFail(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Commit the operation, throwing an exception on any failure.
- commitPendingUploads(CommitContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Commit all the pending uploads.
- commitReverted() - Method in interface org.apache.hadoop.fs.s3a.statistics.CommitterStatistics
-
A commit was reverted.
- commitTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- commitTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- commitTaskInternal(TaskAttemptContext, List<? extends FileStatus>, CommitContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
- commitTaskInternal(TaskAttemptContext, List<? extends FileStatus>, CommitContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Commit the task by uploading all created files and then
writing a pending entry for them.
- COMMITTER_BYTES_COMMITTED_COUNT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Amount of data committed: "committer_bytes_committed".
- COMMITTER_COMMIT_JOB - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Duration Tracking of time to commit an entire job: "committer_commit_job".
- COMMITTER_COMMITS_ABORTED_COUNT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Count of commits aborted: "committer_commits_aborted".
- COMMITTER_COMMITS_FAILED - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Count of commits failed: "committer_commits.failures".
- COMMITTER_COMMITS_REVERTED_COUNT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Count of commits reverted: "committer_commits_reverted".
- COMMITTER_FILE_COUNT_MEAN - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
The number of files in a task.
- COMMITTER_FILE_SIZE_MEAN - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
File Size.
- COMMITTER_FILES_COMMITTED_COUNT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Number of files committed: "committer_files_committed".
- COMMITTER_NAME_DIRECTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- COMMITTER_NAME_FILE - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- COMMITTER_NAME_MAGIC - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- COMMITTER_NAME_PARTITIONED - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- COMMITTER_NAME_STAGING - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
This is the staging committer base class; only used for testing.
- COMMITTER_TASK_DIRECTORY_COUNT_MEAN - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
What is a task attempt's directory count.
- COMMITTER_TASK_DIRECTORY_DEPTH_MEAN - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
What is the depth of a task attempt's directory tree.
- COMMITTER_TASK_FILE_COUNT_MEAN - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
The number of files in a task.
- COMMITTER_TASK_FILE_SIZE_MEAN - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
The number of files in a task.
- COMMITTER_TASKS_COMPLETED_COUNT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
"Count of successful tasks:: "committer_tasks_completed".
- COMMITTER_TASKS_FAILED_COUNT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Count of failed tasks: "committer_tasks_failed".
- CommitterStatisticNames - Class in org.apache.hadoop.fs.s3a.commit
-
Statistic names for committers.
- CommitterStatistics - Interface in org.apache.hadoop.fs.s3a.statistics
-
Statistics for S3A committers.
- commitUpload(String, String, List<CompletedPart>, long) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
This completes a multipart upload to the destination key via
finalizeMultipartUpload().
- commitUpload(String, String, List<CompletedPart>, long) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
This completes a multipart upload to the destination key via
finalizeMultipartUpload().
- commitUploaded(long) - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
Data has been uploaded to be committed in a subsequent operation;
to be called at the end of the write.
- commitUploaded(long) - Method in interface org.apache.hadoop.fs.s3a.statistics.CommitterStatistics
-
Data has been uploaded to be committed in a subsequent operation.
- CommitUtils - Class in org.apache.hadoop.fs.s3a.commit
-
Static utility methods related to S3A commitment processing, both
staging and magic.
- CommitUtilsWithMR - Class in org.apache.hadoop.fs.s3a.commit.impl
-
These are commit utility methods which import classes from
hadoop-mapreduce, and so only work when that module is on the
classpath.
- completeMPUwithRetries(String, String, List<CompletedPart>, long, AtomicInteger, PutObjectOptions) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
This completes a multipart upload to the destination key via
finalizeMultipartUpload().
- completeMPUwithRetries(String, String, List<CompletedPart>, long, AtomicInteger, PutObjectOptions) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
This completes a multipart upload to the destination key via
finalizeMultipartUpload().
- completeMultipartUpload(CompleteMultipartUploadRequest) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Complete a multipart upload.
- completeMultipartUpload(CompleteMultipartUploadRequest) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Complete a multipart upload.
- completeMultipartUpload(CompleteMultipartUploadRequest) - Method in interface org.apache.hadoop.fs.s3a.WriteOperationHelper.WriteOperationHelperCallbacks
-
Initiates a complete multi-part upload request.
- COMPONENT - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
- COMPONENT - Static variable in class org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
-
- COMPRESSION_OPT_GZIP - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Gzipped.
- COMPRESSION_OPT_NONE - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
No compression.
- ConfigurationHelper - Class in org.apache.hadoop.fs.s3a.impl
-
Helper class for configuration; where methods related to extracting
configuration should go instead of S3AUtils.
- ConfigureShadedAWSSocketFactory - Class in org.apache.hadoop.fs.s3a.impl
-
This interacts with the Shaded httpclient library used in the full
AWS SDK.
- ConfigureShadedAWSSocketFactory() - Constructor for class org.apache.hadoop.fs.s3a.impl.ConfigureShadedAWSSocketFactory
-
- configureSocketFactory(ApacheHttpClient.Builder, DelegatingSSLSocketFactory.SSLChannelMode) - Method in class org.apache.hadoop.fs.s3a.impl.ConfigureShadedAWSSocketFactory
-
- CONFLICT_MODE_APPEND - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Conflict mode: "append".
- CONFLICT_MODE_FAIL - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Conflict mode: "fail".
- CONFLICT_MODE_REPLACE - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Conflict mode: "replace".
- ConflictResolution - Enum in org.apache.hadoop.fs.s3a.commit.staging
-
Enum of conflict resolution algorithms.
- CONNECTION_ACQUISITION_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Acquisition timeout for connections from the pool:
"fs.s3a.connection.acquisition.timeout".
- CONNECTION_IDLE_TIME - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Maximum idle time for connections in the pool: "fs.s3a.connection.idle.time".
- CONNECTION_KEEPALIVE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Should TCP Keepalive be enabled on the socket?
This adds some network IO, but finds failures faster.
- CONNECTION_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Configuration option to configure expiration time of
S3 http connection from the connection pool: "fs.s3a.connection.ttl".
- connectivityFailure - Variable in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Client connectivity: baseExponentialRetry without worrying about whether
or not the command is idempotent.
- Constants - Class in org.apache.hadoop.fs.s3a
-
- CONSTRUCTOR_EXCEPTION - Static variable in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
- containsAuditException(Exception) - Static method in class org.apache.hadoop.fs.s3a.audit.AuditIntegration
-
- containsBasePath(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Does the list of magic elements contain a base path marker?
- CONTENT_DISPOSITION - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- CONTENT_ENCODING - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Content encoding: gzip, deflate, compress, br, etc.
- CONTENT_ENCODING - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- CONTENT_LANGUAGE - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- CONTENT_LENGTH - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- CONTENT_MD5 - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- CONTENT_RANGE - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- CONTENT_TYPE - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- CONTENT_TYPE_APPLICATION_XML - Static variable in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
XML content type : "application/xml".
- CONTENT_TYPE_OCTET_STREAM - Static variable in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
Content type of generic binary objects.
- CONTENT_TYPE_X_DIRECTORY - Static variable in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
Directory content type : "application/x-directory".
- CONTEXT - Static variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
"s3aFileSystem" Currently all s3a metrics are placed in a single
"context".
- ContextAccessors - Interface in org.apache.hadoop.fs.s3a.impl
-
An interface to implement for providing accessors to
S3AFileSystem-level API calls.
- continueListObjects(S3ListRequest, S3ListResult, DurationTrackerFactory) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
List the next set of objects.
- continueListObjectsAsync(S3ListRequest, S3ListResult, DurationTrackerFactory, AuditSpan) - Method in interface org.apache.hadoop.fs.s3a.impl.ListingOperationCallbacks
-
List the next set of objects.
- continueListObjectsAsync(S3ListRequest, S3ListResult, DurationTrackerFactory, AuditSpan) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.ListingOperationCallbacksImpl
-
- convertToIOException(Exception) - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
Convert a passed in exception (expected to be an IOE or AWS exception)
into an IOException.
- convertTokenIdentifier(AbstractS3ATokenIdentifier, Class<T>) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Verify that a token identifier is of a specific class.
- COPY - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- copyEncryptionParameters(HeadObjectResponse, CopyObjectRequest.Builder) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
Propagate encryption parameters from source file if set else use the
current filesystem encryption settings.
- copyFile(String, String, S3ObjectAttributes, S3AReadOpContext) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Copy a single object in the bucket via a COPY operation.
- copyFromLocalFile(boolean, boolean, Path, Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
The src file is on the local disk.
- CopyFromLocalOperation - Class in org.apache.hadoop.fs.s3a.impl
-
Implementation of CopyFromLocalOperation.
- CopyFromLocalOperation(StoreContext, Path, Path, boolean, boolean, CopyFromLocalOperation.CopyFromLocalOperationCallbacks) - Constructor for class org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation
-
- CopyFromLocalOperation.CopyFromLocalOperationCallbacks - Interface in org.apache.hadoop.fs.s3a.impl
-
- copyLocalFileFromTo(File, Path, Path) - Method in interface org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.CopyFromLocalOperationCallbacks
-
Copy / Upload a file from a source path to a destination path.
- copyLocalFileFromTo(File, Path, Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.CopyFromLocalCallbacksImpl
-
- counter(String, String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a counter in the registry.
- counter(Statistic) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a counter in the registry.
- CountersAndGauges - Interface in org.apache.hadoop.fs.s3a.statistics
-
This is the foundational API for collecting S3A statistics.
- CountingChangeTracker - Class in org.apache.hadoop.fs.s3a.statistics.impl
-
A change tracker which increments an atomic long.
- CountingChangeTracker(AtomicLong) - Constructor for class org.apache.hadoop.fs.s3a.statistics.impl.CountingChangeTracker
-
- CountingChangeTracker() - Constructor for class org.apache.hadoop.fs.s3a.statistics.impl.CountingChangeTracker
-
- create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
- CREATE - Static variable in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
create command.
- create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- CREATE_FILE_KEYS - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
The known keys used in a createFile call.
- CREATE_NO_OVERWRITE_FLAGS - Static variable in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder
-
Flag set to create without overwrite.
- CREATE_OVERWRITE_FLAGS - Static variable in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder
-
Flag set to create with overwrite.
- CREATE_SUCCESSFUL_JOB_OUTPUT_DIR_MARKER - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to trigger creation of a marker file on job completion.
- createAndInitAuditor(Configuration, String, OperationAuditorOptions) - Static method in class org.apache.hadoop.fs.s3a.audit.AuditIntegration
-
Create and initialize an audit service.
- createAndStartAuditManager(Configuration, IOStatisticsStore) - Static method in class org.apache.hadoop.fs.s3a.audit.AuditIntegration
-
Create and start an audit manager.
- createAndStartNoopAuditor(Configuration, NoopSpan.SpanActivationCallbacks) - Static method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditor
-
Create, init and start an instance.
- createAsyncHttpClientBuilder(Configuration) - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Create and configure the async http client with timeouts for:
connection acquisition, max idle, timeout, TTL, socket and keepalive.
- createAsyncProxyConfiguration(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Configures the proxy for the async http client.
- createAuthScheme(String, HttpSigner<AwsCredentialsIdentity>) - Static method in class org.apache.hadoop.fs.s3a.auth.SignerFactory
-
Create an auth scheme instance from an ID and a signer.
- createAWSCredentialProviderList(URI, Configuration) - Static method in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
- createAWSV1CredentialProvider(Configuration, String, URI, String) - Static method in class org.apache.hadoop.fs.s3a.adapter.AwsV1BindingSupport
-
Create an AWS credential provider from its class by using reflection.
- createBlockManager(BlockManagerParameters, S3ARemoteObjectReader) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ACachingInputStream
-
- createBulkDelete(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
S3A implementation to create a bulk delete operation using
which actual bulk delete calls can be made.
- createBulkDeleteCallbacks(Path, int, AuditSpanS3A) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create the callbacks for the bulk delete operation.
- createClientConfigBuilder(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Create the config for a given service...the service identifier is used
to determine signature implementation.
- createClientConnection(StsClient, Invoker) - Static method in class org.apache.hadoop.fs.s3a.auth.STSClientFactory
-
Create an STS Client instance.
- createClientManager(S3ClientFactory, S3ClientFactory.S3ClientCreationParameters, DurationTrackerFactory) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create the Client Manager; protected to allow for mocking.
- createClientOverrideConfiguration(S3ClientFactory.S3ClientCreationParameters, Configuration) - Method in class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
Create an override configuration for an S3 client.
- createCommitContext(JobContext, Path, int, IOStatisticsContext) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Create a commit context for a job or task.
- createCommitContextForTesting(Path, String, int) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Create a stub commit context for tests.
- createCopyFromLocalCallbacks(AuditSpanS3A) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create the CopyFromLocalCallbacks;
protected to assist in mocking.
- createCredentials(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Implementation point: whatever the subclass must do to load credentials.
- createCredentials(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialProvider
-
Perform the binding, looking up the DT and parsing it.
- createCredentials(Configuration) - Method in class org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
-
The credentials here must include a session token, else this operation
will raise an exception.
- createDefaultOriginMessage() - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Create the default origin text message with local hostname and
timestamp.
- createDelegationOperations() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create an instance of the delegation operations.
- createDelegationToken(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Create a delegation token for the user.
- createDelegationToken(EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Create a delegation token for the user.
- createEmptyDir(Path, StoreContext) - Method in interface org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.CopyFromLocalOperationCallbacks
-
Create empty directory at path.
- createEmptyDir(Path, StoreContext) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.CopyFromLocalCallbacksImpl
-
- createEmptyIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
- createEmptyIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- createEmptyIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
- createEmptyIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- createExceptionMap() - Method in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Subclasses can override this like a constructor to change behavior: call
superclass method, then modify it as needed, and return it.
- createExecutionInterceptors() - Method in interface org.apache.hadoop.fs.s3a.audit.AuditManagerS3A
-
Create the execution interceptor(s) for this audit service.
- createExecutionInterceptors() - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
Return a list of execution interceptors for the AWS SDK which
relays to this class.
- createExecutionInterceptors() - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- createFakeDirectory(Path, boolean) - Method in interface org.apache.hadoop.fs.s3a.impl.MkdirOperation.MkdirCallbacks
-
Create a fake directory, always ending in "/".
- createFakeDirectory(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.MkdirOperationCallbacksImpl
-
- createFile(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create instance of an FSDataOutputStreamBuilder for
creating a file at the given path.
- CreateFileBuilder - Class in org.apache.hadoop.fs.s3a.impl
-
Builder used in create file; takes a callback to the operation
to create the file.
- CreateFileBuilder(FileSystem, Path, CreateFileBuilder.CreateFileBuilderCallbacks) - Constructor for class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder
-
Constructor.
- CreateFileBuilder.CreateFileBuilderCallbacks - Interface in org.apache.hadoop.fs.s3a.impl
-
Callbacks for creating the file.
- CreateFileBuilder.CreateFileOptions - Class in org.apache.hadoop.fs.s3a.impl
-
Create file options as built from the builder set or the classic
entry point.
- createFileFromBuilder(Path, Progressable, CreateFileBuilder.CreateFileOptions) - Method in interface org.apache.hadoop.fs.s3a.impl.CreateFileBuilder.CreateFileBuilderCallbacks
-
Create a file from the builder.
- CreateFileOptions(EnumSet<CreateFlag>, boolean, boolean, Map<String, String>) - Constructor for class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder.CreateFileOptions
-
- createFileStatus(Path, S3Object, long, String, String, String, boolean) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Create a files status instance from a listing.
- createFileStatusListingIterator(Path, S3ListRequest, PathFilter, Listing.FileStatusAcceptor, AuditSpan) - Method in class org.apache.hadoop.fs.s3a.Listing
-
Create a FileStatus iterator against a path, with a given
list object request.
- createGetContentSummaryCallbacks() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override point: create the callbacks for getContentSummary.
- createHeaderProcessingCallbacks() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create the callbacks.
- createHttpClientBuilder(Configuration) - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Create and configure the http client-based connector with timeouts for:
connection acquisition, max idle, timeout, TTL, socket and keepalive.
- createHttpSigner(Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.auth.SignerFactory
-
Create an auth scheme by looking up the signer class in the configuration,
loading and instantiating it.
- createIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding.TokenSecretManager
-
- createListObjectsRequest(String, String, AuditSpan) - Method in interface org.apache.hadoop.fs.s3a.impl.ListingOperationCallbacks
-
Create a ListObjectsRequest request against this bucket,
with the maximum keys returned in a query set in the FS config.
- createListObjectsRequest(String, String, AuditSpan) - Method in class org.apache.hadoop.fs.s3a.Listing
-
- createListObjectsRequest(String, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create a
ListObjectsRequest request against this bucket,
with the maximum keys returned in a query set by
S3AFileSystem.maxKeys.
- createListObjectsRequest(String, String, AuditSpan) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.ListingOperationCallbacksImpl
-
- createLocatedFileStatusIterator(RemoteIterator<S3AFileStatus>) - Method in class org.apache.hadoop.fs.s3a.Listing
-
Create a located status iterator over a file status iterator.
- createMarkerToolOperations(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create a marker tools operations binding for this store.
- createMetricsUpdatingStore() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create an IOStatistics store which updates FS metrics
as well as IOStatistics.
- createMkdirOperationCallbacks() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override point: create the callbacks for Mkdir.
- createMultipartUploader(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- createMultipartUploaderStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
- createMultipartUploaderStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- createMultipartUploaderStatistics() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AStatisticsContext
-
Creaet a multipart statistics collector.
- createNewSpan(String, String, String) - Static method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
A static source of no-op spans, using the same span ID
source as managed spans.
- createNewStream() - Method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders.BaseContentProvider
-
Override point for subclasses to create their new streams.
- createNonRecursive(Path, FsPermission, EnumSet<CreateFlag>, int, short, long, Progressable) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
The S3A implementations downgrades to the recursive creation, to avoid
any race conditions with parent entries "disappearing".
- createObjectAttributes(Path, String, String, long) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Create the attributes of an object for subsequent use.
- createObjectAttributes(S3AFileStatus) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Create the attributes of an object for subsequent use.
- createOutputCommitter(Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
- createPassword(AbstractS3ATokenIdentifier) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding.TokenSecretManager
-
- createPolicy(ChangeDetectionPolicy.Mode, ChangeDetectionPolicy.Source, boolean) - Static method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Create a policy.
- createProxyConfiguration(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Configures the proxy.
- createPutObjectRequest(String, long, PutObjectOptions) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Create a PutObjectRequest request against the specific key.
- createPutObjectRequest(String, long, PutObjectOptions) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Create a PutObjectRequest request against the specific key.
- createReadContext(FileStatus) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Create the read context for reading from the referenced file,
using FS state as well as the status.
- createReadContext(FileStatus, AuditSpan) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create the read context for reading from the referenced file,
using FS state as well as the status.
- createRequestFactory() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Build the request factory.
- createRetryPolicyBuilder(Configuration) - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Configures the retry policy.
- createS3AInternals() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create the S3AInternals; left as something mocking
subclasses may want to override.
- createS3AStore(ClientManager, int) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create the S3AStore instance.
- createS3AsyncClient(URI, S3ClientFactory.S3ClientCreationParameters) - Method in class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
- createS3AsyncClient(URI, S3ClientFactory.S3ClientCreationParameters) - Method in interface org.apache.hadoop.fs.s3a.S3ClientFactory
-
Creates a new S3AsyncClient.
- createS3Client(URI, S3ClientFactory.S3ClientCreationParameters) - Method in class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
- createS3Client(URI, S3ClientFactory.S3ClientCreationParameters) - Method in interface org.apache.hadoop.fs.s3a.S3ClientFactory
-
Creates a new S3Client.
- createS3TransferManager(S3AsyncClient) - Method in class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
- createS3TransferManager(S3AsyncClient) - Method in interface org.apache.hadoop.fs.s3a.S3ClientFactory
-
Creates a new S3TransferManager.
- createSecretMananger() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Create a secret manager.
- createSigner(String, String) - Static method in class org.apache.hadoop.fs.s3a.auth.SignerFactory
-
Create an instance of the given signer.
- createSingleStatusIterator(S3ALocatedFileStatus) - Method in class org.apache.hadoop.fs.s3a.Listing
-
Create a remote iterator from a single status entry.
- createSpan(String, String, String) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
Start an operation; as well as invoking the audit
service to do this, sets the operation as the
active operation for this thread.
- createSpan(String, String, String) - Method in class org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor
-
- createSpan(String, String, String) - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- createSpan(String, String, String) - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditor
-
- createSpan(String, String, String) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Start an operation; this informs the audit service of the event
and then sets it as the active span.
- createSpan(String, String, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Start an operation; this informs the audit service of the event
and then sets it as the active span.
- createSpan(String, String, String) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
- createSpanID() - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractOperationAuditor
-
Create a span ID.
- createStorageStatistics(IOStatistics) - Static method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create the storage statistics or bind to an existing one.
- createStoreContext() - Method in interface org.apache.hadoop.fs.s3a.impl.StoreContextFactory
-
Build an immutable store context, including picking
up the current audit span.
- createStoreContext() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Build an immutable store context.
- createSuccessMarker(Path, SuccessData, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Save the success data to the _SUCCESS file.
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
Implementation point: create a task committer for a specific filesystem.
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory
-
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory
-
Create a task committer.
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory
-
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory
-
- createTaskCommitter(S3AFileSystem, Path, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory
-
- createTempFile(String, long) - Method in interface org.apache.hadoop.fs.s3a.impl.ContextAccessors
-
Create a temporary file.
- createTempFile(String, long) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Create a temporary file somewhere.
- createThrottledExecutor(int) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Create a new executor service with a given capacity.
- createThrottledExecutor() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- createThrottleRetryPolicy(Configuration) - Method in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Create the throttling policy.
- createTokenIdentifier(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Create a token identifier with all the information needed
to be included in a delegation token.
- createTokenIdentifier(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
Create a new delegation token.
- createTokenIdentifier(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
Create the Token Identifier.
- createTokenIdentifier(Optional<RoleModel.Policy>, EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- createTracker(Path, String, PutTrackerStatistics) - Method in class org.apache.hadoop.fs.s3a.commit.MagicCommitIntegration
-
Given a path and a key to that same path, create a tracker for it.
- createTransferListener() - Method in interface org.apache.hadoop.fs.s3a.audit.AuditManagerS3A
-
Return a transfer callback which
fixes the active span context to be that in which
the transfer listener was created.
- createTransferListener() - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- createTransferListener() - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- createUploadFileStatus(Path, boolean, long, long, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Create a file status for object we just uploaded.
- createWrappedCommitter(JobContext, Configuration) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Create the wrapped committer.
- createWriteOperationHelper(AuditSpan) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create a Write Operation Helper with the given span.
- CREDENTIAL_PROVIDER_CLASSNAME - Static variable in class org.apache.hadoop.fs.s3a.adapter.AwsV1BindingSupport
-
V1 credential provider classname: .
- CredentialInitializationException - Exception in org.apache.hadoop.fs.s3a
-
Exception which Hadoop's AWSCredentialsProvider implementations should
throw when there is a problem with the credential setup.
- CredentialInitializationException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.CredentialInitializationException
-
- CredentialInitializationException(String) - Constructor for exception org.apache.hadoop.fs.s3a.CredentialInitializationException
-
- CredentialProviderListFactory - Class in org.apache.hadoop.fs.s3a.auth
-
- CREDENTIALS_CONVERTED_TO_DELEGATION_TOKEN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
A message added to the standard origin string when the DT is
built from session credentials passed in.
- CRYPTO_CEK_ALGORITHM - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Content encryption algorithm, such as "AES/GCM/NoPadding".
- CRYPTO_KEY - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Deprecated.
- CRYPTO_KEYWRAP_ALGORITHM - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Key wrapping algorithm such as "AESWrap" and "RSA/ECB/OAEPWithSHA-256AndMGF1Padding".
- CSE_PADDING_LENGTH - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
S3 client side encryption adds padding to the content length of constant
length of 16 bytes (at the moment, since we have only 1 content
encryption algorithm).
- CSV_HEADER_OPT_IGNORE - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Ignore the header.
- CSV_HEADER_OPT_NONE - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
No header: first row is data.
- CSV_HEADER_OPT_USE - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Use the header.
- CSV_INPUT_COMMENT_MARKER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
String which indicates the row is actually a comment.
- CSV_INPUT_COMMENT_MARKER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default marker.
- CSV_INPUT_FIELD_DELIMITER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default field delimiter.
- CSV_INPUT_HEADER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
How should headers be used?
Value: "fs.s3a.select.input.csv.header".
- CSV_INPUT_HEADER_OPT_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default header mode: "ignore".
- CSV_INPUT_INPUT_FIELD_DELIMITER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Field delimiter.
- CSV_INPUT_QUOTE_CHARACTER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Quote Character.
- CSV_INPUT_QUOTE_CHARACTER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default Quote Character.
- CSV_INPUT_QUOTE_ESCAPE_CHARACTER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Character to escape quotes.
- CSV_INPUT_QUOTE_ESCAPE_CHARACTER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default quote escape character.
- CSV_INPUT_RECORD_DELIMITER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Record delimiter.
- CSV_INPUT_RECORD_DELIMITER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default delimiter
Value: "\n".
- CSV_OUTPUT_FIELD_DELIMITER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Field delimiter.
- CSV_OUTPUT_FIELD_DELIMITER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default field delimiter.
- CSV_OUTPUT_QUOTE_CHARACTER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Quote Character.
- CSV_OUTPUT_QUOTE_CHARACTER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default Quote Character.
- CSV_OUTPUT_QUOTE_ESCAPE_CHARACTER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Character to escape quotes.
- CSV_OUTPUT_QUOTE_ESCAPE_CHARACTER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default quote escape character.
- CSV_OUTPUT_QUOTE_FIELDS - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Should CSV fields be quoted?
One of : ALWAYS, ASNEEDED
Value: "fs.s3a.select.output.csv.quote.fields".
- CSV_OUTPUT_QUOTE_FIELDS_ALWAYS - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Output quotation policy (default): "always".
- CSV_OUTPUT_QUOTE_FIELDS_AS_NEEEDED - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Output quotation policy: "asneeded".
- CSV_OUTPUT_RECORD_DELIMITER - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Record delimiter.
- CSV_OUTPUT_RECORD_DELIMITER_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Default delimiter
Value: "\n".
- CUSTOM_SIGNERS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
List of custom Signers.
- CustomHttpSigner - Class in org.apache.hadoop.fs.s3a.auth
-
Custom signer that delegates to the AWS V4 signer.
- CustomHttpSigner() - Constructor for class org.apache.hadoop.fs.s3a.auth.CustomHttpSigner
-
- CustomSdkSigner - Class in org.apache.hadoop.fs.s3a.auth
-
This class is for testing the SDK's signing: it
can be declared as the signer class in the configuration
and then the full test suite run with it.
- CustomSdkSigner() - Constructor for class org.apache.hadoop.fs.s3a.auth.CustomSdkSigner
-
- CustomSdkSigner.Initializer - Class in org.apache.hadoop.fs.s3a.auth
-
- CYPHER_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "cypher".
- DATE - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- dateToLong(Date) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Date to long conversion.
- DAYS_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- deactivate(AuditSpanS3A) - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- deactivate() - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopSpan
-
- deactivate(AuditSpanS3A) - Method in interface org.apache.hadoop.fs.s3a.audit.impl.NoopSpan.SpanActivationCallbacks
-
Span was deactivated.
- decodeBytes(byte[]) - Static method in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
Get the string value from the bytes.
- decrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Decrement a gauge by a specific value.
- decrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Decrement a gauge by a specific value.
- decrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Decrement a gauge by a specific value.
- decrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Decrement a specific gauge.
- decrementGauge(Statistic, long) - Method in interface org.apache.hadoop.fs.s3a.statistics.CountersAndGauges
-
Decrement a specific gauge.
- decrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
Decrement a specific gauge.
- decrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- DEFAULT_ALLOW_REQUESTER_PAYS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_ASSUMED_ROLE_STS_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default endpoint for session tokens: "".
- DEFAULT_ASYNC_DRAIN_THRESHOLD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
This is a number based purely on experimentation in
ITestS3AInputStreamPerformance.
- DEFAULT_AUTHORITATIVE_PATH - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_AWS_S3_CLASSLOADER_ISOLATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_AWS_S3_VECTOR_ACTIVE_RANGE_READS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Limit of queued range data download operations during vectored
read.
- DEFAULT_AWS_S3_VECTOR_READS_MAX_MERGED_READ_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default maximum read size in bytes during vectored reads : 1048576.
- DEFAULT_AWS_S3_VECTOR_READS_MIN_SEEK_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default minimum seek in bytes during vectored reads : 4096.
- DEFAULT_BLOCKSIZE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Default blocksize as used in blocksize and FS status queries: 33554432.
- DEFAULT_BLOCKSIZE - Static variable in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Default blocksize as used in blocksize and FS status queries.
- DEFAULT_CANNED_ACL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_COMMITTER_THREADS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- DEFAULT_CONFLICT_MODE - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Default conflict mode: "append".
- DEFAULT_CONNECTION_ACQUISITION_TIMEOUT_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default acquisition timeout: 60 seconds.
- DEFAULT_CONNECTION_IDLE_TIME_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default idle time: 60 seconds.
- DEFAULT_CONNECTION_KEEPALIVE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_CONNECTION_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_CONNECTION_TTL_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_CREATE_SUCCESSFUL_JOB_DIR_MARKER - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Default job marker option: true.
- DEFAULT_DELAY_KEY_SUBSTRING - Static variable in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
Keys containing this substring will be subject to delayed visibility.
- DEFAULT_DELEGATION_TOKEN_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Default token binding "".
- DEFAULT_DELEGATION_TOKEN_DURATION - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Default duration of a delegation token: "1h".
- DEFAULT_DELEGATION_TOKEN_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Default endpoint for session tokens: "".
- DEFAULT_DELEGATION_TOKEN_REGION - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Region default: "".
- DEFAULT_DIRECTORY_MARKER_POLICY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default retention policy: "keep".
- DEFAULT_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value of s3 endpoint: "".
- DEFAULT_ESTABLISH_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_ESTABLISH_TIMEOUT_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default TCP/(and TLS?) establish timeout: 30 seconds.
- DEFAULT_EXECUTOR_CAPACITY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_FAST_BUFFER_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_FAST_UPLOAD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- DEFAULT_FAST_UPLOAD_ACTIVE_BLOCKS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Limit of queued block upload operations before writes
block.
- DEFAULT_FAST_UPLOAD_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default buffer option: "disk".
- DEFAULT_FS_S3A_COMMITTER_ABORT_PENDING_UPLOADS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- DEFAULT_KEEPALIVE_TIME - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_KEEPALIVE_TIME_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_LIST_VERSION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MAGIC_COMMITTER_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Is the committer enabled by default: true.
- DEFAULT_MAX_ERROR_RETRIES - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default number of times the AWS client library should retry errors before
escalating to the S3A code: 5.
- DEFAULT_MAX_PAGING_KEYS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MAX_THREADS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MAX_TOTAL_TASKS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MAXIMUM_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_METADATASTORE_AUTHORITATIVE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- DEFAULT_METADATASTORE_METADATA_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- DEFAULT_MIN_MULTIPART_THRESHOLD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MULTIPART_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_MULTIPART_UPLOAD_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value for multipart uploads.
- DEFAULT_PART_UPLOAD_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default part upload timeout: 15 minutes.
- DEFAULT_PREFETCH_MAX_BLOCKS_COUNT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value for max blocks count config.
- DEFAULT_PURGE_EXISTING_MULTIPART - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_PURGE_EXISTING_MULTIPART_AGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default Age.
- DEFAULT_READAHEAD_RANGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_REQUEST_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_REQUEST_TIMEOUT_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default duration of a request before it is timed out: 60s.
- DEFAULT_RETRY_HTTP_5XX_ERRORS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_S3_CLIENT_FACTORY_IMPL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default factory:
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory.
- DEFAULT_S3A_COMMITTER_GENERATE_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- DEFAULT_S3A_COMMITTER_REQUIRE_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- DEFAULT_S3A_IO_RATE_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_S3GUARD_DISABLED_WARN_LEVEL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- DEFAULT_S3GUARD_METASTORE_LOCAL_ENTRY_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- DEFAULT_S3GUARD_METASTORE_LOCAL_MAX_RECORDS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- DEFAULT_SECURE_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_SOCKET_RECV_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_SOCKET_SEND_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_SOCKET_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_SOCKET_TIMEOUT_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default socket timeout: 200 seconds.
- DEFAULT_SSL_CHANNEL_MODE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- DEFAULT_STAGING_COMMITTER_UNIQUE_FILENAMES - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- DEFAULT_UPLOAD_PART_COUNT_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Maximum entries you can upload in a single file write/copy/upload.
- DefaultS3ClientFactory - Class in org.apache.hadoop.fs.s3a
-
- DefaultS3ClientFactory() - Constructor for class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
- DELEGATION_TOKEN_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Property containing classname for token binding: "fs.s3a.delegation.token.binding".
- DELEGATION_TOKEN_CREDENTIALS_PROVIDER - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Key to list AWS credential providers for Session/role
credentials: "fs.s3a.aws.credentials.provider".
- DELEGATION_TOKEN_DURATION - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Duration of tokens in time: "fs.s3a.assumed.role.session.duration".
- DELEGATION_TOKEN_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Endpoint for session tokens, used when building delegation tokens:
"fs.s3a.assumed.role.sts.endpoint".
- DELEGATION_TOKEN_FULL_CREDENTIALS_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Token binding to pass full credentials: "org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding".
- DELEGATION_TOKEN_REGION - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Region for DT issuing; must be non-empty if the endpoint is set: "fs.s3a.assumed.role.sts.endpoint.region".
- DELEGATION_TOKEN_ROLE_ARN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
ARN of the delegation token: "fs.s3a.assumed.role.arn".
- DELEGATION_TOKEN_ROLE_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Role DTs: "org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding".
- DELEGATION_TOKEN_SESSION_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Session Token binding classname: "org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding".
- DelegationBindingInfo - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Binding information returned by the token provider.
- DelegationBindingInfo() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.DelegationBindingInfo
-
- DelegationConstants - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
All the constants related to delegation tokens.
- DelegationOperations - Interface in org.apache.hadoop.fs.s3a.auth.delegation
-
All operations used for delegation which aren't in the store context.
- DelegationTokenIOException - Exception in org.apache.hadoop.fs.s3a.auth.delegation
-
General IOException for Delegation Token issues.
- DelegationTokenIOException(String) - Constructor for exception org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException
-
- DelegationTokenIOException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenIOException
-
- DelegationTokenProvider - Interface in org.apache.hadoop.fs.s3a.auth.delegation
-
Interface for S3A Delegation Token access.
- DelegationTokenStatistics - Interface in org.apache.hadoop.fs.s3a.statistics
-
Instrumentation exported to for S3A Delegation Token support.
- DELETE - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- DELETE - Static variable in class org.apache.hadoop.fs.s3a.impl.DirectoryPolicyImpl
-
Delete all markers.
- delete(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Delete a Path.
- delete(Path, boolean) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- DELETE_BULK - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- DELETE_BULK_ENTRY - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- DELETE_CONSIDERED_IDEMPOTENT - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
This declared delete as idempotent.
- deleteDestinationPaths(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Delete the working paths of a job.
- deleteDirectoryTree(Path, String) - Method in class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
Delete a directory tree.
- deleteLocal(Path, boolean) - Method in interface org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.CopyFromLocalOperationCallbacks
-
Delete file / directory at path.
- deleteLocal(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.CopyFromLocalCallbacksImpl
-
- deleteObject(DeleteObjectRequest) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Delete an object.
- deleteObject(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Delete an object.
- deleteObject(DeleteObjectRequest) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Delete an object.
- deleteObjectAtPath(Path, String, boolean) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Delete an object.
- deleteObjects(DeleteObjectsRequest) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Perform a bulk object delete operation against S3.
- deleteObjects(DeleteObjectsRequest) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Perform a bulk object delete operation against S3.
- deleteOnExit(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
This override bypasses checking for existence.
- DeleteOperation - Class in org.apache.hadoop.fs.s3a.impl
-
Implementation of the delete() operation.
- DeleteOperation(StoreContext, S3AFileStatus, boolean, OperationCallbacks, int, boolean) - Constructor for class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
Constructor.
- deleteQuietly(FileSystem, Path, boolean) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Delete a path quietly: failures are logged at DEBUG.
- deleteStagingUploadsParentDirectory(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Delete the multipart upload staging directory.
- deleteSuccessMarker(Path) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Delete any existing _SUCCESS file.
- deleteTaskAttemptPathQuietly(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Delete the task attempt path without raising any errors.
- deleteTaskWorkingPathQuietly(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Delete the working path of a task; no-op if there is none, that
is: this is a job.
- deleteWithoutCloseCheck(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Same as delete(), except that it does not check if fs is closed.
- deleteWithWarning(FileSystem, Path, boolean) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Delete a path: failures are logged at WARN.
- deletingDirs() - Static method in class org.apache.hadoop.fs.s3a.impl.PutObjectOptions
-
Get the options to delete directory markers.
- deploy(AbstractS3ATokenIdentifier) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Deploy, returning the binding information.
- deployUnbonded() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Perform any actions when deploying unbonded, and return a list
of credential providers.
- deployUnbonded() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- deployUnbonded() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Return an unbonded provider chain.
- describe() - Method in interface org.apache.hadoop.fs.s3a.impl.DirectoryPolicy
-
Describe the policy for marker tools and logs.
- describe() - Method in class org.apache.hadoop.fs.s3a.impl.DirectoryPolicyImpl
-
- description() - Static method in class org.apache.hadoop.fs.s3a.auth.CustomSdkSigner
-
- destinationPath() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Build the destination path of the object.
- DIRECTORIES - Static variable in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
Skip the HEAD and only look for directories.
- directory(Path) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Given a directory path, return the S3 resource to it.
- DIRECTORY_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Directory committer factory: "org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory".
- DIRECTORY_MARKER_POLICY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Policy for directory markers.
- DIRECTORY_MARKER_POLICY_AUTHORITATIVE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Retain directory markers in authoritative directory trees only.
- DIRECTORY_MARKER_POLICY_DELETE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Delete directory markers.
- DIRECTORY_MARKER_POLICY_KEEP - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Retain directory markers.
- DIRECTORY_OPERATIONS_PURGE_UPLOADS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Should directory operations purge uploads?
This adds at least one parallelized list operation to the call,
plus the overhead of deletions.
- DIRECTORY_OPERATIONS_PURGE_UPLOADS_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- directoryCreated() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A created a directory.
- directoryDeleted() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A just deleted a directory.
- DirectoryPolicy - Interface in org.apache.hadoop.fs.s3a.impl
-
Interface for Directory Marker policies to implement.
- DirectoryPolicy.MarkerPolicy - Enum in org.apache.hadoop.fs.s3a.impl
-
Supported retention policies.
- DirectoryPolicyImpl - Class in org.apache.hadoop.fs.s3a.impl
-
Implementation of directory policy.
- DirectoryPolicyImpl(DirectoryPolicy.MarkerPolicy, Predicate<Path>) - Constructor for class org.apache.hadoop.fs.s3a.impl.DirectoryPolicyImpl
-
Constructor.
- DirectoryStagingCommitter - Class in org.apache.hadoop.fs.s3a.commit.staging
-
This commits to a directory.
- DirectoryStagingCommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter
-
- DirectoryStagingCommitterFactory - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Factory for the Directory committer.
- DirectoryStagingCommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory
-
- DirMarkerTracker - Class in org.apache.hadoop.fs.s3a.impl
-
Tracks directory markers which have been reported in object listings.
- DirMarkerTracker(Path, boolean) - Constructor for class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
Construct.
- DirMarkerTracker.Marker - Class in org.apache.hadoop.fs.s3a.impl
-
This is a marker entry stored in the map and
returned as markers are deleted.
- DOES_NOT_IMPLEMENT - Static variable in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
- DOWNGRADE_SYNCABLE_EXCEPTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Rather than raise an exception when an attempt is
made to call the Syncable APIs, warn and downgrade.
- DOWNGRADE_SYNCABLE_EXCEPTIONS_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value for syncable invocation.
- DRAIN_BUFFER_SIZE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
size of a buffer to create when draining the stream.
- dstFileStatus - Variable in class org.apache.hadoop.fs.s3a.S3AOpContext
-
FileStatus for "destination" path being operated on.
- dump(String, String, String, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Dump all the metrics to a string.
- dumpDiagnostics(String, String, String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Dump the diagnostics (if any) to a string.
- dumpFileSystemStatistics(PrintStream) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Dump the filesystem Storage Statistics if the FS is not null.
- dumpMetrics(String, String, String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Dump the metrics (if any) to a string.
- duration(Statistic) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Registering a duration adds the success and failure counters.
- E_BAD_PATH - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error message for bad path: "Path does not represent a magic-commit path".
- E_DELEGATION_TOKENS_DISABLED - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
- E_DEST_EXISTS - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error text when the destination path exists and the committer
must abort the job/task "Destination path exists and committer conflict resolution mode is \"fail\"".
- E_FORBIDDEN_AWS_PROVIDER - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
Error message when the AWS provider list built up contains a forbidden
entry.
- E_NEGATIVE_READAHEAD_VALUE - Static variable in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- E_NO_ARN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
Error message when there is no Role ARN.
- E_NO_AWS_CREDENTIALS - Static variable in exception org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException
-
The default error message: "No AWS Credentials".
- E_NO_MAGIC_PATH_ELEMENT - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error message for a path without a magic element in the list: "No __magic_job- element in path".
- E_NO_ROLE - Static variable in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
- E_NO_SESSION_TOKENS_FOR_ROLE_BINDING - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
If the token binding auth chain is only session-level auth, you
can't use the role binding: "Cannot issue S3A Role Delegation Tokens without full AWS credentials".
- E_NO_SPARK_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
- E_NO_SUCH_BUCKET - Static variable in class org.apache.hadoop.fs.s3a.impl.ErrorTranslation.AwsErrorCodes
-
The AWS S3 error code used to recognize when a 404 means the bucket is
unknown.
- E_NORMAL_FS - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error message if filesystem isn't magic: "Filesystem does not have support for \'magic\' committer enabled in configuration option fs.s3a.committer.magic.enabled".
- E_SELF_GENERATED_JOB_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Error string when task setup fails.
- E_WRONG_FS - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Error message if the dest FS isn't S3A: "Output path is not on an S3A Filesystem".
- EC2_CONTAINER_CREDENTIALS_V1 - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
V1 credential provider: "com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper".
- EC2_IAM_CREDENTIALS_V1 - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
V1 credential provider: "com.amazonaws.auth.InstanceProfileCredentialsProvider".
- EC2_IAM_CREDENTIALS_V2 - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
V2 EC2 instance/container credential provider.
- effect(boolean) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Map a bool to an effect.
- effect - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
Default effect is Deny; forces callers to switch on Allow.
- elementsToKey(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Take a list of elements and create an S3 key by joining them
with "/" between each one.
- empty() - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Return a set of empty credentials.
- empty() - Static method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
Get the empty entry.
- EMPTY - Static variable in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- EMPTY - Static variable in class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
- EMPTY_BLOCK_OUTPUT_STREAM_STATISTICS - Static variable in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- EMPTY_COMMITTER_STATISTICS - Static variable in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- EMPTY_DELEGATION_TOKEN_STATISTICS - Static variable in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- EMPTY_INPUT_STREAM_STATISTICS - Static variable in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- EMPTY_STATISTICS_FROM_AWS_SDK - Static variable in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- EmptyMultipartUploaderStatistics() - Constructor for class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext.EmptyMultipartUploaderStatistics
-
- EmptyS3AStatisticsContext - Class in org.apache.hadoop.fs.s3a.statistics.impl
-
Special statistics context, all of whose context operations are no-ops.
- EmptyS3AStatisticsContext() - Constructor for class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- EmptyS3AStatisticsContext.EmptyMultipartUploaderStatistics - Class in org.apache.hadoop.fs.s3a.statistics.impl
-
Multipart Uploader.
- ENABLE_MULTI_DELETE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- encodeBytes(Object) - Static method in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
Stringify an object and return its bytes in UTF-8 encoding.
- ENCRYPTION_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- EncryptionSecretOperations - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
- EncryptionSecrets - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Encryption options in a form which can serialized or marshalled as a hadoop
Writeable.
- EncryptionSecrets() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Empty constructor, for use in marshalling.
- EncryptionSecrets(S3AEncryptionMethods, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Create a pair of secrets.
- EncryptionSecrets(String, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Create a pair of secrets.
- ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Endpoint.
- ENDPOINT_FIPS_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- ENDPOINT_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "endpoint".
- enforceMinimumDuration(String, Duration, Duration) - Static method in class org.apache.hadoop.fs.s3a.impl.ConfigurationHelper
-
Enforce a minimum duration of a configuration option, if the supplied
value is non-null.
- ensureCurrentBuffer() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ACachingInputStream
-
- ensureCurrentBuffer() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3AInMemoryInputStream
-
Ensures that a non-empty valid buffer is available for immediate reading.
- ensureCurrentBuffer() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Ensures that a non-empty valid buffer is available for immediate reading.
- ensureOutputParameterInRange(String, long) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Ensure that the long value is in the range of an integer.
- entryPoint(Statistic, Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Entry point to an operation.
- entryPoint(Statistic, String, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Entry point to an operation.
- ENVIRONMENT_CREDENTIALS_V1 - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
V1 env var credential provider: "com.amazonaws.auth.EnvironmentVariableCredentialsProvider".
- ENVIRONMENT_CREDENTIALS_V2 - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
V2 environment variables credential provider.
- EOF_MESSAGE_IN_XML_PARSER - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
- EOF_READ_DIFFERENT_LENGTH - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Equality check is on superclass and UUID only.
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Compare if this object is equal to another object.
- equals(Object) - Method in class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- equals(Object) - Method in class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
Equality test matches user and password.
- ERROR_ENDPOINT_WITH_FIPS - Static variable in class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
Error message when an endpoint is set with FIPS enabled: "Non central endpoint cannot be set when fs.s3a.endpoint.fips is true".
- errorIgnored() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Note that an error was ignored.
- errorln() - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- errorln(String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- errors() - Method in exception org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteException
-
- errorToString(S3Error) - Static method in exception org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteException
-
Convert an error to a string.
- ErrorTranslation - Class in org.apache.hadoop.fs.s3a.impl
-
Translate from AWS SDK-wrapped exceptions into IOExceptions with
as much information as possible.
- ErrorTranslation.AwsErrorCodes - Class in org.apache.hadoop.fs.s3a.impl
-
AWS error codes explicitly recognized and processes specially;
kept in their own class for isolation.
- ESTABLISH_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Milliseconds until a connection is established: "fs.s3a.connection.establish.timeout".
- ETAG - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- ETAG_CHECKSUM_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Should etags be exposed as checksums?
- ETAG_CHECKSUM_ENABLED_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value: false.
- exceptionInMultipartAbort() - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
Note an exception in a multipart abort.
- exceptionInMultipartComplete(int) - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
Note exception in a multipart complete.
- execMarkerTool(MarkerTool.ScanArgs) - Static method in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Execute the marker tool, with no checks on return codes.
- execute() - Method in class org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation
-
- execute() - Method in class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
Delete a file or directory tree.
- execute() - Method in class org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation
-
Execute the operation.
- execute() - Method in class org.apache.hadoop.fs.s3a.impl.GetContentSummaryOperation
-
Return the ContentSummary of a given path.
- execute() - Method in class org.apache.hadoop.fs.s3a.impl.MkdirOperation
-
Make the given path and all non-existent parents into
directories.
- execute() - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
- executed() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- executeOnlyOnce() - Method in class org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation
-
Check that the operation has not been invoked twice.
- ExecutingStoreOperation<T> - Class in org.apache.hadoop.fs.s3a.impl
-
- ExecutingStoreOperation(StoreContext) - Constructor for class org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation
-
Constructor.
- ExecutingStoreOperation(StoreContext, AuditSpan) - Constructor for class org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation
-
Constructor.
- EXECUTOR_CAPACITY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- exists(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override superclass so as to add statistic collection.
- exit(int, String) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- exitException(int, String, Object...) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Build a exception to throw with a formatted message.
- EXPERIMENTAL_AWS_INTERNAL_THROTTLING - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Experimental/Unstable feature: should the AWS client library retry
throttle responses before escalating to the S3A code: "fs.s3a.experimental.aws.s3.throttling".
- EXPERIMENTAL_AWS_INTERNAL_THROTTLING_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- extendedRequestId() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- extractException(String, String, ExecutionException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Extract an exception from a failed future, and convert to an IOE.
- extractException(String, String, CompletionException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Extract an exception from a failed future, and convert to an IOE.
- extractIdentifier(Token<? extends AbstractS3ATokenIdentifier>) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
From a token, get the session token identifier.
- extractJobID(Configuration) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Extract the job ID from a configuration.
- extractMagicFileLength(FileSystem, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Get the magic file length of a file.
- extractTaskAttemptIdFromPath(Path) - Static method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTrackerUtils
-
The magic path is of the following format.
- extractXAttrLongValue(byte[]) - Static method in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
Convert an XAttr byte array to a long.
- fail - Variable in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
No retry on network and tangible API issues.
- FAIL_INJECT_CLIENT_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FAIL_INJECT_INCONSISTENCY_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- FAIL_INJECT_INCONSISTENCY_MSEC - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- FAIL_INJECT_INCONSISTENCY_PROBABILITY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- FAIL_INJECT_THROTTLE_PROBABILITY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FAIL_ON_METADATA_WRITE_ERROR - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- FAIL_ON_METADATA_WRITE_ERROR_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- failDestinationExists(Path, String) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Generate a PathExistsException because the destination exists.
- FailureInjectionPolicy - Class in org.apache.hadoop.fs.s3a
-
Simple object which stores current failure injection settings.
- FailureInjectionPolicy(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- fakeDirsDeleted(int) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that fake directory request was made.
- FAST_BUFFER_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- FAST_UPLOAD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- FAST_UPLOAD_ACTIVE_BLOCKS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Maximum Number of blocks a single output stream can have
active (uploading, or queued to the central FileSystem
instance's pool of queued operations.
- FAST_UPLOAD_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
What buffer to use.
- FAST_UPLOAD_BUFFER_ARRAY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Use an in-memory array.
- FAST_UPLOAD_BUFFER_DISK - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Buffer blocks to disk: "disk".
- FAST_UPLOAD_BYTEBUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Use a byte buffer.
- FILE - Static variable in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
Look for files and directories.
- FILE_NOT_FOUND_SINGLE_ATTEMPT - Static variable in exception org.apache.hadoop.fs.s3a.RemoteFileChangedException
-
The file disappeaded during a rename between LIST and COPY.
- fileContentProvider(File, long, int) - Static method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders
-
Create a content provider from a file.
- fileContentProvider(File, long, int, Supplier<Boolean>) - Static method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders
-
Create a content provider from a file.
- fileCreated() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A created a file.
- fileDeleted(int) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A deleted one or more files.
- fileFound(Path, String, S3ALocatedFileStatus) - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
A file has been found.
- filename(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Get the filename of a path: the last element.
- filesCopied(int, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Indicate that S3A copied some files within the store.
- FILESYSTEM_TEMP_PATH - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterConstants
-
The temporary path for staging data, if not explicitly set.
- finalDestination(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Calculates the final destination of a file.
- finalize() - Method in class org.apache.hadoop.fs.s3a.S3A
-
Close the file system; the FileContext API doesn't have an explicit close.
- finish() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool.ScanResult
-
Throw an exception if the exit code is non-zero.
- finishedWrite(String, long, PutObjectOptions) - Method in interface org.apache.hadoop.fs.s3a.WriteOperationHelper.WriteOperationHelperCallbacks
-
Perform post-write actions.
- finishRename(Path, Path) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
The rename has finished; perform any store cleanup operations
such as creating/deleting directory markers.
- FIPS_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Is the endpoint a FIPS endpoint?
Can be queried as a path capability.
- FIPS_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- fixBucketRegion(String) - Static method in class org.apache.hadoop.fs.s3a.impl.NetworkBinding
-
Given an S3 bucket region as returned by a bucket location query,
fix it into a form which can be used by other AWS commands.
- flatmapLocatedFiles(RemoteIterator<LocatedFileStatus>, S3AUtils.LocatedFileStatusMap<Optional<T>>) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Map an operation to every LocatedFileStatus in a remote
iterator, returning a list of the all results which were not empty.
- formatAppAttemptDir(String, int) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Build the name of the job attempt directory.
- formatJobDir(String) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Build the name of the job directory, without
app attempt.
- formatRange(long, long) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Format a byte range for a request header.
- from(Optional<Boolean>) - Static method in enum org.apache.hadoop.fs.s3a.Tristate
-
Build a tristate from a boolean.
- fromAWSCredentials(AwsSessionCredentials) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Create from a set of AWS credentials.
- fromBool(boolean) - Static method in enum org.apache.hadoop.fs.s3a.Tristate
-
- fromEnvironment(Map<String, String>) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Build a set of credentials from the environment.
- fromFileStatus(FileStatus, Tristate, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Convenience constructor for creating from a vanilla FileStatus plus
an isEmptyDirectory flag.
- fromFileSystem(URI, Configuration) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Loads the credentials from the owning S3A FS, including
from Hadoop credential providers.
- fromStatusIterator(FileSystem, RemoteIterator<? extends FileStatus>) - Static method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
Create an active commit of the given pending files.
- fromSTSCredentials(Credentials) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Create a set of marshalled credentials from a set of credentials
issued by an STS call.
- fromSymbol(String) - Static method in enum org.apache.hadoop.fs.s3a.Statistic
-
Get a statistic from a symbol.
- FS_S3A - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FS_S3A_BLOCK_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FS_S3A_BUCKET_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Prefix for S3A bucket-specific properties: "fs.s3a.bucket.".
- FS_S3A_COMMITTER_ABORT_PENDING_UPLOADS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Should committers abort all pending uploads to the destination
directory?
- FS_S3A_COMMITTER_GENERATE_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Generate a UUID in job setup rather than fall back to
YARN Application attempt ID.
- FS_S3A_COMMITTER_MAGIC_TRACK_COMMITS_IN_MEMORY_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Should Magic committer track all the pending commits in memory?
- FS_S3A_COMMITTER_MAGIC_TRACK_COMMITS_IN_MEMORY_ENABLED_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- FS_S3A_COMMITTER_NAME - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Option to identify the S3A committer:
"fs.s3a.committer.name".
- FS_S3A_COMMITTER_REQUIRE_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Require the spark UUID to be passed down: "fs.s3a.committer.require.uuid".
- FS_S3A_COMMITTER_STAGING_ABORT_PENDING_UPLOADS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Deprecated.
- FS_S3A_COMMITTER_STAGING_CONFLICT_MODE - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Staging committer conflict resolution policy: "fs.s3a.committer.staging.conflict-mode".
- FS_S3A_COMMITTER_STAGING_TMP_PATH - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Path in the cluster filesystem for temporary data: "fs.s3a.committer.staging.tmp.path".
- FS_S3A_COMMITTER_STAGING_UNIQUE_FILENAMES - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Option for final files to have a uniqueness name through job attempt info,
falling back to a new UUID if there is no job attempt information to use.
- FS_S3A_COMMITTER_THREADS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Number of threads in committers for parallel operations on files
(upload, commit, abort, delete...): "fs.s3a.committer.threads".
- FS_S3A_COMMITTER_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
A unique identifier to use for this work: "fs.s3a.committer.uuid".
- FS_S3A_COMMITTER_UUID_SOURCE - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Where did the UUID come from? "fs.s3a.committer.uuid.source".
- FS_S3A_CREATE_HEADER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Prefix for adding a header to the object when created.
- FS_S3A_CREATE_PERFORMANCE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Flag for create performance.
- FS_S3A_CREATE_PERFORMANCE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value for create performance in an S3A FS.
- FS_S3A_CREATE_PERFORMANCE_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- FS_S3A_PERFORMANCE_FLAGS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Comma separated list of performance flags.
- FS_S3A_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Prefix for all S3A properties: "fs.s3a.".
- FS_S3A_SELECT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
- FS_S3A_SELECT_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Flag: is S3 select enabled?
Value: "fs.s3a.select.enabled".
- FS_S3A_SELECT_INPUT_CSV - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Prefix for all CSV input options.
- FS_S3A_SELECT_OUTPUT_CSV - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Prefix for all CSV output options.
- FULL_TOKEN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- FULL_TOKEN_KIND - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
- FULL_TOKEN_NAME - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Name of full token: "S3ADelegationToken/Full".
- FullCredentialsTokenBinding - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Full credentials: they are simply passed as-is, rather than
converted to a session.
- FullCredentialsTokenBinding() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- FullCredentialsTokenIdentifier - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
The full credentials payload is the same of that for a session token, but
a different token kind is used.
- FullCredentialsTokenIdentifier() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenIdentifier
-
- FullCredentialsTokenIdentifier(URI, Text, Text, MarshalledCredentials, EncryptionSecrets, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenIdentifier
-
- fullKey(S3AFileStatus) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Build the full S3 key for a request from the status entry,
possibly adding a "/" if it represents directory and it does
not have a trailing slash already.
- gauge(String, String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a gauge in the registry.
- gauge(Statistic) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a gauge in the registry.
- GET - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- get() - Method in class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
Active any span and then call the supplied callable.
- GET_ACL - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- GET_LOGGING_STATUS - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- getAborted() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getAccessKey() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- getActiveAuditSpan() - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
Return the active wrapped span.
- getActiveAuditSpan() - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- getActiveAuditSpan() - Method in interface org.apache.hadoop.fs.s3a.impl.ContextAccessors
-
Return the active audit span.
- getActiveAuditSpan() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Return the active audit span.
- getActiveAuditSpan() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the active audit span.
- getAdditionalTokenIssuers() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get a null/possibly empty list of extra delegation token issuers.
- getAdditionalTokenIssuers() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Ask any DT plugin for any extra token issuers.
- getAmazonS3Client(String) - Method in interface org.apache.hadoop.fs.s3a.S3AInternals
-
Returns the S3 client used by this filesystem.
- getAppAttemptId(JobContext) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Get the Application Attempt ID for this job.
- getAsyncDrainThreshold() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
- getAuditManager() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
The audit manager.
- getAuditor() - Method in interface org.apache.hadoop.fs.s3a.audit.AuditManagerS3A
-
Get the auditor; valid once initialized.
- getAuditor() - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- getAuditor() - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- getAuditor() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the auditor.
- getAuditor() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the auditor; valid once initialized.
- getAuditorId() - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractOperationAuditor
-
- getAuditorId() - Method in interface org.apache.hadoop.fs.s3a.audit.OperationAuditor
-
Get the Auditor ID.
- getAuditSpan() - Method in class org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation
-
Get the audit span this object was created with.
- getAuditSpan() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get the audit which was active when the file was opened.
- getAuditSpan() - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Get the audit span this object was created with.
- getAuditSpanSource() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- getAuditSpanSource() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the audit span source; allows for components like the committers
to have a source of spans without being hard coded to the FS only.
- getAuthoritativePaths(S3AFileSystem) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3Guard
-
- getAWSAccessKeys(URI, Configuration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Return the access key and secret for S3 API use.
- getBackwardSeekOperations() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getBaseMagicTaskAttemptPath(TaskAttemptContext, String, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Get the base Magic attempt path, without any annotations to mark relative
references.
- getBasePath() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
Get the base path of the tracker.
- getBaseTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Compute the base path where the output of a task attempt is written.
- getBaseTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- getBaseTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Return the local work path as the destination for writing work.
- getBlockData() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getBlocksActivelyAllocated() - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
Get counters of blocks actively allocated; may be inaccurate
if the numbers change during the (non-synchronized) calculation.
- getBlocksAllocated() - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
- getBlocksReleased() - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
- getBoundDT() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get any bound DT.
- getBoundOrNewDT(EncryptionSecrets, Text) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get any bound DT or create a new one.
- getBucket() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- getBucket() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getBucket() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- getBucket() - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
Get the target bucket.
- getBucket() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getBucket() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the bucket of this filesystem.
- getBucket() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getBucketLocation() - Method in interface org.apache.hadoop.fs.s3a.impl.ContextAccessors
-
Get the region of a bucket.
- getBucketLocation() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the location of the bucket.
- getBucketLocation() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
S3AInternals method.
- getBucketLocation() - Method in interface org.apache.hadoop.fs.s3a.S3AInternals
-
Get the region of a bucket.
- getBucketLocation(String) - Method in interface org.apache.hadoop.fs.s3a.S3AInternals
-
Get the region of a bucket; fixing up the region so it can be used
in the builders of other AWS clients.
- getBucketMetadata() - Method in interface org.apache.hadoop.fs.s3a.impl.HeaderProcessing.HeaderProcessingCallbacks
-
Retrieve the bucket metadata.
- getBucketMetadata() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Request bucket metadata.
- getBucketMetadata() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.HeaderProcessingCallbacksImpl
-
- getBucketMetadata() - Method in interface org.apache.hadoop.fs.s3a.S3AInternals
-
Request bucket metadata.
- getBucketOption(Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a bucket-specific property.
- getBufferSize() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
- getBytesBackwardsOnSeek() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getBytesDiscardedInAbort() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getBytesPendingUpload() - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
Get the number of bytes pending upload.
- getBytesRead() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
The bytes read in read() operations.
- getBytesReadInClose() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getBytesSkippedOnSeek() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getBytesWritten() - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
Get the current count of bytes written.
- getCannedACL() - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Get the canned ACL of this FS.
- getCannedACL() - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
Get the canned ACL of this FS.
- getCanonicalServiceName() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
The canonical name of the service.
- getCanonicalServiceName() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Return a service name iff delegation tokens are enabled and the
token binding is issuing delegation tokens.
- getCanonicalUri() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Get the canonical URI of the filesystem, which is what is
used to identify the tokens.
- getCanonicalUri() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the canonical URI.
- getCause() - Method in exception org.apache.hadoop.fs.s3a.AWSClientIOException
-
- getCause() - Method in exception org.apache.hadoop.fs.s3a.AWSS3IOException
-
- getCause() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- getChangeDetectionPolicy() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getChangeDetectionPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the change detection policy for this FS instance.
- getChangeDetectionPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
- getChangePolicy() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport
-
- getChangePolicy() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
- getChangeTrackerStatistics() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
Get a reference to the change tracker statistics for this
stream.
- getClassname() - Method in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
- getClosed() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getCloseOperations() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getCommandFormat() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- getCommitOperations() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the commit actions instance.
- getCommits() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- getCommittedBytes() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getCommittedFileCount() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getCommittedObjects() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getCommittedTaskPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Compute the path where the output of a committed task is stored until
the entire job is committed.
- getCommittedTaskPath(int, TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Compute the path where the output of a committed task is stored until the
entire job is committed for a specific application attempt.
- getCommitter() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getCommonPrefixes() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- getConf() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractAWSCredentialProvider
-
- getConf() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- getConf() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Job configuration.
- getConf() - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Get the configuration of this instance; essentially the owning
filesystem configuration.
- getConf() - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Get the configuration of this instance; essentially the owning
filesystem configuration.
- getConfictModeOption(JobContext, Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the conflict mode option string.
- getConfiguration() - Method in class org.apache.hadoop.fs.s3a.audit.OperationAuditorOptions
-
- getConfiguration() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getConfiguration() - Method in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Get the configuration this policy was created from.
- getConfigurationOption(JobContext, Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Get a configuration option, with any value in the job configuration
taking priority over that in the filesystem.
- getConflictResolutionMode(JobContext, Configuration) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- getContentEncoding() - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Get the content encoding (e.g.
- getContentEncoding() - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
Get the content encoding (e.g.
- getContentProvider() - Method in class org.apache.hadoop.fs.s3a.S3ADataBlocks.BlockUploadData
-
The content provider.
- getContentRangeFinish() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- getContentRangeStart() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- getContentSummary(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
This is a very slow operation against object storage.
- GetContentSummaryCallbacksImpl() - Constructor for class org.apache.hadoop.fs.s3a.S3AFileSystem.GetContentSummaryCallbacksImpl
-
- GetContentSummaryOperation - Class in org.apache.hadoop.fs.s3a.impl
-
GetContentSummary operation.
- GetContentSummaryOperation(StoreContext, Path, GetContentSummaryOperation.GetContentSummaryCallbacks) - Constructor for class org.apache.hadoop.fs.s3a.impl.GetContentSummaryOperation
-
Constructor.
- GetContentSummaryOperation.GetContentSummaryCallbacks - Interface in org.apache.hadoop.fs.s3a.impl
-
Callbacks used by the operation.
- getContext() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getContextAccessors() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getCounterValue(Statistic) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Get the value of a counter.
- getCounterValue(String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Get the value of a counter.
- getCreated() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- getCreated() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
When was the upload created?
- getCreationCount() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
How many delegation tokens have been issued?
- getCredentialProviders() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationBindingInfo
-
Get list of credential providers.
- getCredentialProviders() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get the AWS credential provider.
- getCredentialSet() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
- getCurrentStream() - Method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders.BaseContentProvider
-
Current stream.
- getDate() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Timestamp as date; no expectation of parseability.
- getDate() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getDecodedIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get any decoded identifier from the bound DT; empty if not bound.
- getDefaultAsyncDrainThreshold() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport
-
- getDefaultBlockSize(Path) - Method in interface org.apache.hadoop.fs.s3a.impl.ListingOperationCallbacks
-
Return the number of bytes that large input files should be optimally
be split into to minimize I/O time.
- getDefaultBlockSize() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Return the number of bytes that large input files should be optimally
be split into to minimize I/O time.
- getDefaultBlockSize(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.ListingOperationCallbacksImpl
-
- getDefaultBufferSize() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport
-
- getDefaultPort() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getDefaultReadAhead() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport
-
- getDelegationToken(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get a delegation token if the FS is set up for them.
- getDelegationTokens() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the delegation token support for this filesystem;
not null iff delegation support is enabled.
- getDescription() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Return a description.
- getDescription() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
- getDescription() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- getDescription() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getDescription() - Method in enum org.apache.hadoop.fs.s3a.Statistic
-
- getDestFS() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the destination FS, creating it on demand if needed.
- getDestinationFileSystem(Path, JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
Get the destination filesystem, returning null if there is none.
- getDestinationFS(Path, Configuration) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the destination filesystem from the output path and the configuration.
- getDestinationKey() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getDestKey() - Method in class org.apache.hadoop.fs.s3a.commit.PutTracker
-
get the destination key.
- getDestS3AFS() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the destination as an S3A Filesystem; casting it.
- getDiagnostics() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getDirectoryMarkerPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the directory marker policy of this filesystem.
- getDirectoryPolicy(Configuration, Predicate<Path>) - Static method in class org.apache.hadoop.fs.s3a.impl.DirectoryPolicyImpl
-
Create/Get the policy for this configuration.
- getDirSummary(Path) - Method in class org.apache.hadoop.fs.s3a.impl.GetContentSummaryOperation
-
Return the ContentSummary of a given directory.
- getDrained() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- getDstFileStatus() - Method in class org.apache.hadoop.fs.s3a.S3AOpContext
-
- getDuration() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Duration of sessions.
- getDuration(Configuration, String, Duration, TimeUnit, Duration) - Static method in class org.apache.hadoop.fs.s3a.impl.ConfigurationHelper
-
Get a duration.
- getDurationTrackerFactory() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- getDurationTrackerFactory() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the factory for duration tracking.
- getDurationTrackerFactory() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Get the duration tracker factory.
- getDurationTrackerFactory() - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
- getEncryptionAlgorithm() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
- getEncryptionAlgorithm(String, Configuration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get the server-side encryption or client side encryption algorithm.
- getEncryptionKey() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
- getEncryptionMethod() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Get the encryption method.
- getEncryptionSecrets() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Get the encryption secrets.
- getEncryptionSecrets() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get the encryption secrets of the DT.
- getEncryptionSecrets() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the encryption secrets.
- getEndpoint() - Method in class org.apache.hadoop.fs.s3a.ArnResource
-
Formatted endpoint for the resource.
- getEndpoint() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
- getETag() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
- getEtag() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
- getETag() - Method in class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- getEtag() - Method in class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- getETag() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getEtags() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getException() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations.MaybeIOE
-
Get any exception.
- getExecutionInterceptors() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
List of execution interceptors to include in the chain
of interceptors in the SDK.
- getExecutor() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getExitCode() - Method in exception org.apache.hadoop.fs.s3a.RenameFailedException
-
- getExitCode() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool.ScanResult
-
- getExpiration() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Expiration; will be 0 for none known.
- getExpirationDateTime() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Expiration date time as passed in from source.
- getExpirationDateTime() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Get a temporal representing the time of expiration, if there
is one.
- getExpiryTime() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Return the expiry time in seconds since 1970-01-01.
- getExpiryTime() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Return the expiry time in seconds since 1970-01-01.
- getExtraData() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Any custom extra data committer subclasses may choose to add.
- getFailureLimit() - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- getFile() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getFileChecksum(Path, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
When enabled, get the etag of a object at the path via HEAD request and
return it as a checksum object.
- getFileLength() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
Get the file length.
- getFilename() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
This is the filename of the pending file itself.
- getFilenames() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getFilePosition() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getFilesDeleted() - Method in class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
- getFilesFound() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
- getFileStatus(Path) - Method in interface org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.CopyFromLocalOperationCallbacks
-
Get the file status for a path.
- getFileStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.CopyFromLocalCallbacksImpl
-
- getFileStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Return a file status object that represents the path.
- getFileStatus(Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- getFileStatusesAssumingNonEmptyDir(Path, AuditSpan) - Method in class org.apache.hadoop.fs.s3a.Listing
-
Calculate list of file statuses assuming path
to be a non-empty directory.
- getFilesystem() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- getFileUpload() - Method in class org.apache.hadoop.fs.s3a.UploadInfo
-
- getFilteredMarkerCount() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool.ScanResult
-
- getFinalKey(String, JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Returns the final S3 key for a relative path.
- getFinalPath(String, JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Returns the final S3 location for a relative path as a Hadoop Path.
- getFirstSupportedPolicy(Collection<String>, S3AInputPolicy) - Static method in enum org.apache.hadoop.fs.s3a.S3AInputPolicy
-
Scan the list of input policies, returning the first one supported.
- getFlags() - Method in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder.CreateFileOptions
-
- getFlags() - Method in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder
-
make the flag getter public.
- getForwardSeekOperations() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getFsDelegationToken() - Method in interface org.apache.hadoop.fs.s3a.auth.delegation.DelegationTokenProvider
-
- getFsDelegationToken() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getFsStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get FS Statistic for this S3AFS instance.
- getFsURI() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getFullArn() - Method in class org.apache.hadoop.fs.s3a.ArnResource
-
Full arn for resource.
- getFuturePool() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Gets the ExecutorServiceFuturePool used for asynchronous prefetches.
- getHeaders() - Method in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder.CreateFileOptions
-
- getHeaders() - Method in class org.apache.hadoop.fs.s3a.impl.PutObjectOptions
-
Headers for the put/post request.
- getHeaders() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
Get the map of headers.
- getHostname() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getInitializationException() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Get any IOE raised during initialization.
- getInnerSubmitter() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Return a submitter.
- getInputPolicy() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
- getInputPolicy() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getInputPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the input policy for this FS instance.
- getInputPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Get the current input policy.
- getInputPolicy() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get the IO policy.
- getInputPolicy() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getInstanceFromReflection(String, Configuration, URI, Class<? extends InstanceT>, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Creates an instance of a class using reflection.
- getInstanceStatistics() - Method in interface org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext.S3AFSStatisticsSource
-
Get the statistics of the FS instance, shared across all threads.
- getInstantiationCount() - Static method in class org.apache.hadoop.fs.s3a.auth.CustomSdkSigner
-
- getInstrumentation() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the statistics context for this StoreContext.
- getInstrumentation() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get S3A Instrumentation.
- getInstrumentation() - Method in interface org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext.S3AFSStatisticsSource
-
Get the S3A Instrumentation.
- getInvocationCount() - Static method in class org.apache.hadoop.fs.s3a.auth.CustomSdkSigner
-
- getInvoker() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Get the invoker for STS calls.
- getInvoker() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getInvoker() - Method in class org.apache.hadoop.fs.s3a.S3AOpContext
-
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractOperationAuditor
-
Get the IOStatistics Store.
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.impl.GetContentSummaryOperation
-
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Get the instrumentation's IOStatistics.
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Gets the internal IO statistics.
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Gets the internal IO statistics.
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the instrumentation's IOStatistics.
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Get the instance IO Statistics.
- getIOStatistics() - Method in interface org.apache.hadoop.fs.s3a.statistics.CommitterStatistics
-
Return the writeable IOStatisticsStore.
- getIOStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource
-
- getIOStatisticsAggregator() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Return the IOStatistics aggregator.
- getIOStatisticsContext() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
IOStatistics context of the created thread.
- getIoStatisticsStore() - Method in class org.apache.hadoop.fs.s3a.audit.OperationAuditorOptions
-
- getJobAttemptFileSystem(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the filesystem for the job attempt.
- getJobAttemptPath(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Compute the path where the output of a given job attempt will be placed.
- getJobAttemptPath(int) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Compute the path where the output of a given job attempt will be placed.
- getJobAttemptPath(int) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Compute the path where the output of a given job attempt will be placed.
- getJobAttemptPath(JobContext, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Compute the path where the output of a given job attempt will be placed.
- getJobAttemptPath(int) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- getJobAttemptPath(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
For a job attempt path, the staging committer returns that of the
wrapped committer.
- getJobContext() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the job/task context this committer was instantiated with.
- getJobContext() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Job Context.
- getJobId() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- getJobId() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getJobId() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getJobId() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Get the job ID.
- getJobIdSource() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getJobPath() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Compute the path under which all job attempts will be placed.
- getJobPath() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Compute the path under which all job attempts will be placed.
- getJobPath() - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Compute the path under which all job attempts will be placed.
- getKey() - Method in class org.apache.hadoop.fs.s3a.audit.AWSRequestAnalyzer.RequestInfo
-
- getKey() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker.Marker
-
- getKey() - Method in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
- getKey() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getKind() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Get the kind of the tokens managed here.
- getKind() - Method in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
- getLastDirChecked() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
- getLastHeader() - Method in class org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor
-
Get the last header used.
- getLeafMarkers() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
Get the map of leaf markers.
- getLen() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getLength() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Destination file size.
- getLength() - Method in class org.apache.hadoop.fs.s3a.UploadInfo
-
- getListFilesAssumingDir(Path, boolean, Listing.FileStatusAcceptor, AuditSpan) - Method in class org.apache.hadoop.fs.s3a.Listing
-
List files under a path assuming the path to be a directory.
- getListing() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get current listing instance.
- getLocalTaskAttemptTempDir(Configuration, String, TaskAttemptID) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Get the task attempt temporary directory in the local filesystem.
- getLocatedFileStatusIteratorForDir(Path, PathFilter, AuditSpan) - Method in class org.apache.hadoop.fs.s3a.Listing
-
Generate list located status for a directory.
- getLogNoVersionSupport() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getMagicJobAttemptPath(String, int, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Compute the "magic" path for a job attempt.
- getMagicJobAttemptsPath(Path, String) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Get the location of magic job attempts.
- getMagicJobPath(String, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Compute the "magic" path for a job.
- getMagicTaskAttemptPath(TaskAttemptContext, String, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Compute the path where the output of a task attempt is stored until
that task is committed.
- getMagicTaskAttemptsPath(String, Path, int) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Compute the path where the output of magic task attempts are stored.
- getMapping() - Method in enum org.apache.hadoop.fs.s3a.Tristate
-
Get the boolean mapping, if present.
- getMarkerPolicy() - Method in interface org.apache.hadoop.fs.s3a.impl.DirectoryPolicy
-
Get the marker policy.
- getMarkerPolicy() - Method in class org.apache.hadoop.fs.s3a.impl.DirectoryPolicyImpl
-
- getMarkersFound() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
- getMarshalledCredentials() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Get the marshalled credentials.
- getMaxKeys() - Method in interface org.apache.hadoop.fs.s3a.impl.ListingOperationCallbacks
-
Get the maximum key count.
- getMaxKeys() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.ListingOperationCallbacksImpl
-
- getMaxReadSizeForVectorReads() - Method in class org.apache.hadoop.fs.s3a.VectoredIOContext
-
- getMessage() - Method in exception org.apache.hadoop.fs.s3a.AWSClientIOException
-
- getMethod() - Method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
- getMethod(String) - Static method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
Get the encryption mechanism from the value provided.
- getMetrics() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getMetrics(MetricsCollector, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
- getMetrics() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
- getMetricSourceName() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
if registered with the metrics, return the
name of the source.
- getMinimumOperationDuration() - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Get the current minimum operation duration.
- getMinimumPartSize() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
Get the minimum part size for transfer parts.
- getMinSeekForVectorReads() - Method in class org.apache.hadoop.fs.s3a.VectoredIOContext
-
- getMode() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getModificationTime() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Get the modification time of the file/directory.
- getMultipartSizeProperty(Configuration, String, long) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
- getMultiPartThreshold() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the threshold for multipart files.
- getMultiPartThreshold() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
Get the threshold for multipart operations.
- getMultipartUploadCommitsDirectory(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Build a qualified temporary path for the multipart upload commit
information in the cluster filesystem.
- getName() - Method in class org.apache.hadoop.fs.s3a.ArnResource
-
Resource name.
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the name of this committer.
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter
-
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
- getName() - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- getName() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getName() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- getName() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Return sub-command name.
- getName() - Method in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
- getName() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
- getNextReadPos() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getObject(GetObjectRequest) - Method in interface org.apache.hadoop.fs.s3a.S3AInputStream.InputStreamCallbacks
-
Execute the request.
- getObjectMetadata(String) - Method in interface org.apache.hadoop.fs.s3a.impl.HeaderProcessing.HeaderProcessingCallbacks
-
Retrieve the object metadata.
- getObjectMetadata(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getObjectMetadata(String, ChangeTracker, Invoker, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Request object metadata; increments counters in the process.
- getObjectMetadata(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.HeaderProcessingCallbacksImpl
-
- getObjectMetadata(Path) - Method in interface org.apache.hadoop.fs.s3a.S3AInternals
-
Low-level call to get at the object metadata.
- getObjectsFound() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
How many objects were found.
- getOffsetStr(long) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getOpenOperations() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getOperationId() - Method in class org.apache.hadoop.fs.s3a.impl.ActiveOperationContext
-
- getOperationName() - Method in interface org.apache.hadoop.fs.s3a.audit.AWSAuditEventCallbacks
-
Get the name of the operation.
- getOperationName() - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractAuditSpanImpl
-
- getOperationName() - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- getOperationName() - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- getOptionName() - Method in enum org.apache.hadoop.fs.s3a.impl.DirectoryPolicy.MarkerPolicy
-
Get the option name.
- getOptions() - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractOperationAuditor
-
Get the options this auditor was initialized with.
- getOrCreateAsyncClient() - Method in interface org.apache.hadoop.fs.s3a.impl.ClientManager
-
Get the Async S3Client,raising a failure to create as an IOException.
- getOrCreateAsyncClient() - Method in class org.apache.hadoop.fs.s3a.impl.ClientManagerImpl
-
- getOrCreateAsyncClient() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- getOrCreateAsyncS3ClientUnchecked() - Method in interface org.apache.hadoop.fs.s3a.impl.ClientManager
-
Get the AsyncS3Client, raising a failure to create as an UncheckedIOException.
- getOrCreateAsyncS3ClientUnchecked() - Method in class org.apache.hadoop.fs.s3a.impl.ClientManagerImpl
-
Get the AsyncS3Client, raising a failure to create as an UncheckedIOException.
- getOrCreateAsyncS3ClientUnchecked() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- getOrCreateS3Client() - Method in interface org.apache.hadoop.fs.s3a.impl.ClientManager
-
Get the S3Client, raising a failure to create as an IOException.
- getOrCreateS3Client() - Method in class org.apache.hadoop.fs.s3a.impl.ClientManagerImpl
-
- getOrCreateS3Client() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- getOrCreateS3ClientUnchecked() - Method in interface org.apache.hadoop.fs.s3a.impl.ClientManager
-
Get the S3Client, raising a failure to create as an UncheckedIOException.
- getOrCreateS3ClientUnchecked() - Method in class org.apache.hadoop.fs.s3a.impl.ClientManagerImpl
-
Get the S3Client, raising a failure to create as an UncheckedIOException.
- getOrCreateS3ClientUnchecked() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- getOrCreateTransferManager() - Method in interface org.apache.hadoop.fs.s3a.impl.ClientManager
-
Get the transfer manager, creating it and any dependencies if needed.
- getOrCreateTransferManager() - Method in class org.apache.hadoop.fs.s3a.impl.ClientManagerImpl
-
- getOrCreateTransferManager() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- getOrigin() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- getOriginalDestKey() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- getOuterSubmitter() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Return a submitter.
- getOutputPath() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Final path of output, in the destination FS.
- getOwner() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Get the owner of this Service.
- getOwner() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the owner of the filesystem.
- getOwner() - Method in class org.apache.hadoop.fs.s3a.S3ADataBlocks.BlockFactory
-
Owner.
- getOwner() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the owner of this FS: who created it?
- getOwnerAccountId() - Method in class org.apache.hadoop.fs.s3a.ArnResource
-
Return owner's account id.
- getOwnerText() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Return the name of the owner to be used in tokens.
- getParent(String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Get the parent path of a string path: everything up to but excluding
the last "/" in the path.
- getParentAuthChain() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
The auth chain for the parent options.
- getPartCount() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Get the number of etags.
- getPartition(String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Returns the partition of a relative file path, or null if the path is a
file name with no relative directory.
- getPartitions(Path, List<? extends FileStatus>) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Get the set of partitions from the list of files being staged.
- getPartitionSize() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the partition size for multipart operations.
- getPassword() - Method in class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
- getPath() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- getPath() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker.Marker
-
- getPath() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObject
-
Gets the path of this file.
- getPath(S3ObjectAttributes) - Static method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObject
-
Gets the path corresponding to the given s3Attributes.
- getPath() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get the path of this read.
- getPath() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getPathToBytesWritten() - Static method in class org.apache.hadoop.fs.s3a.commit.magic.InMemoryMagicCommitTracker
-
- getPathUri() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
Get the full s3 path.
- getPendingPartKey() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- getPendingSetSerializer() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Get a serializer for .pendingset files.
- getPerformanceFlags() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the performance flags.
- getPerformanceFlags() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- getPerformanceFlags() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the performance flags.
- getPolicy(Configuration) - Static method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Reads the change detection policy from Configuration.
- getPolicy(String, S3AInputPolicy) - Static method in enum org.apache.hadoop.fs.s3a.S3AInputPolicy
-
Choose an access policy.
- getPolicyProvider() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
- getPolicySetCount() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getPos() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Gets the current position.
- getPos() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Gets the current position.
- getPos() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- getPrefetchBlockCount() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Gets the size of prefetch queue (in number of blocks).
- getPrefetchBlockSize() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Gets the size in bytes of a single prefetch block.
- getProviders() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Returns the underlying list of providers.
- getPurgeSummary() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool.ScanResult
-
- getPutRequestLength(PutObjectRequest) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get the length of the PUT, verifying that the length is known.
- getReadahead() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Get the current readahead value.
- getReadahead() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get the readahead for this operation.
- getReadAheadRange() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
- getReader() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ACachingBlockManager
-
- getReader() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getReadExceptions() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getReadFullyOperations() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getReadInvoker() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObject
-
Gets an instance of Invoker for interacting with S3 API.
- getReadInvoker() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get invoker to use for read operations.
- getReadOperations() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getReadsIncomplete() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getReason() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- getRefCount() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Get the current reference count.
- getRegion() - Method in class org.apache.hadoop.fs.s3a.ArnResource
-
Resource region.
- getRegion() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
Get the region.
- getRegistry() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Get the metrics registry.
- getRelativePath(Path, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Using URI#relativize(), build the relative path from the
base path to the full path.
- getRemaining() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- getRequestFactory() - Method in interface org.apache.hadoop.fs.s3a.impl.ContextAccessors
-
Get the request factory.
- getRequestFactory() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- getRequestFactory() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the request factory.
- getRequestFactory() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the request factory which uses this store's audit span.
- getRequestFactory() - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
- getRequestFactory() - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Get the request factory which uses this store's audit span.
- getRetryCallback() - Method in class org.apache.hadoop.fs.s3a.Invoker
-
- getRetryPolicy() - Method in class org.apache.hadoop.fs.s3a.Invoker
-
- getRevisionId(HeadObjectResponse, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Pulls the attribute this policy uses to detect change out of the S3 object
metadata.
- getRevisionId(GetObjectResponse, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getRevisionId(S3ObjectAttributes) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getRevisionId(CopyObjectResponse) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getRevisionId() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- getRole() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Used in logging and reporting to help disentangle messages.
- getRoleARN() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- getS3AFileSystem(Path, Configuration, boolean) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Get the S3A FS of a path.
- getS3AInternals() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the S3AInternals.
- getS3AStatisticsContext() - Method in class org.apache.hadoop.fs.s3a.impl.ActiveOperationContext
-
- getS3AStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Access the input stream statistics.
- getS3AStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Access the input stream statistics.
- getS3AStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Access the input stream statistics.
- getS3Client() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getS3EncryptionAlgorithm() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the encryption algorithm of this connector.
- getS3EncryptionKey(String, Configuration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get any S3 encryption key, without propagating exceptions from
JCEKs files.
- getS3EncryptionKey(String, Configuration, boolean) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get any SSE/CSE key from a configuration/credential provider.
- getS3File() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getS3ObjectAttributes() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- getS3Objects() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- getSaved() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
When was the upload saved?
- getScanCount() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
- getScheme() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Return the protocol scheme for the FileSystem.
- getScheme() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
Return the protocol scheme for the FileSystem.
- getSdkStream() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- getSecretKey() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- getSecretManagerPasssword() - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Get the password to use in secret managers.
- getSeekOperations() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getServerSideEncryptionAlgorithm() - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Get the encryption algorithm of this endpoint.
- getServerSideEncryptionAlgorithm() - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
Get the encryption algorithm of this endpoint.
- getServerSideEncryptionAlgorithm() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getServerSideEncryptionKey() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getService() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get the service identifier of the owning FS.
- getServiceName() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADtFetcher
-
Returns the service name for HDFS, which is also a valid URL prefix.
- getSessionToken() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- getSinglePendingFileSerializer() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Get a serializer for .pending files.
- getSize() - Method in class org.apache.hadoop.fs.s3a.audit.AWSRequestAnalyzer.RequestInfo
-
- getSize() - Method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders.BaseContentProvider
-
Size as set by constructor parameter.
- getSource() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- getSource() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- getSourceFiles() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getSourceFS() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- getSpanId() - Method in interface org.apache.hadoop.fs.s3a.audit.AWSAuditEventCallbacks
-
Return a span ID which must be unique for all spans within
everywhere.
- getSpanId() - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractAuditSpanImpl
-
- getSpanId() - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
The Span ID in the audit manager is the ID of the auditor,
which can be used in the filesystem toString() method
to assist in correlating client logs with S3 logs.
- getSpanId() - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- getSplitEnd() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
- getSplitStart() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
- getSSEAwsKMSKey(EncryptionSecrets) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecretOperations
-
Gets the SSE-KMS key if present, else let S3 use AWS managed key.
- getSSECustomerKey(EncryptionSecrets) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecretOperations
-
Gets the SSE-C client side key if present.
- getStage() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getStagingUploadsParentDirectory(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
- getStartTime() - Method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders.BaseContentProvider
-
When did this upload start?
- getState() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
- getStatistics() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObject
-
Gets an instance of S3AInputStreamStatistics used for reporting access metrics.
- getStatisticsContext() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- getStatisticsContext() - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
- getStats() - Method in class org.apache.hadoop.fs.s3a.S3AOpContext
-
- getStatus() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker.Marker
-
- getStatus() - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
- getStorageClass() - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Get the object storage class, return null if none.
- getStorageClass() - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
Get the object storage class, return null if none.
- getStorageStatistics() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Get the storage statistics of this filesystem.
- getStorageStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the storage statistics of this filesystem.
- getStore() - Method in interface org.apache.hadoop.fs.s3a.S3AInternals
-
Get the store for low-level operations.
- getStoreContext() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
- getStoreContext() - Method in class org.apache.hadoop.fs.s3a.impl.AbstractStoreOperation
-
Get the store context.
- getStoreContext() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
- getStoreContext() - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
- getStreamCreationCount() - Method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders.BaseContentProvider
-
How many times has a stream been created?
- getStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- getSuccess() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Get the success flag.
- getSurplusMarkers() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
Get the map of surplus markers.
- getSymbol() - Method in enum org.apache.hadoop.fs.s3a.Statistic
-
- getTaskAttemptFilesystem(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the task attempt path filesystem.
- getTaskAttemptIdToMpuMetadata() - Static method in class org.apache.hadoop.fs.s3a.commit.magic.InMemoryMagicCommitTracker
-
- getTaskAttemptIdToPath() - Static method in class org.apache.hadoop.fs.s3a.commit.magic.InMemoryMagicCommitTracker
-
- getTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Compute the path where the output of a task attempt is stored until
that task is committed.
- getTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Compute the path where the output of a task attempt is stored until
that task is committed.
- getTaskAttemptPath(TaskAttemptContext, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Compute the path where the output of a task attempt is stored until
that task is committed.
- getTaskId() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getTaskOutput(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Lists the output of a task under the task attempt path.
- getTempJobAttemptPath(String, Path, int) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Compute a path for temporary data associated with a job.
- getTempTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get a temporary directory for data.
- getTempTaskAttemptPath(TaskAttemptContext, String, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitUtilsWithMR
-
Compute the path where the output of a given task attempt will be placed.
- getTempTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Get a temporary directory for data.
- getTempTaskAttemptPath(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- getText() - Method in enum org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.CredentialTypeRequired
-
- getText() - Method in enum org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.JobUUIDSource
-
Source for messages.
- getText() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Arbitrary notes.
- getThisBuilder() - Method in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder
-
- getThisBuilder() - Method in class org.apache.hadoop.fs.s3a.impl.S3AMultipartUploaderBuilder
-
- getThrottleProbability() - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- getThrown() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- getTimestamp() - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractAuditSpanImpl
-
- getTimestamp() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- getTokenIdentifier() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Token identifier bound to.
- getTokenIssuingPolicy() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Predicate: will this binding issue a DT?
That is: should the filesystem declare that it is issuing
delegation tokens? If true
- getTokenIssuingPolicy() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
- getTokenKind() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Get the kind of the issued tokens.
- getTotalBytesRead() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
The total number of bytes read, including
all read and discarded when closing streams
or skipped during seek calls.
- getTotalMarkerCount() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool.ScanResult
-
- getTracker() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool.ScanResult
-
- getTrackerStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- getTransferManagerExecutor() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
Get the executor that the transfer manager will use to execute background tasks.
- getTrimmedStringCollectionSplitByEquals(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get the equal op (=) delimited key-value pairs of the name property as
a collection of pair of Strings, trimmed of the leading and trailing whitespace
after delimiting the name by comma and new line separator.
- getType() - Method in enum org.apache.hadoop.fs.s3a.Statistic
-
What type is this statistic?
- getUnbondedSpan() - Method in class org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor
-
- getUnbondedSpan() - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditor
-
- getUnbondedSpan() - Method in interface org.apache.hadoop.fs.s3a.audit.OperationAuditor
-
Get the unbonded span to use after deactivating an active
span.
- getUploadId() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getUploadsAborted() - Method in class org.apache.hadoop.fs.s3a.impl.DeleteOperation
-
Get the count of uploads aborted.
- getUploadsAborted() - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
Get the count of uploads aborted.
- getUri() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractAWSCredentialProvider
-
Get the binding URI: may be null.
- getUri() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- getUri() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getUri() - Method in class org.apache.hadoop.fs.s3a.impl.ClientManagerImpl
-
Get the URI of the filesystem.
- getUri() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- getUri() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Returns a URI whose scheme and authority identify this FileSystem.
- getUri() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- getUriDefaultPort() - Method in class org.apache.hadoop.fs.s3a.S3A
-
- getUsage() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- getUsage() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- getUsage() - Method in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
- getUsage() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
- getUser() - Method in class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
- getUserAgentField() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Return a string for use in building up the User-Agent field, so
get into the S3 access logs.
- getUserAgentField() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Return a string for use in building up the User-Agent field, so
get into the S3 access logs.
- getUserAgentField() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
UA field contains the UUID of the token if present.
- getUserAgentSuffix() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
- getUsername() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- getUsername() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the username of the FS.
- getUuid() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Get the UUID of this token identifier.
- getUUID() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
The Job UUID, as passed in or generated.
- getUUIDSource() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Source of the UUID.
- getV1() - Method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
- getV1() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- getV2() - Method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
- getV2() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- getVectoredIOContext() - Method in class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Get Vectored IO context for this this read op.
- getVerb() - Method in class org.apache.hadoop.fs.s3a.audit.AWSRequestAnalyzer.RequestInfo
-
- getVersion() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- getVersion() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- getVersionId() - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker.Marker
-
Get the version ID of the status object; may be null.
- getVersionId() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
- getVersionId() - Method in class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- getVersionId() - Method in class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
- getVersionMismatches() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- getVersionMismatches() - Method in interface org.apache.hadoop.fs.s3a.statistics.ChangeTrackerStatistics
-
How many version mismatches have occurred.
- getVersionMismatches() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.CountingChangeTracker
-
- getVersionMismatches() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
- getWorkingDirectory() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get the current working directory for the given file system.
- getWorkingDirectory() - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- getWorkPath() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
This is the critical method for FileOutputFormat; it declares
the path for work.
- getWrappedStream() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Get the wrapped stream.
- getWriteOperationHelper() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create a Write Operation Helper with the current active span.
- getWriter() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- getXAttr(Path, String) - Method in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
Get an XAttr name and value for a file or directory.
- getXAttr(Path, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getXAttrs(Path) - Method in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
See FileSystem.getXAttrs(path.
- getXAttrs(Path, List<String>) - Method in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
See FileSystem.getXAttrs(path, names.
- getXAttrs(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- getXAttrs(Path, List<String>) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- globStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- globStatus(Path, PathFilter) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- GUARDED_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- IAMInstanceCredentialsProvider - Class in org.apache.hadoop.fs.s3a.auth
-
This is an IAM credential provider which wraps
an ContainerCredentialsProvider
to provide credentials when the S3A connector is instantiated on AWS EC2
or the AWS container services.
- IAMInstanceCredentialsProvider() - Constructor for class org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
-
Constructor.
- ignoreIOExceptions(Logger, String, String, CallableRaisingIOE<T>) - Static method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute an operation and ignore all raised IOExceptions; log at INFO;
full stack only at DEBUG.
- ignoreIOExceptions(Logger, String, String, InvocationRaisingIOE) - Static method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute an operation and ignore all raised IOExceptions; log at INFO;
full stack only at DEBUG.
- incCounter(String) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource
-
Increment a named counter by 1.
- incCounter(String, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource
-
DefaultS3ClientFactoryDefaultS3ClientFactory
Increment a named counter by 1.
- incGauge(String, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource
-
- incGauge(String) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource
-
- incrementCounter(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Increments a mutable counter and the matching
instance IOStatistics counter.
- incrementCounter(Statistic, AtomicLong) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Increments a mutable counter and the matching
instance IOStatistics counter with the value of
the atomic long.
- incrementCounter(Statistic, long) - Method in interface org.apache.hadoop.fs.s3a.statistics.CountersAndGauges
-
Increment a specific counter.
- incrementCounter(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
Increment a specific counter.
- incrementCounter(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- incrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Increment a gauge by a specific value.
- incrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Increment a gauge by a specific value.
- incrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment a gauge by a specific value.
- incrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Increment a specific gauge.
- incrementGauge(Statistic, long) - Method in interface org.apache.hadoop.fs.s3a.statistics.CountersAndGauges
-
Increment a specific gauge.
- incrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
Increment a specific gauge.
- incrementGauge(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- incrementPutCompletedStatistics(boolean, long) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
At the end of a put/multipart upload operation, update the
relevant counters and gauges.
- incrementPutCompletedStatistics(boolean, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
At the end of a put/multipart upload operation, update the
relevant counters and gauges.
- incrementPutCompletedStatistics(boolean, long) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
At the end of a put/multipart upload operation, update the
relevant counters and gauges.
- incrementPutProgressStatistics(String, long) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Callback for use in progress callbacks from put/multipart upload events.
- incrementPutProgressStatistics(String, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Callback for use in progress callbacks from put/multipart upload events.
- incrementPutProgressStatistics(String, long) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Callback for use in progress callbacks from put/multipart upload events.
- incrementPutStartStatistics(long) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
At the start of a put/multipart upload operation, update the
relevant counters.
- incrementPutStartStatistics(long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
At the start of a put/multipart upload operation, update the
relevant counters.
- incrementPutStartStatistics(long) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
At the start of a put/multipart upload operation, update the
relevant counters.
- incrementReadOperations() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Increment read operations.
- incrementReadOperations() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment read operations.
- incrementReadOperations() - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Increment read operations.
- incrementStatistic(Statistic) - Method in class org.apache.hadoop.fs.s3a.impl.BulkDeleteRetryHandler
-
Increment a statistic by 1.
- incrementStatistic(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.BulkDeleteRetryHandler
-
Increment a statistic by a specific value.
- incrementStatistic(Statistic) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Increment a statistic by 1.
- incrementStatistic(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Increment a statistic by a specific value.
- incrementStatistic(Statistic) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Increment a statistic by 1.
- incrementStatistic(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Increment a statistic by a specific value.
- incrementStatistic(Statistic) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment a statistic by 1.
- incrementStatistic(Statistic, long) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment a statistic by a specific value.
- incrementWriteOperations() - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Increment the write operation counter.
- incrementWriteOperations() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Increment the write operation counter.
- incrementWriteOperations() - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Increment the write operation counter.
- incrementWriteOperations() - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
- incrementWriteOperations() - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Increment the write operation counter
of the filesystem.
- init(OperationAuditorOptions) - Method in class org.apache.hadoop.fs.s3a.audit.impl.AbstractOperationAuditor
-
Sets the IOStats and then calls init().
- init(OperationAuditorOptions) - Method in interface org.apache.hadoop.fs.s3a.audit.OperationAuditor
-
Initialize.
- init() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
- initCustomSigners() - Method in class org.apache.hadoop.fs.s3a.auth.SignerManager
-
Initialize custom signers and register them with the AWS SDK.
- initFileOutputCommitterOptions(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Init the context config with everything needed for the file output
committer.
- initialize() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
Initialize the tracker.
- initialize() - Method in class org.apache.hadoop.fs.s3a.commit.PutTracker
-
Startup event.
- initialize(URI, Configuration) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Called after a new FileSystem instance is constructed.
- initialize(URI, Configuration) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
Always fail to initialize.
- INITIALIZE_SPAN - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Span name used during initialization.
- initializeAuditService() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Initialize and launch the audit manager and service.
- initializeClass() - Static method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
This is purely for testing, as it force initializes all static
initializers.
- Initializer() - Constructor for class org.apache.hadoop.fs.s3a.auth.CustomSdkSigner.Initializer
-
- initializeStatisticsBinding() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Initialize the statistics binding.
- initiateCopy(S3ALocatedFileStatus, String, String, Path) - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
Initiate a copy operation in the executor.
- initiateGetRequest() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
Initiate a GET request.
- initiateInnerStreamClose(boolean) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
Initiate a stream close/abort.
- initiateJobOperation(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Start the final job commit/abort commit operations.
- initiateMultiPartUpload(String, PutObjectOptions) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Start the multipart upload process.
- initiateMultiPartUpload(String, PutObjectOptions) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Start the multipart upload process.
- initiateTaskOperation(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Start a ask commit/abort commit operations.
- initOutput(Path) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Init the output filesystem and path.
- initS3AFileSystem(String) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Create and initialize a new S3A FileSystem instance.
- InMemoryMagicCommitTracker - Class in org.apache.hadoop.fs.s3a.commit.magic
-
InMemoryMagicCommitTracker stores the commit data in memory.
- InMemoryMagicCommitTracker(Path, String, String, String, String, WriteOperationHelper, PutTrackerStatistics) - Constructor for class org.apache.hadoop.fs.s3a.commit.magic.InMemoryMagicCommitTracker
-
- INPUT_FADV_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- INPUT_FADV_NORMAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
General input.
- INPUT_FADV_RANDOM - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- INPUT_FADV_SEQUENTIAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- INPUT_FADVISE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Which input strategy to use for buffering, seeking and similar when
reading data.
- inputPolicySet(int) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
The input policy has been switched.
- INSTANCE - Static variable in class org.apache.hadoop.fs.s3a.audit.impl.NoopSpan
-
Static public instance.
- instantiated() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext.EmptyMultipartUploaderStatistics
-
- instantiated() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.S3AMultipartUploaderStatisticsImpl
-
- instantiated() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AMultipartUploaderStatistics
-
- INSTANTIATION_EXCEPTION - Static variable in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
- instantiationException(URI, String, String, Throwable) - Static method in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
General instantiation failure.
- InstantiationIOException - Exception in org.apache.hadoop.fs.s3a.impl
-
An instantiation exception raised during reflection-based creation
of classes.
- InstantiationIOException(InstantiationIOException.Kind, URI, String, String, String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
- InstantiationIOException.Kind - Enum in org.apache.hadoop.fs.s3a.impl
-
An (extensible) enum of kinds of instantiation failure.
- InternalCommitterConstants - Class in org.apache.hadoop.fs.s3a.commit
-
These are internal constants not intended for public use.
- InternalConstants - Class in org.apache.hadoop.fs.s3a.impl
-
Internal constants private only to the S3A codebase.
- intOption(Configuration, String, int, int) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a integer option >= the minimum allowed value.
- INVALID_CREDENTIALS - Static variable in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Error text on invalid non-empty credentials: "Invalid AWS credentials".
- invalidArgs(String, Object...) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Build the exception to raise on invalid arguments.
- Invoker - Class in org.apache.hadoop.fs.s3a
-
Class to provide lambda expression invocation of AWS operations.
- Invoker(RetryPolicy, Invoker.Retried) - Constructor for class org.apache.hadoop.fs.s3a.Invoker
-
Instantiate.
- Invoker.Retried - Interface in org.apache.hadoop.fs.s3a
-
Callback for retry and notification operations.
- IS_MARKER_AWARE - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- isAbstract(URI, String, String) - Static method in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
Class is abstract.
- isAwsEndpoint(String) - Static method in class org.apache.hadoop.fs.s3a.impl.NetworkBinding
-
Is this an AWS endpoint? looks at end of FQDN.
- isAwsV1SdkAvailable() - Static method in class org.apache.hadoop.fs.s3a.adapter.AwsV1BindingSupport
-
Is the AWS v1 SDK available?
- isBoolean() - Method in enum org.apache.hadoop.fs.s3a.Tristate
-
Does this value map to a boolean.
- isBoundToDT() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Predicate: is there a bound DT?
- isChecksumValidationEnabled() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
- isClosed() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Get the closed flag.
- isClosed() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
- isClosed() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- isCollectIOStatistics() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Collecting thread level IO statistics?
- isContainerCredentialsProvider() - Method in class org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
-
Is this a container credentials provider?
- isCSEEnabled() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
return if the store context have client side encryption enabled.
- isCSEEnabled() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
a method to know if Client side encryption is enabled or not.
- isDirectory(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Optimized probe for a path referencing a dir.
- isEmpty() - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Is this empty: does it contain any credentials at all?
This test returns true if either the access key or secret key is empty.
- isEmpty() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- isEmptyDirectory() - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
- isExpressCreateSession() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
Should s3express createSession be called?
- isFile(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Optimized probe for a path referencing a file.
- isFipsEnabled() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
Get the FIPS flag.
- isInitialized() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Has an attempt to initialize the credentials been attempted?
- isKeepMarkers() - Method in class org.apache.hadoop.fs.s3a.impl.PutObjectOptions
-
Get the marker retention flag.
- isMagicCommitEnabled() - Method in class org.apache.hadoop.fs.s3a.commit.MagicCommitIntegration
-
Is magic commit enabled?
- isMagicCommitEnabled() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Is magic commit enabled?
- isMagicCommitPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.MagicCommitIntegration
-
Predicate: is a path a magic commit path?
- isMagicCommitPath(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Predicate: is a path a magic commit path?
True if magic commit is enabled and the path qualifies as special,
and is not a a .pending or .pendingset file,
- isMagicPath(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Is a path in the magic tree?
- isMessageTranslatableToEOF(SdkException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Cue that an AWS exception is likely to be an EOF Exception based
on the message coming back from the client.
- isMultiObjectDeleteEnabled() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- isMultipartCopy() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
Get the multipart flag.
- isMultipartCopyEnabled() - Method in interface org.apache.hadoop.fs.s3a.S3AInternals
-
Is multipart copy enabled?
- isMultipartUploadEnabled() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- isMutating() - Method in class org.apache.hadoop.fs.s3a.audit.AWSRequestAnalyzer.RequestInfo
-
- isNotInstanceOf(URI, String, String, String) - Static method in exception org.apache.hadoop.fs.s3a.impl.InstantiationIOException
-
Class does not implement the desired interface.
- isObjectNotFound(AwsServiceException) - Static method in class org.apache.hadoop.fs.s3a.impl.ErrorTranslation
-
Does this exception indicate that a reference to an object
returned a 404.
- isObjectStreamOpen() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Is the inner object stream open?
- isPathStyleAccess() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
- isPerformance() - Method in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder.CreateFileOptions
-
- isRecursive() - Method in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder.CreateFileOptions
-
- isRequesterPays() - Method in class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
- isRequestMultipartIO(Object) - Static method in class org.apache.hadoop.fs.s3a.audit.AWSRequestAnalyzer
-
Predicate which returns true if the request is part of the
multipart upload API -and which therefore must be rejected
if multipart upload is disabled.
- isRequestNotAlwaysInSpan(Object) - Static method in class org.apache.hadoop.fs.s3a.audit.AWSRequestAnalyzer
-
Predicate which returns true if the request is of a kind which
could be outside a span because of how the AWS SDK generates them.
- isRequireVersion() - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
- isS3ExpressStore(String, String) - Static method in class org.apache.hadoop.fs.s3a.impl.S3ExpressStorage
-
Is a bucket an S3Express store?
This may get confused against third party stores, so takes the endpoint
and only supports aws endpoints round the world.
- isServerSide() - Method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
Flag to indicate this is a server-side encryption option.
- isSignerRegistered(String) - Static method in class org.apache.hadoop.fs.s3a.auth.SignerFactory
-
Check if the signer has already been registered.
- isThrottleException(Exception) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Is the exception an instance of a throttling exception.
- isTokenRequired() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADtFetcher
-
- isTrackMagicCommitsInMemoryEnabled(Configuration) - Static method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTrackerUtils
-
Is tracking of magic commit data in-memory enabled.
- isTruncated() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
- isUnderMagicPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.MagicCommitIntegration
-
Is this path in/under a magic path...regardless of file type.
- isUnknownBucket(AwsServiceException) - Static method in class org.apache.hadoop.fs.s3a.impl.ErrorTranslation
-
Does this exception indicate that the AWS Bucket was unknown.
- isUseListV1() - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- isV1() - Method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
Is this a v1 API request or v2?
- isV1() - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
Is this a v1 API result or v2?
- isValid(MarshalledCredentials.CredentialTypeRequired) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Is this a valid set of credentials tokens?
- isVerbose() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
- iterator() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Iterate over the etags.
- iteratorToStatuses(RemoteIterator<S3AFileStatus>) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
- LAST_MODIFIED - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- lastElement(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Last element of a (non-empty) list.
- LIST - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- LIST_ONLY - Static variable in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
List operation only.
- LIST_VERSION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 API level parameters.
- listAndFilter(FileSystem, Path, boolean, PathFilter) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
List located files and filter them as a classic listFiles(path, filter)
would do.
- listAWSPolicyRules(Set<AWSPolicyProvider.AccessLevel>) - Method in interface org.apache.hadoop.fs.s3a.auth.delegation.AWSPolicyProvider
-
Get the AWS policy statements required for accessing this service.
- listAWSPolicyRules(Set<AWSPolicyProvider.AccessLevel>) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Build the AWS policy for restricted access to the resources needed
by this bucket.
- listFiles(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
.
- listFilesAndDirectoryMarkers(Path, S3AFileStatus, boolean) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Recursive list of files and directory markers.
- listFilesAndEmptyDirectories(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Recursive List of files and empty directories.
- listFilesIterator(Path, boolean) - Method in interface org.apache.hadoop.fs.s3a.impl.GetContentSummaryOperation.GetContentSummaryCallbacks
-
List all entries under a path.
- listFilesIterator(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.GetContentSummaryCallbacksImpl
-
- Listing - Class in org.apache.hadoop.fs.s3a
-
Place for the S3A listing classes; keeps all the small classes under control.
- Listing(ListingOperationCallbacks, StoreContext) - Constructor for class org.apache.hadoop.fs.s3a.Listing
-
- Listing.AcceptAllButSelfAndS3nDirs - Class in org.apache.hadoop.fs.s3a
-
Accept all entries except the base path and those which map to S3N
pseudo directory markers.
- ListingOperationCallbacks - Interface in org.apache.hadoop.fs.s3a.impl
-
These are all the callbacks which
Listing operations
need, derived from the actual appropriate S3AFileSystem
methods.
- ListingOperationCallbacksImpl() - Constructor for class org.apache.hadoop.fs.s3a.S3AFileSystem.ListingOperationCallbacksImpl
-
- listLocalStatusIterator(Path) - Method in interface org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.CopyFromLocalOperationCallbacks
-
List all entries (files AND directories) for a path.
- listLocalStatusIterator(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.CopyFromLocalCallbacksImpl
-
- listLocatedStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override superclass so as to add statistic collection.
- listLocatedStatus(Path, PathFilter) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
.
- listMultipartUploads(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Listing all multipart uploads; limited to the first few hundred.
- listMultipartUploads(String) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
- listMultipartUploads(String) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
List in-progress multipart uploads under a path: limited to the first
few hundred.
- listObjects(Path, String) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Create an iterator over objects in S3.
- listObjects(S3ListRequest, DurationTrackerFactory) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Initiate a listObjects operation, incrementing metrics
in the process.
- listObjects(Path, String) - Method in interface org.apache.hadoop.fs.s3a.tools.MarkerToolOperations
-
Create an iterator over objects in S3.
- listObjects(Path, String) - Method in class org.apache.hadoop.fs.s3a.tools.MarkerToolOperationsImpl
-
- listObjectsAsync(S3ListRequest, DurationTrackerFactory, AuditSpan) - Method in interface org.apache.hadoop.fs.s3a.impl.ListingOperationCallbacks
-
Initiate a listObjectsAsync operation, incrementing metrics
in the process.
- listObjectsAsync(S3ListRequest, DurationTrackerFactory, AuditSpan) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.ListingOperationCallbacksImpl
-
- listPendingUploads(CommitContext, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the list of pending uploads for this job attempt.
- listPendingUploadsToAbort(CommitContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the list of pending uploads for this job attempt, swallowing
exceptions.
- listPendingUploadsToCommit(CommitContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Get the list of pending uploads for this job attempt.
- listPendingUploadsToCommit(CommitContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Get the list of pending uploads for this job attempt, by listing
all .pendingset files in the job attempt directory.
- listPendingUploadsToCommit(CommitContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Get the list of pending uploads for this job attempt.
- listPendingUploadsUnderPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
List all pending uploads to the destination FS under a path.
- listProviderNames() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
List all the providers' names.
- listStatus(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
List the statuses of the files/directories in the given path if the path is
a directory.
- listStatus(Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- listStatusIterator(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Override subclass such that we benefit for async listing done
in S3AFileSystem.
- listUploads(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
List any pending multipart uploads whose keys begin with prefix, using
an iterator that can handle an unlimited number of entries.
- listUploadsUnderPrefix(StoreContext, String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
List any pending multipart uploads whose keys begin with prefix, using
an iterator that can handle an unlimited number of entries.
- listXAttrs(Path) - Method in class org.apache.hadoop.fs.s3a.impl.HeaderProcessing
-
See FileSystem.listXAttrs(path).
- listXAttrs(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- load(FileSystem, Path, FileStatus) - Static method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Load an instance from a file, then validate it.
- load(FileSystem, FileStatus, JsonSerialization<T>) - Static method in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Load an instance from a status, then validate it.
- load(FileSystem, Path, FileStatus, JsonSerialization<SinglePendingCommit>) - Static method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Load an instance from a file, then validate it.
- load(FileSystem, Path, JsonSerialization<SinglePendingCommit>, FileStatus) - Static method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Load an instance from a file, then validate it.
- load(FileSystem, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Load an instance from a file, then validate it.
- loadPendingCommits(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Loads pending commits from either memory or from the remote store (S3) based on the config.
- loadSinglePendingCommits(Path, boolean, CommitContext) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Load all single pending commits in the directory, using the
outer submitter.
- locateAllSinglePendingCommits(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Locate all files with the pending suffix under a directory.
- LOCATION_UNKNOWN - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
Output when the client cannot get the location of a bucket.
- LOG - Static variable in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
-
Subclasses refer to this.
- LOG - Static variable in class org.apache.hadoop.fs.s3a.impl.ClientManagerImpl
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders
-
- LOG - Static variable in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- LOG - Static variable in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- LOG_ENTRY_PATTERN - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
And the actual compiled pattern.
- LOG_ENTRY_REGEXP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Construct the log entry pattern.
- LOG_EVENT - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Log retries at debug.
- LOG_EVENT - Static variable in class org.apache.hadoop.fs.s3a.Invoker
-
Log retries at debug.
- LOG_NO_SQL_SELECT - Static variable in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport
-
- logAtDebug(Logger) - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
Dump the result at debug level.
- logDnsLookup(Configuration) - Static method in class org.apache.hadoop.fs.s3a.impl.NetworkBinding
-
Log the dns address associated with s3 endpoint.
- LOGGING_AUDIT_SERVICE - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Classname of the logging auditor: "org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor".
- LoggingAuditor - Class in org.apache.hadoop.fs.s3a.audit.impl
-
The LoggingAuditor logs operations at DEBUG (in SDK Request) and
in span lifecycle and S3 request class construction at TRACE.
- LoggingAuditor() - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor
-
Create the auditor.
- Login() - Constructor for class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
Create an instance with no login details.
- Login(String, String) - Constructor for class org.apache.hadoop.fs.s3native.S3xLoginHelper.Login
-
- LOGIN_WARNING - Static variable in class org.apache.hadoop.fs.s3native.S3xLoginHelper
-
- longBytesOption(Configuration, String, long, long) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a long option >= the minimum allowed value, supporting memory
prefixes K,M,G,T,P.
- longOption(Configuration, String, long, long) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a long option >= the minimum allowed value.
- lookupCounterValue(String) - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
Get the value of a counter.
- lookupCounterValue(String) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource
- lookupCounterValue(String) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
Get the value of a counter.
- lookupGauge(String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Look up a gauge.
- lookupGaugeValue(String) - Method in interface org.apache.hadoop.fs.s3a.statistics.BlockOutputStreamStatistics
-
Get the value of a gauge.
- lookupGaugeValue(String) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource
- lookupGaugeValue(String) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
Get the value of a gauge.
- lookupMetric(String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Look up a metric from both the registered set and the lighter weight
stream entries.
- lookupPassword(String, Configuration, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Deprecated.
- lookupPassword(String, Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a password from a configuration, including JCEKS files, handling both
the absolute key and bucket override.
- lookupPassword(String, Configuration, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get a password from a configuration, including JCEKS files, handling both
the absolute key and bucket override.
- lookupQuantiles(String) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Look up a quantiles.
- lookupS3ADelegationToken(Credentials, URI) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Look for any S3A token for the given FS service.
- lookupToken(Credentials, Text, Text) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Look up a token from the credentials, verify it is of the correct
kind.
- lookupToken(Credentials, Text) - Static method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Look up any token from the service; cast it to one of ours.
- ls(Path, boolean) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
List files.
- MAGIC - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- MAGIC_COMMITTER_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to indicate whether support for the Magic committer is enabled
in the filesystem.
- MAGIC_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Magic committer factory: "org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory".
- MAGIC_COMMITTER_PENDING_OBJECT_ETAG_NAME - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Etag name to be returned on non-committed S3 object: "pending".
- MAGIC_COMMITTER_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to indicate whether support for the Magic committer is enabled
in the filesystem.
- MAGIC_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- MAGIC_PATH_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- MagicCommitIntegration - Class in org.apache.hadoop.fs.s3a.commit
-
Adds the code needed for S3A to support magic committers.
- MagicCommitIntegration(S3AFileSystem, boolean) - Constructor for class org.apache.hadoop.fs.s3a.commit.MagicCommitIntegration
-
Instantiate.
- MagicCommitPaths - Class in org.apache.hadoop.fs.s3a.commit
-
Operations on (magic) paths.
- MagicCommitTracker - Class in org.apache.hadoop.fs.s3a.commit.magic
-
Put tracker for Magic commits.
- MagicCommitTracker(Path, String, String, String, String, WriteOperationHelper, PutTrackerStatistics) - Constructor for class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
Magic commit tracker.
- MagicCommitTrackerUtils - Class in org.apache.hadoop.fs.s3a.commit.magic
-
- magicElementIndex(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Get the index of the magic path element.
- magicPathChildren(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Get the child path elements under the magic path.
- magicPathParents(List<String>) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Get the parent path elements of the magic path.
- MagicS3GuardCommitter - Class in org.apache.hadoop.fs.s3a.commit.magic
-
This is a dedicated committer which requires the "magic" directory feature
of the S3A Filesystem to be enabled; it then uses paths for task and job
attempts in magic paths, so as to ensure that the final output goes direct
to the destination directory.
- MagicS3GuardCommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Create a task committer.
- MagicS3GuardCommitterFactory - Class in org.apache.hadoop.fs.s3a.commit.magic
-
Factory for the Magic committer.
- MagicS3GuardCommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory
-
- main(String[]) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Main entry point.
- makeIOE(String, Exception) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Convert any exception to an IOE, if needed.
- makeQualified(Path) - Method in interface org.apache.hadoop.fs.s3a.impl.ContextAccessors
-
Qualify a path.
- makeQualified(Path) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Qualify a path.
- makeQualified(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- mapErrorStatusCodeToStatisticName(int) - Static method in class org.apache.hadoop.fs.s3a.statistics.impl.StatisticsFromAwsSdkImpl
-
Map error status codes to statistic names, excluding 404.
- mapLocatedFiles(RemoteIterator<? extends LocatedFileStatus>, S3AUtils.LocatedFileStatusMap<T>) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Map an operation to every LocatedFileStatus in a remote
iterator, returning a list of the results.
- mark(int) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- markerFound(Path, String, S3ALocatedFileStatus) - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
A marker has been found; this may or may not be a leaf.
- MarkerPurgeSummary() - Constructor for class org.apache.hadoop.fs.s3a.tools.MarkerTool.MarkerPurgeSummary
-
- MARKERS - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Name of this tool: "markers".
- MARKERS_AWARE - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- MARKERS_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- MarkerTool - Class in org.apache.hadoop.fs.s3a.tools
-
Audit an S3 bucket for directory markers.
- MarkerTool(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Constructor.
- MarkerTool.MarkerPurgeSummary - Class in org.apache.hadoop.fs.s3a.tools
-
- MarkerTool.ScanArgs - Class in org.apache.hadoop.fs.s3a.tools
-
Arguments for the scan.
- MarkerTool.ScanArgsBuilder - Class in org.apache.hadoop.fs.s3a.tools
-
Builder of the scan arguments.
- MarkerTool.ScanResult - Class in org.apache.hadoop.fs.s3a.tools
-
Result of the scan operation.
- MarkerToolOperations - Interface in org.apache.hadoop.fs.s3a.tools
-
Operations which must be offered by the store for
MarkerTool.
- MarkerToolOperationsImpl - Class in org.apache.hadoop.fs.s3a.tools
-
Implement the marker tool operations by forwarding to the
OperationCallbacks instance provided in the constructor.
- MarkerToolOperationsImpl(OperationCallbacks) - Constructor for class org.apache.hadoop.fs.s3a.tools.MarkerToolOperationsImpl
-
Constructor.
- markSupported() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
- markSupported() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- markSupported() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- MarshalledCredentialBinding - Class in org.apache.hadoop.fs.s3a.auth
-
- MarshalledCredentialProvider - Class in org.apache.hadoop.fs.s3a.auth
-
AWS credential provider driven from marshalled session/full credentials
(full, simple session or role).
- MarshalledCredentialProvider(String, URI, Configuration, MarshalledCredentials, MarshalledCredentials.CredentialTypeRequired) - Constructor for class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialProvider
-
Constructor.
- MarshalledCredentials - Class in org.apache.hadoop.fs.s3a.auth
-
Stores the credentials for a session or for a full login.
- MarshalledCredentials() - Constructor for class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Constructor.
- MarshalledCredentials(String, String, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Create from a set of properties.
- MarshalledCredentials.CredentialTypeRequired - Enum in org.apache.hadoop.fs.s3a.auth
-
Enumeration of credential types for use in validation methods.
- MATERIALS_DESCRIPTION - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
JSON-encoded description of encryption materials used during encryption.
- MAX_ENTRIES_TO_DELETE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
The maximum number of entries that can be deleted in any bulk delete
call to S3: 1000.
- MAX_ERROR_RETRIES - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Number of times the AWS client library should retry errors before
escalating to the S3A code: "fs.s3a.attempts.maximum".
- MAX_MULTIPART_COUNT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Maximum number of partitions in a multipart upload: 10000.
- MAX_PAGING_KEYS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- MAX_SECRET_LENGTH - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
- MAX_TEXT_LENGTH - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
The maximum string length supported for text fields.
- MAX_THREADS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The maximum number of threads to allow in the pool used by S3A.
- MAX_TOTAL_TASKS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The maximum number of tasks queued (other than prefetcher tasks) if all threads are
busy: "fs.s3a.max.total.tasks".
- MAXIMUM_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Number of simultaneous connections to S3: "fs.s3a.connection.maximum".
- maxReadSizeForVectorReads() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
.
- maybe(boolean, T) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Convert a value into a non-empty Optional instance if
the value of include is true.
- maybeAddTrailingSlash(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Turns a path (relative or otherwise) into an S3 key, adding a trailing
"/" if the path is not the root and does not already have a "/"
at the end.
- maybeAddTrailingSlash(String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Turns a path (relative or otherwise) into an S3 key, adding a trailing
"/" if the path is not the root and does not already have a "/"
at the end.
- maybeApplyConstraint(GetObjectRequest.Builder) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Apply any revision control set by the policy if it is to be
enforced on the server.
- maybeApplyConstraint(CopyObjectRequest.Builder) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Apply any revision control set by the policy if it is to be
enforced on the server.
- maybeApplyConstraint(HeadObjectRequest.Builder) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
- maybeAwaitCompletion(CompletableFuture<T>) - Static method in class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
Block awaiting completion for any non-null future passed in;
No-op if a null arg was supplied.
- maybeCreateFakeParentDirectory(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create a fake parent directory if required.
- maybeCreateSuccessMarker(JobContext, List<String>, IOStatisticsSnapshot) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
if the job requires a success marker on a successful job,
create the _SUCCESS file.
- maybeCreateSuccessMarkerFromCommits(CommitContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- maybeExtractChannelException(String, String, Throwable) - Static method in class org.apache.hadoop.fs.s3a.impl.ErrorTranslation
-
Extract an AWS HTTP channel exception if the inner exception is considered
an HttpClient NoHttpResponseException or an OpenSSL channel exception.
- maybeExtractIOException(String, Throwable, String) - Static method in class org.apache.hadoop.fs.s3a.impl.ErrorTranslation
-
Translate an exception if it or its inner exception is an
IOException.
- maybeIgnore(boolean, String, InvocationRaisingIOE) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Execute an operation; maybe suppress any raised IOException.
- maybeIgnore(boolean, String, IOException) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Log or rethrow a caught IOException.
- MaybeIOE(IOException) - Constructor for class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations.MaybeIOE
-
Construct with an exception.
- maybeResetIOStatisticsContext() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Reset the IOStatistics context if statistics are being
collected.
- maybeRethrow() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations.MaybeIOE
-
Rethrow any exception.
- maybeRetry(boolean, String, String, boolean, Invoker.Retried, InvocationRaisingIOE) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a void operation with retry processing when doRetry=true, else
just once.
- maybeRetry(boolean, String, String, boolean, InvocationRaisingIOE) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a void operation with the default retry callback invoked when
doRetry=true, else just once.
- maybeRetry(boolean, String, String, boolean, Invoker.Retried, CallableRaisingIOE<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with retry processing when doRetry=true, else just once.
- maybeTranslateAuditException(String, Exception) - Static method in class org.apache.hadoop.fs.s3a.audit.AuditIntegration
-
- maybeTranslateCredentialException(String, Throwable) - Static method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
- META_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- METADATASTORE_AUTHORITATIVE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- METADATASTORE_METADATA_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- METRIC_TAG_BUCKET - Static variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
"bucket" The name of a field added to metrics records
that indicates the hostname portion of the FS URL.
- METRIC_TAG_FILESYSTEM_ID - Static variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
"s3aFileSystemId" The name of a field added to metrics
records that uniquely identifies a specific FileSystem instance.
- METRICS_SYSTEM_NAME - Static variable in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
"s3a-file-system" The name of the s3a-specific metrics
system instance used for s3a metrics.
- MIN_MULTIPART_THRESHOLD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- MINIMUM_NETWORK_OPERATION_DURATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
This is the minimum operation duration unless programmatically set.
- minSeekForVectorReads() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
.
- MINUTES_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- MkdirOperation - Class in org.apache.hadoop.fs.s3a.impl
-
The mkdir operation.
- MkdirOperation(StoreContext, Path, MkdirOperation.MkdirCallbacks, boolean, boolean) - Constructor for class org.apache.hadoop.fs.s3a.impl.MkdirOperation
-
Initialize Mkdir Operation context for S3A.
- MkdirOperation.MkdirCallbacks - Interface in org.apache.hadoop.fs.s3a.impl
-
Callbacks used by mkdir.
- MkdirOperationCallbacksImpl() - Constructor for class org.apache.hadoop.fs.s3a.S3AFileSystem.MkdirOperationCallbacksImpl
-
- mkdirs(Path, FsPermission) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Make the given path and all non-existent parents into
directories.
- mkdirs(Path, FsPermission) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- modifyHttpRequest(Context.ModifyHttpRequest, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- modifyHttpResponse(Context.ModifyHttpResponse, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- modifyRequest(Context.ModifyRequest, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- modifyResponse(Context.ModifyResponse, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- MR_JOB_ID - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
The MR job ID; copies from MRJobConfig so that it can be
referred to without needing hadoop-mapreduce on the classpath.
- MultiObjectDeleteException - Exception in org.apache.hadoop.fs.s3a.impl
-
- MultiObjectDeleteException(List<S3Error>) - Constructor for exception org.apache.hadoop.fs.s3a.impl.MultiObjectDeleteException
-
- MULTIPART_MIN_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The minimum multipart size which S3 supports.
- MULTIPART_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- MULTIPART_UPLOAD_ABORT - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- MULTIPART_UPLOAD_COMPLETE - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- MULTIPART_UPLOAD_PART - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- MULTIPART_UPLOAD_START - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- MULTIPART_UPLOADS_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Option to enable or disable the multipart uploads.
- MULTIPART_UPLOADS_LIST - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- MultipartUtils - Class in org.apache.hadoop.fs.s3a
-
MultipartUtils upload-specific functions for use by S3AFileSystem and Hadoop
CLI.
- MultipartUtils.UploadIterator - Class in org.apache.hadoop.fs.s3a
-
Iterator over multipart uploads.
- NAME - Static variable in class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider
-
- NAME - Static variable in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
- NAME - Static variable in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialProvider
-
Name: "org.apache.hadoop.fs.s3a.auth.MarshalledCredentialProvider".
- NAME - Static variable in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Name to include in persisted data, so as to differentiate from
any other manifests: "org.apache.hadoop.fs.s3a.commit.files.SuccessData/1".
- NAME - Static variable in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Name: "magic".
- NAME - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter
-
Name: "directory".
- NAME - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
Name: "partitioned".
- NAME - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Name: "staging".
- NAME - Static variable in class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- NAME - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- NAME - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
- NAME - Static variable in class org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
-
- NAME - Static variable in class org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
-
- NAME - Static variable in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
Name of this tool: "bucket".
- NativeS3FileSystem - Class in org.apache.hadoop.fs.s3native
-
- NativeS3FileSystem() - Constructor for class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- needsTaskCommit(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Did this task write any files in the work directory?
Probes for a task existing by looking to see if the attempt dir exists.
- needsTaskCommit(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- NetworkBinding - Class in org.apache.hadoop.fs.s3a.impl
-
Configures network settings when communicating with AWS services.
- newAbortMultipartUploadRequestBuilder(String, String) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Abort a multipart upload.
- newAbortMultipartUploadRequestBuilder(String, String) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newBulkDeleteRequestBuilder(List<ObjectIdentifier>) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create a request builder to delete objects in bulk.
- newBulkDeleteRequestBuilder(List<ObjectIdentifier>) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newCommitterStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create a new instance of the committer statistics.
- newCommitterStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a new instance of the committer statistics.
- newCommitterStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
Create a new instance of the committer statistics.
- newCommitterStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- newCommitterStatistics() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AStatisticsContext
-
Create a new instance of the committer statistics.
- newCompleteMultipartUploadRequestBuilder(String, String, List<CompletedPart>) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Complete a multipart upload.
- newCompleteMultipartUploadRequestBuilder(String, String, List<CompletedPart>) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newCopyObjectRequestBuilder(String, String, HeadObjectResponse) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create a copy request builder.
- newCopyObjectRequestBuilder(String, String, HeadObjectResponse) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newDelegationTokenStatistics() - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a delegation token statistics instance.
- newDelegationTokenStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
Create a delegation token statistics instance.
- newDelegationTokenStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- newDelegationTokenStatistics() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AStatisticsContext
-
Create a delegation token statistics instance.
- newDeleteObjectRequestBuilder(String) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create a request builder to delete a single object.
- newDeleteObjectRequestBuilder(String) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newDirectoryMarkerRequest(String) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create a PutObjectRequest request for creating
an empty directory.
- newDirectoryMarkerRequest(String) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newGetObjectRequestBuilder(String) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create a GET request builder.
- newGetObjectRequestBuilder(String) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newGetRequestBuilder(String) - Method in interface org.apache.hadoop.fs.s3a.S3AInputStream.InputStreamCallbacks
-
Create a GET request builder.
- newHeadBucketRequestBuilder(String) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create a HEAD bucket request builder.
- newHeadBucketRequestBuilder(String) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newHeadObjectRequestBuilder(String) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create a HEAD object request builder.
- newHeadObjectRequestBuilder(String) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newInputStreamStatistics(FileSystem.Statistics) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a stream input statistics instance.
- newInputStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
Create a stream input statistics instance.
- newInputStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- newInputStreamStatistics() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AStatisticsContext
-
Create a stream input statistics instance.
- newListMultipartUploadsRequestBuilder(String) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
List all multipart uploads under a prefix.
- newListMultipartUploadsRequestBuilder(String) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newListObjectsV1RequestBuilder(String, String, int) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create the (legacy) V1 list request builder.
- newListObjectsV1RequestBuilder(String, String, int) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newListObjectsV2RequestBuilder(String, String, int) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create a V2 list request builder.
- newListObjectsV2RequestBuilder(String, String, int) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newMultipartUploadRequestBuilder(String, PutObjectOptions) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Start a multipart upload.
- newMultipartUploadRequestBuilder(String, PutObjectOptions) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newOperationId() - Static method in class org.apache.hadoop.fs.s3a.impl.ActiveOperationContext
-
Create an operation ID.
- newOutputStreamStatistics(FileSystem.Statistics) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Create a stream output statistics instance.
- newOutputStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
Create a stream output statistics instance.
- newOutputStreamStatistics() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- newOutputStreamStatistics() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AStatisticsContext
-
Create a stream output statistics instance.
- newPutObjectRequestBuilder(String, PutObjectOptions, long, boolean) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create a PutObjectRequest request builder.
- newPutObjectRequestBuilder(String, PutObjectOptions, long, boolean) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
Create a putObject request.
- newPutObjectRequestBuilder(String, long, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Create a putObject request builder.
- newSid() - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Statement ID factory.
- newStatisticsFromAwsSdk() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
- newStatisticsFromAwsSdk() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- newStatisticsFromAwsSdk() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AStatisticsContext
-
Create a StatisticsFromAwsSdk instance.
- newStream() - Method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders.BaseContentProvider
-
Create a new stream.
- newUploadPartRequestBuilder(String, String, int, long) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Create and initialize a part request builder of a multipart upload.
- newUploadPartRequestBuilder(String, String, int, long) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- newUploadPartRequestBuilder(String, String, int, long) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Create and initialize a part request builder of a multipart upload.
- newUploadPartRequestBuilder(String, String, int, long) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Create and initialize a part request builder of a multipart upload.
- next() - Method in class org.apache.hadoop.fs.s3a.MultipartUtils.UploadIterator
-
- NO_AWS_CREDENTIAL_PROVIDERS - Static variable in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
- NO_AWS_CREDENTIALS - Static variable in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Error text on empty credentials: "No AWS credentials".
- NO_OP - Static variable in class org.apache.hadoop.fs.s3a.Invoker
-
No op for a retrying callback.
- NO_OP_SIGNER - Static variable in class org.apache.hadoop.fs.s3a.auth.SignerFactory
-
- NoAuthWithAWSException - Exception in org.apache.hadoop.fs.s3a.auth
-
A specific subclass of SdkException which is
used in the S3A retry policy to fail fast when there is any
authentication problem.
- NoAuthWithAWSException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException
-
- NoAuthWithAWSException(String) - Constructor for exception org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException
-
- NoAwsCredentialsException - Exception in org.apache.hadoop.fs.s3a.auth
-
A special exception which declares that no credentials were found;
this can be treated specially in logging, handling, etc.
- NoAwsCredentialsException(String, String) - Constructor for exception org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException
-
Construct.
- NoAwsCredentialsException(String) - Constructor for exception org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException
-
- NoAwsCredentialsException(String, String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException
-
Construct with exception.
- NoCredentials() - Constructor for class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider.NoCredentials
-
- NONE - Static variable in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations.MaybeIOE
-
- nonNullDurationTrackerFactory(DurationTrackerFactory) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Given a possibly null duration tracker factory, return a non-null
one for use in tracking durations -either that or the FS tracker
itself.
- nonNullDurationTrackerFactory(DurationTrackerFactory) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Given a possibly null duration tracker factory, return a non-null
one for use in tracking durations -either that or the FS tracker
itself.
- nonNullDurationTrackerFactory(DurationTrackerFactory) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Given a possibly null duration tracker factory, return a non-null
one for use in tracking durations -either that or the FS tracker
itself.
- NOOP_AUDIT_SERVICE - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Classname of the No-op auditor: "org.apache.hadoop.fs.s3a.audit.impl.NoopAuditor".
- NoopAuditManagerS3A - Class in org.apache.hadoop.fs.s3a.audit.impl
-
Simple No-op audit manager for use before a real
audit chain is set up, and for testing.
- NoopAuditManagerS3A() - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
Constructor.
- NoopAuditor - Class in org.apache.hadoop.fs.s3a.audit.impl
-
An audit service which returns the
NoopSpan.
- NoopAuditor() - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditor
-
Constructor.
- NoopAuditor(NoopSpan.SpanActivationCallbacks) - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditor
-
- NoopSpan - Class in org.apache.hadoop.fs.s3a.audit.impl
-
A minimal span with no direct side effects.
- NoopSpan(String, String, String, String, NoopSpan.SpanActivationCallbacks) - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.NoopSpan
-
Create a no-op span.
- NoopSpan() - Constructor for class org.apache.hadoop.fs.s3a.audit.impl.NoopSpan
-
- NoopSpan.SpanActivationCallbacks - Interface in org.apache.hadoop.fs.s3a.audit.impl
-
Activation callbacks.
- NOT_A_WRAPPED_SPAN - Static variable in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- noteAwsClientExecuteTime(Duration) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.StatisticsFromAwsSdkImpl
-
- noteAwsClientExecuteTime(Duration) - Method in interface org.apache.hadoop.fs.s3a.statistics.StatisticsFromAwsSdk
-
Record how long a request took to execute on the
client.
- noteAwsRequestTime(Duration) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.StatisticsFromAwsSdkImpl
-
- noteAwsRequestTime(Duration) - Method in interface org.apache.hadoop.fs.s3a.statistics.StatisticsFromAwsSdk
-
Record how long a request took overall.
- noteRequestMarshallTime(Duration) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.StatisticsFromAwsSdkImpl
-
- noteRequestMarshallTime(Duration) - Method in interface org.apache.hadoop.fs.s3a.statistics.StatisticsFromAwsSdk
-
Record how long a request took to marshall into
XML.
- noteRequestSigningTime(Duration) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.StatisticsFromAwsSdkImpl
-
- noteRequestSigningTime(Duration) - Method in interface org.apache.hadoop.fs.s3a.statistics.StatisticsFromAwsSdk
-
Record how long a request took to sign, including
any calls to EC2 credential endpoints.
- noteResponseProcessingTime(Duration) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.StatisticsFromAwsSdkImpl
-
- noteResponseProcessingTime(Duration) - Method in interface org.apache.hadoop.fs.s3a.statistics.StatisticsFromAwsSdk
-
Record how long it took to process the response.
- noteSpanReferenceLost(long) - Method in interface org.apache.hadoop.fs.s3a.audit.OperationAuditor
-
Span reference lost from GC operations.
- notFound(FileNotFoundException) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Handle FileNotFoundException by converting to an exit exception
with specific error code.
- NoVersionAttributeException - Exception in org.apache.hadoop.fs.s3a
-
Indicates the S3 object does not provide the versioning attribute required
by the configured change detection policy.
- NoVersionAttributeException(String, String) - Constructor for exception org.apache.hadoop.fs.s3a.NoVersionAttributeException
-
Constructs a NoVersionAttributeException.
- OBJECT_LOCK_LEGAL_HOLD_STATUS - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- OBJECT_LOCK_MODE - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- OBJECT_LOCK_RETAIN_UNTIL_DATE - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
- OBJECT_REPLICATION_STATUS - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Header for the replication status of an Amazon S3 Object.
- objectRepresentsDirectory(String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Predicate: does the object represent a directory?.
- OBJECTSIZE_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "objectsize".
- of(IOException) - Static method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations.MaybeIOE
-
- once(String, String, CallableRaisingIOE<T>) - Static method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function, translating any exception into an IOException.
- once(String, String, InvocationRaisingIOE) - Static method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute an operation with no result.
- onceInTheFuture(String, String, Future<T>) - Static method in class org.apache.hadoop.fs.s3a.Invoker
-
Wait for a future, translating SdkException into an IOException.
- onceTrackingDuration(String, String, DurationTracker, CallableRaisingIOE<T>) - Static method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function, translating any exception into an IOException.
- onChangeDetected(String, String, String, long, String, long) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy
-
Takes appropriate action based on
mode when a change has
been detected.
- onExecutionFailure(Context.FailedExecution, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
Forward to active span.
- onFailure(String, IOException, int, boolean) - Method in interface org.apache.hadoop.fs.s3a.Invoker.Retried
-
Retry event in progress (before any sleep).
- OP_CREATE_DIRECTORIES - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Directory creation "op_create_directories".
- OP_CREATE_ONE_DIRECTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Creating a single directory "op_create_one_directory".
- OP_DIRECTORY_SCAN - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Directory scan "op_directory_scan".
- OP_LOAD_ALL_MANIFESTS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
"op_load_all_manifests".
- OP_LOAD_MANIFEST - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Load a task manifest: "op_load_manifest".
- OP_RENAME_FILE - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Rename a file: "op_rename".
- OP_SAVE_TASK_MANIFEST - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Save a task manifest: "op_task_stage_save_task_manifest".
- OP_STAGE_JOB_ABORT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Job abort: "op_job_stage_abort".
- OP_STAGE_JOB_CLEANUP - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Job cleanup: "op_job_stage_cleanup".
- OP_STAGE_JOB_COMMIT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Overall job commit "committer_commit_job".
- OP_STAGE_JOB_CREATE_TARGET_DIRS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Prepare Directories Stage: "op_job_stage_create_target_dirs".
- OP_STAGE_JOB_LOAD_MANIFESTS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Load Manifest Stage: "op_job_stage_load_manifests".
- OP_STAGE_JOB_RENAME_FILES - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Rename files stage duration: "op_job_stage_rename_files".
- OP_STAGE_JOB_SAVE_SUCCESS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Job saving _SUCCESS marker Stage: "op_job_stage_save_success_marker".
- OP_STAGE_JOB_SETUP - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Job Setup Stage: "op_job_stage_setup".
- OP_STAGE_JOB_VALIDATE_OUTPUT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Output Validation (within job commit) Stage: "op_job_stage_optional_validate_output".
- OP_STAGE_TASK_ABORT_TASK - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Task abort: "op_task_stage_abort_task".
- OP_STAGE_TASK_COMMIT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Task Commit Stage: "op_stage_task_commit".
- OP_STAGE_TASK_SAVE_MANIFEST - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Task saving manifest file Stage: "op_task_stage_save_manifest".
- OP_STAGE_TASK_SCAN_DIRECTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Task Scan directory Stage: "op_stage_task_scan_directory".
- OP_STAGE_TASK_SETUP - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitterStatisticNames
-
Task Setup Stage: "op_task_stage_setup".
- open(Path, int) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Opens an FSDataInputStream at the indicated Path.
- open(Path, int) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- OpenFileInformation() - Constructor for class org.apache.hadoop.fs.s3a.impl.OpenFileSupport.OpenFileInformation
-
Constructor.
- OpenFileSupport - Class in org.apache.hadoop.fs.s3a.impl
-
Helper class for openFile() logic, especially processing file status
args and length/etag/versionID.
- OpenFileSupport(ChangeDetectionPolicy, long, String, int, long, S3AInputPolicy) - Constructor for class org.apache.hadoop.fs.s3a.impl.OpenFileSupport
-
Instantiate with the default options from the filesystem.
- OpenFileSupport.OpenFileInformation - Class in org.apache.hadoop.fs.s3a.impl
-
The information on a file needed to open it.
- openFileWithOptions(Path, OpenFileParameters) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Initiate the open() operation.
- openForRead(long, int) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObject
-
Opens a section of the file for reading.
- openSimpleFile(int) - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport
-
Open a simple file, using all the default
options.
- OPENSSL_STREAM_CLOSED - Static variable in class org.apache.hadoop.fs.s3a.impl.ErrorTranslation
-
OpenSSL stream closed error: "WFOPENSSL0035".
- OPERATION_OPEN - Static variable in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- OPERATION_REOPEN - Static variable in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- OperationAuditor - Interface in org.apache.hadoop.fs.s3a.audit
-
Interfaces for audit services to implement.
- OperationAuditorOptions - Class in org.apache.hadoop.fs.s3a.audit
-
- OperationCallbacks - Interface in org.apache.hadoop.fs.s3a.impl
-
- operationRetried(String, Exception, int, boolean) - Method in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
Callback from
Invoker when an operation is retried.
- operationRetried(Exception) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Callback when an operation was retried.
- operationRetried(String, Exception, int, boolean) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Callback from
Invoker when an operation is retried.
- operationRetried(Exception) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Callback when an operation was retried.
- operationRetried(String, Exception, int, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Callback from
Invoker when an operation is retried.
- OPT_AUDIT - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Audit sub-command: "audit".
- OPT_CLEAN - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Clean Sub-command: "clean".
- OPT_ENDPOINT - Static variable in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
endpoint "endpoint".
- OPT_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Limit of objects to scan: "limit".
- OPT_MAX - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Max number of markers to find: "max".
- OPT_MIN - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Min number of markers to find: "min".
- OPT_NONAUTH - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Only consider markers found in non-authoritative paths
as failures: "nonauth".
- OPT_OUT - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Name of a file to save the list of markers to: "out".
- OPT_PREFIX - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Prefix to use for config options: "fs.s3a.committer.".
- OPT_REGION - Static variable in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
region "region".
- OPT_SUMMARY_REPORT_DIR - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Directory for saving job summary reports.
- OPT_ZONE - Static variable in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
Zone for a store.
- OPTIMIZED_COPY_FROM_LOCAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Is the higher performance copy from local file to S3 enabled?
This switch allows for it to be disabled if there are problems.
- OPTIMIZED_COPY_FROM_LOCAL_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- OPTIONS_CREATE_FILE_NO_OVERWRITE - Static variable in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder
-
Classic create file option set: no overwrite.
- OPTIONS_CREATE_FILE_OVERWRITE - Static variable in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder
-
Classic create file option set: overwriting.
- OPTIONS_CREATE_FILE_PERFORMANCE - Static variable in class org.apache.hadoop.fs.s3a.impl.CreateFileBuilder
-
Performance create options.
- org.apache.hadoop.fs.s3a - package org.apache.hadoop.fs.s3a
-
S3A Filesystem.
- org.apache.hadoop.fs.s3a.adapter - package org.apache.hadoop.fs.s3a.adapter
-
Adapter classes for allowing V1 credential providers to be used with SDKV2.
- org.apache.hadoop.fs.s3a.api - package org.apache.hadoop.fs.s3a.api
-
Where the interfaces for classes implemented in o.a.h.fs.s3a.impl
should go.
- org.apache.hadoop.fs.s3a.audit - package org.apache.hadoop.fs.s3a.audit
-
Support for auditing and ultimately tracing operations.
- org.apache.hadoop.fs.s3a.audit.impl - package org.apache.hadoop.fs.s3a.audit.impl
-
The internal implementations of auditing support.
- org.apache.hadoop.fs.s3a.auth - package org.apache.hadoop.fs.s3a.auth
-
Authentication and permissions support.
- org.apache.hadoop.fs.s3a.auth.delegation - package org.apache.hadoop.fs.s3a.auth.delegation
-
Extensible delegation token support for the S3A connector.
- org.apache.hadoop.fs.s3a.commit - package org.apache.hadoop.fs.s3a.commit
-
Support for committing the output of analytics jobs directly to S3.
- org.apache.hadoop.fs.s3a.commit.files - package org.apache.hadoop.fs.s3a.commit.files
-
Persistent data formats for the committers.
- org.apache.hadoop.fs.s3a.commit.impl - package org.apache.hadoop.fs.s3a.commit.impl
-
Internal classes which make use of mapreduce code.
- org.apache.hadoop.fs.s3a.commit.magic - package org.apache.hadoop.fs.s3a.commit.magic
-
This is the "Magic" committer and support.
- org.apache.hadoop.fs.s3a.commit.staging - package org.apache.hadoop.fs.s3a.commit.staging
-
The staging committers.
- org.apache.hadoop.fs.s3a.impl - package org.apache.hadoop.fs.s3a.impl
-
Implementation classes private to the S3A store.
- org.apache.hadoop.fs.s3a.prefetch - package org.apache.hadoop.fs.s3a.prefetch
-
High performance s3 input stream which reads in
blocks and can cache blocks in the local filesystem.
- org.apache.hadoop.fs.s3a.s3guard - package org.apache.hadoop.fs.s3a.s3guard
-
This package contained S3Guard support; now the feature has been removed,
its contents are limited to the public command line and some
methods still used by directory marker code.
- org.apache.hadoop.fs.s3a.select - package org.apache.hadoop.fs.s3a.select
-
Was the location for support for S3 Select.
- org.apache.hadoop.fs.s3a.statistics - package org.apache.hadoop.fs.s3a.statistics
-
Statistics collection for the S3A connector: interfaces.
- org.apache.hadoop.fs.s3a.statistics.impl - package org.apache.hadoop.fs.s3a.statistics.impl
-
Statistics collection for the S3A connector: implementation.
- org.apache.hadoop.fs.s3a.tools - package org.apache.hadoop.fs.s3a.tools
-
S3A Command line tools independent of S3Guard.
- org.apache.hadoop.fs.s3native - package org.apache.hadoop.fs.s3native
-
A distributed implementation of
FileSystem for reading and writing files on
Amazon S3.
- outputImmediatelyVisible() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker
-
Flag to indicate that output is not visible after the stream
is closed.
- outputImmediatelyVisible() - Method in class org.apache.hadoop.fs.s3a.commit.PutTracker
-
Flag to indicate that output is not immediately visible after the stream
is closed.
- OUTSIDE_SPAN - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Operation name for any operation outside of an explicit
span.
- OWNER_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "owner".
- pageSize() - Method in class org.apache.hadoop.fs.s3a.impl.BulkDeleteOperation
-
- PARAM_TASK_ATTEMPT_ID - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Task Attempt ID query header: "ta".
- parseArgs(String[]) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Parse CLI arguments and returns the position arguments.
- parseArgsWithErrorReporting(String[]) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Process the arguments.
- PART_UPLOAD_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Timeout for uploading all of a small object or a single part
of a larger one.
- PARTITION_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Partitioned committer factory: "org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory".
- PartitionedStagingCommitter - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Partitioned committer.
- PartitionedStagingCommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
- PartitionedStagingCommitterFactory - Class in org.apache.hadoop.fs.s3a.commit.staging
-
- PartitionedStagingCommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory
-
- partPut(long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext.EmptyMultipartUploaderStatistics
-
- partPut(long) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.S3AMultipartUploaderStatisticsImpl
-
- partPut(long) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AMultipartUploaderStatistics
-
- path(Path, String...) - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Varags constructor of paths.
- PATH_STYLE_ACCESS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PathCommitException - Exception in org.apache.hadoop.fs.s3a.commit
-
Path exception to use for various commit issues.
- PathCommitException(String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.commit.PathCommitException
-
- PathCommitException(String, String) - Constructor for exception org.apache.hadoop.fs.s3a.commit.PathCommitException
-
- PathCommitException(Path, String) - Constructor for exception org.apache.hadoop.fs.s3a.commit.PathCommitException
-
- PathCommitException(String, String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.commit.PathCommitException
-
- Paths - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Path operations for the staging committers.
- pathToKey(Path) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
- pathToKey(Path) - Method in interface org.apache.hadoop.fs.s3a.impl.ContextAccessors
-
Turns a path (relative or otherwise) into an S3 key.
- pathToKey(Path) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
Turns a path (relative or otherwise) into an S3 key.
- pathToKey(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Turns a path (relative or otherwise) into an S3 key.
- pathToLocalFile(Path) - Method in interface org.apache.hadoop.fs.s3a.impl.CopyFromLocalOperation.CopyFromLocalOperationCallbacks
-
Get the file from a path.
- pathToLocalFile(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.CopyFromLocalCallbacksImpl
-
- PENDING_FILTER - Static variable in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Filter to find all {code .pending} files.
- PENDING_SUFFIX - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Suffix applied to pending commit metadata: ".pending".
- PendingSet - Class in org.apache.hadoop.fs.s3a.commit.files
-
Persistent format for multiple pending commits.
- PendingSet() - Constructor for class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- PendingSet(int) - Constructor for class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- PENDINGSET_FILTER - Static variable in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Filter to find all {code .pendingset} files.
- PENDINGSET_SUFFIX - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Suffix applied to multiple pending commit metadata: ".pendingset".
- pendingsetCommitted(IOStatistics) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
Callback when a pendingset has been committed,
including any source statistics.
- PerformanceFlagEnum - Enum in org.apache.hadoop.fs.s3a.api
-
Enum of performance flags.
- PersistentCommitData<T extends PersistentCommitData> - Class in org.apache.hadoop.fs.s3a.commit.files
-
Class for single/multiple commit data structures.
- PersistentCommitData() - Constructor for class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
- policy(RoleModel.Statement...) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
From a set of statements, create a policy.
- policy(List<RoleModel.Statement>) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
From a set of statements, create a policy.
- Policy() - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Empty constructor: initializes the statements to an empty list.
- Policy(List<RoleModel.Statement>) - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
- Policy(RoleModel.Statement...) - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
- precommitCheckPendingFiles(CommitContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Run a precommit check that all files are loadable.
- preCommitJob(CommitContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Subclass-specific pre-Job-commit actions.
- preCommitJob(CommitContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter
-
Pre-commit actions for a job.
- preCommitJob(CommitContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitter
-
All
Job-side conflict resolution.
- preCommitJob(CommitContext, AbstractS3ACommitter.ActiveCommit) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Pre-commit actions for a job.
- PRECONDITIONS_FAILED - Static variable in exception org.apache.hadoop.fs.s3a.RemoteFileChangedException
-
- PREFETCH_BLOCK_COUNT_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Maximum number of blocks prefetched at any given time.
- PREFETCH_BLOCK_DEFAULT_COUNT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PREFETCH_BLOCK_DEFAULT_SIZE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PREFETCH_BLOCK_SIZE_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
The size of a single prefetched block in number of bytes.
- PREFETCH_ENABLED_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default option as to whether the prefetching input stream is enabled.
- PREFETCH_ENABLED_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Controls whether the prefetching input stream is enabled.
- PREFETCH_MAX_BLOCKS_COUNT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Prefetch max blocks count config.
- prepareRequest(SdkRequest.Builder) - Method in interface org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl.PrepareRequest
-
Post-creation preparation of AWS request.
- prepareSTSClient() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Get the client to AWS STS.
- prepareToOpenFile(Path, OpenFileParameters, long) - Method in class org.apache.hadoop.fs.s3a.impl.OpenFileSupport
-
Prepare to open a file from the openFile parameters.
- println(PrintStream, String, Object...) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Print a formatted string followed by a newline to the output stream.
- PROBE_FAILURE - Static variable in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
Error Message logged/thrown when the tool could not start as
the bucket probe was not disabled and the probe (inevitably)
failed.
- probePathStatus(Path, Set<StatusProbeEnum>) - Method in interface org.apache.hadoop.fs.s3a.impl.GetContentSummaryOperation.GetContentSummaryCallbacks
-
Get the status of a path.
- probePathStatus(Path, Set<StatusProbeEnum>) - Method in interface org.apache.hadoop.fs.s3a.impl.MkdirOperation.MkdirCallbacks
-
Get the status of a path.
- probePathStatus(Path, Set<StatusProbeEnum>) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.GetContentSummaryCallbacksImpl
-
- probePathStatus(Path, Set<StatusProbeEnum>) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem.MkdirOperationCallbacksImpl
-
- processDeleteOnExit() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Delete all paths that were marked as delete-on-exit.
- processException(SdkException, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Process an exception generated against the change policy.
- processMetadata(HeadObjectResponse, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Process metadata response from server for validation against the change
policy.
- processMetadata(GetObjectResponse, String) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Process the response from server for validation against the change
policy.
- processResponse(GetObjectResponse, String, long) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Process the response from the server for validation against the
change policy.
- processResponse(CopyObjectResponse) - Method in class org.apache.hadoop.fs.s3a.impl.ChangeTracker
-
Process the response from the server for validation against the
change policy.
- PRODUCT_NAME - Static variable in class org.apache.hadoop.fs.s3a.impl.S3ExpressStorage
-
What is the official product name? used for error messages and logging: "Amazon S3 Express One Zone Storage".
- PROFILE_CREDENTIALS_V1 - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
V1 profile credential provider: "com.amazonaws.auth.profile.ProfileCredentialsProvider".
- PROFILE_CREDENTIALS_V2 - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
V2 environment variables credential provider.
- ProgressableProgressListener - Class in org.apache.hadoop.fs.s3a
-
Listener to progress from AWS regarding transfers.
- ProgressableProgressListener(S3AStore, String, Progressable) - Constructor for class org.apache.hadoop.fs.s3a.ProgressableProgressListener
-
Instantiate.
- progressChanged(ProgressListenerEvent, long) - Method in interface org.apache.hadoop.fs.s3a.impl.ProgressListener
-
- ProgressListener - Interface in org.apache.hadoop.fs.s3a.impl
-
Interface for progress listeners to implement.
- ProgressListenerEvent - Enum in org.apache.hadoop.fs.s3a.impl
-
Enum for progress listener events.
- propagateBucketOptions(Configuration, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Propagates bucket-specific settings into generic S3A configuration keys.
- PROXY_DOMAIN - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_HOST - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_PASSWORD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_PORT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_SECURED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Is the proxy secured(proxyProtocol = HTTPS)?
- PROXY_USERNAME - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PROXY_WORKSTATION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- publish(MetricCollection) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AwsStatisticsCollector
-
This is the callback from the AWS SDK where metrics
can be collected.
- PURGE_EXISTING_MULTIPART - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- PURGE_EXISTING_MULTIPART_AGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
purge any multipart uploads older than this number of seconds.
- PURPOSE - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- PURPOSE - Static variable in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
Purpose of this tool: "View and manipulate S3 buckets".
- PURPOSE - Static variable in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
Purpose of this tool: "View and manipulate S3 directory markers".
- PUT - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- putDiagnostic(String, String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Add a diagnostics entry.
- putExtraData(String, String) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Set/Update an extra data entry.
- putExtraData(String, String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Set/Update an extra data entry.
- putObject(PutObjectRequest, File, ProgressableProgressListener) - Method in class org.apache.hadoop.fs.s3a.impl.S3AStoreImpl
-
Start a transfer-manager managed async PUT of an object,
incrementing the put requests and put bytes
counters.
- putObject(PutObjectRequest, File, ProgressableProgressListener) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Start a transfer-manager managed async PUT of an object,
incrementing the put requests and put bytes
counters.
- putObject(PutObjectRequest, File, ProgressableProgressListener) - Method in interface org.apache.hadoop.fs.s3a.S3AStore
-
Start a transfer-manager managed async PUT of an object,
incrementing the put requests and put bytes
counters.
- putObject(PutObjectRequest, PutObjectOptions, S3ADataBlocks.BlockUploadData, DurationTrackerFactory) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
PUT an object directly (i.e.
- putObject(PutObjectRequest, PutObjectOptions, S3ADataBlocks.BlockUploadData, DurationTrackerFactory) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
PUT an object directly (i.e.
- PutObjectOptions - Class in org.apache.hadoop.fs.s3a.impl
-
Extensible structure for options when putting/writing objects.
- PutObjectOptions(boolean, String, Map<String, String>) - Constructor for class org.apache.hadoop.fs.s3a.impl.PutObjectOptions
-
Constructor.
- PutTracker - Class in org.apache.hadoop.fs.s3a.commit
-
Multipart put tracker.
- PutTracker(String) - Constructor for class org.apache.hadoop.fs.s3a.commit.PutTracker
-
Instantiate.
- PutTrackerStatistics - Interface in org.apache.hadoop.fs.s3a.statistics
-
Interface for PUT tracking.
- RANGE - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Range header for the get object request.
- RangeNotSatisfiableEOFException - Exception in org.apache.hadoop.fs.s3a
-
Status code 416, range not satisfiable.
- RangeNotSatisfiableEOFException(String, Exception) - Constructor for exception org.apache.hadoop.fs.s3a.RangeNotSatisfiableEOFException
-
- read(ByteBuffer, long, int) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ABlockManager
-
Reads into the given buffer size bytes from the underlying file
starting at startOffset.
- read(ByteBuffer, long, int) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ACachingBlockManager
-
Reads into the given buffer size bytes from the underlying file
starting at startOffset.
- read() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Reads and returns one byte from this stream.
- read(byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Reads up to len bytes from this stream and copies them into
the given buffer starting at the given offset.
- read() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- read(byte[]) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Reads bytes from this stream and copies them into
the given buffer starting at the beginning (offset 0).
- read(byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Reads up to len bytes from this stream and copies them into
the given buffer starting at the given offset.
- read(ByteBuffer, long, int) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObjectReader
-
Stars reading at offset and reads upto size bytes into buffer.
- read() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- read(byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
This updates the statistics on read operations started and whether
or not the read operation "completed", that is: returned the exact
number of bytes requested.
- READAHEAD_RANGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
read ahead buffer size to prevent connection re-establishments.
- readException() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
An ignored stream read exception was received.
- readFields(DataInput) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Read state.
- readFields(DataInput) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.EncryptionSecrets
-
Read in from the writable stream.
- readFields(DataInput) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
- readFields(DataInput) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Read in the fields.
- readFully(long, byte[], int, int) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Subclass readFully() operation which only seeks at the start
of the series of operations; seeking back at the end.
- readFullyOperationStarted(long, long) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
A PositionedRead.read(position, buffer, offset, length)
operation has just started.
- readOperationCompleted(int, int) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
A read operation has completed.
- readOperationStarted(long, long) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
A read(byte[] buf, int off, int len) operation has started.
- readVectored(List<? extends FileRange>, IntFunction<ByteBuffer>) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Vectored read implementation for S3AInputStream.
- readVectoredBytesDiscarded(int) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
Number of bytes discarded during vectored read.
- readVectoredOperationStarted(int, int) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
A vectored read operation has started..
- recordDuration(Statistic, boolean, Duration) - Method in class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Add the duration as a timed statistic, deriving
statistic name from the operation symbol and the outcome.
- recordDuration(Statistic, boolean, Duration) - Method in interface org.apache.hadoop.fs.s3a.statistics.CountersAndGauges
-
Record a duration.
- recordDuration(Statistic, boolean, Duration) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.BondedS3AStatisticsContext
-
- recordDuration(Statistic, boolean, Duration) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.EmptyS3AStatisticsContext
-
- recordJobFailure(Throwable) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Note a failure by setting success flag to false,
then add the exception to the diagnostics.
- recoverTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Task recovery considered Unsupported: Warn and fail.
- recursiveDirectoryRename() - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
Execute a full recursive rename.
- REFERRER_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "referrer".
- REFERRER_HEADER_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Should the logging auditor add the HTTP Referrer header?
"fs.s3a.audit.referrer.enabled".
- REFERRER_HEADER_ENABLED_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Should the logging auditor add the HTTP Referrer header?
Default value: true.
- REFERRER_HEADER_FILTER - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
List of audit fields to strip from referrer headers.
- refresh() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Deprecated.
- registerSigner(String, Class<? extends Signer>) - Static method in class org.apache.hadoop.fs.s3a.auth.SignerFactory
-
Register an implementation class for the given signer type.
- registerStore(String, Configuration, DelegationTokenProvider, UserGroupInformation) - Method in interface org.apache.hadoop.fs.s3a.auth.AwsSignerInitializer
-
Register a store instance.
- registerStore(String, Configuration, DelegationTokenProvider, UserGroupInformation) - Method in class org.apache.hadoop.fs.s3a.auth.CustomSdkSigner.Initializer
-
- REJECT_OUT_OF_SPAN_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.audit.S3AAuditConstants
-
Should operations outside spans be rejected?
This is for testing coverage of the span code; if used
in production there's a risk of unexpected failures.
- rejectSecretsInURIs(URI) - Static method in class org.apache.hadoop.fs.s3native.S3xLoginHelper
-
Extract the login details from a URI, raising an exception if
the URI contains them.
- remainingInCurrentRequest() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Bytes left in the current request.
- remainingInFile() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Bytes left in stream.
- RemoteFileChangedException - Exception in org.apache.hadoop.fs.s3a
-
Indicates the S3 object is out of sync with the expected version.
- RemoteFileChangedException(String, String, String) - Constructor for exception org.apache.hadoop.fs.s3a.RemoteFileChangedException
-
Constructs a RemoteFileChangedException.
- RemoteFileChangedException(String, String, String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.RemoteFileChangedException
-
Constructs a RemoteFileChangedException.
- REMOTEIP_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "remoteip".
- removeAllowedMarkers(DirectoryPolicy) - Method in class org.apache.hadoop.fs.s3a.impl.DirMarkerTracker
-
Scan the surplus marker list and remove from it all where the directory
policy says "keep".
- removeBucketOverrides(String, Configuration, String...) - Static method in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
Remove any values from a bucket.
- removeKeys(List<ObjectIdentifier>, boolean) - Method in interface org.apache.hadoop.fs.s3a.impl.OperationCallbacks
-
Remove keys from the store.
- removeKeys(List<ObjectIdentifier>, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- removeKeys(List<ObjectIdentifier>, boolean) - Method in interface org.apache.hadoop.fs.s3a.tools.MarkerToolOperations
-
Remove keys from the store.
- removeKeys(List<ObjectIdentifier>, boolean) - Method in class org.apache.hadoop.fs.s3a.tools.MarkerToolOperationsImpl
-
- rename(Path, Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Renames Path src to Path dst.
- rename(Path, Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- RENAME_PARALLEL_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
This is an arbitrary value: 10.
- RenameFailedException - Exception in org.apache.hadoop.fs.s3a
-
Error to indicate that a specific rename failed.
- RenameFailedException(String, String, Throwable) - Constructor for exception org.apache.hadoop.fs.s3a.RenameFailedException
-
- RenameFailedException(String, String, String) - Constructor for exception org.apache.hadoop.fs.s3a.RenameFailedException
-
- RenameFailedException(Path, Path, String) - Constructor for exception org.apache.hadoop.fs.s3a.RenameFailedException
-
- renameFileToDest() - Method in class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
The source is a file: rename it to the destination, which
will be under the current destination path if that is a directory.
- RenameOperation - Class in org.apache.hadoop.fs.s3a.impl
-
A parallelized rename operation.
- RenameOperation(StoreContext, Path, String, S3AFileStatus, Path, String, S3AFileStatus, OperationCallbacks, int, boolean) - Constructor for class org.apache.hadoop.fs.s3a.impl.RenameOperation
-
Initiate the rename.
- representsEmptyDirectory(String) - Method in class org.apache.hadoop.fs.s3a.S3ListResult
-
Does this listing represent an empty directory?
- REQUEST_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
How long should the SDK retry/wait on a response from an S3 store: "fs.s3a.connection.request.timeout"
including the time needed to sign the request.
- requestCreated(SdkRequest.Builder) - Method in interface org.apache.hadoop.fs.s3a.audit.AWSAuditEventCallbacks
-
Callback when a request is created in the S3A code.
- requestCreated(SdkRequest.Builder) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
Audit the creation of a request and retrieve
a reference to the active thread span.
- REQUESTER_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "requester".
- REQUESTER_PAYS_HEADER - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Headers in request indicating that the requester must be charged for data
transfer.
- RequestFactory - Interface in org.apache.hadoop.fs.s3a.api
-
Factory for S3 request objects.
- RequestFactoryImpl - Class in org.apache.hadoop.fs.s3a.impl
-
The standard implementation of the request factory.
- RequestFactoryImpl(RequestFactoryImpl.RequestFactoryBuilder) - Constructor for class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
Constructor.
- RequestFactoryImpl.PrepareRequest - Interface in org.apache.hadoop.fs.s3a.impl
-
This is a callback for anything to "prepare" every request
after creation.
- RequestFactoryImpl.RequestFactoryBuilder - Class in org.apache.hadoop.fs.s3a.impl
-
Builder.
- requestId() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- REQUESTID_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "requestid".
- requestRole(String, String, String, long, TimeUnit) - Method in class org.apache.hadoop.fs.s3a.auth.STSClientFactory.STSClient
-
Request a set of role credentials.
- requestSessionCredentials(AwsCredentialsProvider, Configuration, String, String, int, Invoker, String) - Static method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentialBinding
-
Request a set of credentials from an STS endpoint.
- requestSessionCredentials(long, TimeUnit) - Method in class org.apache.hadoop.fs.s3a.auth.STSClientFactory.STSClient
-
Request a set of session credentials.
- REQUESTURI_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "requesturi".
- requiresDelayedCommitOutputInFileSystem() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Flag to indicate whether or not the destination filesystem needs
to be configured to support magic paths where the output isn't immediately
visible.
- requiresDelayedCommitOutputInFileSystem() - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
Require magic paths in the FS client.
- requireServiceStarted() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Require the service to be started.
- requireServiceState(Service.STATE) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
Require that the service is in a given state.
- requiresSecret() - Method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
Does this encryption algorithm require a secret?
- reset() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- resetBindings() - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Reset the store and filesystem bindings.
- resetBindings() - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
- resetConnection() - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
Forcibly reset the stream, by aborting the connection.
- resetCurrentAuditContext() - Method in class org.apache.hadoop.fs.s3a.commit.impl.AuditContextUpdater
-
Remove job/task info from the current audit context.
- resetMinimumOperationDuration() - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Reset the minimum operation duration to the default.
- resetTempFolderCache() - Static method in class org.apache.hadoop.fs.s3a.commit.staging.Paths
-
Reset the temp folder cache; useful in tests.
- resolveCredentials() - Method in class org.apache.hadoop.fs.s3a.adapter.V1ToV2AwsCredentialProviderAdapter
-
Collect v1 credentials and convert to v2.
- resolveCredentials() - Method in class org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider
-
- resolveCredentials() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider
-
Get the credentials.
- resolveCredentials() - Method in class org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
-
Get credentials.
- resolveCredentials() - Method in class org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
-
Ask for the credentials.
- resolveCredentials() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Iterate through the list of providers, to find one with credentials.
- resolveCredentials() - Method in class org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
-
- resource(String, String, boolean) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a resource.
- resource(Path, boolean, boolean) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Given a path, return the S3 resource to it.
- resource - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- REST_GET_POLICY_STATUS - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- REST_GET_PUBLIC_ACCESS_BLOCK - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- REST_GET_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- RESTORE - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Header for the optional restore information of an object.
- Retries - Class in org.apache.hadoop.fs.s3a
-
Annotations to inform the caller of an annotated method whether
the method performs retries and/or exception translation internally.
- Retries() - Constructor for class org.apache.hadoop.fs.s3a.Retries
-
- Retries.OnceExceptionsSwallowed - Annotation Type in org.apache.hadoop.fs.s3a
-
One attempt, Exceptions are swallowed.
- Retries.OnceMixed - Annotation Type in org.apache.hadoop.fs.s3a
-
No retry, expect a bit of both.
- Retries.OnceRaw - Annotation Type in org.apache.hadoop.fs.s3a
-
No retry, exceptions are not translated.
- Retries.OnceTranslated - Annotation Type in org.apache.hadoop.fs.s3a
-
No retry, exceptions are translated.
- Retries.RetryExceptionsSwallowed - Annotation Type in org.apache.hadoop.fs.s3a
-
Retried, Exceptions are swallowed.
- Retries.RetryMixed - Annotation Type in org.apache.hadoop.fs.s3a
-
Retried, mixed translation.
- Retries.RetryRaw - Annotation Type in org.apache.hadoop.fs.s3a
-
Retried, no translation.
- Retries.RetryTranslated - Annotation Type in org.apache.hadoop.fs.s3a
-
Retried, exceptions are translated.
- retrieveAttachedSpan(ExecutionAttributes) - Static method in class org.apache.hadoop.fs.s3a.audit.AuditIntegration
-
Get the span from the execution attributes.
- retrievePassword(AbstractS3ATokenIdentifier) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding.TokenSecretManager
-
- retry(String, String, boolean, Invoker.Retried, InvocationRaisingIOE) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a void operation with retry processing.
- retry(String, String, boolean, InvocationRaisingIOE) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a void operation with the default retry callback invoked.
- retry(String, String, boolean, CallableRaisingIOE<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with the default retry callback invoked.
- retry(String, String, boolean, Invoker.Retried, CallableRaisingIOE<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with retry processing.
- retry(String, String, boolean, CallableRaisingIOE<T>) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Execute a function with retry processing.
- retry(String, String, boolean, CallableRaisingIOE<T>) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Execute a function with retry processing.
- RETRY_HTTP_5XX_ERRORS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Should S3A connector retry on all 5xx errors which don't have
explicit support: "fs.s3a.retry.http.5xx.errors"?
- RETRY_INTERVAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Interval between retry attempts.: "fs.s3a.retry.interval".
- RETRY_INTERVAL_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default retry interval: "500ms".
- RETRY_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Number of times to retry any repeatable S3 client request on failure,
excluding throttling requests: "fs.s3a.retry.limit".
- RETRY_LIMIT_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default retry limit: 7.
- RETRY_THROTTLE_INTERVAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Interval between retry attempts on throttled requests: "fs.s3a.retry.throttle.interval".
- RETRY_THROTTLE_INTERVAL_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default throttled retry interval: "500ms".
- RETRY_THROTTLE_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Number of times to retry any throttled request: "fs.s3a.retry.throttle.limit".
- RETRY_THROTTLE_LIMIT_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default throttled retry limit: 20.
- retryable() - Method in exception org.apache.hadoop.fs.s3a.AWSClientIOException
-
Query inner cause for retryability.
- retryable() - Method in exception org.apache.hadoop.fs.s3a.AWSNoResponseException
-
- retryable() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceThrottledException
-
- retryable() - Method in exception org.apache.hadoop.fs.s3a.AWSUnsupportedFeatureException
-
- retryable() - Method in exception org.apache.hadoop.fs.s3a.CredentialInitializationException
-
This exception is not going to go away if you try calling it again.
- retryAwsClientExceptions - Variable in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Handling of AWSClientIOException and subclasses.
- retryIdempotentCalls - Variable in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Idempotent calls which raise IOEs are retried.
- retryUntranslated(String, boolean, CallableRaisingIOE<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with retry processing and no translation.
- retryUntranslated(String, boolean, Invoker.Retried, CallableRaisingIOE<T>) - Method in class org.apache.hadoop.fs.s3a.Invoker
-
Execute a function with retry processing: AWS SDK Exceptions
are not translated.
- revertCommit(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
- revertCommit(SinglePendingCommit) - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitOperations
-
Revert a pending commit by deleting the destination.
- revertCommit(String) - Method in class org.apache.hadoop.fs.s3a.WriteOperationHelper
-
Revert a commit by deleting the file.
- revertCommit(String) - Method in interface org.apache.hadoop.fs.s3a.WriteOperations
-
Revert a commit by deleting the file.
- ROLE_TOKEN_KIND - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
- ROLE_TOKEN_NAME - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Name of role token: "S3ADelegationToken/Role".
- RoleElt() - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.RoleElt
-
- RoleModel - Class in org.apache.hadoop.fs.s3a.auth
-
Jackson Role Model for Role Properties, for API clients and tests.
- RoleModel() - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel
-
- RoleModel.Effects - Enum in org.apache.hadoop.fs.s3a.auth
-
Effect options.
- RoleModel.Policy - Class in org.apache.hadoop.fs.s3a.auth
-
A policy is one or more statements.
- RoleModel.RoleElt - Class in org.apache.hadoop.fs.s3a.auth
-
Any element in a role.
- RoleModel.Statement - Class in org.apache.hadoop.fs.s3a.auth
-
A single statement.
- RolePolicies - Class in org.apache.hadoop.fs.s3a.auth
-
Operations, statements and policies covering the operations
needed to work with S3.
- RoleTokenBinding - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Role Token support requests an explicit role and automatically restricts
that role to the given policy of the binding.
- RoleTokenBinding() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
Constructor.
- RoleTokenIdentifier - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Role token identifier.
- RoleTokenIdentifier() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenIdentifier
-
- RoleTokenIdentifier(URI, Text, Text, MarshalledCredentials, EncryptionSecrets, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenIdentifier
-
- run(String[], PrintStream) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo
-
- run(String[]) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- run(String[], PrintStream) - Method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Run the tool, capturing the output (if the tool supports that).
- run(Configuration, String...) - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Execute the command with the given arguments.
- run(String[], PrintStream) - Method in class org.apache.hadoop.fs.s3a.tools.BucketTool
-
- run(String[], PrintStream) - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
- S3_ABORT_MULTIPART_UPLOAD - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Abort multipart upload is needed for the S3A Commit protocols.
- S3_ALL_BUCKETS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All S3 buckets: "arn:aws:s3:::*".
- S3_ALL_DELETE - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All s3:Delete* operations.
- S3_ALL_GET - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All s3:Get* operations.
- S3_ALL_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
All S3 operations: "s3:*".
- S3_ALL_PUT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
S3 Put*.
- S3_BUCKET_ALL_LIST - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_BUCKET_LIST_BUCKET - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
List the contents of a bucket.
- S3_BUCKET_LIST_MULTIPART_UPLOADS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
This is used by the abort operation in S3A commit work.
- S3_BUCKET_READ_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Policies which can be applied to bucket resources for read operations.
- S3_CLIENT_FACTORY_IMPL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Client Factory implementation class: "fs.s3a.s3.client.factory.impl".
- S3_DELETE_OBJECT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_DELETE_OBJECT_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_DELETE_OBJECT_VERSION - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_DELETE_OBJECT_VERSION_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_ENCRYPTION_ALGORITHM - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Set S3-server side encryption(SSE) or S3-Client side encryption(CSE)
algorithm.
- S3_ENCRYPTION_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Set S3-SSE or S3-CSE encryption Key if required.
- S3_EXPIRE_OBJECT - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogVerbs
-
- S3_GET_BUCKET_LOCATION - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_ACL - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_TORRENT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_VERSION - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_VERSION_ACL - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_VERSION_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_GET_OBJECT_VERSION_TORRENT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_LIST_MULTIPART_UPLOAD_PARTS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
List multipart upload is needed for the S3A Commit protocols.
- S3_METADATA_STORE_IMPL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3_PATH_RW_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Actions needed to write data to an S3A Path.
- S3_PATH_WRITE_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Actions needed to write data to an S3A Path.
- S3_PUT_OBJECT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_PUT_OBJECT_ACL - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_PUT_OBJECT_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_PUT_OBJECT_VERSION_ACL - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_PUT_OBJECT_VERSION_TAGGING - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_RESTORE_OBJECT - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_ROOT_READ_OPERATIONS_LIST - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
- S3_ROOT_RW_OPERATIONS - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Actions needed for R/W IO from the root of a bucket.
- S3_SELECT_CAPABILITY - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Does the FS Support S3 Select?
This is false everywhere.
- S3_V2_SIGNER - Static variable in class org.apache.hadoop.fs.s3a.auth.SignerFactory
-
The v2 signer is no longer available: "S3SignerType".
- S3_VERSION_ID - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
S3's version ID header.
- S3A - Class in org.apache.hadoop.fs.s3a
-
S3A implementation of AbstractFileSystem.
- S3A(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.S3A
-
- S3A_BUCKET_PROBE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Bucket validation parameter which can be set by client.
- S3A_BUCKET_PROBE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default value of bucket validation parameter.
- S3A_COMMITTER_EXPERIMENTAL_COLLECT_IOSTATISTICS - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Experimental feature to collect thread level IO statistics.
- S3A_COMMITTER_EXPERIMENTAL_COLLECT_IOSTATISTICS_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
- S3A_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
S3 Committer factory: "org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory".
- S3A_COMMITTER_FACTORY_KEY - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Key to set for the S3A schema to use the specific committer.
- S3A_DEFAULT_PORT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Default port for this is 443: HTTPS.
- S3A_DYNAMIC_CAPABILITIES - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Dynamic Path capabilities to be evaluated
in the BucketInfo tool.
- S3A_IO_RATE_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Config to set the rate limit for S3A IO operations.
- S3A_OPENFILE_KEYS - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
The known keys used in a standard openFile call.
- S3A_SECURITY_CREDENTIAL_PROVIDER_PATH - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Extra set of security credentials which will be prepended to that
set in "hadoop.security.credential.provider.path".
- S3AAuditConstants - Class in org.apache.hadoop.fs.s3a.audit
-
- S3ABlockManager - Class in org.apache.hadoop.fs.s3a.prefetch
-
Provides read access to S3 file one block at a time.
- S3ABlockManager(S3ARemoteObjectReader, BlockData) - Constructor for class org.apache.hadoop.fs.s3a.prefetch.S3ABlockManager
-
Constructs an instance of S3ABlockManager.
- S3ACachingBlockManager - Class in org.apache.hadoop.fs.s3a.prefetch
-
Provides access to S3 file one block at a time.
- S3ACachingBlockManager(BlockManagerParameters, S3ARemoteObjectReader) - Constructor for class org.apache.hadoop.fs.s3a.prefetch.S3ACachingBlockManager
-
Constructs an instance of a S3ACachingBlockManager.
- S3ACachingInputStream - Class in org.apache.hadoop.fs.s3a.prefetch
-
Provides an InputStream that allows reading from an S3 file.
- S3ACachingInputStream(S3AReadOpContext, S3ObjectAttributes, S3AInputStream.InputStreamCallbacks, S3AInputStreamStatistics, Configuration, LocalDirAllocator) - Constructor for class org.apache.hadoop.fs.s3a.prefetch.S3ACachingInputStream
-
Initializes a new instance of the S3ACachingInputStream class.
- S3ACommitterFactory - Class in org.apache.hadoop.fs.s3a.commit
-
- S3ACommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory
-
- S3ADataBlocks - Class in org.apache.hadoop.fs.s3a
-
Set of classes to support output streaming into blocks which are then
uploaded as to S3 as a single PUT, or as part of a multipart request.
- S3ADataBlocks.BlockFactory - Class in org.apache.hadoop.fs.s3a
-
Base class for block factories.
- S3ADataBlocks.BlockUploadData - Class in org.apache.hadoop.fs.s3a
-
The output information for an upload.
- S3ADelegationTokens - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
Support for creating a DT from a filesystem.
- S3ADelegationTokens() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Instantiate.
- S3ADelegationTokens.TokenIssuingPolicy - Enum in org.apache.hadoop.fs.s3a.auth.delegation
-
- S3ADtFetcher - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
A DT fetcher for S3A.
- S3ADtFetcher() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.S3ADtFetcher
-
- S3AEncryptionMethods - Enum in org.apache.hadoop.fs.s3a
-
This enum is to centralize the encryption methods and
the value required in the configuration.
- S3AFileStatus - Class in org.apache.hadoop.fs.s3a
-
File status for an S3A "file".
- S3AFileStatus(boolean, Path, String) - Constructor for class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Create a directory status.
- S3AFileStatus(Tristate, Path, String) - Constructor for class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Create a directory status.
- S3AFileStatus(long, long, Path, long, String, String, String) - Constructor for class org.apache.hadoop.fs.s3a.S3AFileStatus
-
A simple file.
- S3AFileSystem - Class in org.apache.hadoop.fs.s3a
-
The core S3A Filesystem implementation.
- S3AFileSystem() - Constructor for class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- S3AFileSystem.CopyFromLocalCallbacksImpl - Class in org.apache.hadoop.fs.s3a
-
- S3AFileSystem.GetContentSummaryCallbacksImpl - Class in org.apache.hadoop.fs.s3a
-
- S3AFileSystem.HeaderProcessingCallbacksImpl - Class in org.apache.hadoop.fs.s3a
-
Operations needed for Header Processing.
- S3AFileSystem.ListingOperationCallbacksImpl - Class in org.apache.hadoop.fs.s3a
-
- S3AFileSystem.MkdirOperationCallbacksImpl - Class in org.apache.hadoop.fs.s3a
-
- S3AInMemoryInputStream - Class in org.apache.hadoop.fs.s3a.prefetch
-
Provides an InputStream that allows reading from an S3 file.
- S3AInMemoryInputStream(S3AReadOpContext, S3ObjectAttributes, S3AInputStream.InputStreamCallbacks, S3AInputStreamStatistics) - Constructor for class org.apache.hadoop.fs.s3a.prefetch.S3AInMemoryInputStream
-
Initializes a new instance of the S3AInMemoryInputStream class.
- S3AInputPolicy - Enum in org.apache.hadoop.fs.s3a
-
Stream input policy.
- S3AInputStream - Class in org.apache.hadoop.fs.s3a
-
The input stream for an S3A object.
- S3AInputStream(S3AReadOpContext, S3ObjectAttributes, S3AInputStream.InputStreamCallbacks, S3AInputStreamStatistics, ExecutorService) - Constructor for class org.apache.hadoop.fs.s3a.S3AInputStream
-
Create the stream.
- S3AInputStream.InputStreamCallbacks - Interface in org.apache.hadoop.fs.s3a
-
Callbacks for input stream IO.
- S3AInputStreamStatistics - Interface in org.apache.hadoop.fs.s3a.statistics
-
- S3AInstrumentation - Class in org.apache.hadoop.fs.s3a
-
Instrumentation of S3A.
- S3AInstrumentation(URI) - Constructor for class org.apache.hadoop.fs.s3a.S3AInstrumentation
-
Construct the instrumentation for a filesystem.
- S3AInternalAuditConstants - Class in org.apache.hadoop.fs.s3a.audit.impl
-
Internal constants; not intended for public use, or
for use by any external implementations.
- S3AInternals - Interface in org.apache.hadoop.fs.s3a
-
This is an unstable interface for access to S3A Internal state, S3 operations
and the S3 client connector itself.
- S3ALocatedFileStatus - Class in org.apache.hadoop.fs.s3a
-
LocatedFileStatus extended to also carry ETag and object version ID.
- S3ALocatedFileStatus(S3AFileStatus, BlockLocation[]) - Constructor for class org.apache.hadoop.fs.s3a.S3ALocatedFileStatus
-
- S3AMultipartUploaderBuilder - Class in org.apache.hadoop.fs.s3a.impl
-
Builder for S3A multipart uploaders.
- S3AMultipartUploaderBuilder(S3AFileSystem, WriteOperations, StoreContext, Path, S3AMultipartUploaderStatistics) - Constructor for class org.apache.hadoop.fs.s3a.impl.S3AMultipartUploaderBuilder
-
- S3AMultipartUploaderStatistics - Interface in org.apache.hadoop.fs.s3a.statistics
-
Statistics for the S3A multipart uploader.
- S3AMultipartUploaderStatisticsImpl - Class in org.apache.hadoop.fs.s3a.statistics.impl
-
Implementation of the uploader statistics.
- S3AMultipartUploaderStatisticsImpl(BiConsumer<Statistic, Long>) - Constructor for class org.apache.hadoop.fs.s3a.statistics.impl.S3AMultipartUploaderStatisticsImpl
-
Constructor.
- S3AOpContext - Class in org.apache.hadoop.fs.s3a
-
Class for operation context struct passed through codepaths for main
S3AFileSystem operations.
- S3AOpContext(Invoker, FileSystem.Statistics, S3AStatisticsContext, FileStatus) - Constructor for class org.apache.hadoop.fs.s3a.S3AOpContext
-
Constructor.
- S3APrefetchingInputStream - Class in org.apache.hadoop.fs.s3a.prefetch
-
Enhanced InputStream for reading from S3.
- S3APrefetchingInputStream(S3AReadOpContext, S3ObjectAttributes, S3AInputStream.InputStreamCallbacks, S3AInputStreamStatistics, Configuration, LocalDirAllocator) - Constructor for class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Initializes a new instance of the S3APrefetchingInputStream class.
- S3AReadOpContext - Class in org.apache.hadoop.fs.s3a
-
Read-specific operation context struct.
- S3AReadOpContext(Path, Invoker, FileSystem.Statistics, S3AStatisticsContext, FileStatus, VectoredIOContext, IOStatisticsAggregator, ExecutorServiceFuturePool, int, int) - Constructor for class org.apache.hadoop.fs.s3a.S3AReadOpContext
-
Instantiate.
- S3ARemoteInputStream - Class in org.apache.hadoop.fs.s3a.prefetch
-
Provides an
InputStream that allows reading from an S3 file.
- S3ARemoteInputStream(S3AReadOpContext, S3ObjectAttributes, S3AInputStream.InputStreamCallbacks, S3AInputStreamStatistics) - Constructor for class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Initializes a new instance of the S3ARemoteInputStream class.
- S3ARemoteObject - Class in org.apache.hadoop.fs.s3a.prefetch
-
Encapsulates low level interactions with S3 object on AWS.
- S3ARemoteObject(S3AReadOpContext, S3ObjectAttributes, S3AInputStream.InputStreamCallbacks, S3AInputStreamStatistics, ChangeTracker) - Constructor for class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObject
-
Initializes a new instance of the S3ARemoteObject class.
- S3ARemoteObjectReader - Class in org.apache.hadoop.fs.s3a.prefetch
-
Provides functionality to read S3 file one block at a time.
- S3ARemoteObjectReader(S3ARemoteObject) - Constructor for class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObjectReader
-
- S3ARetryPolicy - Class in org.apache.hadoop.fs.s3a
-
The S3A request retry policy.
- S3ARetryPolicy(Configuration) - Constructor for class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
Instantiate.
- S3AStatisticInterface - Interface in org.apache.hadoop.fs.s3a.statistics
-
Base Interface for S3A Statistics, as a way of managing
them.
- S3AStatisticsContext - Interface in org.apache.hadoop.fs.s3a.statistics
-
This is the statistics context for ongoing operations in S3A.
- S3AStorageStatistics - Class in org.apache.hadoop.fs.s3a
-
Storage statistics for S3A, dynamically generated from the IOStatistics.
- S3AStorageStatistics(IOStatistics) - Constructor for class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- S3AStorageStatistics() - Constructor for class org.apache.hadoop.fs.s3a.S3AStorageStatistics
-
- S3AStore - Interface in org.apache.hadoop.fs.s3a
-
Interface for the S3A Store;
S3 client interactions should be via this; mocking
is possible for unit tests.
- S3AStoreBuilder - Class in org.apache.hadoop.fs.s3a.impl
-
Builder for the S3AStore.
- S3AStoreBuilder() - Constructor for class org.apache.hadoop.fs.s3a.impl.S3AStoreBuilder
-
- S3AStoreImpl - Class in org.apache.hadoop.fs.s3a.impl
-
Store Layer.
- S3AUtils - Class in org.apache.hadoop.fs.s3a
-
Utility methods for S3A code.
- S3AUtils.CallOnLocatedFileStatus - Interface in org.apache.hadoop.fs.s3a
-
An interface for use in lambda-expressions working with
directory tree listings.
- S3AUtils.LocatedFileStatusMap<T> - Interface in org.apache.hadoop.fs.s3a
-
An interface for use in lambda-expressions working with
directory tree listings.
- S3ClientCreationParameters() - Constructor for class org.apache.hadoop.fs.s3a.S3ClientFactory.S3ClientCreationParameters
-
- S3ClientFactory - Interface in org.apache.hadoop.fs.s3a
-
Factory for creation of S3Client client instances.
- S3ClientFactory.S3ClientCreationParameters - Class in org.apache.hadoop.fs.s3a
-
Settings for the S3 Client.
- S3EXPRESS_CREATE_SESSION - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Use the S3 Express createSession() operation to authenticate with
S3Express storage?
- S3EXPRESS_CREATE_SESSION_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- S3EXPRESS_CREATE_SESSION_POLICY - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
S3Express session permission; required unless sessions are disabled.
- S3EXPRESS_STORE_SUFFIX - Static variable in class org.apache.hadoop.fs.s3a.impl.S3ExpressStorage
-
Suffix of S3Express storage bucket names..
- S3ExpressStorage - Class in org.apache.hadoop.fs.s3a.impl
-
Anything needed to support Amazon S3 Express One Zone Storage.
- S3Guard - Class in org.apache.hadoop.fs.s3a.s3guard
-
Logic for integrating MetadataStore with S3A.
- S3GUARD_CLI_PRUNE_AGE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_CONSISTENCY_RETRY_INTERVAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_CONSISTENCY_RETRY_INTERVAL_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_CONSISTENCY_RETRY_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_CONSISTENCY_RETRY_LIMIT_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_BACKGROUND_SLEEP_MSEC_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_BACKGROUND_SLEEP_MSEC_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_BATCH_WRITE_REQUEST_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_MAX_RETRIES - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_MAX_RETRIES_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_REGION_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_TABLE_CAPACITY_READ_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_TABLE_CAPACITY_READ_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_TABLE_CAPACITY_WRITE_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_TABLE_CAPACITY_WRITE_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_TABLE_CREATE_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_TABLE_NAME_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_TABLE_SSE_CMK - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_TABLE_SSE_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_TABLE_TAG - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_THROTTLE_RETRY_INTERVAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DDB_THROTTLE_RETRY_INTERVAL_DEFAULT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_DISABLED_WARN_LEVEL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_METASTORE_DYNAMO - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_METASTORE_LOCAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_METASTORE_LOCAL_ENTRY_TTL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_METASTORE_LOCAL_MAX_RECORDS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GUARD_METASTORE_NULL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- S3GuardTool - Class in org.apache.hadoop.fs.s3a.s3guard
-
CLI to manage S3Guard Metadata Store.
- S3GuardTool(Configuration, String...) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Constructor a S3Guard tool with HDFS configuration.
- S3GuardTool(Configuration, int, int, String...) - Constructor for class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Constructor a S3Guard tool with HDFS configuration.
- S3GuardTool.BucketInfo - Class in org.apache.hadoop.fs.s3a.s3guard
-
Get info about a bucket and its S3Guard integration status.
- s3guardUnsupported() - Static method in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
Crate an exception declaring S3Guard is unsupported.
- S3ListRequest - Class in org.apache.hadoop.fs.s3a
-
API version-independent container for S3 List requests.
- S3ListResult - Class in org.apache.hadoop.fs.s3a
-
API version-independent container for S3 List responses.
- S3ListResult(ListObjectsResponse, ListObjectsV2Response) - Constructor for class org.apache.hadoop.fs.s3a.S3ListResult
-
- S3LogParser - Class in org.apache.hadoop.fs.s3a.audit
-
Class to help parse AWS S3 Logs.
- S3LogVerbs - Class in org.apache.hadoop.fs.s3a.audit
-
Various verbs in the S3 logs.
- S3MagicCommitTracker - Class in org.apache.hadoop.fs.s3a.commit.magic
-
Stores the commit data under the magic path.
- S3MagicCommitTracker(Path, String, String, String, String, WriteOperationHelper, PutTrackerStatistics) - Constructor for class org.apache.hadoop.fs.s3a.commit.magic.S3MagicCommitTracker
-
- S3N_FOLDER_SUFFIX - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- S3ObjectAttributes - Class in org.apache.hadoop.fs.s3a
-
This class holds attributes of an object independent of the
file status type.
- S3ObjectAttributes(String, Path, String, S3AEncryptionMethods, String, String, String, long) - Constructor for class org.apache.hadoop.fs.s3a.S3ObjectAttributes
-
Constructor.
- S3xLoginHelper - Class in org.apache.hadoop.fs.s3native
-
Class to aid logging in to S3 endpoints.
- S3xLoginHelper.Login - Class in org.apache.hadoop.fs.s3native
-
Simple tuple of login details.
- save(FileSystem, Path, JsonSerialization<PendingSet>) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- save(FileSystem, Path, JsonSerialization<T>) - Method in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Save to a hadoop filesystem.
- save(FileSystem, Path, JsonSerialization<SinglePendingCommit>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- save(FileSystem, Path, JsonSerialization<SuccessData>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- saveFile(FileSystem, Path, T, JsonSerialization<T>, boolean) - Static method in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Save to a file.
- saveToStream(Path, T, FSDataOutputStreamBuilder, JsonSerialization<T>) - Static method in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Save to a file.
- SC_200_OK - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
200 status code: OK.
- SC_301_MOVED_PERMANENTLY - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
301 status code: Moved Permanently.
- SC_307_TEMPORARY_REDIRECT - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
307 status code: Temporary Redirect.
- SC_400_BAD_REQUEST - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
400 status code: Bad Request.
- SC_401_UNAUTHORIZED - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
401 status code: Unauthorized.
- SC_403 - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Deprecated.
- SC_403_FORBIDDEN - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
403 status code: Forbidden.
- SC_404 - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Deprecated.
- SC_404_NOT_FOUND - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
404 status code: Not Found.
- SC_405_METHOD_NOT_ALLOWED - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
405 status code: Method Not Allowed.
- SC_409_CONFLICT - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
409 status code: Conflict.
- SC_410_GONE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
410 status code: Gone.
- SC_412_PRECONDITION_FAILED - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
412 status code: Precondition Failed.
- SC_415_UNSUPPORTED_MEDIA_TYPE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
415 status code: Content type unsupported by this store.
- SC_416_RANGE_NOT_SATISFIABLE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
416 status code: Range Not Satisfiable.
- SC_429_TOO_MANY_REQUESTS_GCS - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
429 status code: This is the google GCS throttle message.
- SC_443_NO_RESPONSE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
443 status code: No Response (unofficial).
- SC_444_NO_RESPONSE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
444 status code: No Response (unofficial).
- SC_500_INTERNAL_SERVER_ERROR - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
500 status code: Internal Server Error.
- SC_501_NOT_IMPLEMENTED - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
501 status code: method not implemented.
- SC_503_SERVICE_UNAVAILABLE - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
503 status code: Service Unavailable.
- SC_504_GATEWAY_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
504 Gateway Timeout.
- ScanArgsBuilder() - Constructor for class org.apache.hadoop.fs.s3a.tools.MarkerTool.ScanArgsBuilder
-
- SDK_V2_UPGRADE_LOG - Static variable in class org.apache.hadoop.fs.s3a.impl.V2Migration
-
- SDK_V2_UPGRADE_LOG_NAME - Static variable in class org.apache.hadoop.fs.s3a.impl.InternalConstants
-
Name of the log for events related to the SDK V2 upgrade.
- SDKStreamDrainer<TStream extends InputStream & software.amazon.awssdk.http.Abortable> - Class in org.apache.hadoop.fs.s3a.impl
-
Drains/aborts s3 or other AWS SDK streams.
- SDKStreamDrainer(String, TStream, boolean, int, S3AInputStreamStatistics, String) - Constructor for class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
Prepare to drain the stream.
- SECONDS_FLAG - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- SECRET_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- secretAccessKey() - Method in class org.apache.hadoop.fs.s3a.auth.AbstractSessionCredentialsProvider.NoCredentials
-
- SECURE_CONNECTIONS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- seek(long) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Updates internal data such that the next read will take place at the given pos.
- seek(long) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Moves the current read position so that the next read will occur at pos.
- seek(long) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- seekBackwards(long) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
Seek backwards, incrementing the seek and backward seek counters.
- seekForwards(long, long) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
Record a forward seek, adding a seek operation, a forward
seek operation, and any bytes skipped.
- seekToNewSource(long) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
- seekToNewSource(long) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- SELECT_ERRORS_INCLUDE_SQL - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Should Select errors include the SQL statement?
It is easier to debug but a security risk if the exceptions
ever get printed/logged and the query contains secrets.
- SELECT_FORMAT_CSV - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
CSV as an input or output format: "csv".
- SELECT_FORMAT_JSON - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
JSON as an input or output format: "json".
- SELECT_INPUT_COMPRESSION - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
How is the input compressed? This applies to all formats.
- SELECT_INPUT_FORMAT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Input format for data.
- SELECT_OUTPUT_FORMAT - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
Output format for data -that is, what the results are generated
as.
- SELECT_SQL - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
This is the big SQL expression: "fs.s3a.select.sql".
- SELECT_UNSUPPORTED - Static variable in class org.apache.hadoop.fs.s3a.select.SelectConstants
-
Deprecated.
- SelectConstants - Class in org.apache.hadoop.fs.s3a.select
-
Deprecated.
- selectTokenFromFSOwner() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Find a token for the FS user and canonical filesystem URI.
- serializer() - Static method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Get a shared JSON serializer for this class.
- serializer() - Static method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
Get a JSON serializer for this class.
- serializer() - Static method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Get a JSON serializer for this class.
- SERVER_SIDE_ENCRYPTION - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Header for optional server-side encryption algorithm.
- SERVER_SIDE_ENCRYPTION_AES256 - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- SERVER_SIDE_ENCRYPTION_ALGORITHM - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- SERVER_SIDE_ENCRYPTION_AWS_KMS_KEY_ID - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Header for optional server-side encryption algorithm.
- SERVER_SIDE_ENCRYPTION_KEY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- serviceInit(Configuration) - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- serviceInit(Configuration) - Method in class org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor
-
Service init, look for jobID and attach as an attribute in log entries.
- serviceInit(Configuration) - Method in class org.apache.hadoop.fs.s3a.audit.impl.NoopAuditManagerS3A
-
- serviceInit(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDTService
-
- serviceInit(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.RoleTokenBinding
-
- serviceInit(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Init the service.
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
After starting the auditor, it is queried for its
unbonded span, which is then wrapped and stored for
use.
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding
-
Service startup: create the secret manager.
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.FullCredentialsTokenBinding
-
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Service startup includes binding to any delegation token, and
deploying unbounded if there is none.
- serviceStart() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Service start will read in all configuration options
then build that client.
- serviceStop() - Method in class org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A
-
- serviceStop() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens
-
Stop the token binding.
- serviceStop() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- SESSION_TOKEN - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- SESSION_TOKEN - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
session token for when using TemporaryAWSCredentialsProvider: : "fs.s3a.session.token".
- SESSION_TOKEN_KIND - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
- SESSION_TOKEN_NAME - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
Name of session token: "S3ADelegationToken/Session".
- SessionTokenBinding - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
The session token DT binding: creates an AWS session token
for the DT, extracts and serves it up afterwards.
- SessionTokenBinding() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Constructor for reflection.
- SessionTokenBinding(String, Text) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
Constructor for subclasses.
- SessionTokenIdentifier - Class in org.apache.hadoop.fs.s3a.auth.delegation
-
A token identifier which contains a set of AWS session credentials,
credentials which will be valid until they expire.
- SessionTokenIdentifier() - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Constructor for service loader use.
- SessionTokenIdentifier(Text) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Constructor for subclasses.
- SessionTokenIdentifier(Text, Text, Text, URI, MarshalledCredentials, EncryptionSecrets, String) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Constructor.
- SessionTokenIdentifier(Text, Text, Text, Text, URI) - Constructor for class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenIdentifier
-
Constructor.
- setAccessKey(String) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setAllowed(boolean) - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- setAmazonS3Client(S3Client) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Set the client -used in mocking tests to force in a different client.
- setAuditor(AuditSpanSource<AuditSpanS3A>) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
Set builder value.
- setBucket(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setBucket(String) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setBucket(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Set the bucket.
- setBucketOption(Configuration, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Set a bucket-specific property to a particular value.
- setChangeDetectionPolicy(ChangeDetectionPolicy) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setCommits(List<SinglePendingCommit>) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- setCommitter(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setConf(Configuration) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
- setConfiguration(Configuration) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setContextAccessors(ContextAccessors) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setCreated(long) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setCurrentStream(T) - Method in class org.apache.hadoop.fs.s3a.impl.UploadContentProviders.BaseContentProvider
-
Set the current stream.
- setDate(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setDate(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setDescription(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setDestFS(FileSystem) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Set the destination FS: the FS of the final output.
- setDestinationKey(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setDiagnostics(Map<String, String>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setDurationAsMillis(Configuration, String, Duration) - Static method in class org.apache.hadoop.fs.s3a.impl.ConfigurationHelper
-
Set a duration as a time in milliseconds, with the suffix "ms" added.
- setDurationAsSeconds(Configuration, String, Duration) - Static method in class org.apache.hadoop.fs.s3a.impl.ConfigurationHelper
-
Set a duration as a time in seconds, with the suffix "s" added.
- setEnableCSE(boolean) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
set is client side encryption boolean value.
- setEncryptionSecrets(EncryptionSecrets) - Method in interface org.apache.hadoop.fs.s3a.api.RequestFactory
-
Set the encryption secrets for all subsequent requests.
- setEncryptionSecrets(EncryptionSecrets) - Method in class org.apache.hadoop.fs.s3a.impl.RequestFactoryImpl
-
- setEncryptionSecrets(EncryptionSecrets) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Set the encryption secrets for requests.
- setEtags(List<String>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setExecutor(ExecutorService) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setExecutorCapacity(int) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setExpiration(long) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setExpirationDateTime(Optional<OffsetDateTime>) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- setExtraData(Map<String, String>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setFailureLimit(int) - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
- setFilename(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setFilenames(List<String>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setFsURI(URI) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setHostname(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setIfDefined(Configuration, String, String, String) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Set a key if the value is non-empty.
- setInputPolicy(S3AInputPolicy) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setInputPolicy(S3AInputPolicy) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- setInstrumentation(S3AStatisticsContext) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setInvoker(Invoker) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setIOStatistics(IOStatisticsSnapshot) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- setIOStatistics(IOStatisticsSnapshot) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setIOStatistics(IOStatisticsSnapshot) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setIOStatistics(IOStatisticsStore) - Method in class org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource
-
Setter.
- setIsEmptyDirectory(Tristate) - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
Update isEmptyDirectory attribute.
- setJobId(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- setJobId(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setJobId(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setJobIdSource(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setLength(long) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setMaxReadSizeForVectoredReads(int) - Method in class org.apache.hadoop.fs.s3a.VectoredIOContext
-
- setMetrics(Map<String, Long>) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setMinimumOperationDuration(Duration) - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Set the minimum operation duration.
- setMinSeekForVectoredReads(int) - Method in class org.apache.hadoop.fs.s3a.VectoredIOContext
-
- setMultiObjectDeleteEnabled(boolean) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setName(String) - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Set the name; adds a ": " if needed.
- setName(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setOrigin(String) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
- setOutputPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Set the output path.
- setOwner(UserGroupInformation) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setParentAuthChain(AWSCredentialProviderList) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- setPerformanceFlags(FlagSet<PerformanceFlagEnum>) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setReadahead(Long) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3APrefetchingInputStream
-
Sets the number of bytes to read ahead each time.
- setReadahead(Long) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
Sets the number of bytes to read ahead each time.
- setReadahead(Long) - Method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
- setRequestTimeout(AwsRequest.Builder, Duration) - Static method in class org.apache.hadoop.fs.s3a.impl.AWSClientConfig
-
Set a custom ApiCallTimeout for a single request.
- setRoleARN(String) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setSaved(long) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setSecretKey(String) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setSecretsInConfiguration(Configuration) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Patch a configuration with the secrets.
- setSessionToken(String) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
- setState(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setStorageStatistics(S3AStorageStatistics) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setSuccess(boolean) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Set the success flag.
- setTaskId(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setText(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setThrottleProbability(float) - Method in class org.apache.hadoop.fs.s3a.FailureInjectionPolicy
-
Set the probability of throttling a request.
- setTimestamp(long) - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- setTokenIdentifier(Optional<SessionTokenIdentifier>) - Method in class org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding
-
- setupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Base job setup (optionally) deletes the success marker and
always creates the destination directory.
- setupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter
-
- setupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitter
-
- setupJob(JobContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Set up the job, including calling the same method on the
wrapped committer.
- setUploadId(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setupTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Task setup.
- setupTask(TaskAttemptContext) - Method in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
- setUri(String) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setUri(URI, boolean) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
- setUseListV1(boolean) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setUsername(String) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- setVerbose(boolean) - Method in class org.apache.hadoop.fs.s3a.tools.MarkerTool
-
- setVersion(int) - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
- setVersion(int) - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- setVersionId(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileStatus
-
set the S3 object versionId, else null.
- setWorkingDirectory(Path) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Set the current working directory for the given file system.
- setWorkingDirectory(Path) - Method in class org.apache.hadoop.fs.s3native.NativeS3FileSystem
-
Deprecated.
- setWorkPath(Path) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Set the work path for this committer.
- share() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Get a reference to this object with an updated reference count.
- shareCredentials(String) - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Get a shared copy of the AWS credentials, with its reference
counter updated.
- shareCredentials(String) - Method in interface org.apache.hadoop.fs.s3a.S3AInternals
-
Get a shared copy of the AWS credentials, with its reference
counter updated.
- SharedInstanceCredentialProvider - Class in org.apache.hadoop.fs.s3a
-
This credential provider has jittered between existing and non-existing,
but it turns up in documentation enough that it has been restored.
- SharedInstanceCredentialProvider() - Constructor for class org.apache.hadoop.fs.s3a.SharedInstanceCredentialProvider
-
- shouldAbort() - Method in class org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer
-
- shouldRetry(Exception, int, int, boolean) - Method in class org.apache.hadoop.fs.s3a.S3ARetryPolicy
-
- sid - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- sign(SignRequest<? extends AwsCredentialsIdentity>) - Method in class org.apache.hadoop.fs.s3a.auth.CustomHttpSigner
-
- sign(SdkHttpFullRequest, ExecutionAttributes) - Method in class org.apache.hadoop.fs.s3a.auth.CustomSdkSigner
-
Method to sign the incoming request with credentials.
- signAsync(AsyncSignRequest<? extends AwsCredentialsIdentity>) - Method in class org.apache.hadoop.fs.s3a.auth.CustomHttpSigner
-
- SignerFactory - Class in org.apache.hadoop.fs.s3a.auth
-
Signer factory used to register and create signers.
- SignerManager - Class in org.apache.hadoop.fs.s3a.auth
-
Class to handle custom signers.
- SignerManager(String, DelegationTokenProvider, Configuration, UserGroupInformation) - Constructor for class org.apache.hadoop.fs.s3a.auth.SignerManager
-
- SIGNING_ALGORITHM - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Multiple parameters can be used to specify a non-default signing
algorithm.
fs.s3a.signing-algorithm - This property has existed for the longest time.
- SIGNING_ALGORITHM_DDB - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- SIGNING_ALGORITHM_S3 - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SIGNING_ALGORITHM_STS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SIGV_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "sigv".
- SimpleAWSCredentialsProvider - Class in org.apache.hadoop.fs.s3a
-
Support simple credentials for authenticating with AWS.
- SimpleAWSCredentialsProvider(URI, Configuration) - Constructor for class org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
-
Build the credentials from a filesystem URI and configuration.
- SinglePendingCommit - Class in org.apache.hadoop.fs.s3a.commit.files
-
This is the serialization format for uploads yet to be committed.
- SinglePendingCommit() - Constructor for class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- size() - Method in class org.apache.hadoop.fs.s3a.AWSCredentialProviderList
-
Get the size of this list.
- size() - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.ActiveCommit
-
- size() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Number of commits.
- size() - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteObject
-
Gets the size of this file.
- skip(long) - Method in class org.apache.hadoop.fs.s3a.prefetch.S3ARemoteInputStream
-
- SOCKET_RECV_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SOCKET_SEND_BUFFER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- SOCKET_TIMEOUT - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Milliseconds until we give up on a connection to s3: "fs.s3a.connection.timeout".
- SPARK_WRITE_UUID - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
The UUID for jobs: "spark.sql.sources.writeJobUUID".
- splitPathToElements(Path) - Static method in class org.apache.hadoop.fs.s3a.commit.MagicCommitPaths
-
Take an absolute path, split it into a list of elements.
- SSE_C_NO_KEY_ERROR - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
Encryption SSE-C used but the config lacks an encryption key.
- SSE_S3_WITH_KEY_ERROR - Static variable in class org.apache.hadoop.fs.s3a.S3AUtils
-
Encryption SSE-S3 is used but the caller also set an encryption key.
- SSL_CHANNEL_MODE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
- STAGING_COMMITTER_FACTORY - Static variable in class org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants
-
Directory committer factory: "org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory".
- STAGING_UPLOADS - Static variable in class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterConstants
-
Filename used under ~/${UUID} for the staging files.
- StagingCommitter - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Committer based on the contributed work of the
Netflix multipart committers.
The working directory of each task is actually under a temporary
path in the local filesystem; jobs write directly into it.
- StagingCommitter(Path, TaskAttemptContext) - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter
-
Committer for a single task attempt.
- StagingCommitterConstants - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Internal staging committer constants.
- StagingCommitterFactory - Class in org.apache.hadoop.fs.s3a.commit.staging
-
Factory for the staging committer.
- StagingCommitterFactory() - Constructor for class org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory
-
- STANDARD_AWS_PROVIDERS - Static variable in class org.apache.hadoop.fs.s3a.auth.CredentialProviderListFactory
-
The standard AWS provider list for AWS connections.
- startOperation(String, String, String) - Method in class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter
-
Start an operation; retrieve an audit span.
- statement - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
- statement(boolean, String, String...) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a statement.
- statement(boolean, String, Collection<String>) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a statement.
- statement(boolean, Path, boolean, boolean, String...) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a statement.
- statement(boolean, Path, boolean, boolean, Collection<String>) - Static method in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
Create a statement.
- Statement(RoleModel.Effects) - Constructor for class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- STATEMENT_ALL_S3 - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Allow all S3 Operations.
- STATEMENT_ALL_S3_GET_BUCKET_LOCATION - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
The s3:GetBucketLocation permission is for all buckets, not for
any named bucket, which complicates permissions.
- STATEMENT_ALLOW_KMS_RW - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Statement to allow KMS R/W access access, so full use of
SSE-KMS.
- STATEMENT_ALLOW_SSE_KMS_READ - Static variable in class org.apache.hadoop.fs.s3a.auth.RolePolicies
-
Statement to allow read access to KMS keys, so the ability
to read SSE-KMS data,, but not decrypt it.
- Statistic - Enum in org.apache.hadoop.fs.s3a
-
Statistic which are collected in S3A.
- StatisticsFromAwsSdk - Interface in org.apache.hadoop.fs.s3a.statistics
-
- StatisticsFromAwsSdkImpl - Class in org.apache.hadoop.fs.s3a.statistics.impl
-
Hook up AWS SDK Statistics to the S3 counters.
- StatisticsFromAwsSdkImpl(CountersAndGauges) - Constructor for class org.apache.hadoop.fs.s3a.statistics.impl.StatisticsFromAwsSdkImpl
-
- StatisticTypeEnum - Enum in org.apache.hadoop.fs.s3a.statistics
-
Enum of statistic types.
- STATUS_CODE - Static variable in exception org.apache.hadoop.fs.s3a.AWSBadRequestException
-
HTTP status code which signals this failure mode was triggered: 400.
- STATUS_CODE - Static variable in exception org.apache.hadoop.fs.s3a.AWSServiceThrottledException
-
HTTP status code which signals this failure mode was triggered: 503.
- statusCode() - Method in exception org.apache.hadoop.fs.s3a.AWSServiceIOException
-
- StatusProbeEnum - Enum in org.apache.hadoop.fs.s3a.impl
-
Enum of probes which can be made of S3.
- stopAllServices() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Stop all services.
- STORAGE_CLASS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 storage class: standard, reduced_redundancy, intelligent_tiering etc.
- STORAGE_CLASS - Static variable in interface org.apache.hadoop.fs.s3a.impl.AWSHeaders
-
Header describing what class of storage a user wants.
- STORAGE_CLASS_DEEP_ARCHIVE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Storage option: "deep_archive".
- STORAGE_CLASS_GLACIER - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Storage option: "glacier".
- STORAGE_CLASS_GLACIER_INSTANT_RETRIEVAL - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Storage option: "glacier_ir".
- STORAGE_CLASS_INTELLIGENT_TIERING - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Storage option: "intelligent_tiering".
- STORAGE_CLASS_ONEZONE_INFREQUENT_ACCESS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Storage option: "onezone_ia".
- STORAGE_CLASS_OUTPOSTS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Storage option: "outposts".
- STORAGE_CLASS_REDUCED_REDUNDANCY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Storage option: "reduced_redundancy".
- STORAGE_CLASS_STANDARD - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Storage option: "standard".
- STORAGE_CLASS_STANDARD_INFREQUENT_ACCESS - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
S3 Storage option: "standard_ia".
- STORE_CAPABILITY_AWS_V2 - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Is this a v2 SDK build? value "fs.s3a.capability.aws.v2".
- STORE_CAPABILITY_DIRECTORY_MARKER_ACTION_DELETE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
PathCapabilities probe to indicate that a path
deletes directory markers.
- STORE_CAPABILITY_DIRECTORY_MARKER_ACTION_KEEP - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
PathCapabilities probe to indicate that a path
keeps directory markers.
- STORE_CAPABILITY_DIRECTORY_MARKER_AWARE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
PathCapabilities probe to verify that an S3A Filesystem
has the changes needed to safely work with buckets where
directoy markers have not been deleted.
- STORE_CAPABILITY_DIRECTORY_MARKER_MULTIPART_UPLOAD_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Deprecated.
- STORE_CAPABILITY_DIRECTORY_MARKER_POLICY_AUTHORITATIVE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
PathCapabilities probe to indicate that the filesystem
keeps directory markers in authoritative paths only.
- STORE_CAPABILITY_DIRECTORY_MARKER_POLICY_DELETE - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
PathCapabilities probe to indicate that the filesystem
deletes directory markers.
- STORE_CAPABILITY_DIRECTORY_MARKER_POLICY_KEEP - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
PathCapabilities probe to indicate that the filesystem
keeps directory markers.
- STORE_CAPABILITY_MAGIC_COMMITTER - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to indicate that a store supports magic committers.
- STORE_CAPABILITY_MAGIC_COMMITTER_OLD - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Deprecated.
- STORE_CAPABILITY_MULTIPART_UPLOAD_ENABLED - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Stream supports multipart uploads to the given path.
- STORE_CAPABILITY_S3_EXPRESS_STORAGE - Static variable in class org.apache.hadoop.fs.s3a.impl.S3ExpressStorage
-
Is this S3Express storage? value "fs.s3a.capability.s3express.storage".
- StoreContext - Class in org.apache.hadoop.fs.s3a.impl
-
This class provides the core context of the S3A filesystem to subsidiary
components, without exposing the entire parent class.
- StoreContextBuilder - Class in org.apache.hadoop.fs.s3a.impl
-
Builder for the store context.
- StoreContextBuilder() - Constructor for class org.apache.hadoop.fs.s3a.impl.StoreContextBuilder
-
- StoreContextFactory - Interface in org.apache.hadoop.fs.s3a.impl
-
Factory for creating store contexts.
- STREAM_CAPABILITY_MAGIC_OUTPUT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Flag to indicate whether a stream is a magic output stream;
returned in StreamCapabilities
Value: "fs.s3a.capability.magic.output.stream".
- STREAM_CAPABILITY_MAGIC_OUTPUT_OLD - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
Deprecated.
- STREAM_READ_GAUGE_INPUT_POLICY - Static variable in class org.apache.hadoop.fs.s3a.Constants
-
Gauge name for the input policy : "stream_read_gauge_input_policy".
- streamClose(boolean, long) - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
The inner stream was closed.
- streamOpened() - Method in interface org.apache.hadoop.fs.s3a.statistics.S3AInputStreamStatistics
-
The inner stream was opened.
- stringify(AwsServiceException) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Get low level details of an amazon exception for logging; multi-line.
- stringify(S3Object) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
String information about a summary entry for debug messages.
- STS_STANDARD - Static variable in class org.apache.hadoop.fs.s3a.auth.delegation.DelegationConstants
-
The standard STS server.
- STSClientFactory - Class in org.apache.hadoop.fs.s3a.auth
-
Factory for creating STS Clients.
- STSClientFactory() - Constructor for class org.apache.hadoop.fs.s3a.auth.STSClientFactory
-
- STSClientFactory.STSClient - Class in org.apache.hadoop.fs.s3a.auth
-
STS client connection with retries.
- stubAuditManager() - Static method in class org.apache.hadoop.fs.s3a.audit.AuditIntegration
-
Return a stub audit manager.
- submit(Executor, Callable<T>) - Static method in class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
Submit a callable into a completable future.
- submit(Executor, AuditSpan, Callable<T>) - Static method in class org.apache.hadoop.fs.s3a.impl.CallableSupplier
-
Submit a callable into a completable future.
- submit(CompletableFuture<T>, Callable<T>) - Method in class org.apache.hadoop.fs.s3a.impl.StoreContext
-
- submit(CallableRaisingIOE<T>) - Method in interface org.apache.hadoop.fs.s3a.S3AInputStream.InputStreamCallbacks
-
Submit some asynchronous work, for example, draining a stream.
- SUCCESS_MARKER_FILE_LIMIT - Static variable in class org.apache.hadoop.fs.s3a.commit.CommitConstants
-
The limit to the number of committed objects tracked during
job commits and saved to the _SUCCESS file.
- SuccessData - Class in org.apache.hadoop.fs.s3a.commit.files
-
Summary data saved into a _SUCCESS marker file.
- SuccessData() - Constructor for class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- switchToIOStatisticsContext() - Method in class org.apache.hadoop.fs.s3a.commit.impl.CommitContext
-
Switch to the context IOStatistics context,
if needed.
- v1(ListObjectsRequest) - Static method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
Restricted constructors to ensure v1 or v2, not both.
- v1(ListObjectsResponse) - Static method in class org.apache.hadoop.fs.s3a.S3ListResult
-
Restricted constructors to ensure v1 or v2, not both.
- v1RequestHandlersUsed(String) - Static method in class org.apache.hadoop.fs.s3a.impl.V2Migration
-
Notes use of request handlers.
- V1ToV2AwsCredentialProviderAdapter - Class in org.apache.hadoop.fs.s3a.adapter
-
Adapts a V1 AWSCredentialsProvider to the V2 AwsCredentialsProvider interface.
- v2(ListObjectsV2Request) - Static method in class org.apache.hadoop.fs.s3a.S3ListRequest
-
Restricted constructors to ensure v1 or v2, not both.
- v2(ListObjectsV2Response) - Static method in class org.apache.hadoop.fs.s3a.S3ListResult
-
Restricted constructors to ensure v1 or v2, not both.
- V2Migration - Class in org.apache.hadoop.fs.s3a.impl
-
This class provides utility methods required for migrating S3A to AWS Java SDK V2.
- validate() - Method in class org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier
-
Validate the token by looking at its fields.
- validate(String, MarshalledCredentials.CredentialTypeRequired) - Method in class org.apache.hadoop.fs.s3a.auth.MarshalledCredentials
-
Verify that a set of credentials is valid.
- validate() - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
Validation includes validating all statements.
- validate() - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.RoleElt
-
validation operation.
- validate() - Method in class org.apache.hadoop.fs.s3a.auth.RoleModel.Statement
-
- validate() - Method in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Validate the data: those fields which must be non empty, must be set.
- validate() - Method in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Validate the data: those fields which must be non empty, must be set.
- validate() - Method in class org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit
-
- validate() - Method in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
- validateCollectionClass(Iterable, Class) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Verify that all instances in a collection are of the given class.
- validateOutputStreamConfiguration(Path, Configuration) - Static method in class org.apache.hadoop.fs.s3a.S3AUtils
-
Validates the output stream configuration.
- validateReadahead(Long) - Static method in class org.apache.hadoop.fs.s3a.S3AInputStream
-
from a possibly null Long value, return a valid
readahead.
- ValidationFailure - Exception in org.apache.hadoop.fs.s3a.commit
-
Exception raised on validation failures; kept as an IOException
for consistency with other failures.
- ValidationFailure(String, Object...) - Constructor for exception org.apache.hadoop.fs.s3a.commit.ValidationFailure
-
Create an instance with string formatting applied to the message
and arguments.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.api.PerformanceFlagEnum
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.auth.delegation.AWSPolicyProvider.AccessLevel
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.TokenIssuingPolicy
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.CredentialTypeRequired
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.auth.RoleModel.Effects
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.JobUUIDSource
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.commit.staging.ConflictResolution
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.AWSCannedACL
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy.Mode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy.Source
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.DirectoryPolicy.MarkerPolicy
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.InstantiationIOException.Kind
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.ProgressListenerEvent
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.S3AInputPolicy
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.Statistic
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.statistics.StatisticTypeEnum
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum org.apache.hadoop.fs.s3a.Tristate
-
Returns the enum constant of this type with the specified name.
- values() - Static method in enum org.apache.hadoop.fs.s3a.api.PerformanceFlagEnum
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.auth.delegation.AWSPolicyProvider.AccessLevel
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.TokenIssuingPolicy
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.auth.MarshalledCredentials.CredentialTypeRequired
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.auth.RoleModel.Effects
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.JobUUIDSource
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.commit.staging.ConflictResolution
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.AWSCannedACL
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy.Mode
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy.Source
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.DirectoryPolicy.MarkerPolicy
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.InstantiationIOException.Kind
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.ProgressListenerEvent
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.impl.StatusProbeEnum
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.S3AEncryptionMethods
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.S3AInputPolicy
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.Statistic
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.statistics.StatisticTypeEnum
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum org.apache.hadoop.fs.s3a.Tristate
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- VectoredIOContext - Class in org.apache.hadoop.fs.s3a
-
Context related to vectored IO operation.
- VectoredIOContext() - Constructor for class org.apache.hadoop.fs.s3a.VectoredIOContext
-
Default no arg constructor.
- VERB_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "verb".
- VERBOSE - Static variable in class org.apache.hadoop.fs.s3a.s3guard.S3GuardTool
-
- verify(boolean, String, Object...) - Static method in exception org.apache.hadoop.fs.s3a.commit.ValidationFailure
-
Verify that a condition holds.
- verifyBucketExists() - Method in class org.apache.hadoop.fs.s3a.S3AFileSystem
-
Verify that the bucket exists.
- verifyIsMagicCommitFS(S3AFileSystem) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Verify that an S3A FS instance is a magic commit FS.
- verifyIsMagicCommitPath(S3AFileSystem, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Verify that the path is a magic one.
- verifyIsS3AFS(FileSystem, Path) - Static method in class org.apache.hadoop.fs.s3a.commit.CommitUtils
-
Verify that an FS is an S3A FS.
- version - Variable in class org.apache.hadoop.fs.s3a.auth.RoleModel.Policy
-
- VERSION - Static variable in class org.apache.hadoop.fs.s3a.auth.RoleModel
-
- VERSION - Static variable in class org.apache.hadoop.fs.s3a.commit.files.PendingSet
-
Supported version value: 3.
- VERSION - Static variable in class org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData
-
Supported version value: 2.
- VERSION - Static variable in class org.apache.hadoop.fs.s3a.commit.files.SuccessData
-
Supported version value: 1.
- VERSION_FOUR_SIGNER - Static variable in class org.apache.hadoop.fs.s3a.auth.SignerFactory
-
- VERSION_FOUR_UNSIGNED_PAYLOAD_SIGNER - Static variable in class org.apache.hadoop.fs.s3a.auth.SignerFactory
-
- VERSION_GROUP - Static variable in class org.apache.hadoop.fs.s3a.audit.S3LogParser
-
Log group "version".
- versionMismatchError() - Method in interface org.apache.hadoop.fs.s3a.statistics.ChangeTrackerStatistics
-
A version mismatch was detected.
- versionMismatchError() - Method in class org.apache.hadoop.fs.s3a.statistics.impl.CountingChangeTracker
-