@InterfaceAudience.Public @InterfaceStability.Evolving public final class Constants extends Object
S3AFileSystem
.
Some of the strings are marked as Unstable
. This means
that they may be Unsupported in future; at which point they will be marked
as deprecated and simply ignored.
All S3Guard related constants are marked as Deprecated and either ignored (ddb config)
or rejected (setting the metastore to anything other than the null store).
Timeout default values are declared as integers or long values in milliseconds and
occasionally seconds.
There are now Duration
constants for these default values; the original
fields are retained for compatibility, and derive their value from the Duration equivalent.
New timeout/duration constants do not get the equivalent integer/long fields.
Modifier and Type | Field and Description |
---|---|
static String |
ACCESS_KEY |
static String |
ALLOW_REQUESTER_PAYS
Configuration option for S3 Requester Pays feature: "fs.s3a.requester.pays.enabled".
|
static String |
ASSUMED_ROLE_ARN
ARN of AWS Role to request: "fs.s3a.assumed.role.arn".
|
static String |
ASSUMED_ROLE_CREDENTIALS_DEFAULT |
static String |
ASSUMED_ROLE_CREDENTIALS_PROVIDER
List of providers to authenticate for the assumed role: "fs.s3a.assumed.role.credentials.provider".
|
static String |
ASSUMED_ROLE_POLICY
JSON policy containing the policy to apply to the role: "fs.s3a.assumed.role.policy".
|
static String |
ASSUMED_ROLE_SESSION_DURATION
Duration of assumed roles before a refresh is attempted.
|
static String |
ASSUMED_ROLE_SESSION_DURATION_DEFAULT
Default duration of an assumed role: "1h".
|
static String |
ASSUMED_ROLE_SESSION_NAME
Session name for the assumed role, must be valid characters according
to the AWS APIs: "fs.s3a.assumed.role.session.name".
|
static String |
ASSUMED_ROLE_STS_ENDPOINT
Security Token Service Endpoint: "fs.s3a.assumed.role.sts.endpoint".
|
static String |
ASSUMED_ROLE_STS_ENDPOINT_REGION
Region for the STS endpoint; needed if the endpoint
is set to anything other then the central one.: "fs.s3a.assumed.role.sts.endpoint.region".
|
static String |
ASSUMED_ROLE_STS_ENDPOINT_REGION_DEFAULT
Default value for the STS endpoint region; needed for
v4 signing: "".
|
static String |
ASYNC_DRAIN_THRESHOLD
The threshold at which drain operations switch
to being asynchronous with the schedule/wait overhead
compared to synchronous.
|
static String |
AUTHORITATIVE_PATH
Paths considered "authoritative".
|
static String |
AWS_AUTH_CLASS_PREFIX
Prefix of auth classes in AWS SDK V1.
|
static String |
AWS_CREDENTIALS_PROVIDER |
static String |
AWS_CREDENTIALS_PROVIDER_MAPPING
AWS credentials providers mapping with key/value pairs.
|
static String |
AWS_REGION
AWS S3 region for the bucket.
|
static String |
AWS_S3_ACCESSPOINT_REQUIRED
Require that all S3 access is made through Access Points.
|
static String |
AWS_S3_CENTRAL_REGION
The special S3 region which can be used to talk to any bucket.
|
static String |
AWS_S3_CLASSLOADER_ISOLATION
Are extensions classes, such as
fs.s3a.aws.credentials.provider ,
going to be loaded from the same classloader that loaded
the S3AFileSystem ?
It is useful to turn classloader isolation off for Apache Spark applications
that might load S3AFileSystem from the Spark distribution (Launcher classloader)
while users might want to provide custom extensions (loaded by Spark MutableClassloader). |
static String |
AWS_S3_DEFAULT_REGION
The default S3 region when using cross region client.
|
static String |
AWS_S3_VECTOR_ACTIVE_RANGE_READS
Maximum number of range reads a single input stream can have
active (downloading, or queued) to the central FileSystem
instance's pool of queued operations.
|
static String |
AWS_S3_VECTOR_READS_MAX_MERGED_READ_SIZE
What is the largest merged read size in bytes such
that we group ranges together during vectored read.
|
static String |
AWS_S3_VECTOR_READS_MIN_SEEK_SIZE
What is the smallest reasonable seek in bytes such
that we group ranges together during vectored read operation.
|
static String |
AWS_SERVICE_IDENTIFIER_DDB
Deprecated.
|
static String |
AWS_SERVICE_IDENTIFIER_S3 |
static String |
AWS_SERVICE_IDENTIFIER_STS |
static String |
BUCKET_REGION_HEADER
The bucket region header.
|
static String |
BUFFER_DIR |
static String |
BULK_DELETE_PAGE_SIZE
Number of objects to delete in a single multi-object delete "fs.s3a.bulk.delete.page.size".
|
static int |
BULK_DELETE_PAGE_SIZE_DEFAULT
Default Number of objects to delete in a single multi-object
delete: 250.
|
static String |
CANNED_ACL |
static String |
CENTRAL_ENDPOINT
The central endpoint :"s3.amazonaws.com".
|
static String |
CHANGE_DETECT_MODE
Mode to run change detection in.
|
static String |
CHANGE_DETECT_MODE_CLIENT
Change is detected on the client side by comparing the returned id with the
expected id.
|
static String |
CHANGE_DETECT_MODE_DEFAULT
Default change detection mode: server.
|
static String |
CHANGE_DETECT_MODE_NONE
Change detection is turned off.
|
static String |
CHANGE_DETECT_MODE_SERVER
Change is detected by passing the expected value in the GetObject request.
|
static String |
CHANGE_DETECT_MODE_WARN
Change is detected on the client side by comparing the returned id with the
expected id.
|
static String |
CHANGE_DETECT_REQUIRE_VERSION
If true, raises a
RemoteFileChangedException exception when S3
doesn't provide the attribute defined by fs.s3a.change.detection.source. |
static boolean |
CHANGE_DETECT_REQUIRE_VERSION_DEFAULT
Default change detection require version: true.
|
static String |
CHANGE_DETECT_SOURCE
Where to get the value to use in change detection.
|
static String |
CHANGE_DETECT_SOURCE_DEFAULT
Default change detection mechanism: eTag.
|
static String |
CHANGE_DETECT_SOURCE_ETAG
eTag as the change detection mechanism.
|
static String |
CHANGE_DETECT_SOURCE_VERSION_ID
Object versionId as the change detection mechanism.
|
static String |
CHECKSUM_VALIDATION
Should checksums be validated on download?
This is slower and not needed on TLS connections.
|
static boolean |
CHECKSUM_VALIDATION_DEFAULT
Default value of
CHECKSUM_VALIDATION . |
static String |
CONNECTION_ACQUISITION_TIMEOUT
Acquisition timeout for connections from the pool:
"fs.s3a.connection.acquisition.timeout".
|
static String |
CONNECTION_IDLE_TIME
Maximum idle time for connections in the pool: "fs.s3a.connection.idle.time".
|
static String |
CONNECTION_KEEPALIVE
Should TCP Keepalive be enabled on the socket?
This adds some network IO, but finds failures faster.
|
static String |
CONNECTION_TTL
Configuration option to configure expiration time of
S3 http connection from the connection pool: "fs.s3a.connection.ttl".
|
static String |
CONTENT_ENCODING
Content encoding: gzip, deflate, compress, br, etc.
|
static String |
CUSTOM_SIGNERS
List of custom Signers.
|
static boolean |
DEFAULT_ALLOW_REQUESTER_PAYS
Default configuration for "fs.s3a.requester.pays.enabled": false.
|
static String |
DEFAULT_ASSUMED_ROLE_STS_ENDPOINT
Default endpoint for session tokens: "".
|
static int |
DEFAULT_ASYNC_DRAIN_THRESHOLD
This is a number based purely on experimentation in
ITestS3AInputStreamPerformance . |
static String[] |
DEFAULT_AUTHORITATIVE_PATH |
static boolean |
DEFAULT_AWS_S3_CLASSLOADER_ISOLATION
Default value for
AWS_S3_CLASSLOADER_ISOLATION . |
static int |
DEFAULT_AWS_S3_VECTOR_ACTIVE_RANGE_READS
Limit of queued range data download operations during vectored
read.
|
static int |
DEFAULT_AWS_S3_VECTOR_READS_MAX_MERGED_READ_SIZE
Default maximum read size in bytes during vectored reads : 1253376.
|
static int |
DEFAULT_AWS_S3_VECTOR_READS_MIN_SEEK_SIZE
Default minimum seek in bytes during vectored reads : 4896.
|
static String |
DEFAULT_CANNED_ACL |
static Duration |
DEFAULT_CONNECTION_ACQUISITION_TIMEOUT_DURATION
Default acquisition timeout: 60 seconds.
|
static Duration |
DEFAULT_CONNECTION_IDLE_TIME_DURATION
Default idle time: 60 seconds.
|
static boolean |
DEFAULT_CONNECTION_KEEPALIVE
Default value of
CONNECTION_KEEPALIVE : false. |
static long |
DEFAULT_CONNECTION_TTL
Deprecated.
|
static Duration |
DEFAULT_CONNECTION_TTL_DURATION
Default duration for
CONNECTION_TTL : 5 minutes. |
static String |
DEFAULT_DIRECTORY_MARKER_POLICY
Default retention policy: "keep".
|
static String |
DEFAULT_ENDPOINT
Default value of s3 endpoint: "".
|
static int |
DEFAULT_ESTABLISH_TIMEOUT
Deprecated.
|
static Duration |
DEFAULT_ESTABLISH_TIMEOUT_DURATION
Default TCP/(and TLS?) establish timeout: 30 seconds.
|
static int |
DEFAULT_EXECUTOR_CAPACITY
The capacity of executor queues for operations other than block
upload, where
FAST_UPLOAD_ACTIVE_BLOCKS is used instead. |
static int |
DEFAULT_FAST_BUFFER_SIZE |
static boolean |
DEFAULT_FAST_UPLOAD
Deprecated.
|
static int |
DEFAULT_FAST_UPLOAD_ACTIVE_BLOCKS
Limit of queued block upload operations before writes
block.
|
static String |
DEFAULT_FAST_UPLOAD_BUFFER
Default buffer option: "disk".
|
static int |
DEFAULT_KEEPALIVE_TIME
Deprecated.
|
static Duration |
DEFAULT_KEEPALIVE_TIME_DURATION
Default value of
KEEPALIVE_TIME : 60s. |
static int |
DEFAULT_LIST_VERSION |
static int |
DEFAULT_MAX_ERROR_RETRIES
Default number of times the AWS client library should retry errors before
escalating to the S3A code: 5.
|
static int |
DEFAULT_MAX_PAGING_KEYS |
static int |
DEFAULT_MAX_THREADS
Default value of
MAX_THREADS : 96. |
static int |
DEFAULT_MAX_TOTAL_TASKS
Default value for
MAX_TOTAL_TASKS : 32. |
static int |
DEFAULT_MAXIMUM_CONNECTIONS
Default value for
MAXIMUM_CONNECTIONS : 500. |
static boolean |
DEFAULT_METADATASTORE_AUTHORITATIVE
Deprecated.
|
static long |
DEFAULT_METADATASTORE_METADATA_TTL
Deprecated.
|
static long |
DEFAULT_MIN_MULTIPART_THRESHOLD |
static long |
DEFAULT_MULTIPART_SIZE |
static boolean |
DEFAULT_MULTIPART_UPLOAD_ENABLED
Default value for multipart uploads.
|
static int |
DEFAULT_PREFETCH_MAX_BLOCKS_COUNT
Default value for max blocks count config.
|
static boolean |
DEFAULT_PURGE_EXISTING_MULTIPART |
static long |
DEFAULT_PURGE_EXISTING_MULTIPART_AGE
Default Age.
|
static long |
DEFAULT_READAHEAD_RANGE |
static int |
DEFAULT_REQUEST_TIMEOUT
Deprecated.
|
static Duration |
DEFAULT_REQUEST_TIMEOUT_DURATION
Default duration of a request before it is timed out: 60s.
|
static Class<? extends S3ClientFactory> |
DEFAULT_S3_CLIENT_FACTORY_IMPL
Default factory:
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory . |
static String |
DEFAULT_S3GUARD_DISABLED_WARN_LEVEL
Deprecated.
|
static int |
DEFAULT_S3GUARD_METASTORE_LOCAL_ENTRY_TTL
Deprecated.
|
static int |
DEFAULT_S3GUARD_METASTORE_LOCAL_MAX_RECORDS
Deprecated.
|
static boolean |
DEFAULT_SECURE_CONNECTIONS |
static int |
DEFAULT_SOCKET_RECV_BUFFER |
static int |
DEFAULT_SOCKET_SEND_BUFFER |
static int |
DEFAULT_SOCKET_TIMEOUT
Deprecated.
|
static Duration |
DEFAULT_SOCKET_TIMEOUT_DURATION
Default socket timeout: 200 seconds.
|
static org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.SSLChannelMode |
DEFAULT_SSL_CHANNEL_MODE |
static String |
DIRECTORY_MARKER_POLICY
Policy for directory markers.
|
static String |
DIRECTORY_MARKER_POLICY_AUTHORITATIVE
Retain directory markers in authoritative directory trees only.
|
static String |
DIRECTORY_MARKER_POLICY_DELETE
Delete directory markers.
|
static String |
DIRECTORY_MARKER_POLICY_KEEP
Retain directory markers.
|
static String |
DIRECTORY_OPERATIONS_PURGE_UPLOADS
Should directory operations purge uploads?
This adds at least one parallelized list operation to the call,
plus the overhead of deletions.
|
static boolean |
DIRECTORY_OPERATIONS_PURGE_UPLOADS_DEFAULT
Default value of
DIRECTORY_OPERATIONS_PURGE_UPLOADS : false. |
static String |
DOWNGRADE_SYNCABLE_EXCEPTIONS
Rather than raise an exception when an attempt is
made to call the Syncable APIs, warn and downgrade.
|
static boolean |
DOWNGRADE_SYNCABLE_EXCEPTIONS_DEFAULT
Default value for syncable invocation.
|
static String |
ENABLE_MULTI_DELETE |
static String |
ENDPOINT
Endpoint.
|
static boolean |
ENDPOINT_FIPS_DEFAULT |
static String |
ESTABLISH_TIMEOUT
Milliseconds until a connection is established: "fs.s3a.connection.establish.timeout".
|
static String |
ETAG_CHECKSUM_ENABLED
Should etags be exposed as checksums?
|
static boolean |
ETAG_CHECKSUM_ENABLED_DEFAULT
Default value: false.
|
static String |
EXECUTOR_CAPACITY
The capacity of executor queues for operations other than block
upload, where
FAST_UPLOAD_ACTIVE_BLOCKS is used instead. |
static String |
EXPERIMENTAL_AWS_INTERNAL_THROTTLING
Experimental/Unstable feature: should the AWS client library retry
throttle responses before escalating to the S3A code: "fs.s3a.experimental.aws.s3.throttling".
|
static boolean |
EXPERIMENTAL_AWS_INTERNAL_THROTTLING_DEFAULT
Default value of
EXPERIMENTAL_AWS_INTERNAL_THROTTLING ,
value: true. |
static String |
FAIL_INJECT_CLIENT_FACTORY |
static String |
FAIL_INJECT_INCONSISTENCY_KEY
Deprecated.
|
static String |
FAIL_INJECT_INCONSISTENCY_MSEC
Deprecated.
|
static String |
FAIL_INJECT_INCONSISTENCY_PROBABILITY
Deprecated.
|
static String |
FAIL_INJECT_THROTTLE_PROBABILITY |
static String |
FAIL_ON_METADATA_WRITE_ERROR
Deprecated.
|
static boolean |
FAIL_ON_METADATA_WRITE_ERROR_DEFAULT
Deprecated.
|
static String |
FAST_BUFFER_SIZE
Deprecated.
|
static String |
FAST_UPLOAD
Deprecated.
|
static String |
FAST_UPLOAD_ACTIVE_BLOCKS
Maximum Number of blocks a single output stream can have
active (uploading, or queued to the central FileSystem
instance's pool of queued operations.
|
static String |
FAST_UPLOAD_BUFFER
What buffer to use.
|
static String |
FAST_UPLOAD_BUFFER_ARRAY
Use an in-memory array.
|
static String |
FAST_UPLOAD_BUFFER_DISK
Buffer blocks to disk: "disk".
|
static String |
FAST_UPLOAD_BYTEBUFFER
Use a byte buffer.
|
static String |
FIPS_ENDPOINT
Is the endpoint a FIPS endpoint?
Can be queried as a path capability.
|
static String |
FS_S3A |
static String |
FS_S3A_BLOCK_SIZE |
static String |
FS_S3A_BUCKET_PREFIX
Prefix for S3A bucket-specific properties: "fs.s3a.bucket.".
|
static String |
FS_S3A_CREATE_HEADER
Prefix for adding a header to the object when created.
|
static String |
FS_S3A_CREATE_PERFORMANCE
Flag for create performance.
|
static boolean |
FS_S3A_CREATE_PERFORMANCE_DEFAULT
Default value for create performance in an S3A FS.
|
static String |
FS_S3A_CREATE_PERFORMANCE_ENABLED
Capability to indicate that the FS has been instantiated with
FS_S3A_CREATE_PERFORMANCE set to true. |
static String |
FS_S3A_PREFIX
Prefix for all S3A properties: "fs.s3a.".
|
static String |
HADOOP_TMP_DIR
default hadoop temp dir on local system: "hadoop.tmp.dir".
|
static String |
HTTP_SIGNER_CLASS_NAME
Classname of the http signer to use when
HTTP_SIGNER_ENABLED
is true: "fs.s3a.http.signer.class". |
static String |
HTTP_SIGNER_ENABLED
Flag to switch to a v2 SDK HTTP signer.
|
static boolean |
HTTP_SIGNER_ENABLED_DEFAULT
Default value of
HTTP_SIGNER_ENABLED : false. |
static String |
INPUT_FADV_DEFAULT
Deprecated.
use the
Options.OpenFileOptions value
in code which only needs to be compiled against newer hadoop
releases. |
static String |
INPUT_FADV_NORMAL
General input.
|
static String |
INPUT_FADV_RANDOM
Deprecated.
use the
Options.OpenFileOptions value
in code which only needs to be compiled against newer hadoop
releases. |
static String |
INPUT_FADV_SEQUENTIAL
Deprecated.
use the
Options.OpenFileOptions value
in code which only needs to be compiled against newer hadoop
releases. |
static String |
INPUT_FADVISE
Which input strategy to use for buffering, seeking and similar when
reading data.
|
static String |
KEEPALIVE_TIME
The time an idle thread waits before terminating: "fs.s3a.threads.keepalivetime".
|
static String |
LIST_VERSION
S3 API level parameters.
|
static String |
MAX_ERROR_RETRIES
Number of times the AWS client library should retry errors before
escalating to the S3A code: "fs.s3a.attempts.maximum".
|
static int |
MAX_MULTIPART_COUNT
Maximum number of partitions in a multipart upload: 10000.
|
static String |
MAX_PAGING_KEYS |
static String |
MAX_THREADS
The maximum number of threads to allow in the pool used by S3A.
|
static String |
MAX_TOTAL_TASKS
The maximum number of tasks queued (other than prefetcher tasks) if all threads are
busy: "fs.s3a.max.total.tasks".
|
static String |
MAXIMUM_CONNECTIONS
Number of simultaneous connections to S3: "fs.s3a.connection.maximum".
|
static String |
METADATASTORE_AUTHORITATIVE
Deprecated.
no longer supported
|
static String |
METADATASTORE_METADATA_TTL
Deprecated.
no longer supported
|
static String |
MIN_MULTIPART_THRESHOLD |
static Duration |
MINIMUM_NETWORK_OPERATION_DURATION
This is the minimum operation duration unless programmatically set.
|
static int |
MULTIPART_MIN_SIZE
The minimum multipart size which S3 supports.
|
static String |
MULTIPART_SIZE |
static String |
MULTIPART_UPLOADS_ENABLED
Option to enable or disable the multipart uploads.
|
static String |
OPTIMIZED_COPY_FROM_LOCAL
Is the higher performance copy from local file to S3 enabled?
This switch allows for it to be disabled if there are problems.
|
static boolean |
OPTIMIZED_COPY_FROM_LOCAL_DEFAULT
Default value for
OPTIMIZED_COPY_FROM_LOCAL . |
static String |
PATH_STYLE_ACCESS |
static String |
PREFETCH_BLOCK_COUNT_KEY
Maximum number of blocks prefetched at any given time.
|
static int |
PREFETCH_BLOCK_DEFAULT_COUNT |
static int |
PREFETCH_BLOCK_DEFAULT_SIZE |
static String |
PREFETCH_BLOCK_SIZE_KEY
The size of a single prefetched block in number of bytes.
|
static boolean |
PREFETCH_ENABLED_DEFAULT
Default option as to whether the prefetching input stream is enabled.
|
static String |
PREFETCH_ENABLED_KEY
Controls whether the prefetching input stream is enabled.
|
static String |
PREFETCH_MAX_BLOCKS_COUNT
Prefetch max blocks count config.
|
static String |
PROXY_DOMAIN |
static String |
PROXY_HOST |
static String |
PROXY_PASSWORD |
static String |
PROXY_PORT |
static String |
PROXY_SECURED
Is the proxy secured(proxyProtocol = HTTPS)?
|
static String |
PROXY_USERNAME |
static String |
PROXY_WORKSTATION |
static String |
PURGE_EXISTING_MULTIPART |
static String |
PURGE_EXISTING_MULTIPART_AGE
purge any multipart uploads older than this number of seconds.
|
static String |
READAHEAD_RANGE
read ahead buffer size to prevent connection re-establishments.
|
static String |
REQUEST_TIMEOUT
How long should the SDK retry/wait on a response from an S3 store: "fs.s3a.connection.request.timeout"
including the time needed to sign the request.
|
static String |
RETRY_INTERVAL
Interval between retry attempts.: "fs.s3a.retry.interval".
|
static String |
RETRY_INTERVAL_DEFAULT
Default retry interval: "500ms".
|
static String |
RETRY_LIMIT
Number of times to retry any repeatable S3 client request on failure,
excluding throttling requests: "fs.s3a.retry.limit".
|
static int |
RETRY_LIMIT_DEFAULT
Default retry limit: 7.
|
static String |
RETRY_THROTTLE_INTERVAL
Interval between retry attempts on throttled requests: "fs.s3a.retry.throttle.interval".
|
static String |
RETRY_THROTTLE_INTERVAL_DEFAULT
Default throttled retry interval: "500ms".
|
static String |
RETRY_THROTTLE_LIMIT
Number of times to retry any throttled request: "fs.s3a.retry.throttle.limit".
|
static int |
RETRY_THROTTLE_LIMIT_DEFAULT
Default throttled retry limit: 20.
|
static String |
S3_CLIENT_FACTORY_IMPL
S3 Client Factory implementation class: "fs.s3a.s3.client.factory.impl".
|
static String |
S3_ENCRYPTION_ALGORITHM
Set S3-server side encryption(SSE) or S3-Client side encryption(CSE)
algorithm.
|
static String |
S3_ENCRYPTION_KEY
Set S3-SSE or S3-CSE encryption Key if required.
|
static String |
S3_METADATA_STORE_IMPL
Deprecated.
|
static String |
S3A_BUCKET_PROBE
Bucket validation parameter which can be set by client.
|
static int |
S3A_BUCKET_PROBE_DEFAULT
Default value of bucket validation parameter.
|
static int |
S3A_DEFAULT_PORT
Default port for this is 443: HTTPS.
|
static String |
S3A_SECURITY_CREDENTIAL_PROVIDER_PATH
Extra set of security credentials which will be prepended to that
set in
"hadoop.security.credential.provider.path" . |
static String |
S3EXPRESS_CREATE_SESSION
Use the S3 Express createSession() operation to authenticate with
S3Express storage?
|
static boolean |
S3EXPRESS_CREATE_SESSION_DEFAULT
Default value of
S3EXPRESS_CREATE_SESSION . |
static String |
S3GUARD_CLI_PRUNE_AGE
Deprecated.
|
static String |
S3GUARD_CONSISTENCY_RETRY_INTERVAL
Deprecated.
|
static String |
S3GUARD_CONSISTENCY_RETRY_INTERVAL_DEFAULT
Deprecated.
|
static String |
S3GUARD_CONSISTENCY_RETRY_LIMIT
Deprecated.
|
static int |
S3GUARD_CONSISTENCY_RETRY_LIMIT_DEFAULT
Deprecated.
|
static int |
S3GUARD_DDB_BACKGROUND_SLEEP_MSEC_DEFAULT
Deprecated.
|
static String |
S3GUARD_DDB_BACKGROUND_SLEEP_MSEC_KEY
Deprecated.
|
static int |
S3GUARD_DDB_BATCH_WRITE_REQUEST_LIMIT
Deprecated.
|
static String |
S3GUARD_DDB_MAX_RETRIES
Deprecated.
|
static int |
S3GUARD_DDB_MAX_RETRIES_DEFAULT
Deprecated.
|
static String |
S3GUARD_DDB_REGION_KEY
Deprecated.
|
static long |
S3GUARD_DDB_TABLE_CAPACITY_READ_DEFAULT
Deprecated.
|
static String |
S3GUARD_DDB_TABLE_CAPACITY_READ_KEY
Deprecated.
|
static long |
S3GUARD_DDB_TABLE_CAPACITY_WRITE_DEFAULT
Deprecated.
|
static String |
S3GUARD_DDB_TABLE_CAPACITY_WRITE_KEY
Deprecated.
|
static String |
S3GUARD_DDB_TABLE_CREATE_KEY
Deprecated.
|
static String |
S3GUARD_DDB_TABLE_NAME_KEY
Deprecated.
|
static String |
S3GUARD_DDB_TABLE_SSE_CMK
Deprecated.
|
static String |
S3GUARD_DDB_TABLE_SSE_ENABLED
Deprecated.
|
static String |
S3GUARD_DDB_TABLE_TAG
Deprecated.
|
static String |
S3GUARD_DDB_THROTTLE_RETRY_INTERVAL
Deprecated.
|
static String |
S3GUARD_DDB_THROTTLE_RETRY_INTERVAL_DEFAULT
Deprecated.
|
static String |
S3GUARD_DISABLED_WARN_LEVEL
Deprecated.
|
static String |
S3GUARD_METASTORE_DYNAMO
Deprecated.
|
static String |
S3GUARD_METASTORE_LOCAL
Deprecated.
|
static String |
S3GUARD_METASTORE_LOCAL_ENTRY_TTL
Deprecated.
|
static String |
S3GUARD_METASTORE_LOCAL_MAX_RECORDS
Deprecated.
|
static String |
S3GUARD_METASTORE_NULL
Deprecated.
|
static String |
S3N_FOLDER_SUFFIX |
static String |
SECRET_KEY |
static String |
SECURE_CONNECTIONS |
static String |
SERVER_SIDE_ENCRYPTION_AES256
Deprecated.
|
static String |
SERVER_SIDE_ENCRYPTION_ALGORITHM
Deprecated.
|
static String |
SERVER_SIDE_ENCRYPTION_KEY
Deprecated.
|
static String |
SESSION_TOKEN
session token for when using TemporaryAWSCredentialsProvider: : "fs.s3a.session.token".
|
static String |
SIGNING_ALGORITHM
Multiple parameters can be used to specify a non-default signing
algorithm.
fs.s3a.signing-algorithm - This property has existed for the longest time. |
static String |
SIGNING_ALGORITHM_DDB
Deprecated.
|
static String |
SIGNING_ALGORITHM_S3 |
static String |
SIGNING_ALGORITHM_STS |
static String |
SOCKET_RECV_BUFFER |
static String |
SOCKET_SEND_BUFFER |
static String |
SOCKET_TIMEOUT
Milliseconds until we give up on a connection to s3: "fs.s3a.connection.timeout".
|
static String |
SSL_CHANNEL_MODE |
static String |
STORAGE_CLASS
S3 storage class: standard, reduced_redundancy, intelligent_tiering etc.
|
static String |
STORAGE_CLASS_DEEP_ARCHIVE
S3 Storage option: "deep_archive".
|
static String |
STORAGE_CLASS_GLACIER
S3 Storage option: "glacier".
|
static String |
STORAGE_CLASS_GLACIER_INSTANT_RETRIEVAL
S3 Storage option: "glacier_ir".
|
static String |
STORAGE_CLASS_INTELLIGENT_TIERING
S3 Storage option: "intelligent_tiering".
|
static String |
STORAGE_CLASS_ONEZONE_INFREQUENT_ACCESS
S3 Storage option: "onezone_ia".
|
static String |
STORAGE_CLASS_OUTPOSTS
S3 Storage option: "outposts".
|
static String |
STORAGE_CLASS_REDUCED_REDUNDANCY
S3 Storage option: "reduced_redundancy".
|
static String |
STORAGE_CLASS_STANDARD
S3 Storage option: "standard".
|
static String |
STORAGE_CLASS_STANDARD_INFREQUENT_ACCESS
S3 Storage option: "standard_ia".
|
static String |
STORE_CAPABILITY_AWS_V2
Is this a v2 SDK build? value "fs.s3a.capability.aws.v2".
|
static String |
STORE_CAPABILITY_DIRECTORY_MARKER_ACTION_DELETE
PathCapabilities probe to indicate that a path
deletes directory markers. |
static String |
STORE_CAPABILITY_DIRECTORY_MARKER_ACTION_KEEP
PathCapabilities probe to indicate that a path
keeps directory markers. |
static String |
STORE_CAPABILITY_DIRECTORY_MARKER_AWARE
PathCapabilities probe to verify that an S3A Filesystem
has the changes needed to safely work with buckets where
directoy markers have not been deleted. |
static String |
STORE_CAPABILITY_DIRECTORY_MARKER_MULTIPART_UPLOAD_ENABLED
Deprecated.
|
static String |
STORE_CAPABILITY_DIRECTORY_MARKER_POLICY_AUTHORITATIVE
PathCapabilities probe to indicate that the filesystem
keeps directory markers in authoritative paths only. |
static String |
STORE_CAPABILITY_DIRECTORY_MARKER_POLICY_DELETE
PathCapabilities probe to indicate that the filesystem
deletes directory markers. |
static String |
STORE_CAPABILITY_DIRECTORY_MARKER_POLICY_KEEP
PathCapabilities probe to indicate that the filesystem
keeps directory markers. |
static String |
STORE_CAPABILITY_MULTIPART_UPLOAD_ENABLED
Stream supports multipart uploads to the given path.
|
static String |
STREAM_READ_GAUGE_INPUT_POLICY
Gauge name for the input policy : "stream_read_gauge_input_policy".
|
static int |
THREAD_POOL_SHUTDOWN_DELAY_SECONDS
How long to wait for the thread pool to terminate when cleaning up.
|
static String |
USER_AGENT_PREFIX |
static String |
XA_HEADER_PREFIX
To comply with the XAttr rules, all headers of the object retrieved
through the getXAttr APIs have the prefix: "header.".
|
public static final String HADOOP_TMP_DIR
public static final int MULTIPART_MIN_SIZE
public static final String ACCESS_KEY
public static final String SECRET_KEY
public static final String AWS_CREDENTIALS_PROVIDER
public static final String AWS_CREDENTIALS_PROVIDER_MAPPING
public static final String S3A_SECURITY_CREDENTIAL_PROVIDER_PATH
"hadoop.security.credential.provider.path"
.
This extra option allows for per-bucket overrides.public static final String SESSION_TOKEN
public static final String ASSUMED_ROLE_ARN
public static final String ASSUMED_ROLE_SESSION_NAME
public static final String ASSUMED_ROLE_SESSION_DURATION
public static final String ASSUMED_ROLE_STS_ENDPOINT
public static final String DEFAULT_ASSUMED_ROLE_STS_ENDPOINT
public static final String ASSUMED_ROLE_STS_ENDPOINT_REGION
public static final String ASSUMED_ROLE_STS_ENDPOINT_REGION_DEFAULT
public static final String ASSUMED_ROLE_SESSION_DURATION_DEFAULT
public static final String ASSUMED_ROLE_CREDENTIALS_PROVIDER
public static final String ASSUMED_ROLE_POLICY
public static final String ASSUMED_ROLE_CREDENTIALS_DEFAULT
public static final String MAX_TOTAL_TASKS
public static final int DEFAULT_MAX_TOTAL_TASKS
MAX_TOTAL_TASKS
: 32.public static final String MAXIMUM_CONNECTIONS
public static final int DEFAULT_MAXIMUM_CONNECTIONS
MAXIMUM_CONNECTIONS
: 500.
Future releases are likely to increase this value.
Keep in sync with the value in core-default.xml
public static final String CONNECTION_TTL
public static final Duration DEFAULT_CONNECTION_TTL_DURATION
CONNECTION_TTL
: 5 minutes.public static final long DEFAULT_CONNECTION_TTL
DEFAULT_CONNECTION_TTL_DURATION
CONNECTION_TTL
: 5 minutes.public static final String SECURE_CONNECTIONS
public static final boolean DEFAULT_SECURE_CONNECTIONS
public static final String ALLOW_REQUESTER_PAYS
public static final boolean DEFAULT_ALLOW_REQUESTER_PAYS
public static final String SSL_CHANNEL_MODE
public static final org.apache.hadoop.security.ssl.DelegatingSSLSocketFactory.SSLChannelMode DEFAULT_SSL_CHANNEL_MODE
public static final String ENDPOINT
public static final String DEFAULT_ENDPOINT
Note: previously this constant was defined as
CENTRAL_ENDPOINT
, however the actual
S3A client code used "" as the default when
ENDPOINT
was unset.
As core-default.xml also set the endpoint to "",
the empty string has long been the real
default value.
public static final String CENTRAL_ENDPOINT
public static final String PATH_STYLE_ACCESS
public static final String PROXY_HOST
public static final String PROXY_PORT
public static final String PROXY_USERNAME
public static final String PROXY_PASSWORD
public static final String PROXY_DOMAIN
public static final String PROXY_WORKSTATION
public static final String PROXY_SECURED
public static final String MAX_ERROR_RETRIES
public static final int DEFAULT_MAX_ERROR_RETRIES
@InterfaceStability.Unstable public static final String EXPERIMENTAL_AWS_INTERNAL_THROTTLING
@InterfaceStability.Unstable public static final boolean EXPERIMENTAL_AWS_INTERNAL_THROTTLING_DEFAULT
EXPERIMENTAL_AWS_INTERNAL_THROTTLING
,
value: true.public static final Duration MINIMUM_NETWORK_OPERATION_DURATION
public static final String ESTABLISH_TIMEOUT
public static final Duration DEFAULT_ESTABLISH_TIMEOUT_DURATION
public static final int DEFAULT_ESTABLISH_TIMEOUT
DEFAULT_ESTABLISH_TIMEOUT_DURATION
public static final String SOCKET_TIMEOUT
public static final Duration DEFAULT_SOCKET_TIMEOUT_DURATION
public static final int DEFAULT_SOCKET_TIMEOUT
DEFAULT_SOCKET_TIMEOUT_DURATION
DEFAULT_SOCKET_TIMEOUT_DURATION
.public static final String REQUEST_TIMEOUT
This is time to response, so for a GET request it is "time to 200 response"
not the time limit to download the requested data.
This makes it different from REQUEST_TIMEOUT
, which is for total
HTTP request.
Default unit is milliseconds.
There is a minimum duration set in MINIMUM_NETWORK_OPERATION_DURATION
;
it is impossible to set a delay less than this, even for testing.
Why so? Too many deployments where the configuration assumed the timeout was in seconds
and that "120" was a reasonable value rather than "too short to work reliably"
Note for anyone writing tests which need to set a low value for this:
to avoid the minimum duration overrides, call
AWSClientConfig.setMinimumOperationDuration()
and set a low value
before creating the filesystem.
public static final Duration DEFAULT_REQUEST_TIMEOUT_DURATION
public static final int DEFAULT_REQUEST_TIMEOUT
DEFAULT_REQUEST_TIMEOUT_DURATION
public static final String CONNECTION_ACQUISITION_TIMEOUT
public static final Duration DEFAULT_CONNECTION_ACQUISITION_TIMEOUT_DURATION
public static final String CONNECTION_KEEPALIVE
public static final boolean DEFAULT_CONNECTION_KEEPALIVE
CONNECTION_KEEPALIVE
: false.public static final String CONNECTION_IDLE_TIME
Too low: overhead of creating connections. Too high, risk of stale connections and inability to use the adaptive load balancing of the S3 front end.
Default unit is milliseconds for consistency with other options.
public static final Duration DEFAULT_CONNECTION_IDLE_TIME_DURATION
public static final String SOCKET_SEND_BUFFER
public static final int DEFAULT_SOCKET_SEND_BUFFER
public static final String SOCKET_RECV_BUFFER
public static final int DEFAULT_SOCKET_RECV_BUFFER
public static final String MAX_PAGING_KEYS
public static final int DEFAULT_MAX_PAGING_KEYS
public static final String MAX_THREADS
public static final int DEFAULT_MAX_THREADS
MAX_THREADS
: 96.public static final String KEEPALIVE_TIME
public static final Duration DEFAULT_KEEPALIVE_TIME_DURATION
KEEPALIVE_TIME
: 60s.public static final int DEFAULT_KEEPALIVE_TIME
DEFAULT_KEEPALIVE_TIME_DURATION
KEEPALIVE_TIME
: 60s.public static final String MULTIPART_SIZE
public static final long DEFAULT_MULTIPART_SIZE
public static final String MIN_MULTIPART_THRESHOLD
public static final long DEFAULT_MIN_MULTIPART_THRESHOLD
public static final String ENABLE_MULTI_DELETE
public static final String BULK_DELETE_PAGE_SIZE
public static final int BULK_DELETE_PAGE_SIZE_DEFAULT
public static final String BUFFER_DIR
@Deprecated public static final String FAST_UPLOAD
@Deprecated public static final boolean DEFAULT_FAST_UPLOAD
@Deprecated public static final String FAST_BUFFER_SIZE
public static final int DEFAULT_FAST_BUFFER_SIZE
public static final String FAST_UPLOAD_BUFFER
FAST_UPLOAD_BUFFER_DISK
Value: "fs.s3a.fast.upload.buffer"public static final String FAST_UPLOAD_BUFFER_DISK
public static final String FAST_UPLOAD_BUFFER_ARRAY
public static final String FAST_UPLOAD_BYTEBUFFER
FAST_UPLOAD_BUFFER_ARRAY
: "bytebuffer".public static final String DEFAULT_FAST_UPLOAD_BUFFER
public static final String FAST_UPLOAD_ACTIVE_BLOCKS
Default is DEFAULT_FAST_UPLOAD_ACTIVE_BLOCKS
public static final int DEFAULT_FAST_UPLOAD_ACTIVE_BLOCKS
public static final String DOWNGRADE_SYNCABLE_EXCEPTIONS
public static final boolean DOWNGRADE_SYNCABLE_EXCEPTIONS_DEFAULT
public static final String EXECUTOR_CAPACITY
FAST_UPLOAD_ACTIVE_BLOCKS
is used instead.
This should be less than MAX_THREADS
for fair
submission.
Value: "fs.s3a.executor.capacity".public static final int DEFAULT_EXECUTOR_CAPACITY
FAST_UPLOAD_ACTIVE_BLOCKS
is used instead.
Value: 16public static final String CANNED_ACL
public static final String DEFAULT_CANNED_ACL
public static final String CONTENT_ENCODING
public static final String STORAGE_CLASS
public static final String STORAGE_CLASS_STANDARD
public static final String STORAGE_CLASS_REDUCED_REDUNDANCY
public static final String STORAGE_CLASS_GLACIER
public static final String STORAGE_CLASS_STANDARD_INFREQUENT_ACCESS
public static final String STORAGE_CLASS_ONEZONE_INFREQUENT_ACCESS
public static final String STORAGE_CLASS_INTELLIGENT_TIERING
public static final String STORAGE_CLASS_DEEP_ARCHIVE
public static final String STORAGE_CLASS_OUTPOSTS
public static final String STORAGE_CLASS_GLACIER_INSTANT_RETRIEVAL
public static final String PURGE_EXISTING_MULTIPART
public static final boolean DEFAULT_PURGE_EXISTING_MULTIPART
public static final String PURGE_EXISTING_MULTIPART_AGE
public static final long DEFAULT_PURGE_EXISTING_MULTIPART_AGE
@Deprecated public static final String SERVER_SIDE_ENCRYPTION_ALGORITHM
S3AEncryptionMethods
for valid options.
"fs.s3a.server-side-encryption-algorithm"@Deprecated public static final String SERVER_SIDE_ENCRYPTION_AES256
@Deprecated public static final String SERVER_SIDE_ENCRYPTION_KEY
SERVER_SIDE_ENCRYPTION_ALGORITHM
is
SSE-KMS
(will default to aws/s3
master key if left blank).
With with SSE_C
, the base-64 encoded AES 256 key.
May be set within a JCEKS file.
Value: ""fs.s3a.server-side-encryption.key"".public static final String S3_ENCRYPTION_ALGORITHM
S3AEncryptionMethods
for valid options.
public static final String S3_ENCRYPTION_KEY
SERVER_SIDE_ENCRYPTION_KEY
public static final String CUSTOM_SIGNERS
public static final String SIGNING_ALGORITHM
fs.s3a.sts.signing-algorithm
: algorithm to use for STS interaction.public static final String SIGNING_ALGORITHM_S3
@Deprecated public static final String SIGNING_ALGORITHM_DDB
public static final String SIGNING_ALGORITHM_STS
public static final String S3N_FOLDER_SUFFIX
public static final String FS_S3A_BLOCK_SIZE
public static final String FS_S3A
public static final String FS_S3A_PREFIX
public static final String FS_S3A_BUCKET_PREFIX
public static final int S3A_DEFAULT_PORT
public static final String USER_AGENT_PREFIX
public static final String AUTHORITATIVE_PATH
public static final String[] DEFAULT_AUTHORITATIVE_PATH
@Deprecated public static final String METADATASTORE_AUTHORITATIVE
@Deprecated public static final boolean DEFAULT_METADATASTORE_AUTHORITATIVE
public static final String S3A_BUCKET_PROBE
S3AFileSystem.initialize(URI, Configuration)
.
Value: "fs.s3a.bucket.probe"public static final int S3A_BUCKET_PROBE_DEFAULT
S3AFileSystem.verifyBucketExistsV2()
.
Value: 0@Deprecated public static final String METADATASTORE_METADATA_TTL
@Deprecated public static final long DEFAULT_METADATASTORE_METADATA_TTL
public static final String READAHEAD_RANGE
public static final long DEFAULT_READAHEAD_RANGE
public static final String ASYNC_DRAIN_THRESHOLD
public static final int DEFAULT_ASYNC_DRAIN_THRESHOLD
ITestS3AInputStreamPerformance
.
Value: 16000public static final String INPUT_FADVISE
public static final String INPUT_FADV_DEFAULT
Options.OpenFileOptions
value
in code which only needs to be compiled against newer hadoop
releases.public static final String INPUT_FADV_NORMAL
public static final String INPUT_FADV_SEQUENTIAL
Options.OpenFileOptions
value
in code which only needs to be compiled against newer hadoop
releases.public static final String INPUT_FADV_RANDOM
Options.OpenFileOptions
value
in code which only needs to be compiled against newer hadoop
releases.seek()
operations.
Value: "random"public static final String STREAM_READ_GAUGE_INPUT_POLICY
@InterfaceAudience.Private @InterfaceStability.Unstable public static final String S3_CLIENT_FACTORY_IMPL
@InterfaceAudience.Private @InterfaceStability.Unstable public static final Class<? extends S3ClientFactory> DEFAULT_S3_CLIENT_FACTORY_IMPL
org.apache.hadoop.fs.s3a.DefaultS3ClientFactory
.@InterfaceAudience.Private public static final int MAX_MULTIPART_COUNT
@Deprecated public static final String S3_METADATA_STORE_IMPL
@Deprecated public static final String FAIL_ON_METADATA_WRITE_ERROR
@Deprecated public static final boolean FAIL_ON_METADATA_WRITE_ERROR_DEFAULT
@InterfaceStability.Unstable @Deprecated public static final String S3GUARD_CLI_PRUNE_AGE
@Deprecated public static final String S3GUARD_DDB_REGION_KEY
@Deprecated public static final String S3GUARD_DDB_TABLE_NAME_KEY
@Deprecated public static final String S3GUARD_DDB_TABLE_TAG
@Deprecated public static final String S3GUARD_DDB_TABLE_CREATE_KEY
@Deprecated public static final String S3GUARD_DDB_TABLE_CAPACITY_READ_KEY
@Deprecated public static final long S3GUARD_DDB_TABLE_CAPACITY_READ_DEFAULT
@Deprecated public static final String S3GUARD_DDB_TABLE_CAPACITY_WRITE_KEY
@Deprecated public static final long S3GUARD_DDB_TABLE_CAPACITY_WRITE_DEFAULT
@Deprecated public static final String S3GUARD_DDB_TABLE_SSE_ENABLED
@Deprecated public static final String S3GUARD_DDB_TABLE_SSE_CMK
@Deprecated public static final int S3GUARD_DDB_BATCH_WRITE_REQUEST_LIMIT
@Deprecated public static final String S3GUARD_DDB_MAX_RETRIES
@Deprecated public static final int S3GUARD_DDB_MAX_RETRIES_DEFAULT
@Deprecated public static final String S3GUARD_DDB_THROTTLE_RETRY_INTERVAL
@Deprecated public static final String S3GUARD_DDB_THROTTLE_RETRY_INTERVAL_DEFAULT
@Deprecated @InterfaceStability.Unstable public static final String S3GUARD_DDB_BACKGROUND_SLEEP_MSEC_KEY
@Deprecated public static final int S3GUARD_DDB_BACKGROUND_SLEEP_MSEC_DEFAULT
@Deprecated public static final String S3GUARD_METASTORE_NULL
@Deprecated @InterfaceStability.Unstable public static final String S3GUARD_METASTORE_LOCAL
@InterfaceStability.Unstable @Deprecated public static final String S3GUARD_METASTORE_LOCAL_MAX_RECORDS
@Deprecated public static final int DEFAULT_S3GUARD_METASTORE_LOCAL_MAX_RECORDS
@InterfaceStability.Unstable @Deprecated public static final String S3GUARD_METASTORE_LOCAL_ENTRY_TTL
@Deprecated public static final int DEFAULT_S3GUARD_METASTORE_LOCAL_ENTRY_TTL
@Deprecated public static final String S3GUARD_METASTORE_DYNAMO
@Deprecated public static final String S3GUARD_DISABLED_WARN_LEVEL
@Deprecated public static final String DEFAULT_S3GUARD_DISABLED_WARN_LEVEL
@Deprecated public static final String FAIL_INJECT_INCONSISTENCY_KEY
@Deprecated public static final String FAIL_INJECT_INCONSISTENCY_MSEC
@Deprecated public static final String FAIL_INJECT_INCONSISTENCY_PROBABILITY
@InterfaceStability.Unstable public static final String LIST_VERSION
@InterfaceStability.Unstable public static final int DEFAULT_LIST_VERSION
@InterfaceStability.Unstable public static final String FAIL_INJECT_THROTTLE_PROBABILITY
@InterfaceStability.Unstable public static final String FAIL_INJECT_CLIENT_FACTORY
public static final String RETRY_LIMIT
public static final int RETRY_LIMIT_DEFAULT
public static final String RETRY_INTERVAL
public static final String RETRY_INTERVAL_DEFAULT
public static final String RETRY_THROTTLE_LIMIT
public static final int RETRY_THROTTLE_LIMIT_DEFAULT
public static final String RETRY_THROTTLE_INTERVAL
public static final String RETRY_THROTTLE_INTERVAL_DEFAULT
public static final String ETAG_CHECKSUM_ENABLED
public static final boolean ETAG_CHECKSUM_ENABLED_DEFAULT
public static final String CHANGE_DETECT_SOURCE
public static final String CHANGE_DETECT_SOURCE_ETAG
public static final String CHANGE_DETECT_SOURCE_VERSION_ID
public static final String CHANGE_DETECT_SOURCE_DEFAULT
public static final String CHANGE_DETECT_MODE
public static final String CHANGE_DETECT_MODE_CLIENT
RemoteFileChangedException
.public static final String CHANGE_DETECT_MODE_SERVER
RemoteFileChangedException
is
thrown.public static final String CHANGE_DETECT_MODE_WARN
public static final String CHANGE_DETECT_MODE_NONE
public static final String CHANGE_DETECT_MODE_DEFAULT
public static final String CHANGE_DETECT_REQUIRE_VERSION
RemoteFileChangedException
exception when S3
doesn't provide the attribute defined by fs.s3a.change.detection.source.
For example, if source is versionId, but object versioning is not enabled
on the bucket, or alternatively if source is eTag and a third-party S3
implementation that doesn't return eTag is used.
When false, only a warning message will be logged for this condition.
public static final boolean CHANGE_DETECT_REQUIRE_VERSION_DEFAULT
@Deprecated public static final String S3GUARD_CONSISTENCY_RETRY_LIMIT
@Deprecated public static final int S3GUARD_CONSISTENCY_RETRY_LIMIT_DEFAULT
@Deprecated public static final String S3GUARD_CONSISTENCY_RETRY_INTERVAL
@Deprecated public static final String S3GUARD_CONSISTENCY_RETRY_INTERVAL_DEFAULT
public static final String AWS_SERVICE_IDENTIFIER_S3
@Deprecated public static final String AWS_SERVICE_IDENTIFIER_DDB
public static final String AWS_SERVICE_IDENTIFIER_STS
public static final int THREAD_POOL_SHUTDOWN_DELAY_SECONDS
public static final String DIRECTORY_MARKER_POLICY
public static final String DIRECTORY_MARKER_POLICY_DELETE
public static final String DIRECTORY_MARKER_POLICY_KEEP
public static final String DIRECTORY_MARKER_POLICY_AUTHORITATIVE
public static final String DEFAULT_DIRECTORY_MARKER_POLICY
public static final String STORE_CAPABILITY_DIRECTORY_MARKER_AWARE
PathCapabilities
probe to verify that an S3A Filesystem
has the changes needed to safely work with buckets where
directoy markers have not been deleted.
Value: "fs.s3a.capability.directory.marker.aware".public static final String STORE_CAPABILITY_DIRECTORY_MARKER_POLICY_KEEP
PathCapabilities
probe to indicate that the filesystem
keeps directory markers.
Value: "fs.s3a.capability.directory.marker.policy.keep".public static final String STORE_CAPABILITY_DIRECTORY_MARKER_POLICY_DELETE
PathCapabilities
probe to indicate that the filesystem
deletes directory markers.
Value: "fs.s3a.capability.directory.marker.policy.delete".public static final String STORE_CAPABILITY_DIRECTORY_MARKER_POLICY_AUTHORITATIVE
PathCapabilities
probe to indicate that the filesystem
keeps directory markers in authoritative paths only.
Value: "fs.s3a.capability.directory.marker.policy.authoritative".public static final String STORE_CAPABILITY_DIRECTORY_MARKER_ACTION_KEEP
PathCapabilities
probe to indicate that a path
keeps directory markers.
Value: "fs.s3a.capability.directory.marker.action.keep".public static final String STORE_CAPABILITY_DIRECTORY_MARKER_ACTION_DELETE
PathCapabilities
probe to indicate that a path
deletes directory markers.
Value: "fs.s3a.capability.directory.marker.action.delete".public static final String XA_HEADER_PREFIX
public static final String AWS_REGION
public static final String AWS_S3_CENTRAL_REGION
public static final String AWS_S3_DEFAULT_REGION
public static final String FIPS_ENDPOINT
public static final boolean ENDPOINT_FIPS_DEFAULT
public static final String AWS_S3_ACCESSPOINT_REQUIRED
public static final String FS_S3A_CREATE_PERFORMANCE
public static final boolean FS_S3A_CREATE_PERFORMANCE_DEFAULT
public static final String FS_S3A_CREATE_PERFORMANCE_ENABLED
FS_S3A_CREATE_PERFORMANCE
set to true.
Value "fs.s3a.create.performance.enabled".public static final String FS_S3A_CREATE_HEADER
public static final String AWS_S3_VECTOR_READS_MIN_SEEK_SIZE
public static final String AWS_S3_VECTOR_READS_MAX_MERGED_READ_SIZE
public static final int DEFAULT_AWS_S3_VECTOR_READS_MIN_SEEK_SIZE
public static final int DEFAULT_AWS_S3_VECTOR_READS_MAX_MERGED_READ_SIZE
public static final String AWS_S3_VECTOR_ACTIVE_RANGE_READS
Default is DEFAULT_AWS_S3_VECTOR_ACTIVE_RANGE_READS
public static final int DEFAULT_AWS_S3_VECTOR_ACTIVE_RANGE_READS
public static final String AWS_AUTH_CLASS_PREFIX
public static final String PREFETCH_ENABLED_KEY
public static final boolean PREFETCH_ENABLED_DEFAULT
public static final String PREFETCH_BLOCK_SIZE_KEY
public static final int PREFETCH_BLOCK_DEFAULT_SIZE
public static final String PREFETCH_BLOCK_COUNT_KEY
public static final int PREFETCH_BLOCK_DEFAULT_COUNT
public static final String MULTIPART_UPLOADS_ENABLED
Default is DEFAULT_MULTIPART_UPLOAD_ENABLED
.
public static final boolean DEFAULT_MULTIPART_UPLOAD_ENABLED
public static final String STORE_CAPABILITY_MULTIPART_UPLOAD_ENABLED
@Deprecated public static final String STORE_CAPABILITY_DIRECTORY_MARKER_MULTIPART_UPLOAD_ENABLED
public static final String PREFETCH_MAX_BLOCKS_COUNT
public static final int DEFAULT_PREFETCH_MAX_BLOCKS_COUNT
public static final String BUCKET_REGION_HEADER
public static final String DIRECTORY_OPERATIONS_PURGE_UPLOADS
public static final boolean DIRECTORY_OPERATIONS_PURGE_UPLOADS_DEFAULT
DIRECTORY_OPERATIONS_PURGE_UPLOADS
: false.public static final String OPTIMIZED_COPY_FROM_LOCAL
public static final boolean OPTIMIZED_COPY_FROM_LOCAL_DEFAULT
OPTIMIZED_COPY_FROM_LOCAL
.
Value: true.public static final String STORE_CAPABILITY_AWS_V2
public static final String S3EXPRESS_CREATE_SESSION
Value: "fs.s3a.s3express.create.session".
This is preferred as it is faster, but it does require extra IAM permissions and is not suited to some deployments, including some of the hadoop-aws test suites.
public static final boolean S3EXPRESS_CREATE_SESSION_DEFAULT
S3EXPRESS_CREATE_SESSION
.
Value: true.public static final String HTTP_SIGNER_ENABLED
public static final boolean HTTP_SIGNER_ENABLED_DEFAULT
HTTP_SIGNER_ENABLED
: false.public static final String HTTP_SIGNER_CLASS_NAME
HTTP_SIGNER_ENABLED
is true: "fs.s3a.http.signer.class".public static final String CHECKSUM_VALIDATION
public static final boolean CHECKSUM_VALIDATION_DEFAULT
CHECKSUM_VALIDATION
.
Value: false.public static final String AWS_S3_CLASSLOADER_ISOLATION
fs.s3a.aws.credentials.provider
,
going to be loaded from the same classloader that loaded
the S3AFileSystem
?
It is useful to turn classloader isolation off for Apache Spark applications
that might load S3AFileSystem
from the Spark distribution (Launcher classloader)
while users might want to provide custom extensions (loaded by Spark MutableClassloader).
Value: "fs.s3a.classloader.isolation".public static final boolean DEFAULT_AWS_S3_CLASSLOADER_ISOLATION
AWS_S3_CLASSLOADER_ISOLATION
.
Value: true.Copyright © 2008–2024 Apache Software Foundation. All rights reserved.