Package | Description |
---|---|
com.google.cloud.hadoop.fs.gcs |
Modifier and Type | Field and Description |
---|---|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.AUTH_CLIENT_ID
Configuration key for GCS client ID.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.AUTH_CLIENT_SECRET
Configuration key for GCS client secret.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.AUTH_SERVICE_ACCOUNT_EMAIL
Configuration key specifying the email address of the service-account with which to
authenticate.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.AUTH_SERVICE_ACCOUNT_ENABLE
Configuration key for enabling GCE service account authentication.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.AUTH_SERVICE_ACCOUNT_KEY_FILE
Configuration key specifying local file containing a service-account private .p12 keyfile.
|
static GoogleHadoopFileSystemConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.BLOCK_SIZE
Configuration key for default block size of a file.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.DELEGATION_TOKEN_BINDING_CLASS
Configuration key for Delegation Token binding class.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCE_BUCKET_DELETE_ENABLE
If true, recursive delete on a path that refers to a GCS bucket itself ('/' for any
bucket-rooted GoogleHadoopFileSystem) or delete on that path when it's empty will result in
fully deleting the GCS bucket.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_APPLICATION_NAME_SUFFIX
Configuration key for adding a suffix to the GHFS application name sent to GCS.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_BATCH_THREADS
Configuration key for a number of threads to execute batch requests.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_CONCURRENT_GLOB_ENABLE
Configuration key for enabling the use of flat and regular glob search algorithms in two
parallel threads.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_CONFIG_OVERRIDE_FILE
Override configuration file path.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_COOPERATIVE_LOCKING_ENABLE
Configuration key for using cooperative locking to achieve a directory mutation operations
isolation.
|
static GoogleHadoopFileSystemConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_COOPERATIVE_LOCKING_EXPIRATION_TIMEOUT_MS
Configuration key for lock expiration when using cooperative locking.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_COOPERATIVE_LOCKING_MAX_CONCURRENT_OPERATIONS
Configuration key for maximum allowed concurrent operations when using cooperative locking.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_COPY_BATCH_THREADS
Configuration key for a number of threads to execute batch requests for copy operations.
|
static GoogleHadoopFileSystemConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_COPY_MAX_REQUESTS_PER_BATCH
Configuration key for a max number of GCS RPCs in batch request for copy operations.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_COPY_WITH_REWRITE_ENABLE
Configuration key for enabling the use of Rewrite requests for copy operations.
|
static GoogleHadoopFileSystemConfigurationProperty<GoogleHadoopFileSystemBase.GcsFileChecksumType> |
GoogleHadoopFileSystemConfiguration.GCS_FILE_CHECKSUM_TYPE
Configuration key for which type of FileChecksum to return; if a particular file doesn't
support the requested type, then getFileChecksum() will return null for that file.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_FLAT_GLOB_ENABLE
Configuration key for enabling the use of a large flat listing to pre-populate possible glob
matches in a single API call before running the core globbing logic in-memory rather than
sequentially and recursively performing API calls.
|
static GoogleHadoopFileSystemConfigurationProperty<GoogleCloudStorageReadOptions.GenerationReadConsistency> |
GoogleHadoopFileSystemConfiguration.GCS_GENERATION_READ_CONSISTENCY
Configuration key for the generation consistency read model.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_HTTP_CONNECT_TIMEOUT
Configuration key for the connect timeout (in millisecond) for HTTP request to GCS.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_HTTP_MAX_RETRY
Configuration key for the max number of retries for failed HTTP request to GCS.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_HTTP_READ_TIMEOUT
Configuration key for the connect timeout (in millisecond) for HTTP request to GCS.
|
static GoogleHadoopFileSystemConfigurationProperty<HttpTransportFactory.HttpTransportType> |
GoogleHadoopFileSystemConfiguration.GCS_HTTP_TRANSPORT
Configuration key for the name of HttpTransport class to use for connecting to GCS.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_INFER_IMPLICIT_DIRECTORIES_ENABLE
Configuration key for enabling automatic inference of implicit directories.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_BUFFER_SIZE
Configuration key for setting read buffer size.
|
static GoogleHadoopFileSystemConfigurationProperty<GoogleCloudStorageReadOptions.Fadvise> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FADVISE
Tunes reading objects behavior to optimize HTTP GET requests for various use cases.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE
If true, on opening a file we will proactively perform a metadata GET to check whether the
object exists, even though the underlying channel will not open a data stream until read() is
actually called so that streams can seek to nonzero file positions without incurring an extra
stream creation.
|
static GoogleHadoopFileSystemConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_INPLACE_SEEK_LIMIT
If forward seeks are within this many bytes of the current position, seeks are performed by
reading and discarding bytes in-place rather than opening a new underlying stream.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_MIN_RANGE_REQUEST_SIZE
Minimum size in bytes of the HTTP Range header set in GCS request when opening new stream to
read an object.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_SUPPORT_GZIP_ENCODING_ENABLE
If true, reading a file with GZIP content encoding (HTTP header "Content-Encoding: gzip") will
result in failure (IOException is thrown).
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_LAZY_INITIALIZATION_ENABLE
Configuration key for enabling lazy initialization of GCS FS instance.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_MARKER_FILE_PATTERN
Configuration key for marker file pattern.
|
static GoogleHadoopFileSystemConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_MAX_LIST_ITEMS_PER_CALL
Configuration key for number of items to return per call to the list* GCS RPCs.
|
static GoogleHadoopFileSystemConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_MAX_REQUESTS_PER_BATCH
Configuration key for a max number of GCS RPCs in batch request.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE
Configuration key for modifying the maximum amount of time to wait for empty object creation.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_BUFFER_SIZE
Configuration key for setting write buffer size.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_DIRECT_UPLOAD_ENABLE
Configuration key for enabling GCS direct upload.
|
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_PIPE_BUFFER_SIZE
Configuration key for setting pipe buffer size.
|
static GoogleHadoopFileSystemConfigurationProperty<GoogleHadoopFileSystemBase.OutputStreamType> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_TYPE
Configuration key for which type of output stream to use; different options may have different
degrees of support for advanced features like
hsync() and different performance
characteristics. |
static GoogleHadoopFileSystemConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_UPLOAD_CHUNK_SIZE
Configuration key for setting GCS upload chunk size.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_ENABLE
Configuration key for using a local item cache to supplement GCS API "getFile" results.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE
Configuration key for whether or not to enable list caching for the performance cache.
|
static GoogleHadoopFileSystemConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS
Configuration key for maximum number of milliseconds a GoogleCloudStorageItemInfo will remain
"valid" in the performance cache before it's invalidated.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_PROJECT_ID
Configuration key for GCS project ID.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_PROXY_ADDRESS
Configuration key for setting a proxy for the connector to use to connect to GCS.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_PROXY_PASSWORD
Configuration key for setting a proxy password for the connector to use to authenticate with
proxy used to connect to GCS.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_PROXY_USERNAME
Configuration key for setting a proxy username for the connector to use to authenticate with
proxy used to connect to GCS.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_REPAIR_IMPLICIT_DIRECTORIES_ENABLE
Configuration key for enabling automatic repair of implicit directories whenever detected
inside delete and rename calls.
|
static GoogleHadoopFileSystemConfigurationProperty<Collection<String>> |
GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_BUCKETS
Configuration key for GCS Requester Pays Buckets.
|
static GoogleHadoopFileSystemConfigurationProperty<RequesterPaysOptions.RequesterPaysMode> |
GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_MODE
Configuration key for GCS project ID.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_PROJECT_ID
Configuration key for GCS Requester Pays Project ID.
|
static GoogleHadoopFileSystemConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_REWRITE_MAX_BYTES_PER_CALL
Configuration key for specifying max number of bytes rewritten in a single rewrite request when
fs.gs.copy.with.rewrite.enable is set to 'true'.
|
static GoogleHadoopFileSystemConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_STATUS_PARALLEL_ENABLE
If true, executes GCS requests in
listStatus and getFileStatus methods in
parallel to reduce latency. |
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_WORKING_DIRECTORY
Configuration key for initial working directory of a GHFS instance.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.PATH_CODEC
Configuration key for changing the path codec from legacy to 'uri path encoding'.
|
static GoogleHadoopFileSystemConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.PERMISSIONS_TO_REPORT
Key for the permissions that we report a file or directory to have.
|
Copyright © 2019. All rights reserved.