com.google.cloud.hadoop.util

Class HadoopCredentialConfiguration

java.lang.Object

com.google.cloud.hadoop.util.HadoopCredentialConfiguration

public class HadoopCredentialConfiguration
extends Object

Hadoop credential configuration.

When reading configuration this class makes use of a list of key prefixes that are each applied to key suffixes to create a complete configuration key. There is a base prefix of 'google.cloud.' that is included by the builder for each EntriesCredentialConfiguration created. When constructing, other prefixes can be specified. Prefixes specified later can be used to override the values of previously set values. In this way a set of global credentials can be specified for most connectors with an override specified for any connectors that need different credentials.

Field Summary

Fields

Modifier and Type	Field and Description
static HadoopConfigurationProperty<Class<? extends AccessTokenProvider>>	ACCESS_TOKEN_PROVIDER_IMPL_SUFFIX Configuration key for the name of the AccessTokenProvider to use to generate AccessTokens.
static HadoopConfigurationProperty<String>	AUTH_CLIENT_ID_SUFFIX Configuration key for defining the OAUth2 client ID.
static HadoopConfigurationProperty<RedactedString>	AUTH_CLIENT_SECRET_SUFFIX Configuration key for defining the OAUth2 client secret.
static HadoopConfigurationProperty<RedactedString>	AUTH_REFRESH_TOKEN_SUFFIX Configuration key for defining the OAuth2 refresh token.
static String	BASE_KEY_PREFIX All instances constructed using the builder will use google.cloud as the first prefix checked.
static HadoopConfigurationProperty<Boolean>	ENABLE_NULL_CREDENTIAL_SUFFIX For unit-testing, the key suffix allowing null to be returned from credential creation instead of logging an error and aborting.
static HadoopConfigurationProperty<Boolean>	ENABLE_SERVICE_ACCOUNTS_SUFFIX Key suffix for enabling GCE service account authentication.
static HadoopConfigurationProperty<Map<String,String>>	GROUP_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX Key prefix for the group identifier associated with the service account to impersonate when accessing GCS.
static HadoopConfigurationProperty<HttpTransportFactory.HttpTransportType>	HTTP_TRANSPORT_SUFFIX Configuration key for the name of HttpTransport class to use for connecting to GCS.
static HadoopConfigurationProperty<String>	IMPERSONATION_SERVICE_ACCOUNT_SUFFIX Key suffix specifying the impersonating service account with which to call GCS API to get access token.
static HadoopConfigurationProperty<String>	PROXY_ADDRESS_SUFFIX Configuration key for setting a proxy for the connector to use to connect to GCS.
static HadoopConfigurationProperty<String>	PROXY_PASSWORD_SUFFIX Configuration key for setting a proxy password for the connector to use to authenticate with proxy used to connect to GCS.
static HadoopConfigurationProperty<String>	PROXY_USERNAME_SUFFIX Configuration key for setting a proxy username for the connector to use to authenticate with proxy used to connect to GCS.
static HadoopConfigurationProperty<String>	SERVICE_ACCOUNT_EMAIL_SUFFIX Key suffix specifying the email address associated with the service account with which to authenticate.
static HadoopConfigurationProperty<String>	SERVICE_ACCOUNT_JSON_KEYFILE_SUFFIX Key suffix used to indicate the path to a JSON file containing a Service Account key and identifier (email).
static HadoopConfigurationProperty<String>	SERVICE_ACCOUNT_KEYFILE_SUFFIX Key suffix specifying local file containing a service account private .p12 keyfile.
static HadoopConfigurationProperty<String>	SERVICE_ACCOUNT_PRIVATE_KEY_ID_SUFFIX Key suffix used to specify private key id for the service account.
static HadoopConfigurationProperty<String>	SERVICE_ACCOUNT_PRIVATE_KEY_SUFFIX Key suffix used to specify private key for the service account.
static HadoopConfigurationProperty<String>	TOKEN_SERVER_URL_SUFFIX Configuration key for setting a token server URL to use to refresh OAuth token.
static HadoopConfigurationProperty<Map<String,String>>	USER_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX Key prefix for the user identifier associated with the service account to impersonate when accessing GCS.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	HadoopCredentialConfiguration()

Method Summary

Modifier and Type	Method and Description
static AccessTokenProvider	getAccessTokenProvider(org.apache.hadoop.conf.Configuration config, List<String> keyPrefixes) Creates an AccessTokenProvider based on the configuration.
static Class<? extends AccessTokenProvider>	getAccessTokenProviderImplClass(org.apache.hadoop.conf.Configuration config, String... keyPrefixes)
static com.google.common.collect.ImmutableList<String>	getConfigKeyPrefixes(String... keyPrefixes) Returns full list of config prefixes that will be resolved based on the order in returned list.
static CredentialFactory	getCredentialFactory(org.apache.hadoop.conf.Configuration config, String... keyPrefixesVararg)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BASE_KEY_PREFIX

public static final String BASE_KEY_PREFIX

All instances constructed using the builder will use google.cloud as the first prefix checked. Other prefixes can be added and will override values in the google.cloud prefix.

See Also:

Constant Field Values

ENABLE_SERVICE_ACCOUNTS_SUFFIX

public static final HadoopConfigurationProperty<Boolean> ENABLE_SERVICE_ACCOUNTS_SUFFIX

Key suffix for enabling GCE service account authentication. A value of false will disable the use of the service accounts for authentication. The default value is true - use a service account for authentication.

SERVICE_ACCOUNT_EMAIL_SUFFIX

public static final HadoopConfigurationProperty<String> SERVICE_ACCOUNT_EMAIL_SUFFIX

Key suffix specifying the email address associated with the service account with which to authenticate. Only required if ENABLE_SERVICE_ACCOUNTS_SUFFIX is true and we're using SERVICE_ACCOUNT_KEYFILE_SUFFIX to authenticate with a private keyfile.

SERVICE_ACCOUNT_PRIVATE_KEY_ID_SUFFIX

public static final HadoopConfigurationProperty<String> SERVICE_ACCOUNT_PRIVATE_KEY_ID_SUFFIX

Key suffix used to specify private key id for the service account.

SERVICE_ACCOUNT_PRIVATE_KEY_SUFFIX

public static final HadoopConfigurationProperty<String> SERVICE_ACCOUNT_PRIVATE_KEY_SUFFIX

Key suffix used to specify private key for the service account.

SERVICE_ACCOUNT_KEYFILE_SUFFIX

public static final HadoopConfigurationProperty<String> SERVICE_ACCOUNT_KEYFILE_SUFFIX

Key suffix specifying local file containing a service account private .p12 keyfile. Only used if SERVICE_ACCOUNT_EMAIL_SUFFIX is true; if provided, triggers private keyfile service account authentication. The file will be required to be present on all nodes and at the same location on all nodes.

SERVICE_ACCOUNT_JSON_KEYFILE_SUFFIX

public static final HadoopConfigurationProperty<String> SERVICE_ACCOUNT_JSON_KEYFILE_SUFFIX

Key suffix used to indicate the path to a JSON file containing a Service Account key and identifier (email). Technically, this could be a JSON containing a non-service account user, but this setting is only used in the service account flow and is namespaced as such.

ENABLE_NULL_CREDENTIAL_SUFFIX

public static final HadoopConfigurationProperty<Boolean> ENABLE_NULL_CREDENTIAL_SUFFIX

For unit-testing, the key suffix allowing null to be returned from credential creation instead of logging an error and aborting.

TOKEN_SERVER_URL_SUFFIX

public static final HadoopConfigurationProperty<String> TOKEN_SERVER_URL_SUFFIX

Configuration key for setting a token server URL to use to refresh OAuth token.

PROXY_ADDRESS_SUFFIX

public static final HadoopConfigurationProperty<String> PROXY_ADDRESS_SUFFIX

Configuration key for setting a proxy for the connector to use to connect to GCS. The proxy must be an HTTP proxy of the form "host:port".

PROXY_USERNAME_SUFFIX

public static final HadoopConfigurationProperty<String> PROXY_USERNAME_SUFFIX

Configuration key for setting a proxy username for the connector to use to authenticate with proxy used to connect to GCS.

PROXY_PASSWORD_SUFFIX

public static final HadoopConfigurationProperty<String> PROXY_PASSWORD_SUFFIX

Configuration key for setting a proxy password for the connector to use to authenticate with proxy used to connect to GCS.

HTTP_TRANSPORT_SUFFIX

public static final HadoopConfigurationProperty<HttpTransportFactory.HttpTransportType> HTTP_TRANSPORT_SUFFIX

Configuration key for the name of HttpTransport class to use for connecting to GCS. Must be the name of an HttpTransportFactory.HttpTransportType (APACHE or JAVA_NET).

ACCESS_TOKEN_PROVIDER_IMPL_SUFFIX

public static final HadoopConfigurationProperty<Class<? extends AccessTokenProvider>> ACCESS_TOKEN_PROVIDER_IMPL_SUFFIX

Configuration key for the name of the AccessTokenProvider to use to generate AccessTokens.

IMPERSONATION_SERVICE_ACCOUNT_SUFFIX

public static final HadoopConfigurationProperty<String> IMPERSONATION_SERVICE_ACCOUNT_SUFFIX

Key suffix specifying the impersonating service account with which to call GCS API to get access token.

USER_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX

public static final HadoopConfigurationProperty<Map<String,String>> USER_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX

Key prefix for the user identifier associated with the service account to impersonate when accessing GCS.

GROUP_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX

public static final HadoopConfigurationProperty<Map<String,String>> GROUP_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX

Key prefix for the group identifier associated with the service account to impersonate when accessing GCS.

AUTH_CLIENT_ID_SUFFIX

public static final HadoopConfigurationProperty<String> AUTH_CLIENT_ID_SUFFIX

Configuration key for defining the OAUth2 client ID.

AUTH_CLIENT_SECRET_SUFFIX

public static final HadoopConfigurationProperty<RedactedString> AUTH_CLIENT_SECRET_SUFFIX

Configuration key for defining the OAUth2 client secret.

AUTH_REFRESH_TOKEN_SUFFIX

public static final HadoopConfigurationProperty<RedactedString> AUTH_REFRESH_TOKEN_SUFFIX

Configuration key for defining the OAuth2 refresh token.

Constructor Detail

HadoopCredentialConfiguration

protected HadoopCredentialConfiguration()

Method Detail

getCredentialFactory

public static CredentialFactory getCredentialFactory(org.apache.hadoop.conf.Configuration config,
                                                     String... keyPrefixesVararg)

getAccessTokenProvider

public static AccessTokenProvider getAccessTokenProvider(org.apache.hadoop.conf.Configuration config,
                                                         List<String> keyPrefixes)
                                                  throws IOException

Creates an AccessTokenProvider based on the configuration.

Throws:

IOException

getAccessTokenProviderImplClass

public static Class<? extends AccessTokenProvider> getAccessTokenProviderImplClass(org.apache.hadoop.conf.Configuration config,
                                                                                   String... keyPrefixes)

getConfigKeyPrefixes

public static com.google.common.collect.ImmutableList<String> getConfigKeyPrefixes(String... keyPrefixes)

Returns full list of config prefixes that will be resolved based on the order in returned list.