org.apache.hadoop.fs.s3native

Class S3xLoginHelper

java.lang.Object

org.apache.hadoop.fs.s3native.S3xLoginHelper

public final class S3xLoginHelper
extends Object

Class to aid logging in to S3 endpoints. It is in S3N so that it can be used across all S3 filesystems.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	S3xLoginHelper.Login Simple tuple of login details.

Field Summary

Fields

Modifier and Type	Field and Description
static String	LOGIN_WARNING
static String	PLUS_ENCODED
static String	PLUS_UNENCODED
static String	PLUS_WARNING

Method Summary

Methods

Modifier and Type	Method and Description
static URI	buildFSURI(URI uri) Build the filesystem URI.
static URI	canonicalizeUri(URI uri, int defaultPort) Canonicalize the given URI.
static void	checkPath(org.apache.hadoop.conf.Configuration conf, URI fsUri, org.apache.hadoop.fs.Path path, int defaultPort) Check the path, ignoring authentication details.
static S3xLoginHelper.Login	extractLoginDetails(URI name) Extract the login details from a URI.
static S3xLoginHelper.Login	extractLoginDetailsWithWarnings(URI name) Extract the login details from a URI, logging a warning if the URI contains these.
static String	toString(URI pathUri) Create a stripped down string value for error messages.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGIN_WARNING

public static final String LOGIN_WARNING

See Also:

Constant Field Values

PLUS_WARNING

public static final String PLUS_WARNING

See Also:

Constant Field Values

PLUS_UNENCODED

public static final String PLUS_UNENCODED

See Also:

Constant Field Values

PLUS_ENCODED

public static final String PLUS_ENCODED

See Also:

Constant Field Values

Method Detail

buildFSURI

public static URI buildFSURI(URI uri)

Build the filesystem URI. This can include stripping down of part of the URI.

Parameters:

uri - filesystem uri

Returns:

the URI to use as the basis for FS operation and qualifying paths.

Throws:

IllegalArgumentException - if the URI is in some way invalid.

toString

public static String toString(URI pathUri)

Create a stripped down string value for error messages.

Parameters:

pathUri - URI

Returns:

a shortened schema://host/path value

extractLoginDetailsWithWarnings

public static S3xLoginHelper.Login extractLoginDetailsWithWarnings(URI name)

Extract the login details from a URI, logging a warning if the URI contains these.

Parameters:

name - URI of the filesystem

Returns:

a login tuple, possibly empty.

extractLoginDetails

public static S3xLoginHelper.Login extractLoginDetails(URI name)

Extract the login details from a URI.

Parameters:

name - URI of the filesystem

Returns:

a login tuple, possibly empty.

canonicalizeUri

public static URI canonicalizeUri(URI uri,
                  int defaultPort)

Canonicalize the given URI. This strips out login information.

Parameters:

uri - the URI to canonicalize

defaultPort - default port to use in canonicalized URI if the input URI has no port and this value is greater than 0

Returns:

a new, canonicalized URI.

checkPath

public static void checkPath(org.apache.hadoop.conf.Configuration conf,
             URI fsUri,
             org.apache.hadoop.fs.Path path,
             int defaultPort)

Check the path, ignoring authentication details. See FileSystem.checkPath(Path) for the operation of this. Essentially

1. The URI is canonicalized.
2. If the schemas match, the hosts are compared.
3. If there is a mismatch between null/non-null host, the default FS values are used to patch in the host.

That all originates in the core FS; the sole change here being to use URI.getHost() over URI.getAuthority(). Some of that code looks a relic of the code anti-pattern of using "hdfs:file.txt" to define the path without declaring the hostname. It's retained for compatibility.

Parameters:

conf - FS configuration

fsUri - the FS URI

path - path to check

defaultPort - default port of FS