org.apache.hadoop.fs.s3a.commit

Class MagicCommitPaths

java.lang.Object

org.apache.hadoop.fs.s3a.commit.MagicCommitPaths

public final class MagicCommitPaths
extends Object

Operations on (magic) paths.

Method Summary

Modifier and Type	Method and Description
static List<String>	basePathChildren(List<String> elements) Get any child path elements under any __base path, or an empty list if there is either: no __base path element, or no child entries under it.
static boolean	containsBasePath(List<String> elements) Does the list of magic elements contain a base path marker?
static String	elementsToKey(List<String> elements) Take a list of elements and create an S3 key by joining them with "/" between each one.
static String	filename(List<String> elements) Get the filename of a path: the last element.
static List<String>	finalDestination(List<String> elements) Calculates the final destination of a file.
static boolean	isMagicPath(List<String> elements) Is a path in the magic tree?
static String	lastElement(List<String> strings) Last element of a (non-empty) list.
static int	magicElementIndex(List<String> elements) Get the index of the magic path element.
static List<String>	magicPathChildren(List<String> elements) Get the child path elements under the magic path.
static List<String>	magicPathParents(List<String> elements) Get the parent path elements of the magic path.
static org.apache.hadoop.fs.Path	magicSubdir(org.apache.hadoop.fs.Path destDir) Get the magic subdirectory of a destination directory.
static List<String>	splitPathToElements(org.apache.hadoop.fs.Path path) Take an absolute path, split it into a list of elements.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

splitPathToElements

public static List<String> splitPathToElements(org.apache.hadoop.fs.Path path)

Take an absolute path, split it into a list of elements. If empty, the path is the root path.

Parameters:

path - input path

Returns:

a possibly empty list of elements.

Throws:

IllegalArgumentException - if the path is invalid -relative, empty...

isMagicPath

public static boolean isMagicPath(List<String> elements)

Is a path in the magic tree?

Parameters:

elements - element list

Returns:

true if a path is considered magic

containsBasePath

public static boolean containsBasePath(List<String> elements)

Does the list of magic elements contain a base path marker?

Parameters:

elements - element list, already stripped out from the magic tree.

Returns:

true if a path has a base directory

magicElementIndex

public static int magicElementIndex(List<String> elements)

Get the index of the magic path element.

Parameters:

elements - full path element list

Returns:

the index.

Throws:

IllegalArgumentException - if there is no magic element

magicPathParents

public static List<String> magicPathParents(List<String> elements)

Get the parent path elements of the magic path. The list may be immutable or may be a view of the underlying list. Both the parameter list and the returned list MUST NOT be modified.

Parameters:

elements - full path element list

Returns:

the parent elements; may be empty

magicPathChildren

public static List<String> magicPathChildren(List<String> elements)

Get the child path elements under the magic path. The list may be immutable or may be a view of the underlying list. Both the parameter list and the returned list MUST NOT be modified.

Parameters:

elements - full path element list

Returns:

the child elements; may be empty

basePathChildren

public static List<String> basePathChildren(List<String> elements)

Get any child path elements under any __base path, or an empty list if there is either: no __base path element, or no child entries under it. The list may be immutable or may be a view of the underlying list. Both the parameter list and the returned list MUST NOT be modified.

Parameters:

elements - full path element list

Returns:

the child elements; may be empty

elementsToKey

public static String elementsToKey(List<String> elements)

Take a list of elements and create an S3 key by joining them with "/" between each one.

Parameters:

elements - path elements

Returns:

a path which can be used in the AWS API

filename

public static String filename(List<String> elements)

Get the filename of a path: the last element.

Parameters:

elements - element list.

Returns:

the filename; the last element.

lastElement

public static String lastElement(List<String> strings)

Last element of a (non-empty) list.

Parameters:

strings - strings in

Returns:

the last one.

magicSubdir

public static org.apache.hadoop.fs.Path magicSubdir(org.apache.hadoop.fs.Path destDir)

Get the magic subdirectory of a destination directory.

Parameters:

destDir - the destination directory

Returns:

a new path.

finalDestination

public static List<String> finalDestination(List<String> elements)

Calculates the final destination of a file. This is the parent of any __magic element, and the filename of the path. That is: all intermediate child path elements are discarded. Why so? paths under the magic path include job attempt and task attempt subdirectories, which need to be skipped. If there is a __base directory in the children, then it becomes a base for unflattened paths, that is: all its children are pulled into the final destination.

Parameters:

elements - element list.

Returns:

the path