@InterfaceAudience.Private @InterfaceStability.Unstable

Package org.apache.hadoop.fs.s3a.commit

Support for committing the output of analytics jobs directly to S3.

See: Description

Interface Summary

Interface	Description
Tasks.FailureTask<I,E extends Exception>	Callback invoked on a failure.
Tasks.Submitter	Interface to whatever lets us submit tasks.
Tasks.Task<I,E extends Exception>	Callback invoked to process an item.

Class Summary

Class	Description
AbstractS3ACommitter	Abstract base class for S3A committers; allows for any commonality between different architectures.
AbstractS3ACommitter.ActiveCommit	State of the active commit operation.
AbstractS3ACommitterFactory	Dynamically create the output committer based on subclass type and settings.
CommitConstants	Constants for working with committers.
CommitOperations	The implementation of the various actions a committer needs.
CommitOperations.MaybeIOE	A holder for a possible IOException; the call CommitOperations.MaybeIOE.maybeRethrow() will throw any exception passed into the constructor, and be a no-op if none was.
CommitUtils	Static utility methods related to S3A commitment processing, both staging and magic.
CommitUtilsWithMR	These are commit utility methods which import classes from hadoop-mapreduce, and so only work when that module is on the classpath.
InternalCommitterConstants	These are internal constants not intended for public use.
MagicCommitIntegration	Adds the code needed for S3A to support magic committers.
MagicCommitPaths	Operations on (magic) paths.
PutTracker	Multipart put tracker.
S3ACommitterFactory	The S3A committer factory which chooses the committer based on the specific option chosen in a per-bucket basis from the property CommitConstants.FS_S3A_COMMITTER_NAME.
Tasks	Utility class for parallel execution, takes closures for the various actions.
Tasks.Builder<I>	Builder for task execution.

Enum Summary

Enum	Description
AbstractS3ACommitter.JobUUIDSource	Enumeration of Job UUID source.

Exception Summary

Exception	Description
PathCommitException	Path exception to use for various commit issues.
ValidationFailure	Exception raised on validation failures; kept as an IOException for consistency with other failures.

Package org.apache.hadoop.fs.s3a.commit Description

Support for committing the output of analytics jobs directly to S3.