org.apache.hadoop.fs.s3a.commit

Class InternalCommitterConstants

java.lang.Object

org.apache.hadoop.fs.s3a.commit.InternalCommitterConstants

@InterfaceAudience.Private
 @InterfaceStability.Unstable
public final class InternalCommitterConstants
extends Object

These are internal constants not intended for public use.

Field Summary

Fields

Modifier and Type	Field and Description
static String	COMMITTER_NAME_STAGING This is the staging committer base class; only used for testing.
static String	DIRECTORY_COMMITTER_FACTORY Directory committer factory: "org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory".
static String	E_BAD_PATH Error message for bad path: "Path does not represent a magic-commit path".
static String	E_DEST_EXISTS Error text when the destination path exists and the committer must abort the job/task "Destination path exists and committer conflict resolution mode is \"fail\"".
static String	E_NO_MAGIC_PATH_ELEMENT Error message for a path without a magic element in the list: "No __magic element in path".
static String	E_NO_SPARK_UUID Incoming Job/task configuration didn't contain any option SPARK_WRITE_UUID.
static String	E_NORMAL_FS Error message if filesystem isn't magic: "Filesystem does not have support for \'magic\' committer enabled in configuration option fs.s3a.committer.magic.enabled".
static String	E_WRONG_FS Error message if the dest FS isn't S3A: "Output path is not on an S3A Filesystem".
static String	FS_S3A_COMMITTER_UUID A unique identifier to use for this work: "fs.s3a.committer.uuid".
static String	FS_S3A_COMMITTER_UUID_SOURCE Where did the UUID come from? "fs.s3a.committer.uuid.source".
static String	JAVA_IO_TMPDIR Java temp dir: "java.io.tmpdir".
static String	MAGIC_COMMITTER_FACTORY Magic committer factory: "org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory".
static String	PARTITION_COMMITTER_FACTORY Partitioned committer factory: "org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory".
static String	SPARK_WRITE_UUID The UUID for jobs: "spark.sql.sources.writeJobUUID".
static String	STAGING_COMMITTER_FACTORY Directory committer factory: "org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory".

Method Summary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

COMMITTER_NAME_STAGING

public static final String COMMITTER_NAME_STAGING

This is the staging committer base class; only used for testing.

See Also:

Constant Field Values

FS_S3A_COMMITTER_UUID

public static final String FS_S3A_COMMITTER_UUID

A unique identifier to use for this work: "fs.s3a.committer.uuid".

See Also:

Constant Field Values

FS_S3A_COMMITTER_UUID_SOURCE

public static final String FS_S3A_COMMITTER_UUID_SOURCE

Where did the UUID come from? "fs.s3a.committer.uuid.source".

See Also:

Constant Field Values

STAGING_COMMITTER_FACTORY

public static final String STAGING_COMMITTER_FACTORY

Directory committer factory: "org.apache.hadoop.fs.s3a.commit.staging.StagingCommitterFactory".

See Also:

Constant Field Values

DIRECTORY_COMMITTER_FACTORY

public static final String DIRECTORY_COMMITTER_FACTORY

Directory committer factory: "org.apache.hadoop.fs.s3a.commit.staging.DirectoryStagingCommitterFactory".

See Also:

Constant Field Values

PARTITION_COMMITTER_FACTORY

public static final String PARTITION_COMMITTER_FACTORY

Partitioned committer factory: "org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory".

See Also:

Constant Field Values

MAGIC_COMMITTER_FACTORY

public static final String MAGIC_COMMITTER_FACTORY

Magic committer factory: "org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitterFactory".

See Also:

Constant Field Values

E_DEST_EXISTS

public static final String E_DEST_EXISTS

Error text when the destination path exists and the committer must abort the job/task "Destination path exists and committer conflict resolution mode is \"fail\"".

See Also:

Constant Field Values

E_BAD_PATH

public static final String E_BAD_PATH

Error message for bad path: "Path does not represent a magic-commit path".

See Also:

Constant Field Values

E_NORMAL_FS

public static final String E_NORMAL_FS

Error message if filesystem isn't magic: "Filesystem does not have support for \'magic\' committer enabled in configuration option fs.s3a.committer.magic.enabled".

See Also:

Constant Field Values

E_WRONG_FS

public static final String E_WRONG_FS

Error message if the dest FS isn't S3A: "Output path is not on an S3A Filesystem".

See Also:

Constant Field Values

E_NO_MAGIC_PATH_ELEMENT

public static final String E_NO_MAGIC_PATH_ELEMENT

Error message for a path without a magic element in the list: "No __magic element in path".

See Also:

Constant Field Values

SPARK_WRITE_UUID

public static final String SPARK_WRITE_UUID

The UUID for jobs: "spark.sql.sources.writeJobUUID". This was historically created in Spark 1.x's SQL queries, but "went away".

See Also:

Constant Field Values

JAVA_IO_TMPDIR

public static final String JAVA_IO_TMPDIR

Java temp dir: "java.io.tmpdir".

See Also:

Constant Field Values

E_NO_SPARK_UUID

public static final String E_NO_SPARK_UUID

Incoming Job/task configuration didn't contain any option SPARK_WRITE_UUID.

See Also:

Constant Field Values