org.apache.hadoop.fs.s3a.commit.magic

Class MagicCommitTracker

java.lang.Object

org.apache.hadoop.fs.s3a.commit.PutTracker

org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTracker

@InterfaceAudience.Private
public class MagicCommitTracker
extends PutTracker

Put tracker for Magic commits.

Important

: must not directly or indirectly import a class which uses any datatype in hadoop-mapreduce.

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
MagicCommitTracker(org.apache.hadoop.fs.Path path, String bucket, String originalDestKey, String destKey, String pendingsetKey, WriteOperationHelper writer, PutTrackerStatistics trackerStatistics) Magic commit tracker.

Method Summary

Modifier and Type	Method and Description
boolean	aboutToComplete(String uploadId, List<software.amazon.awssdk.services.s3.model.CompletedPart> parts, long bytesWritten, org.apache.hadoop.fs.statistics.IOStatistics iostatistics) Complete operation: generate the final commit data, put it.
boolean	initialize() Initialize the tracker.
boolean	outputImmediatelyVisible() Flag to indicate that output is not visible after the stream is closed.
String	toString()

Methods inherited from class org.apache.hadoop.fs.s3a.commit.PutTracker

getDestKey

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

MagicCommitTracker

public MagicCommitTracker(org.apache.hadoop.fs.Path path,
                          String bucket,
                          String originalDestKey,
                          String destKey,
                          String pendingsetKey,
                          WriteOperationHelper writer,
                          PutTrackerStatistics trackerStatistics)

Magic commit tracker.

Parameters:

path - path nominally being written to

bucket - dest bucket

originalDestKey - the original key, in the magic directory.

destKey - key for the destination

pendingsetKey - key of the pendingset file

writer - writer instance to use for operations; includes audit span

trackerStatistics - tracker statistics

Method Detail

initialize

public boolean initialize()
                   throws IOException

Initialize the tracker.

Overrides:

initialize in class PutTracker

Returns:

true, indicating that the multipart commit must start.

Throws:

IOException - any IO problem.

outputImmediatelyVisible

public boolean outputImmediatelyVisible()

Flag to indicate that output is not visible after the stream is closed.

Overrides:

outputImmediatelyVisible in class PutTracker

Returns:

true

aboutToComplete

public boolean aboutToComplete(String uploadId,
                               List<software.amazon.awssdk.services.s3.model.CompletedPart> parts,
                               long bytesWritten,
                               org.apache.hadoop.fs.statistics.IOStatistics iostatistics)
                        throws IOException

Complete operation: generate the final commit data, put it.

Overrides:

aboutToComplete in class PutTracker

Parameters:

uploadId - Upload ID

parts - list of parts

bytesWritten - bytes written

iostatistics - nullable IO statistics

Returns:

false, indicating that the commit must fail.

Throws:

IOException - any IO problem.

IllegalArgumentException - bad argument

toString

public String toString()

Overrides:

toString in class PutTracker