org.apache.hadoop.fs.s3a.impl

Class SDKStreamDrainer

java.lang.Object

org.apache.hadoop.fs.s3a.impl.SDKStreamDrainer

All Implemented Interfaces:

org.apache.hadoop.util.functional.CallableRaisingIOE<Boolean>

public class SDKStreamDrainer
extends Object
implements org.apache.hadoop.util.functional.CallableRaisingIOE<Boolean>

Drains/aborts s3 or other AWS SDK streams. It is callable so can be passed directly to a submitter for async invocation. A request object may be passed in; it will be implicitly cached until this object is GCd. This is because in some versions of the AWS SDK, the S3Object has a finalize() method which releases the http connection, even when the stream is still open. See HADOOP-17338 for details.

Constructor Summary

Constructors

Constructor and Description
SDKStreamDrainer(String uri, Closeable requestObject, com.amazonaws.internal.SdkFilterInputStream sdkStream, boolean shouldAbort, int remaining, S3AInputStreamStatistics streamStatistics, String reason) Prepare to drain the stream.

Method Summary

Modifier and Type	Method and Description
boolean	aborted()
Boolean	apply() drain the stream.
boolean	executed()
int	getDrained()
String	getReason()
int	getRemaining()
Object	getRequestObject()
com.amazonaws.internal.SdkFilterInputStream	getSdkStream()
S3AInputStreamStatistics	getStreamStatistics()
Exception	getThrown()
String	getUri()
boolean	shouldAbort()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

SDKStreamDrainer

public SDKStreamDrainer(String uri,
                        @Nullable
                        Closeable requestObject,
                        com.amazonaws.internal.SdkFilterInputStream sdkStream,
                        boolean shouldAbort,
                        int remaining,
                        S3AInputStreamStatistics streamStatistics,
                        String reason)

Prepare to drain the stream.

Parameters:

uri - URI for messages

requestObject - http request object; needed to avoid GC issues.

sdkStream - stream to close.

shouldAbort - force an abort; used if explicitly requested.

streamStatistics - stats to update

reason - reason for stream being closed; used in messages

remaining - remaining bytes

Method Detail

apply

public Boolean apply()

drain the stream. This method is intended to be used directly or asynchronously, and measures the duration of the operation in the stream statistics.

Specified by:

apply in interface org.apache.hadoop.util.functional.CallableRaisingIOE<Boolean>

Returns:

was the stream aborted?

getUri

public String getUri()

getRequestObject

public Object getRequestObject()

getSdkStream

public com.amazonaws.internal.SdkFilterInputStream getSdkStream()

shouldAbort

public boolean shouldAbort()

getRemaining

public int getRemaining()

getStreamStatistics

public S3AInputStreamStatistics getStreamStatistics()

getReason

public String getReason()

executed

public boolean executed()

getThrown

public Exception getThrown()

getDrained

public int getDrained()

aborted

public boolean aborted()

toString

public String toString()

Overrides:

toString in class Object