org.apache.hadoop.fs.s3a.impl

Class ChangeDetectionPolicy

java.lang.Object

org.apache.hadoop.fs.s3a.impl.ChangeDetectionPolicy

@InterfaceAudience.Private
 @InterfaceStability.Unstable
public abstract class ChangeDetectionPolicy
extends Object

Object change detection policy. Determines which attribute is used to detect change and what to do when change is detected.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ChangeDetectionPolicy.Mode What to do when change is detected.
static class	ChangeDetectionPolicy.Source The S3 object attribute used to detect change.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CHANGE_DETECTED

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ChangeDetectionPolicy(ChangeDetectionPolicy.Mode mode, boolean requireVersion)

Method Summary

Modifier and Type	Method and Description
abstract void	applyRevisionConstraint(com.amazonaws.services.s3.model.GetObjectRequest request, String revisionId) Applies the given revisionId as a server-side qualification on the GetObjectRequest.
static ChangeDetectionPolicy	createPolicy(ChangeDetectionPolicy.Mode mode, ChangeDetectionPolicy.Source source, boolean requireVersion) Create a policy.
LogExactlyOnce	getLogNoVersionSupport()
ChangeDetectionPolicy.Mode	getMode()
static ChangeDetectionPolicy	getPolicy(org.apache.hadoop.conf.Configuration configuration) Reads the change detection policy from Configuration.
abstract String	getRevisionId(com.amazonaws.services.s3.model.ObjectMetadata objectMetadata, String uri) Pulls the attribute this policy uses to detect change out of the S3 object metadata.
abstract ChangeDetectionPolicy.Source	getSource()
boolean	isRequireVersion()
org.apache.commons.lang3.tuple.ImmutablePair<Boolean,RemoteFileChangedException>	onChangeDetected(String revisionId, String newRevisionId, String uri, long position, String operation, long timesAlreadyDetected) Takes appropriate action based on mode when a change has been detected.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CHANGE_DETECTED

public static final String CHANGE_DETECTED

See Also:

Constant Field Values

Constructor Detail

ChangeDetectionPolicy

protected ChangeDetectionPolicy(ChangeDetectionPolicy.Mode mode,
                                boolean requireVersion)

Method Detail

getMode

public ChangeDetectionPolicy.Mode getMode()

getSource

public abstract ChangeDetectionPolicy.Source getSource()

isRequireVersion

public boolean isRequireVersion()

getLogNoVersionSupport

public LogExactlyOnce getLogNoVersionSupport()

getPolicy

public static ChangeDetectionPolicy getPolicy(org.apache.hadoop.conf.Configuration configuration)

Reads the change detection policy from Configuration.

Parameters:

configuration - the configuration

Returns:

the policy

createPolicy

public static ChangeDetectionPolicy createPolicy(ChangeDetectionPolicy.Mode mode,
                                                 ChangeDetectionPolicy.Source source,
                                                 boolean requireVersion)

Create a policy.

Parameters:

mode - mode pf checks

source - source of change

requireVersion - throw exception when no version available?

Returns:

the policy

getRevisionId

public abstract String getRevisionId(com.amazonaws.services.s3.model.ObjectMetadata objectMetadata,
                                     String uri)

Pulls the attribute this policy uses to detect change out of the S3 object metadata. The policy generically refers to this attribute as revisionId.

Parameters:

objectMetadata - the s3 object metadata

uri - the URI of the object

Returns:

the revisionId string as interpreted by this policy, or potentially null if the attribute is unavailable (such as when the policy says to use versionId but object versioning is not enabled for the bucket).

applyRevisionConstraint

public abstract void applyRevisionConstraint(com.amazonaws.services.s3.model.GetObjectRequest request,
                                             String revisionId)

Applies the given revisionId as a server-side qualification on the GetObjectRequest.

Parameters:

request - the request

revisionId - the revision id

onChangeDetected

public org.apache.commons.lang3.tuple.ImmutablePair<Boolean,RemoteFileChangedException> onChangeDetected(String revisionId,
                                                                                                         String newRevisionId,
                                                                                                         String uri,
                                                                                                         long position,
                                                                                                         String operation,
                                                                                                         long timesAlreadyDetected)

Takes appropriate action based on mode when a change has been detected.

Parameters:

revisionId - the expected revision id

newRevisionId - the detected revision id

uri - the URI of the object being accessed

position - the position being read in the object

operation - the operation being performed on the object (e.g. open or re-open) that triggered the change detection

timesAlreadyDetected - number of times a change has already been detected on the current stream

Returns:

a pair of: was a change detected, and any exception to throw. If the change was detected, this updates a counter in the stream statistics; If an exception was returned it is thrown after the counter update.