Package software.amazon.awssdk.services.sagemaker.model

Class ShuffleConfig

java.lang.Object

software.amazon.awssdk.services.sagemaker.model.ShuffleConfig

All Implemented Interfaces:

Serializable, SdkPojo, ToCopyableBuilder<ShuffleConfig.Builder,ShuffleConfig>

@Generated("software.amazon.awssdk:codegen")
public final class ShuffleConfig
extends Object
implements SdkPojo, Serializable, ToCopyableBuilder<ShuffleConfig.Builder,ShuffleConfig>

A configuration for a shuffle option for input data in a channel. If you use S3Prefix for S3DataType, the results of the S3 key prefix matches are shuffled. If you use ManifestFile, the order of the S3 object references in the ManifestFile is shuffled. If you use AugmentedManifestFile, the order of the JSON lines in the AugmentedManifestFile is shuffled. The shuffling order is determined using the Seed value.

For Pipe input mode, when ShuffleConfig is specified shuffling is done at the start of every epoch. With large datasets, this ensures that the order of the training data is different for each epoch, and it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType of ShardedByS3Key, the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	ShuffleConfig.Builder

Method Summary

Modifier and Type	Method	Description
static ShuffleConfig.Builder	builder()
boolean	equals(Object obj)
boolean	equalsBySdkFields(Object obj)
<T> Optional<T>	getValueForField(String fieldName, Class<T> clazz)
int	hashCode()
Map<String,SdkField<?>>	sdkFieldNameToField()
List<SdkField<?>>	sdkFields()
Long	seed()	Determines the shuffling order in ShuffleConfig value.
static Class<? extends ShuffleConfig.Builder>	serializableBuilderClass()
ShuffleConfig.Builder	toBuilder()
String	toString()	Returns a string representation of this object.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface software.amazon.awssdk.utils.builder.ToCopyableBuilder

copy

Method Detail

seed

public final Long seed()

Determines the shuffling order in ShuffleConfig value.

Returns:

Determines the shuffling order in ShuffleConfig value.

toBuilder

public ShuffleConfig.Builder toBuilder()

Specified by:

toBuilder in interface ToCopyableBuilder<ShuffleConfig.Builder,ShuffleConfig>

builder

public static ShuffleConfig.Builder builder()

serializableBuilderClass

public static Class<? extends ShuffleConfig.Builder> serializableBuilderClass()

hashCode

public final int hashCode()

Overrides:

hashCode in class Object

equals

public final boolean equals(Object obj)

Overrides:

equals in class Object

equalsBySdkFields

public final boolean equalsBySdkFields(Object obj)

Specified by:

equalsBySdkFields in interface SdkPojo

toString

public final String toString()

Returns a string representation of this object. This is useful for testing and debugging. Sensitive data will be redacted from this string using a placeholder value.

Overrides:

toString in class Object

getValueForField

public final <T> Optional<T> getValueForField(String fieldName,
                                              Class<T> clazz)

sdkFields

public final List<SdkField<?>> sdkFields()

Specified by:

sdkFields in interface SdkPojo

sdkFieldNameToField

public final Map<String,SdkField<?>> sdkFieldNameToField()

Specified by:

sdkFieldNameToField in interface SdkPojo