Package ai.djl.basicdataset.utils
Class FixedBucketSampler
- java.lang.Object
-
- ai.djl.basicdataset.utils.FixedBucketSampler
-
- All Implemented Interfaces:
ai.djl.training.dataset.Sampler
public class FixedBucketSampler extends java.lang.Object implements ai.djl.training.dataset.Sampler
FixedBucketSampler
is aSampler
to be used withTextDataset
, andPaddingStackBatchifier
. It groups text data of same length, and samples them together so that the amount of padding required is minimised. It also makes sure that the sampling is random across epochs.
-
-
Constructor Summary
Constructors Constructor Description FixedBucketSampler(int batchSize)
Constructs a new instance ofFixedBucketSampler
with the given number of buckets, and the given batch size.FixedBucketSampler(int batchSize, int numBuckets)
Constructs a new instance ofFixedBucketSampler
with the given number of buckets, and the given batch size.FixedBucketSampler(int batchSize, int numBuckets, boolean shuffle)
Constructs a new instance ofFixedBucketSampler
with the given number of buckets, and the given batch size.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description int
getBatchSize()
java.util.Iterator<java.util.List<java.lang.Long>>
sample(ai.djl.training.dataset.RandomAccessDataset dataset)
-
-
-
Constructor Detail
-
FixedBucketSampler
public FixedBucketSampler(int batchSize, int numBuckets, boolean shuffle)
Constructs a new instance ofFixedBucketSampler
with the given number of buckets, and the given batch size.- Parameters:
batchSize
- the batch sizenumBuckets
- the number of bucketsshuffle
- whether to shuffle data randomly while sampling
-
FixedBucketSampler
public FixedBucketSampler(int batchSize, int numBuckets)
Constructs a new instance ofFixedBucketSampler
with the given number of buckets, and the given batch size.- Parameters:
batchSize
- the batch sizenumBuckets
- the number of buckets
-
FixedBucketSampler
public FixedBucketSampler(int batchSize)
Constructs a new instance ofFixedBucketSampler
with the given number of buckets, and the given batch size.- Parameters:
batchSize
- the batch size
-
-
Method Detail
-
sample
public java.util.Iterator<java.util.List<java.lang.Long>> sample(ai.djl.training.dataset.RandomAccessDataset dataset)
- Specified by:
sample
in interfaceai.djl.training.dataset.Sampler
-
getBatchSize
public int getBatchSize()
- Specified by:
getBatchSize
in interfaceai.djl.training.dataset.Sampler
-
-