Package ai.djl.training.dataset

Class DataIterable

java.lang.Object

ai.djl.training.dataset.DataIterable

All Implemented Interfaces:

java.lang.Iterable<Batch>, java.util.Iterator<Batch>

public class DataIterable
extends java.lang.Object
implements java.lang.Iterable<Batch>, java.util.Iterator<Batch>

DataIterable is a data loader that combines Dataset, Batchifier, Pipeline, and Sampler to provide an iterable over the given RandomAccessDataset.

We don't recommended using DataIterable directly. Instead use RandomAccessDataset combined with Trainer to iterate over the RandomAccessDataset}

Constructor Summary

Constructors

Constructor	Description
DataIterable(RandomAccessDataset dataset, NDManager manager, Sampler sampler, Batchifier dataBatchifier, Batchifier labelBatchifier, Pipeline pipeline, Pipeline targetPipeline, java.util.concurrent.ExecutorService executor, int preFetchNumber, Device device)	Creates a new instance of DataIterable with the given parameters.

Method Summary

Modifier and Type	Method	Description
boolean	hasNext()
java.util.Iterator<Batch>	iterator()
Batch	next()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Constructor Detail

DataIterable

public DataIterable(RandomAccessDataset dataset,
                    NDManager manager,
                    Sampler sampler,
                    Batchifier dataBatchifier,
                    Batchifier labelBatchifier,
                    Pipeline pipeline,
                    Pipeline targetPipeline,
                    java.util.concurrent.ExecutorService executor,
                    int preFetchNumber,
                    Device device)

Creates a new instance of DataIterable with the given parameters.

Parameters:

dataset - the dataset to iterate on

manager - the manager to create the arrays

sampler - a sampler to sample data with

dataBatchifier - a batchifier for data

labelBatchifier - a batchifier for labels

pipeline - the pipeline of transforms to apply on the data

targetPipeline - the pipeline of transforms to apply on the labels

executor - an ExecutorService

preFetchNumber - the number of samples to prefetch

device - the Device

Method Detail

iterator

public java.util.Iterator<Batch> iterator()

Specified by:

iterator in interface java.lang.Iterable<Batch>

hasNext

public boolean hasNext()

Specified by:

hasNext in interface java.util.Iterator<Batch>

next

public Batch next()

Specified by:

next in interface java.util.Iterator<Batch>