ai.djl.training.dataset

Class ArrayDataset

java.lang.Object

ai.djl.training.dataset.RandomAccessDataset

ai.djl.training.dataset.ArrayDataset

All Implemented Interfaces:

Dataset, java.util.RandomAccess

public class ArrayDataset
extends RandomAccessDataset

ArrayDataset is an implementation of RandomAccessDataset that consist entirely of large NDArrays. There can be multiple data and label NDArrays within the dataset. Each sample will be retrieved by indexing each NDArray along the first dimension.

The following is an example of how to use ArrayDataset:

     ArrayDataset dataset = new ArrayDataset.Builder()
                              .setData(data)
                              .optLabels(label)
                              .setSampling(20, false)
                              .build();
 

See Also:

Dataset

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ArrayDataset.Builder The Builder to construct an ArrayDataset.

Nested classes/interfaces inherited from class ai.djl.training.dataset.RandomAccessDataset

RandomAccessDataset.BaseBuilder<T extends RandomAccessDataset.BaseBuilder>

Nested classes/interfaces inherited from interface ai.djl.training.dataset.Dataset

Dataset.Usage

Field Summary

Fields

Modifier and Type	Field and Description
protected NDArray[]	data
protected NDArray[]	labels

Fields inherited from class ai.djl.training.dataset.RandomAccessDataset

batchifier, device, executor, pipeline, prefetchNumber, sampler, targetPipeline

Constructor Summary

Constructors

Constructor and Description
ArrayDataset(RandomAccessDataset.BaseBuilder<?> builder) Creates a new instance of ArrayDataset with the arguments in ArrayDataset.Builder.

Method Summary

Modifier and Type	Method and Description
Record	get(NDManager manager, long index) Gets the Record for the given index from the dataset.
long	size() Returns the size of this Dataset.

Methods inherited from class ai.djl.training.dataset.RandomAccessDataset

getData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

data

protected NDArray[] data

labels

protected NDArray[] labels

Constructor Detail

ArrayDataset

public ArrayDataset(RandomAccessDataset.BaseBuilder<?> builder)

Creates a new instance of ArrayDataset with the arguments in ArrayDataset.Builder.

Parameters:

builder - a builder with the required arguments

Method Detail

size

public long size()

Returns the size of this Dataset.

Specified by:

size in class RandomAccessDataset

Returns:

the size of this Dataset

get

public Record get(NDManager manager,
                  long index)

Gets the Record for the given index from the dataset.

Specified by:

get in class RandomAccessDataset

Parameters:

manager - the manager used to create the arrays

index - the index of the requested data item

Returns:

a Record that contains the data and label of the requested data item