Package ai.djl.training.dataset
Interface Dataset
-
- All Known Implementing Classes:
ArrayDataset
,RandomAccessDataset
public interface Dataset
An interface to represent a set of sample data/label pairs to train a model.- See Also:
- The guide on datasets
-
-
Nested Class Summary
Nested Classes Modifier and Type Interface Description static class
Dataset.Usage
An enum that indicates the mode - training, test or validation.
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description java.lang.Iterable<Batch>
getData(NDManager manager)
Fetches an iterator that can iterate through theDataset
.default java.lang.Iterable<Batch>
getData(NDManager manager, java.util.concurrent.ExecutorService executorService)
Fetches an iterator that can iterate through theDataset
with multiple threads.default void
prepare()
Prepares the dataset for use.void
prepare(ai.djl.util.Progress progress)
Prepares the dataset for use with tracked progress.
-
-
-
Method Detail
-
getData
java.lang.Iterable<Batch> getData(NDManager manager) throws java.io.IOException, TranslateException
Fetches an iterator that can iterate through theDataset
.- Parameters:
manager
- the dataset to iterate through- Returns:
- an
Iterable
ofBatch
that contains batches of data from the dataset - Throws:
java.io.IOException
- for various exceptions depending on the datasetTranslateException
- if there is an error while processing input
-
getData
default java.lang.Iterable<Batch> getData(NDManager manager, java.util.concurrent.ExecutorService executorService) throws java.io.IOException, TranslateException
Fetches an iterator that can iterate through theDataset
with multiple threads.- Parameters:
manager
- the dataset to iterate throughexecutorService
- the executorService to use for multi-threading- Returns:
- an
Iterable
ofBatch
that contains batches of data from the dataset - Throws:
java.io.IOException
- for various exceptions depending on the datasetTranslateException
- if there is an error while processing input
-
prepare
default void prepare() throws java.io.IOException, TranslateException
Prepares the dataset for use.- Throws:
java.io.IOException
- for various exceptions depending on the datasetTranslateException
- if there is an error while processing input
-
prepare
void prepare(ai.djl.util.Progress progress) throws java.io.IOException, TranslateException
Prepares the dataset for use with tracked progress.- Parameters:
progress
- the progress tracker- Throws:
java.io.IOException
- for various exceptions depending on the datasetTranslateException
- if there is an error while processing input
-
-