public class AsyncDataSetIterator extends Object implements DataSetIterator
Modifier and Type | Class and Description |
---|---|
protected class |
AsyncDataSetIterator.AsyncPrefetchThread |
Modifier and Type | Field and Description |
---|---|
protected DataSetIterator |
backedIterator |
protected BlockingQueue<DataSet> |
buffer |
protected DataSetCallback |
callback |
protected Integer |
deviceId |
protected AtomicBoolean |
hasDepleted |
protected DataSet |
nextElement |
protected int |
prefetchSize |
protected AtomicBoolean |
shouldWork |
protected DataSet |
terminator |
protected AsyncDataSetIterator.AsyncPrefetchThread |
thread |
protected RuntimeException |
throwable |
protected boolean |
useWorkspace |
protected String |
workspaceId |
Modifier | Constructor and Description |
---|---|
protected |
AsyncDataSetIterator() |
|
AsyncDataSetIterator(DataSetIterator baseIterator)
Create an Async iterator with the default queue size of 8
|
|
AsyncDataSetIterator(DataSetIterator baseIterator,
int queueSize) |
|
AsyncDataSetIterator(DataSetIterator iterator,
int queueSize,
BlockingQueue<DataSet> queue)
Create an Async iterator with the default queue size of 8
|
|
AsyncDataSetIterator(DataSetIterator iterator,
int queueSize,
BlockingQueue<DataSet> queue,
boolean useWorkspace) |
|
AsyncDataSetIterator(DataSetIterator iterator,
int queueSize,
BlockingQueue<DataSet> queue,
boolean useWorkspace,
DataSetCallback callback) |
|
AsyncDataSetIterator(DataSetIterator iterator,
int queueSize,
BlockingQueue<DataSet> queue,
boolean useWorkspace,
DataSetCallback callback,
Integer deviceId) |
|
AsyncDataSetIterator(DataSetIterator baseIterator,
int queueSize,
boolean useWorkspace) |
|
AsyncDataSetIterator(DataSetIterator baseIterator,
int queueSize,
boolean useWorkspace,
DataSetCallback callback) |
|
AsyncDataSetIterator(DataSetIterator baseIterator,
int queueSize,
boolean useWorkspace,
Integer deviceId) |
Modifier and Type | Method and Description |
---|---|
boolean |
asyncSupported()
Does this DataSetIterator support asynchronous prefetching of multiple DataSet objects?
Most DataSetIterators do, but in some cases it may not make sense to wrap this iterator in an
iterator that does asynchronous prefetching.
|
int |
batch()
Batch size
|
protected void |
externalCall() |
List<String> |
getLabels()
Get dataset iterator record reader labels
|
DataSetPreProcessor |
getPreProcessor()
Returns preprocessors, if defined
|
boolean |
hasNext()
Returns
true if the iteration has more elements. |
int |
inputColumns()
Input columns for the dataset
|
DataSet |
next()
Returns the next element in the iteration.
|
DataSet |
next(int num)
Like the standard next method but allows a
customizable number of examples returned
|
void |
remove()
Removes from the underlying collection the last element returned
by this iterator (optional operation).
|
void |
reset()
Resets the iterator back to the beginning
|
boolean |
resetSupported()
Is resetting supported by this DataSetIterator? Many DataSetIterators do support resetting,
but some don't
|
void |
setPreProcessor(DataSetPreProcessor preProcessor)
Set a pre processor
|
void |
shutdown()
This method will terminate background thread AND will destroy attached workspace (if any)
PLEASE NOTE: After shutdown() call, this instance can't be used anymore
|
int |
totalOutcomes()
The number of labels for the dataset
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
forEachRemaining
protected DataSetIterator backedIterator
protected DataSet terminator
protected DataSet nextElement
protected BlockingQueue<DataSet> buffer
protected AsyncDataSetIterator.AsyncPrefetchThread thread
protected AtomicBoolean shouldWork
protected volatile RuntimeException throwable
protected boolean useWorkspace
protected int prefetchSize
protected String workspaceId
protected Integer deviceId
protected AtomicBoolean hasDepleted
protected DataSetCallback callback
protected AsyncDataSetIterator()
public AsyncDataSetIterator(DataSetIterator baseIterator)
baseIterator
- Underlying iterator to wrap and fetch asynchronously frompublic AsyncDataSetIterator(DataSetIterator iterator, int queueSize, BlockingQueue<DataSet> queue)
iterator
- Underlying iterator to wrap and fetch asynchronously fromqueue
- Queue size - number of iterators topublic AsyncDataSetIterator(DataSetIterator baseIterator, int queueSize)
public AsyncDataSetIterator(DataSetIterator baseIterator, int queueSize, boolean useWorkspace)
public AsyncDataSetIterator(DataSetIterator baseIterator, int queueSize, boolean useWorkspace, Integer deviceId)
public AsyncDataSetIterator(DataSetIterator baseIterator, int queueSize, boolean useWorkspace, DataSetCallback callback)
public AsyncDataSetIterator(DataSetIterator iterator, int queueSize, BlockingQueue<DataSet> queue, boolean useWorkspace)
public AsyncDataSetIterator(DataSetIterator iterator, int queueSize, BlockingQueue<DataSet> queue, boolean useWorkspace, DataSetCallback callback)
public AsyncDataSetIterator(DataSetIterator iterator, int queueSize, BlockingQueue<DataSet> queue, boolean useWorkspace, DataSetCallback callback, Integer deviceId)
public DataSet next(int num)
next
in interface DataSetIterator
num
- the number of examplespublic int inputColumns()
inputColumns
in interface DataSetIterator
public int totalOutcomes()
totalOutcomes
in interface DataSetIterator
public boolean resetSupported()
resetSupported
in interface DataSetIterator
public boolean asyncSupported()
asyncSupported
in interface DataSetIterator
protected void externalCall()
public void reset()
reset
in interface DataSetIterator
public void shutdown()
public int batch()
batch
in interface DataSetIterator
public void setPreProcessor(DataSetPreProcessor preProcessor)
setPreProcessor
in interface DataSetIterator
preProcessor
- a pre processor to setpublic DataSetPreProcessor getPreProcessor()
getPreProcessor
in interface DataSetIterator
public List<String> getLabels()
getLabels
in interface DataSetIterator
public boolean hasNext()
true
if the iteration has more elements.
(In other words, returns true
if next(int)
would
return an element rather than throwing an exception.)public DataSet next()
public void remove()
next(int)
. The behavior of an iterator
is unspecified if the underlying collection is modified while the
iteration is in progress in any way other than by calling this
method.remove
in interface Iterator<DataSet>
UnsupportedOperationException
- if the remove
operation is not supported by this iteratorIllegalStateException
- if the next
method has not
yet been called, or the remove
method has already
been called after the last call to the next
methodCopyright © 2021. All rights reserved.