org.platanios.tensorflow.api.ops.io.data
DynamicPaddedBatchDataset
Related Doc:
package data
Input dataset.
Batch size to use.
Shape to which the respective component of each input element should be padded prior to
batching. Any unknown dimensions (e.g., equal to -1) will be padded to the maximum size of
that dimension in each batch.
Scalar tensor structure representing the padding values to use for the respective components. Defaults to zero for numeric types and the empty string for string types.
Name for this dataset.
Batch size to use.
Creates a VARIANT scalar tensor representing this dataset.
Creates a VARIANT scalar tensor representing this dataset. This function adds ops to the current graph, that
create the dataset resource.
Creates an Iterator for enumerating the elements of this dataset.
Creates an Iterator for enumerating the elements of this dataset.
**Note:** The returned iterator will be in an uninitialized state. You must execute the InitializableIterator.initializer op before using it.
If non-empty, then the constructed reader will be shared under the the provided name across multiple sessions that share the same devices (e.g., when using a remote server).
Name for the op created in relation to the iterator.
Created iterator.
Input dataset.
Name for this dataset.
Name for this dataset.
Returns the data types corresponding to each element of this dataset, matching the structure of the elements.
Returns the data types corresponding to each element of this dataset, matching the structure of the elements.
Returns the shapes corresponding to each element of this dataset, matching the structure of the elements.
Returns the shapes corresponding to each element of this dataset, matching the structure of the elements.
Shape to which the respective component of each input element should be padded prior to batching.
Shape to which the respective component of each input element should be padded prior to
batching. Any unknown dimensions (e.g., equal to -1) will be padded to the maximum size of
that dimension in each batch.
Scalar tensor structure representing the padding values to use for the respective components.
Scalar tensor structure representing the padding values to use for the respective components. Defaults to zero for numeric types and the empty string for string types.
Creates a dataset that includes only 1 / numShards of the elements of this dataset.
Creates a dataset that includes only 1 / numShards of the elements of this dataset.
This operator is very useful when running distributed training, as it allows each worker to read a unique subset of the dataset.
When reading a single input file, you can skip elements as follows:
tf.data.TFRecordDataset(inputFile) .shard(numWorkers, workerIndex) .repeat(numEpochs) .shuffle(shuffleBufferSize) .map(parserFn, numParallelCalls)
Important caveats:
tf.data.listFiles(pattern)
.shard(numWorkers, workerIndex)
.repeat(numEpochs)
.shuffle(shuffleBufferSize)
.repeat()
.interleave(tf.data.TFRecordDataset, cycleLength = numReaders, blockLength = 1)
.map(parserFn, numParallelCalls)Number of shards to use.
Index of the shard to obtain.
Created (sharded) dataset.
Applies a transformation function to this dataset.
Applies a transformation function to this dataset.
transform() enables chaining of custom dataset transformations, which are represented as functions that take one
dataset argument and return a transformed dataset.
Dataset transformation function.
Transformed dataset.
Dataset that wraps the application of the
paddedBatchop.$OpDocDatasetPaddedBatch
Tensor type (i.e., nested structure of tensors).
Output type (i.e., nested structure of symbolic tensors).
Data type of the outputs (i.e., nested structure of TensorFlow data types).
Shape type of the outputs (i.e., nested structure of TensorFlow shapes).
Input dataset.
Batch size to use.
Shape to which the respective component of each input element should be padded prior to batching. Any unknown dimensions (e.g., equal to
-1) will be padded to the maximum size of that dimension in each batch.Scalar tensor structure representing the padding values to use for the respective components. Defaults to zero for numeric types and the empty string for string types.
Name for this dataset.