Package org.apache.beam.sdk.io
Class TFRecordIO
- java.lang.Object
-
- org.apache.beam.sdk.io.TFRecordIO
-
public class TFRecordIO extends java.lang.Object
PTransform
s for reading and writing TensorFlow TFRecord files.For reading files, use
read()
.For simple cases of writing files, use
write()
. For more complex cases (such as ability to write windowed data or writing to multiple destinations) usesink()
in combination withFileIO.write()
orFileIO.writeDynamic()
.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
TFRecordIO.CompressionType
Deprecated.UseCompression
.static class
TFRecordIO.Read
Implementation ofread()
.static class
TFRecordIO.ReadFiles
Implementation ofreadFiles()
.static class
TFRecordIO.Sink
static class
TFRecordIO.Write
Implementation ofwrite()
.
-
Field Summary
Fields Modifier and Type Field Description static Coder<byte[]>
DEFAULT_BYTE_ARRAY_CODER
The default coder, which returns each record of the input file as a byte array.
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static TFRecordIO.Read
read()
APTransform
that reads from a TFRecord file (or multiple TFRecord files matching a pattern) and returns aPCollection
containing the decoding of each of the records of the TFRecord file(s) as a byte array.static TFRecordIO.ReadFiles
readFiles()
Likeread()
, but reads each file in aPCollection
ofFileIO.ReadableFile
, returned byFileIO.readMatches()
.static TFRecordIO.Sink
sink()
static TFRecordIO.Write
write()
APTransform
that writes aPCollection
to TFRecord file (or multiple TFRecord files matching a sharding pattern), with each element of the input collection encoded into its own record.
-
-
-
Field Detail
-
DEFAULT_BYTE_ARRAY_CODER
public static final Coder<byte[]> DEFAULT_BYTE_ARRAY_CODER
The default coder, which returns each record of the input file as a byte array.
-
-
Method Detail
-
read
public static TFRecordIO.Read read()
APTransform
that reads from a TFRecord file (or multiple TFRecord files matching a pattern) and returns aPCollection
containing the decoding of each of the records of the TFRecord file(s) as a byte array.
-
readFiles
public static TFRecordIO.ReadFiles readFiles()
Likeread()
, but reads each file in aPCollection
ofFileIO.ReadableFile
, returned byFileIO.readMatches()
.
-
write
public static TFRecordIO.Write write()
APTransform
that writes aPCollection
to TFRecord file (or multiple TFRecord files matching a sharding pattern), with each element of the input collection encoded into its own record.
-
sink
public static TFRecordIO.Sink sink()
-
-