Class TFRecordIO.Read
- java.lang.Object
-
- org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<byte[]>>
-
- org.apache.beam.sdk.io.TFRecordIO.Read
-
- All Implemented Interfaces:
java.io.Serializable
,HasDisplayData
- Enclosing class:
- TFRecordIO
public abstract static class TFRecordIO.Read extends PTransform<PBegin,PCollection<byte[]>>
Implementation ofTFRecordIO.read()
.- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class org.apache.beam.sdk.transforms.PTransform
name, resourceHints
-
-
Constructor Summary
Constructors Constructor Description Read()
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description PCollection<byte[]>
expand(PBegin input)
Override this method to specify how thisPTransform
should be expanded on the givenInputT
.TFRecordIO.Read
from(java.lang.String filepattern)
Returns a transform for reading TFRecord files that reads from the file(s) with the given filename or filename pattern.TFRecordIO.Read
from(ValueProvider<java.lang.String> filepattern)
Same asfrom(filepattern)
, but accepting aValueProvider
.protected FileBasedSource<byte[]>
getSource()
void
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.TFRecordIO.Read
withCompression(Compression compression)
Returns a transform for reading TFRecord files that decompresses all input files using the specified compression type.TFRecordIO.Read
withCompressionType(TFRecordIO.CompressionType compressionType)
Deprecated.TFRecordIO.Read
withoutValidation()
Returns a transform for reading TFRecord files that has GCS path validation on pipeline creation disabled.-
Methods inherited from class org.apache.beam.sdk.transforms.PTransform
compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setResourceHints, toString, validate, validate
-
-
-
-
Method Detail
-
from
public TFRecordIO.Read from(java.lang.String filepattern)
Returns a transform for reading TFRecord files that reads from the file(s) with the given filename or filename pattern. This can be a local path (if running locally), or a Google Cloud Storage filename or filename pattern of the form"gs://<bucket>/<filepath>"
(if running locally or using remote execution). Standard Java Filesystem glob patterns ("*", "?", "[..]") are supported.
-
from
public TFRecordIO.Read from(ValueProvider<java.lang.String> filepattern)
Same asfrom(filepattern)
, but accepting aValueProvider
.
-
withoutValidation
public TFRecordIO.Read withoutValidation()
Returns a transform for reading TFRecord files that has GCS path validation on pipeline creation disabled.This can be useful in the case where the GCS input does not exist at the pipeline creation time, but is expected to be available at execution time.
-
withCompressionType
@Deprecated public TFRecordIO.Read withCompressionType(TFRecordIO.CompressionType compressionType)
Deprecated.
-
withCompression
public TFRecordIO.Read withCompression(Compression compression)
Returns a transform for reading TFRecord files that decompresses all input files using the specified compression type.If no compression type is specified, the default is
Compression.AUTO
. In this mode, the compression type of the file is determined by its extension viaCompression.detect(String)
.
-
expand
public PCollection<byte[]> expand(PBegin input)
Description copied from class:PTransform
Override this method to specify how thisPTransform
should be expanded on the givenInputT
.NOTE: This method should not be called directly. Instead apply the
PTransform
should be applied to theInputT
using theapply
method.Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
- Specified by:
expand
in classPTransform<PBegin,PCollection<byte[]>>
-
getSource
protected FileBasedSource<byte[]> getSource()
-
populateDisplayData
public void populateDisplayData(DisplayData.Builder builder)
Description copied from class:PTransform
Register display data for the given transform or component.populateDisplayData(DisplayData.Builder)
is invoked by Pipeline runners to collect display data viaDisplayData.from(HasDisplayData)
. Implementations may callsuper.populateDisplayData(builder)
in order to register display data in the current namespace, but should otherwise usesubcomponent.populateDisplayData(builder)
to use the namespace of the subcomponent.By default, does not register any display data. Implementors may override this method to provide their own display data.
- Specified by:
populateDisplayData
in interfaceHasDisplayData
- Overrides:
populateDisplayData
in classPTransform<PBegin,PCollection<byte[]>>
- Parameters:
builder
- The builder to populate with display data.- See Also:
HasDisplayData
-
-