public class UnboundedReadFromBoundedSource<T> extends PTransform<PBegin,PCollection<T>>
PTransform that converts a BoundedSource as an UnboundedSource.
BoundedSource is read directly without calling BoundedSource.split(long, org.apache.beam.sdk.options.PipelineOptions), and
element timestamps are propagated. While any elements remain, the watermark is the beginning of
time BoundedWindow.TIMESTAMP_MIN_VALUE, and after all elements have been produced the
watermark goes to the end of time BoundedWindow.TIMESTAMP_MAX_VALUE.
Checkpoints are created by calling BoundedSource.BoundedReader.splitAtFraction(double) on inner BoundedSource. Sources that cannot be split are read entirely into memory, so this transform
does not work well with large, unsplittable sources.
This transform is intended to be used by a runner during pipeline translation to convert a Read.Bounded into a Read.Unbounded.
| Modifier and Type | Class and Description |
|---|---|
static class |
UnboundedReadFromBoundedSource.BoundedToUnboundedSourceAdapter<T>
A
BoundedSource to UnboundedSource adapter. |
annotations, displayData, name, resourceHints| Constructor and Description |
|---|
UnboundedReadFromBoundedSource(BoundedSource<T> source)
Constructs a
PTransform that performs an unbounded read from a BoundedSource. |
| Modifier and Type | Method and Description |
|---|---|
PCollection<T> |
expand(PBegin input)
Override this method to specify how this
PTransform should be expanded on the given
InputT. |
protected Coder<T> |
getDefaultOutputCoder()
Returns the default
Coder to use for the output of this single-output PTransform. |
java.lang.String |
getKindString()
Returns the name to use by default for this
PTransform (not including the names of any
enclosing PTransforms). |
void |
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.
|
addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getName, getResourceHints, setDisplayData, setResourceHints, toString, validate, validatepublic UnboundedReadFromBoundedSource(BoundedSource<T> source)
PTransform that performs an unbounded read from a BoundedSource.public PCollection<T> expand(PBegin input)
PTransformPTransform should be expanded on the given
InputT.
NOTE: This method should not be called directly. Instead apply the PTransform should
be applied to the InputT using the apply method.
Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
expand in class PTransform<PBegin,PCollection<T>>protected Coder<T> getDefaultOutputCoder()
PTransformCoder to use for the output of this single-output PTransform.
By default, always throws
getDefaultOutputCoder in class PTransform<PBegin,PCollection<T>>public java.lang.String getKindString()
PTransformPTransform (not including the names of any
enclosing PTransforms).
By default, returns the base name of this PTransform's class.
The caller is responsible for ensuring that names of applied PTransforms are unique,
e.g., by adding a uniquifying suffix when needed.
getKindString in class PTransform<PBegin,PCollection<T>>public void populateDisplayData(DisplayData.Builder builder)
PTransformpopulateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect
display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace,
but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace
of the subcomponent.
By default, does not register any display data. Implementors may override this method to provide their own display data.
populateDisplayData in interface HasDisplayDatapopulateDisplayData in class PTransform<PBegin,PCollection<T>>builder - The builder to populate with display data.HasDisplayData