Class BoundedReadFromUnboundedSource<T>
- java.lang.Object
-
- org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<T>>
-
- org.apache.beam.sdk.io.BoundedReadFromUnboundedSource<T>
-
- All Implemented Interfaces:
java.io.Serializable
,HasDisplayData
public class BoundedReadFromUnboundedSource<T> extends PTransform<PBegin,PCollection<T>>
PTransform
that reads a bounded amount of data from anUnboundedSource
, specified as one or both of a maximum number of elements or a maximum period of time to read.- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class org.apache.beam.sdk.transforms.PTransform
name, resourceHints
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description PCollection<T>
expand(PBegin input)
Override this method to specify how thisPTransform
should be expanded on the givenInputT
.java.lang.String
getKindString()
Returns the name to use by default for thisPTransform
(not including the names of any enclosingPTransform
s).void
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.BoundedReadFromUnboundedSource<T>
withMaxNumRecords(long maxNumRecords)
Returns a newBoundedReadFromUnboundedSource
that reads a bounded amount of data from the givenUnboundedSource
.BoundedReadFromUnboundedSource<T>
withMaxReadTime(org.joda.time.Duration maxReadTime)
Returns a newBoundedReadFromUnboundedSource
that reads a bounded amount of data from the givenUnboundedSource
.-
Methods inherited from class org.apache.beam.sdk.transforms.PTransform
compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getName, getResourceHints, setResourceHints, toString, validate, validate
-
-
-
-
Method Detail
-
withMaxNumRecords
public BoundedReadFromUnboundedSource<T> withMaxNumRecords(long maxNumRecords)
Returns a newBoundedReadFromUnboundedSource
that reads a bounded amount of data from the givenUnboundedSource
. The bound is specified as a number of records to read.This may take a long time to execute if the splits of this source are slow to read records.
-
withMaxReadTime
public BoundedReadFromUnboundedSource<T> withMaxReadTime(org.joda.time.Duration maxReadTime)
Returns a newBoundedReadFromUnboundedSource
that reads a bounded amount of data from the givenUnboundedSource
. The bound is specified as an amount of time to read for. Each split of the source will read for this much time.
-
expand
public PCollection<T> expand(PBegin input)
Description copied from class:PTransform
Override this method to specify how thisPTransform
should be expanded on the givenInputT
.NOTE: This method should not be called directly. Instead apply the
PTransform
should be applied to theInputT
using theapply
method.Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
- Specified by:
expand
in classPTransform<PBegin,PCollection<T>>
-
getKindString
public java.lang.String getKindString()
Description copied from class:PTransform
Returns the name to use by default for thisPTransform
(not including the names of any enclosingPTransform
s).By default, returns the base name of this
PTransform
's class.The caller is responsible for ensuring that names of applied
PTransform
s are unique, e.g., by adding a uniquifying suffix when needed.- Overrides:
getKindString
in classPTransform<PBegin,PCollection<T>>
-
populateDisplayData
public void populateDisplayData(DisplayData.Builder builder)
Description copied from class:PTransform
Register display data for the given transform or component.populateDisplayData(DisplayData.Builder)
is invoked by Pipeline runners to collect display data viaDisplayData.from(HasDisplayData)
. Implementations may callsuper.populateDisplayData(builder)
in order to register display data in the current namespace, but should otherwise usesubcomponent.populateDisplayData(builder)
to use the namespace of the subcomponent.By default, does not register any display data. Implementors may override this method to provide their own display data.
- Specified by:
populateDisplayData
in interfaceHasDisplayData
- Overrides:
populateDisplayData
in classPTransform<PBegin,PCollection<T>>
- Parameters:
builder
- The builder to populate with display data.- See Also:
HasDisplayData
-
-