public static class CountingInput.UnboundedCountingInput extends PTransform<PBegin,PCollection<Long>>
PTransform
that will produce numbers starting from 0
up to
Long.MAX_VALUE
.
After Long.MAX_VALUE
, the transform never produces more output. (In practice, this
limit should never be reached.)
Elements in the resulting PCollection<Long>
will by default have
timestamps corresponding to processing time at element generation, provided by
Instant.now()
. Use the transform returned by
withTimestampFn(SerializableFunction)
to control the output
timestamps.
name
Modifier and Type | Method and Description |
---|---|
PCollection<Long> |
apply(PBegin begin)
Applies this
PTransform on the given InputT , and returns its
Output . |
void |
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.
|
CountingInput.UnboundedCountingInput |
withMaxNumRecords(long maxRecords)
Returns an
CountingInput.UnboundedCountingInput like this one, but that will read at most the
specified number of elements. |
CountingInput.UnboundedCountingInput |
withMaxReadTime(Duration readTime)
Returns an
CountingInput.UnboundedCountingInput like this one, but that will read for at most the
specified amount of time. |
CountingInput.UnboundedCountingInput |
withRate(long numElements,
Duration periodLength)
Returns an
CountingInput.UnboundedCountingInput like this one, but with output production limited
to an aggregate rate of no more than the number of elements per the period length. |
CountingInput.UnboundedCountingInput |
withTimestampFn(SerializableFunction<Long,Instant> timestampFn)
Returns an
CountingInput.UnboundedCountingInput like this one, but where output elements have the
timestamp specified by the timestampFn. |
getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, toString, validate
public CountingInput.UnboundedCountingInput withTimestampFn(SerializableFunction<Long,Instant> timestampFn)
CountingInput.UnboundedCountingInput
like this one, but where output elements have the
timestamp specified by the timestampFn.
Note that the timestamps produced by timestampFn
may not decrease.
public CountingInput.UnboundedCountingInput withMaxNumRecords(long maxRecords)
CountingInput.UnboundedCountingInput
like this one, but that will read at most the
specified number of elements.
A bounded amount of elements will be produced by the result transform, and the result
PCollection
will be bounded
.
public CountingInput.UnboundedCountingInput withRate(long numElements, Duration periodLength)
CountingInput.UnboundedCountingInput
like this one, but with output production limited
to an aggregate rate of no more than the number of elements per the period length.
Note that when there are multiple splits, each split outputs independently. This may lead to elements not being produced evenly across time, though the aggregate rate will still approach the specified rate.
A duration of Duration.ZERO
will produce output as fast as possible.
public CountingInput.UnboundedCountingInput withMaxReadTime(Duration readTime)
CountingInput.UnboundedCountingInput
like this one, but that will read for at most the
specified amount of time.
A bounded amount of elements will be produced by the result transform, and the result
PCollection
will be bounded
.
public PCollection<Long> apply(PBegin begin)
PTransform
PTransform
on the given InputT
, and returns its
Output
.
Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
The default implementation throws an exception. A derived class must
either implement apply, or else each runner must supply a custom
implementation via
PipelineRunner.apply(com.google.cloud.dataflow.sdk.transforms.PTransform<InputT, OutputT>, InputT)
.
apply
in class PTransform<PBegin,PCollection<Long>>
public void populateDisplayData(DisplayData.Builder builder)
PTransform
populateDisplayData(DisplayData.Builder)
is invoked by Pipeline runners to collect
display data via DisplayData.from(HasDisplayData)
. Implementations may call
super.populateDisplayData(builder)
in order to register display data in the current
namespace, but should otherwise use subcomponent.populateDisplayData(builder)
to use
the namespace of the subcomponent.
By default, does not register any display data. Implementors may override this method to provide their own display data.
populateDisplayData
in interface HasDisplayData
populateDisplayData
in class PTransform<PBegin,PCollection<Long>>
builder
- The builder to populate with display data.HasDisplayData