Class CountingSource
- java.lang.Object
-
- org.apache.beam.sdk.io.CountingSource
-
public class CountingSource extends java.lang.Object
Most users should useGenerateSequence
instead.A source that produces longs. When used as a
BoundedSource
,CountingSource
starts at0
and counts up to a specified maximum. When used as anUnboundedSource
, it counts up toLong.MAX_VALUE
and then never produces more output. (In practice, this limit should never be reached.)The bounded
CountingSource
is implemented based onOffsetBasedSource
andOffsetBasedSource.OffsetBasedReader
, so it performs efficient initial splitting and it supports dynamic work rebalancing.To produce a bounded source, use
createSourceForSubrange(long, long)
. To produce an unbounded source, usecreateUnboundedFrom(long)
.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
CountingSource.CounterMark
The checkpoint for an unboundedCountingSource
is simply the last value produced.
-
Method Summary
All Methods Static Methods Concrete Methods Deprecated Methods Modifier and Type Method Description static UnboundedSource<java.lang.Long,CountingSource.CounterMark>
unbounded()
Deprecated.useGenerateSequence
insteadstatic UnboundedSource<java.lang.Long,CountingSource.CounterMark>
unboundedWithTimestampFn(SerializableFunction<java.lang.Long,org.joda.time.Instant> timestampFn)
Deprecated.useGenerateSequence
and callGenerateSequence.withTimestampFn(SerializableFunction)
insteadstatic BoundedSource<java.lang.Long>
upTo(long numElements)
Deprecated.useGenerateSequence
instead
-
-
-
Method Detail
-
upTo
@Deprecated public static BoundedSource<java.lang.Long> upTo(long numElements)
Deprecated.useGenerateSequence
insteadCreates aBoundedSource
that will produce the specified number of elements, from0
tonumElements - 1
.
-
unbounded
@Deprecated public static UnboundedSource<java.lang.Long,CountingSource.CounterMark> unbounded()
Deprecated.useGenerateSequence
insteadCreates anUnboundedSource
that will produce numbers starting from0
up toLong.MAX_VALUE
.After
Long.MAX_VALUE
, the source never produces more output. (In practice, this limit should never be reached.)Elements in the resulting
PCollection<Long>
will have timestamps corresponding to processing time at element generation, provided byInstant.now()
.
-
unboundedWithTimestampFn
@Deprecated public static UnboundedSource<java.lang.Long,CountingSource.CounterMark> unboundedWithTimestampFn(SerializableFunction<java.lang.Long,org.joda.time.Instant> timestampFn)
Deprecated.useGenerateSequence
and callGenerateSequence.withTimestampFn(SerializableFunction)
insteadCreates anUnboundedSource
that will produce numbers starting from0
up toLong.MAX_VALUE
, with element timestamps supplied by the specified function.After
Long.MAX_VALUE
, the source never produces more output. (In practice, this limit should never be reached.)Note that the timestamps produced by
timestampFn
may not decrease.
-
-