Package org.apache.beam.sdk.transforms
Class Latest
- java.lang.Object
-
- org.apache.beam.sdk.transforms.Latest
-
public class Latest extends java.lang.Object
PTransform
andCombine.CombineFn
for computing the latest element in aPCollection
.Example: compute the latest value for each session:
PCollection<Long> input = ...; PCollection<Long> sessioned = input .apply(Window.<Long>into(Sessions.withGapDuration(Duration.standardMinutes(5))); PCollection<Long> latestValues = sessioned.apply(Latest.<Long>globally());
combineFn()
can also be used manually, in combination with state and with theCombine
transform.For elements with the same timestamp, the element chosen for output is arbitrary.
-
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static <T> Combine.CombineFn<TimestampedValue<T>,?,T>
combineFn()
Returns aCombine.CombineFn
that selects the latest element among its inputs.static <T> PTransform<PCollection<T>,PCollection<T>>
globally()
Returns aPTransform
that takes as input aPCollection<T>
and returns aPCollection<T>
whose contents is the latest element according to its event time, or null if there are no elements.static <K,V>
PTransform<PCollection<KV<K,V>>,PCollection<KV<K,V>>>perKey()
Returns aPTransform
that takes as input aPCollection<KV<K, V>>
and returns aPCollection<KV<K, V>>
whose contents is the latest element per-key according to its event time.
-
-
-
Method Detail
-
combineFn
public static <T> Combine.CombineFn<TimestampedValue<T>,?,T> combineFn()
Returns aCombine.CombineFn
that selects the latest element among its inputs.
-
globally
public static <T> PTransform<PCollection<T>,PCollection<T>> globally()
Returns aPTransform
that takes as input aPCollection<T>
and returns aPCollection<T>
whose contents is the latest element according to its event time, or null if there are no elements.- Type Parameters:
T
- The type of the elements being combined.
-
perKey
public static <K,V> PTransform<PCollection<KV<K,V>>,PCollection<KV<K,V>>> perKey()
Returns aPTransform
that takes as input aPCollection<KV<K, V>>
and returns aPCollection<KV<K, V>>
whose contents is the latest element per-key according to its event time.- Type Parameters:
K
- The key type of the elements being combined.V
- The value type of the elements being combined.
-
-