Package org.apache.beam.sdk.transforms

Class CombineFns

java.lang.Object

org.apache.beam.sdk.transforms.CombineFns

public class CombineFns
extends java.lang.Object

Static utility methods that create combine function instances.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	CombineFns.CoCombineResult	A tuple of outputs produced by a composed combine functions.
static class	CombineFns.ComposeCombineFnBuilder	A builder class to construct a composed CombineFnBase.GlobalCombineFn.
static class	CombineFns.ComposedCombineFn<DataT>	A composed Combine.CombineFn that applies multiple CombineFns.
static class	CombineFns.ComposedCombineFnWithContext<DataT>	A composed CombineWithContext.CombineFnWithContext that applies multiple CombineFnWithContexts.

Constructor Summary

Constructors

Constructor	Description
CombineFns()

Method Summary

Modifier and Type	Method	Description
static CombineFns.ComposeCombineFnBuilder	compose()	Returns a CombineFns.ComposeCombineFnBuilder to construct a composed CombineFnBase.GlobalCombineFn.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CombineFns

public CombineFns()

Method Detail

compose

public static CombineFns.ComposeCombineFnBuilder compose()

Returns a CombineFns.ComposeCombineFnBuilder to construct a composed CombineFnBase.GlobalCombineFn.

The same TupleTag cannot be used in a composition multiple times.

Example:

 PCollection<Integer> globalLatencies = ...;

  TupleTag<Integer> maxLatencyTag = new TupleTag<Integer>();
  TupleTag<Double> meanLatencyTag = new TupleTag<Double>();

 SimpleFunction<Integer, Integer> identityFn =
     new SimpleFunction<Integer, Integer>() {
      @Override
       public Integer apply(Integer input) {
           return input;
       }};

 PCollection<CoCombineResult> maxAndMean = globalLatencies.apply(
     Combine.globally(
         CombineFns.compose()
            .with(identityFn, new MaxIntegerFn(), maxLatencyTag)
            .with(identityFn, new MeanFn<Integer>(), meanLatencyTag)));

 PCollection<T> finalResultCollection = maxAndMean
     .apply(ParDo.of(
         new DoFn<CoCombineResult, T>() {
            @ProcessElement
             public void processElement(
                  @Element CoCombineResult e, OutputReceiver<T> r) throws Exception {
                 Integer maxLatency = e.get(maxLatencyTag);
                 Double meanLatency = e.get(meanLatencyTag);
                 .... Do Something ....
                 r.output(...some T...);
              }
         }));