org.apache.beam.sdk.util.construction.graph

Class GreedyStageFuser

java.lang.Object

org.apache.beam.sdk.util.construction.graph.GreedyStageFuser

public class GreedyStageFuser
extends java.lang.Object

A factory class which produces an ExecutableStage by attempting to fuse all available PCollections when it is constructed.

A PipelineNode.PCollectionNode is fused into a stage if all of its consumers can be fused into the stage. A consumer can be fused into a stage if it is executed within the environment of that ExecutableStage, and receives only per-element inputs. To simplify integration for runners, this fuser specifically does not fuse PTransforms which consume side inputs or have user state, always making them the root of ExecutableStage.

A PipelineNode.PCollectionNode with consumers that execute in an environment other than a stage is materialized, and its consumers execute in independent stages.

Method Summary

Modifier and Type	Method and Description
static ExecutableStage	forGrpcPortRead(QueryablePipeline pipeline, PipelineNode.PCollectionNode inputPCollection, java.util.Set<PipelineNode.PTransformNode> initialNodes) Returns an ExecutableStage where the initial PTransform is a Remote gRPC Port Read, reading elements from the materialized PCollection.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

forGrpcPortRead

public static ExecutableStage forGrpcPortRead(QueryablePipeline pipeline,
                                              PipelineNode.PCollectionNode inputPCollection,
                                              java.util.Set<PipelineNode.PTransformNode> initialNodes)

Returns an ExecutableStage where the initial PTransform is a Remote gRPC Port Read, reading elements from the materialized PCollection.

Parameters:

initialNodes - the initial set of sibling transforms to fuse into this node. All of the transforms must consume the inputPCollection on a per-element basis, and must all be mutually compatible.