org.apache.flink.streaming.api.graph

类 StreamGraphGenerator

java.lang.Object

org.apache.flink.streaming.api.graph.StreamGraphGenerator

@Internal
public class StreamGraphGenerator
extends Object

A generator that generates a StreamGraph from a graph of StreamTransformations.

This traverses the tree of StreamTransformations starting from the sinks. At each transformation we recursively transform the inputs, then create a node in the StreamGraph and add edges from the input Nodes to our newly created node. The transformation methods return the IDs of the nodes in the StreamGraph that represent the input transformation. Several IDs can be returned to be able to deal with feedback transformations and unions.

Partitioning, split/select and union don't create actual nodes in the StreamGraph. For these, we create a virtual node in the StreamGraph that holds the specific property, i.e. partitioning, selector and so on. When an edge is created from a virtual node to a downstream node the StreamGraph resolved the id of the original node and creates an edge in the graph with the desired property. For example, if you have this graph:

     Map-1 -> HashPartition-2 -> Map-3
 

where the numbers represent transformation IDs. We first recurse all the way down. Map-1 is transformed, i.e. we create a StreamNode with ID 1. Then we transform the HashPartition, for this, we create virtual node of ID 4 that holds the property HashPartition. This transformation returns the ID 4. Then we transform the Map-3. We add the edge 4 -> 3. The StreamGraph resolved the actual node with ID 1 and creates and edge 1 -> 3 with the property HashPartition.

字段概要

字段

限定符和类型	字段和说明
static int	DEFAULT_LOWER_BOUND_MAX_PARALLELISM
protected static Integer	iterationIdCounter
static int	UPPER_BOUND_MAX_PARALLELISM

方法概要

限定符和类型	方法和说明
static StreamGraph	generate(StreamExecutionEnvironment env, List<StreamTransformation<?>> transformations) Generates a StreamGraph by traversing the graph of StreamTransformations starting from the given transformations.
static int	getNewIterationNodeId()

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

DEFAULT_LOWER_BOUND_MAX_PARALLELISM

public static final int DEFAULT_LOWER_BOUND_MAX_PARALLELISM

另请参阅:

常量字段值

UPPER_BOUND_MAX_PARALLELISM

public static final int UPPER_BOUND_MAX_PARALLELISM

另请参阅:

常量字段值

iterationIdCounter

protected static Integer iterationIdCounter

方法详细资料

getNewIterationNodeId

public static int getNewIterationNodeId()

generate

public static StreamGraph generate(StreamExecutionEnvironment env,
                                   List<StreamTransformation<?>> transformations)

Generates a StreamGraph by traversing the graph of StreamTransformations starting from the given transformations.

参数:

env - The StreamExecutionEnvironment that is used to set some parameters of the job

transformations - The transformations starting from which to transform the graph

返回:

The generated StreamGraph