org.apache.beam.sdk.util.construction

Interface PTransformTranslation.TransformPayloadTranslator<T extends PTransform<?,?>>

All Known Implementing Classes:

CombineTranslation.CombineGloballyPayloadTranslator, CombineTranslation.CombineGroupedValuesPayloadTranslator, CombineTranslation.CombinePerKeyPayloadTranslator, FlattenTranslator, PTransformTranslation.TransformPayloadTranslator.NotSerializable, ReadTranslation.BoundedReadPayloadTranslator, ReadTranslation.UnboundedReadPayloadTranslator, SplittableParDo.ProcessKeyedElementsTranslator, WindowIntoTranslation.WindowIntoPayloadTranslator

Enclosing class:

PTransformTranslation

public static interface PTransformTranslation.TransformPayloadTranslator<T extends PTransform<?,?>>

A translator between a Java-based PTransform and a protobuf payload for that transform.

When going to a protocol buffer message, the translator produces a payload corresponding to the Java representation while registering components that payload references.

Also, provides methods for generating a Row-based constructor config for the transform that can be later used to re-construct the transform.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	PTransformTranslation.TransformPayloadTranslator.NotSerializable<T extends PTransform<?,?>> A PTransformTranslation.TransformPayloadTranslator for transforms that contain no references to components, so they do not need a specialized rehydration.

Method Summary

Modifier and Type	Method and Description
default T	fromConfigRow(Row configRow, PipelineOptions options) Construts a transform from a provided Row-based construction configuration.
java.lang.String	getUrn() Provides a unique URN for transforms represented by this TransformPayloadTranslator.
default java.lang.String	getUrn(T transform) Same as getUrn() but the returned URN may depend on the transform provided.
default Row	toConfigRow(T transform) Generates a Row-based construction configuration for the provided transform.
@Nullable org.apache.beam.model.pipeline.v1.RunnerApi.FunctionSpec	translate(AppliedPTransform<?,?,T> application, SdkComponents components) Translates the given transform represented by the provided AppliedPTransform to a FunctionSpec with a URN and a payload.

Method Detail

getUrn

java.lang.String getUrn()

Provides a unique URN for transforms represented by this TransformPayloadTranslator.

getUrn

default java.lang.String getUrn(T transform)

Same as getUrn() but the returned URN may depend on the transform provided.

Only override this if the same TransformPayloadTranslator used for multiple transforms. Otherwise, use getUrn().

translate

@Nullable org.apache.beam.model.pipeline.v1.RunnerApi.FunctionSpec translate(AppliedPTransform<?,?,T> application,
                                                                             SdkComponents components)
                                                                      throws java.io.IOException

Translates the given transform represented by the provided AppliedPTransform to a FunctionSpec with a URN and a payload.

Parameters:

application - an AppliedPTransform that includes the transform to be expanded.

components - components of the pipeline that includes the transform.

Returns:

a generated spec for the transform to be included in the pipeline proto. If return value is null, transform should include an empty spec.

Throws:

java.io.IOException

toConfigRow

default Row toConfigRow(T transform)

Generates a Row-based construction configuration for the provided transform.

Parameters:

transform - a transform represented by the current TransformPayloadTranslator.

Returns:

fromConfigRow

default T fromConfigRow(Row configRow,
                        PipelineOptions options)

Construts a transform from a provided Row-based construction configuration.

Parameters:

configRow - a construction configuration similar to what would be generated by the toConfigRow(PTransform) method.

Returns:

a transform represented by the current TransformPayloadTranslator.