com.google.cloud.dataflow.sdk.util

Class DoFnRunner<InputT,OutputT>

java.lang.Object

com.google.cloud.dataflow.sdk.util.DoFnRunner<InputT,OutputT>

Type Parameters:

InputT - the type of the DoFn's (main) input elements

OutputT - the type of the DoFn's (main) output elements

Direct Known Subclasses:

StreamingSideInputDoFnRunner

public class DoFnRunner<InputT,OutputT>
extends Object

Runs a DoFn by constructing the appropriate contexts and passing them in.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DoFnRunner.ListOutputManager An implementation of DoFnRunner.OutputManager using simple lists, for testing and in-memory contexts such as the DirectPipelineRunner.
static interface	DoFnRunner.OutputManager Information about how to create output receivers and output to them.

Field Summary

Fields

Modifier and Type	Field and Description
com.google.cloud.dataflow.sdk.util.DoFnRunner.DoFnContext<InputT,OutputT>	context The context used for running the DoFn.
DoFn<InputT,OutputT>	fn The DoFn being run.

Method Summary

Modifier and Type	Method and Description
static <InputT,OutputT> DoFnRunner<InputT,OutputT>	create(PipelineOptions options, DoFn<InputT,OutputT> fn, SideInputReader sideInputReader, DoFnRunner.OutputManager outputManager, TupleTag<OutputT> mainOutputTag, List<TupleTag<?>> sideOutputTags, ExecutionContext.StepContext stepContext, CounterSet.AddCounterMutator addCounterMutator, WindowingStrategy<?,?> windowingStrategy)
protected DoFn.ProcessContext	createProcessContext(WindowedValue<InputT> elem) Returns a new DoFn.ProcessContext for the given element.
void	finishBundle() Calls DoFn.finishBundle(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.Context).
protected void	invokeProcessElement(WindowedValue<InputT> elem)
void	processElement(WindowedValue<InputT> elem) Calls DoFn.processElement(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.ProcessContext) with a ProcessContext containing the current element.
void	startBundle() Calls DoFn.startBundle(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.Context).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fn

public final DoFn<InputT,OutputT> fn

The DoFn being run.

context

public final com.google.cloud.dataflow.sdk.util.DoFnRunner.DoFnContext<InputT,OutputT> context

The context used for running the DoFn.

Method Detail

create

public static <InputT,OutputT> DoFnRunner<InputT,OutputT> create(PipelineOptions options,
                                                                 DoFn<InputT,OutputT> fn,
                                                                 SideInputReader sideInputReader,
                                                                 DoFnRunner.OutputManager outputManager,
                                                                 TupleTag<OutputT> mainOutputTag,
                                                                 List<TupleTag<?>> sideOutputTags,
                                                                 ExecutionContext.StepContext stepContext,
                                                                 CounterSet.AddCounterMutator addCounterMutator,
                                                                 WindowingStrategy<?,?> windowingStrategy)

startBundle

public void startBundle()

Calls DoFn.startBundle(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.Context).

processElement

public void processElement(WindowedValue<InputT> elem)

Calls DoFn.processElement(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.ProcessContext) with a ProcessContext containing the current element.

invokeProcessElement

protected void invokeProcessElement(WindowedValue<InputT> elem)

finishBundle

public void finishBundle()

Calls DoFn.finishBundle(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.Context).

createProcessContext

protected DoFn.ProcessContext createProcessContext(WindowedValue<InputT> elem)

Returns a new DoFn.ProcessContext for the given element.