DoFnTester (Google Cloud Dataflow SDK 1.3.0 API)

java.lang.Object
- com.google.cloud.dataflow.sdk.transforms.DoFnTester<InputT,OutputT>

Type Parameters:: InputT - the type of the DoFn's (main) input elements; OutputT - the type of the DoFn's (main) output elements

public class DoFnTester<InputT,OutputT>
extends Object

A harness for unit-testing a DoFn.

For example:

 
 DoFn<InputT, OutputT> fn = ...;

 DoFnTester<InputT, OutputT> fnTester = DoFnTester.of(fn);

 // Set arguments shared across all batches:
 fnTester.setSideInputs(...);      // If fn takes side inputs.
 fnTester.setSideOutputTags(...);  // If fn writes to side outputs.

 // Process a batch containing a single input element:
 Input testInput = ...;
 List<OutputT> testOutputs = fnTester.processBatch(testInput);
 Assert.assertThat(testOutputs,
                   JUnitMatchers.hasItems(...));

 // Process a bigger batch:
 Assert.assertThat(fnTester.processBatch(i1, i2, ...),
                   JUnitMatchers.hasItems(...));

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`clearOutputElements()` Clears the record of the elements output so far to the main output.
`<T> void`	`clearSideOutputElements(TupleTag<T> tag)` Clears the record of the elements output so far to the side output with the given tag.
`void`	`finishBundle()` Calls `DoFn.finishBundle(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.Context)` of the `DoFn` under test.
`<OutputT> OutputT`	`getAggregatorValue(Aggregator<?,OutputT> agg)` Returns the value of the provided `Aggregator`.
`static <InputT,OutputT> DoFnTester<InputT,OutputT>`	`of(DoFn<InputT,OutputT> fn)` Returns a `DoFnTester` supporting unit-testing of the given `DoFn`.
`static <InputT,OutputT> DoFnTester<InputT,OutputT>`	`of(DoFnWithContext<InputT,OutputT> fn)` Returns a `DoFnTester` supporting unit-testing of the given `DoFn`.
`List<OutputT>`	`peekOutputElements()` Returns the elements output so far to the main output.
`<T> List<T>`	`peekSideOutputElements(TupleTag<T> tag)` Returns the elements output so far to the side output with the given tag.
`List<OutputT>`	`processBatch(InputT... inputElements)` A convenience operation that first calls `startBundle()`, then calls `processElement(InputT)` on each of the arguments, then calls `finishBundle()`, then returns the result of `takeOutputElements()`.
`void`	`processElement(InputT element)` Calls `DoFn.processElement(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.ProcessContext)` on the `DoFn` under test, in a context where `DoFn.ProcessContext.element()` returns the given element.
`void`	`setSideInput(PCollectionView<?> sideInput, Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<?>> value)` Registers the values of a side input `PCollectionView` to pass to the `DoFn` under test.
`void`	`setSideInputInGlobalWindow(PCollectionView<?> sideInput, Iterable<?> value)` Registers the values for a side input `PCollectionView` to pass to the `DoFn` under test.
`void`	`setSideInputs(Map<PCollectionView<?>,Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<?>>> sideInputs)` Registers the tuple of values of the side input `PCollectionView`s to pass to the `DoFn` under test.
`void`	`setSideOutputTags(TupleTagList sideOutputTags)` Registers the list of `TupleTag`s that can be used by the `DoFn` under test to output to side output `PCollection`s.
`void`	`startBundle()` Calls `DoFn.startBundle(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.Context)` on the `DoFn` under test.
`List<OutputT>`	`takeOutputElements()` Returns the elements output so far to the main output.
`<T> List<T>`	`takeSideOutputElements(TupleTag<T> tag)` Returns the elements output so far to the side output with the given tag.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Method Detail
  - of
```
public static <InputT,OutputT> DoFnTester<InputT,OutputT> of(DoFn<InputT,OutputT> fn)
```
    Returns a DoFnTester supporting unit-testing of the given DoFn.
  - of
```
public static <InputT,OutputT> DoFnTester<InputT,OutputT> of(DoFnWithContext<InputT,OutputT> fn)
```
    Returns a DoFnTester supporting unit-testing of the given DoFn.
  - setSideInputs
```
public void setSideInputs(Map<PCollectionView<?>,Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<?>>> sideInputs)
```
    Registers the tuple of values of the side input PCollectionViews to pass to the DoFn under test.
    If needed, first creates a fresh instance of the DoFn under test.
    If this isn't called, DoFnTester assumes the DoFn takes no side inputs.
  - setSideInput
```
public void setSideInput(PCollectionView<?> sideInput,
                         Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<?>> value)
```
    Registers the values of a side input PCollectionView to pass to the DoFn under test.
    If needed, first creates a fresh instance of the DoFn under test.
    If this isn't called, DoFnTester assumes the DoFn takes no side inputs.
  - setSideInputInGlobalWindow
```
public void setSideInputInGlobalWindow(PCollectionView<?> sideInput,
                                       Iterable<?> value)
```
    Registers the values for a side input PCollectionView to pass to the DoFn under test. All values are placed in the global window.
  - setSideOutputTags
```
public void setSideOutputTags(TupleTagList sideOutputTags)
```
    Registers the list of TupleTags that can be used by the DoFn under test to output to side output PCollections.
    If needed, first creates a fresh instance of the DoFn under test.
    If this isn't called, DoFnTester assumes the DoFn doesn't emit to any side outputs.
  - processBatch
```
public List<OutputT> processBatch(InputT... inputElements)
```
    A convenience operation that first calls startBundle(), then calls processElement(InputT) on each of the arguments, then calls finishBundle(), then returns the result of takeOutputElements().
  - startBundle
```
public void startBundle()
```
    Calls DoFn.startBundle(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.Context) on the DoFn under test.
    If needed, first creates a fresh instance of the DoFn under test.
  - processElement
```
public void processElement(InputT element)
```
    Calls DoFn.processElement(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.ProcessContext) on the DoFn under test, in a context where DoFn.ProcessContext.element() returns the given element.
    Will call startBundle() automatically, if it hasn't already been called.
    
    Throws:
    
    IllegalStateException - if the DoFn under test has already been finished
  - finishBundle
```
public void finishBundle()
```
    Calls DoFn.finishBundle(com.google.cloud.dataflow.sdk.transforms.DoFn<InputT, OutputT>.Context) of the DoFn under test.
    Will call startBundle() automatically, if it hasn't already been called.
    
    Throws:
    
    IllegalStateException - if the DoFn under test has already been finished
  - peekOutputElements
```
public List<OutputT> peekOutputElements()
```
    Returns the elements output so far to the main output. Does not clear them, so subsequent calls will continue to include these elements.
    
    See Also:
    
    takeOutputElements(),
    TODO: provide accessors that take and return {@code WindowedValue}s in order to test timestamp- and window-sensitive DoFns.
  - clearOutputElements
```
public void clearOutputElements()
```
    Clears the record of the elements output so far to the main output.
    
    See Also:
    
    peekOutputElements()
  - takeOutputElements
```
public List<OutputT> takeOutputElements()
```
    Returns the elements output so far to the main output. Clears the list so these elements don't appear in future calls.
    
    See Also:
    
    peekOutputElements()
  - peekSideOutputElements
```
public <T> List<T> peekSideOutputElements(TupleTag<T> tag)
```
    Returns the elements output so far to the side output with the given tag. Does not clear them, so subsequent calls will continue to include these elements.
    
    See Also:
    
    takeSideOutputElements(com.google.cloud.dataflow.sdk.values.TupleTag<T>), clearSideOutputElements(com.google.cloud.dataflow.sdk.values.TupleTag<T>)
  - clearSideOutputElements
```
public <T> void clearSideOutputElements(TupleTag<T> tag)
```
    Clears the record of the elements output so far to the side output with the given tag.
    
    See Also:
    
    peekSideOutputElements(com.google.cloud.dataflow.sdk.values.TupleTag<T>)
  - takeSideOutputElements
```
public <T> List<T> takeSideOutputElements(TupleTag<T> tag)
```
    Returns the elements output so far to the side output with the given tag. Clears the list so these elements don't appear in future calls.
    
    See Also:
    
    peekSideOutputElements(com.google.cloud.dataflow.sdk.values.TupleTag<T>)
  - getAggregatorValue
```
public <OutputT> OutputT getAggregatorValue(Aggregator<?,OutputT> agg)
```
    Returns the value of the provided Aggregator.

Class DoFnTester<InputT,OutputT>

Method Summary

Methods inherited from class java.lang.Object

Method Detail

of

of

setSideInputs

setSideInput

setSideInputInGlobalWindow

setSideOutputTags

processBatch

startBundle

processElement

finishBundle

peekOutputElements

clearOutputElements

takeOutputElements

peekSideOutputElements

clearSideOutputElements

takeSideOutputElements

getAggregatorValue