com.google.cloud.dataflow.sdk.testing

Class TestPipeline

java.lang.Object

com.google.cloud.dataflow.sdk.Pipeline

com.google.cloud.dataflow.sdk.testing.TestPipeline

public class TestPipeline
extends Pipeline

A creator of test pipelines that can be used inside of tests that can be configured to run locally or against the live service.

It is recommended to tag hand-selected tests for this purpose using the RunnableOnService Category annotation, as each test run against the service will spin up and tear down a single VM.

In order to run tests on the dataflow pipeline service, the following conditions must be met:

• runIntegrationTestOnService System property must be set to true.
• System property "projectName" must be set to your Cloud project.
• System property "temp_gcs_directory" must be set to a valid GCS bucket.
• Jars containing the SDK and test classes must be added to the test classpath.

Use DataflowAssert for tests, as it integrates with this test harness in both direct and remote execution modes. For example:

 Pipeline p = TestPipeline.create();
 PCollection<Integer> output = ...

 DataflowAssert.that(output)
     .containsInAnyOrder(1, 2, 3, 4);
 p.run();
 

Nested Class Summary

Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.Pipeline

Pipeline.PipelineExecutionException, Pipeline.PipelineVisitor

Method Summary

Modifier and Type	Method and Description
static TestPipeline	create() Creates and returns a new test pipeline.
static TestDataflowPipelineOptions	getPipelineOptions() Creates PipelineOptions for testing with a DataflowPipelineRunner.
static boolean	isIntegrationTest() Returns whether this test is running on the Cloud Dataflow service as described in TestPipeline.
PipelineResult	run() Runs this TestPipeline, unwrapping any AssertionError that is raised during testing.
String	toString()

Methods inherited from class com.google.cloud.dataflow.sdk.Pipeline

addValueInternal, apply, apply, applyTransform, applyTransform, begin, create, getCoderRegistry, getFullNameForTesting, getOptions, getRunner, setCoderRegistry, traverseTopologically

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

create

public static TestPipeline create()

Creates and returns a new test pipeline.

Use DataflowAssert to add tests, then call Pipeline.run() to execute the pipeline and check the tests.

isIntegrationTest

public static boolean isIntegrationTest()

Returns whether this test is running on the Cloud Dataflow service as described in TestPipeline.

run

public PipelineResult run()

Runs this TestPipeline, unwrapping any AssertionError that is raised during testing.

Overrides:

run in class Pipeline

toString

public String toString()

Overrides:

toString in class Pipeline

getPipelineOptions

public static TestDataflowPipelineOptions getPipelineOptions()

Creates PipelineOptions for testing with a DataflowPipelineRunner.