com.google.cloud.dataflow.sdk.options

Interface DataflowPipelineDebugOptions

All Superinterfaces:

PipelineOptions

All Known Subinterfaces:

BlockingDataflowPipelineOptions, DataflowPipelineOptions, DataflowWorkerHarnessOptions, TestDataflowPipelineOptions

public interface DataflowPipelineDebugOptions
extends PipelineOptions

Internal. Options used to control execution of the Dataflow SDK for debugging and testing purposes.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	DataflowPipelineDebugOptions.DataflowClientFactory Returns the default Dataflow client built from the passed in PipelineOptions.
static class	DataflowPipelineDebugOptions.NameMap Map of old names to new names.
static class	DataflowPipelineDebugOptions.PathValidatorFactory Creates a PathValidator object using the class specified in getPathValidatorClass().
static class	DataflowPipelineDebugOptions.StagerFactory Creates a Stager object using the class specified in getStagerClass().

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

PipelineOptions.CheckEnabled

Method Summary

Modifier and Type	Method and Description
String	getApiRootUrl() The root URL for the Dataflow API.
com.google.api.services.dataflow.Dataflow	getDataflowClient() An instance of the Dataflow client.
String	getDataflowEndpoint() Dataflow endpoint to use.
String	getDataflowJobFile() The path to write the translated Dataflow job specification out to at job submission time.
List<String>	getExperiments() The list of backend experiments to enable.
String	getOverrideWindmillBinary() Custom windmill_main binary to use with the streaming runner.
com.google.cloud.dataflow.sdk.util.PathValidator	getPathValidator() The path validator instance that should be created and used to validate paths.
Class<? extends com.google.cloud.dataflow.sdk.util.PathValidator>	getPathValidatorClass() The class of the validator that should be created and used to validate paths.
String	getPubsubRootUrl() Root URL for use with the Pubsub API.
boolean	getReload() Whether to reload the currently running pipeline with the same name as this one.
com.google.cloud.dataflow.sdk.util.Stager	getStager() The resource stager instance that should be created and used to stage resources.
Class<? extends com.google.cloud.dataflow.sdk.util.Stager>	getStagerClass() The class responsible for staging resources to be accessible by workers during job execution.
DataflowPipelineDebugOptions.NameMap	getTransformNameMapping() Mapping of old PTranform names to new ones, specified as a semicolon-separated list of oldName=newName pairs.
void	setApiRootUrl(String value)
void	setDataflowClient(com.google.api.services.dataflow.Dataflow value)
void	setDataflowEndpoint(String value)
void	setDataflowJobFile(String value)
void	setExperiments(List<String> value)
void	setOverrideWindmillBinary(String value)
void	setPathValidator(com.google.cloud.dataflow.sdk.util.PathValidator validator)
void	setPathValidatorClass(Class<? extends com.google.cloud.dataflow.sdk.util.PathValidator> validatorClass)
void	setPubsubRootUrl(String value)
void	setReload(boolean value)
void	setStager(com.google.cloud.dataflow.sdk.util.Stager stager)
void	setStagerClass(Class<? extends com.google.cloud.dataflow.sdk.util.Stager> stagerClass)
void	setTransformNameMapping(DataflowPipelineDebugOptions.NameMap value)

Methods inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

as, cloneAs, getRunner, getStableUniqueNames, setRunner, setStableUniqueNames

Method Detail

getExperiments

List<String> getExperiments()

The list of backend experiments to enable.

Dataflow provides a number of experimental features that can be enabled with this flag.

Please sync with the Dataflow team before enabling any experiments.

setExperiments

void setExperiments(List<String> value)

getApiRootUrl

String getApiRootUrl()

The root URL for the Dataflow API. dataflowEndpoint can override this value if it contains an absolute URL, otherwise apiRootUrl will be combined with dataflowEndpoint to generate the full URL to communicate with the Dataflow API.

setApiRootUrl

void setApiRootUrl(String value)

getDataflowEndpoint

String getDataflowEndpoint()

Dataflow endpoint to use.

Defaults to the current version of the Google Cloud Dataflow API, at the time the current SDK version was released.

If the string contains "://", then this is treated as a url, otherwise getApiRootUrl() is used as the root url.

setDataflowEndpoint

void setDataflowEndpoint(String value)

getDataflowJobFile

String getDataflowJobFile()

The path to write the translated Dataflow job specification out to at job submission time. The Dataflow job specification will be represented in JSON format.

setDataflowJobFile

void setDataflowJobFile(String value)

getPathValidatorClass

Class<? extends com.google.cloud.dataflow.sdk.util.PathValidator> getPathValidatorClass()

The class of the validator that should be created and used to validate paths. If pathValidator has not been set explicitly, an instance of this class will be constructed and used as the path validator.

setPathValidatorClass

void setPathValidatorClass(Class<? extends com.google.cloud.dataflow.sdk.util.PathValidator> validatorClass)

getPathValidator

com.google.cloud.dataflow.sdk.util.PathValidator getPathValidator()

The path validator instance that should be created and used to validate paths. If no path validator has been set explicitly, the default is to use the instance factory that constructs a path validator based upon the currently set pathValidatorClass.

setPathValidator

void setPathValidator(com.google.cloud.dataflow.sdk.util.PathValidator validator)

getStagerClass

Class<? extends com.google.cloud.dataflow.sdk.util.Stager> getStagerClass()

The class responsible for staging resources to be accessible by workers during job execution.

setStagerClass

void setStagerClass(Class<? extends com.google.cloud.dataflow.sdk.util.Stager> stagerClass)

getStager

com.google.cloud.dataflow.sdk.util.Stager getStager()

The resource stager instance that should be created and used to stage resources. If no stager has been set explicitly, the default is to use the instance factory that constructs a resource stager based upon the currently set stagerClass.

setStager

void setStager(com.google.cloud.dataflow.sdk.util.Stager stager)

getDataflowClient

com.google.api.services.dataflow.Dataflow getDataflowClient()

An instance of the Dataflow client. Defaults to creating a Dataflow client using the current set of options.

setDataflowClient

void setDataflowClient(com.google.api.services.dataflow.Dataflow value)

getPubsubRootUrl

String getPubsubRootUrl()

Root URL for use with the Pubsub API.

setPubsubRootUrl

void setPubsubRootUrl(String value)

getReload

@Experimental
boolean getReload()

Whether to reload the currently running pipeline with the same name as this one.

setReload

void setReload(boolean value)

getTransformNameMapping

@Experimental
DataflowPipelineDebugOptions.NameMap getTransformNameMapping()

Mapping of old PTranform names to new ones, specified as a semicolon-separated list of oldName=newName pairs. To mark a transform as deleted, omit newName.

setTransformNameMapping

void setTransformNameMapping(DataflowPipelineDebugOptions.NameMap value)

getOverrideWindmillBinary

String getOverrideWindmillBinary()

Custom windmill_main binary to use with the streaming runner.

setOverrideWindmillBinary

void setOverrideWindmillBinary(String value)