com.google.cloud.dataflow.sdk.options

Interface DataflowWorkerHarnessOptions

All Superinterfaces:

ApplicationNameOptions, BigQueryOptions, CloudDebuggerOptions, DataflowPipelineDebugOptions, DataflowPipelineOptions, DataflowPipelineWorkerPoolOptions, DataflowWorkerLoggingOptions, GcpOptions, GcsOptions, GoogleApiDebugOptions, PipelineOptions, StreamingOptions

public interface DataflowWorkerHarnessOptions
extends DataflowPipelineOptions

Options that are used exclusively within the Dataflow worker harness. These options have no effect at pipeline creation time.

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions

DataflowPipelineOptions.JobNameFactory

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions

DataflowPipelineDebugOptions.DataflowClientFactory, DataflowPipelineDebugOptions.PathValidatorFactory, DataflowPipelineDebugOptions.StagerFactory

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions

DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType, DataflowPipelineWorkerPoolOptions.TeardownPolicy

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GcpOptions

GcpOptions.CredentialDirFactory, GcpOptions.DefaultProjectFactory, GcpOptions.GcpUserCredentialsFactory

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions

GoogleApiDebugOptions.GoogleApiTracer

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

PipelineOptions.CheckEnabled

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GcsOptions

GcsOptions.ExecutorServiceFactory

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions

DataflowWorkerLoggingOptions.Level, DataflowWorkerLoggingOptions.WorkerLogLevelOverrides

Field Summary

Fields inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions

DATAFLOW_STORAGE_LOCATION

Method Summary

Modifier and Type	Method and Description
String	getJobId() The identity of the Dataflow job.
Integer	getWorkerCacheMb() The size of the worker's in-memory cache, in megabytes.
String	getWorkerId() The identity of the worker running this pipeline.
void	setJobId(String value)
void	setWorkerCacheMb(Integer value)
void	setWorkerId(String value)

Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions

getJobName, getProject, getStagingLocation, getTempLocation, setJobName, setProject, setStagingLocation, setTempLocation

Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions

getApiRootUrl, getDataflowClient, getDataflowEndpoint, getDataflowJobFile, getExperiments, getNumberOfWorkerHarnessThreads, getOverrideWindmillBinary, getPathValidator, getPathValidatorClass, getPubsubRootUrl, getStager, getStagerClass, getTransformNameMapping, getUpdate, setApiRootUrl, setDataflowClient, setDataflowEndpoint, setDataflowJobFile, setExperiments, setNumberOfWorkerHarnessThreads, setOverrideWindmillBinary, setPathValidator, setPathValidatorClass, setPubsubRootUrl, setStager, setStagerClass, setTransformNameMapping, setUpdate

Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions

getAutoscalingAlgorithm, getDiskSizeGb, getDiskSourceImage, getFilesToStage, getMaxNumWorkers, getNetwork, getNumWorkers, getTeardownPolicy, getWorkerDiskType, getWorkerMachineType, getZone, setAutoscalingAlgorithm, setDiskSizeGb, setDiskSourceImage, setFilesToStage, setMaxNumWorkers, setNetwork, setNumWorkers, setTeardownPolicy, setWorkerDiskType, setWorkerMachineType, setZone

Methods inherited from interface com.google.cloud.dataflow.sdk.options.BigQueryOptions

getTempDatasetId, setTempDatasetId

Methods inherited from interface com.google.cloud.dataflow.sdk.options.StreamingOptions

isStreaming, setStreaming

Methods inherited from interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions

getAppName, setAppName

Methods inherited from interface com.google.cloud.dataflow.sdk.options.GcpOptions

getCredentialDir, getCredentialFactoryClass, getCredentialId, getGcpCredential, getSecretsFile, getServiceAccountKeyfile, getServiceAccountName, setCredentialDir, setCredentialFactoryClass, setCredentialId, setGcpCredential, setSecretsFile, setServiceAccountKeyfile, setServiceAccountName

Methods inherited from interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions

getGoogleApiTrace, setGoogleApiTrace

Methods inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

as, cloneAs, getRunner, getStableUniqueNames, setRunner, setStableUniqueNames

Methods inherited from interface com.google.cloud.dataflow.sdk.options.GcsOptions

getExecutorService, getGcsEndpoint, getGcsUtil, setExecutorService, setGcsEndpoint, setGcsUtil

Methods inherited from interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions

getEnableCloudDebugger, setEnableCloudDebugger

Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions

getDefaultWorkerLogLevel, getWorkerLogLevelOverrides, setDefaultWorkerLogLevel, setWorkerLogLevelOverrides

Method Detail

getWorkerId

String getWorkerId()

The identity of the worker running this pipeline.

setWorkerId

void setWorkerId(String value)

getJobId

String getJobId()

The identity of the Dataflow job.

setJobId

void setJobId(String value)

getWorkerCacheMb

Integer getWorkerCacheMb()

The size of the worker's in-memory cache, in megabytes.

Currently, this cache is used for storing read values of side inputs.

setWorkerCacheMb

void setWorkerCacheMb(Integer value)