com.google.cloud.dataflow.sdk.options

Interface DataflowWorkerHarnessOptions

All Superinterfaces:

ApplicationNameOptions, BigQueryOptions, CloudDebuggerOptions, DataflowPipelineDebugOptions, DataflowPipelineOptions, DataflowPipelineWorkerPoolOptions, DataflowProfilingOptions, DataflowWorkerLoggingOptions, GcpOptions, GcsOptions, GoogleApiDebugOptions, HasDisplayData, PipelineOptions, StreamingOptions

@Hidden
public interface DataflowWorkerHarnessOptions
extends DataflowPipelineOptions

Options that are used exclusively within the Dataflow worker harness. These options have no effect at pipeline creation time.

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions

DataflowPipelineOptions.JobNameFactory

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions

DataflowPipelineDebugOptions.DataflowClientFactory, DataflowPipelineDebugOptions.PathValidatorFactory, DataflowPipelineDebugOptions.StagerFactory

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions

DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType, DataflowPipelineWorkerPoolOptions.TeardownPolicy, DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GcpOptions

GcpOptions.CredentialDirFactory, GcpOptions.DefaultProjectFactory, GcpOptions.GcpUserCredentialsFactory

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions

GoogleApiDebugOptions.GoogleApiTracer

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

PipelineOptions.CheckEnabled

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GcsOptions

GcsOptions.ExecutorServiceFactory

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions

DataflowWorkerLoggingOptions.Level, DataflowWorkerLoggingOptions.WorkerLogLevelOverrides

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions

DataflowProfilingOptions.DataflowProfilingAgentConfiguration

Method Summary

Modifier and Type	Method and Description
String	getJobId() The identity of the Dataflow job.
Integer	getWorkerCacheMb() The size of the worker's in-memory cache, in megabytes.
String	getWorkerId() The identity of the worker running this pipeline.
void	setJobId(String value)
void	setWorkerCacheMb(Integer value)
void	setWorkerId(String value)

Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions

getJobName, getProject, getStagingLocation, getUpdate, setJobName, setProject, setStagingLocation, setUpdate

Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions

getApiRootUrl, getDataflowClient, getDataflowEndpoint, getDataflowJobFile, getDumpHeapOnOOM, getExperiments, getNumberOfWorkerHarnessThreads, getOverrideWindmillBinary, getPathValidator, getPathValidatorClass, getPubsubRootUrl, getStager, getStagerClass, getTransformNameMapping, getWindmillServiceEndpoint, getWindmillServicePort, setApiRootUrl, setDataflowClient, setDataflowEndpoint, setDataflowJobFile, setDumpHeapOnOOM, setExperiments, setNumberOfWorkerHarnessThreads, setOverrideWindmillBinary, setPathValidator, setPathValidatorClass, setPubsubRootUrl, setStager, setStagerClass, setTransformNameMapping, setWindmillServiceEndpoint, setWindmillServicePort

Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions

getAutoscalingAlgorithm, getDiskSizeGb, getFilesToStage, getMaxNumWorkers, getNetwork, getNumWorkers, getSubnetwork, getTeardownPolicy, getUsePublicIps, getWorkerDiskType, getWorkerHarnessContainerImage, getWorkerMachineType, getZone, setAutoscalingAlgorithm, setDiskSizeGb, setFilesToStage, setMaxNumWorkers, setNetwork, setNumWorkers, setSubnetwork, setTeardownPolicy, setUsePublicIps, setWorkerDiskType, setWorkerHarnessContainerImage, setWorkerMachineType, setZone

Methods inherited from interface com.google.cloud.dataflow.sdk.options.BigQueryOptions

getTempDatasetId, setTempDatasetId

Methods inherited from interface com.google.cloud.dataflow.sdk.options.StreamingOptions

isStreaming, setStreaming

Methods inherited from interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions

getAppName, setAppName

Methods inherited from interface com.google.cloud.dataflow.sdk.options.GcpOptions

getAuthorizationServerEncodedUrl, getCredentialDir, getCredentialFactoryClass, getCredentialId, getGcpCredential, getSecretsFile, getServiceAccountKeyfile, getServiceAccountName, getTokenServerUrl, setAuthorizationServerEncodedUrl, setCredentialDir, setCredentialFactoryClass, setCredentialId, setGcpCredential, setSecretsFile, setServiceAccountKeyfile, setServiceAccountName, setTokenServerUrl

Methods inherited from interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions

getGoogleApiTrace, setGoogleApiTrace

Methods inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

as, cloneAs, getRunner, getStableUniqueNames, getTempLocation, setRunner, setStableUniqueNames, setTempLocation

Methods inherited from interface com.google.cloud.dataflow.sdk.transforms.display.HasDisplayData

populateDisplayData

Methods inherited from interface com.google.cloud.dataflow.sdk.options.GcsOptions

getExecutorService, getGcsEndpoint, getGcsUploadBufferSizeBytes, getGcsUtil, setExecutorService, setGcsEndpoint, setGcsUploadBufferSizeBytes, setGcsUtil

Methods inherited from interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions

getDebuggee, getEnableCloudDebugger, getMaxConditionCost, setDebuggee, setEnableCloudDebugger, setMaxConditionCost

Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions

getDefaultWorkerLogLevel, getWorkerLogLevelOverrides, setDefaultWorkerLogLevel, setWorkerLogLevelOverrides

Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions

getEnableProfilingAgent, getProfilingAgentConfiguration, setEnableProfilingAgent, setProfilingAgentConfiguration

Method Detail

getWorkerId

String getWorkerId()

The identity of the worker running this pipeline.

setWorkerId

void setWorkerId(String value)

getJobId

String getJobId()

The identity of the Dataflow job.

setJobId

void setJobId(String value)

getWorkerCacheMb

@Default.Integer(value=100)
Integer getWorkerCacheMb()

The size of the worker's in-memory cache, in megabytes.

Currently, this cache is used for storing read values of side inputs.

setWorkerCacheMb

void setWorkerCacheMb(Integer value)