com.google.cloud.dataflow.sdk.options

Interface DataflowPipelineWorkerPoolOptions

All Superinterfaces:

PipelineOptions

All Known Subinterfaces:

BlockingDataflowPipelineOptions, DataflowPipelineOptions, DataflowWorkerHarnessOptions, TestDataflowPipelineOptions

public interface DataflowPipelineWorkerPoolOptions
extends PipelineOptions

Options which are used to configure the Dataflow pipeline worker pool.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType Type of autoscaling algorithm to use.
static class	DataflowPipelineWorkerPoolOptions.ClusterManagerApiType Type of API for handling cluster management,i.e.
static class	DataflowPipelineWorkerPoolOptions.TeardownPolicy The policy for tearing down the workers spun up by the service.

Method Summary

Methods

Modifier and Type	Method and Description
DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType	getAutoscalingAlgorithm()
DataflowPipelineWorkerPoolOptions.ClusterManagerApiType	getClusterManagerApi()
int	getDiskSizeGb() Remote worker disk size, in gigabytes, or 0 to use the default size.
java.lang.String	getDiskSourceImage() Disk source image to use by VMs for jobs.
java.util.List<java.lang.String>	getFilesToStage() List of local files to make available to workers.
int	getMaxNumWorkers() Max number of workers to use when using workerpool autoscaling.
int	getNumWorkers() Number of workers to use in remote execution.
DataflowPipelineWorkerPoolOptions.TeardownPolicy	getTeardownPolicy() Teardown policy for the VMs.
java.lang.String	getWorkerMachineType() Machine type to create worker VMs as.
java.lang.String	getZone() GCE availability zone for launching workers.
void	setAutoscalingAlgorithm(DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType value)
void	setClusterManagerApi(DataflowPipelineWorkerPoolOptions.ClusterManagerApiType value)
void	setDiskSizeGb(int value)
void	setDiskSourceImage(java.lang.String value)
void	setFilesToStage(java.util.List<java.lang.String> value)
void	setMaxNumWorkers(int value)
void	setNumWorkers(int value)
void	setTeardownPolicy(DataflowPipelineWorkerPoolOptions.TeardownPolicy value)
void	setWorkerMachineType(java.lang.String value)
void	setZone(java.lang.String value)

Methods inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

as, getRunner, setRunner

Method Detail

getDiskSourceImage

java.lang.String getDiskSourceImage()

Disk source image to use by VMs for jobs.

See Also:

Compute Engine Images

setDiskSourceImage

void setDiskSourceImage(java.lang.String value)

getNumWorkers

int getNumWorkers()

Number of workers to use in remote execution.

setNumWorkers

void setNumWorkers(int value)

getAutoscalingAlgorithm

DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType getAutoscalingAlgorithm()

setAutoscalingAlgorithm

void setAutoscalingAlgorithm(DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType value)

getMaxNumWorkers

int getMaxNumWorkers()

Max number of workers to use when using workerpool autoscaling. This option is experimental and subject to change.

setMaxNumWorkers

void setMaxNumWorkers(int value)

getDiskSizeGb

int getDiskSizeGb()

Remote worker disk size, in gigabytes, or 0 to use the default size.

setDiskSizeGb

void setDiskSizeGb(int value)

getZone

java.lang.String getZone()

GCE availability zone for launching workers.

Default is up to the service.

setZone

void setZone(java.lang.String value)

getClusterManagerApi

DataflowPipelineWorkerPoolOptions.ClusterManagerApiType getClusterManagerApi()

setClusterManagerApi

void setClusterManagerApi(DataflowPipelineWorkerPoolOptions.ClusterManagerApiType value)

getWorkerMachineType

java.lang.String getWorkerMachineType()

Machine type to create worker VMs as.

setWorkerMachineType

void setWorkerMachineType(java.lang.String value)

getTeardownPolicy

DataflowPipelineWorkerPoolOptions.TeardownPolicy getTeardownPolicy()

Teardown policy for the VMs.

By default this is left unset and the service sets the default policy.

setTeardownPolicy

void setTeardownPolicy(DataflowPipelineWorkerPoolOptions.TeardownPolicy value)

getFilesToStage

java.util.List<java.lang.String> getFilesToStage()

List of local files to make available to workers.

Jars are placed on the worker's classpath.

The default value is the list of jars from the main program's classpath.

setFilesToStage

void setFilesToStage(java.util.List<java.lang.String> value)