Postfix the name field
Postfix the name field
Have a grouping postfix in the id field
Have a grouping postfix in the id field
Id naming is more restrictive, it is recommended to not changing the id unleass you have a good reason
Id field will be prefixed with name
Id field will be prefixed with name
Id naming is more restrictive, it is recommended to not changing the id unless you have a good reason
Name of the pipeline object, if not set, it will defaults to
Name of the pipeline object, if not set, it will defaults to
Option(id)
Give the object a name prefix
Give the object a name prefix
Runs a Spark job on a cluster. The cluster can be an EMR cluster managed by AWS Data Pipeline or another resource if you use TaskRunner. Use SparkActivity when you want to run work in parallel. This allows you to use the scheduling resources of the YARN framework or the MapReduce resource negotiator in Hadoop 1. If you would like to run work sequentially using the Amazon EMR Step action, you can still use SparkActivity.