io.cdap.cdap.api.spark

Class SparkSpecification

java.lang.Object

io.cdap.cdap.api.AbstractProgramSpecification

io.cdap.cdap.api.spark.SparkSpecification

All Implemented Interfaces:

PropertyProvider, ProgramSpecification

@Beta
public final class SparkSpecification
extends AbstractProgramSpecification
implements PropertyProvider

A default specification for Spark programs

Constructor Summary

Constructors

Constructor and Description
SparkSpecification(String className, String name, String description, String mainClassName, Set<String> datasets, Map<String,String> properties, Resources clientResources, Resources driverResources, Resources executorResources, List<SparkHttpServiceHandlerSpecification> handlers, Map<String,Plugin> plugins)

Method Summary

Modifier and Type	Method and Description
Resources	getClientResources()
Set<String>	getDatasets() Returns the set of static datasets used by the Spark program.
Resources	getDriverResources()
Resources	getExecutorResources()
List<SparkHttpServiceHandlerSpecification>	getHandlers()
String	getMainClassName() Get the main class name for the Spark program.
Map<String,String>	getProperties() Returns an immutable Map of all properties.
String	getProperty(String key) Return the property value of a given key.

Methods inherited from class io.cdap.cdap.api.AbstractProgramSpecification

getClassName, getDescription, getName, getPlugins

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkSpecification

public SparkSpecification(String className,
                          String name,
                          String description,
                          @Nullable
                          String mainClassName,
                          Set<String> datasets,
                          Map<String,String> properties,
                          @Nullable
                          Resources clientResources,
                          @Nullable
                          Resources driverResources,
                          @Nullable
                          Resources executorResources,
                          List<SparkHttpServiceHandlerSpecification> handlers,
                          Map<String,Plugin> plugins)

Method Detail

getMainClassName

@Nullable
public String getMainClassName()

Get the main class name for the Spark program.

Returns:

the fully qualified class name or null if it was not set

getProperties

public Map<String,String> getProperties()

Description copied from interface: PropertyProvider

Returns an immutable Map of all properties.

Specified by:

getProperties in interface PropertyProvider

getDatasets

public Set<String> getDatasets()

Returns the set of static datasets used by the Spark program.

getProperty

public String getProperty(String key)

Description copied from interface: PropertyProvider

Return the property value of a given key.

Specified by:

getProperty in interface PropertyProvider

Parameters:

key - for getting specific property value.

Returns:

The value associated with the key or null if not such key exists.

getClientResources

@Nullable
public Resources getClientResources()

Returns:

Resources requirement for the Spark client process or null if not specified.

getDriverResources

@Nullable
public Resources getDriverResources()

Returns:

Resources requirement for the Spark driver process or null if not specified.

getExecutorResources

@Nullable
public Resources getExecutorResources()

Returns:

Resources requirement for the Spark executor processes or null if not specified.

getHandlers

public List<SparkHttpServiceHandlerSpecification> getHandlers()

Returns:

a List of SparkHttpServiceHandlerSpecification defined for the Spark program.