co.cask.cdap.datapipeline

Class ExternalSparkProgram

java.lang.Object

co.cask.cdap.internal.api.AbstractProgramDatasetConfigurable<T>

co.cask.cdap.internal.api.AbstractPluginConfigurable<SparkConfigurer>

co.cask.cdap.api.spark.AbstractSpark

co.cask.cdap.datapipeline.ExternalSparkProgram

All Implemented Interfaces:

ProgramLifecycle<SparkClientContext>, Spark

public class ExternalSparkProgram
extends AbstractSpark

A Spark program that instantiates a sparkprogram plugin and executes it. The plugin is a self contained spark program.

Field Summary

Fields

Modifier and Type	Field and Description
static String	STAGE_NAME

Constructor Summary

Constructors

Constructor and Description
ExternalSparkProgram(BatchPhaseSpec phaseSpec, co.cask.cdap.etl.spec.StageSpec stageSpec)

Method Summary

Modifier and Type	Method and Description
protected void	configure()
void	destroy()
protected void	initialize()

Methods inherited from class co.cask.cdap.api.spark.AbstractSpark

configure, getConfigurer, getContext, initialize, setClientResources, setDescription, setDriverResources, setExecutorResources, setMainClass, setMainClassName, setName, setProperties

Methods inherited from class co.cask.cdap.internal.api.AbstractPluginConfigurable

usePlugin, usePlugin, usePluginClass, usePluginClass

Methods inherited from class co.cask.cdap.internal.api.AbstractProgramDatasetConfigurable

addDatasetModule, addDatasetType, addStream, addStream, createDataset, createDataset, createDataset, createDataset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STAGE_NAME

public static final String STAGE_NAME

See Also:

Constant Field Values

Constructor Detail

ExternalSparkProgram

public ExternalSparkProgram(BatchPhaseSpec phaseSpec,
                            co.cask.cdap.etl.spec.StageSpec stageSpec)

Method Detail

configure

protected void configure()

Overrides:

configure in class AbstractSpark

initialize

protected void initialize()
                   throws Exception

Overrides:

initialize in class AbstractSpark

Throws:

Exception

destroy

public void destroy()

Specified by:

destroy in interface ProgramLifecycle<SparkClientContext>

Overrides:

destroy in class AbstractSpark