io.cdap.cdap.etl.api.batch

Class BatchConfigurable<T extends BatchContext>

java.lang.Object

io.cdap.cdap.etl.api.batch.BatchConfigurable<T>

Type Parameters:

T - batch execution context

All Implemented Interfaces:

PipelineConfigurable, SubmitterLifecycle<T>

Direct Known Subclasses:

BatchAggregator, BatchSink, BatchSource

@Beta
public abstract class BatchConfigurable<T extends BatchContext>
extends Object
implements PipelineConfigurable, SubmitterLifecycle<T>

Base class for Batch run configuration methods.

Constructor Summary

Constructors

Constructor and Description
BatchConfigurable()

Method Summary

Modifier and Type	Method and Description
void	configurePipeline(PipelineConfigurer pipelineConfigurer) Configure an ETL pipeline, registering datasets and plugins that the stage needs.
void	onRunFinish(boolean succeeded, T context) Invoked after the Batch run finishes.
abstract void	prepareRun(T context) Prepare the Batch run.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BatchConfigurable

public BatchConfigurable()

Method Detail

configurePipeline

public void configurePipeline(PipelineConfigurer pipelineConfigurer)

Description copied from interface: PipelineConfigurable

Configure an ETL pipeline, registering datasets and plugins that the stage needs. Validation should be performed in this method, throwing a ValidationException if there are any invalid config properties, or if the input or output schema is not compatible. Output schema should also be set. This method is called many times during the lifecycle of a pipeline so it should not generate any side effects. When the pipeline is being constructed, this is called in order to validate the pipeline and propagate schema. Any datasets registered at this time will be ignored. Config properties that contain macros will not have been evaluated yet. When the pipeline is deployed, this is called in order to validate the pipeline and create any datasets that are registered. Config properties that contain macros will not have been evaluated yet.

Specified by:

configurePipeline in interface PipelineConfigurable

Parameters:

pipelineConfigurer - the configurer used to register required datasets and plugins

prepareRun

public abstract void prepareRun(T context)
                         throws Exception

Prepare the Batch run. Used to configure the job before starting the run.

Specified by:

prepareRun in interface SubmitterLifecycle<T extends BatchContext>

Parameters:

context - batch execution context

Throws:

Exception - if there's an error during this method invocation

onRunFinish

public void onRunFinish(boolean succeeded,
                        T context)

Invoked after the Batch run finishes. Used to perform any end of the run logic.

Specified by:

onRunFinish in interface SubmitterLifecycle<T extends BatchContext>

Parameters:

succeeded - defines the result of batch execution: true if run succeeded, false otherwise

context - batch execution context