Package tech.illuin.pipeline

Class CompositePipeline<I>

java.lang.Object

tech.illuin.pipeline.CompositePipeline<I>

All Implemented Interfaces:

AutoCloseable, Pipeline<I>

public final class CompositePipeline<I>
extends Object
implements Pipeline<I>

The CompositePipeline works by separating its run implementation into predefined categories of components:

• 1 Initializer responsible for creating the pipeline payload, which is reference in the Output and made available to steps and sinks
• 1 to n Indexer responsible for identifying parts (or all) of the payload that will be subjected to steps
• 0 to n Step performing transformative operations, ideally without external side effects, their output is indexed in the ResultContainer
• 0 to n Sink performing terminal operations, expected to be external side effects, they can be executed in a synchronous or asynchronous fashion
• 0 to 1 PipelineErrorHandler responsible for handling and possibly recovering from exceptions occurring during the pipeline's execution
• 0 to n OnCloseHandler responsible for cleaning up when tearing down the pipeline

The pipeline references an ExecutorService which the pipeline will attempt to close when its own close() method is called.

It will additionally handle a variety of MeterRegistry counters reflecting the activity of its components.

Author:

Pierre Lecerf ([email protected])

Constructor Summary

Constructors

Constructor

Description

CompositePipeline(String id, UIDGenerator uidGenerator, InitializerDescriptor<I> initializer, AuthorResolver<I> authorResolver, List<Indexer<?>> indexers, OutputFactory<I> outputFactory, List<StepDescriptor<Indexable,I>> steps, List<SinkDescriptor> sinks, Supplier<ExecutorService> sinkExecutorProvider, PipelineErrorHandler errorHandler, int closeTimeout, List<OnCloseHandler> onCloseHandlers, ObservabilityManager observabilityManager, TagResolver<I> tagResolver, List<Observer> observers)

Method Summary

Modifier and Type	Method	Description
void	close()
PipelineDescription	describe()
String	id()	The returned identifier should ideally be unique throughout an application.
Output	run(I input, Context context)	The run method is the Pipeline's sole entrypoint, responsible for implementing its behaviour.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface tech.illuin.pipeline.Pipeline

asStep, asStep, newContext, newContext, run, run, run

Constructor Details

CompositePipeline

public CompositePipeline(String id,
 UIDGenerator uidGenerator,
 InitializerDescriptor<I> initializer,
 AuthorResolver<I> authorResolver,
 List<Indexer<?>> indexers,
 OutputFactory<I> outputFactory,
 List<StepDescriptor<Indexable,I>> steps,
 List<SinkDescriptor> sinks,
 Supplier<ExecutorService> sinkExecutorProvider,
 PipelineErrorHandler errorHandler,
 int closeTimeout,
 List<OnCloseHandler> onCloseHandlers,
 ObservabilityManager observabilityManager,
 TagResolver<I> tagResolver,
 List<Observer> observers)

Method Details

id

public String id()

Description copied from interface: Pipeline

The returned identifier should ideally be unique throughout an application. It is used in log entries and by the PipelineContainer as an identity and indexing key.

Specified by:

id in interface Pipeline<I>

Returns:

the Pipeline's identifier

run

public Output run(I input,
 Context context)
           throws PipelineException

Description copied from interface: Pipeline

The run method is the Pipeline's sole entrypoint, responsible for implementing its behaviour. It returns an Output which is the composition of Result instances produced during the run itself, as well as eventual historical Result retrieved from the Context at initialization.

Specified by:

run in interface Pipeline<I>

Parameters:

input - an input value which can be directly processed by Step or can contribute to the initialization of a payload

context - an optional Context instance providing information related tu current and eventual previous runs

Returns:

the Pipeline's output

Throws:

PipelineException

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Pipeline<I>

Throws:

Exception

describe

public PipelineDescription describe()

Specified by:

describe in interface Pipeline<I>