Package com.streamsets.pipeline.api
package com.streamsets.pipeline.api
Data Collector API
-
ClassDescriptionSuggestion on remedy of an issue from Antenna Doctor.Context for a batch, primarily used in PushSource to start and finish execution of a batch.General store for various data used by SDC and its various components.Sub-interface to encapsulate tuple of content with it's version.
ChooserValuesprovide a list of values and localizable labels for stage configurations using one of the following models:ValueChooserModelorMultiValueChooserModel.A composed config is an artifact for the UI to show, in read-only form, multiple configs as a single one.AConfigbean holds a configuration (both name and value) of a stage.TheConfigDefannotation is used to define stage configuration variables and their visual representation in the UI.Display mode for the configuration when authoring pipelines.Enum defining the possible EL evaluation modes for configuration variables.Enum defining the possible UI syntax highlighting for a configuration of typeConfigDef.Type.TEXT.Enum defining the possible types of the configuration variable.Defines the upload capability for a configuration.Used to define a configuration bean for stages.Annotation to define the configuration groups for a stage.Standalone class representing configuration issue.Interface for creatingConfigIssueinstances.Interface to retrieve typed configuration from configuration files.TheConnectionDefannotation is used to designate Config Beans as being Connection Definitions too.ChooserValuesfor theConfigDef.connectionType()field.The possible execution engines that a Connection can support.Allows Connections to verify their configuration.Delivery guarantee of a pipeline.A Dependency specifies a configuration this config depends on.Defines an EL constant.Classes annotated withElDefare automatically scanned for EL constants and EL function definitions and they are made available to all configurations of the stage library.Defines an EL function.Methods defined as EL functions must use this annotation to give the UI name to the EL function parameters.Typifies error codes with built-in localization support.If the execution of a pipeline throws an exception, all stages in the pipeline implementing this interface will be notified via theErrorListener.errorNotification(Throwable)method.Used to define destination (Target) stages that specialize in handling records sent to error.Event record is a special Record instance that have few additional required header fields like type, version and creation_time.Defines the execution modes of a pipeline.AExecutoris a special case of a Data collector target.Defines how explorer will help the user to set the value for an explorer aware property.Defines the type of explorer binding the configuration has.Location defines where Explorer UI should place, in a configuration, an explorer chosen value.ExplorerData<T>ExplorerLoaderproducesExplorerDatathat is used byExplorerSearcher.Loads explorer data to be used for Explorer searches.If a stage provides a loader implementing this loader sub-interface, it is possible to use the engine's built-inExplorerSearcherby indicating in the stage's@StageDefexplorerSearchType = ExplorerSearchType.MANAGED_TABULAR.Class that defines the metadata explorer schema for a stage.A leaf schema element.Base schema element class.Empty schema.A schema element that has one or more children schema element.Interface that performs an explorer search against the explorer data produced by the explorer loader.Defines an explorer search request and how to produce the result tree.Searches always return a tree.AFieldis the basic building block to represent data within Data Collector stages.Enum defining allFieldtypes.Field batch is used by to transform individual fields rather than whole records.Field operator is used by to manipulate fields while avoiding object creation for performance.Field processor is a special kind of processor.Annotation for configuration variables that need to be populated with record field names.Field Visitor interface that enables simple iteration over fields.Class which defines a reference to a file through which streams can be created to the file.Annotation to flag the generation of resource bundles for stages and related classes that have labels or messages that show up in the UI and need to be localized.Annotation to hide stage configurations in the UI.Annotation to mark stage that should not be displayed on the pipeline canvas.Type of the hidden stage declaring where else the stage should be used in StreamSets infrastructure.Annotation to inform users of a package, class or method's intended audience.Intended only for professional services consultants.Intended for use only within SDC itself.Intended for use by any project or application.Annotation to inform users of how much to rely on a particular package, class or method not changing over time.Evolving, but can break compatibility at minor release (i.e.Can evolve while retaining compatibility for minor release boundaries.; can break compatibility only at major release (ie.No guarantee is provided as to reliability or stability across any level of release granularity.Exception thrown a field path cannot be parsed.Interface used by stage enums used in the configuration to provide the default UI label for the enums.Annotation for configuration variables that can hold a list of a configuration bean.Annotation for configuration variables that can hold multiple values from a given list of values.Sources that keep track of the data offset on their own should implement this interface.Targets that trigger offset commit should implement this interface.Enumeration defining the error record handling for stages via the built-in 'on record error' configuration.Annotation for configuration variables that define predicate/lane(output-streams) pairs.Describes methods for various context classes that deals with error handling.AProcessoris a Data Collector processor stage.Processorstage context.Shared interface for all active configurable objects - such as Stage or Service.Context that provides runtime information and services to the stage.Context to create and use Java Expression Language (EL) evaluators.ProtoSource<C extends Stage.Context>AProtoSourceis base for Data Collector origin stage.Sourcestage context.PushSourcestage context.Annotation for Data Collector origin (Source) stages that provide a way to view the raw data they produce during preview mode.ARecordrepresents the unit of data Data Collector pipelines process.Metadata of the record.Record Field is a wrapper on top of field describing where the field is located (e.g.ASourceis Data Collector origin stage.Sourcestage context.A SourceResponseSink is used to pass responses from destinations or processors back to the origin about the batch that was processed.Stage<C extends Stage.Context>Internal Base interface for Data Collector stages implementations defining their common context and lifecycle.Interface for configuration issues.Stage Context that provides runtime information and services to the stage.It provides information about the stage.Context to get information about user that started the pipeline or job.Various flags describing how given stage behaves that Data Collector framework can use when executing the flags.Annotation to declare Data Collector stages classes.Java doesn't allow null to be used in annotations, so this DefaultInputStream serves as a constant replacement for null - it's an empty enumeration and corresponds to situation when numberOfInputStreams() returns -1.Default enumeration used by stages having a single output stream.The type of explorer search the stage has.Enumeration to use inStageDef.outputStreams()to indicate that the output streams are variable, driven by the stage configuration specified inStageDef.outputStreamsDrivenByConfig().Exception thrown by stages when there is an error while processing.TheStageUpgraderallows stages to upgrade their configuration from previous versions of the stage.Upgrade context with various upgrade metadata.DefaultStageUpgraderimplementation.Error codes used by the upgrader.Upgrade context with external upgrade metadata.Used to define destination (Target) stages that accept aggregated pipeline statistics, i.e., pipeline metrics, data rule evaluation results and drift rule evaluation results.ATargetis a Data Collector destination stage.Targetstage context.Describes methods for various context classes that deals with error handling.Describes methods for various context classes that deals with error handling.Describes methods for various context classes that deals with source response handling.Annotation for configuration variables that can hold a single value from a given list of values.