api

Type Members

1. trait ExternalChannel extends AnyRef

   

   This is a marker trait tha allows a unified way of creating new instances of sources and sinks from a fully qualified class name.

2. trait MetastoreReader extends AnyRef

   

   Metastore reader allows querying tables registered at the 'metastore' section of the configuration.

   Metastore reader allows querying tables registered at the 'metastore' section of the configuration. It abstracts away the storage provider (HDFS, S3, etc), format (Parquet, Delta, etc.) and partitioning options.

3. sealed trait Query extends AnyRef

   

   Query is an abstraction that unifies the way data can be retrieved from a source.

4. sealed trait Reason extends AnyRef

   

   Reasons a Pramen job has failed.

5. trait Sink extends ExternalChannel with AutoCloseable

   

   A sink is an entity where jobs can write to and it won't be under metastore control (e.g.

   A sink is an entity where jobs can write to and it won't be under metastore control (e.g. Kafka).

   Special Sink jobs are used to write tables from metastore to a sink.

6. trait Source extends ExternalChannel

   

7. trait TableReader extends AnyRef

   

8. trait Transformer extends AnyRef