Class

it.agilelab.bigdata.wasp.models

CdcModel

Related Doc: package models

Permalink

case class CdcModel(name: String, uri: String, schema: String, options: CdcOptions = CdcOptions.default) extends DatastoreModel with Product with Serializable

A named model for mutations coming from a CDC tool. This model should be used together with the Cdc writer plugin in order to write these mutations into a Delta Lake table on HDFS.

uri is the location on HDFS where the Delta Table will be created.

schema is a json-encoded DataFrame schema, that is, a StructType. See DataType.fromJson and DataType.json.

options control the underlying spark DeltaLakeWriter in the writers using an instance of this model.

name

the name of the datastore

uri

the uri where the data are meant to be written

schema

the schema of the data

options

the options for the datastore

Linear Supertypes
Serializable, Serializable, Product, Equals, DatastoreModel, Model, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CdcModel
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. DatastoreModel
  7. Model
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CdcModel(name: String, uri: String, schema: String, options: CdcOptions = CdcOptions.default)

    Permalink

    name

    the name of the datastore

    uri

    the uri where the data are meant to be written

    schema

    the schema of the data

    options

    the options for the datastore

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @HotSpotIntrinsicCandidate() @throws( ... )
  6. def datastoreProduct: DatastoreProduct

    Permalink
    Definition Classes
    CdcModelDatastoreModel
  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  8. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
    Annotations
    @HotSpotIntrinsicCandidate()
  9. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  10. val name: String

    Permalink

    the name of the datastore

    the name of the datastore

    Definition Classes
    CdcModelModel
  11. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  12. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @HotSpotIntrinsicCandidate()
  13. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @HotSpotIntrinsicCandidate()
  14. val options: CdcOptions

    Permalink

    the options for the datastore

  15. val schema: String

    Permalink

    the schema of the data

  16. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  17. val uri: String

    Permalink

    the uri where the data are meant to be written

  18. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  20. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @Deprecated @deprecated @throws( classOf[java.lang.Throwable] )
    Deprecated

    (Since version ) see corresponding Javadoc for more information.

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from DatastoreModel

Inherited from Model

Inherited from AnyRef

Inherited from Any

Ungrouped