Class/Object

io.smartdatalake.definitions

SaveModeMergeOptions

Related Docs: object SaveModeMergeOptions | package definitions

Permalink

case class SaveModeMergeOptions(deleteCondition: Option[String] = None, updateCondition: Option[String] = None, updateColumns: Seq[String] = Seq(), insertCondition: Option[String] = None, insertColumnsToIgnore: Seq[String] = Seq(), additionalMergePredicate: Option[String] = None) extends SaveModeOptions with Product with Serializable

Options to control detailed behaviour of SaveMode.Merge. In Spark expressions use table alias 'existing' to reference columns of the existing table data, and table alias 'new' to reference columns of new data set.

deleteCondition

A condition to control if matched records are deleted. If no condition is given, *no* records are delete.

updateCondition

A condition to control if matched records are updated. If no condition is given all matched records are updated (default). Note that delete is applied before update. Records selected for deletion are automatically excluded from the updates.

updateColumns

List of column names to update in update clause. If empty all columns (except primary keys) are updated (default)

insertCondition

A condition to control if unmatched records are inserted. If no condition is given all unmatched records are inserted (default).

insertColumnsToIgnore

List of column names to ignore in insert clause. If empty all columns are inserted (default).

additionalMergePredicate

To optimize performance for SDLSaveMode.Merge it might be interesting to limit the records read from the existing table data, e.g. merge operation might use only the last 7 days.

Linear Supertypes
Serializable, Serializable, Product, Equals, SaveModeOptions, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SaveModeMergeOptions
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. SaveModeOptions
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SaveModeMergeOptions(deleteCondition: Option[String] = None, updateCondition: Option[String] = None, updateColumns: Seq[String] = Seq(), insertCondition: Option[String] = None, insertColumnsToIgnore: Seq[String] = Seq(), additionalMergePredicate: Option[String] = None)

    Permalink

    deleteCondition

    A condition to control if matched records are deleted. If no condition is given, *no* records are delete.

    updateCondition

    A condition to control if matched records are updated. If no condition is given all matched records are updated (default). Note that delete is applied before update. Records selected for deletion are automatically excluded from the updates.

    updateColumns

    List of column names to update in update clause. If empty all columns (except primary keys) are updated (default)

    insertCondition

    A condition to control if unmatched records are inserted. If no condition is given all unmatched records are inserted (default).

    insertColumnsToIgnore

    List of column names to ignore in insert clause. If empty all columns are inserted (default).

    additionalMergePredicate

    To optimize performance for SDLSaveMode.Merge it might be interesting to limit the records read from the existing table data, e.g. merge operation might use only the last 7 days.

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. val additionalMergePredicate: Option[String]

    Permalink

    To optimize performance for SDLSaveMode.Merge it might be interesting to limit the records read from the existing table data, e.g.

    To optimize performance for SDLSaveMode.Merge it might be interesting to limit the records read from the existing table data, e.g. merge operation might use only the last 7 days.

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  7. val deleteCondition: Option[String]

    Permalink

    A condition to control if matched records are deleted.

    A condition to control if matched records are deleted. If no condition is given, *no* records are delete.

  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  11. val insertColumnsToIgnore: Seq[String]

    Permalink

    List of column names to ignore in insert clause.

    List of column names to ignore in insert clause. If empty all columns are inserted (default).

  12. val insertCondition: Option[String]

    Permalink

    A condition to control if unmatched records are inserted.

    A condition to control if unmatched records are inserted. If no condition is given all unmatched records are inserted (default).

  13. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  14. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  15. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  16. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  17. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  18. val updateColumns: Seq[String]

    Permalink

    List of column names to update in update clause.

    List of column names to update in update clause. If empty all columns (except primary keys) are updated (default)

  19. val updateCondition: Option[String]

    Permalink

    A condition to control if matched records are updated.

    A condition to control if matched records are updated. If no condition is given all matched records are updated (default). Note that delete is applied before update. Records selected for deletion are automatically excluded from the updates.

  20. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from SaveModeOptions

Inherited from AnyRef

Inherited from Any

Ungrouped