Value Members

1.  final def !=(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

2.  final def ##(): Int

   Definition Classes

   AnyRef → Any

3.  final def ==(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

4.  val Append: Value
   See also

   SaveMode

5.  val ErrorIfExists: Value
   See also

   SaveMode

6.  val Ignore: Value
   See also

   SaveMode

7.  val Merge: Value

   Merge new data with existing data by insert new records and update (or delete) existing records.

   Merge new data with existing data by insert new records and update (or delete) existing records. DataObjects need primary key defined to check if a record is new. To delete existing records add column '_deleted' to DataFrame and set its value to 'true' for the records which should be deleted.

   Note that only few DataObjects are able to merge new data, e.g. DeltaLakeTableDataObject and JdbcTableDataObject

8.  val Overwrite: Value
   See also

   SaveMode

9.  val OverwriteOptimized: Value

   This is like SDLSaveMode.Overwrite but processed partitions are manually deleted instead of using dynamic partitioning mode.

   This is like SDLSaveMode.Overwrite but processed partitions are manually deleted instead of using dynamic partitioning mode. Then it uses Sparks append mode to add the new partitions. This helps if there are performance problems when using dynamic partitioning mode with hive tables and many partitions.

   Implementation: This save mode will delete processed partition directories manually. If no partition values are present when writing to a partitioned data object, all partitions are deleted. This is different to Sparks dynamic partitioning, which only deletes partitions where data is present in the DataFrame to be written (enabled by default in SDL). To stop if no partition values are present, configure executionMode.type = FailIfNoPartitionValuesMode on the Action.

10.  val OverwritePreserveDirectories: Value

    This is like SDLSaveMode.Overwrite but doesnt delete the directory of the DataObject and its partition, but only the files inside.

    This is like SDLSaveMode.Overwrite but doesnt delete the directory of the DataObject and its partition, but only the files inside. Then it uses Sparks append mode to add the new files. Like that ACLs set on the base directory are preserved.

    Implementation: This save mode will delete all files inside the base directory, but not the directory itself. If no partition values are present when writing to a partitioned data object, all files in all partitions are deleted, but not the partition directories itself. This is different to Sparks dynamic partitioning, which only deletes partitions where data is present in the DataFrame to be written (enabled by default in SDL). To stop if no partition values are present, configure executionMode.type = FailIfNoPartitionValuesMode on the Action.

11.  final def Value(i: Int, name: String): Value

    Attributes

    protected

    Definition Classes

    Enumeration

12.  final def Value(name: String): Value

    Attributes

    protected

    Definition Classes

    Enumeration

13.  final def Value(i: Int): Value

    Attributes

    protected

    Definition Classes

    Enumeration

14.  final def Value: Value

    Attributes

    protected

    Definition Classes

    Enumeration

15.  final def apply(x: Int): Value

    Definition Classes

    Enumeration

16.  final def asInstanceOf[T0]: T0

    Definition Classes

    Any

17.  def clone(): AnyRef

    Attributes

    protected[lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( ... ) @native()

18.  final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

19.  def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

20.  def finalize(): Unit

    Attributes

    protected[lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

21.  final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

22.  def hashCode(): Int

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

23.  final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

24.  final def maxId: Int

    Definition Classes

    Enumeration

25.  final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

26.  var nextId: Int

    Attributes

    protected

    Definition Classes

    Enumeration

27.  var nextName: Iterator[String]

    Attributes

    protected

    Definition Classes

    Enumeration

28.  final def notify(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

29.  final def notifyAll(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

30.  def readResolve(): AnyRef

    Attributes

    protected

    Definition Classes

    Enumeration

31.  final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

32.  def toString(): String

    Definition Classes

    Enumeration → AnyRef → Any

33.  implicit def value2SparkSaveMode(mode: Value): SDLSaveModeValue
34.  def values: ValueSet

    Definition Classes

    Enumeration

35.  final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

36.  final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

37.  final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... ) @native()

38.  final def withName(s: String): Value

    Definition Classes

    Enumeration