Class/Object

com.krux.hyperion.activity

CopyActivity

Related Docs: object CopyActivity | package activity

Permalink

case class CopyActivity extends PipelineActivity[Ec2Resource] with Product with Serializable

The activity that copies data from one data node to the other.

Source
CopyActivity.scala
Note

it seems that both input and output format needs to be in CsvDataFormat for this copy to work properly and it needs to be a specific variance of the CSV, for more information check the web page: http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-copyactivity.html From our experience it's really hard to export using TsvDataFormat, in both import and export especially for tasks involving RedshiftCopyActivity. A general rule of thumb is always use default CsvDataFormat for tasks involving both exporting to S3 and copy to redshift.

Linear Supertypes
Serializable, Serializable, Product, Equals, PipelineActivity[Ec2Resource], NamedPipelineObject, PipelineObject, Ordered[PipelineObject], Comparable[PipelineObject], AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. CopyActivity
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. PipelineActivity
  7. NamedPipelineObject
  8. PipelineObject
  9. Ordered
  10. Comparable
  11. AnyRef
  12. Any
  1. Hide All
  2. Show all
Visibility
  1. Public
  2. All

Type Members

  1. type Self = CopyActivity

    Permalink
    Definition Classes
    CopyActivity → PipelineActivity → NamedPipelineObject → PipelineObject

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. def <(that: PipelineObject): Boolean

    Permalink
    Definition Classes
    Ordered

  4. def <=(that: PipelineObject): Boolean

    Permalink
    Definition Classes
    Ordered

  5. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  6. def >(that: PipelineObject): Boolean

    Permalink
    Definition Classes
    Ordered

  7. def >=(that: PipelineObject): Boolean

    Permalink
    Definition Classes
    Ordered

  8. val activityFields: ActivityFields[Ec2Resource]

    Permalink
    Definition Classes
    CopyActivity → PipelineActivity

  9. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  10. def attemptTimeout: Option[HDuration]

    Permalink
    Definition Classes
    PipelineActivity

  11. val baseFields: BaseFields

    Permalink
    Definition Classes
    CopyActivity → NamedPipelineObject

  12. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  13. def compare(that: PipelineObject): Int

    Permalink
    Definition Classes
    PipelineObject → Ordered

  14. def compareTo(that: PipelineObject): Int

    Permalink
    Definition Classes
    Ordered → Comparable

  15. def dependsOn: Seq[PipelineActivity[_]]

    Permalink
    Definition Classes
    PipelineActivity

  16. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  17. def failureAndRerunMode: Option[FailureAndRerunMode]

    Permalink
    Definition Classes
    PipelineActivity

  18. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  19. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  20. def groupedBy(group: String): Self

    Permalink
    Definition Classes
    NamedPipelineObject

  21. def id: PipelineObjectId

    Permalink
    Definition Classes
    NamedPipelineObject → PipelineObject

  22. val input: Copyable

    Permalink

  23. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  24. def lateAfterTimeout: Option[HDuration]

    Permalink
    Definition Classes
    PipelineActivity

  25. def maximumRetries: Option[HInt]

    Permalink
    Definition Classes
    PipelineActivity

  26. def named(name: String): Self

    Permalink
    Definition Classes
    NamedPipelineObject

  27. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  28. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  29. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  30. def objects: Iterable[PipelineObject]

    Permalink
    Definition Classes
    PipelineActivity → PipelineObject

  31. def onFail(alarms: SnsAlarm*): Self

    Permalink
    Definition Classes
    PipelineActivity

  32. def onFailAlarms: Seq[SnsAlarm]

    Permalink
    Definition Classes
    PipelineActivity

  33. def onLateAction(alarms: SnsAlarm*): Self

    Permalink
    Definition Classes
    PipelineActivity

  34. def onLateActionAlarms: Seq[SnsAlarm]

    Permalink
    Definition Classes
    PipelineActivity

  35. def onSuccess(alarms: SnsAlarm*): Self

    Permalink
    Definition Classes
    PipelineActivity

  36. def onSuccessAlarms: Seq[SnsAlarm]

    Permalink
    Definition Classes
    PipelineActivity

  37. val output: Copyable

    Permalink

  38. def preconditions: Seq[Precondition]

    Permalink
    Definition Classes
    PipelineActivity

  39. def ref: AdpRef[AdpActivity]

    Permalink
    Definition Classes
    PipelineActivity → PipelineObject

  40. def retryDelay: Option[HDuration]

    Permalink
    Definition Classes
    PipelineActivity

  41. def runsOn: Resource[Ec2Resource]

    Permalink
    Definition Classes
    PipelineActivity

  42. implicit def seq2Option[A](anySeq: Seq[A]): Option[Seq[A]]

    Permalink
    Definition Classes
    PipelineObject

  43. def seqToOption[A, B](anySeq: Seq[A])(transform: (A) ⇒ B): Option[Seq[B]]

    Permalink
    Definition Classes
    PipelineObject

  44. lazy val serialize: AdpCopyActivity

    Permalink
    Definition Classes
    CopyActivity → PipelineActivity → PipelineObject

  45. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  46. implicit def uniquePipelineId2String(id: PipelineObjectId): String

    Permalink
    Definition Classes
    PipelineObject

  47. def updateActivityFields(fields: ActivityFields[Ec2Resource]): CopyActivity

    Permalink
    Definition Classes
    CopyActivity → PipelineActivity

  48. def updateBaseFields(fields: BaseFields): CopyActivity

    Permalink
    Definition Classes
    CopyActivity → NamedPipelineObject

  49. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  50. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  51. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  52. def whenMet(conditions: Precondition*): Self

    Permalink
    Definition Classes
    PipelineActivity

  53. def withAttemptTimeout(duration: HDuration): Self

    Permalink
    Definition Classes
    PipelineActivity

  54. def withFailureAndRerunMode(mode: FailureAndRerunMode): Self

    Permalink
    Definition Classes
    PipelineActivity

  55. def withLateAfterTimeout(duration: HDuration): Self

    Permalink
    Definition Classes
    PipelineActivity

  56. def withMaximumRetries(retries: HInt): Self

    Permalink
    Definition Classes
    PipelineActivity

  57. def withRetryDelay(duration: HDuration): Self

    Permalink
    Definition Classes
    PipelineActivity

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from PipelineActivity[Ec2Resource]

Inherited from NamedPipelineObject

Inherited from PipelineObject

Inherited from Ordered[PipelineObject]

Inherited from Comparable[PipelineObject]

Inherited from AnyRef

Inherited from Any

Ungrouped