org.apache.gearpump.streaming.dsl

op

package op

Visibility
  1. Public
  2. All

Type Members

  1. case class DataSinkOp[T](dataSink: TypedDataSink[T], parallelism: Int, description: String)(implicit evidence$6: ClassTag[T]) extends ParameterizedOp[T] with Product with Serializable

  2. case class DataSourceOp[T](dataSource: TypedDataSource[T], parallelism: Int, description: String)(implicit evidence$5: ClassTag[T]) extends ParameterizedOp[T] with Product with Serializable

  3. case class FlatMapOp[T, R](fun: (T) ⇒ TraversableOnce[R], description: String)(implicit evidence$1: ClassTag[T]) extends SlaveOp[T] with Product with Serializable

  4. case class GroupByOp[T, R](fun: (T) ⇒ R, parallism: Int, description: String)(implicit evidence$3: ClassTag[T]) extends ParameterizedOp[T] with Product with Serializable

  5. trait MasterOp extends Op

  6. case class MergeOp(source: Op, target: Op, description: String) extends MasterOp with Product with Serializable

  7. sealed trait Op extends AnyRef

    Operators for the DSL

  8. case class OpChain(ops: List[Op]) extends Op with Product with Serializable

    Contains operators which can be chained to single one.

    Contains operators which can be chained to single one.

    For example, flatmap().map().reduce() can be chained to single operator as no data shuffling is required.

    ops

    list of operations

  9. trait OpEdge extends AnyRef

  10. trait ParameterizedOp[T] extends MasterOp

  11. case class ProcessorOp[T <: Task](processor: Class[T], parallism: Int, description: String)(implicit evidence$4: ClassTag[T]) extends ParameterizedOp[T] with Product with Serializable

  12. case class ReduceOp[T](fun: (T, T) ⇒ T, description: String)(implicit evidence$2: ClassTag[T]) extends SlaveOp[T] with Product with Serializable

  13. trait SlaveOp[T] extends Op

    When translated to running DAG, SlaveOP can be attach to MasterOP or other SlaveOP "Attach" means running in same Actor.

Value Members

  1. object Direct extends OpEdge with Product with Serializable

    The upstream OP and downstream OP doesn't require network data shuffle.

    The upstream OP and downstream OP doesn't require network data shuffle.

    For example, map, flatmap operation doesn't require network shuffle, we can use Direct to represent the relation with upstream operators.

  2. object Shuffle extends OpEdge with Product with Serializable

    The upstream OP and downstream OP DOES require network data shuffle.

    The upstream OP and downstream OP DOES require network data shuffle.

    For example, map, flatmap operation doesn't require network shuffle, we can use Direct to represent the relation with upstream operators.

Ungrouped