Operators for the DSL
Contains operators which can be chained to single one.
When translated to running DAG, SlaveOP can be attach to MasterOP or other SlaveOP "Attach" means running in same Actor.
The upstream OP and downstream OP doesn't require network data shuffle.
The upstream OP and downstream OP doesn't require network data shuffle.
For example, map, flatmap operation doesn't require network shuffle, we can use Direct to represent the relation with upstream operators.
The upstream OP and downstream OP DOES require network data shuffle.
The upstream OP and downstream OP DOES require network data shuffle.
For example, map, flatmap operation doesn't require network shuffle, we can use Direct to represent the relation with upstream operators.
Contains operators which can be chained to single one.
For example, flatmap().map().reduce() can be chained to single operator as no data shuffling is required.
list of operations