typed

Type Members

1. class BijectedSourceSink[T, U] extends TypedSource[U] with TypedSink[U]

2. trait CoGroupable[K, +R] extends HasReducers with HasDescription with Serializable

   Represents something than can be CoGrouped with another CoGroupable

3. trait CoGrouped[K, +R] extends KeyedListLike[K, R, CoGrouped] with CoGroupable[K, R] with WithReducers[CoGrouped[K, R]] with WithDescription[CoGrouped[K, R]]

4. abstract class CoGroupedJoiner[K] extends Joiner

5. case class ComputedValue[T](toTypedPipe: TypedPipe[T]) extends ValuePipe[T] with Product with Serializable

6. case class Converter[R](conv: TupleConverter[R]) extends FlatMapFn[R] with Product with Serializable

7. class DistinctCoGroupJoiner[K] extends CoGroupedJoiner[K]

8. case class FilteredFn[R](fmap: FlatMapFn[R], fn: (R) ⇒ Boolean) extends FlatMapFn[R] with Product with Serializable

9. sealed trait FlatMapFn[+R] extends (TupleEntry) ⇒ TraversableOnce[R] with Serializable

   Closures are difficult for serialization.

10. case class FlatMappedFn[T, R](fmap: FlatMapFn[T], fn: (T) ⇒ TraversableOnce[R]) extends FlatMapFn[R] with Product with Serializable

11. trait Grouped[K, +V] extends KeyedListLike[K, V, UnsortedGrouped] with HashJoinable[K, V] with Sortable[V, [+x]SortedGrouped[K, x] with Reversable[SortedGrouped[K, x]]] with WithReducers[Grouped[K, V]] with WithDescription[Grouped[K, V]]

    This encodes the rules that 1) sorting is only possible before doing any reduce, 2) reversing is only possible after sorting.

12. trait HasDescription extends AnyRef

    Used for objects that may have a description set to be used in .

13. trait HasReducers extends AnyRef

    used for types that may know how many reducers they need e.

14. sealed trait HashEqualsArrayWrapper[T] extends AnyRef

15. final class HashEqualsBooleanArrayWrapper extends HashEqualsArrayWrapper[Boolean]

16. final class HashEqualsByteArrayWrapper extends HashEqualsArrayWrapper[Byte]

17. final class HashEqualsCharArrayWrapper extends HashEqualsArrayWrapper[Char]

18. final class HashEqualsDoubleArrayWrapper extends HashEqualsArrayWrapper[Double]

19. final class HashEqualsFloatArrayWrapper extends HashEqualsArrayWrapper[Float]

20. final class HashEqualsIntArrayWrapper extends HashEqualsArrayWrapper[Int]

21. final class HashEqualsLongArrayWrapper extends HashEqualsArrayWrapper[Long]

22. final class HashEqualsObjectArrayWrapper[T] extends HashEqualsArrayWrapper[T]

23. final class HashEqualsShortArrayWrapper extends HashEqualsArrayWrapper[Short]

24. trait HashJoinable[K, +V] extends CoGroupable[K, V] with KeyedPipe[K]

    If we can HashJoin, then we can CoGroup, but not vice-versa i.

25. class HashJoiner[K, V, W, R] extends Joiner

    Only intended to be use to implement the hashCogroup on TypedPipe/Grouped

26. case class IdentityReduce[K, V1](keyOrdering: Ordering[K], mapped: TypedPipe[(K, V1)], reducers: Option[Int], descriptions: Seq[String]) extends ReduceStep[K, V1] with Grouped[K, V1] with Product with Serializable

27. case class IdentityValueSortedReduce[K, V1](keyOrdering: Ordering[K], mapped: TypedPipe[(K, V1)], valueSort: Ordering[_ >: V1], reducers: Option[Int], descriptions: Seq[String]) extends ReduceStep[K, V1] with SortedGrouped[K, V1] with Reversable[IdentityValueSortedReduce[K, V1]] with Product with Serializable

28. final case class IterablePipe[T](iterable: Iterable[T]) extends TypedPipe[T] with Product with Serializable

    Creates a TypedPipe from an Iterable[T].

29. case class IteratorMappedReduce[K, V1, V2](keyOrdering: Ordering[K], mapped: TypedPipe[(K, V1)], reduceFn: (K, Iterator[V1]) ⇒ Iterator[V2], reducers: Option[Int], descriptions: Seq[String]) extends ReduceStep[K, V1] with UnsortedGrouped[K, V2] with Product with Serializable

30. trait KeyedList[K, +T] extends KeyedListLike[K, T, KeyedList]

    This is for the case where you don't want to expose any structure but the ability to operate on an iterator of the values

31. trait KeyedListLike[K, +T, +This[K, +T] <: KeyedListLike[K, T, This]] extends Serializable

    Represents sharded lists of items of type T There are exactly two fundamental operations: toTypedPipe: marks the end of the grouped-on-key operations.

32. trait KeyedPipe[K] extends AnyRef

    Represents anything that starts as a TypedPipe of Key Value, where the value type has been erased.

33. case class LiteralValue[T](value: T) extends ValuePipe[T] with Product with Serializable

34. case class MapFn[T, R](fmap: FlatMapFn[T], fn: (T) ⇒ R) extends FlatMapFn[R] with Product with Serializable

35. class MappablePipeJoinEnrichment[T] extends AnyRef

    This class is for the syntax enrichment enabling .

36. class MemorySink[T] extends TypedSink[T]

37. final case class MergedTypedPipe[T](left: TypedPipe[T], right: TypedPipe[T]) extends TypedPipe[T] with Product with Serializable

38. trait MustHaveReducers extends HasReducers

    used for types that must know how many reducers they need e.

39. sealed trait NoStackAndThen[-A, +B] extends Serializable

    This type is used to implement .

40. trait PartitionSchemed[P, T] extends SchemedSource with TypedSink[(P, T)] with Mappable[(P, T)] with HfsTapProvider

    Trait to assist with creating partitioned sources.

41. trait PartitionedDelimited extends Serializable

    Trait to assist with creating objects such as PartitionedTsv to read from separated files.

42. case class PartitionedDelimitedSource[P, T](path: String, template: String, separator: String, fields: Fields, skipHeader: Boolean = false, writeHeader: Boolean = false, quote: String = "\"", strict: Boolean = true, safe: Boolean = true)(implicit mt: Manifest[T], valueSetter: TupleSetter[T], valueConverter: TupleConverter[T], partitionSetter: TupleSetter[P], partitionConverter: TupleConverter[P]) extends SchemedSource with PartitionSchemed[P, T] with Serializable with Product with Serializable

    Scalding source to read or write partitioned delimited text.

43. case class PartitionedTextLine[P](path: String, template: String, encoding: String = ...)(implicit valueSetter: TupleSetter[String], valueConverter: TupleConverter[(Long, String)], partitionSetter: TupleSetter[P], partitionConverter: TupleConverter[P]) extends SchemedSource with TypedSink[(P, String)] with Mappable[(P, (Long, String))] with HfsTapProvider with Serializable with Product with Serializable

    Scalding source to read or write partitioned text.

44. class PipeTExtensions extends Serializable

45. sealed trait ReduceStep[K, V1] extends KeyedPipe[K]

    This is a class that models the logical portion of the reduce step.

46. trait Reversable[+R] extends AnyRef

47. case class SketchJoined[K, V, V2, R](left: Sketched[K, V], right: TypedPipe[(K, V2)], numReducers: Int)(joiner: (K, V, Iterable[V2]) ⇒ Iterator[R])(implicit evidence$1: Ordering[K]) extends MustHaveReducers with Product with Serializable

48. case class Sketched[K, V](pipe: TypedPipe[(K, V)], numReducers: Int, delta: Double, eps: Double, seed: Int)(implicit serialization: (K) ⇒ Array[Byte], ordering: Ordering[K]) extends MustHaveReducers with Product with Serializable

    This class is generally only created by users with the TypedPipe.

49. trait Sortable[+T, +Sorted[+_]] extends AnyRef

    All sorting methods defined here trigger Hadoop secondary sort on key + value.

50. trait SortedGrouped[K, +V] extends KeyedListLike[K, V, SortedGrouped] with WithReducers[SortedGrouped[K, V]] with WithDescription[SortedGrouped[K, V]]

    After sorting, we are no longer CoGroupable, and we can only call reverse in the initial SortedGrouped created from the Sortable: .

51. case class TemplatePartition(partitionFields: Fields, template: String) extends Partition with Product with Serializable

    Creates a partition using the given template string.

52. trait TypedPipe[+T] extends Serializable

    Think of a TypedPipe as a distributed unordered list that may or may not yet have been materialized in memory or disk.

53. class TypedPipeFactory[T] extends TypedPipe[T]

    This is a TypedPipe that delays having access to the FlowDef and Mode until toPipe is called

54. class TypedPipeInst[T] extends TypedPipe[T]

    This is an instance of a TypedPipe that wraps a cascading Pipe

55. trait TypedSink[-T] extends Serializable

    Opposite of TypedSource, used for writing into

56. trait TypedSource[+T] extends Serializable

57. trait UnsortedGrouped[K, +V] extends KeyedListLike[K, V, UnsortedGrouped] with HashJoinable[K, V] with WithReducers[UnsortedGrouped[K, V]] with WithDescription[UnsortedGrouped[K, V]]

    This is the state after we have done some reducing.

58. case class UnsortedIdentityReduce[K, V1](keyOrdering: Ordering[K], mapped: TypedPipe[(K, V1)], reducers: Option[Int], descriptions: Seq[String]) extends ReduceStep[K, V1] with UnsortedGrouped[K, V1] with Product with Serializable

59. sealed trait ValuePipe[+T] extends Serializable

    ValuePipe is special case of a TypedPipe of just a optional single element.

60. case class ValueSortedReduce[K, V1, V2](keyOrdering: Ordering[K], mapped: TypedPipe[(K, V1)], valueSort: Ordering[_ >: V1], reduceFn: (K, Iterator[V1]) ⇒ Iterator[V2], reducers: Option[Int], descriptions: Seq[String]) extends ReduceStep[K, V1] with SortedGrouped[K, V2] with Product with Serializable

61. trait WithDescription[+This <: WithDescription[This]] extends HasDescription

    Used for objects that may _set_ a description to be used in .

62. case class WithDescriptionTypedPipe[T](typedPipe: TypedPipe[T], description: String) extends TypedPipe[T] with Product with Serializable

63. case class WithOnComplete[T](typedPipe: TypedPipe[T], fn: () ⇒ Unit) extends TypedPipe[T] with Product with Serializable

64. trait WithReducers[+This <: WithReducers[This]] extends HasReducers

    used for objects that may _set_ how many reducers they need e.