com.twitter.scalding.typed

HashJoinable

trait HashJoinable[K, +V] extends CoGroupable[K, V] with KeyedPipe[K]

If we can HashJoin, then we can CoGroup, but not vice-versa i.e., HashJoinable is a strict subset of CoGroupable (CoGrouped, for instance is CoGroupable, but not HashJoinable).

Linear Supertypes
KeyedPipe[K], CoGroupable[K, V], Serializable, HasDescription, HasReducers, AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. HashJoinable
  2. KeyedPipe
  3. CoGroupable
  4. Serializable
  5. HasDescription
  6. HasReducers
  7. AnyRef
  8. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def descriptions: Seq[String]

    Definition Classes
    HasDescription
  2. abstract def joinFunction: (K, Iterator[Tuple], Seq[Iterable[Tuple]]) ⇒ Iterator[V]

    This function is not type-safe for others to call, but it should never have an error.

    This function is not type-safe for others to call, but it should never have an error. By construction, we never call it with incorrect types. It would be preferable to have stronger type safety here, but unclear how to achieve, and since it is an internal function, not clear it would actually help anyone for it to be type-safe

    Attributes
    protected
    Definition Classes
    CoGroupable
  3. abstract def keyOrdering: Ordering[K]

    Definition Classes
    KeyedPipe
  4. abstract def mapped: TypedPipe[(K, Any)]

    Definition Classes
    KeyedPipe
  5. abstract def reducers: Option[Int]

    Definition Classes
    HasReducers

Concrete Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  7. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  8. def cogroup[R1, R2](smaller: CoGroupable[K, R1])(fn: (K, Iterator[V], Iterable[R1]) ⇒ Iterator[R2]): CoGrouped[K, R2]

    Smaller is about average values/key not total size (that does not matter, but is clearly related).

    Smaller is about average values/key not total size (that does not matter, but is clearly related).

    Note that from the type signature we see that the right side is iterated (or may be) over and over, but the left side is not. That means that you want the side with fewer values per key on the right. If both sides are similar, no need to worry. If one side is a one-to-one mapping, that should be the "smaller" side.

    Definition Classes
    CoGroupable
  9. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  11. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  13. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  14. def hashCogroupOn[V1, R](mapside: TypedPipe[(K, V1)])(joiner: (K, V1, Iterable[V]) ⇒ Iterator[R]): TypedPipe[(K, R)]

    This fully replicates this entire Grouped to the argument: mapside.

    This fully replicates this entire Grouped to the argument: mapside. This means that we never see the case where the key is absent in the pipe. This means implementing a right-join (from the pipe) is impossible. Note, there is no reduce-phase in this operation. The next issue is that obviously, unlike a cogroup, for a fixed key, each joiner will NOT See all the tuples with those keys. This is because the keys on the left are distributed across many machines See hashjoin: http://docs.cascading.org/cascading/2.0/javadoc/cascading/pipe/HashJoin.html

  15. def inputs: List[TypedPipe[(K, Any)]]

    A HashJoinable has a single input into to the cogroup

    A HashJoinable has a single input into to the cogroup

    Definition Classes
    HashJoinableCoGroupable
  16. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  17. def join[W](smaller: CoGroupable[K, W]): CoGrouped[K, (V, W)]

    Definition Classes
    CoGroupable
  18. def leftJoin[W](smaller: CoGroupable[K, W]): CoGrouped[K, (V, Option[W])]

    Definition Classes
    CoGroupable
  19. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  20. final def notify(): Unit

    Definition Classes
    AnyRef
  21. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  22. def outerJoin[W](smaller: CoGroupable[K, W]): CoGrouped[K, (Option[V], Option[W])]

    Definition Classes
    CoGroupable
  23. def rightJoin[W](smaller: CoGroupable[K, W]): CoGrouped[K, (Option[V], W)]

    Definition Classes
    CoGroupable
  24. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  25. def toString(): String

    Definition Classes
    AnyRef → Any
  26. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from KeyedPipe[K]

Inherited from CoGroupable[K, V]

Inherited from Serializable

Inherited from HasDescription

Inherited from HasReducers

Inherited from AnyRef

Inherited from Any

Ungrouped