spark

UnionRDD

class UnionRDD[T] extends RDD[T] with Serializable

Linear Supertypes
RDD[T], Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Hide All
  2. Show all
  1. UnionRDD
  2. RDD
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
Visibility
  1. Public
  2. All

Instance Constructors

  1. new UnionRDD(sc: SparkContext, rdds: Seq[RDD[T]])(implicit arg0: ClassManifest[T])

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. def ++(other: RDD[T]): RDD[T]

    Definition Classes
    RDD
  5. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  6. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  7. def aggregate[U](zeroValue: U)(seqOp: (U, T) ⇒ U, combOp: (U, U) ⇒ U)(implicit arg0: ClassManifest[U]): U

    Aggregate the elements of each partition, and then the results for all the partitions, using given combine functions and a neutral "zero value".

    Aggregate the elements of each partition, and then the results for all the partitions, using given combine functions and a neutral "zero value". This function can return a different result type, U, than the type of this RDD, T. Thus, we need one operation for merging a T into an U and one operation for merging two U's, as in scala.TraversableOnce. Both of these functions are allowed to modify and return their first argument instead of creating a new U to avoid memory allocation.

    Definition Classes
    RDD
  8. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  9. def cache(): RDD[T]

    Definition Classes
    RDD
  10. def cartesian[U](other: RDD[U])(implicit arg0: ClassManifest[U]): RDD[(T, U)]

    Definition Classes
    RDD
  11. def clone(): AnyRef

    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  12. def collect(): Array[T]

    Definition Classes
    RDD
  13. def compute(s: Split): Iterator[T]

    Definition Classes
    UnionRDDRDD
  14. def context: SparkContext

    Definition Classes
    RDD
  15. def count(): Long

    Definition Classes
    RDD
  16. val dependencies: List[spark.Dependency[_]]

    Definition Classes
    UnionRDDRDD
  17. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  18. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  19. def filter(f: (T) ⇒ Boolean): RDD[T]

    Definition Classes
    RDD
  20. def finalize(): Unit

    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  21. def first(): T

    Definition Classes
    RDD
  22. def flatMap[U](f: (T) ⇒ TraversableOnce[U])(implicit arg0: ClassManifest[U]): RDD[U]

    Definition Classes
    RDD
  23. def fold(zeroValue: T)(op: (T, T) ⇒ T): T

    Aggregate the elements of each partition, and then the results for all the partitions, using a given associative function and a neutral "zero value".

    Aggregate the elements of each partition, and then the results for all the partitions, using a given associative function and a neutral "zero value". The function op(t1, t2) is allowed to modify t1 and return it as its result value to avoid object allocation; however, it should not modify t2.

    Definition Classes
    RDD
  24. def foreach(f: (T) ⇒ Unit): Unit

    Definition Classes
    RDD
  25. final def getClass(): java.lang.Class[_]

    Definition Classes
    AnyRef → Any
  26. def glom(): RDD[Array[T]]

    Definition Classes
    RDD
  27. def groupBy[K](f: (T) ⇒ K)(implicit arg0: ClassManifest[K]): RDD[(K, Seq[T])]

    Definition Classes
    RDD
  28. def groupBy[K](f: (T) ⇒ K, numSplits: Int)(implicit arg0: ClassManifest[K]): RDD[(K, Seq[T])]

    Definition Classes
    RDD
  29. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  30. val id: Int

    Definition Classes
    RDD
  31. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  32. final def iterator(split: Split): Iterator[T]

    Definition Classes
    RDD
  33. def map[U](f: (T) ⇒ U)(implicit arg0: ClassManifest[U]): RDD[U]

    Definition Classes
    RDD
  34. def mapPartitions[U](f: (Iterator[T]) ⇒ Iterator[U])(implicit arg0: ClassManifest[U]): RDD[U]

    Definition Classes
    RDD
  35. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  36. final def notify(): Unit

    Definition Classes
    AnyRef
  37. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  38. val partitioner: Option[Partitioner]

    Definition Classes
    RDD
  39. def pipe(command: Seq[String], env: Map[String, String]): RDD[String]

    Definition Classes
    RDD
  40. def pipe(command: Seq[String]): RDD[String]

    Definition Classes
    RDD
  41. def pipe(command: String): RDD[String]

    Definition Classes
    RDD
  42. def preferredLocations(s: Split): Seq[String]

    Definition Classes
    UnionRDDRDD
  43. def reduce(f: (T, T) ⇒ T): T

    Definition Classes
    RDD
  44. def sample(withReplacement: Boolean, fraction: Double, seed: Int): RDD[T]

    Definition Classes
    RDD
  45. def saveAsObjectFile(path: String): Unit

    Definition Classes
    RDD
  46. def saveAsTextFile(path: String): Unit

    Definition Classes
    RDD
  47. def splits: Array[Split]

    Definition Classes
    UnionRDDRDD
  48. val splits_: Array[Split]

  49. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  50. def take(num: Int): Array[T]

    Take the first num elements of the RDD.

    Take the first num elements of the RDD. This currently scans the partitions *one by one*, so it will be slow if a lot of partitions are required. In that case, use collect() to get the whole RDD instead.

    Definition Classes
    RDD
  51. def takeSample(withReplacement: Boolean, num: Int, seed: Int): Array[T]

    Definition Classes
    RDD
  52. def toArray(): Array[T]

    Definition Classes
    RDD
  53. def toString(): String

    Definition Classes
    AnyRef → Any
  54. def union(other: RDD[T]): RDD[T]

    Definition Classes
    RDD
  55. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  56. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  57. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()

Inherited from RDD[T]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any