com.soundcloud.spark.pagerank

GraphUtils

object GraphUtils

Some general purpose graph operations and utilities. Any operations specific to the complete PageRank graph will be in there and not here.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. GraphUtils
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def areVerticesNormalized[T](vertices: VertexRDD, eps: Value = EPS): Boolean

    Determines if the vertices of a graph are normalized.

    Determines if the vertices of a graph are normalized. Assumes a graph with Double vertex attributes.

  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  8. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. def countDanglingVertices(edges: EdgeRDD): Long

    Counts the number of vertices with no out edges.

    Counts the number of vertices with no out edges. These are considered as "dangling" vertices.

    This is implemented by a set operation, where the dangling vertices are those that appear in the set of destination vertex IDs only (and not in the source vertex IDs).

    Performance note: edges are iterated over twice, so please consider persisting it first.

  10. def countSelfReferences(edges: EdgeRDD): Long

    Counts the number of vertices that have self-referencing edges.

  11. def countVerticesWithoutNormalizedOutEdges(edges: EdgeRDD, eps: Value = EPS): Long

    Counts the number of vertices that do not have edges that sum to 1.

    Counts the number of vertices that do not have edges that sum to 1.0. Assumes edges with Double weights.

  12. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  13. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  14. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  15. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  16. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  17. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  18. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  19. def normalizeOutEdgeWeights(edges: EdgeRDD): EdgeRDD

    Normalizes outgoing edge weights of an EdgeRDD.

    Normalizes outgoing edge weights of an EdgeRDD.

    Performance note: edges are iterated over twice, so please consider persisting it first.

  20. final def notify(): Unit

    Definition Classes
    AnyRef
  21. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  22. def removeSelfReferences(edges: EdgeRDD): EdgeRDD

    Removes any edges that are self-referencing the same vertex.

    Removes any edges that are self-referencing the same vertex. That is, any edges where the source and destination are the same.

  23. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  24. def tagDanglingVertices(srcIds: RDD[Id], dstIds: RDD[Id]): RDD[(Id, Boolean)]

    Given an RDD of source vertex IDs and an RDD of destination vertex IDs (from edges), tag vertex IDs with a flag to indicate if the vertex is dangling (no out edges) or not.

  25. def toString(): String

    Definition Classes
    AnyRef → Any
  26. def unzipDistinct(edges: EdgeRDD): (RDD[Id], RDD[Id])

    Given the edges of a graph, this unzips the source and destination vertex IDs.

    Given the edges of a graph, this unzips the source and destination vertex IDs. ID's in the resulting RDDs are distinct.

  27. def validateStructure(edges: EdgeRDD, vertices: VertexRDD, eps: Value = EPS): Option[Seq[String]]

    Validates the structure of the input PageRank graph, according to the requirements to run PageRank.

    Validates the structure of the input PageRank graph, according to the requirements to run PageRank. Returns a list of validation errors, if any.

    Performance note: edges are iterated over three times, and vertices once, so please consider persisting either or both before running this.

  28. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  30. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped