AutoCacheRule

Value Members

1. final def !=(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

2. final def !=(arg0: Any): Boolean

   Definition Classes

   Any

3. final def ##(): Int

   Definition Classes

   AnyRef → Any

4. final def ==(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

5. final def ==(arg0: Any): Boolean

   Definition Classes

   Any

6. def addCachesToPipeline(pipe: Graph, cachesToAdd: Set[NodeId]): Graph

   Given a pipeline DAG and an additional set of nodes to cache - return a DAG with the nodes cached.

7. def aggressiveCache(graph: Graph): Graph

8. def apply(plan: Graph, prefixes: Map[NodeId, Prefix]): (Graph, Map[NodeId, Prefix])

   Definition Classes

   AutoCacheRule → Rule

9. final def asInstanceOf[T0]: T0

   Definition Classes

   Any

10. def cachedMem(cached: Set[NodeId], profiles: Map[NodeId, Profile]): Long

11. def clone(): AnyRef

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

12. final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

13. def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

14. def estimateCachedRunTime(graph: Graph, linearization: Seq[GraphId], childrenByNode: Map[NodeId, Seq[GraphId]], cached: Set[NodeId], profiles: Map[NodeId, Profile]): Double

    Estimates the total runtime of a pipeline given the cached set of nodes

15. def finalize(): Unit

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

16. def generalizeProfiles(newScale: Long, sampleProfiles: Seq[SampleProfile]): Profile

    This method takes a sequence of profiles at different sample sizes, and generalizes them to a new data scale by fitting then using linear models for memory and cpu usage dependent on data scale.

17. def getChildrenForAllNodes(graph: Graph): Map[NodeId, Seq[GraphId]]

    Get a map representing the children for each node Note: This doesn't capture how many times each child depended on the instruction

18. final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

19. def getDescendantsOfSources(graph: Graph): Set[NodeId]

    Get all descendents of all sources in the graph

20. def getNodeWeights(graph: Graph): Map[NodeId, Int]

    Get the operator weights: estimates for how many passes an operator will make over its input dependencies

21. def getRuns(linearization: Seq[GraphId], childrenByNode: Map[NodeId, Seq[GraphId]], cached: Set[NodeId], nodeWeights: Map[NodeId, Int]): Map[NodeId, Int]

    Get an estimate for how many times the output of each node will be accessed, assuming the given set of nodes have their outputs cached.

    Get an estimate for how many times the output of each node will be accessed, assuming the given set of nodes have their outputs cached.

    Note: This assumes all sinks are accessed exactly once!

22. def greedyCache(graph: Graph, profiles: Map[NodeId, Profile], maxMem: Option[Long]): Graph

23. def hashCode(): Int

    Definition Classes

    AnyRef → Any

24. def initCacheSet(graph: Graph): Set[NodeId]

    Get the initial set of what nodes will have their results effectively cached

25. final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

26. def log: Logger

    Attributes

    protected

    Definition Classes

    Logging

27. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

28. def logDebug(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

29. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

30. def logError(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

31. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

32. def logInfo(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

33. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

34. def logTrace(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

35. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

36. def logWarning(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

37. final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

38. final def notify(): Unit

    Definition Classes

    AnyRef

39. final def notifyAll(): Unit

    Definition Classes

    AnyRef

40. def profileNodes(graph: Graph, linearization: Seq[GraphId], nodesToProfile: Set[NodeId], partitionScales: Seq[Long], numTrials: Int): Map[NodeId, Profile]

    Get profiles of nodes in the pipeline

    Get profiles of nodes in the pipeline

    graph

    The pipeline DAG

    linearization

    A linearization in the nodes of the pipeline DAG

    nodesToProfile

    The nodes to collect profiling information for

    partitionScales

    The scales to profile at (expected number of data points per partition)

    numTrials

    The number of times to profile at each scale

    returns

41. val ruleName: String

    Name for this rule, automatically inferred based on class name.

    Name for this rule, automatically inferred based on class name.

    Definition Classes

    Rule

42. def selectNext(graph: Graph, linearization: Seq[GraphId], profiles: Map[NodeId, Profile], childrenByNode: Map[NodeId, Seq[GraphId]], cached: Set[NodeId], runs: Map[NodeId, Int], spaceLeft: Long): NodeId

43. def stillRoom(cached: Set[NodeId], runs: Map[NodeId, Int], profiles: Map[NodeId, Profile], spaceLeft: Long): Boolean

    Returns true iff there is still an uncached node whose output is used > once, that would fit in memory if cached

44. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

45. def toString(): String

    Definition Classes

    AnyRef → Any

46. final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

47. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

48. final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )