com.groupon.sparklint.analyzer

SparklintAnalyzerLike

trait SparklintAnalyzerLike extends AnyRef

An analyzer is responsible of providing useful stats after processing spark event logs

Since

6/13/16.

Linear Supertypes
AnyRef, Any
Known Subclasses
SparklintStateAnalyzer
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. SparklintAnalyzerLike
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def getCoreUtilizationPercentage: Option[Double]

    returns

    sum(idle time)/sum(allocated cpu time)

  2. abstract def getCumulativeCoreUsage: Option[Map[Int, Long]]

    returns

    cores -> milliseconds with this level of core usage

  3. abstract def getCurrentCores: Option[Int]

    returns

    how many cores are currently being used

  4. abstract def getCurrentTaskByExecutors: Option[Map[String, Iterable[SparklintTaskInfo]]]

    How tasks are allocated on each executor

    How tasks are allocated on each executor

    returns

    ExecutorId -> [TaskInfo]

  5. abstract def getExecutorInfo: Option[Map[String, SparklintExecutorInfo]]

    Get the cores for each executor allocated

    Get the cores for each executor allocated

    returns

    ExecutorId -> [ExecutorInfo]

  6. abstract def getFairSchedulerPools: Seq[String]

    If the app is using fair scheduler, return all the pools being used.

    If the app is using fair scheduler, return all the pools being used. Otherwise, empty

    returns

  7. abstract def getIdleTime: Option[Long]

    returns

    the number of milliseconds when the app is totally idle

  8. abstract def getIdleTimeSinceFirstTask: Option[Long]

    returns

    the number of milliseconds when the app is totally idle after the first task is submitted

  9. abstract def getLastUpdatedAt: Option[Long]

    returns

    Time when the last event was processed

  10. abstract def getLocalityStatsByStageIdentifier(stageIdentifier: SparklintStageIdentifier): Option[SparklintStageMetrics]

    stageIdentifier

    the identifier for a series of stages

    returns

    for each locality level, the cumulative task metrics for that locality level

  11. abstract def getMaxAllocatedCores: Option[Int]

    returns

    the highest allocated cores throughout the app history

  12. abstract def getMaxConcurrentTasks: Option[Int]

    returns

    the highest concurrent tasks throughout the app history

  13. abstract def getMaxCoreUsage: Option[Int]

    returns

    the highest core usage throughout the app history

  14. abstract def getRDDReferencedMoreThan(times: Int): Option[Seq[SparklintRDDInfo]]

    times

    filter rdds that has been referenced more than this many times

    returns

    rdd information

  15. abstract def getRunningTasks: Option[Int]

    Can be different from getCurrentCores since one task can sometimes use more than 1 cores

    Can be different from getCurrentCores since one task can sometimes use more than 1 cores

    returns

    how many tasks are currently being executed.

  16. abstract def getTimeSeriesCoreUsage: Option[Seq[CoreUsage]]

    returns

    [avg load], the time information is contained inside CoreUsage already

  17. abstract def getTimeUntilFirstTask: Option[Long]

    returns

    the number of milliseconds between sparklint listener was created and received first task info

Concrete Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  7. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  8. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  9. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  10. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  11. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  12. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  13. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  14. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  15. final def notify(): Unit

    Definition Classes
    AnyRef

  16. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  17. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  18. def toString(): String

    Definition Classes
    AnyRef → Any

  19. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  20. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  21. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped