:: DeveloperApi :: Information about an org.apache.spark.Accumulable modified during a task or stage.
:: DeveloperApi :: Information about an org.apache.spark.Accumulable modified during a task or stage.
:: DeveloperApi :: Parses and holds information about inputFormat (and files) specified as a parameter.
:: DeveloperApi :: Parses and holds information about inputFormat (and files) specified as a parameter.
:: DeveloperApi :: A result of a job in the DAGScheduler.
:: DeveloperApi :: A result of a job in the DAGScheduler.
:: DeveloperApi :: Interface for listening to events from the Spark scheduler.
:: DeveloperApi :: Interface for listening to events from the Spark scheduler. Note that this is an internal interface which might change in different Spark releases.
Periodic updates from executors.
Periodic updates from executors.
executor id
sequence of (task id, stage id, stage attempt, metrics)
:: DeveloperApi :: Stores information about a stage to pass from the scheduler to SparkListeners.
:: DeveloperApi :: Stores information about a stage to pass from the scheduler to SparkListeners.
:: DeveloperApi :: Simple SparkListener that logs a few summary statistics when each stage completes
:: DeveloperApi :: Simple SparkListener that logs a few summary statistics when each stage completes
:: DeveloperApi :: Information about a running task attempt inside a TaskSet.
:: DeveloperApi :: Information about a running task attempt inside a TaskSet.
:: DeveloperApi :: A logger class to record runtime information for jobs in Spark.
:: DeveloperApi :: A logger class to record runtime information for jobs in Spark. This class outputs one log file for each Spark job, containing tasks start/stop and shuffle information. JobLogger is a subclass of SparkListener, use addSparkListener to add JobLogger to a SparkContext after the SparkContext is created. Note that each JobLogger only works for one SparkContext
NOTE: The functionality of this class is heavily stripped down to accommodate for a general refactor of the SparkListener interface. In its place, the EventLoggingListener is introduced to log application information as SparkListenerEvents. To enable this functionality, set spark.eventLog.enabled to true.
(Since version 1.0.0) Log application information by setting spark.eventLog.enabled.
"FAIR" and "FIFO" determines which policy is used to order tasks amongst a Schedulable's sub-queues "NONE" is used when the a Schedulable has no sub-queues.
Spark's scheduling components. This includes the org.apache.spark.scheduler.DAGScheduler and lower level org.apache.spark.scheduler.TaskScheduler.