executor

Type Members

1. class InputMetrics extends Serializable

   

   :: DeveloperApi :: A collection of accumulators that represents metrics about reading data from external systems.

   :: DeveloperApi :: A collection of accumulators that represents metrics about reading data from external systems.

   Annotations

   @DeveloperApi()

2. class OutputMetrics extends Serializable

   

   :: DeveloperApi :: A collection of accumulators that represents metrics about writing data to external systems.

   :: DeveloperApi :: A collection of accumulators that represents metrics about writing data to external systems.

   Annotations

   @DeveloperApi()

3. class ShuffleReadMetrics extends Serializable

   

   :: DeveloperApi :: A collection of accumulators that represent metrics about reading shuffle data.

   :: DeveloperApi :: A collection of accumulators that represent metrics about reading shuffle data. Operations are not thread-safe.

   Annotations

   @DeveloperApi()

4. class ShuffleWriteMetrics extends Serializable

   

   :: DeveloperApi :: A collection of accumulators that represent metrics about writing shuffle data.

   :: DeveloperApi :: A collection of accumulators that represent metrics about writing shuffle data. Operations are not thread-safe.

   Annotations

   @DeveloperApi()

5. class TaskMetrics extends Serializable

   

   :: DeveloperApi :: Metrics tracked during the execution of a task.

   :: DeveloperApi :: Metrics tracked during the execution of a task.

   This class is wrapper around a collection of internal accumulators that represent metrics associated with a task. The local values of these accumulators are sent from the executor to the driver when the task completes. These values are then merged into the corresponding accumulator previously registered on the driver.

   The accumulator updates are also sent to the driver periodically (on executor heartbeat) and when the task failed with an exception. The TaskMetrics object itself should never be sent to the driver.

   Annotations

   @DeveloperApi()