org.apache.spark.sql.execution.streaming

IncrementalExecution

class IncrementalExecution extends QueryExecution with Logging

A variant of QueryExecution that allows the execution of the given LogicalPlan plan incrementally. Possibly preserving state in between each execution.

Linear Supertypes
Logging, QueryExecution, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. IncrementalExecution
  2. Logging
  3. QueryExecution
  4. AnyRef
  5. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new IncrementalExecution(sparkSession: SparkSession, logicalPlan: LogicalPlan, outputMode: OutputMode, checkpointLocation: String, currentBatchId: Long, offsetSeqMetadata: OffsetSeqMetadata)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. lazy val analyzed: LogicalPlan

    Definition Classes
    QueryExecution

  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  8. def assertAnalyzed(): Unit

    Definition Classes
    QueryExecution

  9. def assertSupported(): Unit

    No need assert supported, as this check has already been done

    No need assert supported, as this check has already been done

    Definition Classes
    IncrementalExecution → QueryExecution

  10. val checkpointLocation: String

  11. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  12. val currentBatchId: Long

  13. object debug

    A special namespace for commands that can be used to debug query execution.

  14. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  15. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  16. lazy val executedPlan: SparkPlan

    Definition Classes
    QueryExecution

  17. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  18. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  19. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  20. def hiveResultString(): Seq[String]

    Returns the result as a hive compatible sequence of strings.

    Returns the result as a hive compatible sequence of strings. This is for testing only.

    Definition Classes
    QueryExecution

  21. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Attributes
    protected
    Definition Classes
    Logging

  22. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  23. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging

  24. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging

  25. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  26. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  27. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  28. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  29. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  30. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  31. def logName: String

    Attributes
    protected
    Definition Classes
    Logging

  32. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  33. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  34. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  35. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  36. val logical: LogicalPlan

    Definition Classes
    QueryExecution

  37. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  38. final def notify(): Unit

    Definition Classes
    AnyRef

  39. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  40. lazy val optimizedPlan: LogicalPlan

    See [SPARK-18339] Walk the optimized logical plan and replace CurrentBatchTimestamp with the desired literal

    See [SPARK-18339] Walk the optimized logical plan and replace CurrentBatchTimestamp with the desired literal

    Definition Classes
    IncrementalExecution → QueryExecution

  41. val outputMode: OutputMode

  42. val planner: SparkPlanner

    Definition Classes
    IncrementalExecution → QueryExecution

  43. def preparations: Seq[Rule[SparkPlan]]

    A sequence of rules that will be applied in order to the physical plan before execution.

    A sequence of rules that will be applied in order to the physical plan before execution.

    Definition Classes
    IncrementalExecution → QueryExecution

  44. def prepareForExecution(plan: SparkPlan): SparkPlan

    Prepares a planned SparkPlan for execution by inserting shuffle operations and internal row format conversions as needed.

    Prepares a planned SparkPlan for execution by inserting shuffle operations and internal row format conversions as needed.

    Attributes
    protected
    Definition Classes
    QueryExecution

  45. def simpleString: String

    Definition Classes
    QueryExecution

  46. lazy val sparkPlan: SparkPlan

    Definition Classes
    QueryExecution

  47. val state: Rule[SparkPlan]

    Locates save/restore pairs surrounding aggregation.

  48. def stringOrError[A](f: ⇒ A): String

    Attributes
    protected
    Definition Classes
    QueryExecution

  49. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  50. lazy val toRdd: RDD[InternalRow]

    Internal version of the RDD.

    Internal version of the RDD. Avoids copies and has no schema

    Definition Classes
    QueryExecution

  51. def toString(): String

    Definition Classes
    QueryExecution → AnyRef → Any

  52. def toStringWithStats: String

    Definition Classes
    QueryExecution

  53. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  54. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  55. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  56. lazy val withCachedData: LogicalPlan

    Definition Classes
    QueryExecution

Inherited from Logging

Inherited from QueryExecution

Inherited from AnyRef

Inherited from Any

Ungrouped