Package

org.apache.spark.sql.execution.streaming

state

Permalink

package state

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. state
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. trait StateStore extends AnyRef

    Permalink

    Base trait for a versioned key-value store used for streaming aggregations

  2. class StateStoreCoordinatorRef extends AnyRef

    Permalink

    Reference to a StateStoreCoordinator that can be used to coordinate instances of StateStores across all the executors, and get their locations for job scheduling.

  3. case class StateStoreId(checkpointLocation: String, operatorId: Long, partitionId: Int) extends Product with Serializable

    Permalink

    Unique identifier for a StateStore

  4. implicit class StateStoreOps[T] extends AnyRef

    Permalink
  5. trait StateStoreProvider extends AnyRef

    Permalink

    Trait representing a provider of a specific version of a StateStore.

  6. class StateStoreRDD[T, U] extends RDD[U]

    Permalink

    An RDD that allows computations to be executed against StateStores.

    An RDD that allows computations to be executed against StateStores. It uses the StateStoreCoordinator to get the locations of loaded state stores and use that as the preferred locations.

  7. sealed trait StoreUpdate extends AnyRef

    Permalink

    Trait representing updates made to a StateStore.

  8. case class ValueAdded(key: UnsafeRow, value: UnsafeRow) extends StoreUpdate with Product with Serializable

    Permalink
  9. case class ValueRemoved(key: UnsafeRow, value: UnsafeRow) extends StoreUpdate with Product with Serializable

    Permalink
  10. case class ValueUpdated(key: UnsafeRow, value: UnsafeRow) extends StoreUpdate with Product with Serializable

    Permalink

Value Members

  1. object StateStore extends Logging

    Permalink

    Companion object to StateStore that provides helper methods to create and retrieve stores by their unique ids.

    Companion object to StateStore that provides helper methods to create and retrieve stores by their unique ids. In addition, when a SparkContext is active (i.e. SparkEnv.get is not null), it also runs a periodic background task to do maintenance on the loaded stores. For each store, it uses the StateStoreCoordinator to ensure whether the current loaded instance of the store is the active instance. Accordingly, it either keeps it loaded and performs maintenance, or unloads the store.

  2. object StateStoreCoordinatorRef extends Logging

    Permalink

    Helper object used to create reference to StateStoreCoordinator.

Inherited from AnyRef

Inherited from Any

Ungrouped