Class

org.apache.spark.sql.catalyst.plans.logical

Statistics

Related Doc: package logical

Permalink

case class Statistics(sizeInBytes: BigInt, rowCount: Option[BigInt] = None, colStats: Map[String, ColumnStat] = Map.empty, isBroadcastable: Boolean = false) extends Product with Serializable

Estimates of various statistics. The default estimation logic simply lazily multiplies the corresponding statistic produced by the children. To override this behavior, override statistics and assign it an overridden version of Statistics.

NOTE: concrete and/or overridden versions of statistics fields should pay attention to the performance of the implementations. The reason is that estimations might get triggered in performance-critical processes, such as query plan planning.

Note that we are using a BigInt here since it is easy to overflow a 64-bit integer in cardinality estimation (e.g. cartesian joins).

sizeInBytes

Physical size in bytes. For leaf operators this defaults to 1, otherwise it defaults to the product of children's sizeInBytes.

rowCount

Estimated number of rows.

colStats

Column-level statistics.

isBroadcastable

If true, output is small enough to be used in a broadcast join.

Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Statistics
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Statistics(sizeInBytes: BigInt, rowCount: Option[BigInt] = None, colStats: Map[String, ColumnStat] = Map.empty, isBroadcastable: Boolean = false)

    Permalink

    sizeInBytes

    Physical size in bytes. For leaf operators this defaults to 1, otherwise it defaults to the product of children's sizeInBytes.

    rowCount

    Estimated number of rows.

    colStats

    Column-level statistics.

    isBroadcastable

    If true, output is small enough to be used in a broadcast join.

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. val colStats: Map[String, ColumnStat]

    Permalink

    Column-level statistics.

  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  10. val isBroadcastable: Boolean

    Permalink

    If true, output is small enough to be used in a broadcast join.

  11. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  14. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  15. val rowCount: Option[BigInt]

    Permalink

    Estimated number of rows.

  16. def simpleString: String

    Permalink

    Readable string representation for the Statistics.

  17. val sizeInBytes: BigInt

    Permalink

    Physical size in bytes.

    Physical size in bytes. For leaf operators this defaults to 1, otherwise it defaults to the product of children's sizeInBytes.

  18. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  19. def toString(): String

    Permalink
    Definition Classes
    Statistics → AnyRef → Any
  20. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped