org.apache.spark.sql.execution.aggregate

AggregationIterator

abstract class AggregationIterator extends Iterator[InternalRow] with Logging

The base class of SortBasedAggregationIterator. It mainly contains two parts: 1. It initializes aggregate functions. 2. It creates two functions, processRow and generateOutput based on AggregateMode of its aggregate functions. processRow is the function to handle an input. generateOutput is used to generate result.

Linear Supertypes
Logging, Iterator[InternalRow], TraversableOnce[InternalRow], GenTraversableOnce[InternalRow], AnyRef, Any
Known Subclasses
SortBasedAggregationIterator
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. AggregationIterator
  2. Logging
  3. Iterator
  4. TraversableOnce
  5. GenTraversableOnce
  6. AnyRef
  7. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new AggregationIterator(groupingKeyAttributes: Seq[Attribute], valueAttributes: Seq[Attribute], nonCompleteAggregateExpressions: Seq[AggregateExpression], nonCompleteAggregateAttributes: Seq[Attribute], completeAggregateExpressions: Seq[AggregateExpression], completeAggregateAttributes: Seq[Attribute], initialInputBufferOffset: Int, resultExpressions: Seq[NamedExpression], newMutableProjection: (Seq[Expression], Seq[Attribute]) ⇒ () ⇒ MutableProjection, outputsUnsafeRows: Boolean)

Type Members

  1. class GroupedIterator[B >: A] extends AbstractIterator[Seq[B]] with Iterator[Seq[B]]

    Definition Classes
    Iterator

Abstract Value Members

  1. abstract def hasNext: Boolean

    Definition Classes
    Iterator

  2. abstract def newBuffer: MutableRow

    Creates a new aggregation buffer and initializes buffer values for all aggregate functions.

    Creates a new aggregation buffer and initializes buffer values for all aggregate functions.

    Attributes
    protected

  3. abstract def next(): InternalRow

    Definition Classes
    Iterator

Concrete Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. def ++[B >: InternalRow](that: ⇒ GenTraversableOnce[B]): Iterator[B]

    Definition Classes
    Iterator

  5. def /:[B](z: B)(op: (B, InternalRow) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce

  6. def :\[B](z: B)(op: (InternalRow, B) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce

  7. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  8. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  9. def addString(b: StringBuilder): StringBuilder

    Definition Classes
    TraversableOnce

  10. def addString(b: StringBuilder, sep: String): StringBuilder

    Definition Classes
    TraversableOnce

  11. def addString(b: StringBuilder, start: String, sep: String, end: String): StringBuilder

    Definition Classes
    TraversableOnce

  12. def aggregate[B](z: B)(seqop: (B, InternalRow) ⇒ B, combop: (B, B) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce

  13. val aggregationMode: (Option[AggregateMode], Option[AggregateMode])

    The distinct modes of AggregateExpressions.

    The distinct modes of AggregateExpressions. Right now, we can handle the following mode:

    • Partial-only: all AggregateExpressions have the mode of Partial;
    • PartialMerge-only: all AggregateExpressions have the mode of PartialMerge);
    • Final-only: all AggregateExpressions have the mode of Final;
    • Final-Complete: some AggregateExpressions have the mode of Final and others have the mode of Complete;
    • Complete-only: nonCompleteAggregateExpressions is empty and we have AggregateExpressions with mode Complete in completeAggregateExpressions; and
    • Grouping-only: there is no AggregateExpression.
    Attributes
    protected

  14. val allAggregateExpressions: Seq[AggregateExpression]

    Attributes
    protected

  15. val allAggregateFunctions: Array[AggregateFunction]

    Attributes
    protected

  16. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  17. def buffered: BufferedIterator[InternalRow]

    Definition Classes
    Iterator

  18. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  19. def collect[B](pf: PartialFunction[InternalRow, B]): Iterator[B]

    Definition Classes
    Iterator
    Annotations
    @migration
    Migration

    (Changed in version 2.8.0) collect has changed. The previous behavior can be reproduced with toSeq.

  20. def collectFirst[B](pf: PartialFunction[InternalRow, B]): Option[B]

    Definition Classes
    TraversableOnce

  21. def contains(elem: Any): Boolean

    Definition Classes
    Iterator

  22. def copyToArray[B >: InternalRow](xs: Array[B], start: Int, len: Int): Unit

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce

  23. def copyToArray[B >: InternalRow](xs: Array[B]): Unit

    Definition Classes
    TraversableOnce → GenTraversableOnce

  24. def copyToArray[B >: InternalRow](xs: Array[B], start: Int): Unit

    Definition Classes
    TraversableOnce → GenTraversableOnce

  25. def copyToBuffer[B >: InternalRow](dest: Buffer[B]): Unit

    Definition Classes
    TraversableOnce

  26. def corresponds[B](that: GenTraversableOnce[B])(p: (InternalRow, B) ⇒ Boolean): Boolean

    Definition Classes
    Iterator

  27. def count(p: (InternalRow) ⇒ Boolean): Int

    Definition Classes
    TraversableOnce → GenTraversableOnce

  28. def drop(n: Int): Iterator[InternalRow]

    Definition Classes
    Iterator

  29. def dropWhile(p: (InternalRow) ⇒ Boolean): Iterator[InternalRow]

    Definition Classes
    Iterator

  30. def duplicate: (Iterator[InternalRow], Iterator[InternalRow])

    Definition Classes
    Iterator

  31. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  32. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  33. def exists(p: (InternalRow) ⇒ Boolean): Boolean

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce

  34. def filter(p: (InternalRow) ⇒ Boolean): Iterator[InternalRow]

    Definition Classes
    Iterator

  35. def filterNot(p: (InternalRow) ⇒ Boolean): Iterator[InternalRow]

    Definition Classes
    Iterator

  36. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  37. def find(p: (InternalRow) ⇒ Boolean): Option[InternalRow]

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce

  38. def flatMap[B](f: (InternalRow) ⇒ GenTraversableOnce[B]): Iterator[B]

    Definition Classes
    Iterator

  39. def fold[A1 >: InternalRow](z: A1)(op: (A1, A1) ⇒ A1): A1

    Definition Classes
    TraversableOnce → GenTraversableOnce

  40. def foldLeft[B](z: B)(op: (B, InternalRow) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce

  41. def foldRight[B](z: B)(op: (InternalRow, B) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce

  42. def forall(p: (InternalRow) ⇒ Boolean): Boolean

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce

  43. def foreach[U](f: (InternalRow) ⇒ U): Unit

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce

  44. val generateOutput: (InternalRow, MutableRow) ⇒ InternalRow

    Attributes
    protected

  45. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  46. def grouped[B >: InternalRow](size: Int): GroupedIterator[B]

    Definition Classes
    Iterator

  47. def hasDefiniteSize: Boolean

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce

  48. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  49. def indexOf[B >: InternalRow](elem: B): Int

    Definition Classes
    Iterator

  50. def indexWhere(p: (InternalRow) ⇒ Boolean): Int

    Definition Classes
    Iterator

  51. def initializeBuffer(buffer: MutableRow): Unit

    Initializes buffer values for all aggregate functions.

    Initializes buffer values for all aggregate functions.

    Attributes
    protected

  52. def isEmpty: Boolean

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce

  53. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  54. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging

  55. def isTraversableAgain: Boolean

    Definition Classes
    Iterator → GenTraversableOnce

  56. def length: Int

    Definition Classes
    Iterator

  57. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging

  58. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  59. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  60. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  61. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  62. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  63. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  64. def logName: String

    Attributes
    protected
    Definition Classes
    Logging

  65. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  66. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  67. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  68. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  69. def map[B](f: (InternalRow) ⇒ B): Iterator[B]

    Definition Classes
    Iterator

  70. def max[B >: InternalRow](implicit cmp: Ordering[B]): InternalRow

    Definition Classes
    TraversableOnce → GenTraversableOnce

  71. def maxBy[B](f: (InternalRow) ⇒ B)(implicit cmp: Ordering[B]): InternalRow

    Definition Classes
    TraversableOnce → GenTraversableOnce

  72. def min[B >: InternalRow](implicit cmp: Ordering[B]): InternalRow

    Definition Classes
    TraversableOnce → GenTraversableOnce

  73. def minBy[B](f: (InternalRow) ⇒ B)(implicit cmp: Ordering[B]): InternalRow

    Definition Classes
    TraversableOnce → GenTraversableOnce

  74. def mkString: String

    Definition Classes
    TraversableOnce → GenTraversableOnce

  75. def mkString(sep: String): String

    Definition Classes
    TraversableOnce → GenTraversableOnce

  76. def mkString(start: String, sep: String, end: String): String

    Definition Classes
    TraversableOnce → GenTraversableOnce

  77. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  78. def nonEmpty: Boolean

    Definition Classes
    TraversableOnce → GenTraversableOnce

  79. final def notify(): Unit

    Definition Classes
    AnyRef

  80. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  81. def padTo[A1 >: InternalRow](len: Int, elem: A1): Iterator[A1]

    Definition Classes
    Iterator

  82. def partition(p: (InternalRow) ⇒ Boolean): (Iterator[InternalRow], Iterator[InternalRow])

    Definition Classes
    Iterator

  83. def patch[B >: InternalRow](from: Int, patchElems: Iterator[B], replaced: Int): Iterator[B]

    Definition Classes
    Iterator

  84. val processRow: (MutableRow, InternalRow) ⇒ Unit

    Attributes
    protected

  85. def product[B >: InternalRow](implicit num: Numeric[B]): B

    Definition Classes
    TraversableOnce → GenTraversableOnce

  86. def reduce[A1 >: InternalRow](op: (A1, A1) ⇒ A1): A1

    Definition Classes
    TraversableOnce → GenTraversableOnce

  87. def reduceLeft[B >: InternalRow](op: (B, InternalRow) ⇒ B): B

    Definition Classes
    TraversableOnce

  88. def reduceLeftOption[B >: InternalRow](op: (B, InternalRow) ⇒ B): Option[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  89. def reduceOption[A1 >: InternalRow](op: (A1, A1) ⇒ A1): Option[A1]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  90. def reduceRight[B >: InternalRow](op: (InternalRow, B) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce

  91. def reduceRightOption[B >: InternalRow](op: (InternalRow, B) ⇒ B): Option[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  92. def reversed: List[InternalRow]

    Attributes
    protected[this]
    Definition Classes
    TraversableOnce

  93. def sameElements(that: Iterator[_]): Boolean

    Definition Classes
    Iterator

  94. def scanLeft[B](z: B)(op: (B, InternalRow) ⇒ B): Iterator[B]

    Definition Classes
    Iterator

  95. def scanRight[B](z: B)(op: (InternalRow, B) ⇒ B): Iterator[B]

    Definition Classes
    Iterator

  96. def seq: Iterator[InternalRow]

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce

  97. def size: Int

    Definition Classes
    TraversableOnce → GenTraversableOnce

  98. def slice(from: Int, until: Int): Iterator[InternalRow]

    Definition Classes
    Iterator

  99. def sliding[B >: InternalRow](size: Int, step: Int): GroupedIterator[B]

    Definition Classes
    Iterator

  100. def span(p: (InternalRow) ⇒ Boolean): (Iterator[InternalRow], Iterator[InternalRow])

    Definition Classes
    Iterator

  101. def sum[B >: InternalRow](implicit num: Numeric[B]): B

    Definition Classes
    TraversableOnce → GenTraversableOnce

  102. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  103. def take(n: Int): Iterator[InternalRow]

    Definition Classes
    Iterator

  104. def takeWhile(p: (InternalRow) ⇒ Boolean): Iterator[InternalRow]

    Definition Classes
    Iterator

  105. def to[Col[_]](implicit cbf: CanBuildFrom[Nothing, InternalRow, Col[InternalRow]]): Col[InternalRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  106. def toArray[B >: InternalRow](implicit arg0: ClassTag[B]): Array[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  107. def toBuffer[B >: InternalRow]: Buffer[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  108. def toIndexedSeq: IndexedSeq[InternalRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  109. def toIterable: Iterable[InternalRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  110. def toIterator: Iterator[InternalRow]

    Definition Classes
    Iterator → GenTraversableOnce

  111. def toList: List[InternalRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  112. def toMap[T, U](implicit ev: <:<[InternalRow, (T, U)]): Map[T, U]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  113. def toSeq: Seq[InternalRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  114. def toSet[B >: InternalRow]: Set[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  115. def toStream: Stream[InternalRow]

    Definition Classes
    Iterator → GenTraversableOnce

  116. def toString(): String

    Definition Classes
    Iterator → AnyRef → Any

  117. def toTraversable: Traversable[InternalRow]

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce

  118. def toVector: Vector[InternalRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce

  119. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  120. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  121. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  122. def withFilter(p: (InternalRow) ⇒ Boolean): Iterator[InternalRow]

    Definition Classes
    Iterator

  123. def zip[B](that: Iterator[B]): Iterator[(InternalRow, B)]

    Definition Classes
    Iterator

  124. def zipAll[B, A1 >: InternalRow, B1 >: B](that: Iterator[B], thisElem: A1, thatElem: B1): Iterator[(A1, B1)]

    Definition Classes
    Iterator

  125. def zipWithIndex: Iterator[(InternalRow, Int)]

    Definition Classes
    Iterator

Deprecated Value Members

  1. def /:\[A1 >: InternalRow](z: A1)(op: (A1, A1) ⇒ A1): A1

    Definition Classes
    GenTraversableOnce
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) use fold instead

Inherited from Logging

Inherited from Iterator[InternalRow]

Inherited from TraversableOnce[InternalRow]

Inherited from GenTraversableOnce[InternalRow]

Inherited from AnyRef

Inherited from Any

Ungrouped