Class

org.apache.flink.streaming.api.scala

AllWindowedStream

Related Doc: package scala

Permalink

class AllWindowedStream[T, W <: Window] extends AnyRef

A AllWindowedStream represents a data stream where the stream of elements is split into windows based on a org.apache.flink.streaming.api.windowing.assigners.WindowAssigner. Window emission is triggered based on a Trigger.

If an Evictor is specified it will be used to evict elements from the window after evaluation was triggered by the Trigger but before the actual evaluation of the window. When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.

Note that the AllWindowedStream() is purely and API construct, during runtime the AllWindowedStream() will be collapsed together with the operation over the window into one single operation.

T

The type of elements in the stream.

W

The type of Window that the org.apache.flink.streaming.api.windowing.assigners.WindowAssigner assigns the elements to.

Annotations
@Public()
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AllWindowedStream
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new AllWindowedStream(javaStream: datastream.AllWindowedStream[T, W])

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def aggregate(aggregationType: AggregationType, position: Int): DataStream[T]

    Permalink
  5. def aggregate[ACC, V, R](preAggregator: AggregateFunction[T, ACC, V], windowFunction: (W, Iterable[V], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[V], arg2: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given aggregation function.

    preAggregator

    The aggregation function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @PublicEvolving()
  6. def aggregate[ACC, V, R](preAggregator: AggregateFunction[T, ACC, V], windowFunction: ProcessAllWindowFunction[V, R, W])(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[V], arg2: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given aggregation function.

    preAggregator

    The aggregation function that is used for pre-aggregation

    windowFunction

    The process window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @PublicEvolving()
  7. def aggregate[ACC, V, R](preAggregator: AggregateFunction[T, ACC, V], windowFunction: AllWindowFunction[V, R, W])(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[V], arg2: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given aggregation function.

    preAggregator

    The aggregation function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @PublicEvolving()
  8. def aggregate[ACC, R](aggregateFunction: AggregateFunction[T, ACC, R])(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given aggregation function to each window.

    Applies the given aggregation function to each window. The aggregation function is called for each element, aggregating values incrementally and keeping the state to one accumulator per window.

    aggregateFunction

    The aggregation function.

    returns

    The data stream that is the result of applying the fold function to the window.

    Annotations
    @PublicEvolving()
  9. def allowedLateness(lateness: Time): AllWindowedStream[T, W]

    Permalink

    Sets the allowed lateness to a user-specified value.

    Sets the allowed lateness to a user-specified value. If not explicitly set, the allowed lateness is 0L. Setting the allowed lateness is only valid for event-time windows. If a value different than 0 is provided with a processing-time org.apache.flink.streaming.api.windowing.assigners.WindowAssigner, then an exception is thrown.

    Annotations
    @PublicEvolving()
  10. def apply[R](function: (W, Iterable[T], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Not that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.

    function

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

  11. def apply[R](function: AllWindowFunction[T, R, W])(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Not that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.

    function

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

  12. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  13. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  14. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  15. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  16. def evictor(evictor: Evictor[_ >: T, _ >: W]): AllWindowedStream[T, W]

    Permalink

    Sets the Evictor that should be used to evict elements from a window before emission.

    Sets the Evictor that should be used to evict elements from a window before emission.

    Note: When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.

    Annotations
    @PublicEvolving()
  17. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  18. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  19. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  20. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  21. def max(field: String): DataStream[T]

    Permalink

    Applies an aggregation that that gives the maximum of the elements in the window at the given field.

  22. def max(position: Int): DataStream[T]

    Permalink

    Applies an aggregation that that gives the maximum of the elements in the window at the given position.

  23. def maxBy(field: String): DataStream[T]

    Permalink

    Applies an aggregation that that gives the maximum element of the window by the given field.

    Applies an aggregation that that gives the maximum element of the window by the given field. When equality, returns the first.

  24. def maxBy(position: Int): DataStream[T]

    Permalink

    Applies an aggregation that that gives the maximum element of the window by the given position.

    Applies an aggregation that that gives the maximum element of the window by the given position. When equality, returns the first.

  25. def min(field: String): DataStream[T]

    Permalink

    Applies an aggregation that that gives the minimum of the elements in the window at the given field.

  26. def min(position: Int): DataStream[T]

    Permalink

    Applies an aggregation that that gives the minimum of the elements in the window at the given position.

  27. def minBy(field: String): DataStream[T]

    Permalink

    Applies an aggregation that that gives the minimum element of the window by the given field.

    Applies an aggregation that that gives the minimum element of the window by the given field. When equality, returns the first.

  28. def minBy(position: Int): DataStream[T]

    Permalink

    Applies an aggregation that that gives the minimum element of the window by the given position.

    Applies an aggregation that that gives the minimum element of the window by the given position. When equality, returns the first.

  29. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  30. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  31. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  32. def process[R](function: ProcessAllWindowFunction[T, R, W])(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Not that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.

    function

    The process window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @PublicEvolving()
  33. def reduce[R](preAggregator: (T, T) ⇒ T, windowFunction: ProcessAllWindowFunction[T, R, W])(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation reducer.

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The process window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @PublicEvolving()
  34. def reduce[R](preAggregator: ReduceFunction[T], windowFunction: ProcessAllWindowFunction[T, R, W])(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation reducer.

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The process window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @PublicEvolving()
  35. def reduce[R](preAggregator: (T, T) ⇒ T, windowFunction: (W, Iterable[T], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation reducer.

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

  36. def reduce[R](preAggregator: ReduceFunction[T], windowFunction: AllWindowFunction[T, R, W])(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation reducer.

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

  37. def reduce(function: (T, T) ⇒ T): DataStream[T]

    Permalink

    Applies a reduce function to the window.

    Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

    This window will try and pre-aggregate data as much as the window policies permit. For example, tumbling time windows can perfectly pre-aggregate the data, meaning that only one element per key is stored. Sliding time windows will pre-aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to pre-aggregate, or may need to store extra values in an aggregation tree.

    function

    The reduce function.

    returns

    The data stream that is the result of applying the reduce function to the window.

  38. def reduce(function: ReduceFunction[T]): DataStream[T]

    Permalink

    Applies a reduce function to the window.

    Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

    This window will try and pre-aggregate data as much as the window policies permit. For example, tumbling time windows can perfectly pre-aggregate the data, meaning that only one element per key is stored. Sliding time windows will pre-aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to pre-aggregate, or may need to store extra values in an aggregation tree.

    function

    The reduce function.

    returns

    The data stream that is the result of applying the reduce function to the window.

  39. def sideOutputLateData(outputTag: OutputTag[T]): AllWindowedStream[T, W]

    Permalink

    Send late arriving data to the side output identified by the given OutputTag.

    Send late arriving data to the side output identified by the given OutputTag. Data is considered late after the watermark has passed the end of the window plus the allowed lateness set using allowedLateness(Time).

    You can get the stream of late data using DataStream.getSideOutput() on the DataStream resulting from the windowed operation with the same OutputTag.

    Annotations
    @PublicEvolving()
  40. def sum(field: String): DataStream[T]

    Permalink

    Applies an aggregation that sums the elements in the window at the given field.

  41. def sum(position: Int): DataStream[T]

    Permalink

    Applies an aggregation that sums the elements in the window at the given position.

  42. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  43. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  44. def trigger(trigger: Trigger[_ >: T, _ >: W]): AllWindowedStream[T, W]

    Permalink

    Sets the Trigger that should be used to trigger window emission.

    Sets the Trigger that should be used to trigger window emission.

    Annotations
    @PublicEvolving()
  45. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  46. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  47. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def apply[R](initialValue: R, preAggregator: (R, T) ⇒ R, windowFunction: (W, Iterable[R], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation folder.

    initialValue

    Initial value of the fold

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @deprecated
    Deprecated
  2. def apply[R](initialValue: R, preAggregator: FoldFunction[T, R], windowFunction: AllWindowFunction[R, R, W])(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation folder.

    initialValue

    Initial value of the fold

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @deprecated
    Deprecated
  3. def apply[R](preAggregator: (T, T) ⇒ T, windowFunction: (W, Iterable[T], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation reducer.

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @deprecated
    Deprecated
  4. def apply[R](preAggregator: ReduceFunction[T], windowFunction: AllWindowFunction[T, R, W])(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation reducer.

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @deprecated
    Deprecated
  5. def fold[ACC, R](initialValue: ACC, preAggregator: (ACC, T) ⇒ ACC, windowFunction: ProcessAllWindowFunction[ACC, R, W])(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation folder.

    initialValue

    Initial value of the fold

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @deprecated @PublicEvolving()
    Deprecated

    use aggregate() instead

  6. def fold[ACC, R](initialValue: ACC, preAggregator: (ACC, T) ⇒ ACC, windowFunction: (W, Iterable[ACC], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation folder.

    initialValue

    Initial value of the fold

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @deprecated
    Deprecated

    use aggregate() instead

  7. def fold[ACC, R](initialValue: ACC, preAggregator: FoldFunction[T, ACC], windowFunction: ProcessAllWindowFunction[ACC, R, W])(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation folder.

    initialValue

    Initial value of the fold

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The process window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @deprecated @PublicEvolving()
    Deprecated

    use aggregate() instead

  8. def fold[ACC, R](initialValue: ACC, preAggregator: FoldFunction[T, ACC], windowFunction: AllWindowFunction[ACC, R, W])(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given window function to each window.

    Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

    Arriving data is pre-aggregated using the given pre-aggregation folder.

    initialValue

    Initial value of the fold

    preAggregator

    The reduce function that is used for pre-aggregation

    windowFunction

    The window function.

    returns

    The data stream that is the result of applying the window function to the window.

    Annotations
    @deprecated
    Deprecated

    use aggregate() instead

  9. def fold[R](initialValue: R)(function: (R, T) ⇒ R)(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given fold function to each window.

    Applies the given fold function to each window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

    function

    The fold function.

    returns

    The data stream that is the result of applying the fold function to the window.

    Annotations
    @deprecated
    Deprecated

    use aggregate() instead

  10. def fold[R](initialValue: R, function: FoldFunction[T, R])(implicit arg0: TypeInformation[R]): DataStream[R]

    Permalink

    Applies the given fold function to each window.

    Applies the given fold function to each window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

    function

    The fold function.

    returns

    The data stream that is the result of applying the fold function to the window.

    Annotations
    @deprecated
    Deprecated

    use aggregate() instead

Inherited from AnyRef

Inherited from Any

Ungrouped