Package org.apache.beam.sdk.transforms.windowing

Enum PaneInfo.Timing

  • All Implemented Interfaces:
    java.io.Serializable, java.lang.Comparable<PaneInfo.Timing>
    Enclosing class:
    PaneInfo
    public static enum PaneInfo.Timing
extends java.lang.Enum<PaneInfo.Timing>
    Enumerates the possibilities for the timing of this pane firing related to the input and output watermarks for its computation.

    A window may fire multiple panes, and the timing of those panes generally follows the regular expression EARLY* ON_TIME? LATE*. Generally a pane is considered:

    1. EARLY if the system cannot be sure it has seen all data which may contribute to the pane's window.
    2. ON_TIME if the system predicts it has seen all the data which may contribute to the pane's window.
    3. LATE if the system has encountered new data after predicting no more could arrive. It is possible an ON_TIME pane has already been emitted, in which case any following panes are considered LATE.

    Only an AfterWatermark.pastEndOfWindow() trigger may produce an ON_TIME pane. With merging WindowFn's, windows may be merged to produce new windows that satisfy their own instance of the above regular expression. The only guarantee is that once a window produces a final pane, it will not be merged into any new windows.

    The predictions above are made using the mechanism of watermarks.

    We can state some properties of LATE and ON_TIME panes, but first need some definitions:

    1. We'll call a pipeline 'simple' if it does not use DoFn.WindowedContext.outputWithTimestamp(OutputT, org.joda.time.Instant) in any DoFn, and it uses the same Window.withAllowedLateness(org.joda.time.Duration) argument value on all windows (or uses the default of Duration.ZERO).
    2. We'll call an element 'locally late', from the point of view of a computation on a worker, if the element's timestamp is before the input watermark for that computation on that worker. The element is otherwise 'locally on-time'.
    3. We'll say 'the pane's timestamp' to mean the timestamp of the element produced to represent the pane's contents.

    Then in simple pipelines:

    1. (Soundness) An ON_TIME pane can never cause a later computation to generate a LATE pane. (If it did, it would imply a later computation's input watermark progressed ahead of an earlier stage's output watermark, which by design is not possible.)
    2. (Liveness) An ON_TIME pane is emitted as soon as possible after the input watermark passes the end of the pane's window.
    3. (Consistency) A pane with only locally on-time elements will always be ON_TIME. And a LATE pane cannot contain locally on-time elements.

    However, note that:

    1. An ON_TIME pane may contain locally late elements. It may even contain only locally late elements. Provided a locally late element finds its way into an ON_TIME pane its lateness becomes unobservable.
    2. A LATE pane does not necessarily cause any following computation panes to be marked as LATE.

    • Enum Constant Summary

      Enum Constants 
      Enum Constant Description
      EARLY
      Pane was fired before the input watermark had progressed after the end of the window.
      LATE
      Pane was fired after the output watermark had progressed past the end of the window.
      ON_TIME
      Pane was fired by a AfterWatermark.pastEndOfWindow() trigger because the input watermark progressed after the end of the window.
      UNKNOWN
      This element was not produced in a triggered pane and its relation to input and output watermarks is unknown.

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static PaneInfo.Timing valueOf​(java.lang.String name)
      Returns the enum constant of this type with the specified name.
      static PaneInfo.Timing[] values()
      Returns an array containing the constants of this enum type, in the order they are declared.

      • Methods inherited from class java.lang.Enum

        clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

      • Methods inherited from class java.lang.Object

        getClass, notify, notifyAll, wait, wait, wait

    • Enum Constant Detail

      • EARLY

        public static final PaneInfo.Timing EARLY
        Pane was fired before the input watermark had progressed after the end of the window.

      • ON_TIME

        public static final PaneInfo.Timing ON_TIME
        Pane was fired by a AfterWatermark.pastEndOfWindow() trigger because the input watermark progressed after the end of the window. However the output watermark has not yet progressed after the end of the window. Thus it is still possible to assign a timestamp to the element representing this pane which cannot be considered locally late by any following computation.

      • LATE

        public static final PaneInfo.Timing LATE
        Pane was fired after the output watermark had progressed past the end of the window.

      • UNKNOWN

        public static final PaneInfo.Timing UNKNOWN
        This element was not produced in a triggered pane and its relation to input and output watermarks is unknown.

    • Method Detail

      • values

        public static PaneInfo.Timing[] values()
        Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows: 
        for (PaneInfo.Timing c : PaneInfo.Timing.values())
    System.out.println(c);
        Returns:
        an array containing the constants of this enum type, in the order they are declared

      • valueOf

        public static PaneInfo.Timing valueOf​(java.lang.String name)
        Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)
        Parameters:
        name - the name of the enum constant to be returned.
        Returns:
        the enum constant with the specified name
        Throws:
        java.lang.IllegalArgumentException - if this enum type has no constant with the specified name
        java.lang.NullPointerException - if the argument is null