Annotation Type DoFn.RequiresStableInput


  • @Documented
    @Experimental
    @Retention(RUNTIME)
    @Target(METHOD)
    public static @interface DoFn.RequiresStableInput
    Experimental - no backwards compatibility guarantees. The exact name or usage of this feature may change.

    Annotation that may be added to a DoFn.ProcessElement, DoFn.OnTimer, or DoFn.OnWindowExpiration method to indicate that the runner must ensure that the observable contents of the input PCollection or mutable state must be stable upon retries.

    This is important for sinks, which must ensure exactly-once semantics when writing to a storage medium outside of your pipeline. A general pattern for a basic sink is to write a DoFn that can perform an idempotent write, and annotate that it requires stable input. Combined, these allow the write to be freely retried until success.

    An example of an unstable input would be anything computed using nondeterministic logic. In Beam, any user-defined function is permitted to be nondeterministic, and any PCollection is permitted to be recomputed in any manner.