Package org.apache.flink.runtime.asyncprocessing.functions

Class DeclaringAsyncKeyedProcessFunction<K,I,O>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.KeyedProcessFunction<K,I,O>

org.apache.flink.runtime.asyncprocessing.functions.DeclaringAsyncKeyedProcessFunction<K,I,O>

Type Parameters:

K - Type of the key.

I - Type of the input elements.

O - Type of the output elements.

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction

@Internal
public abstract class DeclaringAsyncKeyedProcessFunction<K,I,O>
extends KeyedProcessFunction<K,I,O>

A keyed function that processes elements of a stream. This works with AsyncKeyedProcessOperator using the async state access and declares the processing logic as FLIP-455 describes. This function extends from the non-declaring KeyedProcessFunction for the convenience of unifying logic in the operator AsyncKeyedProcessOperator. Maybe later we could extract the common part KeyedProcessFunction and this class to a base class, and deviate from that.

For every element in the input stream the process declareProcess(org.apache.flink.runtime.asyncprocessing.declare.DeclarationContext, org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context, org.apache.flink.util.Collector<O>) declared is invoked. This can produce zero or more elements as output. Implementations can also query the time and set timers through the provided KeyedProcessFunction.Context. For firing timers the process declareOnTimer(org.apache.flink.runtime.asyncprocessing.declare.DeclarationContext, org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext, org.apache.flink.util.Collector<O>) declared will be invoked. This can again produce zero or more elements as output and register further timers.

NOTE: Access to keyed state and timers (which are also scoped to a key) is only available if the DeclaringAsyncKeyedProcessFunction is applied on a KeyedStream.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.KeyedProcessFunction

KeyedProcessFunction.Context, KeyedProcessFunction.OnTimerContext

Constructor Summary

Constructors

Constructor	Description
DeclaringAsyncKeyedProcessFunction()

Method Summary

Modifier and Type	Method	Description
org.apache.flink.util.function.ThrowingConsumer<Long,Exception>	declareOnTimer(DeclarationContext context, KeyedProcessFunction.OnTimerContext ctx, org.apache.flink.util.Collector<O> out)	Declare a procedure which is called when a timer set using TimerService fires.
abstract org.apache.flink.util.function.ThrowingConsumer<I,Exception>	declareProcess(DeclarationContext context, KeyedProcessFunction.Context ctx, org.apache.flink.util.Collector<O> out)	Declare a process for one element from the input stream.
void	declareVariables(DeclarationContext context)	Declaring variables before declareProcess(org.apache.flink.runtime.asyncprocessing.declare.DeclarationContext, org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context, org.apache.flink.util.Collector<O>) and declareOnTimer(org.apache.flink.runtime.asyncprocessing.declare.DeclarationContext, org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext, org.apache.flink.util.Collector<O>).
void	onTimer(long timestamp, KeyedProcessFunction.OnTimerContext ctx, org.apache.flink.util.Collector<O> out)	Override and finalize this method.
void	processElement(I value, KeyedProcessFunction.Context ctx, org.apache.flink.util.Collector<O> out)	Override and finalize this method.

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

close, getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DeclaringAsyncKeyedProcessFunction

public DeclaringAsyncKeyedProcessFunction()

Method Detail

processElement

public final void processElement(I value,
                                 KeyedProcessFunction.Context ctx,
                                 org.apache.flink.util.Collector<O> out)
                          throws Exception

Override and finalize this method. Please use declareProcess(org.apache.flink.runtime.asyncprocessing.declare.DeclarationContext, org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context, org.apache.flink.util.Collector<O>) instead.

Specified by:

processElement in class KeyedProcessFunction<K,I,O>

Parameters:

value - The input value.

ctx - A KeyedProcessFunction.Context that allows querying the timestamp of the element and getting a TimerService for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.

out - The collector for returning result values.

Throws:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.

onTimer

public final void onTimer(long timestamp,
                          KeyedProcessFunction.OnTimerContext ctx,
                          org.apache.flink.util.Collector<O> out)
                   throws Exception

Override and finalize this method. Please use declareOnTimer(org.apache.flink.runtime.asyncprocessing.declare.DeclarationContext, org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext, org.apache.flink.util.Collector<O>) instead.

Overrides:

onTimer in class KeyedProcessFunction<K,I,O>

Parameters:

timestamp - The timestamp of the firing timer.

ctx - An KeyedProcessFunction.OnTimerContext that allows querying the timestamp, the TimeDomain, and the key of the firing timer and getting a TimerService for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.

out - The collector for returning result values.

Throws:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.

declareVariables

public void declareVariables(DeclarationContext context)

Declaring variables before declareProcess(org.apache.flink.runtime.asyncprocessing.declare.DeclarationContext, org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context, org.apache.flink.util.Collector<O>) and declareOnTimer(org.apache.flink.runtime.asyncprocessing.declare.DeclarationContext, org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext, org.apache.flink.util.Collector<O>).

declareProcess

public abstract org.apache.flink.util.function.ThrowingConsumer<I,Exception> declareProcess(DeclarationContext context,
                                                                                                  KeyedProcessFunction.Context ctx,
                                                                                                  org.apache.flink.util.Collector<O> out)
                                                                                           throws DeclarationException

Declare a process for one element from the input stream.

This function can output zero or more elements using the Collector parameter and also update internal state or set timers using the KeyedProcessFunction.Context parameter.

Parameters:

context - the context that provides useful methods to define named callbacks.

ctx - A KeyedProcessFunction.Context that allows querying the timestamp of the element and getting a TimerService for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.

out - The collector for returning result values.

Returns:

the whole processing logic just like processElement.

Throws:

DeclarationException

declareOnTimer

public org.apache.flink.util.function.ThrowingConsumer<Long,Exception> declareOnTimer(DeclarationContext context,
                                                                                            KeyedProcessFunction.OnTimerContext ctx,
                                                                                            org.apache.flink.util.Collector<O> out)
                                                                                     throws DeclarationException

Declare a procedure which is called when a timer set using TimerService fires.

Parameters:

context - the context that provides useful methods to define named callbacks.

ctx - An KeyedProcessFunction.OnTimerContext that allows querying the timestamp, the TimeDomain, and the key of the firing timer and getting a TimerService for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.

out - The processor for processing timestamps.

Throws:

DeclarationException