com.google.cloud.dataflow.sdk.util

Class ReduceFn<K,InputT,OutputT,W extends BoundedWindow>

java.lang.Object

com.google.cloud.dataflow.sdk.util.ReduceFn<K,InputT,OutputT,W>

Type Parameters:

K - The type of key being processed.

InputT - The type of input values associated with the key.

OutputT - The output type that will be produced for each key.

W - The type of windows this operates on.

All Implemented Interfaces:

Serializable

public abstract class ReduceFn<K,InputT,OutputT,W extends BoundedWindow>
extends Object
implements Serializable

Specification for processing to happen after elements have been grouped by key.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	ReduceFn.Context Information accessible to all the processing methods in this ReduceFn.
static interface	ReduceFn.MergingStateContext Interface for interacting with persistent state within onMerge(com.google.cloud.dataflow.sdk.util.ReduceFn<K, InputT, OutputT, W>.OnMergeContext).
class	ReduceFn.OnMergeContext Information accessible within onMerge(com.google.cloud.dataflow.sdk.util.ReduceFn<K, InputT, OutputT, W>.OnMergeContext).
class	ReduceFn.OnTriggerContext Information accessible within onTrigger(com.google.cloud.dataflow.sdk.util.ReduceFn<K, InputT, OutputT, W>.OnTriggerContext).
class	ReduceFn.ProcessValueContext Information accessible within processValue(com.google.cloud.dataflow.sdk.util.ReduceFn<K, InputT, OutputT, W>.ProcessValueContext).
static interface	ReduceFn.StateContext Interface for interacting with persistent state.
static interface	ReduceFn.Timers Interface for interacting with time.

Constructor Summary

Constructors

Constructor and Description
ReduceFn()

Method Summary

Modifier and Type	Method and Description
abstract void	clearState(ReduceFn.Context c) Called to clear any persisted state that the ReduceFn may be holding.
abstract StateContents<Boolean>	isEmpty(ReduceFn.StateContext c) Returns true if the there is no buffered state.
abstract void	onMerge(ReduceFn.OnMergeContext c) Called when windows are merged.
abstract void	onTrigger(ReduceFn.OnTriggerContext c) Called when triggers fire.
void	prefetchOnTrigger(ReduceFn.StateContext c) Called before onTrigger is invoked to provide an opportunity to prefetch any needed state.
abstract void	processValue(ReduceFn.ProcessValueContext c) Called for each value of type InputT associated with the current key.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReduceFn

public ReduceFn()

Method Detail

processValue

public abstract void processValue(ReduceFn.ProcessValueContext c)
                           throws Exception

Called for each value of type InputT associated with the current key.

Throws:

Exception

onMerge

public abstract void onMerge(ReduceFn.OnMergeContext c)
                      throws Exception

Called when windows are merged.

There are generally two strategies for implementing this and handling merging of state:

• Lazily merge the state when outputting. This is especially easy if all the state is stored in MergeableState, since an automatically merged view can be retrieved.
• Eagerly merge the state inside the onMerge(com.google.cloud.dataflow.sdk.util.ReduceFn<K, InputT, OutputT, W>.OnMergeContext) implementation. Load all the state from the merging windows and write it back to the result window. In this case the state in the result window should be cleared into between the read and write in case it was in the source windows.

Throws:

Exception

onTrigger

public abstract void onTrigger(ReduceFn.OnTriggerContext c)
                        throws Exception

Called when triggers fire.

Implementations of ReduceFn should call ReduceFn.OnTriggerContext.output(OutputT) to emit any results that should be included in the pane produced by this trigger firing.

Throws:

Exception

prefetchOnTrigger

public void prefetchOnTrigger(ReduceFn.StateContext c)

Called before onTrigger is invoked to provide an opportunity to prefetch any needed state.

Parameters:

c - Context to use prefetch from.

clearState

public abstract void clearState(ReduceFn.Context c)
                         throws Exception

Called to clear any persisted state that the ReduceFn may be holding. This will be called when the windowing is closing and will receive no future interactions.

Throws:

Exception

isEmpty

public abstract StateContents<Boolean> isEmpty(ReduceFn.StateContext c)

Returns true if the there is no buffered state.