See: Description
| Interface | Description |
|---|---|
| ActiveWindowSet<W extends BoundedWindow> |
Tracks the windows that are active.
|
| ActiveWindowSet.MergeCallback<W extends BoundedWindow> | |
| CredentialFactory |
Construct an oauth credential to be used by the SDK and the SDK workers.
|
| DoFnRunner.OutputManager |
Information about how to create output receivers and output to them.
|
| ExecutionContext |
Context for the current execution.
|
| ExecutionContext.StepContext |
Per-step, per-key context used for retrieving state.
|
| IOChannelFactory |
Defines a factory for working with read and write channels.
|
| MonitoringUtil.JobMessagesHandler |
An interface that can be used for defining callbacks to receive a list
of JobMessages containing monitoring information.
|
| MutationDetector |
An object for detecting illegal mutations.
|
| PathValidator |
Interface for controlling validation of paths.
|
| ReduceFn.MergingStateContext |
Interface for interacting with persistent state within
ReduceFn.onMerge(com.google.cloud.dataflow.sdk.util.ReduceFn<K, InputT, OutputT, W>.OnMergeContext). |
| ReduceFn.StateContext |
Interface for interacting with persistent state.
|
| ReduceFn.Timers |
Interface for interacting with time.
|
| SideInputReader |
The interface to objects that provide side inputs.
|
| SizedSideInputReader |
Extension to
SideInputReader that can approximate the size of the side input. |
| Stager |
Interface for staging files needed for running a Dataflow pipeline.
|
| TimerInternals |
Encapsulate interaction with time within the execution environment.
|
| WindowingInternals<InputT,OutputT> |
Interface that may be required by some (internal)
DoFns to implement windowing. |
| Class | Description |
|---|---|
| ApiSurface |
Represents the API surface of a package prefix.
|
| AppEngineEnvironment |
Stores whether we are running within AppEngine or not.
|
| AppliedCombineFn<K,InputT,AccumT,OutputT> |
A
Combine.KeyedCombineFn with a fixed accumulator coder. |
| AssignWindowsDoFn<T,W extends BoundedWindow> | |
| AttemptAndTimeBoundedExponentialBackOff |
Extension of
AttemptBoundedExponentialBackOff that bounds the total time that the backoff
is happening as well as the amount of retries. |
| AttemptBoundedExponentialBackOff |
Implementation of
BackOff that increases the back off period for each retry attempt
using a randomization function that grows exponentially. |
| Base64Utils |
Utilities related to Base64 encoding.
|
| BaseExecutionContext |
Base class for implementations of
ExecutionContext. |
| BaseExecutionContext.StepContext |
Base class for implementations of
ExecutionContext.StepContext. |
| BatchModeExecutionContext |
ExecutionContext for use in batch mode. |
| BatchTimerInternals |
TimerInternals that uses priority queues to manage the timers that are ready to fire.
|
| BigQueryTableInserter |
Inserts rows into BigQuery.
|
| BigQueryTableRowIterator |
Iterates over all rows in a table.
|
| BoundedQueueExecutor |
Executor that blocks on execute() if its queue is full.
|
| CloudCounterUtils |
Utilities for working with CloudCounters.
|
| CloudMetricUtils |
Utilities for working with Dataflow API Metrics.
|
| CloudObject |
A representation of an arbitrary Java object to be instantiated by Dataflow
workers.
|
| CloudSourceUtils |
Utilities for working with Source Dataflow API definitions and
Reader
objects. |
| CoderUtils |
Utilities for working with Coders.
|
| CounterAggregator<InputT,AccumT,OutputT> |
An implementation of the
Aggregator interface that uses a
Counter as the underlying representation. |
| Credentials |
Provides support for loading credentials.
|
| DataflowPathValidator |
GCP implementation of
PathValidator. |
| DataflowReleaseInfo |
Utilities for working with the Dataflow distribution.
|
| DirectModeExecutionContext |
ExecutionContext for use in direct mode. |
| DirectSideInputReader |
Basic side input reader wrapping a
PTuple of side input iterables. |
| DoFnInfo<InputT,OutputT> |
Wrapper class holding the necessary information to serialize a DoFn.
|
| DoFnRunner<InputT,OutputT> |
Runs a DoFn by constructing the appropriate contexts and passing them in.
|
| DoFnRunner.ListOutputManager |
An implementation of
DoFnRunner.OutputManager using simple lists, for testing and in-memory
contexts such as the DirectPipelineRunner. |
| ExecutableTrigger<W extends BoundedWindow> |
A wrapper around a trigger used during execution.
|
| ExposedByteArrayInputStream |
ByteArrayInputStream that allows accessing the entire internal buffer without copying. |
| ExposedByteArrayOutputStream |
ByteArrayOutputStream special cased to treat writes of a single byte-array specially. |
| FileIOChannelFactory |
Implements IOChannelFactory for local files.
|
| GcpCredentialFactory |
Construct an oauth credential to be used by the SDK and the SDK workers.
|
| GcsIOChannelFactory |
Implements IOChannelFactory for GCS.
|
| GcsStager |
Utility class for staging files to GCS.
|
| GcsUtil |
Provides operations on GCS.
|
| GcsUtil.GcsUtilFactory |
This is a
DefaultValueFactory able to create a GcsUtil using
any transport flags specified on the PipelineOptions. |
| GroupAlsoByWindowsDoFn<K,InputT,OutputT,W extends BoundedWindow> |
DoFn that merges windows and groups elements in those windows, optionally
combining values.
|
| InstanceBuilder<T> |
Utility for creating objects dynamically.
|
| IntervalBoundedExponentialBackOff |
Implementation of
BackOff that increases the back off period for each retry attempt
using a randomization function that grows exponentially. |
| IOChannelUtils |
Provides utilities for creating read and write channels.
|
| MergingActiveWindowSet<W extends BoundedWindow> |
Implementation of
ActiveWindowSet used with WindowFns that support
merging. |
| MimeTypes |
Constants representing various mime types.
|
| MonitoringUtil |
A helper class for monitoring jobs submitted to the service.
|
| MonitoringUtil.PrintHandler |
A handler that prints monitoring messages to a stream.
|
| MonitoringUtil.TimeStampComparator |
Comparator for sorting rows in increasing order based on timestamp.
|
| MutationDetectors |
Static methods for creating and working with
MutationDetector. |
| NonEmptyPanes<W extends BoundedWindow> |
Tracks which windows have non-empty panes.
|
| NonMergingActiveWindowSet<W extends BoundedWindow> |
Implementation of
ActiveWindowSet used with WindowFns that don't support
merging. |
| NoopCredentialFactory |
Construct an oauth credential to be used by the SDK and the SDK workers.
|
| NoopPathValidator |
Noop implementation of
PathValidator. |
| NullSideInputReader |
A
SideInputReader representing a well-defined set of views, but not storing
any values for them. |
| OutputReference |
A representation used by
Steps
to reference the output of other Steps. |
| PackageUtil |
Helper routines for packages.
|
| PaneInfoTracker |
Encapsulates the logic for tracking the current
PaneInfo and producing new PaneInfo for
a trigger firing. |
| PCollectionViews |
Implementations of
PCollectionView shared across the SDK. |
| PCollectionViewWindow<T> |
A pair of a
PCollectionView and a BoundedWindow, which can
be thought of as window "of" the view. |
| PropertyNames |
Constant property names used by the SDK in CloudWorkflow specifications.
|
| PTuple |
A
PTuple is an immutable tuple of
heterogeneously-typed values, "keyed" by TupleTags. |
| ReaderUtils |
Utilities for working with
Reader
objects. |
| ReduceFn<K,InputT,OutputT,W extends BoundedWindow> |
Specification for processing to happen after elements have been grouped by key.
|
| ReduceFnRunner<K,InputT,OutputT,W extends BoundedWindow> |
Manages the execution of a
ReduceFn after a GroupByKey.GroupByKeyOnly has partitioned the
PCollection by key. |
| ReifyTimestampAndWindowsDoFn<K,V> |
DoFn that makes timestamps and window assignments explicit in the value part of each key/value
pair.
|
| Reshuffle<K,V> |
A
PTransform to reshuffle the elements based on their key. |
| RetryHttpRequestInitializer |
Implements a request initializer that adds retry handlers to all
HttpRequests.
|
| SerializableUtils |
Utilities for working with Serializables.
|
| Serializer |
Utility for converting objects between Java and Cloud representations.
|
| ShardingWritableByteChannel |
Implements a WritableByteChannel that may contain multiple output shards.
|
| Sized<T> |
A
T with an accompanying size estimate. |
| SizedSideInputReader.Defaults |
Abstract class providing default implementations for methods of
SizedSideInputReader. |
| StateFetcher |
Class responsible for fetching state from the windmill server.
|
| StreamingGroupAlsoByWindowsDoFn<K,InputT,OutputT,W extends BoundedWindow> |
DoFn that merges windows and groups elements in those windows.
|
| StreamingModeExecutionContext |
ExecutionContext for use in streaming mode. |
| StreamingModeExecutionContext.StreamingModeSideInputReader |
A
SideInputReader that fetches side inputs from the streaming worker's
cache. |
| StreamingPCollectionViewWriterFn<T> |
Utility DoFn for writing streaming PCollectionViews.
|
| StreamingSideInputDoFnRunner<InputT,OutputT,W extends BoundedWindow> |
Runs a DoFn by constructing the appropriate contexts and passing them in.
|
| StreamUtils |
Utility functions for stream operations.
|
| StringUtils |
Utilities for working with JSON and other human-readable string formats.
|
| Structs |
A collection of static methods for manipulating datastructure representations
transferred via the Dataflow API.
|
| TestCredential |
Fake credential, for use in testing.
|
| TimerInternals.TimerData |
Data about a timer as represented within
TimerInternals. |
| TimerOrElement<ElemT> |
Class representing either a timer, or arbitrary element.
|
| TimerOrElement.TimerOrElementCoder<T> |
Coder that forwards
ByteSizeObserver calls to an underlying element coder. |
| TimeUtil |
A helper class for converting between Dataflow API and SDK time
representations.
|
| Transport |
Helpers for cloud communication.
|
| TriggerRunner<W extends BoundedWindow> |
Executes a trigger within the context provided by
ReduceFnRunner. |
| TriggerRunner.Result |
Result of trigger execution.
|
| Values |
A collection of static methods for manipulating value representations
transfered via the Dataflow API.
|
| ValueWithRecordId<ValueT> |
Immutable struct containing a value as well as a unique id identifying the value.
|
| ValueWithRecordId.ValueWithRecordIdCoder<ValueT> | |
| VarInt |
Variable-length encoding for integers.
|
| WatermarkHold<W extends BoundedWindow> |
Implements the logic needed to hold the output watermark back to emit
values with specific timestamps.
|
| WindowedValue<V> |
An immutable triple of value, timestamp, and windows.
|
| WindowedValue.FullWindowedValueCoder<T> |
Coder for
WindowedValue. |
| WindowedValue.ValueOnlyWindowedValueCoder<T> |
Coder for
WindowedValue. |
| WindowedValue.WindowedValueCoder<T> |
Abstract class for
WindowedValue coder. |
| WindowingStrategy<T,W extends BoundedWindow> |
A
WindowingStrategy describes the windowing behavior for a specific collection of values. |
| Enum | Description |
|---|---|
| StateFetcher.SideInputState |
Indicates the caller's knowledge of whether a particular side input has been computed.
|
| TimeDomain |
TimeDomain specifies whether an operation is based on
timestamps of elements or current "real-world" time as reported while processing. |
| WindowingStrategy.AccumulationMode |
The accumulation modes that can be used with windowing.
|
| Exception | Description |
|---|---|
| IllegalMutationException |
Thrown when a value appears to have been mutated, but that mutation is forbidden.
|
| UserCodeException |
An exception that was thrown in user-code.
|
| Annotation Type | Description |
|---|---|
| SystemDoFnInternal |
Annotation to mark
DoFns as an internal component of the Dataflow SDK. |