Skip navigation links
A B C D E F G H I J K L M N O P R S T U V W X Y 

A

AccumulatingCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
 
accumulatingFiredPanes() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.
accumulatingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.
accumulatingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.
add(Long) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique
Adds a value to the heap, returning whether the value is (large enough to be) in the heap.
add(String, String) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
Register the given string display metadata.
add(String, long) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
Register the given numeric display metadata.
add(String, double) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
Register the given floating point display metadata.
add(String, Instant) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
Register the given timestamp display metadata.
add(String, Duration) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
Register the given duration display metadata.
add(String, Class<?>) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
Register the given class display metadata.
addCollectionToSingletonOutput(String, PValue, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an output with the given name to the previously added CollectionToSingleton Dataflow step, consuming the specified input PValue and producing the specified output PValue.
addComposite(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Adds a composite operation to the transform node.
addEncodingInput(Coder<?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Sets the encoding for the current Dataflow step.
addExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
addExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
addInput(String, Boolean) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input with the given name and value to the current Dataflow step.
addInput(String, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input with the given name and value to the current Dataflow step.
addInput(String, Long) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input with the given name and value to the current Dataflow step.
addInput(String, PInput) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input with the given name to the previously added Dataflow step, coming from the specified input PValue.
addInput(String, Map<String, Object>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input that is a dictionary of strings to objects.
addInput(String, List<? extends Map<String, Object>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input that is a list of objects.
addInput(TransformTreeNode, PInput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Adds an input to the given node.
addInput(ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique, T) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
addInput(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn.Accumulator
Adds the given input value to this accumulator, modifying this accumulator.
addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
 
addInput(double[], Double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
addInput(Combine.Holder<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
addInput(int[], Integer) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
addInput(long[], Long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Adds the given input value to the given accumulator, returning the new accumulator value.
addInput(List<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
 
addInput(K, AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Adds the given input value to the given accumulator, returning the new accumulator value.
addInput(Object[], DataT) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
 
addInput(Object[], DataT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
 
addInput(K, Object[], DataT) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
 
addInput(K, Object[], DataT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
 
addInput(AccumT, InputT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
Adds the given input value to the given accumulator, returning the new accumulator value.
addInput(K, AccumT, InputT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
Adds the given input value to the given accumulator, returning the new accumulator value.
addInput(Top.BoundedHeap<KV<Integer, T>, SerializableComparator<KV<Integer, T>>>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
addInputProducer(PValue, TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Adds an input to the transform node.
addInputsRandomly(PerKeyCombineFnRunner<? super K, ? super InputT, AccumT, ?>, K, Iterable<InputT>, Random, DoFn<?, ?>.ProcessContext) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
Create a random list of accumulators from the given list of values.
addOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.
addOverrideForClass(Class<?>, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
Overrides the default log level for the passed in class.
addOverrideForName(String, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
Overrides the default log level for the passed in name.
addOverrideForPackage(Package, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
Overrides the default log level for the passed in package.
addStep(PTransform<?, ?>, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds a step to the Dataflow workflow for the given transform, with the given Dataflow step type.
addStep(PTransform<?, ? extends PValue>, Step) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds a pre-defined step to the Dataflow workflow.
addTraceFor(AbstractGoogleClient, String) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
Creates a GoogleApiDebugOptions.GoogleApiTracer that sets the trace destination on all calls that match the given client type.
addTraceFor(AbstractGoogleClientRequest<?>, String) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
Creates a GoogleApiDebugOptions.GoogleApiTracer that sets the trace traceDestination on all calls that match for the given request type.
addValue(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator
Adds a new value into the Aggregator.
addValueInternal(PValue) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Adds the given PValue to this Pipeline.
addValueOnlyOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.
advance() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
advance() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
 
advance() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
Advances the reader to the next valid record.
advance() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Advances the reader to the next valid record.
advanceImpl() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
 
advanceImpl() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
Advances to the next record and returns true, or returns false if there is no next record.
AfterAll<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Create a Trigger that fires and finishes once after all of its sub-triggers have fired.
AfterDelayFromFirstElement<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A base class for triggers that happen after a processing time delay from the arrival of the first element in a pane.
AfterDelayFromFirstElement(TimeDomain, List<SerializableFunction<Instant, Instant>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
AfterEach<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A composite Trigger that executes its sub-triggers in order.
AfterFirst<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Create a composite Trigger that fires once after at least one of its sub-triggers have fired.
AfterPane<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Triggers that fire based on properties of the elements in the current pane.
AfterProcessingTime<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
AfterProcessingTime triggers fire based on the current processing time.
AfterWatermark<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
AfterWatermark triggers fire based on progress of the system watermark.
AfterWatermark.AfterWatermarkEarly<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
Interface for building an AfterWatermarkTrigger with early firings already filled in.
AfterWatermark.AfterWatermarkLate<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
Interface for building an AfterWatermarkTrigger with late firings already filled in.
AfterWatermark.FromEndOfWindow<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A watermark trigger targeted relative to the end of the window.
Aggregator<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
An Aggregator<InputT> enables monitoring of values of type InputT, to be combined across all bundles.
AggregatorPipelineExtractor - Class in com.google.cloud.dataflow.sdk.runners
Retrieves Aggregators at each ParDo and returns a Map of Aggregator to the PTransforms in which it is present.
AggregatorPipelineExtractor(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorPipelineExtractor
Creates an AggregatorPipelineExtractor for the given Pipeline.
AggregatorRetrievalException - Exception in com.google.cloud.dataflow.sdk.runners
Signals that an exception has occurred while retrieving Aggregators.
AggregatorRetrievalException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.AggregatorRetrievalException
Constructs a new AggregatorRetrievalException with the specified detail message and cause.
AggregatorRetriever - Class in com.google.cloud.dataflow.sdk.transforms
An internal class for extracting Aggregators from DoFns.
AggregatorValues<T> - Class in com.google.cloud.dataflow.sdk.runners
A collection of values associated with an Aggregator.
AggregatorValues() - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorValues
 
alignedTo(Duration, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
Aligns timestamps to the smallest multiple of size since the offset greater than the timestamp.
alignedTo(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
Aligns the time to be the smallest multiple of size greater than the timestamp since the epoch.
ALL_CONTEXTS - Static variable in class com.google.cloud.dataflow.sdk.testing.CoderProperties
All the contexts, for use in test cases.
ALL_KEYS - Static variable in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
The range of all keys, with empty start and end keys.
allowsDynamicSplitting() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
Whether this source should allow dynamic splitting of the offset ranges.
and(TupleTag<V>, List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns a new CoGbkResult based on this, with the given tag and given data added to it.
and(TupleTag<V>, PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns a new KeyedPCollectionTuple<K> that is the same as this, appended with the given PCollection.
and(PCollection.IsBounded) - Method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded
Returns the composed IsBounded property.
and(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollection appended to the end.
and(Iterable<PCollection<T>>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollections appended to the end, in order.
and(TupleTag<T>, PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns a new PCollectionTuple that has each PCollection and TupleTag of this PCollectionTuple plus the given PCollection associated with the given TupleTag.
and(TupleTag<?>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns a new TupleTagList that has all the TupleTags of this TupleTagList plus the given TupleTag appended to the end.
and(List<TupleTag<?>>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns a new TupleTagList that has all the TupleTags of this TupleTagList plus the given TupleTags appended to the end, in order.
any(long) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample
Sample#any(long) takes a PCollection<T> and a limit, and produces a new PCollection<T> containing up to limit elements of the input PCollection.
applicableTo(PCollection<?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
 
ApplicationNameOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that allow setting the application name.
AppliedPTransform<InputT extends PInput,OutputT extends POutput,TransformT extends PTransform<? super InputT,OutputT>> - Class in com.google.cloud.dataflow.sdk.transforms
Represents the application of a PTransform to a specific input to produce a specific output.
apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.coders.DelegateCoder.CodingFunction
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
 
apply(PCollection<TableRow>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
 
apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
 
apply(PCollection<KV<ByteString, Iterable<Mutation>>>) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
 
apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.BoundedCountingInput
 
apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.Write.Bound
 
apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Like Pipeline.apply(String, PTransform) but the transform node in the Pipeline graph will be named according to PTransform.getName().
apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Adds a root PTransform, such as Read or Create, to this Pipeline.
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
 
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Applies the given transform to the input.
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
 
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
 
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner
Applies a transform to the given input, returning the output.
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
 
apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
Applies the binary operation to the two operands, returning the result.
apply(V, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
Applies the binary operation to the two operands, returning the result.
apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
Applies the binary operation to the two operands, returning the result.
apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
Applies the binary operation to the two operands, returning the result.
apply(Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Applies this CombineFn to a collection of input values to produce a combined output value.
apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
 
apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView
 
apply(PCollection<? extends KV<K, ? extends Iterable<InputT>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
 
apply(K, Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Applies this KeyedCombineFn to a key and a collection of input values to produce a combined output value.
apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
 
apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKeyWithHotKeyFanout
 
apply(K, Iterable<? extends InputT>, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
Applies this KeyedCombineFnWithContext to a key and a collection of input values to produce a combined output value.
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Count.PerElement
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.TimestampedValues
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Filter
 
apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements
 
apply(PCollection<? extends Iterable<T>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenIterables
 
apply(PCollectionList<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenPCollectionList
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
 
apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupAlsoByWindow
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.ReifyTimestampsAndWindows
 
apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.SortValuesByTimestamp
 
apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound
 
apply(KeyedPCollectionTuple<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGroupByKey
 
apply(PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Like KeyedPCollectionTuple.apply(String, PTransform) but defaulting to the name provided by the PTransform.
apply(String, PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Applies the given PTransform to this input KeyedPCollectionTuple and returns its OutputT.
apply(PCollection<? extends KV<K, ?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Keys
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.KvSwap
 
apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.MapElements
 
apply(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
 
apply(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
 
apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Partition
 
apply(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Applies this PTransform on the given InputT, and returns its Output.
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates.WithRepresentativeValues
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.SampleAny
 
apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.SerializableFunction
Returns the result of invoking this function on the given input.
apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
 
apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
 
apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
 
apply(PCollection<? extends KV<?, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Values
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsIterable
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsList
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMultimap
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
 
apply(PCollection<ElemT>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.CreatePCollectionView
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Remerge
 
apply(PCollection<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithKeys
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithTimestamps
 
apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin
Like PBegin.apply(String, PTransform) but defaulting to the name of the PTransform.
apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin
Applies the given PTransform to this input PBegin, using name to identify this specific application of the transform.
apply(PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Like PCollection.apply(String, PTransform) but defaulting to the name of the PTransform.
apply(String, PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Applies the given PTransform to this input PCollection, using name to identify this specific application of the transform.
apply(PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Like PCollectionList.apply(String, PTransform) but defaulting to the name of the PTransform.
apply(String, PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Applies the given PTransform to this input PCollectionList, using name to identify this specific application of the transform.
apply(PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Like PCollectionTuple.apply(String, PTransform) but defaulting to the name of the PTransform.
apply(String, PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Applies the given PTransform to this input PCollectionTuple, using name to identify this specific application of the transform.
applyTransform(InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline
Like Pipeline.applyTransform(String, PInput, PTransform) but defaulting to the name provided by the PTransform.
applyTransform(String, InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline
Applies the given PTransform to this input InputT and returns its OutputT.
ApproximateQuantiles - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for getting an idea of a PCollection's data distribution using approximate N-tiles (e.g.
ApproximateQuantiles.ApproximateQuantilesCombineFn<T,ComparatorT extends Comparator<T> & Serializable> - Class in com.google.cloud.dataflow.sdk.transforms
The ApproximateQuantilesCombineFn combiner gives an idea of the distribution of a collection of values using approximate N-tiles.
ApproximateUnique - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for estimating the number of distinct elements in a PCollection, or the number of distinct values associated with each key in a PCollection of KVs.
ApproximateUnique() - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
 
ApproximateUnique.ApproximateUniqueCombineFn<T> - Class in com.google.cloud.dataflow.sdk.transforms
CombineFn that computes an estimate of the number of distinct values that were combined.
ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique - Class in com.google.cloud.dataflow.sdk.transforms
A heap utility class to efficiently track the largest added elements.
ApproximateUniqueCombineFn(long, Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
areAllSubtriggersFinished() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Returns true if all the sub-triggers of the current trigger are marked finished.
as(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
Transforms this object into an object of type <T> saving each property that has been manipulated.
as(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Creates and returns an object that implements <T>.
as(Class<T>) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
Creates and returns an object that implements <T> using the values configured on this builder during construction.
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
asCloudObject() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Returns the CloudObject that represents this Coder.
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
Returns the CloudObject that represents this Coder.
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
 
asIterable() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns a View.AsIterable transform that takes a PCollection as input and produces a PCollectionView mapping each window to an Iterable of the values in that window.
asKeyedFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
 
asKeyedFn() - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
Converts this GloballyCombineFn into an equivalent CombineFnBase.PerKeyCombineFn that ignores the keys passed to it and combines the values according to this GloballyCombineFn.
asKeyedFn() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
 
asList() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns a View.AsList transform that takes a PCollection and returns a PCollectionView mapping each window to a List containing all of the elements in the window.
asMap() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData
 
asMap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns a View.AsMap transform that takes a PCollection<KV<K V>> as input and produces a PCollectionView mapping each window to a Map>K, V>.
asMultimap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns a View.AsMultimap transform that takes a PCollection<KV<K, V>> as input and produces a PCollectionView mapping each window to its contents as a Map<K, Iterable<V>> for use as a side input.
asOutputReference(PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Encode a PValue reference as an output reference.
asPath() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
Returns the string representation of this subscription as a path used in the Cloud Pub/Sub API.
asPath() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
Returns the string representation of this topic as a path used in the Cloud Pub/Sub API.
assertSourcesEqualReferenceSource(BoundedSource<T>, List<? extends BoundedSource<T>>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Given a reference Source and a list of Sources, assert that the union of the records read from the list of sources is equal to the records read from the reference source.
assertSplitAtFractionBehavior(BoundedSource<T>, int, double, SourceTestUtils.ExpectedSplitOutcome, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Asserts that the source's reader either fails to splitAtFraction(fraction) after reading numItemsToReadBeforeSplit items, or succeeds in a way that is consistent according to SourceTestUtils.assertSplitAtFractionSucceedsAndConsistent(com.google.cloud.dataflow.sdk.io.BoundedSource<T>, int, double, com.google.cloud.dataflow.sdk.options.PipelineOptions).
assertSplitAtFractionExhaustive(BoundedSource<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Asserts that for each possible start position, BoundedSource.BoundedReader#splitAtFraction at every interesting fraction (halfway between two fractions that differ by at least one item) can be called successfully and the results are consistent if a split succeeds.
assertSplitAtFractionFails(BoundedSource<T>, int, double, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Asserts that the source's reader fails to splitAtFraction(fraction) after reading numItemsToReadBeforeSplit items.
assertSplitAtFractionSucceedsAndConsistent(BoundedSource<T>, int, double, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Verifies some consistency properties of BoundedSource.BoundedReader#splitAtFraction on the given source.
assertUnstartedReaderReadsSameAsItsSource(BoundedSource.BoundedReader<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Assert that a Reader returns a Source that, when read from, produces the same records as the reader.
AssignContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
 
assignedWindows(WindowFn<T, W>, long) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
 
assignMessageTimestamp(PubsubMessage, String, Clock) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO
Returns the Instant that corresponds to the timestamp in the supplied PubsubMessage under the specified ink label.
assignOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn
Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.
assignOutputTime(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
Returns the output timestamp to use for data in the specified window.
assignOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.
assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
 
assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns true if this WindowFn assigns each element to a single window.
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
Returns the single window to which elements with this timestamp belong.
assignWindows(WindowFn<Object, GlobalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
assignWindows(WindowFn<Object, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
 
assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Given a timestamp and element, returns the set of windows into which it should be placed.
asSingleton() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns a View.AsSingleton transform that takes a PCollection with a single value per window as input and produces a PCollectionView that returns the value in the main input window when read as a side input.
asSingletonView() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
Returns a PTransform that produces a PCollectionView whose elements are the result of combining elements per-window in the input PCollection.
asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
Deprecated.
the v1beta1 API for Cloud Pub/Sub is deprecated.
asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
Deprecated.
the v1beta1 API for Cloud Pub/Sub is deprecated.
asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
Deprecated.
the v1beta2 API for Cloud Pub/Sub is deprecated.
asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
Deprecated.
the v1beta2 API for Cloud Pub/Sub is deprecated.
AtomicCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A Coder that has no component Coders or other state.
AtomicCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.AtomicCoder
 
AvroCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A Coder using Avro binary format.
AvroCoder(Class<T>, Schema) - Constructor for class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
AvroIO - Class in com.google.cloud.dataflow.sdk.io
PTransforms for reading and writing Avro files.
AvroIO.Read - Class in com.google.cloud.dataflow.sdk.io
A root PTransform that reads from an Avro file (or multiple Avro files matching a pattern) and returns a PCollection containing the decoding of each record.
AvroIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from an Avro file (or multiple Avro files matching a pattern) and returns a bounded PCollection containing the decoding of each record.
AvroIO.Write - Class in com.google.cloud.dataflow.sdk.io
A root PTransform that writes a PCollection to an Avro file (or multiple Avro files matching a sharding pattern).
AvroIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes a bounded PCollection to an Avro file (or multiple Avro files matching a sharding pattern).
AvroReader(AvroSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
Reads Avro records of type T from the specified source.
AvroSource<T> - Class in com.google.cloud.dataflow.sdk.io
A FileBasedSource for reading Avro files.
AvroSource.AvroReader<T> - Class in com.google.cloud.dataflow.sdk.io
A BlockBasedSource.BlockBasedReader for reading blocks from Avro files.

B

BACKLOG_UNKNOWN - Static variable in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Constant representing an unknown amount of backlog.
baseOutputFilename - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink
Base filename for final output files.
baseTemporaryFilename - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Base filename used for temporary output files.
BATCH_WORKER_HARNESS_CONTAINER_IMAGE - Static variable in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
 
begin() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Returns a PBegin owned by this Pipeline.
beginningOnDay(int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
beginningOnDay(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
BigEndianIntegerCoder - Class in com.google.cloud.dataflow.sdk.coders
A BigEndianIntegerCoder encodes Integers in 4 bytes, big-endian.
BigEndianLongCoder - Class in com.google.cloud.dataflow.sdk.coders
A BigEndianLongCoder encodes Longs in 8 bytes, big-endian.
BigQueryIO - Class in com.google.cloud.dataflow.sdk.io
PTransforms for reading and writing BigQuery tables.
BigQueryIO.Read - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from a BigQuery table and returns a PCollection of TableRows containing each of the rows of the table.
BigQueryIO.Read.Bound - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from a BigQuery table and returns a bounded PCollection of TableRows.
BigQueryIO.Write - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes a PCollection containing TableRows to a BigQuery table.
BigQueryIO.Write.Bound - Class in com.google.cloud.dataflow.sdk.io
A PTransform that can write either a bounded or unbounded PCollection of TableRows to a BigQuery table.
BigQueryIO.Write.CreateDisposition - Enum in com.google.cloud.dataflow.sdk.io
An enumeration type for the BigQuery create disposition strings.
BigQueryIO.Write.WriteDisposition - Enum in com.google.cloud.dataflow.sdk.io
An enumeration type for the BigQuery write disposition strings.
BigQueryOptions - Interface in com.google.cloud.dataflow.sdk.options
Properties needed when using BigQuery with the Dataflow SDK.
BigtableIO - Class in com.google.cloud.dataflow.sdk.io.bigtable
A bounded source and sink for Google Cloud Bigtable.
BigtableIO.Read - Class in com.google.cloud.dataflow.sdk.io.bigtable
A PTransform that reads from Google Cloud Bigtable.
BigtableIO.Write - Class in com.google.cloud.dataflow.sdk.io.bigtable
A PTransform that writes to Google Cloud Bigtable.
BinaryCombineDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
BinaryCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
BinaryCombineIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
BinaryCombineLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
Block() - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
 
BlockBasedReader(BlockBasedSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
 
BlockBasedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A BlockBasedSource is a FileBasedSource where a file consists of blocks of records.
BlockBasedSource(String, long) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource
Creates a BlockBasedSource based on a file name or pattern.
BlockBasedSource(String, long, long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource
Creates a BlockBasedSource for a single file.
BlockBasedSource.Block<T> - Class in com.google.cloud.dataflow.sdk.io
A Block represents a block of records that can be read.
BlockBasedSource.BlockBasedReader<T> - Class in com.google.cloud.dataflow.sdk.io
A Reader that reads records from a BlockBasedSource.
BlockingDataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that are used to configure the BlockingDataflowPipelineRunner.
BlockingDataflowPipelineOptions.StandardOutputFactory - Class in com.google.cloud.dataflow.sdk.options
Returns a default of System.out.
BlockingDataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners
A PipelineRunner that's like DataflowPipelineRunner but that waits for the launched job to finish.
BlockingDataflowPipelineRunner(DataflowPipelineRunner, BlockingDataflowPipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
 
Bound() - Constructor for class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Deprecated.
Should be private. Instead, use one of the factory methods in BigQueryIO.Write, such as BigQueryIO.Write.to(String), to create an instance of this class.
BoundedReader() - Constructor for class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
 
BoundedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A Source that reads a finite amount of input and, because of that, supports some additional operations.
BoundedSource() - Constructor for class com.google.cloud.dataflow.sdk.io.BoundedSource
 
BoundedSource.BoundedReader<T> - Class in com.google.cloud.dataflow.sdk.io
A Reader that reads a bounded amount of input and supports some additional operations, such as progress estimation and dynamic work rebalancing.
BoundedWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A BoundedWindow represents a finite grouping of elements, with an upper bound (larger timestamps represent more recent data) on the timestamps of elements that can be placed in the window.
BoundedWindow() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
 
buildTemporaryFilename(String, String) - Static method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Build a temporary filename using the temporary filename separator with the given prefix and suffix.
buildTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
buildTrigger() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.TriggerBuilder
Return the Trigger built by this builder.
by(PredicateT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Deprecated.
use Filter.byPredicate(PredicateT), which returns a Filter transform instead of a ParDo.Bound.
byPredicate(PredicateT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that satisfy the given predicate.
ByteArrayCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder for byte[].
ByteCoder - Class in com.google.cloud.dataflow.sdk.coders
A ByteCoder encodes Byte values in 1 byte using Java serialization.
ByteKey - Class in com.google.cloud.dataflow.sdk.io.range
A class representing a key consisting of an array of bytes.
ByteKeyRange - Class in com.google.cloud.dataflow.sdk.io.range
A class representing a range of ByteKeys.
ByteKeyRangeTracker - Class in com.google.cloud.dataflow.sdk.io.range
ByteStringCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder for ByteString objects based on their encoded Protocol Buffer form.

C

CalendarWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A collection of WindowFns that windows values into calendar-based windows such as spans of days, months, or years.
CalendarWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
 
CalendarWindows.DaysWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows elements into periods measured by days.
CalendarWindows.MonthsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows elements into periods measured by months.
CalendarWindows.YearsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows elements into periods measured by years.
cancel() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Cancels the job.
CannotProvideCoderException - Exception in com.google.cloud.dataflow.sdk.coders
The exception thrown when a CoderProvider cannot provide a Coder that has been requested.
CannotProvideCoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
CannotProvideCoderException(String, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
CannotProvideCoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
CannotProvideCoderException(String, Throwable, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
CannotProvideCoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
CannotProvideCoderException(Throwable, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
CannotProvideCoderException.ReasonCode - Enum in com.google.cloud.dataflow.sdk.coders
Indicates the reason that Coder inference failed.
clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Clear any state associated with this trigger in the given window.
clearOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Clears the record of the elements output so far to the main output.
clearSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Clears the record of the elements output so far to the side output with the given tag.
cloneAs(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
Makes a deep clone of this object, and transforms the cloned object into the specified type kls.
close() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Closes the channel and return the bundle result.
close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
Closes any ReadableByteChannel created for the current reader.
close() - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
Finishes writing the bundle.
close() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
Closes the reader.
CloudDebuggerOptions - Interface in com.google.cloud.dataflow.sdk.options
Options for controlling Cloud Debugger.
Coder<T> - Interface in com.google.cloud.dataflow.sdk.coders
A Coder<T> defines how to encode and decode values of type T into byte streams.
Coder.Context - Class in com.google.cloud.dataflow.sdk.coders
The context in which encoding or decoding is being done.
Coder.NonDeterministicException - Class in com.google.cloud.dataflow.sdk.coders
Exception thrown by Coder.verifyDeterministic() if the encoding is not deterministic, including details of why the encoding is not deterministic.
coderAllowsEncoding(Coder<T>, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderConsistentWithEquals(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderConsistentWithEqualsInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderDecodeEncodeContentsEqual(Coder<CollectionT>, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in any Coder.Context.
coderDecodeEncodeContentsEqualInContext(Coder<CollectionT>, Coder.Context, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in the given Coder.Context.
coderDecodeEncodeContentsInSameOrder(Coder<IterableT>, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in any Coder.Context.
coderDecodeEncodeContentsInSameOrderInContext(Coder<IterableT>, Coder.Context, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<Iterable<T>>, and value of type Iterable<T>, encoding followed by decoding yields an equal value of type Collection<T>, in the given Coder.Context.
coderDecodeEncodeEqual(Coder<T>, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<T>, and value of type T, encoding followed by decoding yields an equal value of type T, in any Coder.Context.
coderDecodeEncodeEqualInContext(Coder<T>, Coder.Context, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<T>, Coder.Context, and value of type T, encoding followed by decoding yields an equal value of type T.
coderDecodesBase64(Coder<T>, String, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderDecodesBase64(Coder<T>, List<String>, List<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderDecodesBase64ContentsEqual(Coder<IterableT>, String, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderDecodesBase64ContentsEqual(Coder<IterableT>, List<String>, List<IterableT>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderDeterministic(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<T>, and values of type T, if the values are equal then the encoded bytes are equal, in any Coder.Context.
coderDeterministicInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<T>, Coder.Context, and values of type T, if the values are equal then the encoded bytes are equal.
coderEncodesBase64(Coder<T>, T, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderEncodesBase64(Coder<T>, List<T>, List<String>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
CoderException - Exception in com.google.cloud.dataflow.sdk.coders
An Exception thrown if there is a problem encoding or decoding a value.
CoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException
 
CoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException
 
CoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException
 
CoderFactories - Class in com.google.cloud.dataflow.sdk.coders
Static utility methods for creating and working with Coders.
CoderFactory - Interface in com.google.cloud.dataflow.sdk.coders
A CoderFactory creates coders and decomposes values.
coderHasEncodingId(Coder<T>, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
CoderProperties - Class in com.google.cloud.dataflow.sdk.testing
Properties for use in Coder tests.
CoderProperties() - Constructor for class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
CoderProvider - Interface in com.google.cloud.dataflow.sdk.coders
A CoderProvider may create a Coder for any concrete class.
coderProvider() - Static method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
 
coderProvider() - Static method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
A CoderProvider that returns a ProtoCoder with an empty ExtensionRegistry.
CoderProviders - Class in com.google.cloud.dataflow.sdk.coders
Static utility methods for working with CoderProviders.
CoderRegistry - Class in com.google.cloud.dataflow.sdk.coders
A CoderRegistry allows registering the default Coder to use for a Java class, and looking up and instantiating the default Coder for a Java type.
CoderRegistry() - Constructor for class com.google.cloud.dataflow.sdk.coders.CoderRegistry
 
coderSerializable(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
CoGbkResult - Class in com.google.cloud.dataflow.sdk.transforms.join
A row result of a CoGroupByKey.
CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
A row in the PCollection resulting from a CoGroupByKey transform.
CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>, int) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
 
CoGbkResult.CoGbkResultCoder - Class in com.google.cloud.dataflow.sdk.transforms.join
CoGbkResultSchema - Class in com.google.cloud.dataflow.sdk.transforms.join
A schema for the results of a CoGroupByKey.
CoGbkResultSchema(TupleTagList) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Builds a schema from a tuple of TupleTag<?>s.
CoGroupByKey<K> - Class in com.google.cloud.dataflow.sdk.transforms.join
A PTransform that performs a CoGroupByKey on a tuple of tables.
CollectionCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A CollectionCoder encodes Collections in the format of IterableLikeCoder.
CollectionCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.CollectionCoder
 
com.google.cloud.dataflow.sdk - package com.google.cloud.dataflow.sdk
Provides a simple, powerful model for building both batch and streaming parallel data processing Pipelines.
com.google.cloud.dataflow.sdk.annotations - package com.google.cloud.dataflow.sdk.annotations
Defines annotations used across the SDK.
com.google.cloud.dataflow.sdk.coders - package com.google.cloud.dataflow.sdk.coders
Defines Coders to specify how data is encoded to and decoded from byte strings.
com.google.cloud.dataflow.sdk.coders.protobuf - package com.google.cloud.dataflow.sdk.coders.protobuf
Defines a Coder for Protocol Buffers messages, ProtoCoder.
com.google.cloud.dataflow.sdk.io - package com.google.cloud.dataflow.sdk.io
Defines transforms for reading and writing common storage formats, including AvroIO, BigQueryIO, and TextIO.
com.google.cloud.dataflow.sdk.io.bigtable - package com.google.cloud.dataflow.sdk.io.bigtable
Defines transforms for reading and writing from Google Cloud Bigtable.
com.google.cloud.dataflow.sdk.io.range - package com.google.cloud.dataflow.sdk.io.range
Provides thread-safe helpers for implementing dynamic work rebalancing in position-based bounded sources.
com.google.cloud.dataflow.sdk.options - package com.google.cloud.dataflow.sdk.options
Defines PipelineOptions for configuring pipeline execution.
com.google.cloud.dataflow.sdk.runners - package com.google.cloud.dataflow.sdk.runners
Defines runners for executing Pipelines in different modes, including DirectPipelineRunner and DataflowPipelineRunner.
com.google.cloud.dataflow.sdk.testing - package com.google.cloud.dataflow.sdk.testing
Defines utilities for unit testing Dataflow pipelines.
com.google.cloud.dataflow.sdk.transforms - package com.google.cloud.dataflow.sdk.transforms
Defines PTransforms for transforming data in a pipeline.
com.google.cloud.dataflow.sdk.transforms.display - package com.google.cloud.dataflow.sdk.transforms.display
 
com.google.cloud.dataflow.sdk.transforms.join - package com.google.cloud.dataflow.sdk.transforms.join
Defines the CoGroupByKey transform for joining multiple PCollections.
com.google.cloud.dataflow.sdk.transforms.windowing - package com.google.cloud.dataflow.sdk.transforms.windowing
Defines the Window transform for dividing the elements in a PCollection into windows, and the Trigger for controlling when those elements are output.
com.google.cloud.dataflow.sdk.values - package com.google.cloud.dataflow.sdk.values
Defines PCollection and other classes for representing data in a Pipeline.
Combine - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for combining PCollection elements globally and per-key.
combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn
Combines the given output times, which must be from the same window, into an output time for a computed value.
combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
Combines the given output times, which must be from the same window, into an output time for a computed value.
combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
Combines the given output times, which must be from the same window, into an output time for a computed value.
Combine.AccumulatingCombineFn<InputT,AccumT extends Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT>,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A CombineFn that uses a subclass of Combine.AccumulatingCombineFn.Accumulator as its accumulator type.
Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
The type of mutable accumulator values used by this AccumulatingCombineFn.
Combine.BinaryCombineDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
An abstract subclass of Combine.CombineFn for implementing combiners that are more easily and efficiently expressed as binary operations on doubles.
Combine.BinaryCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms
An abstract subclass of Combine.CombineFn for implementing combiners that are more easily expressed as binary operations.
Combine.BinaryCombineIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
An abstract subclass of Combine.CombineFn for implementing combiners that are more easily and efficiently expressed as binary operations on ints
Combine.BinaryCombineLongFn - Class in com.google.cloud.dataflow.sdk.transforms
An abstract subclass of Combine.CombineFn for implementing combiners that are more easily and efficiently expressed as binary operations on longs.
Combine.CombineFn<InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A CombineFn<InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT into a single output value of type OutputT.
Combine.Globally<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
Combine.Globally<InputT, OutputT> takes a PCollection<InputT> and returns a PCollection<OutputT> whose elements are the result of combining all the elements in each window of the input PCollection, using a specified CombineFn<InputT, AccumT, OutputT>.
Combine.GloballyAsSingletonView<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
Combine.GloballyAsSingletonView<InputT, OutputT> takes a PCollection<InputT> and returns a PCollectionView<OutputT> whose elements are the result of combining all the elements in each window of the input PCollection, using a specified CombineFn<InputT, AccumT, OutputT>.
Combine.GroupedValues<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
GroupedValues<K, InputT, OutputT> takes a PCollection<KV<K, Iterable<InputT>>>, such as the result of GroupByKey, applies a specified KeyedCombineFn<K, InputT, AccumT, OutputT> to each of the input KV<K, Iterable<InputT>> elements to produce a combined output KV<K, OutputT> element, and returns a PCollection<KV<K, OutputT>> containing all the combined output elements.
Combine.Holder<V> - Class in com.google.cloud.dataflow.sdk.transforms
Holds a single value value of type V which may or may not be present.
Combine.IterableCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms
Converts a SerializableFunction from Iterable<V>s to Vs into a simple Combine.CombineFn over Vs.
Combine.KeyedCombineFn<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A KeyedCombineFn<K, InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT, associated with a key of type K, into a single output value of type OutputT.
Combine.PerKey<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
PerKey<K, InputT, OutputT> takes a PCollection<KV<K, InputT>>, groups it by key, applies a combining function to the InputT values associated with each key to produce a combined OutputT value, and returns a PCollection<KV<K, OutputT>> representing a map from each distinct key of the input PCollection to the corresponding combined value.
Combine.PerKeyWithHotKeyFanout<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
Like Combine.PerKey, but sharding the combining of hot keys.
Combine.SimpleCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms
Deprecated.
CombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
 
CombineFnBase - Class in com.google.cloud.dataflow.sdk.transforms
This class contains the shared interfaces and abstract classes for different types of combine functions.
CombineFnBase() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFnBase
 
CombineFnBase.GlobalCombineFn<InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
A GloballyCombineFn<InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT into a single output value of type OutputT.
CombineFnBase.PerKeyCombineFn<K,InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
A PerKeyCombineFn<K, InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT, associated with a key of type K, into a single output value of type OutputT.
CombineFns - Class in com.google.cloud.dataflow.sdk.transforms
Static utility methods that create combine function instances.
CombineFns() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns
 
CombineFns.CoCombineResult - Class in com.google.cloud.dataflow.sdk.transforms
A tuple of outputs produced by a composed combine functions.
CombineFns.ComposeCombineFnBuilder - Class in com.google.cloud.dataflow.sdk.transforms
A builder class to construct a composed CombineFnBase.GlobalCombineFn.
CombineFns.ComposedCombineFn<DataT> - Class in com.google.cloud.dataflow.sdk.transforms
A composed Combine.CombineFn that applies multiple CombineFns.
CombineFns.ComposedCombineFnWithContext<DataT> - Class in com.google.cloud.dataflow.sdk.transforms
CombineFns.ComposedKeyedCombineFn<DataT,K> - Class in com.google.cloud.dataflow.sdk.transforms
A composed Combine.KeyedCombineFn that applies multiple KeyedCombineFns.
CombineFns.ComposedKeyedCombineFnWithContext<DataT,K> - Class in com.google.cloud.dataflow.sdk.transforms
CombineFns.ComposeKeyedCombineFnBuilder - Class in com.google.cloud.dataflow.sdk.transforms
A builder class to construct a composed CombineFnBase.PerKeyCombineFn.
CombineFnWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
 
combineOutputTimes(OutputTimeFn<?>, Iterable<? extends Instant>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFns
Applies the given OutputTimeFn to the given output times, obtaining the output time for a value computed.
CombineWithContext - Class in com.google.cloud.dataflow.sdk.transforms
This class contains combine functions that have access to PipelineOptions and side inputs through CombineWithContext.Context.
CombineWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext
 
CombineWithContext.CombineFnWithContext<InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A combine function that has access to PipelineOptions and side inputs through CombineWithContext.Context.
CombineWithContext.Context - Class in com.google.cloud.dataflow.sdk.transforms
Information accessible to all methods in CombineFnWithContext and KeyedCombineFnWithContext.
CombineWithContext.KeyedCombineFnWithContext<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A keyed combine function that has access to PipelineOptions and side inputs through CombineWithContext.Context.
CombineWithContext.RequiresContextInternal - Interface in com.google.cloud.dataflow.sdk.transforms
An internal interface for signaling that a GloballyCombineFn or a PerKeyCombineFn needs to access CombineWithContext.Context.
compact(AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.
compact(List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
 
compact(K, AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.
compact(Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
 
compact(Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
 
compact(K, Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
 
compact(K, Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
 
compact(AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.
compact(K, AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.
compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Largest
 
compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Smallest
 
compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByKey
 
compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByValue
 
compareTo(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
ByteKey implements Comparable<ByteKey> by comparing the arrays in lexicographic order.
compareTo(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
compose() - Static method in class com.google.cloud.dataflow.sdk.transforms.CombineFns
ComposeCombineFnBuilder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeCombineFnBuilder
 
composeKeyed() - Static method in class com.google.cloud.dataflow.sdk.transforms.CombineFns
ComposeKeyedCombineFnBuilder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
 
CompressedReader(CompressedSource<T>, FileBasedSource.FileBasedReader<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Create a CompressedReader from a CompressedSource and delegate reader.
CompressedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A Source that reads from compressed files.
CompressedSource.CompressedReader<T> - Class in com.google.cloud.dataflow.sdk.io
Reader for a CompressedSource.
CompressedSource.CompressionMode - Enum in com.google.cloud.dataflow.sdk.io
Default compression types supported by the CompressedSource.
CompressedSource.DecompressingChannelFactory - Interface in com.google.cloud.dataflow.sdk.io
Factory interface for creating channels that decompress the content of an underlying channel.
computeTargetTimestamp(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
NullableCoder is consistent with equals if the nested Coder is.
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
contains(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns whether this window contains the given window.
containsInAnyOrder(Iterable<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Checks that the Iterable contains the expected elements, in any order.
containsInAnyOrder(T...) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Checks that the Iterable contains the expected elements, in any order.
containsKey(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Returns true if the specified ByteKey is contained within this range.
Context(boolean) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.Context
 
Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.Context
 
Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
 
Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
 
copyFrom(byte[]) - Static method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
Creates a new ByteKey backed by a copy of the specified byte[].
copyToOutputFiles(List<String>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Copy temporary files to final output filenames using the file naming template.
Count - Class in com.google.cloud.dataflow.sdk.transforms
PTransorms to count the elements in a PCollection.
Count.PerElement<T> - Class in com.google.cloud.dataflow.sdk.transforms
Count.PerElement<T> takes a PCollection<T> and returns a PCollection<KV<T, Long>> representing a map from each distinct element of the input PCollection to the number of times that element occurs in the input.
CounterMark(long) - Constructor for class com.google.cloud.dataflow.sdk.io.CountingSource.CounterMark
Creates a checkpoint mark reflecting the last emitted value.
CountingInput - Class in com.google.cloud.dataflow.sdk.io
A PTransform that produces longs.
CountingInput() - Constructor for class com.google.cloud.dataflow.sdk.io.CountingInput
 
CountingInput.BoundedCountingInput - Class in com.google.cloud.dataflow.sdk.io
A PTransform that will produce a specified number of Longs starting from 0.
CountingInput.UnboundedCountingInput - Class in com.google.cloud.dataflow.sdk.io
A PTransform that will produce numbers starting from 0 up to Long.MAX_VALUE.
CountingSource - Class in com.google.cloud.dataflow.sdk.io
A source that produces longs.
CountingSource.CounterMark - Class in com.google.cloud.dataflow.sdk.io
The checkpoint for an unbounded CountingSource is simply the last value produced.
create(List<? extends Coder<?>>) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderFactory
Returns a Coder<?>, given argument coder to use for values of a particular type, given the Coders for each of the type's generic parameter types.
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions.StandardOutputFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.DataflowClientFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.PathValidatorFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.StagerFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions.JobNameFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory
 
create(PipelineOptions) - Method in interface com.google.cloud.dataflow.sdk.options.DefaultValueFactory
Creates a default value for a getter marked with Default.InstanceFactory.
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.CredentialDirFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.DefaultProjectFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.GcpUserCredentialsFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcsOptions.ExecutorServiceFactory
 
create() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
Creates and returns an object that implements PipelineOptions using the values configured on this builder during construction.
create() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Creates and returns an object that implements PipelineOptions.
create(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.Pipeline
Constructs a pipeline from the provided options.
create(DataflowPipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
Creates and returns a new DataflowPipeline instance for tests.
create(Combine.GroupedValues<K, InputT, OutputT>, PCollection<KV<K, Iterable<InputT>>>, boolean, Random) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
 
create() - Static method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
Creates and returns a new test pipeline.
create(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Returns an approximate quantiles combiner with the given compareFn and desired number of quantiles.
create(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Like ApproximateQuantiles.ApproximateQuantilesCombineFn.create(int, Comparator), but sorts values using their natural ordering.
create(int, ComparatorT, long, double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Creates an approximate quantiles combiner with the given compareFn and desired number of quantiles.
Create<T> - Class in com.google.cloud.dataflow.sdk.transforms
Create<T> takes a collection of elements of type T known when the pipeline is constructed and returns a PCollection<T> containing the elements.
Create() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Create
 
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
Returns a GroupByKey<K, V> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGroupByKey
Returns a CoGroupByKey<K> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.Keys
Returns a Keys<K> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.KvSwap
Returns a KvSwap<K, V> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates
Returns a RemoveDuplicates<T> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.Values
Returns a Values<V> PTransform.
Create.TimestampedValues<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that creates a PCollection whose elements have associated timestamps.
Create.Values<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that creates a PCollection from a set of in-memory objects.
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns a new, mutable accumulator value, representing the accumulation of zero input values.
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
 
createAccumulator(K) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Returns a new, mutable accumulator value representing the accumulation of zero input values.
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
 
createAccumulator(CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
 
createAccumulator(K) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
 
createAccumulator(K, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
 
createAccumulator(CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
Returns a new, mutable accumulator value, representing the accumulation of zero input values.
createAccumulator(K, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
Returns a new, mutable accumulator value representing the accumulation of zero input values.
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
 
createAggregator(String, Combine.CombineFn<? super AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Returns an Aggregator with aggregation logic specified by the Combine.CombineFn argument.
createAggregator(String, SerializableFunction<Iterable<AggInputT>, AggInputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Returns an Aggregator with the aggregation logic specified by the SerializableFunction argument.
createAggregator(String, Combine.CombineFn<? super AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns an Aggregator with aggregation logic specified by the Combine.CombineFn argument.
createAggregator(String, SerializableFunction<Iterable<AggInputT>, AggInputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns an Aggregator with the aggregation logic specified by the SerializableFunction argument.
createAggregatorInternal(String, Combine.CombineFn<AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Creates an Aggregator in the DoFn context with the specified name and aggregation logic specified by Combine.CombineFn.
createDatumReader() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Deprecated.
For AvroCoder internal use only.
createDatumWriter() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Deprecated.
For AvroCoder internal use only.
createDecompressingChannel(ReadableByteChannel) - Method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
 
createDecompressingChannel(ReadableByteChannel) - Method in interface com.google.cloud.dataflow.sdk.io.CompressedSource.DecompressingChannelFactory
Given a channel, create a channel that decompresses the content read from the channel.
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource
Creates a BlockBasedSource for the specified range in a single file.
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Creates a CompressedSource for a subrange of a file.
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
Creates and returns a new FileBasedSource of the same type as the current FileBasedSource backed by a given file and an offset range.
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
createForTest() - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
Creates and returns a new DirectPipeline instance for tests.
createForTest() - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Constructs a runner with default properties for testing.
createFromSystemProperties() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Deprecated.
for internal use only
createFromSystemPropertiesInternal() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Creates a set of Dataflow worker harness options based of a set of known system properties.
createPane(boolean, boolean, PaneInfo.Timing) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
 
createPane(boolean, boolean, PaneInfo.Timing, long, long) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
Factory method to create a PaneInfo with the specified parameters.
createPrimitiveOutputInternal(Pipeline, WindowingStrategy<?, ?>, PCollection.IsBounded) - Static method in class com.google.cloud.dataflow.sdk.values.PCollection
Creates and returns a new PCollection for a primitive output.
createReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
Returns a new BoundedSource.BoundedReader that reads from this source.
createReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
createReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
createReader(PipelineOptions, CheckpointMarkT) - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
Create a new UnboundedSource.UnboundedReader to read from this source, resuming from the given checkpoint if present.
createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource
Creates a BlockBasedReader.
createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Creates a FileBasedReader to read a single file.
createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
Creates and returns an instance of a FileBasedReader implementation for the current source assuming the source represents a single file.
createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
createSourceForSubrange(long, long) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
createSourceForSubrange(long, long) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
Returns an OffsetBasedSource for a subrange of the current source.
createStructuralValues(Coder<T>, List<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Testing utilities below depend on standard assertions and matchers to compare elements read by sources.
createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
 
createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink
Return a subclass of FileBasedSink.FileBasedWriteOperation that will manage the write to the sink.
createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink
Returns an instance of a Sink.WriteOperation that can write to this Sink.
createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Clients must implement to return a subclass of FileBasedSink.FileBasedWriter.
createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Creates a new Sink.Writer to write a bundle of the input to the sink.
createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriteOperation
Creates a XmlSink.XmlWriter with a marshaller for the type it will write.
CredentialDirFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcpOptions.CredentialDirFactory
 
currentEventTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
The current event time for the input or null if unknown.
currentProcessingTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
The current processing time.
currentSynchronizedProcessingTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
The current synchronized upstream processing time or null if unknown.
CustomCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
An abstract base class for writing a Coder class that encodes itself via Java serialization.
CustomCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.CustomCoder
 

D

DataflowAssert - Class in com.google.cloud.dataflow.sdk.testing
An assertion on the contents of a PCollection incorporated into the pipeline.
DataflowAssert.IterableAssert<T> - Class in com.google.cloud.dataflow.sdk.testing
An assertion about the contents of a PCollectionView yielding an Iterable<T>.
DataflowAssert.SingletonAssert<T> - Class in com.google.cloud.dataflow.sdk.testing
An assertion about the single value of type T associated with a PCollectionView.
DataflowClientFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.DataflowClientFactory
 
DataflowJobAlreadyExistsException - Exception in com.google.cloud.dataflow.sdk.runners
An exception that is thrown if the unique job name constraint of the Dataflow service is broken because an existing job with the same job name is currently active.
DataflowJobAlreadyExistsException(DataflowPipelineJob, String) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobAlreadyExistsException
Create a new DataflowJobAlreadyExistsException with the specified DataflowPipelineJob and message.
DataflowJobAlreadyUpdatedException - Exception in com.google.cloud.dataflow.sdk.runners
An exception that is thrown if the existing job has already been updated within the Dataflow service and is no longer able to be updated.
DataflowJobAlreadyUpdatedException(DataflowPipelineJob, String) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobAlreadyUpdatedException
Create a new DataflowJobAlreadyUpdatedException with the specified DataflowPipelineJob and message.
DataflowJobCancelledException - Exception in com.google.cloud.dataflow.sdk.runners
Signals that a job run by a BlockingDataflowPipelineRunner was updated during execution.
DataflowJobCancelledException(DataflowPipelineJob, String) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobCancelledException
Create a new DataflowJobAlreadyUpdatedException with the specified DataflowPipelineJob and message.
DataflowJobCancelledException(DataflowPipelineJob, String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobCancelledException
Create a new DataflowJobAlreadyUpdatedException with the specified DataflowPipelineJob, message, and cause.
DataflowJobException - Exception in com.google.cloud.dataflow.sdk.runners
A RuntimeException that contains information about a DataflowPipelineJob.
DataflowJobExecutionException - Exception in com.google.cloud.dataflow.sdk.runners
Signals that a job run by a BlockingDataflowPipelineRunner fails during execution, and provides access to the failed job.
DataflowJobUpdatedException - Exception in com.google.cloud.dataflow.sdk.runners
Signals that a job run by a BlockingDataflowPipelineRunner was updated during execution.
DataflowJobUpdatedException(DataflowPipelineJob, String, DataflowPipelineJob) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobUpdatedException
Create a new DataflowJobUpdatedException with the specified original DataflowPipelineJob, message, and replacement DataflowPipelineJob.
DataflowJobUpdatedException(DataflowPipelineJob, String, DataflowPipelineJob, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobUpdatedException
Create a new DataflowJobUpdatedException with the specified original DataflowPipelineJob, message, replacement DataflowPipelineJob, and cause.
DataflowPipeline - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineDebugOptions - Interface in com.google.cloud.dataflow.sdk.options
Internal.
DataflowPipelineDebugOptions.DataflowClientFactory - Class in com.google.cloud.dataflow.sdk.options
Returns the default Dataflow client built from the passed in PipelineOptions.
DataflowPipelineDebugOptions.PathValidatorFactory - Class in com.google.cloud.dataflow.sdk.options
Creates a PathValidator object using the class specified in DataflowPipelineDebugOptions.getPathValidatorClass().
DataflowPipelineDebugOptions.StagerFactory - Class in com.google.cloud.dataflow.sdk.options
Creates a Stager object using the class specified in DataflowPipelineDebugOptions.getStagerClass().
DataflowPipelineJob - Class in com.google.cloud.dataflow.sdk.runners
A DataflowPipelineJob represents a job submitted to Dataflow using DataflowPipelineRunner.
DataflowPipelineJob(String, String, Dataflow, DataflowAggregatorTransforms) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Constructs the job.
DataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that can be used to configure the DataflowPipeline.
DataflowPipelineOptions.JobNameFactory - Class in com.google.cloud.dataflow.sdk.options
Returns a normalized job name constructed from ApplicationNameOptions.getAppName(), the local system user name (if available), and the current time.
DataflowPipelineRegistrar - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineRegistrar.Options - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineRegistrar.Runner - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners
A PipelineRunner that executes the operations in the pipeline by first translating them to the Dataflow representation using the DataflowPipelineTranslator and then submitting them to a Dataflow service for execution.
DataflowPipelineRunner(DataflowPipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
 
DataflowPipelineRunner.StreamingPubsubIOWrite<T> - Class in com.google.cloud.dataflow.sdk.runners
Specialized implementation for PubsubIO.Write for the Dataflow runner in streaming mode.
DataflowPipelineRunnerHooks - Class in com.google.cloud.dataflow.sdk.runners
An instance of this class can be passed to the DataflowPipelineRunner to add user defined hooks to be invoked at various times during pipeline execution.
DataflowPipelineRunnerHooks() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunnerHooks
 
DataflowPipelineTranslator - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineTranslator knows how to translate Pipeline objects into Cloud Dataflow Service API Jobs.
DataflowPipelineTranslator.JobSpecification - Class in com.google.cloud.dataflow.sdk.runners
The result of a job translation.
DataflowPipelineTranslator.TransformTranslator<TransformT extends PTransform> - Interface in com.google.cloud.dataflow.sdk.runners
A DataflowPipelineTranslator.TransformTranslator knows how to translate a particular subclass of PTransform for the Cloud Dataflow service.
DataflowPipelineTranslator.TranslationContext - Interface in com.google.cloud.dataflow.sdk.runners
The interface provided to registered callbacks for interacting with the DataflowPipelineRunner, including reading and writing the values of PCollections and side inputs (PCollectionViews).
DataflowPipelineWorkerPoolOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that are used to configure the Dataflow pipeline worker pool.
DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType - Enum in com.google.cloud.dataflow.sdk.options
Type of autoscaling algorithm to use.
DataflowPipelineWorkerPoolOptions.TeardownPolicy - Enum in com.google.cloud.dataflow.sdk.options
The policy for tearing down the workers spun up by the service.
DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory - Class in com.google.cloud.dataflow.sdk.options
Returns the default Docker container image that executes Dataflow worker harness, residing in Google Container Registry.
DataflowProfilingAgentConfiguration() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions.DataflowProfilingAgentConfiguration
 
DataflowProfilingOptions - Interface in com.google.cloud.dataflow.sdk.options
Options for controlling profiling of pipeline execution.
DataflowProfilingOptions.DataflowProfilingAgentConfiguration - Class in com.google.cloud.dataflow.sdk.options
Configuration the for profiling agent.
DataflowServiceException - Exception in com.google.cloud.dataflow.sdk.runners
Signals there was an error retrieving information about a job from the Cloud Dataflow Service.
DataflowWorkerHarnessOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that are used exclusively within the Dataflow worker harness.
DataflowWorkerLoggingOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that are used to control logging configuration on the Dataflow worker.
DataflowWorkerLoggingOptions.Level - Enum in com.google.cloud.dataflow.sdk.options
The set of log levels that can be used on the Dataflow worker.
DataflowWorkerLoggingOptions.WorkerLogLevelOverrides - Class in com.google.cloud.dataflow.sdk.options
Defines a log level override for a specific class, package, or name.
DATASTORE_BATCH_UPDATE_LIMIT - Static variable in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Datastore has a limit of 500 mutations per batch operation, so we flush changes to Datastore every 500 entities.
DatastoreIO - Class in com.google.cloud.dataflow.sdk.io
DatastoreIO provides an API to Read and Write PCollections of Google Cloud Datastore DatastoreV1.Entity objects.
DatastoreIO() - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO
 
DatastoreIO.DatastoreReader - Class in com.google.cloud.dataflow.sdk.io
A Source.Reader over the records from a query of the datastore.
DatastoreIO.Sink - Class in com.google.cloud.dataflow.sdk.io
A Sink that writes a PCollection containing Entities to a Datastore kind.
DatastoreIO.Source - Class in com.google.cloud.dataflow.sdk.io
A Source that reads the result rows of a Datastore query as Entity objects.
DatastoreReader(DatastoreIO.Source, Datastore) - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
Returns a DatastoreReader with Source and Datastore object set.
days(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
Returns a WindowFn that windows elements into periods measured by days.
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
 
decode(InputStream, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Decodes a value of type T from the given input stream in the given context.
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow.Coder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
decodePane(byte) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
 
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
Builds an instance of IterableT, this coder's associated Iterable-like subtype, from a list of decoded elements.
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
Builds an instance of IterableT, this coder's associated Iterable-like subtype, from a list of decoded elements.
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.ListCoder
 
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.SetCoder
Builds an instance of IterableT, this coder's associated Iterable-like subtype, from a list of decoded elements.
Default - Annotation Type in com.google.cloud.dataflow.sdk.options
Default represents a set of annotations that can be used to annotate getter properties on PipelineOptions with information representing the default value to be returned if no value is specified.
Default.Boolean - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified boolean primitive value.
Default.Byte - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified byte primitive value.
Default.Character - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified char primitive value.
Default.Class - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified Class value.
Default.Double - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified double primitive value.
Default.Enum - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified enum.
Default.Float - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified float primitive value.
Default.InstanceFactory - Annotation Type in com.google.cloud.dataflow.sdk.options
Value must be of type DefaultValueFactory and have a default constructor.
Default.Integer - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified int primitive value.
Default.Long - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified long primitive value.
Default.Short - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified short primitive value.
Default.String - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified String value.
DEFAULT_HOST - Static variable in class com.google.cloud.dataflow.sdk.io.DatastoreIO
 
DEFAULT_MAX_NUM_ELEMENTS - Static variable in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
The cost (in time and space) to compute quantiles to a given accuracy is a function of the total number of elements in the data set.
DEFAULT_PUBSUB_CODER - Static variable in class com.google.cloud.dataflow.sdk.io.PubsubIO
The default Coder used to translate to/from Cloud Pub/Sub messages.
DEFAULT_TEXT_CODER - Static variable in class com.google.cloud.dataflow.sdk.io.TextIO
The default coder, which returns each line of the input file as a string.
DefaultCoder - Annotation Type in com.google.cloud.dataflow.sdk.coders
The DefaultCoder annotation specifies a default Coder class to handle encoding and decoding instances of the annotated class.
DefaultProjectFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcpOptions.DefaultProjectFactory
 
Defaults() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
 
DefaultTrigger<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A trigger that is equivalent to Repeatedly.forever(AfterWatermark.pastEndOfWindow()).
defaultValue() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns the default value when there are no values added to the accumulator.
defaultValue() - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
Returns the default value when there are no values added to the accumulator.
defaultValue() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
 
defaultValue() - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
Returns the default value of this transform, or null if there isn't one.
DefaultValueFactory<T> - Interface in com.google.cloud.dataflow.sdk.options
An interface used with the Default.InstanceFactory annotation to specify the class that will be an instance factory to produce default values for a given getter on PipelineOptions.
DELAYED_UNTIL_TAG - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
DelegateCoder<T,IntermediateT> - Class in com.google.cloud.dataflow.sdk.coders
A DelegateCoder<T, IntermediateT> wraps a Coder for IntermediateT and encodes/decodes values of type T by converting to/from IntermediateT and then encoding/decoding using the underlying Coder<IntermediateT>.
DelegateCoder(Coder<IntermediateT>, DelegateCoder.CodingFunction<T, IntermediateT>, DelegateCoder.CodingFunction<IntermediateT, T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
DelegateCoder.CodingFunction<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.coders
A CodingFunction<InputT, OutputT> is a serializable function from InputT to OutputT that may throw any Exception.
deleteTimer(Instant, TimeDomain) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Removes the timer set in this trigger context for the given Instant and TimeDomain.
dependsOnlyOnEarliestInputTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
Returns true if the result of combination of many output timestamps actually depends only on the earliest.
dependsOnlyOnEarliestInputTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn
Returns true if the result of combination of many output timestamps actually depends only on the earliest.
dependsOnlyOnEarliestInputTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
Returns true if the result of combination of many output timestamps actually depends only on the earliest.
dependsOnlyOnWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
Returns true if the result does not depend on what outputs were combined but only the window they are in.
dependsOnlyOnWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn
Returns true if the result does not depend on what outputs were combined but only the window they are in.
DependsOnlyOnWindow() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
 
dependsOnlyOnWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
Returns true if the result does not depend on what outputs were combined but only the window they are in.
Description - Annotation Type in com.google.cloud.dataflow.sdk.options
Descriptions are used to generate human readable output when the --help command is specified.
detectClassPathResourcesToStage(ClassLoader) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Attempts to detect all the resources the class loader has access to.
DeterministicStandardCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A DeterministicStandardCoder is a StandardCoder that is deterministic, in the sense that for objects considered equal according to Object.equals(Object), the encoded bytes are also equal.
DeterministicStandardCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.DeterministicStandardCoder
 
DIRECTORY_CONTAINER - Static variable in class com.google.cloud.dataflow.sdk.io.ShardNameTemplate
Shard is a file within a directory.
DirectPipeline - Class in com.google.cloud.dataflow.sdk.runners
DirectPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that can be used to configure the DirectPipeline.
DirectPipelineRegistrar - Class in com.google.cloud.dataflow.sdk.runners
DirectPipelineRegistrar.Options - Class in com.google.cloud.dataflow.sdk.runners
Register the DirectPipelineOptions.
DirectPipelineRegistrar.Runner - Class in com.google.cloud.dataflow.sdk.runners
Register the DirectPipelineRunner.
DirectPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners
Executes the operations in the pipeline directly, in this process, without any optimization.
DirectPipelineRunner.EvaluationContext - Interface in com.google.cloud.dataflow.sdk.runners
The interface provided to registered callbacks for interacting with the DirectPipelineRunner, including reading and writing the values of PCollections and PCollectionViews.
DirectPipelineRunner.EvaluationResults - Interface in com.google.cloud.dataflow.sdk.runners
The interface provided to registered callbacks for interacting with the DirectPipelineRunner, including reading and writing the values of PCollections and PCollectionViews.
DirectPipelineRunner.TestCombineDoFn<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.runners
The implementation may split the Combine.KeyedCombineFn into ADD, MERGE and EXTRACT phases ( see com.google.cloud.dataflow.sdk.runners.worker.CombineValuesFn).
DirectPipelineRunner.TransformEvaluator<TransformT extends PTransform> - Interface in com.google.cloud.dataflow.sdk.runners
An evaluator of a PTransform.
DirectPipelineRunner.ValueWithMetadata<V> - Class in com.google.cloud.dataflow.sdk.runners
An immutable (value, timestamp) pair, along with other metadata necessary for the implementation of DirectPipelineRunner.
discardingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that discards elements in a pane after they are triggered.
discardingFiredPanes() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that discards elements in a pane after they are triggered.
discardingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that discards elements in a pane after they are triggered.
DisplayData - Class in com.google.cloud.dataflow.sdk.transforms.display
Static display metadata associated with a pipeline component.
DisplayData.Builder - Interface in com.google.cloud.dataflow.sdk.transforms.display
Utility to build up display metadata from a component and its included subcomponents.
DisplayData.Identifier - Class in com.google.cloud.dataflow.sdk.transforms.display
Unique identifier for a display metadata item within a component.
DisplayData.Item - Class in com.google.cloud.dataflow.sdk.transforms.display
A display metadata item.
DisplayData.ItemBuilder - Interface in com.google.cloud.dataflow.sdk.transforms.display
Utility to append optional fields to display metadata, or register additional display metadata items.
DoFn<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
The argument to ParDo providing the code to use to process elements of the input PCollection.
DoFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn
 
DoFn.Context - Class in com.google.cloud.dataflow.sdk.transforms
Information accessible to all methods in this DoFn.
DoFn.ProcessContext - Class in com.google.cloud.dataflow.sdk.transforms
DoFn.RequiresWindowAccess - Interface in com.google.cloud.dataflow.sdk.transforms
Interface for signaling that a DoFn needs to access the window the element is being processed in, via DoFn.ProcessContext.window().
DoFnReflector - Class in com.google.cloud.dataflow.sdk.transforms
Utility implementing the necessary reflection for working with DoFnWithContexts.
DoFnReflector() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
 
DoFnTester<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A harness for unit-testing a DoFn.
DoFnTester.OutputElementWithTimestamp<OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
Holder for an OutputElement along with its associated timestamp.
DoFnWithContext<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
The argument to ParDo providing the code to use to process elements of the input PCollection.
DoFnWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
 
DoFnWithContext.Context - Class in com.google.cloud.dataflow.sdk.transforms
Information accessible to all methods in this DoFnWithContext.
DoFnWithContext.ExtraContextFactory<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
Interface for runner implementors to provide implementations of extra context information.
DoFnWithContext.FinishBundle - Annotation Type in com.google.cloud.dataflow.sdk.transforms
Annotation for the method to use to prepare an instance for processing a batch of elements.
DoFnWithContext.ProcessContext - Class in com.google.cloud.dataflow.sdk.transforms
DoFnWithContext.ProcessElement - Annotation Type in com.google.cloud.dataflow.sdk.transforms
Annotation for the method to use for processing elements.
DoFnWithContext.StartBundle - Annotation Type in com.google.cloud.dataflow.sdk.transforms
Annotation for the method to use to prepare an instance for processing a batch of elements.
DoubleCoder - Class in com.google.cloud.dataflow.sdk.coders
A DoubleCoder encodes Double values in 8 bytes using Java serialization.
doublesGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<Double> and returns a PCollection<Double> whose contents is the maximum of the input PCollection's elements, or Double.NEGATIVE_INFINITY if there are no elements.
doublesGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<Double> and returns a PCollection<Double> whose contents is the minimum of the input PCollection's elements, or Double.POSITIVE_INFINITY if there are no elements.
doublesGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<Double> and returns a PCollection<Double> whose contents is the sum of the input PCollection's elements, or 0 if there are no elements.
doublesPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<KV<K, Double>> and returns a PCollection<KV<K, Double>> that contains an output element mapping each distinct key in the input PCollection to the maximum of the values associated with that key in the input PCollection.
doublesPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<KV<K, Double>> and returns a PCollection<KV<K, Double>> that contains an output element mapping each distinct key in the input PCollection to the minimum of the values associated with that key in the input PCollection.
doublesPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<KV<K, Double>> and returns a PCollection<KV<K, Double>> that contains an output element mapping each distinct key in the input PCollection to the sum of the values associated with that key in the input PCollection.
DurationCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder that encodes a joda Duration as a Long using the format of VarLongCoder.

E

element() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the input element to be processed.
element() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
Returns the input element to be processed.
element() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
Returns the current element.
elementCountAtLeast(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
Creates a trigger that fires when the pane contains at least countElems elements.
EMPTY - Static variable in class com.google.cloud.dataflow.sdk.io.range.ByteKey
An empty key.
empty() - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Checks that the Iterable is empty.
empty() - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns an empty CoGbkResult.
empty(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns an empty KeyedPCollectionTuple<K> on the given pipeline.
empty(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns an empty PCollectionList that is part of the given Pipeline.
empty(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns an empty PCollectionTuple that is part of the given Pipeline.
empty() - Static method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns an empty TupleTagList.
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
encode(Integer, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
 
encode(Long, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
 
encode(byte[], OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
 
encode(Byte, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
 
encode(ByteString, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
 
encode(T, OutputStream, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Encodes the given value of type T onto the given output stream in the given context.
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
encode(Double, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
 
encode(ReadableDuration, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder
 
encode(DatastoreV1.Entity, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
 
encode(Instant, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
 
encode(IterableT, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
 
encode(KV<K, V>, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
encode(Map<K, V>, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
 
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
 
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
 
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
encode(String, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
 
encode(TableRow, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
 
encode(Integer, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
encode(Integer, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
 
encode(Long, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
encode(Void, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
 
encode(CoGbkResult, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
encode(GlobalWindow, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow.Coder
 
encode(PaneInfo, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
 
encode(TimestampedValue<T>, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
encodeAndOwn(byte[], OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
Encodes the provided value with the identical encoding to ByteArrayCoder.encode(byte[], java.io.OutputStream, com.google.cloud.dataflow.sdk.coders.Coder.Context), but with optimizations that take ownership of the value.
end() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns the end of this window, exclusive.
ensureCoderSerializable(Coder<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
If the evaluation context is testing serializability, ensures that the argument Coder is serializable and deserializable by encoding it and then decoding it, and returning the result.
ensureElementEncodable(TypedPValue<T>, T) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Ensures that the element is encodable and decodable using the TypePValue's coder, by encoding it and decoding it, and returning the result.
ensureSerializable(FunctionT) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
If the evaluation context is testing serializability, ensures that the argument function is serializable and deserializable by encoding it and then decoding it, and returning the result.
ensureSerializableByCoder(Coder<T>, T, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
If the evaluation context is testing serializability, ensures that the given data is serializable and deserializable with the given Coder by encoding it and then decoding it, and returning the result.
ensureSerializableByCoder(Coder<T>, T, String) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
 
enterCompositeTransform(TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
Called for each composite transform after all topological predecessors have been visited but before any of its component transforms.
enterCompositeTransform(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
EntityCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder for DatastoreV1.Entity objects based on their encoded Protocol Buffer form.
equals(Object) - Method in class com.google.cloud.dataflow.sdk.coders.Coder.Context
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
equals(Object) - Method in class com.google.cloud.dataflow.sdk.coders.StructuralByteArray
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Deprecated.
Object.equals(Object) is not supported on DataflowAssert objects. If you meant to test object equality, use a variant of DataflowAssert.IterableAssert.containsInAnyOrder(java.lang.Iterable<T>) instead.
equals(Object) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Deprecated.
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Identifier
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester.OutputElementWithTimestamp
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.values.KV
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Two type descriptor are equal if and only if they represent the same type.
estimateFractionForKey(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Returns the fraction of this range [startKey, endKey) that is in the interval [startKey, key).
evaluate(TransformT, DirectPipelineRunner.EvaluationContext) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TransformEvaluator
 
eventTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
The event timestamp of the element currently being processed.
every(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
Returns a new SlidingWindows with the original size, that assigns timestamps into half-open intervals of the form [N * period, N * period + size), where 0 is the epoch.
ExecutorServiceFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcsOptions.ExecutorServiceFactory
 
expand() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Expands the component PCollections, stripping off any tag-specific information.
expand() - Method in class com.google.cloud.dataflow.sdk.values.PBegin
 
expand() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
expand() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
expand() - Method in class com.google.cloud.dataflow.sdk.values.PDone
 
expand() - Method in interface com.google.cloud.dataflow.sdk.values.PInput
Expands this PInput into a list of its component output PValues.
expand() - Method in interface com.google.cloud.dataflow.sdk.values.POutput
Expands this POutput into a list of its component output PValues.
expand() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
 
expandFilePattern(String) - Static method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
Experimental - Annotation Type in com.google.cloud.dataflow.sdk.annotations
Signifies that a public API (public class, method or field) is subject to incompatible changes, or even removal, in a future release.
Experimental.Kind - Enum in com.google.cloud.dataflow.sdk.annotations
An enumeration of various kinds of experimental APIs.
extension - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink
The extension to be used for the final output files.
extractOrderedList() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique
Returns the values in the heap, ordered largest to smallest.
extractOutput(ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
extractOutput() - Method in interface com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn.Accumulator
Returns the output value that is the result of combining all the input values represented by this accumulator.
extractOutput(AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
 
extractOutput(double[]) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
extractOutput(Combine.Holder<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
extractOutput(int[]) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
extractOutput(long[]) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
extractOutput(AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns the output value that is the result of combining all the input values represented by the given accumulator.
extractOutput(List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
 
extractOutput(K, AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Returns the output value that is the result of combining all the input values represented by the given accumulator.
extractOutput(Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
 
extractOutput(Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
 
extractOutput(K, Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
 
extractOutput(K, Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
 
extractOutput(AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
Returns the output value that is the result of combining all the input values represented by the given accumulator.
extractOutput(K, AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
Returns the output value that is the result of combining all the input values represented by the given accumulator.
extractOutput(Top.BoundedHeap<KV<Integer, T>, SerializableComparator<KV<Integer, T>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 

F

fewKeys() - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
Returns whether it groups just few keys.
FileBasedReader(FileBasedSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
Subclasses should not perform IO operations at the constructor.
FileBasedSink<T> - Class in com.google.cloud.dataflow.sdk.io
Abstract Sink for file-based output.
FileBasedSink(String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink
Construct a FileBasedSink with the given base output filename and extension.
FileBasedSink(String, String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink
Construct a FileBasedSink with the given base output filename, extension, and file naming template.
FileBasedSink.FileBasedWriteOperation<T> - Class in com.google.cloud.dataflow.sdk.io
Abstract Sink.WriteOperation that manages the process of writing to a FileBasedSink.
FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention - Enum in com.google.cloud.dataflow.sdk.io
Options for handling of temporary output files.
FileBasedSink.FileBasedWriter<T> - Class in com.google.cloud.dataflow.sdk.io
Abstract Sink.Writer that writes a bundle to a FileBasedSink.
FileBasedSink.FileResult - Class in com.google.cloud.dataflow.sdk.io
Result of a single bundle write.
FileBasedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A common base class for all file-based Sources.
FileBasedSource(String, long) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSource
Create a FileBaseSource based on a file or a file pattern specification.
FileBasedSource(String, long, long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSource
Create a FileBasedSource based on a single file.
FileBasedSource.FileBasedReader<T> - Class in com.google.cloud.dataflow.sdk.io
A reader that implements code common to readers of FileBasedSources.
FileBasedSource.Mode - Enum in com.google.cloud.dataflow.sdk.io
A given FileBasedSource represents a file resource of one of these types.
FileBasedWriteOperation(FileBasedSink<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Construct a FileBasedWriteOperation using the same base filename for both temporary and output files.
FileBasedWriteOperation(FileBasedSink<T>, String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Construct a FileBasedWriteOperation.
FileBasedWriteOperation(FileBasedSink<T>, String, FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Create a new FileBasedWriteOperation.
FileBasedWriter(FileBasedSink.FileBasedWriteOperation<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Construct a new FileBasedWriter with a base filename.
fileNamingTemplate - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink
Naming template for output files.
FileResult(String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileResult
 
Filter<T> - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for filtering from a PCollection the elements satisfying a predicate, or satisfying an inequality with a given value based on the elements' natural ordering.
finalize(Iterable<FileBasedSink.FileResult>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Finalizes writing by copying temporary output files to their final location and optionally removing temporary files.
finalize(Iterable<WriteT>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Given an Iterable of results from bundle writes, performs finalization after writing and closes the sink.
finalizeCheckpoint() - Method in class com.google.cloud.dataflow.sdk.io.CountingSource.CounterMark
 
finalizeCheckpoint() - Method in interface com.google.cloud.dataflow.sdk.io.UnboundedSource.CheckpointMark
Perform any finalization that needs to happen after a bundle of data read from the source has been processed and committed.
finishBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Finishes processing this batch of elements.
finishBundle() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
finishBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
finishedInAllMergingWindows() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergingTriggerInfo
Return true if the trigger is finished in all windows being merged.
finishedInAnyMergingWindow() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergingTriggerInfo
Return true if the trigger is finished in any window being merged.
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Finish specifying a transform.
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
 
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PBegin
 
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
finishSpecifying() - Method in interface com.google.cloud.dataflow.sdk.values.PInput
After building, finalizes this PInput to make it ready for being used as an input to a PTransform.
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
 
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
After building, finalizes this PValue to make it ready for running.
finishSpecifyingOutput() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
finishSpecifyingOutput() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
finishSpecifyingOutput() - Method in interface com.google.cloud.dataflow.sdk.values.POutput
As part of applying the producing PTransform, finalizes this output to make it ready for being used as an input and for running.
finishSpecifyingOutput() - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
Default behavior for POutputValueBase.finishSpecifyingOutput() is to do nothing.
firstOf(CoderProvider...) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderProviders
Returns a CoderProvider that consults each of the provider coderProviders and returns the first Coder provided.
firstUnfinishedSubTrigger() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Returns the first unfinished sub-trigger.
fixedSizeGlobally(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample
Returns a PTransform that takes a PCollection<T>, selects sampleSize elements, uniformly at random, and returns a PCollection<Iterable<T>> containing the selected elements.
fixedSizePerKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample
Returns a PTransform that takes an input PCollection<KV<K, V>> and returns a PCollection<KV<K, Iterable<V>>> that contains an output element mapping each distinct key in the input PCollection to a sample of sampleSize values associated with that key in the input PCollection, taken uniformly at random.
FixedWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows values into fixed-size timestamp-based windows.
FlatMapElements<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for mapping a simple function that returns iterables over the elements of a PCollection and merging the results.
FlatMapElements.MissingOutputTypeDescriptor<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
An intermediate builder for a FlatMapElements transform.
Flatten - Class in com.google.cloud.dataflow.sdk.transforms
Flatten<T> takes multiple PCollection<T>s bundled into a PCollectionList<T> and returns a single PCollection<T> containing all the elements in all the input PCollections.
Flatten() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Flatten
 
Flatten.FlattenIterables<T> - Class in com.google.cloud.dataflow.sdk.transforms
FlattenIterables<T> takes a PCollection<Iterable<T>> and returns a PCollection<T> that contains all the elements from each iterable.
Flatten.FlattenPCollectionList<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.
FlattenIterables() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenIterables
 
forCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderFactories
Creates a CoderFactory that always returns the given coder.
forever(Trigger<W>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
Create a composite trigger that repeatedly executes the trigger toRepeat, firing each time it fires and ignoring any indications to finish.
forKey(K, Coder<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
 
forKey(K, Coder<K>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.PerKeyCombineFn
Returns the a regular CombineFnBase.GlobalCombineFn that operates on a specific key.
forKey(K, Coder<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
 
forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
Create an OnElementContext for executing the given trigger.
forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
Create an OnMergeContext for executing the given trigger.
forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Create a sub-context for the given sub-trigger.
from(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new PTransform that's like this one but that reads from the file(s) with the given name or pattern.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns a PTransform that reads from the file(s) with the given name or pattern.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroSource
Creates an AvroSource that reads from the given file name or pattern ("glob").
from(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns a copy of this transform that reads from the specified table.
from(TableReference) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns a copy of this transform that reads from the specified table.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.
from(TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
Reads a BigQuery table specified as a TableReference object.
from(FileBasedSource<T>) - Static method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Creates a CompressedSource from an underlying FileBasedSource.
from(BoundedSource<T>) - Method in class com.google.cloud.dataflow.sdk.io.Read.Builder
Returns a new Read.Bounded PTransform reading from the given BoundedSource.
from(UnboundedSource<T, ?>) - Method in class com.google.cloud.dataflow.sdk.io.Read.Builder
Returns a new Read.Unbounded PTransform reading from the given UnboundedSource.
from(BoundedSource<T>) - Static method in class com.google.cloud.dataflow.sdk.io.Read
Returns a new Read.Bounded PTransform reading from the given BoundedSource.
from(UnboundedSource<T, ?>) - Static method in class com.google.cloud.dataflow.sdk.io.Read
Returns a new Read.Unbounded PTransform reading from the given UnboundedSource.
from(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new transform for reading from text files that's like this one but that reads from the file(s) with the given name or pattern.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a transform for reading text files that reads from the file(s) with the given filename or filename pattern.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.XmlSource
Creates an XmlSource for a single XML file or a set of XML files defined by a Java "glob" file pattern.
from(Map<String, String>) - Static method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
Expects a map keyed by logger Names with values representing Levels.
from(HasDisplayData) - Static method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData
Collect the DisplayData from a component.
fromArgs(String[]) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
Sets the command line arguments to parse when constructing the PipelineOptions.
fromArgs(String[]) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Sets the command line arguments to parse when constructing the PipelineOptions.
fromIterableInternal(Iterable<WindowedValue<?>>) - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
For internal use only.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
Constructs a runner from the provided options.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Construct a runner from the provided options.
fromOptions(DataflowPipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
Constructs a translator from the provided options.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Constructs a DirectPipelineRunner from the given options.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner
Constructs a runner from the provided options.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
Constructs a runner from the provided options.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
 
fromPath(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
Creates a class representing a Pub/Sub subscription from the specified subscription path.
fromPath(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
Creates a class representing a Cloud Pub/Sub topic from the specified topic path.
fromQuery(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns a copy of this transform that reads the results of the specified query.
fromQuery(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
Reads results received after executing the given query.
fromStaticMethods(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderFactories
Creates a CoderFactory built from particular static methods of a class that implements Coder.
fromStaticMethods(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderProviders
Creates a CoderProvider built from particular static methods of a class that implements Coder.

G

GcpOptions - Interface in com.google.cloud.dataflow.sdk.options
Options used to configure Google Cloud Platform project and credentials.
GcpOptions.CredentialDirFactory - Class in com.google.cloud.dataflow.sdk.options
Returns the default credential directory of ${user.home}/.store/data-flow.
GcpOptions.DefaultProjectFactory - Class in com.google.cloud.dataflow.sdk.options
Attempts to infer the default project based upon the environment this application is executing within.
GcpOptions.GcpUserCredentialsFactory - Class in com.google.cloud.dataflow.sdk.options
Attempts to load the GCP credentials.
GcpUserCredentialsFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcpOptions.GcpUserCredentialsFactory
 
GcsOptions - Interface in com.google.cloud.dataflow.sdk.options
Options used to configure Google Cloud Storage.
GcsOptions.ExecutorServiceFactory - Class in com.google.cloud.dataflow.sdk.options
Returns the default ExecutorService to use within the Dataflow SDK.
generateDestinationFilenames(int) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Generate output bundle filenames.
generateInitialSplits(int, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
Returns a list of UnboundedSource objects representing the instances of this source that should be used when executing the workflow.
get(TupleTag<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.CoCombineResult
Returns the value represented by the given TupleTag.
get(int) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns the PCollection at the given index (origin zero).
get(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns the PCollection associated with the given TupleTag in this PCollectionTuple.
get(int) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns the TupleTag at the given index (origin zero).
getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
 
getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
getAccumulatorCoder(CoderRegistry, Coder<Double>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
getAccumulatorCoder(CoderRegistry, Coder<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
getAccumulatorCoder(CoderRegistry, Coder<Integer>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
getAccumulatorCoder(CoderRegistry, Coder<Long>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
getAccumulatorCoder(CoderRegistry, Coder<InputT>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
Returns the Coder to use for accumulator AccumT values, or null if it is not able to be inferred.
getAccumulatorCoder(CoderRegistry, Coder<K>, Coder<InputT>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.PerKeyCombineFn
Returns the Coder to use for accumulator AccumT values, or null if it is not able to be inferred.
getAccumulatorCoder(CoderRegistry, Coder<DataT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
 
getAccumulatorCoder(CoderRegistry, Coder<DataT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
 
getAccumulatorCoder(CoderRegistry, Coder<K>, Coder<DataT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
 
getAccumulatorCoder(CoderRegistry, Coder<K>, Coder<DataT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
 
getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
 
getAddCounterMutator() - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Returns a mutator, which can be used to add additional counters to this EvaluationContext.
getAggregators(DoFn<?, ?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.AggregatorRetriever
Returns the Aggregators created by the provided DoFn.
getAggregatorSteps() - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorPipelineExtractor
Returns a Map between each Aggregator in the Pipeline to the PTransforms in which it is used.
getAggregatorValue(Aggregator<?, AggregateT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the value of the provided Aggregator.
getAggregatorValues(Aggregator<?, T>) - Method in interface com.google.cloud.dataflow.sdk.PipelineResult
Retrieves the current value of the provided Aggregator.
getAggregatorValues(Aggregator<?, OutputT>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
 
getAlgorithm() - Method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
Returns the string representation of this type.
getAll(TupleTag<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns the values from the table represented by the given TupleTag<V> as an Iterable<V> (which may be empty if there are no results).
getAll() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns an immutable List of all the PCollections in this PCollectionList.
getAll() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns an immutable Map from TupleTag to corresponding PCollection, for all the members of this PCollectionTuple.
getAll() - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns an immutable List of all the TupleTags in this TupleTagList.
getAllowedEncodings() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
A collection of encodings supported by Coder.decode(java.io.InputStream, com.google.cloud.dataflow.sdk.coders.Coder.Context) in addition to the encoding from Coder.getEncodingId() (which is assumed supported).
getAllowedEncodings() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
A collection of encodings supported by Coder.decode(java.io.InputStream, com.google.cloud.dataflow.sdk.coders.Coder.Context) in addition to the encoding from Coder.getEncodingId() (which is assumed supported).
getAllowedEncodings() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
getAllowedTimestampSkew() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Deprecated.
does not interact well with the watermark.
getAllowedTimestampSkew() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns the allowed timestamp skew duration, which is the maximum duration that timestamps can be shifted backward in DoFnWithContext.Context.outputWithTimestamp(OutputT, org.joda.time.Instant).
getAllowedTimestampSkew() - Method in class com.google.cloud.dataflow.sdk.transforms.WithTimestamps
Returns the allowed timestamp skew duration, which is the maximum duration that timestamps can be shifted backwards from the timestamp of the input element.
getApiRootUrl() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The root URL for the Dataflow API.
getAppliedFn(CoderRegistry, Coder<? extends KV<K, ? extends Iterable<InputT>>>, WindowingStrategy<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
Returns the Combine.CombineFn bound to its coders.
getAppName() - Method in interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions
Name of application, for display purposes.
getArgumentTypes(Method) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a list of argument types for the given method, which must be a part of the class.
getAuthorizationServerEncodedUrl() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
The authorization server URL to use for OAuth 2 authentication.
getAutoscalingAlgorithm() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
[Experimental] The autoscaling algorithm to use for the workerpool.
getBaseOutputFilename() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink
Returns the base output filename for this file based sink.
getBigtableOptions() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
Returns the Google Cloud Bigtable cluster being read from, and other parameters.
getBigtableOptions() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
Returns the Google Cloud Bigtable cluster being written to, and other parameters.
getBytes() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
Returns a newly-allocated byte[] representing this ByteKey.
getBytesPerOffset() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
Returns approximately how many bytes of data correspond to a single offset in this source.
getCause() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
Returns the reason that this WindowFn is invalid.
getChannelFactory() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
 
getCheckpointMark() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns a UnboundedSource.CheckpointMark representing the progress of this UnboundedReader.
getCheckpointMarkCoder() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
Returns a Coder for encoding and decoding the checkpoints for this source, or null if the checkpoints do not need to be durably committed.
getClasses() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a set of TypeDescriptors, one for each superclass (including this class).
getCoder(TypeDescriptor<T>) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderProvider
Provides a coder for a given class, if possible.
getCoder(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
getCoder() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
Returns the coder used to encode/decode the intermediate values produced/consumed by the coding functions of this DelegateCoder.
getCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getCoder() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getCoder() - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Gets the coder, which may yet be absent.
getCoder() - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Gets the coder, which may yet be absent.
getCoder() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns a Coder suitable for IntervalWindow.
getCoder() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Returns the Coder used by this PCollection to encode and decode the values stored in it.
getCoder() - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
Returns the Coder used by this TypedPValue to encode and decode the values stored in it.
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.AtomicCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
getCoderArguments() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters, or returns null if this cannot be done or this is not a parameterized type.
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters, or returns null if this cannot be done or this is not a parameterized type.
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
getCoderInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
For internal use only.
getCoderRegistry() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Returns the CoderRegistry that this Pipeline uses.
getCoGbkResultSchema() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns the CoGbkResultSchema associated with this KeyedPCollectionTuple.
getCollection() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple.TaggedKeyedPCollection
Returns the underlying PCollection of this TaggedKeyedPCollection.
getCombineFn() - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator
Returns the Combine.CombineFn, which combines input elements in the aggregator.
getCombineFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView
 
getComponents() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
Returns the list of Coders that are components of this Coder.
getComponents() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
getComponentType() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns the component type if this type is an array type, otherwise returns null.
getCompressionType() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
getContinuationTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Return a trigger to use after a GroupByKey to preserve the intention of this trigger.
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Return the Trigger.getContinuationTrigger() of this Trigger.
getContinuationTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxDoubleFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxIntegerFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxLongFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinDoubleFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinIntegerFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinLongFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
 
getCreateDisposition() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns the create disposition.
getCredentialDir() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
Directory for storing dataflow credentials after execution of the OAuth 2 webflow.
getCredentialFactoryClass() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
The class of the credential factory that should be created and used to create credentials.
getCredentialId() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
This option controls which credential store to use when creating the credentials using the OAuth 2 webflow.
getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
 
getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Gets the current record from the delegate reader.
getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
Returns the value of the data item that was read by the last Source.Reader.start() or Source.Reader.advance() call.
getCurrent() - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Returns the last TransformTreeNode on the stack.
getCurrentBlock() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
getCurrentBlock() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Returns the current block (the block that was read by the last successful call to BlockBasedSource.BlockBasedReader.readNextBlock()).
getCurrentBlockOffset() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
getCurrentBlockOffset() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Returns the largest offset such that starting to read from that offset includes the current block.
getCurrentBlockSize() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
getCurrentBlockSize() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Returns the size of the current block in bytes as it is represented in the underlying file, if possible.
getCurrentOffset() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
 
getCurrentOffset() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Returns the delegate reader's current offset in the decompressed input.
getCurrentOffset() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
Returns the starting offset of the current record, which has been read by the last successful Source.Reader.start() or Source.Reader.advance() call.
getCurrentRecord() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
Returns the current record.
getCurrentRecordId() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns a unique identifier for the current record.
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
Returns a Source describing the same input that this Reader currently reads (including items already read).
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
 
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
 
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
Returns a Source describing the same input that this Reader currently reads (including items already read).
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns the UnboundedSource that created this reader.
getCurrentTime(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
To complete an implementation, return the desired time from the TriggerContext.
getCurrentTime(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
getCurrentTimestamp() - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
By default, returns the minimum possible timestamp.
getCurrentTimestamp() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
Returns the timestamp associated with the current data item.
getDataflowClient() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
An instance of the Dataflow client.
getDataflowClient() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Get the Cloud Dataflow API Client used by this job.
getDataflowEndpoint() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Dataflow endpoint to use.
getDataflowJobFile() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The path to write the translated Dataflow job specification out to at job submission time.
getDataset() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
getDayOfMonth() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
getDayOfMonth() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getDebuggee() - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
The Cloud Debugger debuggee to associate with.
getDefaultCoder(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use by default for values of the given type.
getDefaultCoder(TypeDescriptor<OutputT>, TypeDescriptor<InputT>, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use by default for values of the given type, where the given input type uses the given Coder.
getDefaultCoder(Class<? extends T>, Class<T>, Map<Type, ? extends Coder<?>>, TypeVariable<?>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use for the specified type parameter specialization of the subclass, given Coders to use for all other type parameters (if any).
getDefaultCoder(T) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use for the provided example value, if it can be determined.
getDefaultCoder(Class<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use by default for values of the given class.
getDefaultCoders(Class<? extends T>, Class<T>, Map<Type, ? extends Coder<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Deprecated.
this method is not part of the public interface and will be made private
getDefaultOutputCoder(SerializableFunction<InputT, OutputT>, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use on elements produced by this function, given the Coder used for its input elements.
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Returns the delegate source's default output coder.
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.Source
Returns the default Coder to use for the data read from this source.
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
getDefaultOutputCoder(CoderRegistry, Coder<Double>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
getDefaultOutputCoder(CoderRegistry, Coder<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
getDefaultOutputCoder(CoderRegistry, Coder<Integer>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
getDefaultOutputCoder(CoderRegistry, Coder<Long>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
getDefaultOutputCoder(PCollection<? extends KV<K, ? extends Iterable<InputT>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
 
getDefaultOutputCoder(CoderRegistry, Coder<InputT>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
Returns the Coder to use by default for output OutputT values, or null if it is not able to be inferred.
getDefaultOutputCoder(CoderRegistry, Coder<K>, Coder<InputT>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.PerKeyCombineFn
Returns the Coder to use by default for output OutputT values, or null if it is not able to be inferred.
getDefaultOutputCoder(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
 
getDefaultOutputCoder(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Filter
 
getDefaultOutputCoder(PCollectionList<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenPCollectionList
 
getDefaultOutputCoder(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
 
getDefaultOutputCoder(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
 
getDefaultOutputCoder(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getDefaultOutputCoder(PCollection<? extends InputT>, TypedPValue<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the default Coder to use for the output of this single-output PTransform.
getDefaultOutputCoder(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the default Coder to use for the output of this single-output PTransform when applied to the given input.
getDefaultOutputCoder(InputT, TypedPValue<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the default Coder to use for the given output of this single-output PTransform when applied to the given input.
getDefaultOutputCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
getDefaultOutputCoder(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
 
getDefaultWorkerLogLevel() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
This option controls the default log level of all loggers without a log level override.
getDirectPipelineRunnerRandomSeed() - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
The random seed to use for pseudorandom behaviors in the DirectPipelineRunner.
getDiskSizeGb() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Remote worker disk size, in gigabytes, or 0 to use the default size.
getDoFnClass(DoFn<?, ?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
 
getDumpHeapOnOOM() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
If true, save a heap dump before killing a thread or process which is GC thrashing or out of memory.
getElemCoder() - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
getElements() - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
 
getEnableCloudDebugger() - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
Whether to enable the Cloud Debugger snapshot agent for the current job.
getEnableProfilingAgent() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions
 
getEnclosingNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the enclosing composite transform node, or null if there is none.
getEncodedElementByteSize(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
Returns the size in bytes of the encoded value using this coder.
getEncodedElementByteSize(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
Returns the size in bytes of the encoded value using this coder.
getEncodedElementByteSize(byte[], Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
 
getEncodedElementByteSize(Byte, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
Returns the size in bytes of the encoded value using this coder.
getEncodedElementByteSize(ByteString, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
 
getEncodedElementByteSize(Double, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
Returns the size in bytes of the encoded value using this coder.
getEncodedElementByteSize(DatastoreV1.Entity, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
 
getEncodedElementByteSize(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
Overridden to short-circuit the default StandardCoder behavior of encoding and counting the bytes.
getEncodedElementByteSize(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
Returns the size in bytes of the encoded value using this coder.
getEncodedElementByteSize(String, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
Returns the size in bytes of the encoded value using this coder.
getEncodedElementByteSize(TableRow, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
 
getEncodedElementByteSize(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
getEncodedElementByteSize(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
 
getEncodedElementByteSize(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
getEncodedElementByteSize(Void, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
 
getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
The encoding identifier is designed to support evolution as per the design of Avro In order to use this class effectively, carefully read the Avro documentation at Schema Resolution to ensure that the old and new schema match.
getEncodingId() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
 
getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
The encoding identifier is designed to support evolution as per the design of Protocol Buffers.
getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
The encoding identifier is designed to support evolution as per the design of Protocol Buffers.
getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
The encoding id is the fully qualified name of the encoded/decoded class.
getEndKey() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Returns the ByteKey representing the upper bound of this ByteKeyRange.
getEndOffset() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
Returns the specified ending offset of the source.
getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
An estimate of the total size (in bytes) of the data that would be read from this source.
getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
 
getExecutorService() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
The ExecutorService instance to use to create threads, can be overridden to specify an ExecutorService that is compatible with the users environment.
getExpandedOutputs() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the transform outputs, in expanded form.
getExperiments() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The list of backend experiments to enable.
getExtensionRegistry() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
Returns the ExtensionRegistry listing all known Protocol Buffers extension messages to T registered with this ProtoCoder.
getFallbackCoderProvider() - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the fallback CoderProvider for this registry.
getFanout() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView
 
getFilename() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileResult
 
getFilenamePrefix() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getFilenamePrefix() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getFilenameSuffix() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getFilenameSuffix() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getFileOrPatternSpec() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
getFilepattern() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
getFilepattern() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
getFilesToStage() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
List of local files to make available to workers.
getFinishedMergingWindows() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergingTriggerInfo
Return the merging windows in which the trigger is finished.
getFlattenResults() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns true/false if result flattening is enabled/disabled, or null if not applicable.
getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
Returns the KeyedCombineFn used by this Combine operation.
getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
Returns the CombineFnBase.PerKeyCombineFn used by this Combine operation.
getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
 
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
Returns a value in [0, 1] representing approximately what fraction of the current source this reader has read so far, or null if such an estimate is not available.
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
 
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
 
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
getFractionConsumed() - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Returns the approximate fraction of positions in the source that have been consumed by successful RangeTracker.tryReturnRecordAt(boolean, PositionT) calls, or 0.0 if no such calls have happened.
getFractionOfBlockConsumed() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
Returns the fraction of the block already consumed, if possible, as a value in [0, 1].
getFullName(PTransform<?, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Returns the full name of the currently being translated transform.
getFullName() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
 
getFullName() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
getFullNameForTesting(PTransform<?, ?>) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Deprecated.
this method is no longer compatible with the design of Pipeline, as PTransforms can be applied multiple times, with different names each time.
getGapDuration() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
getGcpCredential() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
The credential instance that should be used to authenticate against GCP services.
getGcsEndpoint() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
GCS endpoint to use.
getGcsUploadBufferSizeBytes() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
The buffer size (in bytes) to use when uploading files to GCS.
getGcsUtil() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
The GcsUtil instance that should be used to communicate with Google Cloud Storage.
getGoogleApiTrace() - Method in interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions
This option enables tracing of API calls to Google services used within the Dataflow SDK.
getHost() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
getId() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
Returns the id of this TupleTag.
getIdLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getIdLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getIncompatibleGlobalWindowErrorMessage() - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
Returns the error message for not supported default values in Combine.globally().
getIncompatibleGlobalWindowErrorMessage() - Method in class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
 
getIndex(TupleTag<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Returns the index for the given tuple tag, if the tag is present in this schema, -1 if it isn't.
getIndex() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
The zero-based index of this trigger firing that produced this pane.
getInput(PTransform<InputT, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Returns the input of the currently being translated transform.
getInput(PTransform<InputT, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Returns the input of the currently being processed transform.
getInput() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the transform input, in unexpanded form.
getInput() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
getInputs() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns a mapping of inputs to the producing nodes for all inputs to the transform.
getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Returns a TypeDescriptor capturing what is known statically about the input type of this DoFn instance's most-derived class.
getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns a TypeDescriptor capturing what is known statically about the input type of this DoFnWithContext instance's most-derived class.
getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.SimpleFunction
Returns a TypeDescriptor capturing what is known statically about the input type of this DoFn instance's most-derived class.
getInputValueCoder(Coder<KV<K, V>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
Returns the Coder of the values of the input to this transform.
getInsertDefault() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView
 
getInstanceComponents(T) - Static method in class com.google.cloud.dataflow.sdk.coders.AtomicCoder
Returns a list of values contained in the provided example value, one per type parameter.
getInstanceComponents(Object) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderFactory
Returns a list of objects contained in value, one per type argument, or null if none can be determined.
getInstanceComponents(Collection<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
Returns the first element in this collection if it is non-empty, otherwise returns null.
getInstanceComponents(Iterable<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
Returns the first element in this iterable if it is non-empty, otherwise returns null.
getInstanceComponents(KV<K, V>) - Static method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
getInstanceComponents(List<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.ListCoder
Returns the first element in this list if it is non-empty, otherwise returns null.
getInstanceComponents(Map<K, V>) - Static method in class com.google.cloud.dataflow.sdk.coders.MapCoder
Returns the key and value for an arbitrary element of this map, if it is non-empty, otherwise returns null.
getInstanceComponents(Set<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.SetCoder
Returns the first element in this set if it is non-empty, otherwise returns null.
getInstanceComponents(TimestampedValue<T>) - Static method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
getInstanceComponentsHelper(IterableT) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
Returns the first element in the iterable-like exampleValue if it is non-empty, otherwise returns null.
getInterfaces() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a set of TypeDescriptors, one for each interface implemented by this class.
getJAXBClass() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
 
getJob() - Method in exception com.google.cloud.dataflow.sdk.runners.DataflowJobException
Returns the failed job.
getJob() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.JobSpecification
 
getJobId() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
The identity of the Dataflow job.
getJobId() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Get the id of this job.
getJobMessageOutput() - Method in interface com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions
Output stream for job status messages.
getJobName() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
The Dataflow job name is used as an idempotence key within the Dataflow service.
getKey() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the key associated with this element.
getKey() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Identifier
 
getKey() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
 
getKey() - Method in class com.google.cloud.dataflow.sdk.values.KV
Returns the key of this KV.
getKeyCoder() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
getKeyCoder() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
getKeyCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns the key Coder for all PCollections in this KeyedPCollectionTuple.
getKeyedCollections() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns a list of TaggedKeyedPCollections for the PCollections contained in this KeyedPCollectionTuple.
getKindString() - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the name to use by default for this PTransform (not including the names of any enclosing PTransforms).
getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Returns a String capturing the kind of this PValueBase.
getLabel() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
Retrieve the optional label for an item.
getLastEmitted() - Method in class com.google.cloud.dataflow.sdk.io.CountingSource.CounterMark
Returns the last value emitted by the reader.
getLinkUrl() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
Retrieve the optional link URL for an item.
getMainOutputTag() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getMaxConditionCost() - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
The maximum cost (as a ratio of CPU time) allowed for evaluating conditional snapshots.
getMaxEndOffset(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
getMaxEndOffset(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
Returns the actual ending offset of the current source.
getMaxNumRecords() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getMaxNumWorkers() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
The maximum number of workers to use for the workerpool.
getMaxReadTime() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getMessage() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
getMessageType() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
Returns the Protocol Buffers Message type this ProtoCoder supports.
getMinBundleSize() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
Returns the minimum bundle size that should be used when splitting the source into sub-sources.
getMode() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
getMonthOfYear() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getName() - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator
Returns the name of the Aggregator.
getName() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the transform name.
getName() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Returns the name of this PCollection.
getName() - Method in interface com.google.cloud.dataflow.sdk.values.PValue
Returns the name of this PValue.
getName() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Returns the name of this PValueBase.
getNamespace() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
getNamespace() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Identifier
 
getNamespace() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
 
getNetwork() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
GCE network for launching workers.
getNonSpeculativeIndex() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
The zero-based index of this trigger firing among non-speculative panes.
getNumber() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
getNumber() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
getNumber() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getNumberOfWorkerHarnessThreads() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Number of threads to use on the Dataflow worker harness.
getNumShards() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getNumShards() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getNumWorkers() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Number of workers to use when executing the Dataflow job.
getOffset() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
getOffset() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
getOnly(TupleTag<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
If there is a singleton value for the given tag, returns it.
getOnly(TupleTag<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
If there is a singleton value for the given tag, returns it.
getOptions() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Returns the configured PipelineOptions.
getOriginalWindowFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
Returns the original windowFn that this InvalidWindows replaced.
getOutName(int) - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
If this TupleTag is tagging output outputIndex of a PTransform, returns the name that should be used by default for the output.
getOutput(PTransform<?, OutputT>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Returns the output of the currently being translated transform.
getOutput(PTransform<?, OutputT>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Returns the output of the currently being processed transform.
getOutput() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the transform output, in unexpanded form.
getOutput() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
getOutputStrategyInternal(WindowingStrategy<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Get the output strategy of this Window PTransform.
getOutputTime(Instant, GlobalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
getOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
getOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
 
getOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Deprecated.
Implement WindowFn.getOutputTimeFn() to return one of the appropriate OutputTimeFns, or a custom OutputTimeFn extending OutputTimeFn.Defaults.
getOutputTimeFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
getOutputTimeFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
Ensures that later sliding windows have an output time that is past the end of earlier windows.
getOutputTimeFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Provides a default implementation for WindowingStrategy.getOutputTimeFn().
getOutputType() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns a TypeDescriptor capturing what is known statically about the output type of this CombineFn instance's most-derived class.
getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Returns a TypeDescriptor capturing what is known statically about the output type of this DoFn instance's most-derived class.
getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns a TypeDescriptor capturing what is known statically about the output type of this DoFnWithContext instance's most-derived class.
getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.SimpleFunction
Returns a TypeDescriptor capturing what is known statically about the output type of this DoFn instance's most-derived class.
getOverriddenTransform() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
 
getOverrideWindmillBinary() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Custom windmill_main binary to use with the streaming runner.
getPathValidator() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The path validator instance that should be used to validate paths.
getPathValidatorClass() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The class of the validator that should be created and used to validate paths.
getPCollection(PCollection<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
Retrieves the value of the given PCollection.
getPCollectionList(PCollectionList<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
Retrieves the values of each PCollection in the given PCollectionList.
getPCollectionValuesWithMetadata(PCollection<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Retrieves the value of the given PCollection, along with element metadata such as timestamps and windows.
getPCollectionView(PCollectionView<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
Retrieves the values indicated by the given PCollectionView.
getPCollectionWindowedValues(PCollection<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
Retrieves the windowed value of the given PCollection.
getPeriod() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
getPipeline() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
 
getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.PBegin
 
getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
getPipeline() - Method in interface com.google.cloud.dataflow.sdk.values.PInput
Returns the owning Pipeline of this PInput.
getPipeline() - Method in interface com.google.cloud.dataflow.sdk.values.POutput
Returns the owning Pipeline of this POutput.
getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
 
getPipelineOptions() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptionsRegistrar
 
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Options
 
getPipelineOptions() - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Returns the configured pipeline options.
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Options
 
getPipelineOptions() - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Returns the configured pipeline options.
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Get the options used in this Pipeline.
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.Context
Returns the PipelineOptions specified with the PipelineRunner invoking this KeyedCombineFn.
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Returns the PipelineOptions specified with the PipelineRunner invoking this DoFn.
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Returns the PipelineOptions specified with the PipelineRunner invoking this DoFnWithContext.
getPipelineRunners() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Runner
 
getPipelineRunners() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Runner
 
getPipelineRunners() - Method in interface com.google.cloud.dataflow.sdk.runners.PipelineRunnerRegistrar
Get the set of PipelineRunners to register.
getPositionForFractionConsumed(double) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
Returns a position P such that the range [start, P) represents approximately the given fraction of the range [start, end).
getProducingTransformInternal() - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
Returns the AppliedPTransform that this POutputValueBase is an output of.
getProducingTransformInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PValue
Returns the AppliedPTransform that this PValue is an output of.
getProfilingAgentConfiguration() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions
 
getProject() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
getProject() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
Project id to use when launching jobs.
getProjectId() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Get the project this job exists in.
getPubsubRootUrl() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Root URL for use with the Pubsub API.
getQuery() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns the query to be read, or null if reading from a table instead.
getQuery() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
getRawType() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns the Class underlying the Type represented by this TypeDescriptor.
getReason() - Method in exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
getReasons() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
getRecordClass() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
getRecordElement() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
getRecordType() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
getRegisteredOptions() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
 
getReplacedByJob() - Method in exception com.google.cloud.dataflow.sdk.runners.DataflowJobUpdatedException
The new job that replaces the job terminated with this exception.
getReplacedByJob() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Returns a new DataflowPipelineJob for the job that replaced this one, if applicable.
getRootCause() - Method in exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
Returns the inner-most CannotProvideCoderException when they are deeply nested.
getRootElement() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
getRunner() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
The pipeline runner that will be used to execute the pipeline.
getRunner() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Returns the configured PipelineRunner.
getRunner() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
 
getRunner() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
 
getSchema() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns the schema used by this coder.
getSchema() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
getSchema() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getSchema() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
getSchema() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns the table schema.
getSchema() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns the schema used by this CoGbkResult.
getSecretsFile() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
This option controls which file to use when attempting to create the credentials using the OAuth 2 webflow.
getServiceAccountKeyfile() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
This option controls which file to use when attempting to create the credentials using the service account method.
getServiceAccountName() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
This option controls which service account to use when attempting to create the credentials using the service account method.
getShardNameTemplate() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns the current shard name template string.
getShardNameTemplate() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns the current shard name template string.
getShardTemplate() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getShardTemplate() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getShortValue() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
Return the optional short value for an item.
getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
 
getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
Returns the side inputs used by this Combine operation.
getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
Return the earliest window that contains the end of the main-input window.
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns the window of the side input corresponding to the given window of the main input.
getSideOutputTags() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getSink() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Returns the FileBasedSink for this write operation.
getSink() - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Returns the Sink that this write operation writes to.
getSink() - Method in class com.google.cloud.dataflow.sdk.io.Write.Bound
Returns the Sink associated with this PTransform.
getSink() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriteOperation
Return the XmlSink.Bound for this write operation.
getSize() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
getSize() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
getSource() - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded
Returns the BoundedSource used to create this Read PTransform.
getSource() - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
Returns the UnboundedSource used to create this Read PTransform.
getSplitBacklogBytes() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns the size of the backlog of unread data in the underlying data source represented by this split of this source.
getStableUniqueNames() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
Whether to check for stable unique names on each transform.
getStager() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The resource stager instance that should be used to stage resources.
getStagerClass() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The class responsible for staging resources to be accessible by workers during job execution.
getStagingLocation() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
GCS path for staging local files, e.g.
getStartDate() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
getStartDate() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
getStartDate() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getStartKey() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Returns the ByteKey representing the lower bound of this ByteKeyRange.
getStartOffset() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
Returns the starting offset of the source.
getStartPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
 
getStartPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
getStartPosition() - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Returns the starting position of the current range, inclusive.
getState() - Method in interface com.google.cloud.dataflow.sdk.PipelineResult
Retrieves the current state of the pipeline execution.
getState() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
 
getStepName(PTransform<?, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Gets the step name for this transform.
getStepNames() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.JobSpecification
Returns the mapping of AppliedPTransforms to the internal step name for that AppliedPTransform.
getStopPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
 
getStopPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
getStopPosition() - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Returns the ending position of the current range, exclusive.
getSubnetwork() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
GCE subnetwork for launching workers.
getSubscription() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getSupertype(Class<? super T>) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns the generic form of a supertype.
getTable() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns the table to write, or null if reading from a query instead.
getTable() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns the table reference, or null if a .
getTableId() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
Returns the table being read from.
getTableId() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
Returns the table being written to.
getTag(int) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Returns the tuple tag at the given index.
getTagInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
A unique identifier, for internal use.
getTeardownPolicy() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
The teardown policy for the VMs.
getTeardownPolicyName() - Method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.TeardownPolicy
 
getTempDatasetId() - Method in interface com.google.cloud.dataflow.sdk.options.BigQueryOptions
 
getTempLocation() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
A pipeline level default location for storing temporary files.
getTimestamp() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the timestamp associated with this element.
getTimestamp() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
getTimestampLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getTimestampLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getTimeZone() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
getTimeZone() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
getTimeZone() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getTiming() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
Return true if this is the last pane that will be produced in the associated window.
getTokenServerUrl() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
The token server URL to use for OAuth 2 authentication.
getTopic() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getTopic() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getTotalBacklogBytes() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns the size of the backlog of unread data in the underlying data source represented by all splits of this source.
getTotalValue(Combine.CombineFn<T, ?, T>) - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorValues
Get the total value of this Aggregator by applying the specified Combine.CombineFn.
getTransform() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the transform associated with this transform node.
getTransform() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
getTransformEvaluator(Class<TransformT>) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Returns the TransformEvaluator to use for instances of the specified PTransform class, or null if none registered.
getTransformNameMapping() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Mapping of old PTranform names to new ones, specified as JSON {"oldName":"newName",...}.
getTransformTranslator(Class<TransformT>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
Returns the DataflowPipelineTranslator.TransformTranslator to use for instances of the specified PTransform class, or null if none registered.
getTranslator() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Returns the DataflowPipelineTranslator associated with this object.
getTupleTag() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple.TaggedKeyedPCollection
Returns the TupleTag of this TaggedKeyedPCollection.
getTupleTagList() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Returns the TupleTagList tuple associated with this schema.
getType() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns the type this coder encodes/decodes.
getType() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getType() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
Retrieve the DisplayData.Type of display metadata.
getType() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns the Type represented by this TypeDescriptor.
getTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
Returns a TypeDescriptor capturing what is known statically about the type of this TupleTag instance's most-derived class.
getTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
Returns a TypeDescriptor<T> with some reflective information about T, if possible.
getTypeParameter(String) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a TypeVariable for the named type parameter.
getUnionTag() - Method in class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
 
getUpdate() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Deprecated.
This property is replaced by DataflowPipelineOptions.getUpdate()
getUpdate() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
Whether to update the currently running pipeline with the same name as this one.
getValidate() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns true if table validation is enabled.
getValidate() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns true if table validation is enabled.
getValue() - Method in class com.google.cloud.dataflow.sdk.coders.StructuralByteArray
 
getValue() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
Returns an immutable ByteString representing this ByteKey.
getValue() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the value associated with this element.
getValue() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
Retrieve the value of the metadata item.
getValue() - Method in class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
 
getValue() - Method in class com.google.cloud.dataflow.sdk.values.KV
Returns the value of this KV.
getValue() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
getValueCoder() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
getValueCoder() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
getValues() - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorValues
Get the values of the Aggregator at all steps it was used.
getValuesAtSteps() - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorValues
Get the values of the Aggregator by the user name at each step it was used.
getView() - Method in class com.google.cloud.dataflow.sdk.transforms.View.CreatePCollectionView
 
getWatermark() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns a timestamp before or at the timestamps of all future elements read by this reader.
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Returns a bound in watermark time by which this trigger would have fired at least once for a given window had there been input data.
getWindowedValue() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the WindowedValue associated with this element.
getWindowFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Get the WindowFn of this Window PTransform.
getWindowingStrategy() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Returns the WindowingStrategy of this PCollection.
getWindowingStrategyInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
For internal use only.
getWindows() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the collection of windows this element has been placed into.
getWorkerCacheMb() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
The size of the worker's in-memory cache, in megabytes.
getWorkerDiskType() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Specifies what type of persistent disk should be used.
getWorkerHarnessContainerImage() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Docker container image that executes Dataflow worker harness, residing in Google Container Registry.
getWorkerId() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
The identity of the worker running this pipeline.
getWorkerLogLevelOverrides() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
This option controls the log levels for specifically named loggers.
getWorkerMachineType() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Machine type to create Dataflow worker VMs as.
getWriteDisposition() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns the write disposition.
getWriteOperation() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Return the FileBasedWriteOperation that this Writer belongs to.
getWriteOperation() - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
Returns the write operation this writer belongs to.
getWriteOperation() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Return the XmlWriteOperation this write belongs to.
getWriterResultCoder() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Provides a coder for FileBasedSink.FileResult.
getWriterResultCoder() - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Returns a coder for the writer result type.
getZone() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
GCE availability zone for launching workers.
globally(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
Returns a PTransform that takes a PCollection<T> and returns a PCollection<List<T>> whose single value is a List of the approximate N-tiles of the elements of the input PCollection.
globally(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
Like ApproximateQuantiles.globally(int, Comparator), but sorts using the elements' natural ordering.
globally(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
Returns a PTransform that takes a PCollection<T> and returns a PCollection<Long> containing a single value that is an estimate of the number of distinct elements in the input PCollection.
globally(double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
Like ApproximateUnique.globally(int), but specifies the desired maximum estimation error instead of the sample size.
globally(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.Globally PTransform that uses the given SerializableFunction to combine all the elements in each window of the input PCollection into a single value in the output PCollection.
globally(CombineFnBase.GlobalCombineFn<? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.Globally PTransform that uses the given GloballyCombineFn to combine all the elements in each window of the input PCollection into a single value in the output PCollection.
globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Count
Returns a Combine.Globally PTransform that counts the number of elements in its input PCollection.
globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> whose contents is the maximum according to the natural ordering of T of the input PCollection's elements, or null if there are no elements.
globally(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> whose contents is the maximum of the input PCollection's elements, or null if there are no elements.
globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Mean
Returns a PTransform that takes an input PCollection<NumT> and returns a PCollection<Double> whose contents is the mean of the input PCollection's elements, or 0 if there are no elements.
globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> whose contents is the minimum according to the natural ordering of T of the input PCollection's elements, or null if there are no elements.
globally(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> whose contents is the minimum of the input PCollection's elements, or null if there are no elements.
GlobalWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing
The default window into which all data is placed (via GlobalWindows).
GlobalWindow.Coder - Class in com.google.cloud.dataflow.sdk.transforms.windowing
GlobalWindow.Coder for encoding and decoding GlobalWindows.
GlobalWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Default WindowFn that assigns all data to the same window.
GlobalWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
GoogleApiDebugOptions - Interface in com.google.cloud.dataflow.sdk.options
These options configure debug settings for Google API clients created within the Dataflow SDK.
GoogleApiDebugOptions.GoogleApiTracer - Class in com.google.cloud.dataflow.sdk.options
A GoogleClientRequestInitializer that adds the trace destination to Google API calls.
GoogleApiTracer() - Constructor for class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
 
greaterThan(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that are greater than a given value, based on the elements' natural ordering.
greaterThanEq(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that are greater than or equal to a given value, based on the elements' natural ordering.
GroupAlsoByWindow(WindowingStrategy<?, ?>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupAlsoByWindow
 
GroupByKey<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
GroupByKey<K, V> takes a PCollection<KV<K, V>>, groups the values by key and windows, and returns a PCollection<KV<K, Iterable<V>>> representing a map from each distinct key and window of the input PCollection to an Iterable over all the values associated with that key in the input per window.
GroupByKey.GroupAlsoByWindow<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Helper transform that takes a collection of timestamp-ordered values associated with each key, groups the values by window, combines windows as needed, and for each window in each key, outputs a collection of key/value-list pairs implicitly assigned to the window and with the timestamp derived from that window.
GroupByKey.GroupByKeyOnly<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Primitive helper transform that groups by key only, ignoring any window assignments.
GroupByKey.ReifyTimestampsAndWindows<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Helper transform that makes timestamps and window assignments explicit in the value part of each key/value pair.
GroupByKey.SortValuesByTimestamp<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Helper transform that sorts the values associated with each key by timestamp.
GroupByKeyOnly() - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
 
groupedValues(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.GroupedValues PTransform that takes a PCollection of KVs where a key maps to an Iterable of values, e.g., the result of a GroupByKey, then uses the given SerializableFunction to combine all the values associated with a key, ignoring the key.
groupedValues(CombineFnBase.GlobalCombineFn<? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.GroupedValues PTransform that takes a PCollection of KVs where a key maps to an Iterable of values, e.g., the result of a GroupByKey, then uses the given CombineFn to combine all the values associated with a key, ignoring the key.
groupedValues(CombineFnBase.PerKeyCombineFn<? super K, ? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.GroupedValues PTransform that takes a PCollection of KVs where a key maps to an Iterable of values, e.g., the result of a GroupByKey, then uses the given KeyedCombineFn to combine all the values associated with each key.

H

has(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns whether this PCollectionTuple contains a PCollection with the given tag.
hasDefaultValue() - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
Returns whether this transform has a default value.
HasDisplayData - Interface in com.google.cloud.dataflow.sdk.transforms.display
Marker interface for PTransforms and components used within PTransforms to specify display metadata to be used within UIs and diagnostic tools.
hashCode() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.Context
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.coders.StructuralByteArray
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Deprecated.
Object.hashCode() is not supported on DataflowAssert objects.
hashCode() - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Deprecated.
Object.hashCode() is not supported on DataflowAssert objects.
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Identifier
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester.OutputElementWithTimestamp
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.values.KV
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
 
hasReplacementJob() - Method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
 
Hidden - Annotation Type in com.google.cloud.dataflow.sdk.options
Methods and/or interfaces annotated with @Hidden will be suppressed from being output when --help is specified on the command-line.

I

identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
Returns the identity element of this operation, i.e.
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
Returns the value that should be used for the combine of the empty set.
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
Returns the identity element of this operation, i.e.
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
Returns the identity element of this operation, i.e.
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
 
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
 
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
 
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
 
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
 
IDENTITY - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
idLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a transform that's like this one but that reads unique message IDs from the given message attribute.
idLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a transform for reading from Cloud Pub/Sub where unique record identifiers are expected to be provided as Pub/Sub message attributes.
idLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new transform that's like this one but that publishes unique record IDs to a message attribute with the specified name.
idLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
Creates a transform that writes to Pub/Sub, adding each record's unique identifier to the published messages in an attribute with the specified name.
in(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PBegin
Returns a PBegin in the given Pipeline.
in(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PDone
Creates a PDone in the given Pipeline.
include(HasDisplayData) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
Include display metadata from the specified subcomponent.
INDEX_OF_MAX - Static variable in class com.google.cloud.dataflow.sdk.io.ShardNameTemplate
Shard name containing the index and max.
initialize(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Initialization of the sink.
initialize(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Performs initialization before writing to the sink.
initialize(AbstractGoogleClientRequest<?>) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
 
inOrder(Trigger<W>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
Returns an AfterEach Trigger with the given subtriggers.
INSTANCE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow.Coder
 
INSTANCE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow
Singleton instance of GlobalWindow.
INSTANCE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
 
InstantCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder for joda Instant that encodes it as a big endian Long shifted such that lexicographic ordering of the bytes corresponds to chronological order.
integersGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<Integer> and returns a PCollection<Integer> whose contents is the maximum of the input PCollection's elements, or Integer.MIN_VALUE if there are no elements.
integersGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<Integer> and returns a PCollection<Integer> whose contents is a single value that is the minimum of the input PCollection's elements, or Integer.MAX_VALUE if there are no elements.
integersGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<Integer> and returns a PCollection<Integer> whose contents is the sum of the input PCollection's elements, or 0 if there are no elements.
integersPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<KV<K, Integer>> and returns a PCollection<KV<K, Integer>> that contains an output element mapping each distinct key in the input PCollection to the maximum of the values associated with that key in the input PCollection.
integersPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<KV<K, Integer>> and returns a PCollection<KV<K, Integer>> that contains an output element mapping each distinct key in the input PCollection to the minimum of the values associated with that key in the input PCollection.
integersPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<KV<K, Integer>> and returns a PCollection<KV<K, Integer>> that contains an output element mapping each distinct key in the input PCollection to the sum of the values associated with that key in the input PCollection.
interpolateKey(double) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Returns a ByteKey key such that [startKey, key) represents approximately the specified fraction of the range [startKey, endKey).
intersects(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns whether this window intersects the given window.
IntervalWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing
An implementation of BoundedWindow that represents an interval from IntervalWindow.start (inclusive) to IntervalWindow.end (exclusive).
IntervalWindow(Instant, Instant) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Creates a new IntervalWindow that represents the half-open time interval [start, end).
IntervalWindow(Instant, ReadableDuration) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
into(WindowFn<? super T, ?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Creates a Window PTransform that uses the given WindowFn to window the data.
into(WindowFn<? super T, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
Returns a new Window PTransform that's like this transform but that will use the given WindowFn, and that has its input and output types bound.
IntraBundleParallelization - Class in com.google.cloud.dataflow.sdk.transforms
Provides multi-threading of DoFns, using threaded execution to process multiple elements concurrently within a bundle.
IntraBundleParallelization() - Constructor for class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization
 
IntraBundleParallelization.Bound<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that, when applied to a PCollection<InputT>, invokes a user-specified DoFn<InputT, OutputT> on all its elements, with all its outputs collected into an output PCollection<OutputT>.
IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A multi-threaded DoFn wrapper.
IntraBundleParallelization.Unbound - Class in com.google.cloud.dataflow.sdk.transforms
An incomplete IntraBundleParallelization transform, with unbound input/output types.
InvalidWindows<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that represents an invalid pipeline state.
InvalidWindows(String, WindowFn<?, W>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
is(T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
isArray() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns true if this type is known to be an array type.
isAtSplitPoint() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Returns true if the reader is at a split point.
isAtSplitPoint() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Returns true only for the first record; compressed sources cannot be split.
isAtSplitPoint() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
Returns whether the current record is at a split point (i.e., whether the current record would be the first record to be read by a source with a specified start offset of OffsetBasedSource.OffsetBasedReader.getCurrentOffset()).
isBounded() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
 
isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
InvalidWindows objects with the same originalWindowFn are compatible.
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Returns whether this performs the same triggering as the given Trigger.
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns whether this performs the same merging as the given WindowFn.
isCompositeNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns true if this node represents a composite transform that does not perform processing of its own, but merely encapsulates a sub-pipeline (which may be empty).
isDisjoint(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns whether this window is disjoint from the given window.
isEmpty() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
Returns true if the byte[] backing this ByteKey is of length 0.
isEmpty() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
 
isEmpty() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
 
isEqualTo(T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Checks that the value of this SingletonAssert's view is equal to the expected value.
isFinished() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Returns true if the current trigger is marked finished.
isFinished(int) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Return true if the given subtrigger is marked finished.
isFinishedSpecifyingInternal() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Returns whether this PValueBase has been finalized, and its core properties, e.g., name, can no longer be changed.
isFirst() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
Return true if this is the first pane produced for the associated window.
isLast() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
Return true if this is the last pane that will be produced in the associated window.
isMerging() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Returns true if the windowing strategy of the current PCollection is a merging WindowFn.
isNonMerging() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.NonMergingWindowFn
 
isNonMerging() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns true if this WindowFn never needs to merge any windows.
isRegisterByteSizeObserverCheap(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(byte[], Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(Byte, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(ByteString, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(T, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(Double, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(ReadableDuration, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(Instant, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(IterableT, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(KV<K, V>, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
Returns whether both keyCoder and valueCoder are considered not expensive.
isRegisterByteSizeObserverCheap(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
NullableCoder is cheap if valueCoder is cheap.
isRegisterByteSizeObserverCheap(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(Void, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRootNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
 
isSplittable() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Determines whether a single file represented by this source is splittable.
isSplittable() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
Determines whether a file represented by this source is can be split into bundles.
isStreaming() - Method in interface com.google.cloud.dataflow.sdk.options.StreamingOptions
Set to true if running a streaming pipeline.
isSubtypeOf(TypeDescriptor<?>) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Return true if this type is a subtype of the given type.
isSupertypeOf(TypeDescriptor<?>) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns true if this type is assignable from the given type.
isTerminal() - Method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
 
isTestEncodability() - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
Controls whether the runner should ensure that all of the elements of every PCollection can be encoded using the appropriate Coder.
isTestSerializability() - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
Controls whether the runner should ensure that all of the elements of the pipeline, such as DoFns, can be serialized.
isTestUnorderedness() - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
Controls whether the runner should randomize the order of each PCollection.
isUnknown() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
Return true if there is no timing information for the current PaneInfo.
isWholeStream - Variable in class com.google.cloud.dataflow.sdk.coders.Coder.Context
Whether the encoded or decoded value fills the remainder of the output or input (resp.) record/stream contents.
items() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData
 
IterableAssert(PTransform<PBegin, PCollectionView<Iterable<T>>>, Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
 
IterableCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
An IterableCoder encodes any Iterable in the format of IterableLikeCoder.
IterableCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
IterableLikeCoder<T,IterableT extends Iterable<T>> - Class in com.google.cloud.dataflow.sdk.coders
An abstract base class with functionality for assembling a Coder for a class that implements Iterable.
IterableLikeCoder(Coder<T>, String) - Constructor for class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
iterables() - Static method in class com.google.cloud.dataflow.sdk.transforms.Flatten
Returns a PTransform that takes a PCollection<Iterable<T>> and returns a PCollection<T> containing all the elements from all the Iterables.

J

JAXBCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A coder for JAXB annotated objects.
JobNameFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions.JobNameFactory
 
JobSpecification(Job, Map<AppliedPTransform<?, ?, ?>, String>) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.JobSpecification
 
jobToString(Job) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
Renders a Job as a string.

K

KeyedCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
 
KeyedCombineFnWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
 
KeyedPCollectionTuple<K> - Class in com.google.cloud.dataflow.sdk.transforms.join
An immutable tuple of keyed PCollections with key type K.
KeyedPCollectionTuple.TaggedKeyedPCollection<K,V> - Class in com.google.cloud.dataflow.sdk.transforms.join
A utility class to help ensure coherence of tag and input PCollection types.
Keys<K> - Class in com.google.cloud.dataflow.sdk.transforms
Keys<K> takes a PCollection of KV<K, V>s and returns a PCollection<K> of the keys.
KV<K,V> - Class in com.google.cloud.dataflow.sdk.values
An immutable key/value pair.
KV.OrderByKey<K extends Comparable<? super K>,V> - Class in com.google.cloud.dataflow.sdk.values
A Comparator that orders KVs by the natural ordering of their keys.
KV.OrderByValue<K,V extends Comparable<? super V>> - Class in com.google.cloud.dataflow.sdk.values
A Comparator that orders KVs by the natural ordering of their values.
KvCoder<K,V> - Class in com.google.cloud.dataflow.sdk.coders
A KvCoder encodes KVs.
KvCoderBase<T> - Class in com.google.cloud.dataflow.sdk.coders
Deprecated.
KvCoderBase() - Constructor for class com.google.cloud.dataflow.sdk.coders.KvCoderBase
Deprecated.
 
KvSwap<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
KvSwap<K, V> takes a PCollection<KV<K, V>> and returns a PCollection<KV<V, K>>, where all the keys and values have been swapped.

L

largest(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<List<T>> with a single element containing the largest count elements of the input PCollection<T>, in decreasing order, sorted according to their natural order.
Largest() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Top.Largest
 
largestPerKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<KV<K, V>> and returns a PCollection<KV<K, List<V>>> that contains an output element mapping each distinct key in the input PCollection to the largest count values associated with that key in the input PCollection<KV<K, V>>, in decreasing order, sorted according to their natural order.
LargestUnique(long) - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique
Creates a heap to track the largest sampleSize elements.
leaveCompositeTransform(TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
Called for each composite transform after all of its component transforms and their outputs have been visited.
leaveCompositeTransform(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
lessThan(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection and returns a PCollection with elements that are less than a given value, based on the elements' natural ordering.
lessThanEq(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that are less than or equal to a given value, based on the elements' natural ordering.
ListCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A Coder for List, using the format of IterableLikeCoder.
ListCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.ListCoder
 
longsGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<Long> and returns a PCollection<Long> whose contents is the maximum of the input PCollection's elements, or Long.MIN_VALUE if there are no elements.
longsGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<Long> and returns a PCollection<Long> whose contents is the minimum of the input PCollection's elements, or Long.MAX_VALUE if there are no elements.
longsGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<Long> and returns a PCollection<Long> whose contents is the sum of the input PCollection's elements, or 0 if there are no elements.
longsPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<KV<K, Long>> and returns a PCollection<KV<K, Long>> that contains an output element mapping each distinct key in the input PCollection to the maximum of the values associated with that key in the input PCollection.
longsPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<KV<K, Long>> and returns a PCollection<KV<K, Long>> that contains an output element mapping each distinct key in the input PCollection to the minimum of the values associated with that key in the input PCollection.
longsPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<KV<K, Long>> and returns a PCollection<KV<K, Long>> that contains an output element mapping each distinct key in the input PCollection to the sum of the values associated with that key in the input PCollection.

M

MapCoder<K,V> - Class in com.google.cloud.dataflow.sdk.coders
A Coder for Maps that encodes them according to provided coders for keys and values.
MapCoderBase<T> - Class in com.google.cloud.dataflow.sdk.coders
Deprecated.
MapCoderBase() - Constructor for class com.google.cloud.dataflow.sdk.coders.MapCoderBase
Deprecated.
 
MapElements<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for mapping a simple function over the elements of a PCollection.
MapElements.MissingOutputTypeDescriptor<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
An intermediate builder for a MapElements transform.
mappedTo(SerializableFunction<Instant, Instant>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
matches(String) - Method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
Returns true if the given file name implies that the contents are compressed according to the compression embodied by this factory.
matches(String) - Method in enum com.google.cloud.dataflow.sdk.io.TextIO.CompressionType
Determine if a given filename matches a compression type based on its extension.
Max - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for computing the maximum of the elements in a PCollection, or the maximum of the values associated with each key in a PCollection of KVs.
Max.MaxDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
Max.MaxFn<T> - Class in com.google.cloud.dataflow.sdk.transforms
Max.MaxIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
Max.MaxLongFn - Class in com.google.cloud.dataflow.sdk.transforms
MaxDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Max.MaxDoubleFn
 
MaxIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Max.MaxIntegerFn
 
MaxLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Max.MaxLongFn
 
maxNumRecords(int) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a transform that's like this one but will only read up to the specified maximum number of records from Cloud Pub/Sub.
maxNumRecords(int) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a transform for reading from Cloud Pub/Sub with a maximum number of records that will be read.
maxReadTime(Duration) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a transform that's like this one but will only read during the specified duration from Cloud Pub/Sub.
maxReadTime(Duration) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a transform for reading from Cloud Pub/Sub with a maximum number of duration during which records will be read.
maxTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
Returns the inclusive upper bound of timestamps for values in this window.
maxTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow
 
maxTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns the largest timestamp that can be included in this window.
Mean - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for computing the arithmetic mean (a.k.a.
merge(W, Iterable<? extends Instant>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
Merges the given output times, presumed to be combined output times for windows that are merging, into an output time for the resultWindow.
merge(W, Iterable<? extends Instant>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
Merges the given output times, presumed to be combined output times for windows that are merging, into an output time for the resultWindow.
merge(W, Iterable<? extends Instant>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn
Merges the given output times, presumed to be combined output times for windows that are merging, into an output time for the resultWindow.
merge(Collection<W>, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.MergeContext
Signals to the framework that the windows in toBeMerged should be merged together to form mergeResult.
mergeAccumulator(AccumT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn.Accumulator
Adds the input values represented by the given accumulator into this accumulator.
mergeAccumulators(Iterable<ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique>) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
mergeAccumulators(Iterable<AccumT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
 
mergeAccumulators(Iterable<double[]>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
mergeAccumulators(Iterable<Combine.Holder<V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
mergeAccumulators(Iterable<int[]>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
mergeAccumulators(Iterable<long[]>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
mergeAccumulators(Iterable<AccumT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns an accumulator representing the accumulation of all the input values accumulated in the merging accumulators.
mergeAccumulators(Iterable<List<V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
 
mergeAccumulators(K, Iterable<AccumT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Returns an accumulator representing the accumulation of all the input values accumulated in the merging accumulators.
mergeAccumulators(Iterable<Object[]>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
 
mergeAccumulators(Iterable<Object[]>, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
 
mergeAccumulators(K, Iterable<Object[]>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
 
mergeAccumulators(K, Iterable<Object[]>, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
 
mergeAccumulators(Iterable<AccumT>, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
Returns an accumulator representing the accumulation of all the input values accumulated in the merging accumulators.
mergeAccumulators(K, Iterable<AccumT>, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
Returns an accumulator representing the accumulation of all the input values accumulated in the merging accumulators.
mergeAccumulators(Iterable<Top.BoundedHeap<KV<Integer, T>, SerializableComparator<KV<Integer, T>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
MergeContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.MergeContext
 
MergeOverlappingIntervalWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A utility function for merging overlapping IntervalWindows.
MergeOverlappingIntervalWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.MergeOverlappingIntervalWindows
 
mergeWindows(WindowFn<Object, W>.MergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
mergeWindows(WindowFn<?, IntervalWindow>.MergeContext) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.MergeOverlappingIntervalWindows
Merge overlapping IntervalWindows.
mergeWindows(WindowFn<T, W>.MergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.NonMergingWindowFn
 
mergeWindows(WindowFn<Object, IntervalWindow>.MergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
mergeWindows(WindowFn<T, W>.MergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Does whatever merging of windows is necessary.
mimeType - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
The MIME type used in the creation of the output channel (if the file system supports it).
Min - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for computing the minimum of the elements in a PCollection, or the minimum of the values associated with each key in a PCollection of KVs.
Min.MinDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
Min.MinFn<T> - Class in com.google.cloud.dataflow.sdk.transforms
Min.MinIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
Min.MinLongFn - Class in com.google.cloud.dataflow.sdk.transforms
MinDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Min.MinDoubleFn
 
MinIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Min.MinIntegerFn
 
MinLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Min.MinLongFn
 
modifyEnvironmentBeforeSubmission(Environment) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunnerHooks
Allows the user to modify the environment of their job before their job is submitted to the service for execution.
months(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
Returns a WindowFn that windows elements into periods measured by months.
MultiThreadedIntraBundleProcessingDoFn(DoFn<InputT, OutputT>, int) - Constructor for class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 

N

name - Variable in class com.google.cloud.dataflow.sdk.transforms.PTransform
The base name of this PTransform, e.g., from ParDo.named(String), or from defaults, or null if not yet assigned.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new PTransform that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns a PTransform with the given step name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform with the given step name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns a copy of this transform using the name associated with this transformation.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
Returns a BigQueryIO.Read.Bound with the given name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns a copy of this write transformation, but with the specified transform name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation with the given transform name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a transform that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a transform for reading from Cloud Pub/Sub with the specified transform name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new transform that's like this one but with the specified step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
Creates a transform that writes to Pub/Sub with the given step name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded
Returns a new Bounded PTransform that's like this one but has the given name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.Read
Returns a new Read PTransform builder with the given name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
Returns a new Unbounded PTransform that's like this one but has the given name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new transform for reading from text files that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a transform for reading text files that uses the given step name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a transform for writing to text files that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a transform for writing to text files with the given step name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
Return a new Globally transform that's like this transform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
Return a new Globally transform that's like this transform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Filter
 
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
Returns a new ParDo PTransform that's like this PTransform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
Returns a new multi-output ParDo PTransform that's like this PTransform but with the specified name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo PTransform with the given name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo transform that's like this transform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo transform that's like this transform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Returns a new Window PTransform that's like this PTransform but with the specified name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Creates a Window PTransform with the given name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
Returns a new Window transform that's like this transform but with the specified name.
naturalOrder(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
 
naturalOrder() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
 
naturalOrder(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
 
naturalOrder() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
 
needsValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
needsValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
needsValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
needsValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
NESTED - Static variable in class com.google.cloud.dataflow.sdk.coders.Coder.Context
The nested context: the value being encoded or decoded is (potentially) a part of a larger record/stream contents, and may have other parts encoded or decoded after it.
nested() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.Context
 
newWith(List<SerializableFunction<Instant, Instant>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
To complete an implementation, return a new instance like this one, but incorporating the provided timestamp mapping functions.
newWith(List<SerializableFunction<Instant, Instant>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
NO_FIRING - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
PaneInfo to use for elements on (and before) initial window assignemnt (including elements read from sources) before they have passed through a GroupByKey and are associated with a particular trigger firing.
NonDeterministicException(Coder<?>, String, Coder.NonDeterministicException) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
NonDeterministicException(Coder<?>, String) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
NonDeterministicException(Coder<?>, List<String>) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
NonDeterministicException(Coder<?>, List<String>, Coder.NonDeterministicException) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
none() - Static method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData
Default empty DisplayData instance.
NonMergingWindowFn<T,W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Abstract base class for WindowFns that do not merge windows.
NonMergingWindowFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.NonMergingWindowFn
 
notEqualTo(T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Checks that the value of this SingletonAssert's view is not equal to the expected value.
NullableCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A NullableCoder encodes nullable values of type T using a nested Coder<T> that does not tolerate null values.

O

of(TypeDescriptor<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns an AvroCoder instance for the provided element type.
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns an AvroCoder instance for the provided element class.
of(Schema) - Static method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns an AvroCoder instance for the Avro schema.
of(Class<T>, Schema) - Static method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns an AvroCoder instance for the provided element type using the provided Avro schema.
of(String, String) - Static method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
 
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
 
of(List<Object>) - Static method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
 
of(String, String, String, String) - Static method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
 
of(Coder<IntermediateT>, DelegateCoder.CodingFunction<T, IntermediateT>, DelegateCoder.CodingFunction<IntermediateT, T>) - Static method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.DurationCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
 
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
of(List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
Create a coder for a given type of JAXB annotated objects.
of(String) - Static method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
Constructor for JSON deserialization only.
of(Coder<K>, Coder<V>) - Static method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
of(List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
of(String, boolean, List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.KvCoderBase
Deprecated.
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.ListCoder
 
of(List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.ListCoder
 
of(Coder<K>, Coder<V>) - Static method in class com.google.cloud.dataflow.sdk.coders.MapCoder
Produces a MapCoder with the given keyCoder and valueCoder.
of(List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
of(String, List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.MapCoderBase
Deprecated.
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
 
of(List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
 
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
Returns a Proto2Coder for the given Protobuf message class.
of(TypeDescriptor<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
Returns a Proto2Coder for the given Protobuf message class.
of(String, List<String>) - Static method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
Constructor for JSON deserialization only.
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
Returns a ProtoCoder for the given Protocol Buffers Message.
of(TypeDescriptor<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
Returns a ProtoCoder for the Protocol Buffers Message indicated by the given TypeDescriptor.
of(String, List<String>) - Static method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
Deprecated.
For JSON deserialization only.
of(TypeDescriptor<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
Returns a SerializableCoder instance for the provided element type.
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
Returns a SerializableCoder instance for the provided element class.
of(String) - Static method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.SetCoder
Produces a SetCoder with the given elementCoder.
of(List<Object>) - Static method in class com.google.cloud.dataflow.sdk.coders.SetCoder
Dynamically typed constructor for JSON deserialization.
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
 
of(ByteString) - Static method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
Creates a new ByteKey backed by the specified ByteString.
of(int...) - Static method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
Creates a new ByteKey backed by a copy of the specified int[].
of(ByteKey, ByteKey) - Static method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Creates a new ByteKeyRange with the given start and end keys.
of(ByteKeyRange) - Static method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
Instantiates a new ByteKeyRangeTracker with the specified range.
of(WindowedValue<V>) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns a new ValueWithMetadata with the WindowedValue.
of(String, InputT, OutputT, TransformT) - Static method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
of(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
Returns a CombineFn that uses the given SerializableFunction to combine values.
of(SerializableFunction<Iterable<V>, V>, int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
Returns a CombineFn that uses the given SerializableFunction to combine values, attempting to buffer at least bufferSize values between invocations.
of(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine.SimpleCombineFn
Deprecated.
of(Iterable<T>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.Values transform that produces a PCollection containing elements of the provided Iterable.
of(T...) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.Values transform that produces a PCollection containing the specified elements.
of(Map<K, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.Values transform that produces a PCollection of KVs corresponding to the keys and values of the specified Map.
of(Class<? extends DoFnWithContext>) - Static method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
 
of(DoFn<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns a DoFnTester supporting unit-testing of the given DoFn.
of(DoFnWithContext<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns a DoFnTester supporting unit-testing of the given DoFn.
of(DoFn<NewInputT, NewOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound
Returns a new IntraBundleParallelization PTransform like this one with the specified DoFn.
of(DoFn<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization
Creates a IntraBundleParallelization PTransform for the given DoFn that processes elements using multiple threads.
of(DoFn<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Unbound
Returns a new IntraBundleParallelization PTransform like this one with the specified DoFn.
of(CoGbkResultSchema, UnionCoder) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
Returns a CoGbkResult.CoGbkResultCoder for the given schema and UnionCoder.
of(List<Coder<?>>, CoGbkResultSchema) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
of(TupleTag<V>, List<V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns a new CoGbkResult that contains just the given tag and given data.
of(List<TupleTag<?>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
of(TupleTag<InputT>, PCollection<KV<K, InputT>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns a new KeyedPCollectionTuple<K> with the given tag and initial PCollection.
of(T, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
 
of(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
 
of(T, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
 
of(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
 
of(DoFn<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo PTransform that will invoke the given DoFn function.
of(DoFnWithContext<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo PTransform that will invoke the given DoFnWithContext function.
of(DoFn<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo PTransform that's like this transform but that will invoke the given DoFn function, and that has its input and output types bound.
of(DoFnWithContext<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo PTransform that's like this transform but which will invoke the given DoFnWithContext function, and which has its input and output types bound.
of(DoFn<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo PTransform that's like this transform but that will invoke the given DoFn function, and that has its input type bound.
of(DoFnWithContext<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo PTransform that's like this transform but which will invoke the given DoFnWithContext function, and which has its input type bound.
of(int, Partition.PartitionFn<? super T>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Partition
Returns a new Partition PTransform that divides its input PCollection into the given number of partitions, using the given partitioning function.
of(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<List<T>> with a single element containing the largest count elements of the input PCollection<T>, in decreasing order, sorted using the given Comparator<T>.
of(PCollectionView<ViewT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.View.CreatePCollectionView
 
of(Trigger.OnceTrigger<W>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
Returns an AfterAll Trigger with the given subtriggers.
of(Trigger.OnceTrigger<W>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
Returns an AfterFirst Trigger with the given subtriggers.
of() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
Returns the default trigger.
of(Duration) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
Partitions the timestamp space into half-open intervals of the form [N * size, (N + 1) * size), where 0 is the epoch.
of(Duration) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
Assigns timestamps into half-open intervals of the form [N * period, N * period + size), where 0 is the epoch.
of(SerializableFunction<V, K>) - Static method in class com.google.cloud.dataflow.sdk.transforms.WithKeys
Returns a PTransform that takes a PCollection<V> and returns a PCollection<KV<K, V>>, where each of the values in the input PCollection has been paired with a key computed from the value by invoking the given SerializableFunction.
of(K) - Static method in class com.google.cloud.dataflow.sdk.transforms.WithKeys
Returns a PTransform that takes a PCollection<V> and returns a PCollection<KV<K, V>>, where each of the values in the input PCollection has been paired with the given key.
of(SerializableFunction<T, Instant>) - Static method in class com.google.cloud.dataflow.sdk.transforms.WithTimestamps
For a SerializableFunction fn from T to Instant, outputs a PTransform that takes an input PCollection<T> and outputs a PCollection<T> containing every element v in the input where each element is output with a timestamp obtained as the result of fn.apply(v).
of(K, V) - Static method in class com.google.cloud.dataflow.sdk.values.KV
Returns a KV with the given key and value.
of(PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns a singleton PCollectionList containing the given PCollection.
of(Iterable<PCollection<T>>) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns a PCollectionList containing the given PCollections, in order.
of(TupleTag<T>, PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns a singleton PCollectionTuple containing the given PCollection keyed by the given TupleTag.
of(V, Instant) - Static method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
Returns a new TimestampedValue with the given value and timestamp.
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
of(List<Object>) - Static method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
of(TupleTag<?>) - Static method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns a singleton TupleTagList containing the given TupleTag.
of(List<TupleTag<?>>) - Static method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns a TupleTagList containing the given TupleTags, in order.
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a TypeDescriptor representing the given type.
of(Type) - Static method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a TypeDescriptor representing the given type.
OFFSET_INFINITY - Static variable in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
Offset corresponding to infinity.
OffsetBasedReader(OffsetBasedSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
 
OffsetBasedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A BoundedSource that uses offsets to define starting and ending positions.
OffsetBasedSource(long, long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
 
OffsetBasedSource.OffsetBasedReader<T> - Class in com.google.cloud.dataflow.sdk.io
A Source.Reader that implements code common to readers of all OffsetBasedSources.
OffsetRangeTracker - Class in com.google.cloud.dataflow.sdk.io.range
A RangeTracker for non-negative positions of type long.
OffsetRangeTracker(long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
Creates an OffsetRangeTracker for the specified range.
ofPrimitiveOutputsInternal(Pipeline, TupleTagList, WindowingStrategy<?, ?>, PCollection.IsBounded) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns a PCollectionTuple with each of the given tags mapping to a new output PCollection.
ofRecordClass(Class<T>) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
Returns an XmlSink that writes objects of the class specified as XML elements.
ON_TIME_AND_ONLY_FIRING - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
PaneInfo to use when there will be exactly one firing and it is on time.
OnceTrigger(List<Trigger<W>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Called immediately after an element is first incorporated into a window.
OnElementContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
 
onFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
onFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
onFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
onFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger
Adjusts the state of the trigger to be ready for the next pane.
onFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Adjusts the state of the trigger to be ready for the next pane.
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Called immediately after windows have been merged.
OnMergeContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
 
onOnlyFiring(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
onOnlyFiring(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
onOnlyFiring(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
onOnlyFiring(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
onOnlyFiring(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
 
onOnlyFiring(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger
open(String) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Opens the channel.
open(String) - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
Performs bundle initialization.
Options() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Options
 
Options() - Constructor for class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Options
 
OrderByKey() - Constructor for class com.google.cloud.dataflow.sdk.values.KV.OrderByKey
 
OrderByValue() - Constructor for class com.google.cloud.dataflow.sdk.values.KV.OrderByValue
 
orFinally(Trigger.OnceTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Specify an ending condition for this trigger.
OUTER - Static variable in class com.google.cloud.dataflow.sdk.coders.Coder.Context
The outer context: the value being encoded or decoded takes up the remainder of the record/stream contents.
output(OutputT) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Adds the given element to the main output PCollection.
output(OutputT) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Adds the given element to the main output PCollection.
outputAtEarliestInputTimestamp() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFns
The policy of outputting at the earliest of the input timestamps for non-late input data that led to a computed value.
outputAtEndOfWindow() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFns
The policy of outputting with timestamps at the end of the window.
outputAtLatestInputTimestamp() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFns
The policy of holding the watermark to the latest of the input timestamps for non-late input data that led to a computed value.
OutputTimeFn<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
(Experimental) A function from timestamps of input values to the timestamp for a computed value.
OutputTimeFn.Defaults<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
(Experimental) Default method implementations for OutputTimeFn where the output time depends on the input element timestamps and possibly the window.
OutputTimeFn.DependsOnlyOnWindow<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
(Experimental) Default method implementations for OutputTimeFn when the output time depends only on the window.
OutputTimeFns - Class in com.google.cloud.dataflow.sdk.transforms.windowing
(Experimental) Static utility methods and provided implementations for OutputTimeFn.
OutputTimeFns() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFns
 
outputWithTimestamp(OutputT, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Adds the given element to the main output PCollection, with the given timestamp.
outputWithTimestamp(OutputT, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Adds the given element to the main output PCollection, with the given timestamp.
overlaps(ByteKeyRange) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Returns true if the specified ByteKeyRange overlaps this range.

P

pane() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns information about the pane within this window into which the input element has been assigned.
pane() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
Returns information about the pane within this window into which the input element has been assigned.
PaneInfo - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Provides information about the pane an element belongs to.
PaneInfo.PaneInfoCoder - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A Coder for encoding PaneInfo instances.
PaneInfo.Timing - Enum in com.google.cloud.dataflow.sdk.transforms.windowing
Enumerates the possibilities for the timing of this pane firing related to the input and output watermarks for its computation.
PaneInfoCoder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
 
ParDo - Class in com.google.cloud.dataflow.sdk.transforms
ParDo is the core element-wise transform in Google Cloud Dataflow, invoking a user-specified function on each of the elements of the input PCollection to produce zero or more output elements, all of which are collected into the output PCollection.
ParDo() - Constructor for class com.google.cloud.dataflow.sdk.transforms.ParDo
 
ParDo.Bound<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that, when applied to a PCollection<InputT>, invokes a user-specified DoFn<InputT, OutputT> on all its elements, with all its outputs collected into an output PCollection<OutputT>.
ParDo.BoundMulti<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that, when applied to a PCollection<InputT>, invokes a user-specified DoFn<InputT, OutputT> on all its elements, which can emit elements to any of the PTransform's main and side output PCollections, which are bundled into a result PCollectionTuple.
ParDo.Unbound - Class in com.google.cloud.dataflow.sdk.transforms
An incomplete ParDo transform, with unbound input/output types.
ParDo.UnboundMulti<OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
An incomplete multi-output ParDo transform, with unbound input type.
parseTableSpec(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO
Parse a table specification in the form "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]".
Partition<T> - Class in com.google.cloud.dataflow.sdk.transforms
Partition takes a PCollection<T> and a PartitionFn, uses the PartitionFn to split the elements of the input PCollection into N partitions, and returns a PCollectionList<T> that bundles N PCollection<T>s containing the split elements.
Partition.PartitionFn<T> - Interface in com.google.cloud.dataflow.sdk.transforms
A function object that chooses an output partition for an element.
partitionFor(T, int) - Method in interface com.google.cloud.dataflow.sdk.transforms.Partition.PartitionFn
Chooses the partition into which to put the given element.
PartitioningWindowFn<T,W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that places each value into exactly one window based on its timestamp and never merges windows.
PartitioningWindowFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
 
pastEndOfWindow() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark
Creates a trigger that fires when the watermark passes the end of the window.
pastFirstElementInPane() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
Creates a trigger that fires when the current processing time passes the processing time at which this trigger saw the first element in a pane.
PathValidatorFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.PathValidatorFactory
 
PBegin - Class in com.google.cloud.dataflow.sdk.values
PBegin is the "input" to a root PTransform, such as Read or Create.
PBegin(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.PBegin
Constructs a PBegin in the given Pipeline.
PCollection<T> - Class in com.google.cloud.dataflow.sdk.values
A PCollection<T> is an immutable collection of values of type T.
PCollection.IsBounded - Enum in com.google.cloud.dataflow.sdk.values
The enumeration of cases for whether a PCollection is bounded.
PCollectionList<T> - Class in com.google.cloud.dataflow.sdk.values
A PCollectionList<T> is an immutable list of homogeneously typed PCollection<T>s.
pCollections() - Static method in class com.google.cloud.dataflow.sdk.transforms.Flatten
Returns a PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.
PCollectionTuple - Class in com.google.cloud.dataflow.sdk.values
A PCollectionTuple is an immutable tuple of heterogeneously-typed PCollections, "keyed" by TupleTags.
PCollectionView<T> - Interface in com.google.cloud.dataflow.sdk.values
A PCollectionView<T> is an immutable view of a PCollection as a value of type T that can be accessed as a side input to a ParDo transform.
PDone - Class in com.google.cloud.dataflow.sdk.values
PDone is the output of a PTransform that has a trivial result, such as a Write.
peekOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the main output.
peekOutputElementsWithTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the main output with associated timestamps.
peekSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the side output with the given tag.
perElement() - Static method in class com.google.cloud.dataflow.sdk.transforms.Count
Returns a Count.PerElement PTransform that counts the number of occurrences of each element in its input PCollection.
PerElement() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Count.PerElement
 
perKey(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
Returns a PTransform that takes a PCollection<KV<K, V>> and returns a PCollection<KV<K, List<V>>> that contains an output element mapping each distinct key in the input PCollection to a List of the approximate N-tiles of the values associated with that key in the input PCollection.
perKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
Like ApproximateQuantiles.perKey(int, Comparator), but sorts values using the their natural ordering.
perKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
Returns a PTransform that takes a PCollection<KV<K, V>> and returns a PCollection<KV<K, Long>> that contains an output element mapping each distinct key in the input PCollection to an estimate of the number of distinct values associated with that key in the input PCollection.
perKey(double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
Like ApproximateUnique.perKey(int), but specifies the desired maximum estimation error instead of the sample size.
perKey(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.PerKey PTransform that first groups its input PCollection of KVs by keys and windows, then invokes the given function on each of the values lists to produce a combined value, and then returns a PCollection of KVs mapping each distinct key to its combined value for each window.
perKey(CombineFnBase.GlobalCombineFn<? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.PerKey PTransform that first groups its input PCollection of KVs by keys and windows, then invokes the given function on each of the values lists to produce a combined value, and then returns a PCollection of KVs mapping each distinct key to its combined value for each window.
perKey(CombineFnBase.PerKeyCombineFn<? super K, ? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.PerKey PTransform that first groups its input PCollection of KVs by keys and windows, then invokes the given function on each of the key/values-lists pairs to produce a combined value, and then returns a PCollection of KVs mapping each distinct key to its combined value for each window.
perKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Count
Returns a Combine.PerKey PTransform that counts the number of elements associated with each key of its input PCollection.
perKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<KV<K, T>> and returns a PCollection<KV<K, T>> that contains an output element mapping each distinct key in the input PCollection to the maximum according to the natural ordering of T of the values associated with that key in the input PCollection.
perKey(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<KV<K, T>> and returns a PCollection<KV<K, T>> that contains one output element per key mapping each to the maximum of the values associated with that key in the input PCollection.
perKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Mean
Returns a PTransform that takes an input PCollection<KV<K, N>> and returns a PCollection<KV<K, Double>> that contains an output element mapping each distinct key in the input PCollection to the mean of the values associated with that key in the input PCollection.
perKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<KV<K, T>> and returns a PCollection<KV<K, T>> that contains an output element mapping each distinct key in the input PCollection to the minimum according to the natural ordering of T of the values associated with that key in the input PCollection.
perKey(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<KV<K, T>> and returns a PCollection<KV<K, T>> that contains one output element per key mapping each to the minimum of the values associated with that key in the input PCollection.
perKey(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<KV<K, V>> and returns a PCollection<KV<K, List<V>>> that contains an output element mapping each distinct key in the input PCollection to the largest count values associated with that key in the input PCollection<KV<K, V>>, in decreasing order, sorted using the given Comparator<V>.
PInput - Interface in com.google.cloud.dataflow.sdk.values
The interface for things that might be input to a PTransform.
Pipeline - Class in com.google.cloud.dataflow.sdk
A Pipeline manages a directed acyclic graph of PTransforms, and the PCollections that the PTransforms consume and produce.
Pipeline(PipelineRunner<?>) - Constructor for class com.google.cloud.dataflow.sdk.Pipeline
Pipeline(PipelineRunner<?>, PipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.Pipeline
 
Pipeline.PipelineExecutionException - Exception in com.google.cloud.dataflow.sdk
Thrown during execution of a Pipeline, whenever user code within that Pipeline throws an exception.
Pipeline.PipelineVisitor - Interface in com.google.cloud.dataflow.sdk
A Pipeline.PipelineVisitor can be passed into Pipeline.traverseTopologically(com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor) to be called for each of the transforms and values in the Pipeline.
PipelineExecutionException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.Pipeline.PipelineExecutionException
PipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
PipelineOptions are used to configure Pipelines.
PipelineOptions.CheckEnabled - Enum in com.google.cloud.dataflow.sdk.options
Enumeration of the possible states for a given check.
PipelineOptionsFactory - Class in com.google.cloud.dataflow.sdk.options
Constructs a PipelineOptions or any derived interface that is composable to any other derived interface of PipelineOptions via the PipelineOptions.as(java.lang.Class<T>) method.
PipelineOptionsFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
 
PipelineOptionsFactory.Builder - Class in com.google.cloud.dataflow.sdk.options
A fluent PipelineOptions builder.
PipelineOptionsRegistrar - Interface in com.google.cloud.dataflow.sdk.options
PipelineOptions creators have the ability to automatically have their PipelineOptions registered with this SDK by creating a ServiceLoader entry and a concrete implementation of this interface.
PipelineOptionsValidator - Class in com.google.cloud.dataflow.sdk.options
Validates that the PipelineOptions conforms to all the Validation criteria.
PipelineOptionsValidator() - Constructor for class com.google.cloud.dataflow.sdk.options.PipelineOptionsValidator
 
PipelineResult - Interface in com.google.cloud.dataflow.sdk
Result of Pipeline.run().
PipelineResult.State - Enum in com.google.cloud.dataflow.sdk
Named constants for common values for the job state.
PipelineRunner<ResultT extends PipelineResult> - Class in com.google.cloud.dataflow.sdk.runners
A PipelineRunner can execute, translate, or otherwise process a Pipeline.
PipelineRunner() - Constructor for class com.google.cloud.dataflow.sdk.runners.PipelineRunner
 
PipelineRunnerRegistrar - Interface in com.google.cloud.dataflow.sdk.runners
PipelineRunner creators have the ability to automatically have their PipelineRunner registered with this SDK by creating a ServiceLoader entry and a concrete implementation of this interface.
plusDelayOf(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
Adds some delay to the original target time.
popNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Removes the last TransformTreeNode from the stack.
populateDisplayData(DisplayData.Builder) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.HasDisplayData
Register display metadata for the given transform or component.
populateDisplayData(DisplayData.Builder) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Register display metadata for the given transform or component.
populateDisplayData(DisplayData.Builder) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
Register display metadata for the given transform or component.
populateDisplayData(DisplayData.Builder) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Register display metadata for the given transform or component.
POutput - Interface in com.google.cloud.dataflow.sdk.values
The interface for things that might be output from a PTransform.
POutputValueBase - Class in com.google.cloud.dataflow.sdk.values
A POutputValueBase is the abstract base class of PTransform outputs.
POutputValueBase(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.POutputValueBase
 
POutputValueBase() - Constructor for class com.google.cloud.dataflow.sdk.values.POutputValueBase
No-arg constructor for Java serialization only.
prefetchOnElement(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
prefetchOnElement(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Called to allow the trigger to prefetch any state it will likely need to read from during an Trigger.onElement(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnElementContext) call.
prefetchOnFire(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Called to allow the trigger to prefetch any state it will likely need to read from during an Trigger.onFire(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.TriggerContext) call.
prefetchOnMerge(MergingStateAccessor<?, W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
prefetchOnMerge(MergingStateAccessor<?, W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
prefetchOnMerge(MergingStateAccessor<?, W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Called to allow the trigger to prefetch any state it will likely need to read from during an Trigger.onMerge(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.OnMergeContext) call.
prefetchShouldFire(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
prefetchShouldFire(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
prefetchShouldFire(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Called to allow the trigger to prefetch any state it will likely need to read from during an Trigger.shouldFire(com.google.cloud.dataflow.sdk.transforms.windowing.Trigger<W>.TriggerContext) call.
prepareWrite(WritableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Called with the channel that a subclass will write its header, footer, and values to.
prepareWrite(WritableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Creates the output stream that elements will be written to.
printHelp(PrintStream) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Outputs the set of registered options with the PipelineOptionsFactory with a description for each one if available to the output stream.
printHelp(PrintStream, Class<? extends PipelineOptions>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Outputs the set of options available to be set for the passed in PipelineOptions interface.
processBatch(Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
A convenience operation that first calls DoFnTester.startBundle(), then calls DoFnTester.processElement(InputT) on each of the input elements, then calls DoFnTester.finishBundle(), then returns the result of DoFnTester.takeOutputElements().
processBatch(InputT...) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
A convenience method for testing DoFns with bundles of elements.
ProcessContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
 
ProcessContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
 
processElement(DoFn<KV<K, Iterable<InputT>>, KV<K, OutputT>>.ProcessContext) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
 
processElement(DoFn<InputT, OutputT>.ProcessContext) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Processes one input element.
processElement(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
processElement(DoFn<InputT, OutputT>.ProcessContext) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
Whether this source is known to produce key/value pairs sorted by lexicographic order on the bytes of the encoded key.
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Returns whether the delegate source produces sorted keys.
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
PROJECT_ID_REGEXP - Static variable in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Project IDs must contain lowercase letters, digits, or dashes.
Proto2Coder<T extends com.google.protobuf.Message> - Class in com.google.cloud.dataflow.sdk.coders
Deprecated.
ProtoCoder<T extends com.google.protobuf.Message> - Class in com.google.cloud.dataflow.sdk.coders.protobuf
A Coder using Google Protocol Buffers binary format.
PROVIDER - Static variable in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
PROVIDER - Static variable in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
A CoderProvider that constructs a SerializableCoder for any class that implements serializable.
PTransform<InputT extends PInput,OutputT extends POutput> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform<InputT, OutputT> is an operation that takes an InputT (some subtype of PInput) and produces an OutputT (some subtype of POutput).
PTransform() - Constructor for class com.google.cloud.dataflow.sdk.transforms.PTransform
 
PTransform(String) - Constructor for class com.google.cloud.dataflow.sdk.transforms.PTransform
 
PubsubIO - Class in com.google.cloud.dataflow.sdk.io
Read and Write PTransforms for Cloud Pub/Sub streams.
PubsubIO.PubsubSubscription - Class in com.google.cloud.dataflow.sdk.io
Class representing a Cloud Pub/Sub Subscription.
PubsubIO.PubsubTopic - Class in com.google.cloud.dataflow.sdk.io
Class representing a Cloud Pub/Sub Topic.
PubsubIO.Read - Class in com.google.cloud.dataflow.sdk.io
A PTransform that continuously reads from a Cloud Pub/Sub stream and returns a PCollection of Strings containing the items from the stream.
PubsubIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from a Cloud Pub/Sub source and returns a unbounded PCollection containing the items from the stream.
PubsubIO.Write - Class in com.google.cloud.dataflow.sdk.io
A PTransform that continuously writes a PCollection of Strings to a Cloud Pub/Sub stream.
PubsubIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes an unbounded PCollection of Strings to a Cloud Pub/Sub stream.
pushNode(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Add a TransformTreeNode to the stack.
PValue - Interface in com.google.cloud.dataflow.sdk.values
The interface for values that can be input to and output from PTransforms.
PValueBase - Class in com.google.cloud.dataflow.sdk.values
A PValueBase is an abstract base class that provides sensible default implementations for methods of PValue.
PValueBase(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.PValueBase
 
PValueBase() - Constructor for class com.google.cloud.dataflow.sdk.values.PValueBase
No-arg constructor for Java serialization only.

R

randomizeIfUnordered(List<T>, boolean) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
If the evaluation context is testing unorderedness, randomly permutes the order of the elements, in a copy if !inPlaceAllowed, and returns the permuted list, otherwise returns the argument unchanged.
RangeTracker<PositionT> - Interface in com.google.cloud.dataflow.sdk.io.range
A RangeTracker is a thread-safe helper object for implementing dynamic work rebalancing in position-based BoundedSource.BoundedReader subclasses.
RawUnionValue - Class in com.google.cloud.dataflow.sdk.transforms.join
This corresponds to an integer union tag and value.
RawUnionValue(int, Object) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
Constructs a partial union from the given union tag and value.
read() - Static method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO
Creates an uninitialized BigtableIO.Read.
read() - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Deprecated.
the name and return type do not match. Use DatastoreIO.source().
Read - Class in com.google.cloud.dataflow.sdk.io
A PTransform for reading from a Source.
Read() - Constructor for class com.google.cloud.dataflow.sdk.io.Read
 
Read.Bounded<T> - Class in com.google.cloud.dataflow.sdk.io
PTransform that reads from a BoundedSource.
Read.Builder - Class in com.google.cloud.dataflow.sdk.io
Helper class for building Read transforms.
Read.Unbounded<T> - Class in com.google.cloud.dataflow.sdk.io
PTransform that reads from a UnboundedSource.
Reader() - Constructor for class com.google.cloud.dataflow.sdk.io.Source.Reader
 
readFrom(String, DatastoreV1.Query) - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Returns a PTransform that reads Datastore entities from the query against the given dataset.
readFrom(String, String, DatastoreV1.Query) - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
readFromFileWithClass(String, Class<T>) - Static method in class com.google.cloud.dataflow.sdk.io.AvroSource
Creates a Read transform that will read from an AvroSource that is configured to read records of the given type from a file pattern.
readFromSource(FileBasedSource<T>, CompressedSource.DecompressingChannelFactory) - Static method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Creates a Read transform that reads from that reads from the underlying FileBasedSource sourceDelegate after decompressing it with a CompressedSource.DecompressingChannelFactory.
readFromSource(BoundedSource<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Reads all elements from the given BoundedSource.
readFromStartedReader(Source.Reader<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Reads all elements from the given started Source.Reader.
readFromUnstartedReader(Source.Reader<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Reads all elements from the given unstarted Source.Reader.
readNextBlock() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
readNextBlock() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Read the next block from the input.
readNextRecord() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
Reads the next record from the block and returns true iff one exists.
readNextRecord() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Reads the next record from the current block if possible.
readNextRecord() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Reads the next record via the delegate reader.
readNextRecord() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
readNItemsFromStartedReader(Source.Reader<T>, int) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Read elements from a Source.Reader that has already had Source.Reader#start called on it, until n elements are read.
readNItemsFromUnstartedReader(Source.Reader<T>, int) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Read elements from a Source.Reader until n elements are read.
readRemainingFromReader(Source.Reader<T>, boolean) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
Read all remaining elements from a Source.Reader.
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in interface com.google.cloud.dataflow.sdk.values.POutput
Records that this POutput is an output of the given PTransform.
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
Records that this POutputValueBase is an output with the given name of the given AppliedPTransform.
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
 
recordAsOutput(AppliedPTransform<?, ?, ?>, String) - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Records that this POutputValueBase is an output with the given name of the given AppliedPTransform in the given Pipeline.
RecordingPipelineVisitor - Class in com.google.cloud.dataflow.sdk.runners
Provides a simple Pipeline.PipelineVisitor that records the transformation tree.
RecordingPipelineVisitor() - Constructor for class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
register(Class<? extends PipelineOptions>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
This registers the interface with this factory.
registerByteSizeObserver(T, ElementByteSizeObserver, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.
registerByteSizeObserver(ReadableDuration, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder
 
registerByteSizeObserver(Instant, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
 
registerByteSizeObserver(IterableT, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
registerByteSizeObserver(KV<K, V>, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
Notifies ElementByteSizeObserver about the byte size of the encoded value using this coder.
registerByteSizeObserver(Map<K, V>, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
registerByteSizeObserver(T, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
Overridden to short-circuit the default StandardCoder behavior of encoding and counting the bytes.
registerByteSizeObserver(T, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.
registerCoder(Class<?>, Class<?>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Registers coderClazz as the default Coder class to handle encoding and decoding instances of clazz, overriding prior registrations if any exist.
registerCoder(Class<?>, CoderFactory) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Registers coderFactory as the default CoderFactory to produce Coder instances to decode and encode instances of clazz.
registerCoder(Class<T>, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Register the provided Coder for encoding all values of the specified Class.
registerDefaultTransformEvaluator(Class<TransformT>, DirectPipelineRunner.TransformEvaluator<? super TransformT>) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Records that instances of the specified PTransform class should be evaluated by default by the corresponding TransformEvaluator.
registerStandardCoders() - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Registers standard Coders with this CoderRegistry.
registerTransformEvaluator(Class<TransformT>, DirectPipelineRunner.TransformEvaluator<TransformT>) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Records that instances of the specified PTransform class should be evaluated by the corresponding TransformEvaluator.
registerTransformTranslator(Class<TransformT>, DataflowPipelineTranslator.TransformTranslator<? extends TransformT>) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
Records that instances of the specified PTransform class should be translated by default by the corresponding DataflowPipelineTranslator.TransformTranslator.
ReifyTimestampsAndWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.ReifyTimestampsAndWindows
 
remerge() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Creates a Window PTransform that does not change assigned windows, but will cause windows to be merged again as part of the next GroupByKey.
Remerge() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Remerge
 
RemoveDuplicates<T> - Class in com.google.cloud.dataflow.sdk.transforms
RemoveDuplicates<T> takes a PCollection<T> and returns a PCollection<T> that has all the elements of the input but with duplicate elements removed such that each element is unique within each window.
RemoveDuplicates() - Constructor for class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates
 
RemoveDuplicates.WithRepresentativeValues<T,IdT> - Class in com.google.cloud.dataflow.sdk.transforms
A RemoveDuplicates PTransform that uses a SerializableFunction to obtain a representative value for each input element.
removeTemporaryFiles(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Removes temporary output files.
Repeatedly<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Repeat a trigger, either until some condition is met or forever.
requiresDeduping() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
Returns whether this source requires explicit deduping.
resetTree() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Clears all keyed state for triggers in the current sub-tree and unsets all the associated finished bits.
resolveType(Type) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a TypeDescriptor representing the given type, with type variables resolved according to the specialization in this type.
run() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Runs the Pipeline using its PipelineRunner.
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
Processes the given Pipeline, returning the results.
run() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
 
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
 
run() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
 
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
 
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner
Processes the given Pipeline, returning the results.
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
 
run() - Method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
Runs this TestPipeline, unwrapping any AssertionError that is raised during testing.
RunnableOnService - Interface in com.google.cloud.dataflow.sdk.testing
Category tag for tests that can be run on the DataflowPipelineRunner if the runIntegrationTestOnService System property is set to true.
Runner() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Runner
 
Runner() - Constructor for class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Runner
 
runWindowFn(WindowFn<T, W>, List<Long>) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
Runs the WindowFn over the provided input, returning a map of windows to the timestamps in those windows.

S

Sample - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for taking samples of the elements in a PCollection, or samples of the values associated with each key in a PCollection of KVs.
Sample() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sample
 
Sample.FixedSizedSampleFn<T> - Class in com.google.cloud.dataflow.sdk.transforms
CombineFn that computes a fixed-size sample of a collection of values.
Sample.SampleAny<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that takes a PCollection<T> and a limit, and produces a new PCollection<T> containing up to limit elements of the input PCollection.
satisfies(SerializableFunction<Iterable<T>, Void>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Applies a SerializableFunction to check the elements of the Iterable.
satisfies(DataflowAssert.AssertRelation<Iterable<T>, Iterable<T>>, Iterable<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Applies a SerializableFunction to check the elements of the Iterable.
satisfies(SerializableFunction<T, Void>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Applies a SerializableFunction to check the value of this SingletonAssert's view.
satisfies(DataflowAssert.AssertRelation<T, T>, T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Applies an AssertRelation to check the provided relation against the value of this assert and the provided expected value.
SerializableCoder<T extends Serializable> - Class in com.google.cloud.dataflow.sdk.coders
A Coder for Java classes that implement Serializable.
SerializableCoder(Class<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
SerializableComparator<T> - Interface in com.google.cloud.dataflow.sdk.transforms
A Comparator that is also Serializable.
SerializableFunction<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
A function that computes an output value of type OutputT from an input value of type InputT and is Serializable.
Sessions - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn windowing values into sessions separated by Sessions.gapDuration-long periods with no elements.
set(long...) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
SET_PROJECT_FROM_OPTIONS_WARNING - Static variable in class com.google.cloud.dataflow.sdk.io.BigQueryIO
 
setApiRootUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setAppName(String) - Method in interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions
 
setAuthorizationServerEncodedUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setAutoscalingAlgorithm(DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
SetCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A SetCoder encodes any Set using the format of IterableLikeCoder.
SetCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.SetCoder
 
setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Sets the coder to use for elements of type T, as needed for internal purposes.
setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Sets the coder to use for elements of type T, as needed for internal purposes.
setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Sets the Coder used by this PCollection to encode and decode the values stored in it.
setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
Sets the Coder used by this TypedPValue to encode and decode the values stored in it.
setCoderRegistry(CoderRegistry) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Sets the CoderRegistry that this Pipeline uses.
setCredentialDir(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setCredentialFactoryClass(Class<? extends CredentialFactory>) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setCredentialId(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setDataflowClient(Dataflow) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setDataflowEndpoint(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setDataflowJobFile(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setDebuggee(Debuggee) - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
 
setDefaultWorkerLogLevel(DataflowWorkerLoggingOptions.Level) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
 
setDirectPipelineRunnerRandomSeed(Long) - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
 
setDiskSizeGb(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setDumpHeapOnOOM(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setEnableCloudDebugger(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
 
setEnableProfilingAgent(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions
 
setExecutorService(ExecutorService) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
 
setExperiments(List<String>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setFallbackCoderProvider(CoderProvider) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Sets the fallback CoderProvider for this registry.
setFilesToStage(List<String>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setFinished(boolean) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Sets the finished bit for the current trigger.
setFinished(boolean, int) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Sets the finished bit for the given sub-trigger.
setGcpCredential(Credential) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setGcsEndpoint(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
 
setGcsUploadBufferSizeBytes(Integer) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
 
setGcsUtil(GcsUtil) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
 
setGoogleApiTrace(GoogleApiDebugOptions.GoogleApiTracer) - Method in interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions
 
setHooks(DataflowPipelineRunnerHooks) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
Sets callbacks to invoke during execution.
setHooks(DataflowPipelineRunnerHooks) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Sets callbacks to invoke during execution see DataflowPipelineRunnerHooks.
setIsBoundedInternal(PCollection.IsBounded) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
setJobId(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
 
setJobMessageOutput(PrintStream) - Method in interface com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions
 
setJobName(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
setMaxConditionCost(double) - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
 
setMaxNumWorkers(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setName(String) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Sets the name of this PCollection.
setName(String) - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Sets the name of this PValueBase.
setNetwork(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setNumberOfWorkerHarnessThreads(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setNumWorkers(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setOutput(TransformTreeNode, POutput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Sets the output of a transform node.
setOutput(POutput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Adds an output to the transform node.
setOverrideWindmillBinary(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setPathValidator(PathValidator) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setPathValidatorClass(Class<? extends PathValidator>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setPCollection(PCollection<T>, List<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Shorthand for setting the value of a PCollection where the elements do not have timestamps or windows.
setPCollectionValuesWithMetadata(PCollection<T>, List<DirectPipelineRunner.ValueWithMetadata<T>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Sets the value of the given PCollection, where each element also has a timestamp and collection of windows.
setPCollectionView(PCollectionView<T>, Iterable<WindowedValue<ElemT>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Sets the value associated with the given PCollectionView.
setPCollectionWindowedValue(PCollection<T>, List<WindowedValue<T>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Sets the value of the given PCollection, where each element also has a timestamp and collection of windows.
setProfilingAgentConfiguration(DataflowProfilingOptions.DataflowProfilingAgentConfiguration) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions
 
setProject(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
setProject(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setPubsubRootUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setRunner(Class<? extends PipelineRunner<?>>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
 
setSecretsFile(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setServiceAccountKeyfile(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setServiceAccountName(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setSideInput(PCollectionView<?>, Iterable<WindowedValue<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Registers the values of a side input PCollectionView to pass to the DoFn under test.
setSideInputInGlobalWindow(PCollectionView<?>, Iterable<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Registers the values for a side input PCollectionView to pass to the DoFn under test.
setSideInputs(Map<PCollectionView<?>, Iterable<WindowedValue<?>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Registers the tuple of values of the side input PCollectionViews to pass to the DoFn under test.
setSideOutputTags(TupleTagList) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Registers the list of TupleTags that can be used by the DoFn under test to output to side output PCollections.
setStableUniqueNames(PipelineOptions.CheckEnabled) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
 
setStager(Stager) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setStagerClass(Class<? extends Stager>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setStagingLocation(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
setStreaming(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.StreamingOptions
 
setSubnetwork(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setTeardownPolicy(DataflowPipelineWorkerPoolOptions.TeardownPolicy) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setTempDatasetId(String) - Method in interface com.google.cloud.dataflow.sdk.options.BigQueryOptions
 
setTempLocation(String) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
 
setTestEncodability(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
 
setTestSerializability(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
 
setTestUnorderedness(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
 
setTimer(Instant, TimeDomain) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
Sets a timer to fire when the watermark or processing time is beyond the given timestamp.
setTimer(Instant, TimeDomain) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
Sets a timer to fire when the watermark or processing time is beyond the given timestamp.
setTokenServerUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setTransformNameMapping(Map<String, String>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setTypeDescriptorInternal(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
setTypeDescriptorInternal(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
Sets the TypeDescriptor<T> associated with this class.
setUpdate(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Deprecated.
setUpdate(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
setupDelegateAggregators() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Sets up Aggregators created by the DoFn so they are usable within this context.
setWindowingStrategyInternal(WindowingStrategy<?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Sets the WindowingStrategy of this PCollection.
setWorkerCacheMb(Integer) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
 
setWorkerDiskType(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setWorkerHarnessContainerImage(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setWorkerId(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
 
setWorkerLogLevelOverrides(DataflowWorkerLoggingOptions.WorkerLogLevelOverrides) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
 
setWorkerMachineType(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setZone(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
ShardNameTemplate - Class in com.google.cloud.dataflow.sdk.io
Standard shard naming templates.
ShardNameTemplate() - Constructor for class com.google.cloud.dataflow.sdk.io.ShardNameTemplate
 
shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
Returns true if the current state of the trigger indicates that its condition is satisfied and it is ready to fire.
shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
 
shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
 
shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Returns true if the current state of the trigger indicates that its condition is satisfied and it is ready to fire.
sideInput(PCollectionView<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.Context
Returns the value of the side input for the window corresponding to the window of the main input element.
sideInput(PCollectionView<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the value of the side input for the window corresponding to the window of the main input element.
sideInput(PCollectionView<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
Returns the value of the side input.
sideOutput(TupleTag<T>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Adds the given element to the side output PCollection with the given tag.
sideOutput(TupleTag<T>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Adds the given element to the side output PCollection with the given tag.
sideOutputWithTimestamp(TupleTag<T>, T, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Adds the given element to the specified side output PCollection, with the given timestamp.
sideOutputWithTimestamp(TupleTag<T>, T, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Adds the given element to the specified side output PCollection, with the given timestamp.
SimpleCombineFn(SerializableFunction<Iterable<V>, V>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.SimpleCombineFn
Deprecated.
 
SimpleFunction<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A SerializableFunction which is not a functional interface.
SimpleFunction() - Constructor for class com.google.cloud.dataflow.sdk.transforms.SimpleFunction
 
SingletonAssert(DataflowAssert.CreateActual<?, T>, Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
 
sink() - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Returns a new DatastoreIO.Sink builder using the default host.
Sink(String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
Constructs a Sink with given host and dataset.
sink - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
The Sink that this WriteOperation will write to.
Sink<T> - Class in com.google.cloud.dataflow.sdk.io
A Sink represents a resource that can be written to using the Write transform.
Sink() - Constructor for class com.google.cloud.dataflow.sdk.io.Sink
 
Sink.WriteOperation<T,WriteT> - Class in com.google.cloud.dataflow.sdk.io
A Sink.WriteOperation defines the process of a parallel write of objects to a Sink.
Sink.Writer<T,WriteT> - Class in com.google.cloud.dataflow.sdk.io
A Writer writes a bundle of elements from a PCollection to a sink.
size() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Returns the number of columns for this schema.
size() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns the number of PCollections in this PCollectionList.
size() - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns the number of TupleTags in this TupleTagList.
SlidingWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows values into possibly overlapping fixed-size timestamp-based windows.
smallest(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<List<T>> with a single element containing the smallest count elements of the input PCollection<T>, in increasing order, sorted according to their natural order.
Smallest() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Top.Smallest
 
smallestPerKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<KV<K, V>> and returns a PCollection<KV<K, List<V>>> that contains an output element mapping each distinct key in the input PCollection to the smallest count values associated with that key in the input PCollection<KV<K, V>>, in increasing order, sorted according to their natural order.
SortValuesByTimestamp() - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.SortValuesByTimestamp
 
source() - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Returns an empty DatastoreIO.Source builder with the default host.
Source<T> - Class in com.google.cloud.dataflow.sdk.io
Base class for defining input formats and creating a Source for reading the input.
Source() - Constructor for class com.google.cloud.dataflow.sdk.io.Source
 
Source.Reader<T> - Class in com.google.cloud.dataflow.sdk.io
The interface that readers of custom input sources must implement.
SourceTestUtils - Class in com.google.cloud.dataflow.sdk.testing
Helper functions and test harnesses for checking correctness of Source implementations.
SourceTestUtils() - Constructor for class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
 
SourceTestUtils.ExpectedSplitOutcome - Enum in com.google.cloud.dataflow.sdk.testing
span(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns the minimal window that includes both this window and the given window.
split(int) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Returns a list of up to numSplits + 1 ByteKeys in ascending order, where the keys have been interpolated to form roughly equal sub-ranges of this ByteKeyRange, assuming a uniform distribution of keys within this range.
splitAtFraction(double) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
Tells the reader to narrow the range of the input it's going to read and give up the remainder, so that the new range would contain approximately the given fraction of the amount of data in the current range.
splitAtFraction(double) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
splitAtFraction(double) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
 
splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
Splits the source into bundles of approximately desiredBundleSizeBytes.
splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
 
StagerFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.StagerFactory
 
StandardCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
An abstract base class to implement a Coder that defines equality, hashing, and printing via the class name and recursively using StandardCoder.getComponents().
StandardCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
StandardOutputFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions.StandardOutputFactory
 
start() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
start() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
 
start() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
Initializes the reader and advances the reader to the first record.
start() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Initializes the reader and advances the reader to the first record.
start() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns the start of this window, inclusive.
startBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Prepares this DoFn instance for processing a batch of elements.
startBundle() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
startBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
startImpl() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
 
startImpl() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
Initializes the OffsetBasedSource.OffsetBasedReader and advances to the first record, returning true if there is a record available to be read.
startReading(ReadableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
Starts reading from the provided channel.
startReading(ReadableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Creates a decompressing channel from the input channel and passes it to its delegate reader's FileBasedReader#startReading(ReadableByteChannel).
startReading(ReadableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
Performs any initialization of the subclass of FileBasedReader that involves IO operations.
state() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
 
state() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Returns the interface for accessing persistent state.
STREAMING_WORKER_HARNESS_CONTAINER_IMAGE - Static variable in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
 
StreamingOptions - Interface in com.google.cloud.dataflow.sdk.options
Options used to configure streaming.
StreamingPubsubIOWrite(DataflowPipelineRunner, PubsubIO.Write.Bound<T>) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
Builds an instance of this class from the overridden transform.
StringDelegateCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A Coder that wraps a Coder<String> and encodes/decodes values via string representations.
StringDelegateCoder(Class<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
 
StringUtf8Coder - Class in com.google.cloud.dataflow.sdk.coders
A Coder that encodes Strings in UTF-8 encoding.
StructuralByteArray - Class in com.google.cloud.dataflow.sdk.coders
A wrapper around a byte[] that uses structural, value-based equality rather than byte[]'s normal object identity.
StructuralByteArray(byte[]) - Constructor for class com.google.cloud.dataflow.sdk.coders.StructuralByteArray
 
structuralValue(byte[]) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
Returns an object with an Object.equals() method that represents structural equality on the argument.
structuralValue(T) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Returns an object with an Object.equals() method that represents structural equality on the argument.
structuralValue(T) - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
Returns an object with an Object.equals() method that represents structural equality on the argument.
structuralValue(KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
structuralValue(T) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
 
structuralValue(T) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
structuralValueConsistentWithEquals(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
structuralValueConsistentWithEqualsInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
subscription(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a transform that's like this one but reading from the given subscription.
subscription(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a transform for reading from a specific Cloud Pub/Sub subscription.
subTrigger(int) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Access the executable version of the specified sub-trigger.
subTriggers - Variable in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
subTriggers() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
subTriggers() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Access the executable versions of the sub-triggers of the current trigger.
Sum - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for computing the sum of the elements in a PCollection, or the sum of the values associated with each key in a PCollection of KVs.
Sum.SumDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
Sum.SumIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
Sum.SumLongFn - Class in com.google.cloud.dataflow.sdk.transforms
SumDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
 
SumIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
 
SumLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
 
supportsDynamicWorkRebalancing() - Method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
Returns whether a TestPipeline supports dynamic work rebalancing, and thus tests of dynamic work rebalancing are expected to pass.

T

TableRowJsonCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder that encodes BigQuery TableRow objects in their native JSON format.
TaggedKeyedPCollection(TupleTag<V>, PCollection<KV<K, V>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple.TaggedKeyedPCollection
 
takeOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the main output.
takeOutputElementsWithTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the main output with associated timestamps.
takeSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the side output with the given tag.
TEMPORARY_FILENAME_SEPARATOR - Static variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Name separator for temporary files.
temporaryFileRetention - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Option to keep or remove temporary output files.
TestCombineDoFn(PerKeyCombineFnRunner<? super K, ? super InputT, AccumT, OutputT>, Coder<AccumT>, boolean, Random) - Constructor for class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
 
TestDataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.testing
A set of options used to configure the TestPipeline.
TestDataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.testing
TestDataflowPipelineRunner is a pipeline runner that wraps a DataflowPipelineRunner when running tests against the TestPipeline.
testingPipelineOptions() - Static method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
Creates PipelineOptions for testing.
TestPipeline - Class in com.google.cloud.dataflow.sdk.testing
A creator of test pipelines that can be used inside of tests that can be configured to run locally or against the live service.
TextIO - Class in com.google.cloud.dataflow.sdk.io
PTransforms for reading and writing text files.
TextIO.CompressionType - Enum in com.google.cloud.dataflow.sdk.io
Possible text file compression types.
TextIO.Read - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from a text file (or multiple text files matching a pattern) and returns a PCollection containing the decoding of each of the lines of the text file(s).
TextIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from one or more text files and returns a bounded PCollection containing one element for each line of the input files.
TextIO.Write - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes a PCollection to text file (or multiple text files matching a sharding pattern), with each element of the input collection encoded into its own line.
TextIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes a bounded PCollection to a text file (or multiple text files matching a sharding pattern), with each PCollection element being encoded into its own line.
TextualIntegerCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder that encodes Integer Integers as the ASCII bytes of their textual, decimal, representation.
TextualIntegerCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
that(PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs an DataflowAssert.IterableAssert for the elements of the provided PCollection.
thatIterable(PCollectionView<Iterable<T>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs an DataflowAssert.IterableAssert for the value of the provided PCollectionView PCollectionView<Iterable<T>>.
thatMap(PCollection<KV<K, V>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs a DataflowAssert.SingletonAssert for the value of the provided PCollection, which must have at most one value per key.
thatMultimap(PCollection<KV<K, V>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs a DataflowAssert.SingletonAssert for the value of the provided PCollection.
thatSingleton(PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs a DataflowAssert.SingletonAssert for the value of the provided PCollection PCollection<T>, which must be a singleton.
thatSingletonIterable(PCollection<? extends Iterable<T>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs an DataflowAssert.IterableAssert for the value of the provided PCollection which must contain a single Iterable<T> value.
timestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the timestamp of the input element.
timestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
Returns the timestamp of the input element.
timestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
Returns the timestamp of the current element.
TIMESTAMP_MAX_VALUE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
 
TIMESTAMP_MIN_VALUE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
 
timestamped(Iterable<TimestampedValue<T>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.TimestampedValues transform that produces a PCollection containing the elements of the provided Iterable with the specified timestamps.
timestamped(TimestampedValue<T>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.TimestampedValues transform that produces a PCollection containing the specified elements with the specified timestamps.
timestamped(Iterable<T>, Iterable<Long>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new root transform that produces a PCollection containing the specified elements with the specified timestamps.
TimestampedValue<V> - Class in com.google.cloud.dataflow.sdk.values
An immutable pair of a value and a timestamp.
TimestampedValue(V, Instant) - Constructor for class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
TimestampedValue.TimestampedValueCoder<T> - Class in com.google.cloud.dataflow.sdk.values
timestampLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a transform that's like this one but that reads message timestamps from the given message attribute.
timestampLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a transform reading from Cloud Pub/Sub where record timestamps are expected to be provided as Pub/Sub message attributes.
timestampLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new transform that's like this one but that publishes record timestamps to a message attribute with the specified name.
timestampLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
Creates a transform that writes to Pub/Sub, adds each record's timestamp to the published messages in an attribute with the specified name.
timestampMappers - Variable in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
A list of timestampMappers m1, m2, m3, ...
to(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but that writes to the file(s) with the given filename prefix.
to(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform that writes to the file(s) with the given prefix.
to(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns a copy of this write transformation, but writing to the specified table.
to(TableReference) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns a copy of this write transformation, but writing to the specified table.
to(SerializableFunction<BoundedWindow, String>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns a copy of this write transformation, but using the specified function to determine which table to write to for each window.
to(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation for the given table specification.
to(TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation for the given table.
to(SerializableFunction<BoundedWindow, String>) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation from a function that maps windows to table specifications.
to(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a transform for writing to text files that's like this one but that writes to the file(s) with the given filename prefix.
to(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a transform for writing to text files that writes to the file(s) with the given prefix.
to(Sink<T>) - Static method in class com.google.cloud.dataflow.sdk.io.Write
Creates a Write transform that writes to the given Sink.
toDoFn(DoFnWithContext<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
Create a DoFn that the DoFnWithContext.
toFilenamePrefix(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
Returns an XmlSink that writes to files with the given prefix.
Top - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for finding the largest (or smallest) set of elements in a PCollection, or the largest (or smallest) set of values associated with each key in a PCollection of KVs.
Top.Largest<T extends Comparable<? super T>> - Class in com.google.cloud.dataflow.sdk.transforms
A Serializable Comparator that that uses the compared elements' natural ordering.
Top.Smallest<T extends Comparable<? super T>> - Class in com.google.cloud.dataflow.sdk.transforms
Serializable Comparator that that uses the reverse of the compared elements' natural ordering.
Top.TopCombineFn<T,ComparatorT extends Comparator<T> & Serializable> - Class in com.google.cloud.dataflow.sdk.transforms
CombineFn for Top transforms that combines a bunch of Ts into a single count-long List<T>, using compareFn to choose the largest Ts.
TopCombineFn(int, ComparatorT) - Constructor for class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
 
topic(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a transform that's like this one but that reads from the specified topic.
topic(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a transform for reading from a Cloud Pub/Sub topic.
topic(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new transform that's like this one but that writes to the specified topic.
topic(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
Creates a transform that publishes to the specified topic.
toString() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.Context
 
toString() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
toString() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
toString() - Method in class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
 
toString() - Method in class com.google.cloud.dataflow.sdk.coders.StructuralByteArray
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
toString() - Method in class com.google.cloud.dataflow.sdk.Pipeline
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
 
toString() - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
 
toString() - Method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Identifier
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.KV
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
 
toTableReference(SerializableFunction<BoundedWindow, TableReference>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns a copy of this write transformation, but using the specified function to determine which table to write to for each window.
toTableReference(SerializableFunction<BoundedWindow, TableReference>) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation from a function that maps windows to TableReference objects.
toTableSpec(TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO
Returns a canonical string representation of the TableReference.
TransformHierarchy - Class in com.google.cloud.dataflow.sdk.runners
Captures information about a collection of transformations and their associated PValues.
TransformHierarchy() - Constructor for class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Create a TransformHierarchy containing a root node.
transforms - Variable in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
TransformTreeNode - Class in com.google.cloud.dataflow.sdk.runners
Provides internal tracking of transform relationships with helper methods for initialization and ordered visitation.
TransformTreeNode(TransformTreeNode, PTransform<?, ?>, String, PInput) - Constructor for class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Creates a new TransformTreeNode with the given parent and transform.
translate(TransformT, DataflowPipelineTranslator.TranslationContext) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TransformTranslator
 
translate(Pipeline, DataflowPipelineRunner, List<DataflowPackage>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
Translates a Pipeline into a JobSpecification.
traverseTopologically(Pipeline.PipelineVisitor) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Trigger<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Triggers control when the elements for a specific key and window are output.
Trigger(List<Trigger<W>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
trigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
 
trigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Returns the interface for accessing trigger info.
Trigger.MergingTriggerInfo<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
Interact with properties of the trigger being executed, with extensions to deal with the merging windows.
Trigger.OnceTrigger<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Triggers that are guaranteed to fire at most once should extend from this, rather than the general Trigger class to indicate that behavior.
Trigger.OnElementContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Trigger.OnMergeContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Trigger.TriggerContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Information accessible to all operational hooks in this Trigger.
Trigger.TriggerInfo<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
Interface for accessing information about the trigger being executed and other triggers in the same tree.
TriggerBuilder<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
Anything that can be used to create an instance of a Trigger implements this interface.
TriggerContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
 
triggering(TriggerBuilder<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Sets a non-default trigger for this Window PTransform.
triggering(TriggerBuilder<?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Sets a non-default trigger for this Window PTransform.
triggering(TriggerBuilder<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
Sets a non-default trigger for this Window PTransform.
tryReturnRecordAt(boolean, ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
 
tryReturnRecordAt(boolean, Long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
tryReturnRecordAt(boolean, long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
tryReturnRecordAt(boolean, PositionT) - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Atomically determines whether a record at the given position can be returned and updates internal state.
trySplitAtPosition(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
 
trySplitAtPosition(Long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
trySplitAtPosition(long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
trySplitAtPosition(PositionT) - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Atomically splits the current range [RangeTracker.getStartPosition(), RangeTracker.getStopPosition()) into a "primary" part [RangeTracker.getStartPosition(), splitPosition) and a "residual" part [splitPosition, RangeTracker.getStopPosition()), assuming the current last-consumed position is within [RangeTracker.getStartPosition(), splitPosition) (i.e., splitPosition has not been consumed yet).
TupleTag<V> - Class in com.google.cloud.dataflow.sdk.values
A TupleTag is a typed tag to use as the key of a heterogeneously typed tuple, like PCollectionTuple.
TupleTag() - Constructor for class com.google.cloud.dataflow.sdk.values.TupleTag
Constructs a new TupleTag, with a fresh unique id.
TupleTag(String) - Constructor for class com.google.cloud.dataflow.sdk.values.TupleTag
Constructs a new TupleTag with the given id.
TupleTagList - Class in com.google.cloud.dataflow.sdk.values
A TupleTagList is an immutable list of heterogeneously typed TupleTags.
TypeDescriptor<T> - Class in com.google.cloud.dataflow.sdk.values
A description of a Java type, including actual generic parameters where possible.
TypeDescriptor() - Constructor for class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Creates a TypeDescriptor representing the type parameter T.
TypeDescriptor(Object) - Constructor for class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Creates a TypeDescriptor representing the type parameter T, which should resolve to a concrete type in the context of the class clazz.
TypeDescriptor(Class<?>) - Constructor for class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Creates a TypeDescriptor representing the type parameter T, which should resolve to a concrete type in the context of the class clazz.
TypedPValue<T> - Class in com.google.cloud.dataflow.sdk.values
A TypedPValue<T> is the abstract base class of things that store some number of values of type T.
TypedPValue(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.TypedPValue
 

U

unbounded() - Static method in class com.google.cloud.dataflow.sdk.io.CountingInput
Creates an CountingInput.UnboundedCountingInput that will produce numbers starting from 0 up to Long.MAX_VALUE.
unbounded() - Static method in class com.google.cloud.dataflow.sdk.io.CountingSource
Deprecated.
UnboundedReader() - Constructor for class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
 
UnboundedSource<OutputT,CheckpointMarkT extends UnboundedSource.CheckpointMark> - Class in com.google.cloud.dataflow.sdk.io
A Source that reads an unbounded amount of input and, because of that, supports some additional operations such as checkpointing, watermarks, and record ids.
UnboundedSource() - Constructor for class com.google.cloud.dataflow.sdk.io.UnboundedSource
 
UnboundedSource.CheckpointMark - Interface in com.google.cloud.dataflow.sdk.io
A marker representing the progress and state of an UnboundedSource.UnboundedReader.
UnboundedSource.UnboundedReader<OutputT> - Class in com.google.cloud.dataflow.sdk.io
A Reader that reads an unbounded amount of input.
unboundedWithTimestampFn(SerializableFunction<Long, Instant>) - Static method in class com.google.cloud.dataflow.sdk.io.CountingSource
unfinishedSubTriggers() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
Returns an iterable over the unfinished sub-triggers of the current trigger.
updateWindowingStrategy(WindowingStrategy<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
 
upTo(long) - Static method in class com.google.cloud.dataflow.sdk.io.CountingInput
Creates a CountingInput.BoundedCountingInput that will produce the specified number of elements, from 0 to numElements - 1.
upTo(long) - Static method in class com.google.cloud.dataflow.sdk.io.CountingSource
Deprecated.
usesSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
 

V

validate() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
validate(PInput) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Validates the current PTransform.
validate(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
 
validate(PCollection<KV<ByteString, Iterable<Mutation>>>) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
 
validate() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Validates that the delegate source is a valid source and that the channel factory is not null.
validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
Ensures the host and dataset are set.
validate() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink
Perform pipeline-construction-time validation.
validate() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
validate() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
 
validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink
Ensures that the sink is valid and can be written to before the write operation begins.
validate() - Method in class com.google.cloud.dataflow.sdk.io.Source
Checks that this source is valid, before it can be used in a pipeline.
validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
Validates that the root element, class to bind to a JAXB context, and filenamePrefix have been set and that the class can be bound in a JAXB context.
validate() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
validate(Class<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsValidator
Validates that the passed PipelineOptions conforms to all the validation criteria from the passed in interface.
validate(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
 
validate(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Called before invoking apply (which may be intercepted by the runner) to verify this transform is fully specified and applicable to the specified input.
validate(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsIterable
 
validate(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsList
 
validate(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap
 
validate(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMultimap
 
validate(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
 
validate(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
 
validateGetOutputTimestamp(WindowFn<T, W>, long) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTime for later windows (as defined by maxTimestamp won't prevent the watermark from passing the end of earlier windows.
validateGetOutputTimestamps(WindowFn<T, W>, OutputTimeFn<? super W>, List<List<Long>>) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
Verifies that later-ending merged windows from any of the timestamps hold up output of earlier-ending windows, using the provided WindowFn and OutputTimeFn.
validateNonInterferingOutputTimes(WindowFn<T, W>, long) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
Assigns the given timestamp to windows using the specified windowFn, and verifies that result of windowFn.getOutputTimestamp for each window is within the proper bound.
Validation - Annotation Type in com.google.cloud.dataflow.sdk.options
Validation represents a set of annotations that can be used to annotate getter properties on PipelineOptions with information representing the validation criteria to be used when validating with the PipelineOptionsValidator.
Validation.Required - Annotation Type in com.google.cloud.dataflow.sdk.options
This criteria specifies that the value must be not null.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.annotations.Experimental.Kind
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException.ReasonCode
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.CreateDisposition
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.WriteDisposition
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSource.Mode
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.TextIO.CompressionType
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.TeardownPolicy
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.Level
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.PipelineOptions.CheckEnabled
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.testing.SourceTestUtils.ExpectedSplitOutcome
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.Timing
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Window.ClosingBehavior
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded
Returns the enum constant of this type with the specified name.
values() - Static method in enum com.google.cloud.dataflow.sdk.annotations.Experimental.Kind
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException.ReasonCode
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.CreateDisposition
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.WriteDisposition
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSource.Mode
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.TextIO.CompressionType
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.TeardownPolicy
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.Level
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.options.PipelineOptions.CheckEnabled
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
Returns an array containing the constants of this enum type, in the order they are declared.
values - Variable in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
values() - Static method in enum com.google.cloud.dataflow.sdk.testing.SourceTestUtils.ExpectedSplitOutcome
Returns an array containing the constants of this enum type, in the order they are declared.
Values<V> - Class in com.google.cloud.dataflow.sdk.transforms
Values<V> takes a PCollection of KV<K, V>s and returns a PCollection<V> of the values.
values() - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.Timing
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Window.ClosingBehavior
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded
Returns an array containing the constants of this enum type, in the order they are declared.
VarIntCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder that encodes Integers using between 1 and 5 bytes.
VarLongCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder that encodes Longs using between 1 and 10 bytes.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.DeterministicStandardCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.ListCoder
List sizes are always known, so ListIterable may be deterministic while the general IterableLikeCoder is not.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
NullableCoder is deterministic if the nested Coder is.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.SetCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic(String, Iterable<Coder<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
verifyDeterministic(String, Coder<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
via(SerializableFunction<InputT, ? extends Iterable<OutputT>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements
For a SerializableFunction<InputT, ? extends Iterable<OutputT>> fn, returns a PTransform that applies fn to every element of the input PCollection<InputT> and outputs all of the elements to the output PCollection<OutputT>.
via(SimpleFunction<InputT, ? extends Iterable<OutputT>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements
For a SimpleFunction<InputT, ? extends Iterable<OutputT>> fn, return a PTransform that applies fn to every element of the input PCollection<InputT> and outputs all of the elements to the output PCollection<OutputT>.
via(SerializableFunction<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.MapElements
For a SerializableFunction<InputT, OutputT> fn and output type descriptor, returns a PTransform that takes an input PCollection<InputT> and returns a PCollection<OutputT> containing fn.apply(v) for every element v in the input.
via(SimpleFunction<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.MapElements
For a SimpleFunction<InputT, OutputT> fn, returns a PTransform that takes an input PCollection<InputT> and returns a PCollection<OutputT> containing fn.apply(v) for every element v in the input.
View - Class in com.google.cloud.dataflow.sdk.transforms
Transforms for creating PCollectionViews from PCollections (to read them as side inputs).
View.AsIterable<T> - Class in com.google.cloud.dataflow.sdk.transforms
Not intended for direct use by pipeline authors; public only so a PipelineRunner may override its behavior.
View.AsList<T> - Class in com.google.cloud.dataflow.sdk.transforms
Not intended for direct use by pipeline authors; public only so a PipelineRunner may override its behavior.
View.AsMap<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Not intended for direct use by pipeline authors; public only so a PipelineRunner may override its behavior.
View.AsMultimap<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Not intended for direct use by pipeline authors; public only so a PipelineRunner may override its behavior.
View.AsSingleton<T> - Class in com.google.cloud.dataflow.sdk.transforms
Not intended for direct use by pipeline authors; public only so a PipelineRunner may override its behavior.
View.CreatePCollectionView<ElemT,ViewT> - Class in com.google.cloud.dataflow.sdk.transforms
Creates a primitive PCollectionView.
visit(Pipeline.PipelineVisitor, Set<PValue>) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Visits all nodes in the transform hierarchy, in transitive order.
visit(Pipeline.PipelineVisitor, Set<PValue>) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Visit the transform node.
visitTransform(TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
Called for each primitive transform after all of its topological predecessors and inputs have been visited.
visitTransform(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
visitValue(PValue, TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
Called for each value after the transform that produced the value has been visited.
visitValue(PValue, TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
VoidCoder - Class in com.google.cloud.dataflow.sdk.coders
A Coder for Void.

W

waitToFinish(long, TimeUnit, MonitoringUtil.JobMessagesHandler) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Waits for the job to finish and return the final status.
weeks(int, int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
Returns a WindowFn that windows elements into periods measured by weeks.
window() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the window into which the input element has been assigned.
window() - Method in interface com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ExtraContextFactory
Construct the BoundedWindow to use within a DoFnWithContext that needs it.
window() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
The window that the current context is executing in.
Window - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Window logically divides up or groups the elements of a PCollection into finite windows according to a WindowFn.
Window() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Window
 
Window.Bound<T> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A PTransform that windows the elements of a PCollection<T>, into finite windows according to a user-specified WindowFn.
Window.ClosingBehavior - Enum in com.google.cloud.dataflow.sdk.transforms.windowing
Specifies the conditions under which a final pane will be created when a window is permanently closed.
Window.Remerge<T> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
PTransform that does not change assigned windows, but will cause windows to be merged again as part of the next GroupByKey.
Window.Unbound - Class in com.google.cloud.dataflow.sdk.transforms.windowing
An incomplete Window transform, with unbound input/output type.
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns the Coder used for serializing the windows used by this windowFn.
WindowFn<T,W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
The argument to the Window transform used to assign elements into windows and to determine how windows are merged.
WindowFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
 
WindowFn.AssignContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
WindowFn.MergeContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
WindowFnTestUtils - Class in com.google.cloud.dataflow.sdk.testing
A utility class for testing WindowFns.
WindowFnTestUtils() - Constructor for class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
 
windowingInternals() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the process context to use for implementing windowing.
windowingInternals() - Method in interface com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ExtraContextFactory
Construct the WindowingInternals to use within a DoFnWithContext that needs it.
windows() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
Returns the windows the current element was in, prior to this WindowFn being called.
windows() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.MergeContext
Returns the current set of windows.
with(SimpleFunction<DataT, InputT>, Combine.CombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeCombineFnBuilder
Returns a CombineFns.ComposedCombineFn that can take additional GlobalCombineFns and apply them as a single combine function.
with(SimpleFunction<DataT, InputT>, CombineWithContext.CombineFnWithContext<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeCombineFnBuilder
Returns a CombineFns.ComposedCombineFnWithContext that can take additional GlobalCombineFns and apply them as a single combine function.
with(SimpleFunction<DataT, InputT>, Combine.CombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
Returns a CombineFns.ComposedCombineFn with an additional Combine.CombineFn.
with(SimpleFunction<DataT, InputT>, CombineWithContext.CombineFnWithContext<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
with(SimpleFunction<DataT, InputT>, CombineFnBase.GlobalCombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
with(SimpleFunction<DataT, InputT>, Combine.KeyedCombineFn<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
with(SimpleFunction<DataT, InputT>, CombineWithContext.KeyedCombineFnWithContext<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
with(SimpleFunction<DataT, InputT>, Combine.CombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
with(SimpleFunction<DataT, InputT>, CombineWithContext.CombineFnWithContext<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
with(SimpleFunction<DataT, InputT>, CombineFnBase.PerKeyCombineFn<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
with(SimpleFunction<DataT, InputT>, CombineFnBase.GlobalCombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
with(SimpleFunction<DataT, InputT>, Combine.KeyedCombineFn<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
Returns a CombineFns.ComposedKeyedCombineFn that can take additional PerKeyCombineFns and apply them as a single combine function.
with(SimpleFunction<DataT, InputT>, CombineWithContext.KeyedCombineFnWithContext<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
Returns a CombineFns.ComposedKeyedCombineFnWithContext that can take additional PerKeyCombineFns and apply them as a single combine function.
with(SimpleFunction<DataT, InputT>, Combine.CombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
Returns a CombineFns.ComposedKeyedCombineFn that can take additional PerKeyCombineFns and apply them as a single combine function.
with(SimpleFunction<DataT, InputT>, CombineWithContext.CombineFnWithContext<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
Returns a CombineFns.ComposedKeyedCombineFnWithContext that can take additional PerKeyCombineFns and apply them as a single combine function.
withAllowedLateness(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Override the amount of lateness allowed for data elements in the pipeline.
withAllowedLateness(Duration, Window.ClosingBehavior) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Override the amount of lateness allowed for data elements in the pipeline.
withAllowedLateness(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
Override the amount of lateness allowed for data elements in the pipeline.
withAllowedLateness(Duration, Window.ClosingBehavior) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
Override the amount of lateness allowed for data elements in the pipeline.
withAllowedLateness(Duration) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Override the amount of lateness allowed for data elements in the pipeline.
withAllowedTimestampSkew(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.WithTimestamps
Return a new WithTimestamps like this one with updated allowed timestamp skew, which is the maximum duration that timestamps can be shifted backward.
withBigtableOptions(BigtableOptions) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
Returns a new BigtableIO.Read that will read from the Cloud Bigtable cluster indicated by the given options, and using any other specified customizations.
withBigtableOptions(BigtableOptions.Builder) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
Returns a new BigtableIO.Read that will read from the Cloud Bigtable cluster indicated by the given options, and using any other specified customizations.
withBigtableOptions(BigtableOptions) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
Returns a new BigtableIO.Write that will write to the Cloud Bigtable cluster indicated by the given options, and using any other specified customizations.
withBigtableOptions(BigtableOptions.Builder) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
Returns a new BigtableIO.Write that will write to the Cloud Bigtable cluster indicated by the given options, and using any other specified customizations.
withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a transform that's like this one but that uses the given Coder to decode each record into a value of type X.
withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a transform for reading from Cloud Pub/Sub that uses the given Coder to decode Pub/Sub messages into a value of type T.
withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new transform that's like this one but that uses the given Coder to encode each of the elements of the input PCollection into an output record.
withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
Creates a transform that uses the given Coder to encode each of the elements of the input collection into an output message.
withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new transform for reading from text files that's like this one but that uses the given Coder to decode each of the lines of the file into a value of type X.
withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a transform for reading text files that uses the given Coder<T> to decode each of the lines of the file into a value of type T.
withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a transform for writing to text files that's like this one but that uses the given Coder to encode each of the elements of the input PCollection into an output text line.
withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a transform for writing to text files that uses the given Coder to encode each of the elements of the input PCollection into an output text line.
withCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.TimestampedValues
Returns a Create.TimestampedValues PTransform like this one that uses the given Coder<T> to decode each of the objects into a value of type T.
withCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
Returns a Create.Values PTransform like this one that uses the given Coder<T> to decode each of the objects into a value of type T.
withCompressionType(TextIO.CompressionType) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new transform for reading from text files that's like this one but reads from input sources using the specified compression type.
withCompressionType(TextIO.CompressionType) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a transform for reading text files that decompresses all input files using the specified compression type.
withCreateDisposition(BigQueryIO.Write.CreateDisposition) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns a copy of this write transformation, but using the specified create disposition.
withCreateDisposition(BigQueryIO.Write.CreateDisposition) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation with the specified options for creating the table.
withDataset(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
Returns a Sink that is like this one, but will write to the specified dataset.
withDataset(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
withDecompression(CompressedSource.DecompressingChannelFactory) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Return a CompressedSource that is like this one but will decompress its underlying file with the given CompressedSource.DecompressingChannelFactory.
withDefaultValue(T) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
Default value to return for windows with no value in them.
withEarlyFirings(Trigger.OnceTrigger<W>) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.AfterWatermarkLate
Creates a new Trigger like the this, except that it fires repeatedly whenever the given Trigger fires before the watermark has passed the end of the window.
withEarlyFirings(Trigger.OnceTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
Creates a new Trigger like the this, except that it fires repeatedly whenever the given Trigger fires before the watermark has passed the end of the window.
withEncodabilityTesting(boolean) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Enable runtime testing to verify that all values can be encoded.
withEndKey(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Returns new ByteKeyRange like this one, but with the specified end key.
withEpsilon(double) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Returns an ApproximateQuantilesCombineFn that's like this one except that it uses the specified epsilon value.
withExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
Produces a Proto2Coder like this one, but with the extensions from the given classes registered.
withExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Deprecated.
withExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
Returns a ProtoCoder like this one, but with the extensions from the given classes registered.
withExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
withFanout(int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
Returns a PTransform identical to this, but that uses an intermediate node to combine parts of the data to reduce load on the final global combine step.
withGapDuration(Duration) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
Creates a Sessions WindowFn with the specified gap duration.
withHost(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
Returns a Sink that is like this one, but will use the given host.
withHost(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
withHotKeyFanout(SerializableFunction<? super K, Integer>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
If a single key has disproportionately many values, it may become a bottleneck, especially in streaming mode.
withHotKeyFanout(int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
Like Combine.PerKey.withHotKeyFanout(SerializableFunction), but returning the given constant value for every key.
withKey(Object) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns a new ValueWithMetadata with the implicit key associated with this value set.
WithKeys<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
WithKeys<K, V> takes a PCollection<V>, and either a constant key of type K or a function from V to K, and returns a PCollection<KV<K, V>>, where each of the values in the input PCollection has been paired with either the constant key or a key computed from the value.
withKeyType(TypeDescriptor<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithKeys
Return a WithKeys that is like this one with the specified key type descriptor.
withLabel(String) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.ItemBuilder
Add a human-readable label to describe the most-recently added metadata field.
withLateFirings(Trigger.OnceTrigger<W>) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.AfterWatermarkEarly
Creates a new Trigger like the this, except that it fires repeatedly whenever the given Trigger fires after the watermark has passed the end of the window.
withLateFirings(Trigger.OnceTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
Creates a new Trigger like the this, except that it fires repeatedly whenever the given Trigger fires after the watermark has passed the end of the window.
withLinkUrl(String) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.ItemBuilder
Add a link URL to the most-recently added display metadata.
withMaxInputSize(long) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Returns an ApproximateQuantilesCombineFn that's like this one except that it uses the specified maxNumElements value.
withMaxNumRecords(long) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput
Returns an CountingInput.UnboundedCountingInput like this one, but that will read at most the specified number of elements.
withMaxNumRecords(long) - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
Returns a new BoundedReadFromUnboundedSource that reads a bounded amount of data from the given UnboundedSource.
withMaxParallelism(int) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound
Returns a new IntraBundleParallelization PTransform like this one with the specified maximum concurrency level.
withMaxParallelism(int) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Unbound
Returns a new IntraBundleParallelization PTransform like this one with the specified maximum concurrency level.
withMaxParallelism(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization
Creates a IntraBundleParallelization PTransform with the specified maximum concurrency level.
withMaxReadTime(Duration) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput
Returns an CountingInput.UnboundedCountingInput like this one, but that will read for at most the specified amount of time.
withMaxReadTime(Duration) - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
Returns a new BoundedReadFromUnboundedSource that reads a bounded amount of data from the given UnboundedSource.
withMinBundleSize(long) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
Returns an AvroSource that's like this one but uses the supplied minimum bundle size.
withMinBundleSize(long) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
Sets a parameter minBundleSize for the minimum bundle size of the source.
withNamespace(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
withNumShards(int) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but that uses the provided shard count.
withNumShards(int) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform that uses the provided shard count.
withNumShards(int) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a transform for writing to text files that's like this one but that uses the provided shard count.
withNumShards(int) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a transform for writing to text files that uses the provided shard count.
withOffset(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
Partitions the timestamp space into half-open intervals of the form [N * size + offset, (N + 1) * size + offset), where 0 is the epoch.
withOffset(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
Assigns timestamps into half-open intervals of the form [N * period + offset, N * period + offset + size).
withoutDefaults() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
Returns a PTransform identical to this, but that does not attempt to provide a default value in the case of empty input.
withOutputTags(TupleTag<OutputT>, TupleTagList) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
Returns a new multi-output ParDo PTransform that's like this PTransform but with the specified main and side output tags.
withOutputTags(TupleTag<OutputT>, TupleTagList) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new multi-output ParDo transform that's like this transform but with the specified main and side output tags.
withOutputTags(TupleTag<OutputT>, TupleTagList) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a multi-output ParDo PTransform whose output PCollections will be referenced using the given main output and side output tags.
withOutputTimeFn(OutputTimeFn<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
(Experimental) Override the default OutputTimeFn, to control the output timestamp of values output from a GroupByKey operation.
withOutputType(TypeDescriptor<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements.MissingOutputTypeDescriptor
 
withOutputType(TypeDescriptor<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.MapElements.MissingOutputTypeDescriptor
 
withoutResultFlattening() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
withoutSharding() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but that forces a single file as output.
withoutSharding() - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform that forces a single file as output.
withoutSharding() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a transform for writing to text files that's like this one but that forces a single file as output.
withoutSharding() - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a transform for writing to text files that forces a single file as output.
withoutStrictParsing() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
During parsing of the arguments, we will skip over improperly formatted and unknown arguments.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new PTransform that's like this one but that has GCS input path validation on pipeline creation disabled.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns a PTransform that reads Avro file(s) that has GCS path validation on pipeline creation disabled.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but that has GCS output path validation on pipeline creation disabled.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform that writes Avro file(s) that has GCS path validation on pipeline creation disabled.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Disable table validation.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
Disables BigQuery table validation, which is enabled by default.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns a copy of this write transformation, but without BigQuery table validation.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation with BigQuery table validation disabled.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new transform for reading from text files that's like this one but that has GCS path validation on pipeline creation disabled.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a transform for reading text files that has GCS path validation on pipeline creation disabled.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a transform for writing to text files that's like this one but that has GCS output path validation on pipeline creation disabled.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a transform for writing to text files that has GCS path validation on pipeline creation disabled.
withQuery(DatastoreV1.Query) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
Returns a new Source that reads the results of the specified query.
withRecordClass(Class<T>) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
Sets a JAXB annotated class that can be populated using a record of the provided XML file.
withRecordElement(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
Sets name of the record element of the XML document.
withRepresentativeType(TypeDescriptor<IdT>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates.WithRepresentativeValues
Return a WithRepresentativeValues PTransform that is like this one, but with the specified output type descriptor.
withRepresentativeValueFn(SerializableFunction<T, IdT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates
Returns a RemoveDuplicates<T, IdT> PTransform.
withRootElement(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
Returns an XmlSink that writes XML files with an enclosing root element of the supplied name.
withRootElement(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
Sets name of the root element of the XML document.
withRowFilter(RowFilter) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
Returns a new BigtableIO.Read that will filter the rows read from Cloud Bigtable using the given row filter.
withSchema(Class<X>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new PTransform that's like this one but that reads Avro file(s) containing records whose type is the specified Avro-generated class.
withSchema(Schema) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new PTransform that's like this one but that reads Avro file(s) containing records of the specified schema.
withSchema(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new PTransform that's like this one but that reads Avro file(s) containing records of the specified schema in a JSON-encoded string form.
withSchema(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns a PTransform that reads Avro file(s) containing records whose type is the specified Avro-generated class.
withSchema(Schema) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns a PTransform that reads Avro file(s) containing records of the specified schema.
withSchema(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns a PTransform that reads Avro file(s) containing records of the specified schema in a JSON-encoded string form.
withSchema(Class<X>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but that writes to Avro file(s) containing records whose type is the specified Avro-generated class.
withSchema(Schema) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but that writes to Avro file(s) containing records of the specified schema.
withSchema(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but that writes to Avro file(s) containing records of the specified schema in a JSON-encoded string form.
withSchema(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform that writes Avro file(s) containing records whose type is the specified Avro-generated class.
withSchema(Schema) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform that writes Avro file(s) containing records of the specified schema.
withSchema(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform that writes Avro file(s) containing records of the specified schema in a JSON-encoded string form.
withSchema(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
Returns an AvroSource that's like this one but reads files containing records that conform to the given schema.
withSchema(Schema) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
Returns an AvroSource that's like this one but reads files containing records that conform to the given schema.
withSchema(Class<X>) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
Returns an AvroSource that's like this one but reads files containing records of the type of the given class.
withSchema(TableSchema) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns a copy of this write transformation, but using the specified schema for rows to be written.
withSchema(TableSchema) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation with the specified schema to use in table creation.
withSerializabilityTesting(boolean) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Enable runtime testing to verify that all functions and Coder instances can be serialized.
withShardNameTemplate(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but that uses the given shard name template.
withShardNameTemplate(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform that uses the given shard name template.
withShardNameTemplate(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a transform for writing to text files that's like this one but that uses the given shard name template.
withShardNameTemplate(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a transform for writing to text files that uses the given shard name template.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
Returns a PTransform identical to this, but with the specified side inputs to use in CombineWithContext.CombineFnWithContext.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
 
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
Returns a PTransform identical to this, but with the specified side inputs to use in CombineWithContext.KeyedCombineFnWithContext.
withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
Returns a new ParDo PTransform that's like this PTransform but with the specified additional side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
Returns a new ParDo PTransform that's like this PTransform but with the specified additional side inputs.
withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
Returns a new multi-output ParDo PTransform that's like this PTransform but with the specified additional side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
Returns a new multi-output ParDo PTransform that's like this PTransform but with the specified additional side inputs.
withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo transform that's like this transform but with the specified additional side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo transform that is like this transform but with the specified additional side inputs.
withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo transform that's like this transform but with the specified side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo transform that's like this transform but with the specified additional side inputs.
withSideInputs(PCollectionView<?>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo PTransform with the given side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo with the given side inputs.
withSingletonValues() - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap
Deprecated.
this method simply returns this AsMap unmodified
withStartingDay(int, int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
withStartingMonth(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
withStartingYear(int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
withStartKey(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
Returns new ByteKeyRange like this one, but with the specified start key.
withSuffix(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new PTransform that's like this one but that writes to the file(s) with the given filename suffix.
withSuffix(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a PTransform that writes to the file(s) with the given filename suffix.
withSuffix(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a transform for writing to text files that that's like this one but that writes to the file(s) with the given filename suffix.
withSuffix(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a transform for writing to text files that appends the specified suffix to the created files.
withTableId(String) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
Returns a new BigtableIO.Read that will read from the specified table.
withTableId(String) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
Returns a new BigtableIO.Write that will write to the specified table.
withTimestampFn(SerializableFunction<Long, Instant>) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput
Returns an CountingInput.UnboundedCountingInput like this one, but where output elements have the timestamp specified by the timestampFn.
WithTimestamps<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform for assigning timestamps to all the elements of a PCollection.
withTimeZone(DateTimeZone) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
withTimeZone(DateTimeZone) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
withTimeZone(DateTimeZone) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
withUnorderednessTesting(boolean) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Enable runtime testing to verify that functions do not depend on order of the elements.
withValidation() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
After creation we will validate that PipelineOptions conforms to all the validation criteria from <T>.
withValidation() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
After creation we will validate that <T> conforms to all the validation criteria.
withValue(T) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns a new ValueWithMetadata that is a copy of this one, but with a different value.
withWriteDisposition(BigQueryIO.Write.WriteDisposition) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns a copy of this write transformation, but using the specified write disposition.
withWriteDisposition(BigQueryIO.Write.WriteDisposition) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation with the specified options for writing to the table.
WorkerHarnessContainerImageFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory
 
WorkerLogLevelOverrides() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
 
write() - Static method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO
Creates an uninitialized BigtableIO.Write.
write(T) - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
Called for each value in the bundle.
Write() - Constructor for class com.google.cloud.dataflow.sdk.io.TextIO.Write
 
Write - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes to a Sink.
Write() - Constructor for class com.google.cloud.dataflow.sdk.io.Write
 
write() - Static method in class com.google.cloud.dataflow.sdk.io.XmlSink
Returns a builder for an XmlSink.
write(T) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Writes a value to the stream.
Write - Class in com.google.cloud.dataflow.sdk.transforms
Deprecated.
Write() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Write
Deprecated.
 
Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes to a Sink.
writeFooter() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Writes footer at the end of output files.
writeFooter() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Writes the root element closing tag.
writeHeader() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Writes header at the beginning of output files.
writeHeader() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Writes the root element opening tag.
writeOf(Class<T>, String, String) - Static method in class com.google.cloud.dataflow.sdk.io.XmlSink
Returns an XmlSink that writes objects as XML entities.
WriteOperation() - Constructor for class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
 
Writer() - Constructor for class com.google.cloud.dataflow.sdk.io.Sink.Writer
 
writeTo(String) - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Returns a new Write transform that will write to a Sink.

X

XML_EXTENSION - Static variable in class com.google.cloud.dataflow.sdk.io.XmlSink
 
XmlSink - Class in com.google.cloud.dataflow.sdk.io
A Sink that outputs records as XML-formatted elements.
XmlSink() - Constructor for class com.google.cloud.dataflow.sdk.io.XmlSink
 
XmlSink.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A FileBasedSink that writes objects as XML elements.
XmlSink.XmlWriteOperation<T> - Class in com.google.cloud.dataflow.sdk.io
XmlSink.XmlWriter<T> - Class in com.google.cloud.dataflow.sdk.io
A Sink.Writer that can write objects as XML elements.
XmlSource<T> - Class in com.google.cloud.dataflow.sdk.io
A source that can be used to read XML files.
XmlWriteOperation(XmlSink.Bound<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriteOperation
 
XmlWriter(XmlSink.XmlWriteOperation<T>, Marshaller) - Constructor for class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
 

Y

years(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
Returns a WindowFn that windows elements into periods measured by years.
A B C D E F G H I J K L M N O P R S T U V W X Y 
Skip navigation links