Skip navigation links
A B C D E F G H I J K L M N O P R S T U V W X Y 

A

AbstractBoundedReader() - Constructor for class com.google.cloud.dataflow.sdk.io.BoundedSource.AbstractBoundedReader
 
AbstractJobException - Exception in com.google.cloud.dataflow.sdk.runners
A RuntimeException that contains information about a DataflowPipelineJob.
AbstractReader() - Constructor for class com.google.cloud.dataflow.sdk.io.Source.AbstractReader
 
AccumulatingCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
 
accumulatingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Triggering
Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.
add(Long) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique
Adds a value to the heap, returning whether the value is (large enough to be) in the heap.
addCollectionToSingletonOutput(String, PValue, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an output with the given name to the previously added CollectionToSingleton Dataflow step, consuming the specified input PValue and producing the specified output PValue.
addComposite(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Adds a composite operation to the transform node.
addEncodingInput(Coder<?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Sets the encoding for the current Dataflow step.
addExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
addExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
addInput(String, Boolean) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input with the given name and value to the current Dataflow step.
addInput(String, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input with the given name and value to the current Dataflow step.
addInput(String, Long) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input with the given name and value to the current Dataflow step.
addInput(String, PInput) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input with the given name to the previously added Dataflow step, coming from the specified input PValue.
addInput(String, Map<String, Object>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input that is a dictionary of strings to objects.
addInput(String, List<? extends Map<String, Object>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an input that is a list of objects.
addInput(TransformTreeNode, PInput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Adds an input to the given node.
addInput(ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique, T) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
addInput(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn.Accumulator
Adds the given input value to this accumulator, modifying this accumulator.
addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
 
addInput(double[], Double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
addInput(Combine.BinaryCombineFn.Holder<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
addInput(int[], Integer) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
addInput(long[], Long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Adds the given input value to the given accumulator, returning the new accumulator value.
addInput(K, AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Adds the given input value to the given accumulator, modifying the accumulator.
addInput(List<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.SimpleCombineFn
 
addInput(Top.BoundedHeap<KV<Integer, T>, SerializableComparator<KV<Integer, T>>>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
addInput(List<T>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.View.Concatenate
 
addInputProducer(PValue, TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Adds an input to the transform node.
addInputsRandomly(Combine.KeyedCombineFn<? super K, ? super InputT, AccumT, ?>, K, Iterable<InputT>, Random) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
 
addOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.
addStep(PTransform<?, ?>, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds a step to the Dataflow workflow for the given transform, with the given Dataflow step type.
addStep(PTransform<?, ? extends PValue>, Step) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds a pre-defined step to the Dataflow workflow.
addValue(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator
Adds a new value into the Aggregator.
addValueInternal(PValue) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Adds the given PValue to this Pipeline.
addValueOnlyOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.
advance() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
advance() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
 
advance() - Method in interface com.google.cloud.dataflow.sdk.io.Source.Reader
Advances the reader to the next valid record.
advance() - Method in interface com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Advances the reader to the next valid record.
AfterAll<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Create a Trigger that fires and finishes once after all of its sub-triggers have fired.
AfterEach<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A composite Trigger that executes its sub-triggers in order.
AfterFirst<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Create a composite Trigger that fires once after at least one of its sub-triggers have fired.
AfterPane<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Triggers that fire based on properties of the elements in the current pane.
AfterProcessingTime<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
AfterProcessingTime triggers fire based on the current processing time.
AfterWatermark<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
AfterWatermark triggers fire based on progress of the system watermark.
AfterWatermark(List<SerializableFunction<Instant, Instant>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark
 
Aggregator<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
An Aggregator<InputT> enables monitoring of values of type InputT, to be combined across all bundles.
AggregatorPipelineExtractor - Class in com.google.cloud.dataflow.sdk.runners
An AggregatorPipelineExtractor retrieves Aggregators at each ParDo and returns a Map of Aggregator to the PTransforms in which it is present.
AggregatorPipelineExtractor(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorPipelineExtractor
Creates an AggregatorPipelineExtractor for the given Pipeline.
AggregatorRetrievalException - Exception in com.google.cloud.dataflow.sdk.runners
Signals that an exception has occurred while retrieving Aggregators.
AggregatorRetrievalException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.AggregatorRetrievalException
 
AggregatorRetriever - Class in com.google.cloud.dataflow.sdk.transforms
A class for extracting Aggregators from DoFns.
AggregatorValues<T> - Class in com.google.cloud.dataflow.sdk.runners
A collection of values associated with an Aggregator.
AggregatorValues() - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorValues
 
alignedTo(Duration, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
Aligns timestamps to the smallest multiple of size since the offset greater than the timestamp.
alignedTo(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
Aligns the time to be the smallest multiple of size greater than the timestamp since the epoch.
ALL_CONTEXTS - Static variable in class com.google.cloud.dataflow.sdk.testing.CoderProperties
All the contexts, for use in test cases.
and(TupleTag<V>, List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns a new CoGbkResult based on this, with the given tag and given data added to it.
and(TupleTag<V>, PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns a new KeyedPCollectionTuple<K> that is the same as this, appended with the given PCollection.
and(PCollection.IsBounded) - Method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded
Returns the composed IsBounded property.
and(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollection appended to the end.
and(Iterable<PCollection<T>>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollections appended to the end, in order.
and(TupleTag<T>, PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns a new PCollectionTuple that has each PCollection and TupleTag of this PCollectionTuple plus the given PCollection associated with the given TupleTag.
and(TupleTag<?>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns a new TupleTagList that has all the TupleTags of this TupleTagList plus the given TupleTag appended to the end.
and(List<TupleTag<?>>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns a new TupleTagList that has all the TupleTags of this TupleTagList plus the given TupleTags appended to the end, in order.
any(long) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample
Sample#any(long) takes a PCollection<T> and a limit, and produces a new PCollection<T> containing up to limit elements of the input PCollection.
ApplicationNameOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that allow setting the application name.
AppliedPTransform<InputT extends PInput,OutputT extends POutput,TransformT extends PTransform<? super InputT,OutputT>> - Class in com.google.cloud.dataflow.sdk.transforms
Represents the application of a PTransform to a specific input to produce a specific output.
apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.coders.DelegateCoder.CodingFunction
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
 
apply(PCollection<TableRow>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.Read.Bound
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Like Pipeline.apply(String, PTransform) but defaulting to the name of the PTransform.
apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Starts using this pipeline with a root PTransform such as TextIO.READ or Create.
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
 
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Applies the given transform to the input.
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
 
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
 
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner
Applies a transform to the given input, returning the output.
apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
 
apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
Applies the binary operation to the two operands, returning the result.
apply(V, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
Applies the binary operation to the two operands, returning the result.
apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
Applies the binary operation to the two operands, returning the result.
apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
Applies the binary operation to the two operands, returning the result.
apply(Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Applies this CombineFn to a collection of input values to produce a combined output value.
apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
 
apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView
 
apply(PCollection<? extends KV<K, ? extends Iterable<InputT>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
 
apply(K, Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Applies this KeyedCombineFn to a key and a collection of input values to produce a combined output value.
apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
 
apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKeyWithHotKeyFanout
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Count.Globally
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Count.PerElement
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.TimestampedValues
 
apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
 
apply(PCollection<Iterable<T>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenIterables
 
apply(PCollectionList<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenPCollectionList
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
 
apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupAlsoByWindow
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.ReifyTimestampsAndWindows
 
apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.SortValuesByTimestamp
 
apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound
 
apply(KeyedPCollectionTuple<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGroupByKey
 
apply(PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Like KeyedPCollectionTuple.apply(String, PTransform) but defaulting to the name provided by the PTransform.
apply(String, PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Applies the given PTransform to this input KeyedPCollectionTuple and returns its OutputT.
apply(PCollection<? extends KV<K, ?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Keys
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.KvSwap
 
apply(NumT, NumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
 
apply(NumT, NumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
 
apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Partition
 
apply(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Applies this PTransform on the given InputT, and returns its Output.
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.SampleAny
 
apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.SerializableFunction
Returns the result of invoking this function on the given input.
apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
 
apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
 
apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
 
apply(PCollection<? extends KV<?, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Values
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsIterable
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap
 
apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMultimap
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
 
apply(PCollection<ElemT>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.CreatePCollectionView
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Remerge
 
apply(PCollection<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithKeys
 
apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Write.Bound
 
apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin
Like PBegin.apply(String, PTransform) but defaulting to the name of the PTransform.
apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin
Applies the given PTransform to this input PBegin, using name to identify this specific application of the transform.
apply(PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Like PCollection.apply(String, PTransform) but defaulting to the name of the PTransform.
apply(String, PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Applies the given PTransform to this input PCollection<T>, using name to identify this specific application of the transform.
apply(PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Like PCollectionList.apply(String, PTransform) but defaulting to the name of the PTransform.
apply(String, PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Applies the given PTransform to this input PCollectionList<T>, using name to identify this specific application of the transform.
apply(PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Like PCollectionTuple.apply(String, PTransform) but defaulting to the name of the PTransform.
apply(String, PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Applies the given PTransform to this input PCollectionTuple, using name to identify this specific application of the transform.
applyTransform(InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline
Like Pipeline.applyTransform(String, PInput, PTransform) but defaulting to the name provided by the PTransform.
applyTransform(String, InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline
Applies the given PTransform to this input InputT and returns its OutputT.
ApproximateQuantiles - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for getting an idea of a PCollection's data distribution using approximate N-tiles, either globally or per-key.
ApproximateQuantiles() - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
 
ApproximateQuantiles.ApproximateQuantilesCombineFn<T,ComparatorT extends Comparator<T> & Serializable> - Class in com.google.cloud.dataflow.sdk.transforms
The ApproximateQuantilesCombineFn combiner gives an idea of the distribution of a collection of values using approximate N-tiles.
ApproximateUnique - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for estimating the number of distinct elements in a PCollection, or the number of distinct values associated with each key in a PCollection of KVs.
ApproximateUnique() - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
 
ApproximateUnique.ApproximateUniqueCombineFn<T> - Class in com.google.cloud.dataflow.sdk.transforms
CombineFn that computes an estimate of the number of distinct values that were combined.
ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique - Class in com.google.cloud.dataflow.sdk.transforms
A heap utility class to efficiently track the largest added elements.
ApproximateUniqueCombineFn(long, Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
areAllSubtriggersFinished() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Returns true if all the sub-triggers of the current trigger are marked finished.
as(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
Transforms this object into an object of type <T> saving each property that has been manipulated.
as(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Creates and returns an object that implements <T>.
as(Class<T>) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
Creates and returns an object that implements <T> using the values configured on this builder during construction.
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
asCloudObject() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Returns the CloudObject that represents this Coder.
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
asCloudObject() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
 
asIterable() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns a View.AsIterable that takes a PCollection as input and produces a PCollectionView of the values, to be consumed as an iterable side input.
asKeyedFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Converts this CombineFn into an equivalent Combine.KeyedCombineFn that ignores the keys passed to it and combines the values according to this CombineFn.
asList() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns a transform that takes a PCollection and returns a List containing all of its elements, to be consumed as a side input.
asMap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns an View.AsMap transform that takes a PCollection as input and produces a PCollectionView of the values to be consumed as a Map<K, V> side input.
asMultimap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns an View.AsMultimap transform that takes a PCollection of KV<K, V> pairs as input and produces a PCollectionView of its contents as a Map<K, Iterable<V>> for use as a side input.
asOutputReference(PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Encode a PValue reference as an output reference.
assertFor(ExpectedT) - Method in interface com.google.cloud.dataflow.sdk.testing.DataflowAssert.AssertRelation
 
AssignContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
 
assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
 
assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns true if this WindowFn assigns each element to a single window.
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
Returns the single window to which elements with this timestamp belong.
assignWindows(WindowFn<Object, GlobalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
assignWindows(WindowFn<Object, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
 
assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Given a timestamp and element, returns the set of windows into which it should be placed.
asSingleton() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
Returns a View.AsSingleton transform that takes a singleton PCollection as input and produces a PCollectionView of the single value, to be consumed as a side input.
asSingletonView() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
Returns a PTransform that produces a PCollectionView whose elements are the result of combining elements per-window in the input PCollection.
asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
 
asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
 
asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
 
asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
 
AtomicCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
An AtomicCoder is a Coder that has no component Coders or other state.
AtomicCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.AtomicCoder
 
AvroCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A Coder using Avro binary format.
AvroCoder(Class<T>, Schema) - Constructor for class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
AvroCoder.AvroDeterminismChecker - Class in com.google.cloud.dataflow.sdk.coders
Helper class encapsulating the various pieces of state maintained by the recursive walk used for checking if the encoding will be deterministic.
AvroDeterminismChecker() - Constructor for class com.google.cloud.dataflow.sdk.coders.AvroCoder.AvroDeterminismChecker
 
AvroIO - Class in com.google.cloud.dataflow.sdk.io
PTransforms for reading and writing Avro files.
AvroIO() - Constructor for class com.google.cloud.dataflow.sdk.io.AvroIO
 
AvroIO.Read - Class in com.google.cloud.dataflow.sdk.io
A root PTransform that reads from an Avro file (or multiple Avro files matching a pattern) and returns a PCollection containing the decoding of each record.
AvroIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from an Avro file (or multiple Avro files matching a pattern) and returns a bounded PCollection containing the decoding of each record.
AvroIO.Write - Class in com.google.cloud.dataflow.sdk.io
A root PTransform that writes a PCollection to an Avro file (or multiple Avro files matching a sharding pattern).
AvroIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes a bounded PCollection to an Avro file (or multiple Avro files matching a sharding pattern).
AvroReader(AvroSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
AvroSource<T> - Class in com.google.cloud.dataflow.sdk.io
A FileBasedSource for reading Avro-format files.
AvroSource.AvroReader<T> - Class in com.google.cloud.dataflow.sdk.io
A BlockBasedSource.BlockBasedReader for reading blocks from Avro files.

B

baseOutputFilename - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink
Base filename for final output files.
baseTemporaryFilename - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Base filename used for temporary output files.
begin() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Returns a PBegin owned by this Pipeline.
beginningOnDay(int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
beginningOnDay(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
BigEndianIntegerCoder - Class in com.google.cloud.dataflow.sdk.coders
A BigEndianIntegerCoder encodes Integers in 4 bytes, big-endian.
BigEndianLongCoder - Class in com.google.cloud.dataflow.sdk.coders
A BigEndianLongCoder encodes Longs in 8 bytes, big-endian.
BigQueryIO - Class in com.google.cloud.dataflow.sdk.io
PTransforms for reading and writing BigQuery tables.
BigQueryIO() - Constructor for class com.google.cloud.dataflow.sdk.io.BigQueryIO
 
BigQueryIO.Read - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from a BigQuery table and returns a PCollection of TableRows containing each of the rows of the table.
BigQueryIO.Read.Bound - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from a BigQuery table and returns a bounded PCollection of TableRows.
BigQueryIO.Write - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes a PCollection containing TableRows to a BigQuery table.
BigQueryIO.Write.Bound - Class in com.google.cloud.dataflow.sdk.io
A PTransform that can write either a bounded or unbounded PCollection of TableRows to a BigQuery table.
BigQueryIO.Write.CreateDisposition - Enum in com.google.cloud.dataflow.sdk.io
An enumeration type for the BigQuery create disposition strings publicly documented as CREATE_NEVER, and CREATE_IF_NEEDED.
BigQueryIO.Write.WriteDisposition - Enum in com.google.cloud.dataflow.sdk.io
An enumeration type for the BigQuery write disposition strings publicly documented as WRITE_TRUNCATE, WRITE_APPEND, and WRITE_EMPTY.
BigQueryOptions - Interface in com.google.cloud.dataflow.sdk.options
Properties needed when using BigQuery with the Dataflow SDK.
BinaryCombineDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
BinaryCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
BinaryCombineIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
BinaryCombineLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
Block() - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
 
BlockBasedReader(BlockBasedSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
 
BlockBasedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A BlockBasedSource is a FileBasedSource where a file consists of blocks of records.
BlockBasedSource(String, long) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource
Creates a BlockBasedSource based on a file name or pattern.
BlockBasedSource(String, long, long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource
Creates a BlockBasedSource for a single file.
BlockBasedSource.Block<T> - Class in com.google.cloud.dataflow.sdk.io
A Block represents a block of records that can be read.
BlockBasedSource.BlockBasedReader<T> - Class in com.google.cloud.dataflow.sdk.io
A Reader that reads records from a BlockBasedSource.
BlockingDataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that are used to configure the BlockingDataflowPipelineRunner.
BlockingDataflowPipelineOptions.StandardOutputFactory - Class in com.google.cloud.dataflow.sdk.options
Returns a default of System.out.
BlockingDataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners
A PipelineRunner that's like DataflowPipelineRunner but that waits for the launched job to finish.
BlockingDataflowPipelineRunner(DataflowPipelineRunner, BlockingDataflowPipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
 
Bound() - Constructor for class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
 
BoundedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A Source that reads a finite amount of input and, because of that, supports some additional operations.
BoundedSource() - Constructor for class com.google.cloud.dataflow.sdk.io.BoundedSource
 
BoundedSource.AbstractBoundedReader<T> - Class in com.google.cloud.dataflow.sdk.io
A base class implementing some optional methods of BoundedSource.BoundedReader in a default way: Progress estimation (BoundedSource.AbstractBoundedReader.getFractionConsumed()) is not supported.
BoundedSource.BoundedReader<T> - Interface in com.google.cloud.dataflow.sdk.io
A Reader that reads a bounded amount of input and supports some additional operations, such as progress estimation and dynamic work rebalancing.
BoundedWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A BoundedWindow represents a finite grouping of elements, with an upper bound (larger timestamps represent more recent data) on the timestamps of elements that can be placed in the window.
BoundedWindow() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
 
buildTemporaryFilename(String, String) - Static method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Build a temporary filename using the temporary filename separator with the given prefix and suffix.
by(PredicateT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that satisfy the given predicate.
ByteArrayCoder - Class in com.google.cloud.dataflow.sdk.coders
A ByteArrayCoder encodes byte[] objects.
ByteOffsetBasedReader(ByteOffsetBasedSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource.ByteOffsetBasedReader
 
ByteOffsetBasedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A Source that uses byte offsets to define starting and ending positions.
ByteOffsetBasedSource(long, long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource
 
ByteOffsetBasedSource.ByteOffsetBasedReader<T> - Class in com.google.cloud.dataflow.sdk.io
A Source.Reader that implements code common to readers of all ByteOffsetBasedSources.

C

CalendarWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A collection of WindowFns that windows values into calendar-based windows such as spans of days, months, or years.
CalendarWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
 
CalendarWindows.DaysWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows elements into periods measured by days.
CalendarWindows.MonthsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows elements into periods measured by months.
CalendarWindows.YearsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows elements into periods measured by years.
cancel() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Cancels the job.
CannotProvideCoderException - Exception in com.google.cloud.dataflow.sdk.coders
The exception thrown when a CoderProvider cannot provide a Coder that has been requested.
CannotProvideCoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
CannotProvideCoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
CannotProvideCoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
 
check(TypeDescriptor<?>, Schema) - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder.AvroDeterminismChecker
 
clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Clear any state associated with this trigger in the given window.
clearOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Clears the record of the elements output so far to the main output.
clearSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Clears the record of the elements output so far to the side output with the given tag.
cloneAs(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
Makes a deep clone of this object, and transforms the cloned object into the specified type kls.
close() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Closes the channel and return the bundle result.
close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
Closes any ReadableByteChannel created for the current reader.
close() - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
Finishes writing the bundle.
close() - Method in interface com.google.cloud.dataflow.sdk.io.Source.Reader
Closes the reader.
CloudDebuggerOptions - Interface in com.google.cloud.dataflow.sdk.options
Options for controlling Cloud Debugger.
CodedTupleTag<T> - Class in com.google.cloud.dataflow.sdk.values
A TupleTag combined with the Coder to use for values associated with the tag.
CodedTupleTagMap - Class in com.google.cloud.dataflow.sdk.values
A mapping of CodedTupleTags to associated values.
Coder<T> - Interface in com.google.cloud.dataflow.sdk.coders
A Coder<T> defines how to encode and decode values of type T into byte streams.
Coder.Context - Class in com.google.cloud.dataflow.sdk.coders
The context in which encoding or decoding is being done.
Coder.NonDeterministicException - Class in com.google.cloud.dataflow.sdk.coders
Exception thrown by Coder.verifyDeterministic() if the encoding is not deterministic.
coderConsistentWithEquals(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderConsistentWithEqualsInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
coderDecodeEncodeContentsEqual(Coder<CollectionT>, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in any Coder.Context.
coderDecodeEncodeContentsEqualInContext(Coder<CollectionT>, Coder.Context, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in the given Coder.Context.
coderDecodeEncodeContentsInSameOrder(Coder<IterableT>, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in any Coder.Context.
coderDecodeEncodeContentsInSameOrderInContext(Coder<IterableT>, Coder.Context, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<Iterable<T>>, and value of type Iterable<T>, encoding followed by decoding yields an equal value of type Collection<T>, in the given Coder.Context.
coderDecodeEncodeEqual(Coder<T>, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<T>, and value of type T, encoding followed by decoding yields an equal value of type T, in any Coder.Context.
coderDecodeEncodeEqualInContext(Coder<T>, Coder.Context, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<T>, Coder.Context, and value of type T, encoding followed by decoding yields an equal value of type T.
coderDeterministic(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<T>, and values of type T, if the values are equal then the encoded bytes are equal, in any Coder.Context.
coderDeterministicInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
Verifies that for the given Coder<T>, Coder.Context, and values of type T, if the values are equal then the encoded bytes are equal.
CoderException - Exception in com.google.cloud.dataflow.sdk.coders
A CoderException is thrown if there is a problem encoding or decoding a value.
CoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException
 
CoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException
 
CoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException
 
CoderFactories - Class in com.google.cloud.dataflow.sdk.coders
Static utility methods for creating and working with Coders.
CoderFactory - Interface in com.google.cloud.dataflow.sdk.coders
A CoderFactory creates coders and decomposes values.
CoderProperties - Class in com.google.cloud.dataflow.sdk.testing
Properties for use in Coder tests.
CoderProperties() - Constructor for class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
CoderProvider - Interface in com.google.cloud.dataflow.sdk.coders
A CoderProvider may create a Coder for any concrete class.
CoderRegistry - Class in com.google.cloud.dataflow.sdk.coders
A CoderRegistry allows registering the default Coder to use for a Java class, and looking up and instantiating the default Coder for a Java type.
CoderRegistry() - Constructor for class com.google.cloud.dataflow.sdk.coders.CoderRegistry
 
coderSerializable(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
CoGbkResult - Class in com.google.cloud.dataflow.sdk.transforms.join
A row result of a CoGroupByKey.
CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
A row in the PCollection resulting from a CoGroupByKey transform.
CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>, int) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
 
CoGbkResult.CoGbkResultCoder - Class in com.google.cloud.dataflow.sdk.transforms.join
CoGbkResultSchema - Class in com.google.cloud.dataflow.sdk.transforms.join
A schema for the results of a CoGroupByKey.
CoGbkResultSchema(TupleTagList) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Builds a schema from a tuple of TupleTag<?>s.
CoGroupByKey<K> - Class in com.google.cloud.dataflow.sdk.transforms.join
A PTransform that performs a CoGroupByKey on a tuple of tables.
CollectionCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A CollectionCoder encodes Collections.
CollectionCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.CollectionCoder
 
com.google.cloud.dataflow.sdk - package com.google.cloud.dataflow.sdk
Provides a simple, powerful model for building both batch and streaming parallel data processing Pipelines.
com.google.cloud.dataflow.sdk.annotations - package com.google.cloud.dataflow.sdk.annotations
Defines annotations used across the SDK.
com.google.cloud.dataflow.sdk.coders - package com.google.cloud.dataflow.sdk.coders
Defines Coders to specify how data is encoded to and decoded from byte strings.
com.google.cloud.dataflow.sdk.io - package com.google.cloud.dataflow.sdk.io
Defines transforms for reading and writing common storage formats, including AvroIO, BigQueryIO, and TextIO.
com.google.cloud.dataflow.sdk.io.range - package com.google.cloud.dataflow.sdk.io.range
 
com.google.cloud.dataflow.sdk.options - package com.google.cloud.dataflow.sdk.options
Defines PipelineOptions for configuring pipeline execution.
com.google.cloud.dataflow.sdk.runners - package com.google.cloud.dataflow.sdk.runners
Defines runners for executing Pipelines in different modes, including DirectPipelineRunner and DataflowPipelineRunner.
com.google.cloud.dataflow.sdk.testing - package com.google.cloud.dataflow.sdk.testing
Defines utilities for unit testing Dataflow pipelines.
com.google.cloud.dataflow.sdk.transforms - package com.google.cloud.dataflow.sdk.transforms
Defines PTransforms for transforming data in a pipeline.
com.google.cloud.dataflow.sdk.transforms.join - package com.google.cloud.dataflow.sdk.transforms.join
Defines the CoGroupByKey transform for joining multiple PCollections.
com.google.cloud.dataflow.sdk.transforms.windowing - package com.google.cloud.dataflow.sdk.transforms.windowing
Defines the Window transform for dividing the elements in a PCollection into windows, and the Trigger for controlling when those elements are output.
com.google.cloud.dataflow.sdk.values - package com.google.cloud.dataflow.sdk.values
Defines PCollection and other classes for representing data in a Pipeline.
Combine - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for combining PCollection elements globally and per-key.
Combine() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine
 
Combine.AccumulatingCombineFn<InputT,AccumT extends Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT>,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A CombineFn that uses a subclass of Combine.AccumulatingCombineFn.Accumulator as its accumulator type.
Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
The type of mutable accumulator values used by this AccumulatingCombineFn.
Combine.BinaryCombineDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
An abstract subclass of Combine.CombineFn for implementing combiners that are more easily expressed as binary operations on doubles.
Combine.BinaryCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms
An abstract subclass of Combine.CombineFn for implementing combiners that are more easily expressed as binary operations.
Combine.BinaryCombineIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
An abstract subclass of Combine.CombineFn for implementing combiners that are more easily expressed as binary operations on ints.
Combine.BinaryCombineLongFn - Class in com.google.cloud.dataflow.sdk.transforms
An abstract subclass of Combine.CombineFn for implementing combiners that are more easily expressed as binary operations on longs.
Combine.CombineFn<InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A CombineFn<InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT into a single output value of type OutputT.
Combine.Globally<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
Combine.Globally<InputT, OutputT> takes a PCollection<InputT> and returns a PCollection<OutputT> whose elements are the result of combining all the elements in each window of the input PCollection, using a specified CombineFn<InputT, AccumT, OutputT>.
Combine.GloballyAsSingletonView<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
Combine.GloballyAsSingletonView<InputT, OutputT> takes a PCollection<InputT> and returns a PCollectionView<OutputT> whose elements are the result of combining all the elements in each window of the input PCollection, using a specified CombineFn<InputT, AccumT, OutputT>.
Combine.GroupedValues<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
GroupedValues<K, InputT, OutputT> takes a PCollection<KV<K, Iterable<InputT>>>, such as the result of GroupByKey, applies a specified KeyedCombineFn<K, InputT, AccumT, OutputT> to each of the input KV<K, Iterable<InputT>> elements to produce a combined output KV<K, OutputT> element, and returns a PCollection<KV<K, OutputT>> containing all the combined output elements.
Combine.KeyedCombineFn<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A KeyedCombineFn<K, InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT, associated with a key of type K, into a single output value of type OutputT.
Combine.PerKey<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
PerKey<K, InputT, OutputT> takes a PCollection<KV<K, InputT>>, groups it by key, applies a combining function to the InputT values associated with each key to produce a combined OutputT value, and returns a PCollection<KV<K, OutputT>> representing a map from each distinct key of the input PCollection to the corresponding combined value.
Combine.PerKeyWithHotKeyFanout<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
Like Combine.PerKey, but sharding the combining of hot keys.
Combine.SimpleCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms
Converts a SerializableFunction from Iterable<V>s to Vs into a simple Combine.CombineFn over Vs.
CombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
 
compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Largest
 
compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Smallest
 
compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByKey
 
compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByValue
 
compareTo(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
CompressedReader(CompressedSource<T>, FileBasedSource.FileBasedReader<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Create a CompressedReader from a CompressedSource and delegate reader.
CompressedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A Source that reads from compressed files.
CompressedSource.CompressedReader<T> - Class in com.google.cloud.dataflow.sdk.io
Reader for a CompressedSource.
CompressedSource.CompressionMode - Enum in com.google.cloud.dataflow.sdk.io
Default compression types supported by the CompressedSource.
CompressedSource.DecompressingChannelFactory - Interface in com.google.cloud.dataflow.sdk.io
Factory interface for creating channels that decompress the content of an underlying channel.
computeTargetTimestamp(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
 
Concatenate() - Constructor for class com.google.cloud.dataflow.sdk.transforms.View.Concatenate
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
 
consistentWithEquals() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Returns true if the encoded bytes of two objects are equal only when they are also equal according to Object.equals().
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
 
contains(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns whether this window contains the given window.
containsInAnyOrder(Iterable<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Checks that the Iterable contains the expected elements, in any order.
containsInAnyOrder(T...) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Checks that the Iterable contains the expected elements, in any order.
Context(boolean) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.Context
 
Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
 
Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
 
copyToOutputFiles(List<String>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Copy temporary files to final output filenames using the file naming template.
Count - Class in com.google.cloud.dataflow.sdk.transforms
PTransorms to count the elements in a PCollection.
Count() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Count
 
Count.Globally<T> - Class in com.google.cloud.dataflow.sdk.transforms
Count.Globally<T> takes a PCollection<T> and returns a PCollection<Long> containing a single element that is the total number of elements in the PCollection.
Count.PerElement<T> - Class in com.google.cloud.dataflow.sdk.transforms
Count.PerElement<T> takes a PCollection<T> and returns a PCollection<KV<T, Long>> representing a map from each distinct element of the input PCollection to the number of times that element occurs in the input.
create(List<? extends Coder<?>>) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderFactory
Returns a Coder<?>, given argument coder to use for values of a particular type, given the Coders for each of the type's generic parameter types.
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions.StandardOutputFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.DataflowClientFactory
 
create(String) - Static method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.NameMap
Parses a NameMap from a String.
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.PathValidatorFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.StagerFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions.JobNameFactory
 
create(String) - Static method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverride
Expects a value of the form Name#Level.
create(PipelineOptions) - Method in interface com.google.cloud.dataflow.sdk.options.DefaultValueFactory
Creates a default value for a getter marked with Default.InstanceFactory.
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.CredentialDirFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.GcpUserCredentialsFactory
 
create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcsOptions.ExecutorServiceFactory
 
create(AbstractGoogleClient, String) - Static method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
Creates a GoogleApiDebugOptions.GoogleApiTracer that sets the trace destination on all calls that match the given client type.
create(AbstractGoogleClientRequest<?>, String) - Static method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
Creates a GoogleApiDebugOptions.GoogleApiTracer that sets the trace traceDestination on all calls that match for the given request type.
create(String) - Static method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
Creates a GoogleClientRequestInitializer that adds the trace destination based upon the passed in value.
create() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
Creates and returns an object that implements PipelineOptions using the values configured on this builder during construction.
create() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Creates and returns an object that implements PipelineOptions.
create(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.Pipeline
Constructs a pipeline from the provided options.
create(DataflowPipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
Creates and returns a new DataflowPipeline instance for tests.
create(Combine.GroupedValues<K, InputT, OutputT>, PCollection<KV<K, Iterable<InputT>>>, boolean) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
 
create() - Static method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
Creates and returns a new test pipeline.
create(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Returns an approximate quantiles combiner with the given compareFn and desired number of quantiles.
create(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Like ApproximateQuantiles.ApproximateQuantilesCombineFn.create(int, Comparator), but sorts values using their natural ordering.
create(int, ComparatorT, long, double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Creates an approximate quantiles combiner with the given compareFn and desired number of quantiles.
Create<T> - Class in com.google.cloud.dataflow.sdk.transforms
Create<T> takes a collection of elements of type T known when the pipeline is constructed and returns a PCollection<T> containing the elements.
Create() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Create
 
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
Returns a GroupByKey<K, V> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGroupByKey
Returns a CoGroupByKey<K> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.Keys
Returns a Keys<K> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.KvSwap
Returns a KvSwap<K, V> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates
Returns a RemoveDuplicates<T> PTransform.
create() - Static method in class com.google.cloud.dataflow.sdk.transforms.Values
Returns a Values<V> PTransform.
Create.TimestampedValues<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that creates a PCollection whose elements have associated timestamps.
Create.Values<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that creates a PCollection from a set of in-memory objects.
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns a new, mutable accumulator value, representing the accumulation of zero input values.
createAccumulator(K) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Returns a new, mutable accumulator value representing the accumulation of zero input values.
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.SimpleCombineFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
 
createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.View.Concatenate
 
createAggregator(String, Combine.CombineFn<? super AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Returns an Aggregator with aggregation logic specified by the Combine.CombineFn argument.
createAggregator(String, SerializableFunction<Iterable<AggInputT>, AggInputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Returns an Aggregator with the aggregation logic specified by the SerializableFunction argument.
createAggregator(String, Combine.CombineFn<? super AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns an Aggregator with aggregation logic specified by the Combine.CombineFn argument.
createAggregator(String, SerializableFunction<Iterable<AggInputT>, AggInputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns an Aggregator with the aggregation logic specified by the SerializableFunction argument.
createAggregatorInternal(String, Combine.CombineFn<AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Creates an Aggregator in the DoFn context with the specified name and aggregation logic specified by Combine.CombineFn.
createDatumReader() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns a new DatumReader that can be used to read from an Avro file directly.
createDatumWriter() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns a new DatumWriter that can be used to write to an Avro file directly.
createDecompressingChannel(ReadableByteChannel) - Method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
 
createDecompressingChannel(ReadableByteChannel) - Method in interface com.google.cloud.dataflow.sdk.io.CompressedSource.DecompressingChannelFactory
Given a channel, create a channel that decompresses the content read from the channel.
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource
Creates a BlockBasedSource for the specified range in a single file.
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Creates a CompressedSource for a subrange of a file.
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
Creates and returns a new FileBasedSource of the same type as the current FileBasedSource backed by a given file and an offset range.
createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
createForTest() - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
Creates and returns a new DirectPipeline instance for tests.
createForTest() - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Constructs a runner with default properties for testing.
createFromSystemProperties() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Deprecated.
for internal use only
createFromSystemPropertiesInternal() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Creates a set of DataflowWorkerHarnessOptions based of a set of known system properties.
createInputStream(InputStream) - Method in enum com.google.cloud.dataflow.sdk.io.TextIO.CompressionType
 
createPaneInternal() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
 
createPrimitiveOutputInternal(Pipeline, WindowingStrategy<?, ?>, PCollection.IsBounded) - Static method in class com.google.cloud.dataflow.sdk.values.PCollection
Creates and returns a new PCollection for a primitive output.
createReader(PipelineOptions, ExecutionContext) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
 
createReader(PipelineOptions, ExecutionContext) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
createReader(PipelineOptions, ExecutionContext) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
createReader(PipelineOptions, CheckpointMarkT) - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
Create a new UnboundedSource.UnboundedReader to read from this source, resuming from the given checkpoint if present.
createSingleFileReader(PipelineOptions, ExecutionContext) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
createSingleFileReader(PipelineOptions, ExecutionContext) - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource
Creates a BlockBasedReader.
createSingleFileReader(PipelineOptions, ExecutionContext) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Creates a CompressedReader to read a single file.
createSingleFileReader(PipelineOptions, ExecutionContext) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
Creates and returns an instance of a FileBasedReader implementation for the current source assuming the source represents a single file.
createSingleFileReader(PipelineOptions, ExecutionContext) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
createSourceForSubrange(long, long) - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource
Returns a ByteOffsetBasedSource for a subrange of the current source.
createSourceForSubrange(long, long) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
 
createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink
Return a subclass of FileBasedSink.FileBasedWriteOperation that will manage the write to the sink.
createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink
Returns an instance of a Sink.WriteOperation that can write to this Sink.
createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Clients must implement to return a subclass of FileBasedSink.FileBasedWriter.
createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Creates a new Sink.Writer to write a bundle of the input to the sink.
createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriteOperation
Creates a XmlSink.XmlWriter with a marshaller for the type it will write.
CredentialDirFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcpOptions.CredentialDirFactory
 
currentProcessingTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Returns the current processing time.
CustomCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
An abstract base class for writing Coders that encodes itself via java serialization.
CustomCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.CustomCoder
 

D

DATAFLOW_STORAGE_LOCATION - Static variable in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
DataflowAssert - Class in com.google.cloud.dataflow.sdk.testing
An assertion on the contents of a PCollection incorporated into the pipeline.
DataflowAssert.AssertRelation<ActualT,ExpectedT> - Interface in com.google.cloud.dataflow.sdk.testing
A binary predicate between types Actual and Expected.
DataflowAssert.IterableAssert<T> - Class in com.google.cloud.dataflow.sdk.testing
An assertion about the contents of a PCollectionView.
DataflowAssert.SingletonAssert<T> - Class in com.google.cloud.dataflow.sdk.testing
An assertion about the single value of type T associated with a PCollectionView.
DataflowClientFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.DataflowClientFactory
 
DataflowPipeline - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineDebugOptions - Interface in com.google.cloud.dataflow.sdk.options
Internal.
DataflowPipelineDebugOptions.DataflowClientFactory - Class in com.google.cloud.dataflow.sdk.options
Returns the default Dataflow client built from the passed in PipelineOptions.
DataflowPipelineDebugOptions.NameMap - Class in com.google.cloud.dataflow.sdk.options
Map of old names to new names.
DataflowPipelineDebugOptions.PathValidatorFactory - Class in com.google.cloud.dataflow.sdk.options
Creates a PathValidator object using the class specified in DataflowPipelineDebugOptions.getPathValidatorClass().
DataflowPipelineDebugOptions.StagerFactory - Class in com.google.cloud.dataflow.sdk.options
Creates a Stager object using the class specified in DataflowPipelineDebugOptions.getStagerClass().
DataflowPipelineJob - Class in com.google.cloud.dataflow.sdk.runners
A DataflowPipelineJob represents a job submitted to Dataflow using DataflowPipelineRunner.
DataflowPipelineJob(String, String, Dataflow, DataflowAggregatorTransforms) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Constructs the job.
DataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that can be used to configure the DataflowPipeline.
DataflowPipelineOptions.JobNameFactory - Class in com.google.cloud.dataflow.sdk.options
Returns a normalized job name constructed from ApplicationNameOptions.getAppName(), the local system user name (if available), and the current time.
DataflowPipelineRegistrar - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineRegistrar.Options - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineRegistrar.Runner - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners
A PipelineRunner that executes the operations in the pipeline by first translating them to the Dataflow representation using the DataflowPipelineTranslator and then submitting them to a Dataflow service for execution.
DataflowPipelineRunner.StreamingPubsubIOWrite<T> - Class in com.google.cloud.dataflow.sdk.runners
Specialized implementation for PubsubIO.Write for the Dataflow runner in streaming mode.
DataflowPipelineRunnerHooks - Class in com.google.cloud.dataflow.sdk.runners
An instance of this class can be passed to the DataflowPipelineRunner to add user defined hooks to be invoked at various times during pipeline execution.
DataflowPipelineRunnerHooks() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunnerHooks
 
DataflowPipelineTranslator - Class in com.google.cloud.dataflow.sdk.runners
DataflowPipelineTranslator knows how to translate Pipeline objects into Cloud Dataflow Service API Jobs.
DataflowPipelineTranslator.JobSpecification - Class in com.google.cloud.dataflow.sdk.runners
The result of a job translation.
DataflowPipelineTranslator.TransformTranslator<TransformT extends PTransform> - Interface in com.google.cloud.dataflow.sdk.runners
A DataflowPipelineTranslator.TransformTranslator knows how to translate a particular subclass of PTransform for the Cloud Dataflow service.
DataflowPipelineTranslator.TranslationContext - Interface in com.google.cloud.dataflow.sdk.runners
The interface provided to registered callbacks for interacting with the DataflowPipelineRunner, including reading and writing the values of PCollections and side inputs (PCollectionViews).
DataflowPipelineWorkerPoolOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that are used to configure the Dataflow pipeline worker pool.
DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType - Enum in com.google.cloud.dataflow.sdk.options
Type of autoscaling algorithm to use.
DataflowPipelineWorkerPoolOptions.ClusterManagerApiType - Enum in com.google.cloud.dataflow.sdk.options
Type of API for handling cluster management, i.e.
DataflowPipelineWorkerPoolOptions.TeardownPolicy - Enum in com.google.cloud.dataflow.sdk.options
The policy for tearing down the workers spun up by the service.
DataflowWorkerHarnessOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that are used exclusively within the Dataflow worker harness.
DataflowWorkerLoggingOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that are used to control logging configuration on the Dataflow worker.
DataflowWorkerLoggingOptions.Level - Enum in com.google.cloud.dataflow.sdk.options
The set of log levels that can be used on the Dataflow worker.
DataflowWorkerLoggingOptions.WorkerLogLevelOverride - Class in com.google.cloud.dataflow.sdk.options
Defines a log level override for a specific class, package, or name.
DATASTORE_BATCH_UPDATE_LIMIT - Static variable in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Datastore has a limit of 500 mutations per batch operation, so we flush changes to Datastore every 500 entities.
DatastoreIO - Class in com.google.cloud.dataflow.sdk.io
PTransforms for reading and writing Google Cloud Datastore entities.
DatastoreIO() - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO
 
DatastoreIO.DatastoreReader - Class in com.google.cloud.dataflow.sdk.io
A Source.Reader over the records from a query of the datastore.
DatastoreIO.Sink - Class in com.google.cloud.dataflow.sdk.io
A Sink that writes a PCollection containing Entities to a Datastore kind.
DatastoreIO.Source - Class in com.google.cloud.dataflow.sdk.io
A Source that reads the result rows of a Datastore query as Entity objects.
DatastoreReader(DatastoreIO.Source, Datastore) - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
Returns a DatastoreReader with Source and Datastore object set.
days(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
Returns a WindowFn that windows elements into periods measured by days.
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
 
decode(InputStream, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Decodes a value of type T from the given input stream in the given context.
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow.Coder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
 
decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
 
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
Builds an instance of the coder's associated Iterable from a list of decoded elements.
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.ListCoder
 
decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.SetCoder
 
Default - Annotation Type in com.google.cloud.dataflow.sdk.options
Default represents a set of annotations that can be used to annotate getter properties on PipelineOptions with information representing the default value to be returned if no value is specified.
Default.Boolean - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified boolean primitive value.
Default.Byte - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified byte primitive value.
Default.Character - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified char primitive value.
Default.Class - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified Class value.
Default.Double - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified double primitive value.
Default.Enum - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified enum.
Default.Float - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified float primitive value.
Default.InstanceFactory - Annotation Type in com.google.cloud.dataflow.sdk.options
Value must be of type DefaultValueFactory and have a default constructor.
Default.Integer - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified int primitive value.
Default.Long - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified long primitive value.
Default.Short - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified short primitive value.
Default.String - Annotation Type in com.google.cloud.dataflow.sdk.options
This represents that the default of the option is the specified String value.
DEFAULT_HOST - Static variable in class com.google.cloud.dataflow.sdk.io.DatastoreIO
 
DEFAULT_MAX_NUM_ELEMENTS - Static variable in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
The cost (in time and space) to compute quantiles to a given accuracy is a function of the total number of elements in the data set.
DEFAULT_PANE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
 
DEFAULT_PUBSUB_CODER - Static variable in class com.google.cloud.dataflow.sdk.io.PubsubIO
 
DEFAULT_TEXT_CODER - Static variable in class com.google.cloud.dataflow.sdk.io.TextIO
 
DefaultCoder - Annotation Type in com.google.cloud.dataflow.sdk.coders
The DefaultCoder annotation specifies a default Coder class to handle encoding and decoding instances of the annotated class.
DefaultTrigger<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A trigger that is equivalent to Repeatedly.forever(AfterWatermark.pastEndOfWindow()).
DefaultValueFactory<T> - Interface in com.google.cloud.dataflow.sdk.options
An interface used with Default.InstanceFactory annotation to specify the class that will be an instance factory to produce default values for a given getter on PipelineOptions.
DelegateCoder<T,IntermediateT> - Class in com.google.cloud.dataflow.sdk.coders
A DelegateCoder<T, IntermediateT> wraps a Coder for IntermediateT and encodes/decodes values of type Ts by converting to/from IntermediateT and then encoding/decoding using the underlying Coder<IntermediateT>.
DelegateCoder(Coder<IntermediateT>, DelegateCoder.CodingFunction<T, IntermediateT>, DelegateCoder.CodingFunction<IntermediateT, T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
DelegateCoder.CodingFunction<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.coders
A CodingFunction<InputT, OutputT> is a serializable function from InputT to OutputT that may throw any Exception.
deleteTimer(TimerManager.TimeDomain) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Removes the timer set in this trigger context for the given window and timeDomain.
Description - Annotation Type in com.google.cloud.dataflow.sdk.options
Descriptions are used to generate human readable output when the --help command is specified.
destinationId - Variable in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnTimerContext
 
detectClassPathResourcesToStage(ClassLoader) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Attempts to detect all the resources the class loader has access to.
DeterministicStandardCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A DeterministicStandardCoder is a StandardCoder that is deterministic, in the sense that for objects considered equal according to Object.equals(), the encoded bytes are also equal.
DeterministicStandardCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.DeterministicStandardCoder
 
DIRECTORY_CONTAINER - Static variable in class com.google.cloud.dataflow.sdk.io.ShardNameTemplate
Shard is a file within a directory.
DirectPipeline - Class in com.google.cloud.dataflow.sdk.runners
DirectPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
Options that can be used to configure the DirectPipeline.
DirectPipelineRegistrar - Class in com.google.cloud.dataflow.sdk.runners
DirectPipelineRegistrar.Options - Class in com.google.cloud.dataflow.sdk.runners
Register the DirectPipelineRunner.
DirectPipelineRegistrar.Runner - Class in com.google.cloud.dataflow.sdk.runners
Register the DirectPipelineOptions.
DirectPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners
Executes the operations in the pipeline directly, in this process, without any optimization.
DirectPipelineRunner.EvaluationContext - Interface in com.google.cloud.dataflow.sdk.runners
The interface provided to registered callbacks for interacting with the DirectPipelineRunner, including reading and writing the values of PCollections and PCollectionViews.
DirectPipelineRunner.EvaluationResults - Interface in com.google.cloud.dataflow.sdk.runners
The interface provided to registered callbacks for interacting with the DirectPipelineRunner, including reading and writing the values of PCollections and PCollectionViews.
DirectPipelineRunner.TestCombineDoFn<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.runners
The implementation may split the Combine.KeyedCombineFn into ADD, MERGE and EXTRACT phases (see com.google.cloud.dataflow.sdk.runners.worker.CombineValuesFn).
DirectPipelineRunner.TransformEvaluator<TransformT extends PTransform> - Interface in com.google.cloud.dataflow.sdk.runners
An evaluator of a PTransform.
DirectPipelineRunner.ValueWithMetadata<V> - Class in com.google.cloud.dataflow.sdk.runners
An immutable (value, timestamp) pair, along with other metadata necessary for the implementation of DirectPipelineRunner.
discardingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Triggering
Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that discards elements in a pane after they are triggered.
DoFn<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
The argument to ParDo providing the code to use to process elements of the input PCollection.
DoFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn
 
DoFn.Context - Class in com.google.cloud.dataflow.sdk.transforms
Information accessible to all methods in this DoFn.
DoFn.ProcessContext - Class in com.google.cloud.dataflow.sdk.transforms
DoFn.RequiresWindowAccess - Interface in com.google.cloud.dataflow.sdk.transforms
Interface for signaling that a DoFn needs to access the window the element is being processed in, via DoFn.ProcessContext.window().
DoFnReflector - Class in com.google.cloud.dataflow.sdk.transforms
Utility implementing the necessary reflection for working with DoFnWithContexts.
DoFnReflector() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
 
DoFnTester<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A harness for unit-testing a DoFn.
DoFnWithContext<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
The argument to ParDo providing the code to use to process elements of the input PCollection.
DoFnWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
 
DoFnWithContext.Context - Class in com.google.cloud.dataflow.sdk.transforms
Information accessible to all methods in this DoFnWithContext.
DoFnWithContext.ExtraContextFactory<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
Interface for runner implementors to provide implementations of extra context information.
DoFnWithContext.FinishBundle - Annotation Type in com.google.cloud.dataflow.sdk.transforms
Annotation for the method to use to prepare an instance for processing a batch of elements.
DoFnWithContext.ProcessContext - Class in com.google.cloud.dataflow.sdk.transforms
DoFnWithContext.ProcessElement - Annotation Type in com.google.cloud.dataflow.sdk.transforms
Annotation for the method to use for processing elements.
DoFnWithContext.StartBundle - Annotation Type in com.google.cloud.dataflow.sdk.transforms
Annotation for the method to use to prepare an instance for processing a batch of elements.
DoubleCoder - Class in com.google.cloud.dataflow.sdk.coders
A DoubleCoder encodes Doubles in 8 bytes.
doublesGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<Double> and returns a PCollection<Double> whose contents is the maximum of the input PCollection's elements, or Double.NEGATIVE_INFINITY if there are no elements.
doublesGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<Double> and returns a PCollection<Double> whose contents is the minimum of the input PCollection's elements, or Double.POSITIVE_INFINITY if there are no elements.
doublesGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<Double> and returns a PCollection<Double> whose contents is the sum of the input PCollection's elements, or 0 if there are no elements.
doublesPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<KV<K, Double>> and returns a PCollection<KV<K, Double>> that contains an output element mapping each distinct key in the input PCollection to the maximum of the values associated with that key in the input PCollection.
doublesPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<KV<K, Double>> and returns a PCollection<KV<K, Double>> that contains an output element mapping each distinct key in the input PCollection to the minimum of the values associated with that key in the input PCollection.
doublesPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<KV<K, Double>> and returns a PCollection<KV<K, Double>> that contains an output element mapping each distinct key in the input PCollection to the sum of the values associated with that key in the input PCollection.

E

element() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the input element to be processed.
element() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
Returns the input element to be processed.
element() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
element() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
Returns the current element.
elementCountAtLeast(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
Creates a trigger that fires when the pane contains at least countElems elements.
empty() - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns an empty CoGbkResult.
empty(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns an empty KeyedPCollectionTuple<K> on the given pipeline.
empty() - Static method in class com.google.cloud.dataflow.sdk.values.CodedTupleTagMap
Returns an empty CodedTupleTagMap.
empty(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns an empty PCollectionList that is part of the given Pipeline.
empty(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns an empty PCollectionTuple that is part of the given Pipeline.
empty() - Static method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns an empty TupleTagList.
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
encode(Integer, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
 
encode(Long, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
 
encode(byte[], OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
 
encode(T, OutputStream, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Encodes the given value of type T onto the given output stream in the given context.
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
encode(Double, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
 
encode(DatastoreV1.Entity, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
 
encode(Instant, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
 
encode(IterableT, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
 
encode(KV<K, V>, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
encode(Map<K, V>, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
 
encode(T, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
encode(String, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
 
encode(TableRow, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
 
encode(Integer, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
encode(Integer, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
 
encode(Long, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
encode(Void, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
 
encode(CoGbkResult, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
encode(GlobalWindow, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow.Coder
 
encode(PaneInfo, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
 
encode(TimestampedValue<T>, OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
encodeAndOwn(byte[], OutputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
Encodes the provided value with the identical encoding to ByteArrayCoder.encode(byte[], java.io.OutputStream, com.google.cloud.dataflow.sdk.coders.Coder.Context), but with optimizations that take ownership of the value.
end() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns the end of this window, exclusive.
ensureCoderSerializable(Coder<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
If the evaluation context is testing serializability, ensures that the argument Coder is serializable and deserializable by encoding it and then decoding it, and returning the result.
ensureElementEncodable(TypedPValue<T>, T) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Ensures that the element is encodable and decodable using the TypePValue's coder, by encoding it and decoding it, and returning the result.
ensureSerializable(FunctionT) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
If the evaluation context is testing serializability, ensures that the argument function is serializable and deserializable by encoding it and then decoding it, and returning the result.
ensureSerializableByCoder(Coder<T>, T, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
If the evaluation context is testing serializability, ensures that the given data is serializable and deserializable with the given Coder by encoding it and then decoding it, and returning the result.
ensureSerializableByCoder(Coder<T>, T, String) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
 
enterCompositeTransform(TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
Called for each composite transform after all topological predecessors have been visited but before any of the component transforms.
enterCompositeTransform(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
EntityCoder - Class in com.google.cloud.dataflow.sdk.coders
An EntityCoder is a Coder for DatastoreV1.Entity objects.
equals(Object) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.values.KV
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
 
equals(Object) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Two type descriptor are equal if and only if they represent the same type.
evaluate(TransformT, DirectPipelineRunner.EvaluationContext) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TransformEvaluator
 
eventTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
The event timestamp of the element being processed.
every(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
Returns a new SlidingWindows with the original size, that assigns timestamps into half-open intervals of the form [N * period, N * period + size), where 0 is the epoch.
ExecutorServiceFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcsOptions.ExecutorServiceFactory
 
expand() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Expands the component PCollections, stripping off any tag-specific information.
expand() - Method in class com.google.cloud.dataflow.sdk.values.PBegin
 
expand() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
expand() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
expand() - Method in class com.google.cloud.dataflow.sdk.values.PDone
 
expand() - Method in interface com.google.cloud.dataflow.sdk.values.PInput
Expands this PInput into a list of its component input PValues.
expand() - Method in interface com.google.cloud.dataflow.sdk.values.POutput
Expands this POutput into a list of its component output PValues.
expand() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
 
Experimental - Annotation Type in com.google.cloud.dataflow.sdk.annotations
Signifies that a public API (public class, method or field) is subject to incompatible changes, or even removal, in a future release.
Experimental.Kind - Enum in com.google.cloud.dataflow.sdk.annotations
An enumeration of various kinds of experimental APIs.
extension - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink
The extension to be used for the final output files.
extractOrderedList() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique
Returns the values in the heap, ordered largest to smallest.
extractOutput(ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
extractOutput() - Method in interface com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn.Accumulator
Returns the output value that is the result of combining all the input values represented by this accumulator.
extractOutput(AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
 
extractOutput(double[]) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
extractOutput(Combine.BinaryCombineFn.Holder<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
extractOutput(int[]) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
extractOutput(long[]) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
extractOutput(AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns the output value that is the result of combining all the input values represented by the given accumulator.
extractOutput(K, AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Returns the output value that is the result of combining all the input values represented by the given accumulator.
extractOutput(List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.SimpleCombineFn
 
extractOutput(Top.BoundedHeap<KV<Integer, T>, SerializableComparator<KV<Integer, T>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
extractOutput(List<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.Concatenate
 

F

fewKeys() - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
Returns whether it groups just few keys.
FileBasedReader(FileBasedSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
Subclasses should not perform IO operations at the constructor.
FileBasedSink<T> - Class in com.google.cloud.dataflow.sdk.io
Abstract Sink for file-based output.
FileBasedSink(String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink
Construct a FileBasedSink with the given base output filename and extension.
FileBasedSink(String, String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink
Construct a FileBasedSink with the given base output filename, extension, and file naming template.
FileBasedSink.FileBasedWriteOperation<T> - Class in com.google.cloud.dataflow.sdk.io
Abstract Sink.WriteOperation that manages the process of writing to a FileBasedSink.
FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention - Enum in com.google.cloud.dataflow.sdk.io
Options for handling of temporary output files.
FileBasedSink.FileBasedWriter<T> - Class in com.google.cloud.dataflow.sdk.io
Abstract Sink.Writer that writes a bundle to a FileBasedSink.
FileBasedSink.FileResult - Class in com.google.cloud.dataflow.sdk.io
Result of a single bundle write.
FileBasedSource<T> - Class in com.google.cloud.dataflow.sdk.io
A common base class for all file-based Sources.
FileBasedSource(String, long) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSource
Create a FileBaseSource based on a file or a file pattern specification.
FileBasedSource(String, long, long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSource
Create a FileBasedSource based on a single file.
FileBasedSource.FileBasedReader<T> - Class in com.google.cloud.dataflow.sdk.io
A reader that implements code common to readers of FileBasedSources.
FileBasedSource.Mode - Enum in com.google.cloud.dataflow.sdk.io
A given FileBasedSource represents a file resource of one of these types.
FileBasedWriteOperation(FileBasedSink<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Construct a FileBasedWriteOperation using the same base filename for both temporary and output files.
FileBasedWriteOperation(FileBasedSink<T>, String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Construct a FileBasedWriteOperation.
FileBasedWriteOperation(FileBasedSink<T>, String, FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Create a new FileBasedWriteOperation.
FileBasedWriter(FileBasedSink.FileBasedWriteOperation<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Construct a new FileBasedWriter with a base filename.
fileNamingTemplate - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink
Naming template for output files.
FileResult(String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileResult
 
Filter<T> - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for filtering from a PCollection the elements satisfying a predicate, or satisfying an inequality with a given value based on the elements' natural ordering.
Filter() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Filter
 
finalize(Iterable<FileBasedSink.FileResult>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Finalizes writing by copying temporary output files to their final location and optionally removing temporary files.
finalize(Iterable<WriteT>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Given an Iterable of results from bundle writes, performs finalization after writing and closes the sink.
finalizeCheckpoint() - Method in interface com.google.cloud.dataflow.sdk.io.UnboundedSource.CheckpointMark
Perform any finalization that needs to happen after a bundle of data read from the source has been processed and committed.
finishBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Finishes processing this batch of elements.
finishBundle() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
finishBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
finishedInAnyMergingWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
Return true if the trigger is finished in any window being merged.
finishedSets - Variable in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
 
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Finish specifying a transform.
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
 
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PBegin
 
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
finishSpecifying() - Method in interface com.google.cloud.dataflow.sdk.values.PInput
After building, finalizes this PInput to make it ready for being used as an input to a PTransform.
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
 
finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
After building, finalizes this PValue to make it ready for running.
finishSpecifyingOutput() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
finishSpecifyingOutput() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
finishSpecifyingOutput() - Method in interface com.google.cloud.dataflow.sdk.values.POutput
As part of applying the producing PTransform, finalizes this output to make it ready for being used as an input and for running.
finishSpecifyingOutput() - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
Default behavior for finishSpecifyingOutput() is to do nothing.
firstUnfinishedSubTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Returns the first unfinished sub-trigger.
fixedSizeGlobally(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample
Returns a PTransform that takes a PCollection<T>, selects sampleSize elements, uniformly at random, and returns a PCollection<Iterable<T>> containing the selected elements.
fixedSizePerKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample
Returns a PTransform that takes an input PCollection<KV<K, V>> and returns a PCollection<KV<K, Iterable<V>>> that contains an output element mapping each distinct key in the input PCollection to a sample of sampleSize values associated with that key in the input PCollection, taken uniformly at random.
FixedWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows values into fixed-size timestamp-based windows.
Flatten - Class in com.google.cloud.dataflow.sdk.transforms
Flatten<T> takes multiple PCollection<T>s bundled into a PCollectionList<T> and returns a single PCollection<T> containing all the elements in all the input PCollections.
Flatten() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Flatten
 
Flatten.FlattenIterables<T> - Class in com.google.cloud.dataflow.sdk.transforms
FlattenIterables<T> takes a PCollection<Iterable<T>> and returns a PCollection<T> that contains all the elements from each iterable.
Flatten.FlattenPCollectionList<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.
FlattenIterables() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenIterables
 
forClass(Class<?>, DataflowWorkerLoggingOptions.Level) - Static method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverride
Overrides the default log level for the passed in class.
forCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderFactories
Creates a CoderFactory that always returns the given coder.
forever(Trigger<W>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
Create a composite trigger that repeatedly executes the trigger toRepeat, firing each time it fires and ignoring any indications to finish.
forName(String, DataflowWorkerLoggingOptions.Level) - Static method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverride
Overrides the default log level for the passed in name.
forPackage(Package, DataflowWorkerLoggingOptions.Level) - Static method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverride
Overrides the default log level for the passed in package.
forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
Create an OnElementContext for executing the given trigger.
forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
Create an OnMergeContext for executing the given trigger.
forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnTimerContext
Create an OnTimerContext for executing the given trigger.
forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Create a TriggerContext for executing the given trigger.
from(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new AvroIO.Read PTransform that's like this one but that reads from the file(s) with the given name or pattern.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns an AvroIO.Read PTransform that reads from the file(s) with the given name or pattern.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroSource
Creates an AvroSource that reads from the given file name or pattern ("glob").
from(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Sets the table specification.
from(TableReference) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Sets the table specification.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.
from(TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
Reads a BigQuery table specified as a TableReference object.
from(FileBasedSource<T>) - Static method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Creates a CompressedSource from an underlying FileBasedSource that must be further configured with CompressedSource.withDecompression(com.google.cloud.dataflow.sdk.io.CompressedSource.DecompressingChannelFactory).
from(Source<T>) - Method in class com.google.cloud.dataflow.sdk.io.Read.Bound
Returns a new Read PTransform that's like this one but reads from the given Source.
from(BoundedSource<T>) - Static method in class com.google.cloud.dataflow.sdk.io.Read
Returns a new Read.Bound PTransform reading from the given BoundedSource.
from(UnboundedSource<T, ?>) - Static method in class com.google.cloud.dataflow.sdk.io.Read
Returns a new Read.Bound PTransform reading from the given UnboundedSource.
from(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new TextIO.Read PTransform that's like this one but that reads from the file(s) with the given name or pattern.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a TextIO.Read PTransform that reads from the file(s) with the given name or pattern.
from(String) - Static method in class com.google.cloud.dataflow.sdk.io.XmlSource
Creates an XmlSource for a single XML file or a set of XML files defined by a Java "glob" file pattern.
fromArgs(String[]) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
Sets the command line arguments to parse when constructing the PipelineOptions.
fromArgs(String[]) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Sets the command line arguments to parse when constructing the PipelineOptions.
fromIterableInternal(Iterable<WindowedValue<?>>) - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
For internal use only.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
Constructs a runner from the provided options.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Construct a runner from the provided options.
fromOptions(DataflowPipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
Constructs a translator from the provided options.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Constructs a DirectPipelineRunner from the given options.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner
Constructs a runner from the provided options.
fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
Constructs a runner from the provided options.
fromPath(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
 
fromPath(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
 
fromQuery(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Sets the BigQuery query to be used.
fromQuery(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
Reads results received after executing the given query.
fromStaticMethods(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderFactories
Creates a CoderFactory built from particular static methods of a class that implements Coder.

G

GcpOptions - Interface in com.google.cloud.dataflow.sdk.options
Options used to configure Google Cloud Platform project and credentials.
GcpOptions.CredentialDirFactory - Class in com.google.cloud.dataflow.sdk.options
Returns the default credential directory of ${user.home}/.store/data-flow.
GcpOptions.GcpUserCredentialsFactory - Class in com.google.cloud.dataflow.sdk.options
Attempts to load the GCP credentials.
GcpUserCredentialsFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcpOptions.GcpUserCredentialsFactory
 
GcsOptions - Interface in com.google.cloud.dataflow.sdk.options
Options used to configure Google Cloud Storage.
GcsOptions.ExecutorServiceFactory - Class in com.google.cloud.dataflow.sdk.options
Returns the default ExecutorService to use within the Dataflow SDK.
generateDestinationFilenames(int) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Generate output bundle filenames.
generateInitialSplits(int, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
Returns a list of UnboundedSource objects representing the instances of this source that should be used when executing the workflow.
get(CodedTupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.values.CodedTupleTagMap
Returns the value associated with the given tag in this CodedTupleTagMap, or null if the tag has no asssociated value.
get(int) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns the PCollection at the given index (origin zero).
get(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns the PCollection associated with the given TupleTag in this PCollectionTuple.
get(int) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns the TupleTag at the given index (origin zero).
getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
 
getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
getAccumulatorCoder(CoderRegistry, Coder<Double>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
getAccumulatorCoder(CoderRegistry, Coder<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
getAccumulatorCoder(CoderRegistry, Coder<Integer>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
getAccumulatorCoder(CoderRegistry, Coder<Long>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
getAccumulatorCoder(CoderRegistry, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns the Coder to use for accumulator AccumT values, or null if it is not able to be inferred.
getAccumulatorCoder(CoderRegistry, PCollection<? extends KV<K, ? extends Iterable<InputT>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
 
getAccumulatorCoder(CoderRegistry, Coder<K>, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Returns the Coder to use for accumulator AccumT values, or null if it is not able to be inferred.
getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
 
getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.Concatenate
 
getAddCounterMutator() - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Returns a mutator, which can be used to add additional counters to this EvaluationContext.
getAggregators(DoFn<?, ?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.AggregatorRetriever
Returns the Aggregators created by the provided DoFn.
getAggregatorSteps() - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorPipelineExtractor
Returns a Map mapping each Aggregator in the Pipeline to the PTransforms in which it is used.
getAggregatorValues(Aggregator<?, T>) - Method in interface com.google.cloud.dataflow.sdk.PipelineResult
Retrieves the current value of the provided Aggregator.
getAggregatorValues(Aggregator<?, OutputT>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
 
getAlgorithm() - Method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
Returns the string representation of this type.
getAll(TupleTag<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns the values from the table represented by the given TupleTag<V> as an Iterable<V> (which may be empty if there are no results).
getAll() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns an immutable List of all the PCollections in this PCollectionList.
getAll() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns an immutable Map from TupleTag to corresponding PCollection, for all the members of this PCollectionTuple.
getAll() - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns an immutable List of all the TupleTags in this TupleTagList.
getAllowedTimestampSkew() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Returns the allowed timestamp skew duration, which is the maximum duration that timestamps can be shifted backward in DoFn.Context.outputWithTimestamp(OutputT, org.joda.time.Instant).
getAllowedTimestampSkew() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns the allowed timestamp skew duration, which is the maximum duration that timestamps can be shifted backward in DoFnWithContext.Context.outputWithTimestamp(OutputT, org.joda.time.Instant).
getApiRootUrl() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The root URL for the Dataflow API.
getApiServiceName() - Method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.ClusterManagerApiType
 
getAppName() - Method in interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions
Name of application, for display purposes.
getArgumentTypes(Method) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a list of argument types for the given method, which must be a part of the class.
getAutoscalingAlgorithm() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
getCause() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
Returns the reason that this WindowFn is invalid.
getChannelFactory() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
 
getCheckpointMark() - Method in interface com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns a UnboundedSource.CheckpointMark representing the progress of this UnboundedReader.
getCheckpointMarkCoder() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
Returns a Coder for encoding and decoding the checkpoints for this source, or null if the checkpoints do not need to be durably committed.
getClasses() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a set of TypeDescriptors, one for each superclass (including this class).
getClusterManagerApi() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Type of API for handling cluster management, i.e.
getCoder(TypeDescriptor<T>) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderProvider
Provides a coder for a given class, if possible.
getCoder(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
getCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getCoder() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getCoder() - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Gets the coder, which may yet be absent.
getCoder() - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Gets the coder, which may yet be absent.
getCoder() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns a Coder suitable for encoding IntervalWindows.
getCoder() - Method in class com.google.cloud.dataflow.sdk.values.CodedTupleTag
Returns the Coder used for values associated with this tag.
getCoder() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Returns the Coder used by this PCollection to encode and decode the values stored in it.
getCoder() - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
Returns the Coder used by this TypedPValue to encode and decode the values stored in it.
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.AtomicCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
getCoderArguments() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
If this is a Coder for a parameterized type, returns the list of Coders being used for each of the parameters, or returns null if this cannot be done or this is not a parameterized type.
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
getCoderInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
For internal use only.
getCoderRegistry() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Returns the CoderRegistry that this Pipeline uses.
getCoGbkResultSchema() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns the CoGbkResultSchema associated with this KeyedPCollectionTuple.
getCollection() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple.TaggedKeyedPCollection
Returns the underlying PCollection of this TaggedKeyedPCollection.
getCombineFn() - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator
Returns the Combine.CombineFn, which combines input elements in the aggregator.
getComponents() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
Returns the list of Coders that are components of this Coder.
getComponents() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
getComponentType() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns the component type if this type is an array type, otherwise returns null.
getCompressionType() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
getContinuationTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Return a trigger to use after a GroupByKey to preserve the intention of this trigger.
getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Return the Trigger.getContinuationTrigger() of this Trigger.
getContinuationTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxDoubleFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxIntegerFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxLongFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinDoubleFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinIntegerFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinLongFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
 
getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
 
getCreateDisposition() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns the create disposition.
getCredentialDir() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
Directory for storing dataflow credentials after execution of the OAuth 2 webflow.
getCredentialFactoryClass() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
The class of the credential factory that should be created and used to create credentials.
getCredentialId() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
This option controls which credential store to use when creating the credentials using the OAuth 2 webflow.
getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
 
getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Gets the current record from the delegate reader.
getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
getCurrent() - Method in interface com.google.cloud.dataflow.sdk.io.Source.Reader
Returns the value of the data item that was read by the last Source.Reader.start() or Source.Reader.advance() call.
getCurrent() - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Returns the last TransformTreeNode on the stack.
getCurrentBlock() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
getCurrentBlock() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Returns the current block (the block that was read by the previous call to BlockBasedSource.BlockBasedReader.readNextBlock()).
getCurrentBlockOffset() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
getCurrentBlockOffset() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Returns the largest offset such that starting to read from that offset includes the current block.
getCurrentBlockSize() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
getCurrentBlockSize() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Returns the size of the current block in bytes as it is represented in the underlying file, if possible.
getCurrentOffset() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
 
getCurrentOffset() - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource.ByteOffsetBasedReader
Returns the starting offset of the current record, which has been read by the last successful Source.Reader.start() or Source.Reader.advance() call.
getCurrentOffset() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Returns the delegate reader's current offset in the decompressed input.
getCurrentRecord() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
Returns the current record.
getCurrentRecordId() - Method in interface com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns a unique identifier for the current record.
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
getCurrentSource() - Method in interface com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
 
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource.ByteOffsetBasedReader
 
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
 
getCurrentSource() - Method in interface com.google.cloud.dataflow.sdk.io.Source.Reader
Returns a Source describing the same input that this Reader reads (including items already read).
getCurrentSource() - Method in interface com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns the UnboundedSource that created this reader.
getCurrentTimestamp() - Method in class com.google.cloud.dataflow.sdk.io.Source.AbstractReader
 
getCurrentTimestamp() - Method in interface com.google.cloud.dataflow.sdk.io.Source.Reader
Returns the timestamp associated with the current data item.
getDataflowClient() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
An instance of the Dataflow client.
getDataflowClient() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
 
getDataflowEndpoint() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Dataflow endpoint to use.
getDataflowJobFile() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The path to write the translated Dataflow job specification out to at job submission time.
getDayOfMonth() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
getDayOfMonth() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getDefaultCoder(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use by default for values of the given type.
getDefaultCoder(TypeDescriptor<OutputT>, TypeDescriptor<InputT>, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use by default for values of the given type, where the given context type uses the given context coder.
getDefaultCoder(Class<? extends T>, Class<T>, Map<Type, ? extends Coder<?>>, TypeVariable<?>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use for the specified type parameter specialization of the subclass, given Coders to use for all other type parameters (if any).
getDefaultCoder(T) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use for the provided example value, if it can be determined.
getDefaultCoder(Class<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use by default for values of the given class.
getDefaultCoders(Class<? extends T>, Class<T>, Map<Type, ? extends Coder<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns a Map from each of baseClass's type parameters to the Coder to use by default for it, in the context of subClass's specialization of baseClass.
getDefaultOutputCoder(SerializableFunction<InputT, OutputT>, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Returns the Coder to use on elements produced by this function, given the coder used for its input elements.
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Returns the delegate source's default output coder.
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.Read.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.Source
Returns the default Coder to use for the data read from this source.
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
getDefaultOutputCoder(CoderRegistry, Coder<Double>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
getDefaultOutputCoder(CoderRegistry, Coder<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
getDefaultOutputCoder(CoderRegistry, Coder<Integer>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
getDefaultOutputCoder(CoderRegistry, Coder<Long>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
getDefaultOutputCoder(CoderRegistry, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns the Coder to use by default for output OutputT values, or null if it is not able to be inferred.
getDefaultOutputCoder(PCollection<? extends KV<K, ? extends Iterable<InputT>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
 
getDefaultOutputCoder(CoderRegistry, Coder<K>, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Returns the Coder to use by default for output OutputT values, or null if it is not able to be inferred.
getDefaultOutputCoder(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
 
getDefaultOutputCoder(PCollectionList<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenPCollectionList
 
getDefaultOutputCoder(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
 
getDefaultOutputCoder(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
 
getDefaultOutputCoder(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getDefaultOutputCoder(PCollection<? extends InputT>, TypedPValue<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the default Coder to use for the output of this single-output PTransform.
getDefaultOutputCoder(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the default Coder to use for the output of this single-output PTransform when applied to the given input.
getDefaultOutputCoder(InputT, TypedPValue<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the default Coder to use for the given output of this single-output PTransform when applied to the given input.
getDefaultOutputCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
getDefaultOutputCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.Concatenate
 
getDefaultOutputCoder(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
 
getDefaultWorkerLogLevel() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
This option controls the default log level of all loggers without a log level override.
getDestinationIndex() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnTimerContext
 
getDiskSizeGb() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Remote worker disk size, in gigabytes, or 0 to use the default size.
getDiskSourceImage() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Disk source image to use by VMs for jobs.
getDoFnClass(DoFn<?, ?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
 
getElemCoder() - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
getElements() - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
 
getEnableCloudDebugger() - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
Whether to enable the Cloud Debugger snapshot agent for the current job.
getEnclosingNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the enclosing composite transform node, or null if there is none.
getEncodedElementByteSize(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
 
getEncodedElementByteSize(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
 
getEncodedElementByteSize(byte[], Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
 
getEncodedElementByteSize(Double, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
 
getEncodedElementByteSize(DatastoreV1.Entity, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
 
getEncodedElementByteSize(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
Returns the size in bytes of the encoded value using this coder.
getEncodedElementByteSize(String, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
 
getEncodedElementByteSize(TableRow, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
 
getEncodedElementByteSize(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
getEncodedElementByteSize(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
 
getEncodedElementByteSize(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
getEncodedElementByteSize(Void, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
 
getEndOffset() - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource
Returns the specified ending offset of the source.
getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
An estimate of the total size (in bytes) of the data that would be read from this source.
getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
getExecutorService() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
The ExecutorService instance to use to create threads, can be overridden to specify an ExecutorService that is compatible with the users environment.
getExpandedOutputs() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the transform outputs, in expanded form.
getExperiments() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The list of backend experiments to enable.
getFallbackCoderProvider() - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
 
getFilename() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileResult
 
getFilenamePrefix() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getFilenamePrefix() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getFilenameSuffix() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getFilenameSuffix() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getFileOrPatternSpec() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
getFilepattern() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
getFilepattern() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
getFilesToStage() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
List of local files to make available to workers.
getFinishedMergingWindows() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
Return the merging windows in which the trigger is finished.
getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
Returns the KeyedCombineFn used by this Combine operation.
getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
Returns the KeyedCombineFn used by this Combine operation.
getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
 
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.AbstractBoundedReader
 
getFractionConsumed() - Method in interface com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
Returns a value in [0, 1] representing approximately what fraction of the source (BoundedSource.BoundedReader.getCurrentSource()) this reader has read so far.
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource.ByteOffsetBasedReader
 
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
getFractionConsumed() - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Returns the approximate fraction of positions in the source that have been consumed by successful RangeTracker.tryReturnRecordAt(boolean, PositionT) calls, or 0.0 if no such calls have happened.
getFractionOfBlockConsumed() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
Returns the fraction of the block already consumed (i.e., not including the current record), if possible, as a value in [0, 1].
getFullName(PTransform<?, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Returns the full name of the currently being translated transform.
getFullName() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
 
getFullName() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
getFullNameForTesting(PTransform<?, ?>) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Deprecated.
getGapDuration() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
getGcpCredential() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
The credential instance that should be used to authenticate against GCP services.
getGcsEndpoint() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
GCS endpoint to use.
getGcsUtil() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
The GcsUtil instance that should be used to communicate with Google Cloud Storage.
getGoogleApiTrace() - Method in interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions
This option enables tracing of API calls to Google services used within the Dataflow SDK.
getId() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
Returns the id of this TupleTag.
getIdLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getIdLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getIndex(TupleTag<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Returns the index for the given tuple tag, if the tag is present in this schema, -1 if it isn't.
getInput(PTransform<InputT, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Returns the input of the currently being translated transform.
getInput(PTransform<InputT, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Returns the input of the currently being processed transform.
getInput() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the transform input, in unexpanded form.
getInput() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
getInputs() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns a mapping of inputs to the producing nodes for all inputs to the transform.
getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Returns a TypeDescriptor capturing what is known statically about the input type of this DoFn instance's most-derived class.
getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns a TypeDescriptor capturing what is known statically about the input type of this DoFnWithContext instance's most-derived class.
getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
getInstanceComponents(T) - Static method in class com.google.cloud.dataflow.sdk.coders.AtomicCoder
Returns a list of values contained in the provided example value, one per type parameter.
getInstanceComponents(Object) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderFactory
Returns a list of objects contained in value, one per type argument, or null if none can be determined.
getInstanceComponents(Collection<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
Returns the first element in this collection if it is non-empty, otherwise returns null.
getInstanceComponents(Iterable<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
Returns the first element in this iterable if it is non-empty, otherwise returns null.
getInstanceComponents(KV<K, V>) - Static method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
getInstanceComponents(List<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.ListCoder
Returns the first element in this list if it is non-empty, otherwise returns null.
getInstanceComponents(Map<K, V>) - Static method in class com.google.cloud.dataflow.sdk.coders.MapCoder
Returns the key and value for an arbitrary element of this map, if it is non-empty, otherwise returns null.
getInstanceComponents(Set<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.SetCoder
Returns the first element in this set if it is non-empty, otherwise returns null.
getInstanceComponents(TimestampedValue<T>) - Static method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
getInstanceComponentsHelper(IterableT) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
Returns the first element in this iterable-like if it is non-empty, otherwise returns null.
getInterfaces() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a set of TypeDescriptors, one for each interface implemented by this class.
getJAXBClass() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
 
getJob() - Method in exception com.google.cloud.dataflow.sdk.runners.AbstractJobException
Returns the failed job.
getJob() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.JobSpecification
 
getJobId() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
The identity of the Dataflow job.
getJobId() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
 
getJobMessageOutput() - Method in interface com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions
Output stream for job status messages.
getJobName() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
The Dataflow job name is used as an idempotence key within the Dataflow service.
getKey() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the key associated with this element.
getKey() - Method in class com.google.cloud.dataflow.sdk.values.KV
Returns the key of this KV.
getKeyCoder() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
getKeyCoder() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
getKeyCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns the key Coder for all PCollections in this KeyedPCollectionTuple.
getKeyedCollections() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns a list of TaggedKeyedPCollections for the PCollections contained in this KeyedPCollectionTuple.
getKindString() - Method in class com.google.cloud.dataflow.sdk.io.Read.Bound
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the name to use by default for this PTransform (not including the names of any enclosing PTransforms).
getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
 
getKindString() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Returns a String capturing the kind of this PValueBase.
getLevel() - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverride
 
getMainOutputTag() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getMaxEndOffset(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource
Returns the exact ending offset of the current source.
getMaxEndOffset(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
getMaxNumRecords() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getMaxNumWorkers() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
The maximum number of workers to use when using workerpool autoscaling.
getMaxReadTime() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getMessage() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
getMinBundleSize() - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource
Returns the minimum bundle size that should be used when splitting the source into sub-sources.
getMode() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
getMonthOfYear() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getName() - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverride
 
getName() - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator
Returns the name of the Aggregator.
getName() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Returns the transform name.
getName() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Returns the name of this PCollection.
getName() - Method in interface com.google.cloud.dataflow.sdk.values.PValue
Returns the name of this PValue.
getName() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Returns the name of this PValueBase.
getNetwork() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
GCE network for launching workers.
getNumber() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
getNumber() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
getNumber() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getNumShards() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getNumShards() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getNumWorkers() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Number of workers to use when executing the Dataflow job.
getOffset() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
getOffset() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
getOnly(TupleTag<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
If there is a singleton value for the given tag, returns it.
getOnly(TupleTag<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
If there is a singleton value for the given tag, returns it.
getOptions() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Returns the configured pipeline options.
getOriginalWindowFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
Returns the original windowFn that this InvalidWindows replaced.
getOutName(int) - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
If this TupleTag is tagging output outputIndex of a PTransform, returns the name that should be used by default for the output.
getOutput(PTransform<?, OutputT>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Returns the output of the currently being translated transform.
getOutput(PTransform<?, OutputT>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Returns the output of the currently being processed transform.
getOutput() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the transform output, in unexpanded form.
getOutput() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
getOutputType() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns a TypeDescriptor capturing what is known statically about the output type of this CombineFn instance's most-derived class.
getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Returns a TypeDescriptor capturing what is known statically about the output type of this DoFn instance's most-derived class.
getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
Returns a TypeDescriptor capturing what is known statically about the output type of this DoFnWithContext instance's most-derived class.
getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
getOverriddenTransform() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
 
getOverrideWindmillBinary() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Custom windmill_main binary to use with the streaming runner.
getPathValidator() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The path validator instance that should be created and used to validate paths.
getPathValidatorClass() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The class of the validator that should be created and used to validate paths.
getPCollection(PCollection<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
Retrieves the value of the given PCollection.
getPCollectionList(PCollectionList<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
Retrieves the values of each PCollection in the given PCollectionList.
getPCollectionValuesWithMetadata(PCollection<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Retrieves the value of the given PCollection, along with element metadata such as timestamps and windows.
getPCollectionView(PCollectionView<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
Retrieves the values indicated by the given PCollectionView.
getPCollectionWindowedValues(PCollection<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
Retrieves the windowed value of the given PCollection.
getPeriod() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
getPipeline() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
 
getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.PBegin
 
getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
getPipeline() - Method in interface com.google.cloud.dataflow.sdk.values.PInput
Returns the owning Pipeline of this PInput.
getPipeline() - Method in interface com.google.cloud.dataflow.sdk.values.POutput
Returns the owning Pipeline of this POutput.
getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
 
getPipelineOptions() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptionsRegistrar
 
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Options
 
getPipelineOptions() - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
Returns the configured pipeline options.
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Options
 
getPipelineOptions() - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Returns the configured pipeline options.
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
 
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Returns the PipelineOptions specified with the PipelineRunner invoking this DoFn.
getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Returns the PipelineOptions specified with the PipelineRunner invoking this DoFnWithContext.
getPipelineRunners() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Runner
 
getPipelineRunners() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Runner
 
getPipelineRunners() - Method in interface com.google.cloud.dataflow.sdk.runners.PipelineRunnerRegistrar
 
getPositionForFractionConsumed(double) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
Returns a position P such that the range [start, P) represents approximately the given fraction of the range [start, end).
getProducingTransformInternal() - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
Returns the AppliedPTransform that this POutputValueBase is an output of.
getProducingTransformInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PValue
Returns the AppliedPTransform that this POutputValueBase is an output of.
getProject() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
getProject() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
Project id to use when launching jobs.
getProjectId() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
 
getPubsubRootUrl() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Root URL for use with the Pubsub API.
getQuery() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
 
getRawType() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns the Class underlying the Type represented by this TypeDescriptor.
getReasons() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
getRecordClass() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
getRecordElement() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
getRecordType() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
getRegisteredOptions() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
 
getReload() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Whether to reload the currently running pipeline with the same name as this one.
getRootCause() - Method in exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
Returns the inner-most CannotProvideCoderException when they are deeply nested.
getRootElement() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
getRunner() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
 
getRunner() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Returns the configured pipeline runner.
getRunner() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
 
getRunner() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
 
getSchema() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns the schema used by this coder.
getSchema() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
getSchema() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getSchema() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
getSchema() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns the table schema.
getSchema() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns the schema used by this CoGbkResult.
getSecretsFile() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
This option controls which file to use when attempting to create the credentials using the OAuth 2 webflow.
getServiceAccountKeyfile() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
This option controls which file to use when attempting to create the credentials using the service account method.
getServiceAccountName() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
This option controls which service account to use when attempting to create the credentials using the service account method.
getShardNameTemplate() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns the current shard name template string.
getShardNameTemplate() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns the current shard name template string.
getShardTemplate() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getShardTemplate() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
 
getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
Return the earliest window that contains the end of the main-input window.
getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns the window of the side input corresponding to the given window of the main input.
getSink() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Returns the FileBasedSink for this write operation.
getSink() - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Returns the Sink that this write operation writes to.
getSink() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriteOperation
Return the XmlSink.Bound for this write operation.
getSize() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
getSize() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
getSource() - Method in class com.google.cloud.dataflow.sdk.io.Read.Bound
Returns the Source used to create this Read PTransform.
getStableUniqueNames() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
 
getStager() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The resource stager instance that should be created and used to stage resources.
getStagerClass() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
The class responsible for staging resources to be accessible by workers during job execution.
getStagingLocation() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
GCS path for staging local files, e.g.
getStartDate() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
getStartDate() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
getStartDate() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getStartOffset() - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource
Returns the starting offset of the source.
getStartPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
getStartPosition() - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Returns the starting position of the current range, inclusive.
getState() - Method in interface com.google.cloud.dataflow.sdk.PipelineResult
Retrieves the current state of the pipeline execution.
getState() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
 
getStepName(PTransform<?, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Gets the step name for this transform.
getStepNames() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.JobSpecification
Returns the mapping of AppliedPTransforms to the internal step name for that AppliedPTransform.
getStopPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
getStopPosition() - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Returns the ending position of the current range, exclusive.
getSubscription() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getSupertype(Class<? super T>) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns the generic form of a supertype.
getTable() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns the table to write.
getTable() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns the table reference.
getTag(int) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Returns the JoinTupleTag at the given index.
getTagInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
A unique identifier, for internal use.
getTeardownPolicy() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
The teardown policy for the VMs.
getTeardownPolicyName() - Method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.TeardownPolicy
 
getTempDatasetId() - Method in interface com.google.cloud.dataflow.sdk.options.BigQueryOptions
 
getTempLocation() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
GCS path for temporary files, e.g.
getTimestamp() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the timestamp associated with this element.
getTimestamp() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
getTimestampLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getTimestampLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getTimeZone() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
getTimeZone() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
getTimeZone() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
getTopic() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
 
getTopic() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
 
getTotalValue(Combine.CombineFn<T, ?, T>) - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorValues
Get the total value of this Aggregator by applying the specified Combine.CombineFn.
getTransform() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns the transform associated with this transform node.
getTransform() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
getTransformEvaluator(Class<TransformT>) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Returns the TransformEvaluator to use for instances of the specified PTransform class, or null if none registered.
getTransformNameMapping() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
Mapping of old PTranform names to new ones, specified as a semicolon-separated list of oldName=newName pairs.
getTransformTranslator(Class<TransformT>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
Returns the DataflowPipelineTranslator.TransformTranslator to use for instances of the specified PTransform class, or null if none registered.
getTranslator() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Returns the DataflowPipelineTranslator associated with this object.
getTriggerIdx() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerId
 
getTriggerResult() - Method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergeResult
 
getTupleTag() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple.TaggedKeyedPCollection
Returns the TupleTag of this TaggedKeyedPCollection.
getTupleTagList() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Returns the TupleTagList tuple associated with this schema.
getType() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
getType() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns the Type represented by this TypeDescriptor.
getTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
Returns a TypeDescriptor capturing what is known statically about the type of this TupleTag instance's most-derived class.
getTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
Returns a TypeDescriptor<T> with some reflective information about T, if possible.
getTypeParameter(String) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a TypeVariable for the named type parameter.
getUnionTag() - Method in class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
 
getValidate() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Returns true if table validation is enabled.
getValidate() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns true if table validation is enabled.
getValue() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the value associated with this element.
getValue() - Method in class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
 
getValue() - Method in class com.google.cloud.dataflow.sdk.values.KV
Returns the value of this KV.
getValue() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
getValueCoder() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
getValueCoder() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
getValues() - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorValues
Get the values of the Aggregator at all steps it was used.
getValuesAtSteps() - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorValues
Get the values of the Aggregator by the user name at each step it was used.
getWatermark() - Method in interface com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Returns a lower bound on timestamps of future elements read by this reader.
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Returns a bound in watermark time by which this trigger would have fired at least once for a given window had there been input data.
getWindowedValue() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the WindowedValue associated with this element.
getWindowingStrategy() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
 
getWindowingStrategy() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Returns the WindowingStrategy of this PCollection.
getWindowingStrategyInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
For internal use only.
getWindows() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns the collection of windows this element has been placed into.
getWorkerCacheMb() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
The size of the worker's in-memory cache, in megabytes.
getWorkerDiskType() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Specifies what type of persistent disk should be used.
getWorkerId() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
The identity of the worker running this pipeline.
getWorkerLogLevelOverrides() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
This option controls the log levels for specifically named loggers.
getWorkerMachineType() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Machine type to create Dataflow worker VMs as.
getWorkerPoolType() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
Specifies what type of worker pool should be used.
getWriteDisposition() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Returns the write disposition.
getWriteOperation() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Return the FileBasedWriteOperation that this Writer belongs to.
getWriteOperation() - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
Returns the write operation this writer belongs to.
getWriteOperation() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Return the XmlWriteOperation this write belongs to.
getWriterResultCoder() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Provides a coder for FileBasedSink.FileResult.
getWriterResultCoder() - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Returns a coder for the writer result type.
getZone() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
GCE availability zone for launching workers.
globally(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
Returns a PTransform that takes a PCollection<T> and returns a PCollection<List<T>> whose single value is a List of the approximate N-tiles of the elements of the input PCollection.
globally(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
Like ApproximateQuantiles.globally(int, Comparator), but sorts using the elements' natural ordering.
globally(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
Returns a PTransform that takes a PCollection<T> and returns a PCollection<Long> containing a single value that is an estimate of the number of distinct elements in the input PCollection.
globally(double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
Like ApproximateUnique.globally(int), but specifies the desired maximum estimation error instead of the sample size.
globally(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.Globally PTransform that uses the given SerializableFunction to combine all the elements in each window of the input PCollection into a single value in the output PCollection.
globally(Combine.CombineFn<? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.Globally PTransform that uses the given SerializableFunction to combine all the elements in each window of the input PCollection into a single value in the output PCollection.
globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Count
Returns a Count.Globally PTransform that counts the number of elements in its input PCollection.
Globally() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Count.Globally
 
globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Mean
Returns a PTransform that takes an input PCollection<NumT> and returns a PCollection<Double> whose contents is the mean of the input PCollection's elements, or 0 if there are no elements.
GlobalWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing
The default window into which all data is placed (via GlobalWindows).
GlobalWindow.Coder - Class in com.google.cloud.dataflow.sdk.transforms.windowing
GlobalWindow.Coder for encoding and decoding Windows.
GlobalWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Default WindowFn where all data is in the same window.
GlobalWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
GoogleApiDebugOptions - Interface in com.google.cloud.dataflow.sdk.options
These options configure debug settings for Google API clients created within the Dataflow SDK.
GoogleApiDebugOptions.GoogleApiTracer - Class in com.google.cloud.dataflow.sdk.options
A GoogleClientRequestInitializer that adds the trace destination to Google API calls.
greaterThan(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that are greater than a given value, based on the elements' natural ordering.
greaterThanEq(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that are greater than or equal to a given value, based on the elements' natural ordering.
GroupAlsoByWindow(WindowingStrategy<?, ?>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupAlsoByWindow
 
GroupByKey<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
GroupByKey<K, V> takes a PCollection<KV<K, V>>, groups the values by key and windows, and returns a PCollection<KV<K, Iterable<V>>> representing a map from each distinct key and window of the input PCollection to an Iterable over all the values associated with that key in the input.
GroupByKey.GroupAlsoByWindow<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Helper transform that takes a collection of timestamp-ordered values associated with each key, groups the values by window, combines windows as needed, and for each window in each key, outputs a collection of key/value-list pairs implicitly assigned to the window and with the timestamp derived from that window.
GroupByKey.GroupByKeyOnly<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Primitive helper transform that groups by key only, ignoring any window assignments.
GroupByKey.ReifyTimestampsAndWindows<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Helper transform that makes timestamps and window assignments explicit in the value part of each key/value pair.
GroupByKey.SortValuesByTimestamp<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
Helper transform that sorts the values associated with each key by timestamp.
GroupByKeyOnly() - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
 
groupedValues(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.GroupedValues PTransform that takes a PCollection of KVs where a key maps to an Iterable of values, e.g., the result of a GroupByKey, then uses the given SerializableFunction to combine all the values associated with a key, ignoring the key.
groupedValues(Combine.CombineFn<? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.GroupedValues PTransform that takes a PCollection of KVs where a key maps to an Iterable of values, e.g., the result of a GroupByKey, then uses the given CombineFn to combine all the values associated with a key, ignoring the key.
groupedValues(Combine.KeyedCombineFn<? super K, ? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.GroupedValues PTransform that takes a PCollection of KVs where a key maps to an Iterable of values, e.g., the result of a GroupByKey, then uses the given KeyedCombineFn to combine all the values associated with each key.

H

has(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns whether this PCollectionTuple contains a PCollection with the given tag.
hashCode() - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.values.KV
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
 
hashCode() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
 
Hidden - Annotation Type in com.google.cloud.dataflow.sdk.options
Methods and/or interfaces annotated with @Hidden will be suppressed from being output when --help is specified on the command-line.

I

identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
Returns the identity element of this operation, i.e.
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
Returns the value that should be used for the combine of the empty set.
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
Returns the identity element of this operation, i.e.
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
Returns the identity element of this operation, i.e.
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
 
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
 
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
 
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
 
identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
 
IDENTITY - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
 
idLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a new PubsubIO.Read PTransform that's like this one but reading unique ids from the given PubSub label.
idLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a PubSubIO.Read PTransform where unique record identifiers are expected to be provided using the PubSub labeling API.
idLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new PubsubIO.Write PTransform that's like this one but publishing record ids to the given PubSub label.
idLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
If specified, Dataflow will add a Pubsub label to each output record containing a unique identifier for that record.
in(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PBegin
Returns a PBegin in the given Pipeline.
in(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PDone
Creates a PDone in the given Pipeline.
INDEX_OF_MAX - Static variable in class com.google.cloud.dataflow.sdk.io.ShardNameTemplate
Shard name containing the index and max.
initialize(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Initialization of the sink.
initialize(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
Performs initialization before writing to the sink.
initialize(AbstractGoogleClientRequest<?>) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
 
inOrder(Trigger<W>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
INSTANCE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow.Coder
 
INSTANCE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow
 
INSTANCE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
 
InstantCoder - Class in com.google.cloud.dataflow.sdk.coders
A InstantCoder is a Coder for a joda Instant.
integersGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<Integer> and returns a PCollection<Integer> whose contents is the maximum of the input PCollection's elements, or Integer.MIN_VALUE if there are no elements.
integersGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<Integer> and returns a PCollection<Integer> whose contents is a single value that is the minimum of the input PCollection's elements, or Integer.MAX_VALUE if there are no elements.
integersGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<Integer> and returns a PCollection<Integer> whose contents is the sum of the input PCollection's elements, or 0 if there are no elements.
integersPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<KV<K, Integer>> and returns a PCollection<KV<K, Integer>> that contains an output element mapping each distinct key in the input PCollection to the maximum of the values associated with that key in the input PCollection.
integersPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<KV<K, Integer>> and returns a PCollection<KV<K, Integer>> that contains an output element mapping each distinct key in the input PCollection to the minimum of the values associated with that key in the input PCollection.
integersPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<KV<K, Integer>> and returns a PCollection<KV<K, Integer>> that contains an output element mapping each distinct key in the input PCollection to the sum of the values associated with that key in the input PCollection.
intersects(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns whether this window intersects the given window.
IntervalWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing
An implementation of BoundedWindow that represents an interval from IntervalWindow.start (inclusive) to IntervalWindow.end (exclusive).
IntervalWindow(Instant, Instant) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Creates a new IntervalWindow that represents the half-open time interval [start, end).
IntervalWindow(Instant, Duration) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
into(WindowFn<? super T, ?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Creates a Window PTransform that uses the given WindowFn to window the data.
into(WindowFn<? super T, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
Returns a new Window PTransform that's like this transform but that will use the given WindowFn, and that has its input and output types bound.
IntraBundleParallelization - Class in com.google.cloud.dataflow.sdk.transforms
Provides multi-threading of DoFns, using threaded execution to process multiple elements concurrently within a bundle.
IntraBundleParallelization() - Constructor for class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization
 
IntraBundleParallelization.Bound<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that, when applied to a PCollection<InputT>, invokes a user-specified DoFn<InputT, OutputT> on all its elements, with all its outputs collected into an output PCollection<OutputT>.
IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A multi-threaded DoFn wrapper.
IntraBundleParallelization.Unbound - Class in com.google.cloud.dataflow.sdk.transforms
An incomplete IntraBundleParallelization transform, with unbound input/output types.
InvalidWindows<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that represents an invalid pipeline state.
InvalidWindows(String, WindowFn<?, W>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
is(T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
isArray() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns true if this type is known to be an array type.
isAtSplitPoint() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Returns true if the reader is at a split point.
isAtSplitPoint() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Returns true only for the first record; compressed sources cannot be split.
isAtSplitPoint() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
Specifies if the current record of the reader is at a split point.
isBounded() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
 
isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
isCompatible(WindowFn) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
isCompatible(WindowFn) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
isCompatible(WindowFn) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
InvalidWindows objects with the same originalWindowFn are compatible.
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
 
isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Returns whether this performs the same triggering as the given Trigger.
isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns whether this performs the same merging as the given WindowFn.
isCompositeNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Returns true if this node represents a composite transform that does not perform processing of its own, but merely encapsulates a sub-pipeline (which may be empty).
isDefaultPane(PaneInfo) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
 
isDestination() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnTimerContext
Returns true if the given trigger index corresponds to the current trigger.
isDisjoint(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns whether this window is disjoint from the given window.
isEmpty() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
 
isEmpty() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
 
isEqualTo(T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Checks that the value of this SingletonAssert's view is equal to the expected value.
isFinish() - Method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergeResult
 
isFinish() - Method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerResult
 
isFinished() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Returns true if the current trigger is marked finished.
isFinishedSpecifyingInternal() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Returns whether this PValueBase has been finalized, and its core properties, e.g., name, can no longer be changed.
isFire() - Method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergeResult
 
isFire() - Method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerResult
 
isNonMerging() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.NonMergingWindowFn
 
isNonMerging() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns true if this WindowFn never needs to merge any windows.
isRegisterByteSizeObserverCheap(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
Returns true since registerByteSizeObserver() runs in constant time.
isRegisterByteSizeObserverCheap(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
Returns true since registerByteSizeObserver() runs in constant time.
isRegisterByteSizeObserverCheap(byte[], Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
Returns true since registerByteSizeObserver() runs in constant time.
isRegisterByteSizeObserverCheap(T, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Returns whether Coder.registerByteSizeObserver(T, com.google.cloud.dataflow.sdk.util.common.ElementByteSizeObserver, com.google.cloud.dataflow.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
isRegisterByteSizeObserverCheap(Double, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
Returns true since registerByteSizeObserver() runs in constant time.
isRegisterByteSizeObserverCheap(IterableT, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
Returns whether iterable can use lazy counting, since that requires minimal extra computation.
isRegisterByteSizeObserverCheap(KV<K, V>, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
Returns whether both keyCoder and valueCoder are considered not expensive.
isRegisterByteSizeObserverCheap(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
StandardCoder requires elements to be fully encoded and copied into a byte stream to determine the byte size of the element, which is considered expensive.
isRegisterByteSizeObserverCheap(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
Returns true since registerByteSizeObserver() runs in constant time.
isRegisterByteSizeObserverCheap(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
Returns true since registerByteSizeObserver() runs in constant time.
isRegisterByteSizeObserverCheap(Void, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
Returns true since registerByteSizeObserver() runs in constant time.
isRootNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
 
isSplittable() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Determines whether a single file represented by this source is splittable.
isSplittable() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
Determines whether a file represented by this source is can be split into bundles.
isStreaming() - Method in interface com.google.cloud.dataflow.sdk.options.StreamingOptions
 
isSubtypeOf(TypeDescriptor<?>) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Return true if this type is a subtype of the given type.
isSupertypeOf(TypeDescriptor<?>) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns true if this type is assignable from the given type.
isTerminal() - Method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
Returns if the job state can no longer complete work.
isWholeStream - Variable in class com.google.cloud.dataflow.sdk.coders.Coder.Context
Whether the encoded or decoded value fills the remainder of the output or input (resp.) record/stream contents.
IterableAssert(PTransform<PBegin, PCollectionView<Iterable<T>>>, Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
 
IterableCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
An IterableCoder encodes any Iterable as the sequence of its elements, encoded according to the component coder.
IterableCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
IterableLikeCoder<T,IterableT extends Iterable<T>> - Class in com.google.cloud.dataflow.sdk.coders
An abstract base class with functionality for assembling a Coder for a class that implements Iterable.
IterableLikeCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
 
iterables() - Static method in class com.google.cloud.dataflow.sdk.transforms.Flatten
Returns a PTransform that takes a PCollection<Iterable<T>> and returns a PCollection<T> containing all the elements from all the Iterables.

J

JAXBCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A coder for JAXB annotated objects.
JobExecutionException - Exception in com.google.cloud.dataflow.sdk.runners
Signals that a job run by a BlockingDataflowPipelineRunner fails during execution, and provides access to the failed job.
JobNameFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions.JobNameFactory
 
JobSpecification(Job, Map<AppliedPTransform<?, ?, ?>, String>) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.JobSpecification
 
jobToString(Job) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
 

K

KeyedCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
 
KeyedPCollectionTuple<K> - Class in com.google.cloud.dataflow.sdk.transforms.join
An immutable tuple of keyed PCollections with key type K.
KeyedPCollectionTuple.TaggedKeyedPCollection<K,V> - Class in com.google.cloud.dataflow.sdk.transforms.join
A utility class to help ensure coherence of tag and input PCollection types.
Keys<K> - Class in com.google.cloud.dataflow.sdk.transforms
Keys<K> takes a PCollection of KV<K, V>s and returns a PCollection<K> of the keys.
KV<K,V> - Class in com.google.cloud.dataflow.sdk.values
An immutable key/value pair.
KV.OrderByKey<K extends Comparable<? super K>,V> - Class in com.google.cloud.dataflow.sdk.values
Orders the KV by the key.
KV.OrderByValue<K,V extends Comparable<? super V>> - Class in com.google.cloud.dataflow.sdk.values
Orders the KV by the value.
KvCoder<K,V> - Class in com.google.cloud.dataflow.sdk.coders
A KvCoder encodes KVs.
KvCoderBase<T> - Class in com.google.cloud.dataflow.sdk.coders
A abstract base class for KvCoder.
KvCoderBase() - Constructor for class com.google.cloud.dataflow.sdk.coders.KvCoderBase
 
KvSwap<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
KvSwap<K, V> takes a PCollection<KV<K, V>> and returns a PCollection<KV<V, K>>, where all the keys and values have been swapped.

L

largest(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<List<T>> with a single element containing the largest count elements of the input PCollection<T>, in decreasing order, sorted according to their natural order.
Largest() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Top.Largest
 
largestPerKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<KV<K, V>> and returns a PCollection<KV<K, List<V>>> that contains an output element mapping each distinct key in the input PCollection to the largest count values associated with that key in the input PCollection<KV<K, V>>, in decreasing order, sorted according to their natural order.
LargestUnique(long) - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique
Creates a heap to track the largest sampleSize elements.
leaveCompositeTransform(TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
Called for each composite transform after all of its component transforms and their ouputs have been visited.
leaveCompositeTransform(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
lessThan(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that are less than a given value, based on the elements' natural ordering.
lessThanEq(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that are less than or equal to a given value, based on the elements' natural ordering.
ListCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A ListCoder encodes Lists.
longsGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<Long> and returns a PCollection<Long> whose contents is the maximum of the input PCollection's elements, or Long.MIN_VALUE if there are no elements.
longsGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<Long> and returns a PCollection<Long> whose contents is the minimum of the input PCollection's elements, or Long.MAX_VALUE if there are no elements.
longsGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<Long> and returns a PCollection<Long> whose contents is the sum of the input PCollection's elements, or 0 if there are no elements.
longsPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
Returns a PTransform that takes an input PCollection<KV<K, Long>> and returns a PCollection<KV<K, Long>> that contains an output element mapping each distinct key in the input PCollection to the maximum of the values associated with that key in the input PCollection.
longsPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
Returns a PTransform that takes an input PCollection<KV<K, Long>> and returns a PCollection<KV<K, Long>> that contains an output element mapping each distinct key in the input PCollection to the minimum of the values associated with that key in the input PCollection.
longsPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
Returns a PTransform that takes an input PCollection<KV<K, Long>> and returns a PCollection<KV<K, Long>> that contains an output element mapping each distinct key in the input PCollection to the sum of the values associated with that key in the input PCollection.
lookup(CodedTupleTag<T>, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Lookup the value stored for the given tag and window.
lookup(CodedTupleTag<T>, Iterable<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Lookup the value stored for a given tag in a bunch of windows.

M

MapCoder<K,V> - Class in com.google.cloud.dataflow.sdk.coders
A MapCoder encodes Maps.
MapCoderBase<T> - Class in com.google.cloud.dataflow.sdk.coders
A abstract base class for MapCoder.
MapCoderBase() - Constructor for class com.google.cloud.dataflow.sdk.coders.MapCoderBase
 
mappedTo(SerializableFunction<Instant, Instant>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
Adjust the time at which the trigger will fire.
matches(String) - Method in enum com.google.cloud.dataflow.sdk.io.TextIO.CompressionType
Determine if a given filename matches a compression type based on its extension.
Max - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for computing the maximum of the elements in a PCollection, or the maximum of the values associated with each key in a PCollection of KVs.
Max() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Max
 
Max.MaxDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
Max.MaxFn<NumT extends Comparable<NumT>> - Class in com.google.cloud.dataflow.sdk.transforms
Max.MaxIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
Max.MaxLongFn - Class in com.google.cloud.dataflow.sdk.transforms
MaxDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Max.MaxDoubleFn
 
MaxFn(NumT) - Constructor for class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
Constructs a combining function that computes the maximum over a collection of values of type N, given the smallest value of type N, which is the identity value for the maximum operation over Ns.
MaxIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Max.MaxIntegerFn
 
MaxLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Max.MaxLongFn
 
maxNumRecords(int) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Sets the maximum number of records that will be read from Pubsub.
maxNumRecords(int) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Sets the maximum number of records that will be read from Pubsub.
maxReadTime(Duration) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Sets the maximum duration during which records will be read from Pubsub.
maxReadTime(Duration) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Sets the maximum duration during which records will be read from Pubsub.
maxTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
Returns the upper bound of timestamps for values in this window.
maxTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow
 
maxTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns the largest timestamp that can be included in this window.
Mean - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for computing the arithmetic mean (a.k.a.
merge(Collection<W>, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.MergeContext
Signals to the framework that the windows in toBeMerged should be merged together to form mergeResult.
mergeAccumulator(AccumT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn.Accumulator
Adds the input values represented by the given accumulator into this accumulator.
mergeAccumulators(Iterable<ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique>) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
 
mergeAccumulators(Iterable<AccumT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
 
mergeAccumulators(Iterable<double[]>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
 
mergeAccumulators(Iterable<Combine.BinaryCombineFn.Holder<V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
 
mergeAccumulators(Iterable<int[]>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
 
mergeAccumulators(Iterable<long[]>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
 
mergeAccumulators(Iterable<AccumT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
Returns an accumulator representing the accumulation of all the input values accumulated in the merging accumulators.
mergeAccumulators(K, Iterable<AccumT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
Returns an accumulator representing the accumulation of all the input values accumulated in the merging accumulators.
mergeAccumulators(Iterable<List<V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.SimpleCombineFn
 
mergeAccumulators(Iterable<Top.BoundedHeap<KV<Integer, T>, SerializableComparator<KV<Integer, T>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
 
mergeAccumulators(Iterable<List<T>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.Concatenate
 
MergeContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.MergeContext
 
MergeOverlappingIntervalWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that merges overlapping IntervalWindows.
MergeOverlappingIntervalWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.MergeOverlappingIntervalWindows
 
mergeWindows(WindowFn<Object, W>.MergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
mergeWindows(WindowFn<?, IntervalWindow>.MergeContext) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.MergeOverlappingIntervalWindows
Merge overlapping intervals.
mergeWindows(WindowFn<T, W>.MergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.NonMergingWindowFn
 
mergeWindows(WindowFn<Object, IntervalWindow>.MergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
mergeWindows(WindowFn<T, W>.MergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Does whatever merging of windows is necessary.
mimeType - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
The MIME type used in the creation of the output channel (if the file system supports it).
Min - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for computing the minimum of the elements in a PCollection, or the minimum of the values associated with each key in a PCollection of KVs.
Min() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Min
 
Min.MinDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
Min.MinFn<NumT extends Comparable<NumT>> - Class in com.google.cloud.dataflow.sdk.transforms
Min.MinIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
Min.MinLongFn - Class in com.google.cloud.dataflow.sdk.transforms
MinDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Min.MinDoubleFn
 
MinFn(NumT) - Constructor for class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
Constructs a combining function that computes the minimum over a collection of values of type N, given the largest value of type N, which is the identity value for the minimum operation over Ns.
MinIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Min.MinIntegerFn
 
MinLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Min.MinLongFn
 
modifyEnvironmentBeforeSubmission(Environment) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunnerHooks
Allows the user to modify the environment of their job before their job is submitted to the service for execution.
months(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
Returns a WindowFn that windows elements into periods measured by months.
MultiThreadedIntraBundleProcessingDoFn(DoFn<InputT, OutputT>, int) - Constructor for class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 

N

name - Variable in class com.google.cloud.dataflow.sdk.transforms.PTransform
The base name of this PTransform, e.g., from ParDo.named(String), or from defaults, or null if not yet assigned.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new AvroIO.Read PTransform that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns an AvroIO.Read PTransform with the given step name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new AvroIO.Write PTransform that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns an AvroIO.Write PTransform with the given step name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Sets the name associated with this transformation.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
 
named(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Sets the name associated with this transformation.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Sets the name associated with this transformation.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a new PubsubIO.Read PTransform that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
 
named(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new PubsubIO.Write PTransform that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
 
named(String) - Method in class com.google.cloud.dataflow.sdk.io.Read.Bound
Returns a new Read PTransform that's like this one but has the given name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.Read
Returns a new Read.Bound PTransform with the given name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new TextIO.Read PTransform that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a TextIO.Read PTransform with the given step name.
named(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a new TextIO.Write PTransform that's like this one but with the given step name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a TextIO.Write PTransform with the given step name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
Return a new Globally transform that's like this transform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
Return a new Globally transform that's like this transform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
Returns a new ParDo PTransform that's like this PTransform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
Returns a new multi-output ParDo PTransform that's like this PTransform but with the specified name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo PTransform with the given name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo transform that's like this transform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo transform that's like this transform but with the specified name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Returns a new Window PTransform that's like this PTransform but with the specified name.
named(String) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Creates a Window PTransform with the given name.
named(String) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
Returns a new Window transform that's like this transform but with the specified name.
NameMap() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.NameMap
 
needsValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
 
needsValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
 
needsValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
 
needsValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
 
NESTED - Static variable in class com.google.cloud.dataflow.sdk.coders.Coder.Context
The nested context.
nested() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.Context
 
newWith(List<SerializableFunction<Instant, Instant>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
newWith(List<SerializableFunction<Instant, Instant>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
Method to create an updated version of this TimeTrigger modified to use the specified transform.
nextStepTowardsDestination() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnTimerContext
Returns the sub-trigger of the current trigger that is the next step towards the destination.
NonDeterministicException(Coder<?>, String, Coder.NonDeterministicException) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
NonDeterministicException(Coder<?>, String) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
NonDeterministicException(Coder<?>, List<String>) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
NonDeterministicException(Coder<?>, List<String>, Coder.NonDeterministicException) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
 
NonMergingWindowFn<T,W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Abstract base class for WindowFns that do not merge windows.
NonMergingWindowFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.NonMergingWindowFn
 

O

of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns an AvroCoder instance for the provided element class.
of(Schema) - Static method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns an AvroCoder instance for the Avro schema.
of(Class<T>, Schema) - Static method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Returns an AvroCoder instance for the provided element type using the provided Avro schema.
of(String, String) - Static method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
 
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
 
of(List<Object>) - Static method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
 
of(String, String, String) - Static method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
 
of(Coder<IntermediateT>, DelegateCoder.CodingFunction<T, IntermediateT>, DelegateCoder.CodingFunction<IntermediateT, T>) - Static method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
 
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
of(List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
 
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
Create a coder for a given type of JAXB annotated objects.
of(String) - Static method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
Constructor for JSON deserialization only.
of(Coder<K>, Coder<V>) - Static method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
of(List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
of(String, boolean, List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.KvCoderBase
 
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.ListCoder
 
of(List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.ListCoder
 
of(Coder<K>, Coder<V>) - Static method in class com.google.cloud.dataflow.sdk.coders.MapCoder
Produces a MapCoder with the given keyCoder and valueCoder.
of(List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.MapCoder
 
of(String, List<Coder<?>>) - Static method in class com.google.cloud.dataflow.sdk.coders.MapCoderBase
 
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Returns a Proto2Coder for the given Protobuf message class.
of(String, List<String>) - Static method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Constructor for JSON deserialization only.
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
Returns a SerializableCoder instance for the provided element class.
of(String) - Static method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.SetCoder
Produces a SetCoder with the given elementCoder.
of(List<Object>) - Static method in class com.google.cloud.dataflow.sdk.coders.SetCoder
Dynamically typed constructor for JSON deserialization.
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
of() - Static method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
 
of(WindowedValue<V>) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns a new ValueWithMetadata with the WindowedValue.
of(String, InputT, OutputT, TransformT) - Static method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
of(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine.SimpleCombineFn
Returns a CombineFn that uses the given SerializableFunction to combine values.
of(Iterable<T>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.Values transform that produces a PCollection containing elements of the provided Iterable.
of(T...) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.Values transform that produces a PCollection containing the specified elements.
of(Map<K, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.Values transform that produces a PCollection of KVs corresponding to the keys and values of the specified Map.
of(Class<? extends DoFnWithContext>) - Static method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
 
of(DoFn<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns a DoFnTester supporting unit-testing of the given DoFn.
of(DoFnWithContext<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns a DoFnTester supporting unit-testing of the given DoFn.
of(DoFn<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound
Returns a new IntraBundleParallelization PTransform like this one with the specified DoFn.
of(DoFn<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization
Creates a IntraBundleParallelization PTransform for the given DoFn that processes elements using multiple threads.
of(DoFn<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Unbound
Returns a new IntraBundleParallelization PTransform like this one with the specified DoFn.
of(CoGbkResultSchema, UnionCoder) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
Returns a CoGbkResultCoder for the given schema and unionCoder.
of(List<Coder<?>>, CoGbkResultSchema) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
of(TupleTag<V>, List<V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
Returns a new CoGbkResult that contains just the given tag and given data.
of(List<TupleTag<?>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
of(TupleTag<InputT>, PCollection<KV<K, InputT>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
Returns a new KeyedPCollectionTuple<K> with the given tag and initial PCollection.
of(DoFn<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo PTransform that will invoke the given DoFn function.
of(DoFnWithContext<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo PTransform that will invoke the given DoFnWithContext function.
of(DoFn<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo PTransform that's like this transform but that will invoke the given DoFn function, and that has its input and output types bound.
of(DoFnWithContext<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo PTransform that's like this transform but which will invoke the given DoFnWithContext function, and which has its input and output types bound.
of(DoFn<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo PTransform that's like this transform but that will invoke the given DoFn function, and that has its input type bound.
of(DoFnWithContext<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo PTransform that's like this transform but which will invoke the given DoFnWithContext function, and which has its input type bound.
of(int, Partition.PartitionFn<? super T>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Partition
Returns a new Partition PTransform that divides its input PCollection into the given number of partitions, using the given partitioning function.
of(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<List<T>> with a single element containing the largest count elements of the input PCollection<T>, in decreasing order, sorted using the given Comparator<T>.
of(PCollectionView<ViewT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.View.CreatePCollectionView
 
of(Trigger.OnceTrigger<W>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
of(Trigger.OnceTrigger<W>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
of() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
Returns the default trigger.
of(Duration) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
Partitions the timestamp space into half-open intervals of the form [N * size, (N + 1) * size), where 0 is the epoch.
of(Duration) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
Assigns timestamps into half-open intervals of the form [N * period, N * period + size), where 0 is the epoch.
of(SerializableFunction<V, K>) - Static method in class com.google.cloud.dataflow.sdk.transforms.WithKeys
Returns a PTransform that takes a PCollection<V> and returns a PCollection<KV<K, V>>, where each of the values in the input PCollection has been paired with a key computed from the value by invoking the given SerializableFunction.
of(K) - Static method in class com.google.cloud.dataflow.sdk.transforms.WithKeys
Returns a PTransform that takes a PCollection<V> and returns a PCollection<KV<K, V>>, where each of the values in the input PCollection has been paired with the given key.
of(String, Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.values.CodedTupleTag
Returns a CodedTupleTag with the given id that uses the given Coder whenever a value associated with the tag needs to be serialized.
of(Map<CodedTupleTag<?>, Object>) - Static method in class com.google.cloud.dataflow.sdk.values.CodedTupleTagMap
Returns a CodedTupleTagMap containing the given mappings.
of(K, V) - Static method in class com.google.cloud.dataflow.sdk.values.KV
Returns a KV with the given key and value.
of(PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns a singleton PCollectionList containing the given PCollection.
of(Iterable<PCollection<T>>) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns a PCollectionList containing the given PCollections, in order.
of(TupleTag<T>, PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns a singleton PCollectionTuple containing the given PCollection keyed by the given TupleTag.
of(V, Instant) - Static method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
Returns a new TimestampedValue with the given value and timestamp.
of(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
of(List<Object>) - Static method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
of(TupleTag<?>) - Static method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns a singleton TupleTagList containing the given TupleTag.
of(List<TupleTag<?>>) - Static method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns a TupleTagList containing the given TupleTags, in order.
of(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a TypeDescriptor representing the given type.
of(Type) - Static method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a TypeDescriptor representing the given type.
OFFSET_INFINITY - Static variable in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
Offset corresponding to infinity.
OffsetRangeTracker - Class in com.google.cloud.dataflow.sdk.io.range
A RangeTracker for non-negative positions of type long.
OffsetRangeTracker(long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
Creates an OffsetRangeTracker for the specified range.
ofPrimitiveOutputsInternal(Pipeline, TupleTagList, WindowingStrategy<?, ?>, PCollection.IsBounded) - Static method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
Returns a PCollectionTuple with each of the given tags mapping to a new output PCollection.
ofRecordClass(Class<T>) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
Returns an XmlSink that writes objects of the class specified as XML elements.
oldWindows() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
The old windows that were merged.
OnceTrigger(List<Trigger<W>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
onElement(Trigger<W>.OnElementContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Called immediately after an element is first incorporated into a window.
OnElementContext(Object, Instant) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
onMerge(Trigger<W>.OnMergeContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Called immediately after windows have been merged.
OnMergeContext(Iterable<W>, Map<W, BitSet>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
 
onTimer(Trigger<W>.OnTimerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
 
onTimer(Trigger<W>.OnTimerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
 
onTimer(Trigger<W>.OnTimerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
 
onTimer(Trigger<W>.OnTimerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
onTimer(Trigger<W>.OnTimerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
onTimer(Trigger<W>.OnTimerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
 
onTimer(Trigger<W>.OnTimerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
 
onTimer(Trigger<W>.OnTimerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Called when a timer has fired for the trigger or one of its sub-triggers.
OnTimerContext(Trigger.TriggerId<W>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnTimerContext
 
open(String) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Opens the channel.
open(String) - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
Performs bundle initialization.
Options() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Options
 
Options() - Constructor for class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Options
 
OrderByKey() - Constructor for class com.google.cloud.dataflow.sdk.values.KV.OrderByKey
 
OrderByValue() - Constructor for class com.google.cloud.dataflow.sdk.values.KV.OrderByValue
 
orFinally(Trigger.OnceTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
Specify an ending condition for this trigger.
OUTER - Static variable in class com.google.cloud.dataflow.sdk.coders.Coder.Context
The outer context.
output(OutputT) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Adds the given element to the main output PCollection.
output(OutputT) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Adds the given element to the main output PCollection.
outputWithTimestamp(OutputT, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Adds the given element to the main output PCollection, with the given timestamp.
outputWithTimestamp(OutputT, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Adds the given element to the main output PCollection, with the given timestamp.

P

pane() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns information about the pane within this window into which the input element has been assigned.
PaneInfo - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Provides information about the pane this value belongs to, e.g.
PaneInfo.PaneInfoCoder - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A Coder for encoding PaneInfo instances.
PaneInfoCoder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
 
ParDo - Class in com.google.cloud.dataflow.sdk.transforms
ParDo is the core element-wise transform in Google Cloud Dataflow, invoking a user-specified function (from I to Output) on each of the elements of the input PCollection<InputT> to produce zero or more output elements, all of which are collected into the output PCollection<OutputT>.
ParDo() - Constructor for class com.google.cloud.dataflow.sdk.transforms.ParDo
 
ParDo.Bound<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that, when applied to a PCollection<InputT>, invokes a user-specified DoFn<InputT, OutputT> on all its elements, with all its outputs collected into an output PCollection<OutputT>.
ParDo.BoundMulti<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that, when applied to a PCollection<InputT>, invokes a user-specified DoFn<InputT, OutputT> on all its elements, which can emit elements to any of the PTransform's main and side output PCollections, which are bundled into a result PCollectionTuple.
ParDo.Unbound - Class in com.google.cloud.dataflow.sdk.transforms
An incomplete ParDo transform, with unbound input/output types.
ParDo.UnboundMulti<OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
An incomplete multi-output ParDo transform, with unbound input type.
parseTableSpec(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO
Parse a table specification in the form "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]".
Partition<T> - Class in com.google.cloud.dataflow.sdk.transforms
Partition takes a PCollection<T> and a PartitionFn, uses the PartitionFn to split the elements of the input PCollection into N partitions, and returns a PCollectionList<T> that bundles N PCollection<T>s containing the split elements.
Partition.PartitionFn<T> - Interface in com.google.cloud.dataflow.sdk.transforms
A function object that chooses an output partition for an element.
partitionFor(T, int) - Method in interface com.google.cloud.dataflow.sdk.transforms.Partition.PartitionFn
Chooses the partition into which to put the given element.
PartitioningWindowFn<T,W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that places each value into exactly one window based on its timestamp and never merges windows.
PartitioningWindowFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
 
pastEndOfWindow() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark
Creates a trigger that fires when the watermark passes the end of the window.
pastFirstElementInPane() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
Creates a trigger that fires when the current processing time passes the processing time at which this trigger saw the first element in a pane.
PathValidatorFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.PathValidatorFactory
 
PBegin - Class in com.google.cloud.dataflow.sdk.values
PBegin is used as the "input" to a root PTransform that is the first operation in a Pipeline, such as TextIO.Read or Create.
PBegin(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.PBegin
Constructs a PBegin in the given Pipeline.
PCollection<T> - Class in com.google.cloud.dataflow.sdk.values
A PCollection<T> is an immutable collection of values of type T.
PCollection.IsBounded - Enum in com.google.cloud.dataflow.sdk.values
The PCollection IsBounded property.
PCollectionList<T> - Class in com.google.cloud.dataflow.sdk.values
A PCollectionList<T> is an immutable list of homogeneously typed PCollection<T>s.
pCollections() - Static method in class com.google.cloud.dataflow.sdk.transforms.Flatten
Returns a PTransform that flattens a PCollectionList into a PCollection containing all the elements of all the PCollections in its input.
PCollectionTuple - Class in com.google.cloud.dataflow.sdk.values
A PCollectionTuple is an immutable tuple of heterogeneously-typed PCollections, "keyed" by TupleTags.
PCollectionView<T> - Interface in com.google.cloud.dataflow.sdk.values
A PCollectionView<T> is an immutable view of a PCollection as a value of type T that can be accessed e.g.
PDone - Class in com.google.cloud.dataflow.sdk.values
PDone is the output of a PTransform that doesn't have a non-trival result, e.g., a Write.
peekOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the main output.
peekSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the side output with the given tag.
perElement() - Static method in class com.google.cloud.dataflow.sdk.transforms.Count
Returns a Count.PerElement PTransform that counts the number of occurrences of each element in its input PCollection.
PerElement() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Count.PerElement
 
perKey(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
Returns a PTransform that takes a PCollection<KV<K, V>> and returns a PCollection<KV<K, List<V>>> that contains an output element mapping each distinct key in the input PCollection to a List of the approximate N-tiles of the values associated with that key in the input PCollection.
perKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
Like ApproximateQuantiles.perKey(int, Comparator), but sorts values using the their natural ordering.
perKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
Returns a PTransform that takes a PCollection<KV<K, V>> and returns a PCollection<KV<K, Long>> that contains an output element mapping each distinct key in the input PCollection to an estimate of the number of distinct values associated with that key in the input PCollection.
perKey(double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
Like ApproximateUnique.perKey(int), but specifies the desired maximum estimation error instead of the sample size.
perKey(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.PerKey PTransform that first groups its input PCollection of KVs by keys and windows, then invokes the given function on each of the values lists to produce a combined value, and then returns a PCollection of KVs mapping each distinct key to its combined value for each window.
perKey(Combine.CombineFn<? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.PerKey PTransform that first groups its input PCollection of KVs by keys and windows, then invokes the given function on each of the values lists to produce a combined value, and then returns a PCollection of KVs mapping each distinct key to its combined value for each window.
perKey(Combine.KeyedCombineFn<? super K, ? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
Returns a Combine.PerKey PTransform that first groups its input PCollection of KVs by keys and windows, then invokes the given function on each of the key/values-lists pairs to produce a combined value, and then returns a PCollection of KVs mapping each distinct key to its combined value for each window.
perKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Mean
Returns a PTransform that takes an input PCollection<KV<K, N>> and returns a PCollection<KV<K, Double>> that contains an output element mapping each distinct key in the input PCollection to the mean of the values associated with that key in the input PCollection.
perKey(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<KV<K, V>> and returns a PCollection<KV<K, List<V>>> that contains an output element mapping each distinct key in the input PCollection to the largest count values associated with that key in the input PCollection<KV<K, V>>, in decreasing order, sorted using the given Comparator<V>.
PInput - Interface in com.google.cloud.dataflow.sdk.values
The interface for things that might be input to a PTransform.
Pipeline - Class in com.google.cloud.dataflow.sdk
A Pipeline manages a DAG of PTransforms, and the PCollections that the PTransforms consume and produce.
Pipeline(PipelineRunner<?>) - Constructor for class com.google.cloud.dataflow.sdk.Pipeline
Pipeline(PipelineRunner<?>, PipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.Pipeline
 
Pipeline.PipelineVisitor - Interface in com.google.cloud.dataflow.sdk
A Pipeline.PipelineVisitor can be passed into Pipeline.traverseTopologically(com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor) to be called for each of the transforms and values in the Pipeline.
PipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
PipelineOptions are used to configure Pipelines.
PipelineOptions.CheckEnabled - Enum in com.google.cloud.dataflow.sdk.options
Enumeration of the possible states for a given check.
PipelineOptionsFactory - Class in com.google.cloud.dataflow.sdk.options
Constructs a PipelineOptions or any derived interface that is composable to any other derived interface of PipelineOptions via the PipelineOptions.as(java.lang.Class<T>) method.
PipelineOptionsFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
 
PipelineOptionsFactory.Builder - Class in com.google.cloud.dataflow.sdk.options
A fluent PipelineOptions builder.
PipelineOptionsRegistrar - Interface in com.google.cloud.dataflow.sdk.options
PipelineOptions creators have the ability to automatically have their PipelineOptions registered with this SDK by creating a ServiceLoader entry and a concrete implementation of this interface.
PipelineOptionsValidator - Class in com.google.cloud.dataflow.sdk.options
Validates that the PipelineOptions conforms to all the Validation criteria.
PipelineOptionsValidator() - Constructor for class com.google.cloud.dataflow.sdk.options.PipelineOptionsValidator
 
PipelineResult - Interface in com.google.cloud.dataflow.sdk
Result of Pipeline.run().
PipelineResult.State - Enum in com.google.cloud.dataflow.sdk
Named constants for common values for the job state.
PipelineRunner<ResultT extends PipelineResult> - Class in com.google.cloud.dataflow.sdk.runners
A PipelineRunner can execute, translate, or otherwise process a Pipeline.
PipelineRunner() - Constructor for class com.google.cloud.dataflow.sdk.runners.PipelineRunner
 
PipelineRunnerRegistrar - Interface in com.google.cloud.dataflow.sdk.runners
PipelineRunner creators have the ability to automatically have their PipelineRunner registered with this SDK by creating a ServiceLoader entry and a concrete implementation of this interface.
plusDelayOf(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
Adds some delay to the original target time.
popNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Removes the last TransformTreeNode from the stack.
POutput - Interface in com.google.cloud.dataflow.sdk.values
The interface for things that might be output from a PTransform.
POutputValueBase - Class in com.google.cloud.dataflow.sdk.values
A POutputValueBase is the abstract base class of PTransform outputs.
POutputValueBase(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.POutputValueBase
 
POutputValueBase() - Constructor for class com.google.cloud.dataflow.sdk.values.POutputValueBase
No-arg constructor for Java serialization only.
prepareWrite(WritableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Called with the channel that a subclass will write its header, footer, and values to.
prepareWrite(WritableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Creates the output stream that elements will be written to.
printHelp(PrintStream) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Outputs the set of registered options with the PipelineOptionsFactory with a description for each one if available to the output stream.
printHelp(PrintStream, Class<? extends PipelineOptions>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
Outputs the set of options available to be set for the passed in PipelineOptions interface.
processBatch(InputT...) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
A convenience operation that first calls DoFnTester.startBundle(), then calls DoFnTester.processElement(InputT) on each of the arguments, then calls DoFnTester.finishBundle(), then returns the result of DoFnTester.takeOutputElements().
ProcessContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
 
ProcessContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
 
processElement(DoFn<KV<K, Iterable<InputT>>, KV<K, OutputT>>.ProcessContext) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
 
processElement(DoFn<InputT, OutputT>.ProcessContext) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Processes an input element.
processElement(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
processElement(DoFn<InputT, OutputT>.ProcessContext) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
Whether this source is known to produce key/value pairs with the (encoded) keys in lexicographically sorted order.
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Returns whether the delegate source produces sorted keys.
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
Proto2Coder<T extends com.google.protobuf.Message> - Class in com.google.cloud.dataflow.sdk.coders
An encoder using Google Protocol Buffers 2 binary format.
PROVIDER - Static variable in class com.google.cloud.dataflow.sdk.coders.AvroCoder
 
PROVIDER - Static variable in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
A CoderProvider that constructs a SerializableCoder for any class that implements serializable.
PTransform<InputT extends PInput,OutputT extends POutput> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform<InputT, OutputT> is an operation that takes an InputT (some subtype of PInput) and produces an OutputT (some subtype of POutput).
PTransform() - Constructor for class com.google.cloud.dataflow.sdk.transforms.PTransform
 
PTransform(String) - Constructor for class com.google.cloud.dataflow.sdk.transforms.PTransform
 
PubsubIO - Class in com.google.cloud.dataflow.sdk.io
Read and Write PTransforms for Pub/Sub streams.
PubsubIO() - Constructor for class com.google.cloud.dataflow.sdk.io.PubsubIO
 
PubsubIO.PubsubSubscription - Class in com.google.cloud.dataflow.sdk.io
Class representing a Pubsub Subscription.
PubsubIO.PubsubTopic - Class in com.google.cloud.dataflow.sdk.io
Class representing a Pubsub Topic.
PubsubIO.Read - Class in com.google.cloud.dataflow.sdk.io
A PTransform that continuously reads from a Pubsub stream and returns a PCollection<String> containing the items from the stream.
PubsubIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from a PubSub source and returns a unbounded PCollection containing the items from the stream.
PubsubIO.Write - Class in com.google.cloud.dataflow.sdk.io
A PTransform that continuously writes a PCollection<String> to a Pubsub stream.
PubsubIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes an unbounded PCollection<String> to a PubSub stream.
PubsubTopic(PubsubIO.PubsubTopic.Type, String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
 
pushNode(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Add a TransformTreeNode to the stack.
PValue - Interface in com.google.cloud.dataflow.sdk.values
A PValue is the interface to values that can be input and output from PTransforms.
PValueBase - Class in com.google.cloud.dataflow.sdk.values
A PValueBase is an abstract base class that provides sensible default implementations for methods of PValue.
PValueBase(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.PValueBase
 
PValueBase() - Constructor for class com.google.cloud.dataflow.sdk.values.PValueBase
No-arg constructor for Java serialization only.

R

randomizeIfUnordered(List<T>, boolean) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
If the evaluation context is testing unorderedness, randomly permutes the order of the elements, in a copy if !inPlaceAllowed, and returns the permuted list, otherwise returns the argument unchanged.
rangeTracker - Variable in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource.ByteOffsetBasedReader
The OffsetRangeTracker managing the range and current position of the source.
RangeTracker<PositionT> - Interface in com.google.cloud.dataflow.sdk.io.range
A RangeTracker is a thread-safe helper object for implementing dynamic work rebalancing in position-based BoundedSource.BoundedReader subclasses.
RawUnionValue - Class in com.google.cloud.dataflow.sdk.transforms.join
This corresponds to an integer union tag and value.
RawUnionValue(int, Object) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
Constructs a partial union from the given union tag and value.
Read() - Constructor for class com.google.cloud.dataflow.sdk.io.AvroIO.Read
 
Read() - Constructor for class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
 
read() - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Returns an empty DatastoreIO.Read builder with the default host.
Read() - Constructor for class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
 
Read - Class in com.google.cloud.dataflow.sdk.io
A PTransform for reading from a Source.
Read() - Constructor for class com.google.cloud.dataflow.sdk.io.Read
 
Read() - Constructor for class com.google.cloud.dataflow.sdk.io.TextIO.Read
 
Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
Implementation of the Read PTransform builder.
readFrom(String, DatastoreV1.Query) - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Returns a PTransform that reads Datastore entities from the query against the given dataset.
readFrom(String, String, DatastoreV1.Query) - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Returns a PTransform that reads Datastore entities from the query against the given dataset and host.
readFromFileWithClass(String, Class<T>) - Static method in class com.google.cloud.dataflow.sdk.io.AvroSource
Creates a Read transform that will read from an AvroSource that is configured to read records of the given type from a file pattern.
readFromSource(FileBasedSource<T>, CompressedSource.DecompressingChannelFactory) - Static method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Creates a Read transform that reads from a CompressedSource that reads from an underlying FileBasedSource after decompressing it with a CompressedSource.DecompressingChannelFactory.
readNextBlock() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
 
readNextBlock() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
Read the next block from the input.
readNextRecord() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
Reads the next record from the block and returns true iff one exists.
readNextRecord() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
 
readNextRecord() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Reads the next record via the delegate reader.
readNextRecord() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
 
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
 
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in interface com.google.cloud.dataflow.sdk.values.POutput
Records that this POutput is an output of the given PTransform.
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
Records that this POutputValueBase is an output with the given name of the given AppliedPTransform.
recordAsOutput(AppliedPTransform<?, ?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
 
recordAsOutput(AppliedPTransform<?, ?, ?>, String) - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Records that this POutputValueBase is an output with the given name of the given AppliedPTransform in the given Pipeline.
RecordingPipelineVisitor - Class in com.google.cloud.dataflow.sdk.runners
Provides a simple Pipeline.PipelineVisitor that records the transformation tree.
RecordingPipelineVisitor() - Constructor for class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
register(Class<? extends PipelineOptions>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
This registers the interface with this factory.
registerByteSizeObserver(T, ElementByteSizeObserver, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.
registerByteSizeObserver(IterableT, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
Notifies ElementByteSizeObserver about the byte size of the encoded value using this coder.
registerByteSizeObserver(KV<K, V>, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
Notifies ElementByteSizeObserver about the byte size of the encoded value using this coder.
registerByteSizeObserver(Map<K, V>, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
Notifies ElementByteSizeObserver about the byte size of the encoded value using this coder.
registerByteSizeObserver(T, ElementByteSizeObserver, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
Notifies ElementByteSizeObserver about the byte size of the encoded value using this coder.
registerCoder(Class<?>, Class<?>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Registers coderClazz as the default Coder class to handle encoding and decoding instances of clazz, overriding prior registrations if any exist.
registerCoder(Class<?>, CoderFactory) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Registers coderFactory as the default CoderFactory to produce Coder instances to decode and encode instances of clazz.
registerCoder(Class<T>, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Register the provided Coder for encoding all values of the specified Class.
registerDefaultTransformEvaluator(Class<TransformT>, DirectPipelineRunner.TransformEvaluator<TransformT>) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Records that instances of the specified PTransform class should be evaluated by default by the corresponding TransformEvaluator.
registerStandardCoders() - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
Registers standard Coders with this CoderRegistry.
registerTransformEvaluator(Class<TransformT>, DirectPipelineRunner.TransformEvaluator<TransformT>) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Records that instances of the specified PTransform class should be evaluated by the corresponding TransformEvaluator.
registerTransformTranslator(Class<TransformT>, DataflowPipelineTranslator.TransformTranslator<? extends TransformT>) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
Records that instances of the specified PTransform class should be translated by default by the corresponding DataflowPipelineTranslator.TransformTranslator.
ReifyTimestampsAndWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.ReifyTimestampsAndWindows
 
remerge() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
Creates a Window PTransform that does not change assigned windows, but will cause windows to be merged again as part of the next GroupByKey.
Remerge() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Remerge
 
remove(CodedTupleTag<T>, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Removes the keyed state associated with the given tag and window.
RemoveDuplicates<T> - Class in com.google.cloud.dataflow.sdk.transforms
RemoveDuplicates<T> takes a PCollection<T> and returns a PCollection<T> that has all the elements of the input but with duplicate elements removed such that each element is unique within each window.
removeTemporaryFiles(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Removes temporary output files.
Repeatedly<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Repeat a trigger, either until some condition is met or forever.
resetTree() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Clears all keyed state for triggers in the current sub-tree and unsets all the associated finished bits.
resolveType(Type) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Returns a TypeDescriptor representing the given type, with type variables resolved according to the specialization in this type.
run() - Method in class com.google.cloud.dataflow.sdk.Pipeline
Runs the Pipeline.
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
Processes the given Pipeline, returning the results.
run() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
 
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
 
run() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
 
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
 
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner
Processes the given Pipeline, returning the results.
run(Pipeline) - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
 
run() - Method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
Runs this TestPipeline, unwrapping any AssertionError that is raised during testing.
RunnableOnService - Interface in com.google.cloud.dataflow.sdk.testing
Category tag for tests that can be run on the DataflowPipelineRunner if the runIntegrationTestOnService System property is set to true.
Runner() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Runner
 
Runner() - Constructor for class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Runner
 
runWindowFn(WindowFn<T, W>, List<Long>) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
Runs the WindowFn over the provided input, returning a map of windows to the timestamps in those windows.

S

Sample - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for taking samples of the elements in a PCollection, or samples of the values associated with each key in a PCollection of KVs.
Sample() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sample
 
Sample.FixedSizedSampleFn<T> - Class in com.google.cloud.dataflow.sdk.transforms
CombineFn that computes a fixed-size sample of a collection of values.
Sample.SampleAny<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that takes a PCollection<T> and a limit, and produces a new PCollection<T> containing up to limit elements of the input PCollection.
satisfies(SerializableFunction<Iterable<T>, Void>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Applies a SerializableFunction to check the elements of the Iterable.
satisfies(DataflowAssert.AssertRelation<Iterable<T>, Iterable<T>>, Iterable<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Applies a SerializableFunction to check the elements of the Iterable.
satisfies(SerializableFunction<T, Void>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Applies a SerializableFunction to check the value of this SingletonAssert's view.
satisfies(DataflowAssert.AssertRelation<T, T>, T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Applies an DataflowAssert.AssertRelation to check the provided relation against the value of this assert and the provided expected value.
SerializableCoder<T extends Serializable> - Class in com.google.cloud.dataflow.sdk.coders
A SerializableCoder is a Coder for a Java class that implements Serializable.
SerializableCoder(Class<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
SerializableComparator<T> - Interface in com.google.cloud.dataflow.sdk.transforms
A Comparator that is also Serializable.
SerializableFunction<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
A function that computes an output value of type OutputT from an input value of type InputT and is Serializable.
ServiceException - Exception in com.google.cloud.dataflow.sdk.runners
Signals there was an error retrieving information about a job from the Cloud Dataflow Service.
Sessions - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn windowing values into sessions separated by Sessions.gapDuration-long periods with no elements.
set(long...) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
setApiRootUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setAppName(String) - Method in interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions
 
setAutoscalingAlgorithm(DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setClusterManagerApi(DataflowPipelineWorkerPoolOptions.ClusterManagerApiType) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
SetCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A SetCoder<T> encodes any Set<T> as an encoding of an iterable of its elements.
SetCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.SetCoder
 
setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
Sets the coder to use for elements of type T, as needed for internal purposes.
setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
Sets the coder to use for elements of type T, as needed for internal purposes.
setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Sets the Coder used by this PCollection to encode and decode the values stored in it.
setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
Sets the Coder used by this TypedPValue to encode and decode the values stored in it.
setCoderRegistry(CoderRegistry) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Sets the CoderRegistry that this Pipeline uses.
setCredentialDir(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setCredentialFactoryClass(Class<? extends CredentialFactory>) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setCredentialId(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setDataflowClient(Dataflow) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setDataflowEndpoint(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setDataflowJobFile(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setDefaultWorkerLogLevel(DataflowWorkerLoggingOptions.Level) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
 
setDiskSizeGb(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setDiskSourceImage(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setEnableCloudDebugger(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
 
setExecutorService(ExecutorService) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
 
setExperiments(List<String>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setFallbackCoderProvider(CoderProvider) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
 
setFilesToStage(List<String>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setFinished(boolean) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Sets the finished bit for the current trigger.
setGcpCredential(Credential) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setGcsEndpoint(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
 
setGcsUtil(GcsUtil) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
 
setGoogleApiTrace(GoogleApiDebugOptions.GoogleApiTracer...) - Method in interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions
 
setHooks(DataflowPipelineRunnerHooks) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
Sets callbacks to invoke during execution see DataflowPipelineRunnerHooks.
setHooks(DataflowPipelineRunnerHooks) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
Sets callbacks to invoke during execution see DataflowPipelineRunnerHooks.
setJobId(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
 
setJobMessageOutput(PrintStream) - Method in interface com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions
 
setJobName(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
setMaxNumWorkers(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setName(String) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Sets the name of this PCollection.
setName(String) - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
Sets the name of this PValueBase.
setNetwork(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setNumWorkers(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setOutput(TransformTreeNode, POutput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Sets the output of a transform node.
setOutput(POutput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Adds an output to the transform node.
setOverrideWindmillBinary(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setPathValidator(PathValidator) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setPathValidatorClass(Class<? extends PathValidator>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setPCollection(PCollection<T>, List<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Shorthand for setting the value of a PCollection where the elements do not have timestamps or windows.
setPCollectionValuesWithMetadata(PCollection<T>, List<DirectPipelineRunner.ValueWithMetadata<T>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Sets the value of the given PCollection, where each element also has a timestamp and collection of windows.
setPCollectionView(PCollectionView<T>, Iterable<WindowedValue<ElemT>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Sets the value associated with the given PCollectionView.
setPCollectionWindowedValue(PCollection<T>, List<WindowedValue<T>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
Sets the value of the given PCollection, where each element also has a timestamp and collection of windows.
setProject(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
setProject(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setPubsubRootUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setReload(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setRunner(Class<? extends PipelineRunner<?>>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
 
setSecretsFile(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setServiceAccountKeyfile(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setServiceAccountName(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
 
setSideInput(PCollectionView<?>, Iterable<WindowedValue<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Registers the values of a side input PCollectionView to pass to the DoFn under test.
setSideInputInGlobalWindow(PCollectionView<?>, Iterable<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Registers the values for a side input PCollectionView to pass to the DoFn under test.
setSideInputs(Map<PCollectionView<?>, Iterable<WindowedValue<?>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Registers the tuple of values of the side input PCollectionViews to pass to the DoFn under test.
setSideOutputTags(TupleTagList) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Registers the list of TupleTags that can be used by the DoFn under test to output to side output PCollections.
setStableUniqueNames(PipelineOptions.CheckEnabled) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
 
setStager(Stager) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setStagerClass(Class<? extends Stager>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setStagingLocation(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
setStreaming(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.StreamingOptions
 
setTeardownPolicy(DataflowPipelineWorkerPoolOptions.TeardownPolicy) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setTempDatasetId(String) - Method in interface com.google.cloud.dataflow.sdk.options.BigQueryOptions
 
setTempLocation(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
 
setTimer(Instant, TimerManager.TimeDomain) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Sets a timer to fire when the watermark or processing time is beyond the given timestamp.
setTransformNameMapping(DataflowPipelineDebugOptions.NameMap) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
 
setTypeDescriptorInternal(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Sets the TypeDescriptor<T> for this PCollection<T>, so that the enclosing PCollectionTuple, PCollectionList<T>, or PTransform<?, PCollection<T>>, etc., can provide more detailed reflective information.
setTypeDescriptorInternal(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
Sets the TypeDescriptor<T> associated with this class.
setupDelegateAggregators() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Sets up Aggregators created by the DoFn so they are usable within this context.
setWindowingStrategyInternal(WindowingStrategy<?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
Sets the WindowingStrategy of this PCollection.
setWorkerCacheMb(Integer) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
 
setWorkerDiskType(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setWorkerId(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
 
setWorkerLogLevelOverrides(DataflowWorkerLoggingOptions.WorkerLogLevelOverride...) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
 
setWorkerMachineType(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setWorkerPoolType(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
setZone(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
 
ShardNameTemplate - Class in com.google.cloud.dataflow.sdk.io
Standard shard naming templates.
ShardNameTemplate() - Constructor for class com.google.cloud.dataflow.sdk.io.ShardNameTemplate
 
sideInput(PCollectionView<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the value of the side input for the window corresponding to the window of the main input element.
sideInput(PCollectionView<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
Returns the value of the side input.
sideOutput(TupleTag<T>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Adds the given element to the side output PCollection with the given tag.
sideOutput(TupleTag<T>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Adds the given element to the side output PCollection with the given tag.
sideOutputWithTimestamp(TupleTag<T>, T, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
Adds the given element to the specified side output PCollection, with the given timestamp.
sideOutputWithTimestamp(TupleTag<T>, T, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
Adds the given element to the specified side output PCollection, with the given timestamp.
SingletonAssert(DataflowAssert.CreateActual<?, T>, Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
 
sink() - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Returns a new DatastoreIO.Sink builder using the default host.
Sink(String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
Constructs a Sink with given host and dataset.
sink - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
The Sink that this WriteOperation will write to.
Sink<T> - Class in com.google.cloud.dataflow.sdk.io
A Sink represents a resource that can be written to using the Write transform.
Sink() - Constructor for class com.google.cloud.dataflow.sdk.io.Sink
 
Sink.WriteOperation<T,WriteT> - Class in com.google.cloud.dataflow.sdk.io
A Sink.WriteOperation defines the process of a parallel write of objects to a Sink.
Sink.Writer<T,WriteT> - Class in com.google.cloud.dataflow.sdk.io
A Writer writes a bundle of elements from a PCollection to a sink.
size() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
Returns the number of columns for this schema.
size() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
Returns the number of PCollections in this PCollectionList.
size() - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
Returns the number of TupleTags in this TupleTagList.
SlidingWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A WindowFn that windows values into possibly overlapping fixed-size timestamp-based windows.
smallest(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<List<T>> with a single element containing the smallest count elements of the input PCollection<T>, in increasing order, sorted according to their natural order.
Smallest() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Top.Smallest
 
smallestPerKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
Returns a PTransform that takes an input PCollection<KV<K, V>> and returns a PCollection<KV<K, List<V>>> that contains an output element mapping each distinct key in the input PCollection to the smallest count values associated with that key in the input PCollection<KV<K, V>>, in increasing order, sorted according to their natural order.
SortValuesByTimestamp() - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.SortValuesByTimestamp
 
Source<T> - Class in com.google.cloud.dataflow.sdk.io
Base class for defining input formats and creating a Source for reading the input.
Source() - Constructor for class com.google.cloud.dataflow.sdk.io.Source
 
Source.AbstractReader<T> - Class in com.google.cloud.dataflow.sdk.io
A base class implementing optional methods of Source.Reader in a default way: All values have the timestamp of BoundedWindow.TIMESTAMP_MIN_VALUE.
Source.Reader<T> - Interface in com.google.cloud.dataflow.sdk.io
The interface that readers of custom input sources must implement.
span(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns the minimal window that includes both this window and the given window.
splitAtFraction(double) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.AbstractBoundedReader
 
splitAtFraction(double) - Method in interface com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
Tells the reader to narrow the range of the input it's going to read and give up the remainder, so that the new range would contain approximately the given fraction of the amount of data in the current range.
splitAtFraction(double) - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource.ByteOffsetBasedReader
 
splitAtFraction(double) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
Splits the source into bundles of approximately given size (in bytes).
splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource
 
splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
StagerFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.StagerFactory
 
StandardCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A StandardCoder is a Coder that defines equality, hashing, and printing via the class name and recursively using StandardCoder.getComponents().
StandardCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
StandardOutputFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions.StandardOutputFactory
 
start() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
 
start() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
 
start() - Method in interface com.google.cloud.dataflow.sdk.io.Source.Reader
Initializes the reader and advances the reader to the first record.
start() - Method in interface com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
Initializes the reader and advances the reader to the first record.
start() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
Returns the start of this window, inclusive.
startBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
Prepares this DoFn instance for processing a batch of elements.
startBundle() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
startBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
 
startReading(ReadableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
Starts reading from the provided channel.
startReading(ReadableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
Creates a decompressing channel from the input channel and passes it to its delegate reader's FileBasedSource.FileBasedReader.startReading(ReadableByteChannel).
startReading(ReadableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
Performs any initialization of the subclass of FileBasedReader that involves IO operations.
store(CodedTupleTag<T>, W, T) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Updates the value stored in keyed state for the given tag and window.
StreamingOptions - Interface in com.google.cloud.dataflow.sdk.options
Options used to configure streaming.
StreamingPubsubIOWrite(PubsubIO.Write.Bound<T>) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
Builds an instance of this class from the overridden transform.
StringDelegateCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
A StringDelegateCoder<T> wraps a Coder<String> and encodes/decodes values of type T via string representations.
StringDelegateCoder(Class<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
 
StringUtf8Coder - Class in com.google.cloud.dataflow.sdk.coders
A StringUtf8Coder encodes Java Strings in UTF-8 encoding.
structuralValue(byte[]) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
 
structuralValue(T) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Returns an object with an Object.equals() method that represents structural equality on the argument.
structuralValue(T) - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
structuralValue(KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
structuralValue(T) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
structuralValueConsistentWithEquals(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
structuralValueConsistentWithEqualsInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
 
subscription(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a new PubsubIO.Read PTransform that's like this one but reading from the given subscription.
subscription(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a PubsubIO.Read PTransform for reading from a specific Pubsub subscription.
subTrigger(int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Access the executable version of the specified sub-trigger.
subTriggers - Variable in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
subTriggers() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
subTriggers() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Access the executable versions of the sub-triggers of the current trigger.
Sum - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for computing the sum of the elements in a PCollection, or the sum of the values associated with each key in a PCollection of KVs.
Sum() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum
 
Sum.SumDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
Sum.SumIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
Sum.SumLongFn - Class in com.google.cloud.dataflow.sdk.transforms
SumDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
 
SumIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
 
SumLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
 

T

TableRowJsonCoder - Class in com.google.cloud.dataflow.sdk.coders
A TableRowJsonCoder encodes BigQuery TableRow objects.
TaggedKeyedPCollection(TupleTag<V>, PCollection<KV<K, V>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple.TaggedKeyedPCollection
 
takeOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the main output.
takeSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
Returns the elements output so far to the side output with the given tag.
TEMPORARY_FILENAME_SEPARATOR - Static variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Name separator for temporary files.
temporaryFileRetention - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
Option to keep or remove temporary output files.
TestCombineDoFn(Combine.KeyedCombineFn<? super K, ? super InputT, AccumT, OutputT>, Coder<AccumT>, boolean) - Constructor for class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
 
TestDataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.testing
A set of options used to configure the TestPipeline.
TestDataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.testing
TestDataflowPipelineRunner is a pipeline runner that wraps a DataflowPipelineRunner when running tests against the TestPipeline.
TestPipeline - Class in com.google.cloud.dataflow.sdk.testing
A creator of test pipelines that can be used inside of tests that can be configured to run locally or against the live service.
TextIO - Class in com.google.cloud.dataflow.sdk.io
PTransforms for reading and writing text files.
TextIO() - Constructor for class com.google.cloud.dataflow.sdk.io.TextIO
 
TextIO.CompressionType - Enum in com.google.cloud.dataflow.sdk.io
Possible text file compression types.
TextIO.Read - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from a text file (or multiple text files matching a pattern) and returns a PCollection containing the decoding of each of the lines of the text file(s).
TextIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that reads from a text file (or multiple text files matching a pattern) and returns a bounded PCollection containing the decoding of each of the lines of the text file(s).
TextIO.Write - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes a PCollection to a text file (or multiple text files matching a sharding pattern), with each PCollection element being encoded into its own line.
TextIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A PTransform that writes a bounded PCollection to a text file (or multiple text files matching a sharding pattern), with each PCollection element being encoded into its own line.
TextualIntegerCoder - Class in com.google.cloud.dataflow.sdk.coders
A TextualIntegerCoder encodes Integers as their textual, decimal, representation.
TextualIntegerCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
that(PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs an DataflowAssert.IterableAssert for the elements of the provided PCollection.
thatIterable(PCollectionView<Iterable<T>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs an DataflowAssert.IterableAssert for the value of the provided PCollectionView PCollectionView<Iterable<T>>.
thatMap(PCollection<KV<K, V>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs a DataflowAssert.SingletonAssert for the value of the provided PCollection, which must have at most one value per key.
thatMultimap(PCollection<KV<K, V>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs a DataflowAssert.SingletonAssert for the value of the provided PCollection.
thatSingleton(PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs a DataflowAssert.SingletonAssert for the value of the provided PCollection PCollection<T>, which must be a singleton.
thatSingletonIterable(PCollection<? extends Iterable<T>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
Constructs an DataflowAssert.IterableAssert for the value of the provided PCollection which must contain a single Iterable<T> value.
timestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the timestamp of the input element.
timestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
Returns the timestamp of the input element.
timestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
Returns the timestamp of the current element.
TIMESTAMP_MAX_VALUE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
 
TIMESTAMP_MIN_VALUE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
 
timestamped(Iterable<TimestampedValue<T>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.TimestampedValues transform that produces a PCollection containing the elements of the provided Iterable with the specified timestamps.
timestamped(TimestampedValue<T>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new Create.TimestampedValues transform that produces a PCollection containing the specified elements with the specified timestamps.
timestamped(Iterable<T>, Iterable<Long>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
Returns a new root transform that produces a PCollection containing the specified elements with the specified timestamps.
TimestampedValue<V> - Class in com.google.cloud.dataflow.sdk.values
An immutable (value, timestamp) pair.
TimestampedValue(V, Instant) - Constructor for class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
TimestampedValue.TimestampedValueCoder<T> - Class in com.google.cloud.dataflow.sdk.values
A Coder for TimestampedValue.
timestampLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a new PubsubIO.Read PTransform that's like this one but reading timestamps from the given PubSub label.
timestampLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a PubsubIO.Read PTransform where record timestamps are expected to be provided using the PubSub labeling API.
timestampLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new PubsubIO.Write PTransform that's like this one but publishing timestamps to the given PubSub label.
timestampLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
If specified, Dataflow will add a Pubsub label to each output record specifying the logical timestamp of the record.
timestampMappers - Variable in class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
A list of timestampMappers m1, m2, m3, ...
TimeTrigger<W extends BoundedWindow,T extends TimeTrigger<W,T>> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Support for manipulating the time at which time-based Triggers fire.
TimeTrigger(List<SerializableFunction<Instant, Instant>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.TimeTrigger
 
to(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new AvroIO.Write PTransform that's like this one but that writes to the file(s) with the given filename prefix.
to(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns an AvroIO.Write PTransform that writes to the file(s) with the given prefix.
to(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Specifies the table specification.
to(TableReference) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Specifies the table to be written to.
to(SerializableFunction<BoundedWindow, String>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
 
to(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation for the given table specification.
to(TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation for the given table.
to(SerializableFunction<BoundedWindow, String>) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation from a function that maps windows to table specifications.
to(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a new TextIO.Write PTransform that's like this one but that writes to the file(s) with the given filename prefix.
to(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a TextIO.Write PTransform that writes to the file(s) with the given prefix.
to(Sink<T>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Write
Creates a Write transform that writes to the given Sink.
toDoFn(DoFnWithContext<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
Create a DoFn that the DoFnWithContext.
toFilenamePrefix(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
Returns an XmlSink that writes to files with the given prefix.
Top - Class in com.google.cloud.dataflow.sdk.transforms
PTransforms for finding the largest (or smallest) set of elements in a PCollection, or the largest (or smallest) set of values associated with each key in a PCollection of KVs.
Top() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Top
 
Top.Largest<T extends Comparable<T>> - Class in com.google.cloud.dataflow.sdk.transforms
A Serializable Comparator that that uses the compared elements' natural ordering.
Top.Smallest<T extends Comparable<T>> - Class in com.google.cloud.dataflow.sdk.transforms
Serializable Comparator that that uses the reverse of the compared elements' natural ordering.
Top.TopCombineFn<T,ComparatorT extends Comparator<T> & Serializable> - Class in com.google.cloud.dataflow.sdk.transforms
CombineFn for Top transforms that combines a bunch of Ts into a single count-long List<T>, using compareFn to choose the largest Ts.
TopCombineFn(int, ComparatorT) - Constructor for class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
 
topic(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a new PubsubIO.Read PTransform that's like this one but reading from the give topic.
topic(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a PubsubIO.Read PTransform for reading from a Pubsub topic with the specified publisher topic.
topic(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new PubsubIO.Write PTransform that's like this one but writing to the given topic.
topic(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
The topic to publish to.
toString() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
 
toString() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
toString() - Method in class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
toString() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
toString() - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverride
 
toString() - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
 
toString() - Method in class com.google.cloud.dataflow.sdk.Pipeline
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
 
toString() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
 
toString() - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
 
toString() - Method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
 
toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.CodedTupleTag
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.KV
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
 
toString() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
 
toTableReference(SerializableFunction<BoundedWindow, TableReference>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
 
toTableReference(SerializableFunction<BoundedWindow, TableReference>) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Creates a write transformation from a function that maps windows to TableReference objects.
toTableSpec(TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO
Returns a canonical string representation of the TableReference.
TransformHierarchy - Class in com.google.cloud.dataflow.sdk.runners
Captures information about a collection of transformations and their associated PValues.
TransformHierarchy() - Constructor for class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
 
transforms - Variable in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
TransformTreeNode - Class in com.google.cloud.dataflow.sdk.runners
Provides internal tracking of transform relationships with helper methods for initialization and ordered visitation.
TransformTreeNode(TransformTreeNode, PTransform<?, ?>, String, PInput) - Constructor for class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Creates a new TransformTreeNode with the given parent and transform.
translate(TransformT, DataflowPipelineTranslator.TranslationContext) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TransformTranslator
 
translate(Pipeline, List<DataflowPackage>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
Translates a Pipeline into a JobSpecification.
traverseTopologically(Pipeline.PipelineVisitor) - Method in class com.google.cloud.dataflow.sdk.Pipeline
Trigger<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Triggers control when the elements for a specific key and window are output.
Trigger(List<Trigger<W>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
 
Trigger.MergeResult - Enum in com.google.cloud.dataflow.sdk.transforms.windowing
TriggerResult enumerates the possible result a trigger can have when it is merged.
Trigger.OnceTrigger<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Triggers that are guaranteed to fire at most once should extend from this, rather than the general Trigger class to indicate that behavior.
Trigger.OnElementContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Trigger.OnMergeContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Trigger.OnTimerContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Trigger.TriggerContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Information accessible to all of the callbacks that are executed on a Trigger.
Trigger.TriggerId<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Identifies a unique Trigger instance, by the window it is in and the identifier of the trigger within the trigger tree.
Trigger.TriggerResult - Enum in com.google.cloud.dataflow.sdk.transforms.windowing
TriggerResult enumerates the possible result a trigger can have when it is executed.
TriggerContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
 
TriggerId(W, int) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerId
 
triggering(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Sets a non-default trigger for this Window PTransform.
tryReturnRecordAt(boolean, Long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
tryReturnRecordAt(boolean, long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
tryReturnRecordAt(boolean, PositionT) - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Atomically determines whether a record at the given position can be returned and updates internal state.
trySplitAtPosition(Long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
trySplitAtPosition(long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
 
trySplitAtPosition(PositionT) - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
Atomically splits the current range [RangeTracker.getStartPosition(), RangeTracker.getStopPosition()) into a "primary" part [RangeTracker.getStartPosition(), splitPosition) and a "residual" part [splitPosition, RangeTracker.getStopPosition()), assuming the current last-consumed position is within [RangeTracker.getStartPosition(), splitPosition) (i.e., splitPosition has not been consumed yet).
TupleTag<V> - Class in com.google.cloud.dataflow.sdk.values
A TupleTag is a typed tag to use as the key of a heterogeneously typed tuple, like PCollectionTuple or Its generic type parameter allows tracking the static type of things stored in tuples.
TupleTag() - Constructor for class com.google.cloud.dataflow.sdk.values.TupleTag
Constructs a new TupleTag, with a fresh unique id.
TupleTag(String) - Constructor for class com.google.cloud.dataflow.sdk.values.TupleTag
Constructs a new TupleTag with the given id.
TupleTagList - Class in com.google.cloud.dataflow.sdk.values
A TupleTagList is an immutable list of heterogeneously typed TupleTags.
TypeDescriptor<T> - Class in com.google.cloud.dataflow.sdk.values
A description of a Java type, including actual generic parameters where possible.
TypeDescriptor() - Constructor for class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Creates a TypeDescriptor representing the type parameter T.
TypeDescriptor(Class<?>) - Constructor for class com.google.cloud.dataflow.sdk.values.TypeDescriptor
Creates a TypeDescriptor representing the type parameter T, which should resolve to a concrete type in the context of the class clazz.
TypedPValue<T> - Class in com.google.cloud.dataflow.sdk.values
A TypedPValue<T> is the abstract base class of things that store some number of values of type T.
TypedPValue(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.TypedPValue
 

U

UnboundedSource<OutputT,CheckpointMarkT extends UnboundedSource.CheckpointMark> - Class in com.google.cloud.dataflow.sdk.io
A Source that reads an unbounded amount of input and, because of that, supports some additional operations such as checkpointing, watermarks, and record ids.
UnboundedSource() - Constructor for class com.google.cloud.dataflow.sdk.io.UnboundedSource
 
UnboundedSource.CheckpointMark - Interface in com.google.cloud.dataflow.sdk.io
A marker representing the progress and state of an UnboundedSource.UnboundedReader.
UnboundedSource.UnboundedReader<OutputT> - Interface in com.google.cloud.dataflow.sdk.io
A Reader that reads an unbounded amount of input.
unfinishedSubTriggers() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
Returns an iterable over the unfinished sub-triggers of the current trigger.
usesSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
 

V

validate() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
 
validate(PInput) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Validates the current PTransform.
validate() - Method in class com.google.cloud.dataflow.sdk.io.ByteOffsetBasedSource
 
validate() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Validates that the delegate source is a valid source and that the channel factory is not null.
validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
Ensures the host and dataset are set.
validate() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink
Perform pipeline-construction-time validation.
validate() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
 
validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink
Ensures that the sink is valid and can be written to before the write operation begins.
validate() - Method in class com.google.cloud.dataflow.sdk.io.Source
Checks that this source is valid, before it can be used in a pipeline.
validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
Validates that the root element, class to bind to a JAXB context, and filenamePrefix have been set and that the class can be bound in a JAXB context.
validate() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
 
validate(Class<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsValidator
Validates that the passed PipelineOptions conforms to all the validation criteria from the passed in interface.
validate(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
 
validate(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
Called before invoking apply (which may be intercepted by the runner) to verify this transform is fully specified and applicable to the specified input.
Validation - Annotation Type in com.google.cloud.dataflow.sdk.options
Validation represents a set of annotations that can be used to annotate getter properties on PipelineOptions with information representing the validation criteria to be used when validating with the PipelineOptionsValidator.
Validation.Required - Annotation Type in com.google.cloud.dataflow.sdk.options
This criteria specifies that the value must be not null.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.annotations.Experimental.Kind
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.CreateDisposition
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.WriteDisposition
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSource.Mode
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.TextIO.CompressionType
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.ClusterManagerApiType
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.TeardownPolicy
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.Level
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.PipelineOptions.CheckEnabled
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergeResult
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerResult
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded
Returns the enum constant of this type with the specified name.
values() - Static method in enum com.google.cloud.dataflow.sdk.annotations.Experimental.Kind
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.CreateDisposition
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.WriteDisposition
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSource.Mode
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.io.TextIO.CompressionType
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.ClusterManagerApiType
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.TeardownPolicy
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.Level
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.options.PipelineOptions.CheckEnabled
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
Returns an array containing the constants of this enum type, in the order they are declared.
values - Variable in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
Values<V> - Class in com.google.cloud.dataflow.sdk.transforms
Values<V> takes a PCollection of KV<K, V>s and returns a PCollection<V> of the values.
values() - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergeResult
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerResult
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded
Returns an array containing the constants of this enum type, in the order they are declared.
VarIntCoder - Class in com.google.cloud.dataflow.sdk.coders
A VarIntCoder encodes Integers using between 1 and 5 bytes.
VarLongCoder - Class in com.google.cloud.dataflow.sdk.coders
A VarLongCoder encodes longs using between 1 and 10 bytes.
verifyDatasetPresence(BigQueryOptions, TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
Raises an exception describing reasons why the type may not be deterministically encoded using the given Schema, the directBinaryEncoder, and the ReflectDatumWriter or GenericDatumWriter.
verifyDeterministic() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
Throw Coder.NonDeterministicException if the coding is not deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
A delegate coder is deterministic if the underlying coder is deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.DeterministicStandardCoder
As a DeterministicStandardCoder is presumed deterministic, this method does nothing.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
Floating-point operations are not guaranteed to be deterministic, even if the storage format might be, so floating point representations are not recommended for use in operations that require deterministic inputs.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
A datastore kind can hold arbitrary Object instances, which makes the encoding non-deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
Encoding is not deterministic for the general Iterable case, as it depends upon the type of iterable.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.ListCoder
List sizes are always known, so ListIterable may be deterministic while the general IterableLikeCoder is not.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
Not all maps have a deterministic encoding.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.SetCoder
Not all sets have a deterministic encoding.
verifyDeterministic(String, Iterable<Coder<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
verifyDeterministic(String, Coder<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
TableCell can hold arbitrary Object instances, which makes the encoding non-deterministic.
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
 
verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
 
verifyTablePresence(BigQueryOptions, TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
 
View - Class in com.google.cloud.dataflow.sdk.transforms
Transforms for creating PCollectionViews from PCollections, for consuming the contents of those PCollections as side inputs to ParDo transforms.
View.AsIterable<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that produces a PCollectionView of a singleton PCollection yielding the single element it contains.
View.AsMap<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that produces a PCollectionView of a keyed PCollection yielding a map from each key to its unique associated value.
View.AsMultimap<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that produces a PCollectionView of a keyed PCollection yielding a map of keys to all associated values.
View.AsSingleton<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that produces a PCollectionView of a singleton PCollection yielding the single element it contains.
View.Concatenate<T> - Class in com.google.cloud.dataflow.sdk.transforms
Combiner that combines Ts into a single List<T> containing all inputs.
View.CreatePCollectionView<ElemT,ViewT> - Class in com.google.cloud.dataflow.sdk.transforms
Creates a primitive PCollectionView.
visit(Pipeline.PipelineVisitor, Set<PValue>) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
Visits all nodes in the transform hierarchy, in transitive order.
visit(Pipeline.PipelineVisitor, Set<PValue>) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
Visit the transform node.
visitTransform(TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
Called for each primitive transform after all of its topological predecessors and inputs have been visited.
visitTransform(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
visitValue(PValue, TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
Called for each value after the transform that produced the value has been visited.
visitValue(PValue, TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
 
VoidCoder - Class in com.google.cloud.dataflow.sdk.coders
A VoidCoder encodes Voids.

W

waitToFinish(long, TimeUnit, MonitoringUtil.JobMessagesHandler) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
Waits for the job to finish and return the final status.
weeks(int, int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
Returns a WindowFn that windows elements into periods measured by weeks.
window() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the window into which the input element has been assigned.
window() - Method in interface com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ExtraContextFactory
Construct the BoundedWindow to use within a DoFnWithContext that needs it.
window() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
The window that the current context is executing in.
window() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerId
 
Window - Class in com.google.cloud.dataflow.sdk.transforms.windowing
Window logically divides up or groups the elements of a PCollection into finite windows according to a WindowFn.
Window() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Window
 
Window.Bound<T> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
A PTransform that windows the elements of a PCollection<T>, into finite windows according to a user-specified WindowFn<T, B>.
Window.Remerge<T> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
PTransform that does not change assigned windows, but will cause windows to be merged again as part of the next GroupByKey.
Window.Triggering<T> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
An incomplete Window transform that has a trigger specified but has an unspecified accumulation mode.
Window.Unbound - Class in com.google.cloud.dataflow.sdk.transforms.windowing
An incomplete Window transform, with unbound input/output type.
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
 
windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
Returns the Coder used for serializing the windows used by this windowFn.
WindowFn<T,W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
The argument to the Window transform used to assign elements into windows and to determine how windows are merged.
WindowFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
 
WindowFn.AssignContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
WindowFn.MergeContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
WindowFnTestUtils - Class in com.google.cloud.dataflow.sdk.testing
A utility class for testing WindowFns.
WindowFnTestUtils() - Constructor for class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
 
windowingInternals() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
Returns the process context to use for implementing windowing.
windowingInternals() - Method in interface com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ExtraContextFactory
Construct the WindowingInternals to use within a DoFnWithContext that needs it.
windows() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
Returns the windows the current element was in, prior to this AssignFn being called.
windows() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.MergeContext
Returns the current set of windows.
withAllowedLateness(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
Override the amount of lateness allowed for data elements in the pipeline.
withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
Returns a new PubsubIO.Read PTransform that's like this one but that uses the given Coder<X> to decode each record into a value of type X.
withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
Creates and returns a PubsubIO.Read PTransform that uses the given Coder<T> to decode PubSub record into a value of type T.
withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
Returns a new PubsubIO.Write PTransform that's like this one but that uses the given Coder<X> to encode each of the elements of the input PCollection<X> into an output record.
withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
Returns a TextIO.Write PTransform that uses the given Coder<T> to encode each of the elements of the input PCollection<T> into an output PubSub record.
withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new TextIO.Read PTransform that's like this one but that uses the given Coder<X> to decode each of the lines of the file into a value of type X.
withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a TextIO.Read PTransform that uses the given Coder<T> to decode each of the lines of the file into a value of type T.
withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a new TextIO.Write PTransform that's like this one but that uses the given Coder<X> to encode each of the elements of the input PCollection<X> into an output text line.
withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a TextIO.Write PTransform that uses the given Coder<T> to encode each of the elements of the input PCollection<T> into an output text line.
withCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.TimestampedValues
Returns a Create.TimestampedValues PTransform like this one that uses the given Coder<T> to decode each of the objects into a value of type T.
withCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
Returns a Create.Values PTransform like this one that uses the given Coder<T> to decode each of the objects into a value of type T.
withCompressionType(TextIO.CompressionType) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new TextIO.Read PTransform that's like this one but reads from input sources using the specified compression type.
withCompressionType(TextIO.CompressionType) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a TextIO.Read PTransform that reads from a file with the specified compression type.
withCreateDisposition(BigQueryIO.Write.CreateDisposition) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Specifies options for creating the table.
withCreateDisposition(BigQueryIO.Write.CreateDisposition) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Specifies options for creating the table.
withDataset(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
Returns a Sink that is like this one, but will write to the specified dataset.
withDataset(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
withDecompression(CompressedSource.DecompressingChannelFactory) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
Return a CompressedSource that is like this one but will decompress its underlying file with the given CompressedSource.DecompressingChannelFactory.
withDefaultValue(T) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
Default value to return for windows with no value in them.
withEncodabilityTesting(boolean) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Enable runtime testing to verify that all values can be encoded.
withEpsilon(double) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Returns an ApproximateQuantilesCombineFn that's like this one except that it uses the specified epsilon value.
withExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
Produces a Proto2Coder like this one, but with the extensions from the given classes registered.
withExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
withFanout(int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
Returns a PTransform identical to this, but that uses an intermediate node to combine parts of the data to reduce load on the final global combine step.
withGapDuration(Duration) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
Creates a Sessions WindowFn with the specified gap duration.
withHost(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
Returns a Sink that is like this one, but will use the given host.
withHost(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
withHotKeyFanout(SerializableFunction<? super K, Integer>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
If a single key has disproportionately many values, it may become a bottleneck, especially in streaming mode.
withHotKeyFanout(int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
Like Combine.PerKey.withHotKeyFanout(SerializableFunction), but returning the given constant value for every key.
withHotKeyFanout(int) - Method in class com.google.cloud.dataflow.sdk.transforms.Count.Globally
Returns a PTransform identical to Globally(), but that uses an intermedate combining node to improve performance.
withKey(Object) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns a new ValueWithMetadata with the implicit key associated with this value set.
WithKeys<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
WithKeys<K, V> takes a PCollection<V>, and either a constant key of type K or a function from V to K, and returns a PCollection<KV<K, V>>, where each of the values in the input PCollection has been paired with either the constant key or a key computed from the value.
withMaxInputSize(long) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
Returns an ApproximateQuantilesCombineFn that's like this one except that it uses the specified maxNumElements value.
withMaxParallelism(int) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound
Returns a new IntraBundleParallelization PTransform like this one with the specified maximum concurrency level.
withMaxParallelism(int) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Unbound
Returns a new IntraBundleParallelization PTransform like this one with the specified maximum concurrency level.
withMaxParallelism(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization
Creates a IntraBundleParallelization PTransform with the specified maximum concurrency level.
withMinBundleSize(long) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
Returns an AvroSource that's like this one but uses the supplied minimum bundle size.
withMinBundleSize(long) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
Sets a parameter minBundleSize for the minimum bundle size of the source.
withMockEstimateSizeBytes(Long) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
For testing only.
withNumShards(int) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new AvroIO.Write PTransform that's like this one but that uses the provided shard count.
withNumShards(int) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns an AvroIO.Write PTransform that uses the provided shard count.
withNumShards(int) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a new TextIO.Write PTransform that's like this one but that uses the provided shard count.
withNumShards(int) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a TextIO.Write PTransform that uses the provided shard count.
withOffset(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
Partitions the timestamp space into half-open intervals of the form [N * size + offset, (N + 1) * size + offset), where 0 is the epoch.
withOffset(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
Assigns timestamps into half-open intervals of the form [N * period + offset, N * period + offset + size).
withoutDefaults() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
Returns a PTransform identical to this, but that does not attempt to provide a default value in the case of empty input.
withoutDefaults() - Method in class com.google.cloud.dataflow.sdk.transforms.Count.Globally
Returns a PTransform identical to Globally(), but that does not attempt to provide a default value in the case of empty input.
withOutputTags(TupleTag<OutputT>, TupleTagList) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
Returns a new multi-output ParDo PTransform that's like this PTransform but with the specified main and side output tags.
withOutputTags(TupleTag<OutputT>, TupleTagList) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new multi-output ParDo transform that's like this transform but with the specified main and side output tags.
withOutputTags(TupleTag<OutputT>, TupleTagList) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a multi-output ParDo PTransform whose output PCollections will be referenced using the given main output and side output tags.
withoutSharding() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new AvroIO.Write PTransform that's like this one but that forces a single file as output.
withoutSharding() - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns an AvroIO.Write PTransform that forces a single file as output.
withoutSharding() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a new TextIO.Write PTransform that's like this one but that forces a single file as output.
withoutSharding() - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a TextIO.Write PTransform that forces a single file as output.
withoutStrictParsing() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
During parsing of the arguments, we will skip over improperly formatted and unknown arguments.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new TextIO.Read PTransform that's like this one but that has GCS input path validation on pipeline creation disabled.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns a AvroIO.Read PTransform that has GCS path validation on pipeline creation disabled.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new TextIO.Write PTransform that's like this one but that has GCS output path validation on pipeline creation disabled.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns a AvroIO.Write PTransform that has GCS path validation on pipeline creation disabled.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
Disable table validation.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
Disables BigQuery table validation, which is enabled by default.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Disable table validation.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Disables BigQuery table validation, which is enabled by default.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
Returns a new TextIO.Read PTransform that's like this one but that has GCS path validation on pipeline creation disabled.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
Returns a TextIO.Read PTransform that has GCS path validation on pipeline creation disabled.
withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a new TextIO.Write PTransform that's like this one but that has GCS output path validation on pipeline creation disabled.
withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a TextIO.Write PTransform that has GCS path validation on pipeline creation disabled.
withQuery(DatastoreV1.Query) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
 
withRecordClass(Class<T>) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
Sets a JAXB annotated class that can be populated using a record of the provided XML file.
withRecordElement(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
Sets name of the record element of the XML document.
withRootElement(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
Returns an XmlSink that writes XML files with an enclosing root element of the supplied name.
withRootElement(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
Sets name of the root element of the XML document.
withSchema(Class<X>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new AvroIO.Read PTransform that's like this one but that reads Avro file(s) containing records whose type is the specified Avro-generated class.
withSchema(Schema) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new AvroIO.Read PTransform that's like this one but that reads Avro file(s) containing records of the specified schema.
withSchema(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
Returns a new AvroIO.Read PTransform that's like this one but that reads Avro file(s) containing records of the specified schema in a JSON-encoded string form.
withSchema(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns an AvroIO.Read PTransform that reads Avro file(s) containing records whose type is the specified Avro-generated class.
withSchema(Schema) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns an AvroIO.Read PTransform that reads Avro file(s) containing records of the specified schema.
withSchema(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
Returns an AvroIO.Read PTransform that reads Avro file(s) containing records of the specified schema in a JSON-encoded string form.
withSchema(Class<X>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new AvroIO.Write PTransform that's like this one but that writes to Avro file(s) containing records whose type is the specified Avro-generated class.
withSchema(Schema) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new AvroIO.Write PTransform that's like this one but that writes to Avro file(s) containing records of the specified schema.
withSchema(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new AvroIO.Write PTransform that's like this one but that writes to Avro file(s) containing records of the specified schema in a JSON-encoded string form.
withSchema(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns an AvroIO.Write PTransform that writes Avro file(s) containing records whose type is the specified Avro-generated class.
withSchema(Schema) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns an AvroIO.Write PTransform that writes Avro file(s) containing records of the specified schema.
withSchema(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns an AvroIO.Write PTransform that writes Avro file(s) containing records of the specified schema in a JSON-encoded string form.
withSchema(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
Returns an AvroSource that's like this one but reads files containing records that conform to the given schema.
withSchema(Schema) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
Returns an AvroSource that's like this one but reads files containing records that conform to the given schema.
withSchema(Class<X>) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
Returns an AvroSource that's like this one but reads files containing records of the type of the given class.
withSchema(TableSchema) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Specifies the table schema, used if the table is created.
withSchema(TableSchema) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Specifies a table schema to use in table creation.
withSerializabilityTesting(boolean) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Enable runtime testing to verify that all functions and Coder instances can be serialized.
withShardNameTemplate(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new AvroIO.Write PTransform that's like this one but that uses the given shard name template.
withShardNameTemplate(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns an AvroIO.Write PTransform that uses the given shard name template.
withShardNameTemplate(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a new TextIO.Write PTransform that's like this one but that uses the given shard name template.
withShardNameTemplate(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a TextIO.Write PTransform that uses the given shard name template.
withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
Returns a new ParDo PTransform that's like this PTransform but with the specified side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
Returns a new ParDo PTransform that's like this PTransform but with the specified side inputs.
withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
Returns a new multi-output ParDo PTransform that's like this PTransform but with the specified side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
Returns a new multi-output ParDo PTransform that's like this PTransform but with the specified side inputs.
withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo transform that's like this transform but with the specified side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
Returns a new ParDo transform that's like this transform but with the specified side inputs.
withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo transform that's like this transform but with the specified side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
Returns a new multi-output ParDo transform that's like this transform but with the specified side inputs.
withSideInputs(PCollectionView<?>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo PTransform with the given side inputs.
withSideInputs(Iterable<? extends PCollectionView<?>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
Creates a ParDo with the given side inputs.
withSingletonValues() - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap
Deprecated.
this method simply returns this AsMap unmodified
withStartingDay(int, int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
withStartingMonth(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
withStartingYear(int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
withSuffix(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
Returns a new AvroIO.Write PTransform that's like this one but that writes to the file(s) with the given filename suffix.
withSuffix(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
Returns an AvroIO.Write PTransform that writes to the file(s) with the given filename suffix.
withSuffix(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
Returns a new TextIO.Write PTransform that's like this one but that writes to the file(s) with the given filename suffix.
withSuffix(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
Returns a TextIO.Write PTransform that writes to the file(s) with the given filename suffix.
withTimeZone(DateTimeZone) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
 
withTimeZone(DateTimeZone) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
 
withTimeZone(DateTimeZone) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
 
withUnorderednessTesting(boolean) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
Enable runtime testing to verify that functions do not depend on order of the elements.
withValidation() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
After creation we will validate that PipelineOptions conforms to all the validation criteria from <T>.
withValidation() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
After creation we will validate that PipelineOptions conforms to all the validation criteria from <T>.
withValue(T) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
Returns a new ValueWithMetadata that is a copy of this one, but with a different value.
withWriteDisposition(BigQueryIO.Write.WriteDisposition) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
Specifies options for writing the table.
withWriteDisposition(BigQueryIO.Write.WriteDisposition) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
Specifies options for writing to the table.
Write() - Constructor for class com.google.cloud.dataflow.sdk.io.AvroIO.Write
 
Write() - Constructor for class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
 
Write() - Constructor for class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
 
write(T) - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
Called for each value in the bundle.
Write() - Constructor for class com.google.cloud.dataflow.sdk.io.TextIO.Write
 
write() - Static method in class com.google.cloud.dataflow.sdk.io.XmlSink
Returns a builder for an XmlSink.
write(T) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Writes a value to the stream.
Write - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that writes to a Sink.
Write() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Write
 
Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.transforms
A PTransform that writes to a Sink.
writeFooter() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Writes footer at the end of output files.
writeFooter() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Writes the root element closing tag.
writeHeader() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
Writes header at the beginning of output files.
writeHeader() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
Writes the root element opening tag.
writeOf(Class<T>, String, String) - Static method in class com.google.cloud.dataflow.sdk.io.XmlSink
Returns an XmlSink that writes objects as XML entities.
WriteOperation() - Constructor for class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
 
Writer() - Constructor for class com.google.cloud.dataflow.sdk.io.Sink.Writer
 
writeTo(String) - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
Returns a new Write transform that will write to a Sink.

X

XML_EXTENSION - Static variable in class com.google.cloud.dataflow.sdk.io.XmlSink
 
XmlSink - Class in com.google.cloud.dataflow.sdk.io
A Sink that outputs records as XML-formatted elements.
XmlSink() - Constructor for class com.google.cloud.dataflow.sdk.io.XmlSink
 
XmlSink.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
A FileBasedSink that writes objects as XML elements.
XmlSink.XmlWriteOperation<T> - Class in com.google.cloud.dataflow.sdk.io
XmlSink.XmlWriter<T> - Class in com.google.cloud.dataflow.sdk.io
A Sink.Writer that can write objects as XML elements.
XmlSource<T> - Class in com.google.cloud.dataflow.sdk.io
A source that can be used to read XML files.
XmlWriteOperation(XmlSink.Bound<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriteOperation
 
XmlWriter(XmlSink.XmlWriteOperation<T>, Marshaller) - Constructor for class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
 

Y

years(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
Returns a WindowFn that windows elements into periods measured by years.
A B C D E F G H I J K L M N O P R S T U V W X Y 
Skip navigation links