- AccumulatingCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
-
- accumulatingFiredPanes() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
-
Returns a new Window PTransform that uses the registered WindowFn and
Triggering behavior, and that accumulates elements in a pane after they are triggered.
- accumulatingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
Returns a new Window PTransform that uses the registered WindowFn and
Triggering behavior, and that accumulates elements in a pane after they are triggered.
- accumulatingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
-
Returns a new Window PTransform that uses the registered WindowFn and
Triggering behavior, and that accumulates elements in a pane after they are triggered.
- add(Long) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique
-
Adds a value to the heap, returning whether the value is (large enough
to be) in the heap.
- add(String, String) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
-
Register the given string display metadata.
- add(String, long) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
-
Register the given numeric display metadata.
- add(String, double) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
-
Register the given floating point display metadata.
- add(String, Instant) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
-
Register the given timestamp display metadata.
- add(String, Duration) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
-
Register the given duration display metadata.
- add(String, Class<?>) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
-
Register the given class display metadata.
- addCollectionToSingletonOutput(String, PValue, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds an output with the given name to the previously added
CollectionToSingleton Dataflow step, consuming the specified
input PValue and producing the specified output
PValue.
- addComposite(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Adds a composite operation to the transform node.
- addEncodingInput(Coder<?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Sets the encoding for the current Dataflow step.
- addExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
-
- addExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
-
- addInput(String, Boolean) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds an input with the given name and value to the current
Dataflow step.
- addInput(String, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds an input with the given name and value to the current
Dataflow step.
- addInput(String, Long) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds an input with the given name and value to the current
Dataflow step.
- addInput(String, PInput) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds an input with the given name to the previously added Dataflow
step, coming from the specified input PValue.
- addInput(String, Map<String, Object>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds an input that is a dictionary of strings to objects.
- addInput(String, List<? extends Map<String, Object>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds an input that is a list of objects.
- addInput(TransformTreeNode, PInput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
-
Adds an input to the given node.
- addInput(ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique, T) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
-
- addInput(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn.Accumulator
-
Adds the given input value to this accumulator, modifying
this accumulator.
- addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn
-
- addInput(double[], Double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
-
- addInput(Combine.Holder<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
-
- addInput(int[], Integer) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
-
- addInput(long[], Long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
-
- addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
-
Adds the given input value to the given accumulator, returning the
new accumulator value.
- addInput(List<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
-
- addInput(K, AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
-
Adds the given input value to the given accumulator, returning the new accumulator value.
- addInput(Object[], DataT) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
-
- addInput(Object[], DataT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
-
- addInput(K, Object[], DataT) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
-
- addInput(K, Object[], DataT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
-
- addInput(AccumT, InputT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
-
Adds the given input value to the given accumulator, returning the
new accumulator value.
- addInput(K, AccumT, InputT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
-
Adds the given input value to the given accumulator, returning the new accumulator value.
- addInput(Top.BoundedHeap<KV<Integer, T>, SerializableComparator<KV<Integer, T>>>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
-
- addInputProducer(PValue, TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Adds an input to the transform node.
- addInputsRandomly(PerKeyCombineFnRunner<? super K, ? super InputT, AccumT, ?>, K, Iterable<InputT>, Random, DoFn<?, ?>.ProcessContext) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
-
Create a random list of accumulators from the given list of values.
- addOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds an output with the given name to the previously added
Dataflow step, producing the specified output PValue,
including its Coder if a TypedPValue.
- addOverrideForClass(Class<?>, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
-
Overrides the default log level for the passed in class.
- addOverrideForName(String, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
-
Overrides the default log level for the passed in name.
- addOverrideForPackage(Package, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
-
Overrides the default log level for the passed in package.
- addStep(PTransform<?, ?>, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds a step to the Dataflow workflow for the given transform, with
the given Dataflow step type.
- addStep(PTransform<?, ? extends PValue>, Step) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds a pre-defined step to the Dataflow workflow.
- addTraceFor(AbstractGoogleClient, String) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
-
- addTraceFor(AbstractGoogleClientRequest<?>, String) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
-
- addValue(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator
-
Adds a new value into the Aggregator.
- addValueInternal(PValue) - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- addValueOnlyOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Adds an output with the given name to the previously added
Dataflow step, producing the specified output PValue,
including its Coder if a TypedPValue.
- advance() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
-
- advance() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
-
- advance() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
-
Advances the reader to the next valid record.
- advance() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
-
Advances the reader to the next valid record.
- advanceImpl() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
-
- advanceImpl() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
-
Advances to the next record and returns true, or returns false if there is no next
record.
- AfterAll<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
Create a
Trigger that fires and finishes once after all of its sub-triggers have fired.
- AfterDelayFromFirstElement<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A base class for triggers that happen after a processing time delay from the arrival
of the first element in a pane.
- AfterDelayFromFirstElement(TimeDomain, List<SerializableFunction<Instant, Instant>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- AfterEach<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A composite
Trigger that executes its sub-triggers in order.
- AfterFirst<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
Create a composite
Trigger that fires once after at least one of its sub-triggers have
fired.
- AfterPane<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
Triggers that fire based on properties of the elements in the current pane.
- AfterProcessingTime<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
AfterProcessingTime triggers fire based on the current processing time.
- AfterWatermark<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
AfterWatermark triggers fire based on progress of the system watermark.
- AfterWatermark.AfterWatermarkEarly<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
-
Interface for building an AfterWatermarkTrigger with early firings already filled in.
- AfterWatermark.AfterWatermarkLate<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
-
Interface for building an AfterWatermarkTrigger with late firings already filled in.
- AfterWatermark.FromEndOfWindow<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A watermark trigger targeted relative to the end of the window.
- Aggregator<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
-
An Aggregator<InputT> enables monitoring of values of type InputT,
to be combined across all bundles.
- AggregatorPipelineExtractor - Class in com.google.cloud.dataflow.sdk.runners
-
- AggregatorPipelineExtractor(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorPipelineExtractor
-
Creates an
AggregatorPipelineExtractor for the given
Pipeline.
- AggregatorRetrievalException - Exception in com.google.cloud.dataflow.sdk.runners
-
Signals that an exception has occurred while retrieving
Aggregators.
- AggregatorRetrievalException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.AggregatorRetrievalException
-
Constructs a new AggregatorRetrievalException with the specified detail message and
cause.
- AggregatorRetriever - Class in com.google.cloud.dataflow.sdk.transforms
-
- AggregatorValues<T> - Class in com.google.cloud.dataflow.sdk.runners
-
A collection of values associated with an
Aggregator.
- AggregatorValues() - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorValues
-
- alignedTo(Duration, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
Aligns timestamps to the smallest multiple of size since the offset greater
than the timestamp.
- alignedTo(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
Aligns the time to be the smallest multiple of size greater than the timestamp
since the epoch.
- ALL_CONTEXTS - Static variable in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
All the contexts, for use in test cases.
- ALL_KEYS - Static variable in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
-
The range of all keys, with empty start and end keys.
- allowsDynamicSplitting() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
Whether this source should allow dynamic splitting of the offset ranges.
- and(TupleTag<V>, List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
-
Returns a new
CoGbkResult based on this, with the given tag and given data
added to it.
- and(TupleTag<V>, PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
-
Returns a new KeyedPCollectionTuple<K> that is the same as this,
appended with the given PCollection.
- and(PCollection.IsBounded) - Method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded
-
Returns the composed IsBounded property.
- and(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
- and(Iterable<PCollection<T>>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
- and(TupleTag<T>, PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
-
- and(TupleTag<?>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
-
- and(List<TupleTag<?>>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
-
- any(long) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample
-
Sample#any(long) takes a PCollection<T> and a limit, and
produces a new PCollection<T> containing up to limit
elements of the input PCollection.
- applicableTo(PCollection<?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
-
- ApplicationNameOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Options that allow setting the application name.
- AppliedPTransform<InputT extends PInput,OutputT extends POutput,TransformT extends PTransform<? super InputT,OutputT>> - Class in com.google.cloud.dataflow.sdk.transforms
-
Represents the application of a
PTransform to a specific input to produce
a specific output.
- apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.coders.DelegateCoder.CodingFunction
-
- apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
- apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
- apply(PCollection<TableRow>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
- apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
-
- apply(PCollection<KV<ByteString, Iterable<Mutation>>>) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
-
- apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.BoundedCountingInput
-
- apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput
-
- apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
- apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded
-
- apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
-
- apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.Write.Bound
-
- apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
-
- apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
-
Applies the given transform to the input.
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
-
- apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
-
- apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner
-
Applies a transform to the given input, returning the output.
- apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
-
- apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
-
Applies the binary operation to the two operands, returning the result.
- apply(V, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
-
Applies the binary operation to the two operands, returning the result.
- apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
-
Applies the binary operation to the two operands, returning the result.
- apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
-
Applies the binary operation to the two operands, returning the result.
- apply(Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
-
Applies this CombineFn to a collection of input values
to produce a combined output value.
- apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
-
- apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView
-
- apply(PCollection<? extends KV<K, ? extends Iterable<InputT>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
-
- apply(K, Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
-
Applies this KeyedCombineFn to a key and a collection
of input values to produce a combined output value.
- apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
-
- apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKeyWithHotKeyFanout
-
- apply(K, Iterable<? extends InputT>, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
-
Applies this KeyedCombineFnWithContext to a key and a collection
of input values to produce a combined output value.
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Count.PerElement
-
- apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.TimestampedValues
-
- apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Filter
-
- apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements
-
- apply(PCollection<? extends Iterable<T>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenIterables
-
- apply(PCollectionList<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenPCollectionList
-
- apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
-
- apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupAlsoByWindow
-
- apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
-
- apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.ReifyTimestampsAndWindows
-
- apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.SortValuesByTimestamp
-
- apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound
-
- apply(KeyedPCollectionTuple<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGroupByKey
-
- apply(PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
-
- apply(String, PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
-
Applies the given
PTransform to this input
KeyedPCollectionTuple and returns
its
OutputT.
- apply(PCollection<? extends KV<K, ?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Keys
-
- apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.KvSwap
-
- apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.MapElements
-
- apply(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
-
- apply(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
-
- apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
-
- apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Partition
-
- apply(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
-
Applies this PTransform on the given InputT, and returns its
Output.
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates.WithRepresentativeValues
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.SampleAny
-
- apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.SerializableFunction
-
Returns the result of invoking this function on the given input.
- apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
-
- apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
-
- apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
-
- apply(PCollection<? extends KV<?, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Values
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsIterable
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsList
-
- apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap
-
- apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMultimap
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
-
- apply(PCollection<ElemT>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.CreatePCollectionView
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Remerge
-
- apply(PCollection<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithKeys
-
- apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithTimestamps
-
- apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin
-
- apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin
-
Applies the given
PTransform to this input
PBegin,
using
name to identify this specific application of the transform.
- apply(PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
- apply(String, PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
Applies the given
PTransform to this input
PCollection,
using
name to identify this specific application of the transform.
- apply(PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
- apply(String, PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
Applies the given
PTransform to this input
PCollectionList,
using
name to identify this specific application of the transform.
- apply(PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
-
- apply(String, PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
-
- applyTransform(InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline
-
- applyTransform(String, InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline
-
Applies the given PTransform to this input InputT and returns
its OutputT.
- ApproximateQuantiles - Class in com.google.cloud.dataflow.sdk.transforms
-
PTransforms for getting an idea of a PCollection's
data distribution using approximate N-tiles (e.g.
- ApproximateQuantiles.ApproximateQuantilesCombineFn<T,ComparatorT extends Comparator<T> & Serializable> - Class in com.google.cloud.dataflow.sdk.transforms
-
The ApproximateQuantilesCombineFn combiner gives an idea
of the distribution of a collection of values using approximate
N-tiles.
- ApproximateUnique - Class in com.google.cloud.dataflow.sdk.transforms
-
PTransforms for estimating the number of distinct elements
in a PCollection, or the number of distinct values
associated with each key in a PCollection of KVs.
- ApproximateUnique() - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
-
- ApproximateUnique.ApproximateUniqueCombineFn<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
CombineFn that computes an estimate of the number of
distinct values that were combined.
- ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique - Class in com.google.cloud.dataflow.sdk.transforms
-
A heap utility class to efficiently track the largest added elements.
- ApproximateUniqueCombineFn(long, Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
-
- areAllSubtriggersFinished() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
-
Returns true if all the sub-triggers of the current trigger are marked finished.
- as(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
-
Transforms this object into an object of type <T> saving each property
that has been manipulated.
- as(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
Creates and returns an object that implements <T>.
- as(Class<T>) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
-
Creates and returns an object that implements <T> using the values configured on
this builder during construction.
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
- asCloudObject() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
-
Returns the CloudObject that represents this Coder.
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
-
Returns the CloudObject that represents this Coder.
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
-
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
-
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
-
Deprecated.
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
-
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
-
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
-
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
-
- asCloudObject() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
-
- asIterable() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
-
- asKeyedFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
-
- asKeyedFn() - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
-
Converts this
GloballyCombineFn into an equivalent
CombineFnBase.PerKeyCombineFn that ignores the keys passed to it and
combines the values according to this
GloballyCombineFn.
- asKeyedFn() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
-
- asList() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
-
- asMap() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData
-
- asMap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
-
- asMultimap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
-
- asOutputReference(PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Encode a PValue reference as an output reference.
- asPath() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
-
Returns the string representation of this subscription as a path used in the Cloud Pub/Sub
API.
- asPath() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
-
Returns the string representation of this topic as a path used in the Cloud Pub/Sub
API.
- assertSourcesEqualReferenceSource(BoundedSource<T>, List<? extends BoundedSource<T>>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
-
Given a reference Source and a list of Sources, assert that the union of
the records read from the list of sources is equal to the records read from the reference
source.
- assertSplitAtFractionBehavior(BoundedSource<T>, int, double, SourceTestUtils.ExpectedSplitOutcome, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
-
- assertSplitAtFractionExhaustive(BoundedSource<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
-
Asserts that for each possible start position,
BoundedSource.BoundedReader#splitAtFraction at every interesting fraction (halfway
between two fractions that differ by at least one item) can be called successfully and the
results are consistent if a split succeeds.
- assertSplitAtFractionFails(BoundedSource<T>, int, double, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
-
Asserts that the source's reader fails to splitAtFraction(fraction)
after reading numItemsToReadBeforeSplit items.
- assertSplitAtFractionSucceedsAndConsistent(BoundedSource<T>, int, double, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
-
Verifies some consistency properties of
BoundedSource.BoundedReader#splitAtFraction on the given source.
- assertUnstartedReaderReadsSameAsItsSource(BoundedSource.BoundedReader<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
-
Assert that a Reader returns a Source that, when read from, produces the same
records as the reader.
- AssignContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
-
- assignedWindows(WindowFn<T, W>, long) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
-
- assignMessageTimestamp(PubsubMessage, String, Clock) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO
-
Returns the
Instant that corresponds to the timestamp in the supplied
PubsubMessage under the specified
ink label.
- assignOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn
-
Returns the output timestamp to use for data depending on the given
inputTimestamp in the specified window.
- assignOutputTime(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
-
Returns the output timestamp to use for data in the specified window.
- assignOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
-
Returns the output timestamp to use for data depending on the given
inputTimestamp in the specified window.
- assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
-
- assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
-
- assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
-
Returns true if this WindowFn assigns each element to a single window.
- assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
-
- assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
-
- assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
-
- assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
-
Returns the single window to which elements with this timestamp belong.
- assignWindows(WindowFn<Object, GlobalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
-
- assignWindows(WindowFn<Object, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
-
- assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
-
- assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
-
- assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
-
- assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
-
Given a timestamp and element, returns the set of windows into which it
should be placed.
- asSingleton() - Static method in class com.google.cloud.dataflow.sdk.transforms.View
-
- asSingletonView() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
-
Returns a
PTransform that produces a
PCollectionView
whose elements are the result of combining elements per-window in
the input
PCollection.
- asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
-
- asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
-
- asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
-
- asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
-
- AtomicCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
-
- AtomicCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.AtomicCoder
-
- AvroCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
-
A
Coder using Avro binary format.
- AvroCoder(Class<T>, Schema) - Constructor for class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
- AvroIO - Class in com.google.cloud.dataflow.sdk.io
-
- AvroIO.Read - Class in com.google.cloud.dataflow.sdk.io
-
A root
PTransform that reads from an Avro file (or multiple Avro
files matching a pattern) and returns a
PCollection containing
the decoding of each record.
- AvroIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
-
A
PTransform that reads from an Avro file (or multiple Avro
files matching a pattern) and returns a bounded
PCollection containing
the decoding of each record.
- AvroIO.Write - Class in com.google.cloud.dataflow.sdk.io
-
A root
PTransform that writes a
PCollection to an Avro file (or
multiple Avro files matching a sharding pattern).
- AvroIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
-
A
PTransform that writes a bounded
PCollection to an Avro file (or
multiple Avro files matching a sharding pattern).
- AvroReader(AvroSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
-
Reads Avro records of type T from the specified source.
- AvroSource<T> - Class in com.google.cloud.dataflow.sdk.io
-
- AvroSource.AvroReader<T> - Class in com.google.cloud.dataflow.sdk.io
-
- CalendarWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A collection of
WindowFns that windows values into calendar-based
windows such as spans of days, months, or years.
- CalendarWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
-
- CalendarWindows.DaysWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A
WindowFn that windows elements into periods measured by days.
- CalendarWindows.MonthsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A
WindowFn that windows elements into periods measured by months.
- CalendarWindows.YearsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A
WindowFn that windows elements into periods measured by years.
- cancel() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
-
Cancels the job.
- CannotProvideCoderException - Exception in com.google.cloud.dataflow.sdk.coders
-
- CannotProvideCoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
-
- CannotProvideCoderException(String, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
-
- CannotProvideCoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
-
- CannotProvideCoderException(String, Throwable, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
-
- CannotProvideCoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
-
- CannotProvideCoderException(Throwable, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
-
- CannotProvideCoderException.ReasonCode - Enum in com.google.cloud.dataflow.sdk.coders
-
Indicates the reason that
Coder inference failed.
- clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
-
- clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
-
- clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
Clear any state associated with this trigger in the given window.
- clearOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Clears the record of the elements output so far to the main output.
- clearSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Clears the record of the elements output so far to the side
output with the given tag.
- cloneAs(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
-
Makes a deep clone of this object, and transforms the cloned object into the specified
type kls.
- close() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
-
- close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
-
Closes the channel and return the bundle result.
- close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
-
- close() - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
-
Finishes writing the bundle.
- close() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
-
Closes the reader.
- CloudDebuggerOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Options for controlling Cloud Debugger.
- Coder<T> - Interface in com.google.cloud.dataflow.sdk.coders
-
A
Coder<T> defines how to encode and decode values of type
T into
byte streams.
- Coder.Context - Class in com.google.cloud.dataflow.sdk.coders
-
The context in which encoding or decoding is being done.
- Coder.NonDeterministicException - Class in com.google.cloud.dataflow.sdk.coders
-
Exception thrown by
Coder.verifyDeterministic() if the encoding is
not deterministic, including details of why the encoding is not deterministic.
- coderAllowsEncoding(Coder<T>, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- coderConsistentWithEquals(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- coderConsistentWithEqualsInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- coderDecodeEncodeContentsEqual(Coder<CollectionT>, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
Verifies that for the given Coder<Collection<T>>,
and value of type Collection<T>, encoding followed by decoding yields an
equal value of type Collection<T>, in any Coder.Context.
- coderDecodeEncodeContentsEqualInContext(Coder<CollectionT>, Coder.Context, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
Verifies that for the given Coder<Collection<T>>,
and value of type Collection<T>, encoding followed by decoding yields an
equal value of type Collection<T>, in the given Coder.Context.
- coderDecodeEncodeContentsInSameOrder(Coder<IterableT>, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
Verifies that for the given Coder<Collection<T>>,
and value of type Collection<T>, encoding followed by decoding yields an
equal value of type Collection<T>, in any Coder.Context.
- coderDecodeEncodeContentsInSameOrderInContext(Coder<IterableT>, Coder.Context, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
Verifies that for the given Coder<Iterable<T>>,
and value of type Iterable<T>, encoding followed by decoding yields an
equal value of type Collection<T>, in the given Coder.Context.
- coderDecodeEncodeEqual(Coder<T>, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
Verifies that for the given Coder<T>,
and value of type T, encoding followed by decoding yields an
equal value of type T, in any Coder.Context.
- coderDecodeEncodeEqualInContext(Coder<T>, Coder.Context, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
Verifies that for the given Coder<T>, Coder.Context,
and value of type T, encoding followed by decoding yields an
equal value of type T.
- coderDecodesBase64(Coder<T>, String, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- coderDecodesBase64(Coder<T>, List<String>, List<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- coderDecodesBase64ContentsEqual(Coder<IterableT>, String, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- coderDecodesBase64ContentsEqual(Coder<IterableT>, List<String>, List<IterableT>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- coderDeterministic(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
Verifies that for the given Coder<T>, and values of
type T, if the values are equal then the encoded bytes are equal, in any
Coder.Context.
- coderDeterministicInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
Verifies that for the given Coder<T>, Coder.Context, and values of
type T, if the values are equal then the encoded bytes are equal.
- coderEncodesBase64(Coder<T>, T, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- coderEncodesBase64(Coder<T>, List<T>, List<String>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- CoderException - Exception in com.google.cloud.dataflow.sdk.coders
-
An
Exception thrown if there is a problem encoding or decoding a value.
- CoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException
-
- CoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException
-
- CoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException
-
- CoderFactories - Class in com.google.cloud.dataflow.sdk.coders
-
Static utility methods for creating and working with
Coders.
- CoderFactory - Interface in com.google.cloud.dataflow.sdk.coders
-
- coderHasEncodingId(Coder<T>, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- CoderProperties - Class in com.google.cloud.dataflow.sdk.testing
-
Properties for use in
Coder tests.
- CoderProperties() - Constructor for class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- CoderProvider - Interface in com.google.cloud.dataflow.sdk.coders
-
- coderProvider() - Static method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
-
Deprecated.
- coderProvider() - Static method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
-
- CoderProviders - Class in com.google.cloud.dataflow.sdk.coders
-
- CoderRegistry - Class in com.google.cloud.dataflow.sdk.coders
-
A
CoderRegistry allows registering the default
Coder to use for a Java class,
and looking up and instantiating the default
Coder for a Java type.
- CoderRegistry() - Constructor for class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
- coderSerializable(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- CoGbkResult - Class in com.google.cloud.dataflow.sdk.transforms.join
-
- CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
-
A row in the
PCollection resulting from a
CoGroupByKey transform.
- CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>, int) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
-
- CoGbkResult.CoGbkResultCoder - Class in com.google.cloud.dataflow.sdk.transforms.join
-
- CoGbkResultSchema - Class in com.google.cloud.dataflow.sdk.transforms.join
-
- CoGbkResultSchema(TupleTagList) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
-
Builds a schema from a tuple of TupleTag<?>s.
- CoGroupByKey<K> - Class in com.google.cloud.dataflow.sdk.transforms.join
-
- CollectionCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
-
- CollectionCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.CollectionCoder
-
- com.google.cloud.dataflow.sdk - package com.google.cloud.dataflow.sdk
-
Provides a simple, powerful model for building both batch and
streaming parallel data processing
Pipelines.
- com.google.cloud.dataflow.sdk.annotations - package com.google.cloud.dataflow.sdk.annotations
-
Defines annotations used across the SDK.
- com.google.cloud.dataflow.sdk.coders - package com.google.cloud.dataflow.sdk.coders
-
Defines
Coders
to specify how data is encoded to and decoded from byte strings.
- com.google.cloud.dataflow.sdk.coders.protobuf - package com.google.cloud.dataflow.sdk.coders.protobuf
-
Defines a
Coder
for Protocol Buffers messages,
ProtoCoder.
- com.google.cloud.dataflow.sdk.io - package com.google.cloud.dataflow.sdk.io
-
- com.google.cloud.dataflow.sdk.io.bigtable - package com.google.cloud.dataflow.sdk.io.bigtable
-
Defines transforms for reading and writing from Google Cloud Bigtable.
- com.google.cloud.dataflow.sdk.io.range - package com.google.cloud.dataflow.sdk.io.range
-
Provides thread-safe helpers for implementing dynamic work rebalancing in position-based
bounded sources.
- com.google.cloud.dataflow.sdk.options - package com.google.cloud.dataflow.sdk.options
-
- com.google.cloud.dataflow.sdk.runners - package com.google.cloud.dataflow.sdk.runners
-
- com.google.cloud.dataflow.sdk.testing - package com.google.cloud.dataflow.sdk.testing
-
Defines utilities for unit testing Dataflow pipelines.
- com.google.cloud.dataflow.sdk.transforms - package com.google.cloud.dataflow.sdk.transforms
-
Defines
PTransforms for transforming
data in a pipeline.
- com.google.cloud.dataflow.sdk.transforms.display - package com.google.cloud.dataflow.sdk.transforms.display
-
- com.google.cloud.dataflow.sdk.transforms.join - package com.google.cloud.dataflow.sdk.transforms.join
-
Defines the
CoGroupByKey transform
for joining multiple PCollections.
- com.google.cloud.dataflow.sdk.transforms.windowing - package com.google.cloud.dataflow.sdk.transforms.windowing
-
Defines the
Window transform
for dividing the elements in a PCollection into windows, and the
Trigger for controlling when those
elements are output.
- com.google.cloud.dataflow.sdk.values - package com.google.cloud.dataflow.sdk.values
-
- Combine - Class in com.google.cloud.dataflow.sdk.transforms
-
PTransforms for combining PCollection elements
globally and per-key.
- combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn
-
Combines the given output times, which must be from the same window, into an output time
for a computed value.
- combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
-
Combines the given output times, which must be from the same window, into an output time
for a computed value.
- combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
-
Combines the given output times, which must be from the same window, into an output time
for a computed value.
- Combine.AccumulatingCombineFn<InputT,AccumT extends Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT>,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
- Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
-
The type of mutable accumulator values used by this
AccumulatingCombineFn.
- Combine.BinaryCombineDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
-
An abstract subclass of
Combine.CombineFn for implementing combiners that are more
easily and efficiently expressed as binary operations on
doubles.
- Combine.BinaryCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms
-
An abstract subclass of
Combine.CombineFn for implementing combiners that are more
easily expressed as binary operations.
- Combine.BinaryCombineIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
-
An abstract subclass of
Combine.CombineFn for implementing combiners that are more
easily and efficiently expressed as binary operations on
ints
- Combine.BinaryCombineLongFn - Class in com.google.cloud.dataflow.sdk.transforms
-
An abstract subclass of
Combine.CombineFn for implementing combiners that are more
easily and efficiently expressed as binary operations on
longs.
- Combine.CombineFn<InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
A CombineFn<InputT, AccumT, OutputT> specifies how to combine a
collection of input values of type InputT into a single
output value of type OutputT.
- Combine.Globally<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
Combine.Globally<InputT, OutputT> takes a
PCollection<InputT>
and returns a
PCollection<OutputT> whose elements are the result of
combining all the elements in each window of the input
PCollection,
using a specified
CombineFn<InputT, AccumT, OutputT>.
- Combine.GloballyAsSingletonView<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
Combine.GloballyAsSingletonView<InputT, OutputT> takes a
PCollection<InputT>
and returns a
PCollectionView<OutputT> whose elements are the result of
combining all the elements in each window of the input
PCollection,
using a specified
CombineFn<InputT, AccumT, OutputT>.
- Combine.GroupedValues<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
GroupedValues<K, InputT, OutputT> takes a
PCollection<KV<K, Iterable<InputT>>>, such as the result of
GroupByKey, applies a specified
KeyedCombineFn<K, InputT, AccumT, OutputT>
to each of the input
KV<K, Iterable<InputT>> elements to
produce a combined output
KV<K, OutputT> element, and returns a
PCollection<KV<K, OutputT>> containing all the combined output
elements.
- Combine.Holder<V> - Class in com.google.cloud.dataflow.sdk.transforms
-
Holds a single value value of type V which may or may not be present.
- Combine.IterableCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms
-
- Combine.KeyedCombineFn<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
A KeyedCombineFn<K, InputT, AccumT, OutputT> specifies how to combine
a collection of input values of type InputT, associated with
a key of type K, into a single output value of type
OutputT.
- Combine.PerKey<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
PerKey<K, InputT, OutputT> takes a
PCollection<KV<K, InputT>>, groups it by key, applies a
combining function to the InputT values associated with each
key to produce a combined OutputT value, and returns a
PCollection<KV<K, OutputT>> representing a map from each
distinct key of the input PCollection to the corresponding
combined value.
- Combine.PerKeyWithHotKeyFanout<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
- Combine.SimpleCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms
-
Deprecated.
- CombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
-
- CombineFnBase - Class in com.google.cloud.dataflow.sdk.transforms
-
This class contains the shared interfaces and abstract classes for different types of combine
functions.
- CombineFnBase() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFnBase
-
- CombineFnBase.GlobalCombineFn<InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
-
A GloballyCombineFn<InputT, AccumT, OutputT> specifies how to combine a
collection of input values of type InputT into a single
output value of type OutputT.
- CombineFnBase.PerKeyCombineFn<K,InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
-
A PerKeyCombineFn<K, InputT, AccumT, OutputT> specifies how to combine
a collection of input values of type InputT, associated with
a key of type K, into a single output value of type
OutputT.
- CombineFns - Class in com.google.cloud.dataflow.sdk.transforms
-
Static utility methods that create combine function instances.
- CombineFns() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns
-
- CombineFns.CoCombineResult - Class in com.google.cloud.dataflow.sdk.transforms
-
A tuple of outputs produced by a composed combine functions.
- CombineFns.ComposeCombineFnBuilder - Class in com.google.cloud.dataflow.sdk.transforms
-
- CombineFns.ComposedCombineFn<DataT> - Class in com.google.cloud.dataflow.sdk.transforms
-
- CombineFns.ComposedCombineFnWithContext<DataT> - Class in com.google.cloud.dataflow.sdk.transforms
-
- CombineFns.ComposedKeyedCombineFn<DataT,K> - Class in com.google.cloud.dataflow.sdk.transforms
-
- CombineFns.ComposedKeyedCombineFnWithContext<DataT,K> - Class in com.google.cloud.dataflow.sdk.transforms
-
- CombineFns.ComposeKeyedCombineFnBuilder - Class in com.google.cloud.dataflow.sdk.transforms
-
- CombineFnWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
-
- combineOutputTimes(OutputTimeFn<?>, Iterable<? extends Instant>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFns
-
Applies the given
OutputTimeFn to the given output times, obtaining
the output time for a value computed.
- CombineWithContext - Class in com.google.cloud.dataflow.sdk.transforms
-
This class contains combine functions that have access to PipelineOptions and side inputs
through CombineWithContext.Context.
- CombineWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext
-
- CombineWithContext.CombineFnWithContext<InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
A combine function that has access to PipelineOptions and side inputs through
CombineWithContext.Context.
- CombineWithContext.Context - Class in com.google.cloud.dataflow.sdk.transforms
-
Information accessible to all methods in CombineFnWithContext
and KeyedCombineFnWithContext.
- CombineWithContext.KeyedCombineFnWithContext<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
A keyed combine function that has access to PipelineOptions and side inputs through
CombineWithContext.Context.
- CombineWithContext.RequiresContextInternal - Interface in com.google.cloud.dataflow.sdk.transforms
-
An internal interface for signaling that a GloballyCombineFn
or a PerKeyCombineFn needs to access CombineWithContext.Context.
- compact(AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
-
Returns an accumulator that represents the same logical value as the
input accumulator, but may have a more compact representation.
- compact(List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
-
- compact(K, AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
-
Returns an accumulator that represents the same logical value as the
input accumulator, but may have a more compact representation.
- compact(Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
-
- compact(Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
-
- compact(K, Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
-
- compact(K, Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
-
- compact(AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
-
Returns an accumulator that represents the same logical value as the
input accumulator, but may have a more compact representation.
- compact(K, AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
-
Returns an accumulator that represents the same logical value as the
input accumulator, but may have a more compact representation.
- compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Largest
-
- compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Smallest
-
- compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByKey
-
- compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByValue
-
- compareTo(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
-
- compareTo(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
- compose() - Static method in class com.google.cloud.dataflow.sdk.transforms.CombineFns
-
- ComposeCombineFnBuilder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeCombineFnBuilder
-
- composeKeyed() - Static method in class com.google.cloud.dataflow.sdk.transforms.CombineFns
-
- ComposeKeyedCombineFnBuilder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
-
- CompressedReader(CompressedSource<T>, FileBasedSource.FileBasedReader<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
-
Create a CompressedReader from a CompressedSource and delegate reader.
- CompressedSource<T> - Class in com.google.cloud.dataflow.sdk.io
-
A Source that reads from compressed files.
- CompressedSource.CompressedReader<T> - Class in com.google.cloud.dataflow.sdk.io
-
- CompressedSource.CompressionMode - Enum in com.google.cloud.dataflow.sdk.io
-
Default compression types supported by the CompressedSource.
- CompressedSource.DecompressingChannelFactory - Interface in com.google.cloud.dataflow.sdk.io
-
Factory interface for creating channels that decompress the content of an underlying channel.
- computeTargetTimestamp(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
-
- consistentWithEquals() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
-
NullableCoder is consistent with equals if the nested Coder is.
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
-
- consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
-
- contains(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
Returns whether this window contains the given window.
- containsInAnyOrder(Iterable<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
-
Checks that the Iterable contains the expected elements, in any
order.
- containsInAnyOrder(T...) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
-
Checks that the Iterable contains the expected elements, in any
order.
- containsKey(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
-
Returns
true if the specified
ByteKey is contained within this range.
- Context(boolean) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.Context
-
- Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.Context
-
- Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
-
- Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
-
- copyFrom(byte[]) - Static method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
-
Creates a new
ByteKey backed by a copy of the specified
byte[].
- copyToOutputFiles(List<String>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Copy temporary files to final output filenames using the file naming template.
- Count - Class in com.google.cloud.dataflow.sdk.transforms
-
- Count.PerElement<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
Count.PerElement<T> takes a PCollection<T> and returns a
PCollection<KV<T, Long>> representing a map from each distinct element of the input
PCollection to the number of times that element occurs in the input.
- CounterMark(long) - Constructor for class com.google.cloud.dataflow.sdk.io.CountingSource.CounterMark
-
Creates a checkpoint mark reflecting the last emitted value.
- CountingInput - Class in com.google.cloud.dataflow.sdk.io
-
- CountingInput() - Constructor for class com.google.cloud.dataflow.sdk.io.CountingInput
-
- CountingInput.BoundedCountingInput - Class in com.google.cloud.dataflow.sdk.io
-
A
PTransform that will produce a specified number of
Longs starting from
0.
- CountingInput.UnboundedCountingInput - Class in com.google.cloud.dataflow.sdk.io
-
- CountingSource - Class in com.google.cloud.dataflow.sdk.io
-
A source that produces longs.
- CountingSource.CounterMark - Class in com.google.cloud.dataflow.sdk.io
-
The checkpoint for an unbounded
CountingSource is simply the last value produced.
- create(List<? extends Coder<?>>) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderFactory
-
Returns a Coder<?>, given argument coder to use for
values of a particular type, given the Coders for each of
the type's generic parameter types.
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions.StandardOutputFactory
-
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.DataflowClientFactory
-
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.PathValidatorFactory
-
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.StagerFactory
-
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions.JobNameFactory
-
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory
-
- create(PipelineOptions) - Method in interface com.google.cloud.dataflow.sdk.options.DefaultValueFactory
-
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.CredentialDirFactory
-
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.DefaultProjectFactory
-
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.GcpUserCredentialsFactory
-
- create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcsOptions.ExecutorServiceFactory
-
- create() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
-
Creates and returns an object that implements
PipelineOptions using the values
configured on this builder during construction.
- create() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
- create(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.Pipeline
-
Constructs a pipeline from the provided options.
- create(DataflowPipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
-
- create(Combine.GroupedValues<K, InputT, OutputT>, PCollection<KV<K, Iterable<InputT>>>, boolean, Random) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
-
- create() - Static method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
-
Creates and returns a new test pipeline.
- create(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
-
Returns an approximate quantiles combiner with the given
compareFn and desired number of quantiles.
- create(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
-
- create(int, ComparatorT, long, double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
-
Creates an approximate quantiles combiner with the given
compareFn and desired number of quantiles.
- Create<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
Create<T> takes a collection of elements of type T
known when the pipeline is constructed and returns a
PCollection<T> containing the elements.
- Create() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Create
-
- create() - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
-
Returns a GroupByKey<K, V> PTransform.
- create() - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGroupByKey
-
Returns a CoGroupByKey<K> PTransform.
- create() - Static method in class com.google.cloud.dataflow.sdk.transforms.Keys
-
Returns a Keys<K> PTransform.
- create() - Static method in class com.google.cloud.dataflow.sdk.transforms.KvSwap
-
Returns a KvSwap<K, V> PTransform.
- create() - Static method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates
-
Returns a RemoveDuplicates<T> PTransform.
- create() - Static method in class com.google.cloud.dataflow.sdk.transforms.Values
-
Returns a Values<V> PTransform.
- Create.TimestampedValues<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
A PTransform that creates a PCollection whose elements have
associated timestamps.
- Create.Values<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
A PTransform that creates a PCollection from a set of in-memory objects.
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
-
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
-
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
-
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
-
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
-
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
-
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
-
Returns a new, mutable accumulator value, representing the accumulation of zero input values.
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn
-
- createAccumulator(K) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
-
Returns a new, mutable accumulator value representing the accumulation of zero input values.
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
-
- createAccumulator(CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
-
- createAccumulator(K) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
-
- createAccumulator(K, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
-
- createAccumulator(CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
-
Returns a new, mutable accumulator value, representing the accumulation of zero input values.
- createAccumulator(K, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
-
Returns a new, mutable accumulator value representing the accumulation of zero input values.
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
-
- createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
-
- createAggregator(String, Combine.CombineFn<? super AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
-
- createAggregator(String, SerializableFunction<Iterable<AggInputT>, AggInputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
-
- createAggregator(String, Combine.CombineFn<? super AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
-
- createAggregator(String, SerializableFunction<Iterable<AggInputT>, AggInputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
-
- createAggregatorInternal(String, Combine.CombineFn<AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
-
- createDatumReader() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
- createDatumWriter() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
- createDecompressingChannel(ReadableByteChannel) - Method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
-
- createDecompressingChannel(ReadableByteChannel) - Method in interface com.google.cloud.dataflow.sdk.io.CompressedSource.DecompressingChannelFactory
-
Given a channel, create a channel that decompresses the content read from the channel.
- createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
- createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource
-
Creates a BlockBasedSource for the specified range in a single file.
- createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
-
Creates a CompressedSource for a subrange of a file.
- createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
Creates and returns a new FileBasedSource of the same type as the current
FileBasedSource backed by a given file and an offset range.
- createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
- createForTest() - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
-
Creates and returns a new DirectPipeline instance for tests.
- createForTest() - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
-
Constructs a runner with default properties for testing.
- createFromSystemProperties() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
- createFromSystemPropertiesInternal() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
Creates a set of Dataflow worker harness options based of a set of known system
properties.
- createPane(boolean, boolean, PaneInfo.Timing) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
- createPane(boolean, boolean, PaneInfo.Timing, long, long) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
Factory method to create a
PaneInfo with the specified parameters.
- createPrimitiveOutputInternal(Pipeline, WindowingStrategy<?, ?>, PCollection.IsBounded) - Static method in class com.google.cloud.dataflow.sdk.values.PCollection
-
Creates and returns a new
PCollection for a primitive output.
- createReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
-
- createReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- createReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
- createReader(PipelineOptions, CheckpointMarkT) - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
-
- createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
- createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource
-
Creates a BlockBasedReader.
- createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
-
Creates a FileBasedReader to read a single file.
- createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
Creates and returns an instance of a FileBasedReader implementation for the current
source assuming the source represents a single file.
- createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
- createSourceForSubrange(long, long) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
- createSourceForSubrange(long, long) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
- createStructuralValues(Coder<T>, List<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
-
Testing utilities below depend on standard assertions and matchers to compare elements read by
sources.
- createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
-
- createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink
-
- createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink
-
- createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
-
- createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
- createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
-
Creates a new
Sink.Writer to write a bundle of the input to the sink.
- createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriteOperation
-
- CredentialDirFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcpOptions.CredentialDirFactory
-
- currentEventTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
-
The current event time for the input or null if unknown.
- currentProcessingTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
-
The current processing time.
- currentSynchronizedProcessingTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
-
The current synchronized upstream processing time or null if unknown.
- CustomCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
-
An abstract base class for writing a
Coder class that encodes itself via Java
serialization.
- CustomCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.CustomCoder
-
- DataflowAssert - Class in com.google.cloud.dataflow.sdk.testing
-
An assertion on the contents of a
PCollection
incorporated into the pipeline.
- DataflowAssert.IterableAssert<T> - Class in com.google.cloud.dataflow.sdk.testing
-
An assertion about the contents of a
PCollectionView yielding an
Iterable<T>.
- DataflowAssert.SingletonAssert<T> - Class in com.google.cloud.dataflow.sdk.testing
-
An assertion about the single value of type
T
associated with a
PCollectionView.
- DataflowClientFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.DataflowClientFactory
-
- DataflowJobAlreadyExistsException - Exception in com.google.cloud.dataflow.sdk.runners
-
An exception that is thrown if the unique job name constraint of the Dataflow
service is broken because an existing job with the same job name is currently active.
- DataflowJobAlreadyExistsException(DataflowPipelineJob, String) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobAlreadyExistsException
-
Create a new
DataflowJobAlreadyExistsException with the specified
DataflowPipelineJob and message.
- DataflowJobAlreadyUpdatedException - Exception in com.google.cloud.dataflow.sdk.runners
-
An exception that is thrown if the existing job has already been updated within the Dataflow
service and is no longer able to be updated.
- DataflowJobAlreadyUpdatedException(DataflowPipelineJob, String) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobAlreadyUpdatedException
-
Create a new
DataflowJobAlreadyUpdatedException with the specified
DataflowPipelineJob and message.
- DataflowJobCancelledException - Exception in com.google.cloud.dataflow.sdk.runners
-
- DataflowJobCancelledException(DataflowPipelineJob, String) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobCancelledException
-
Create a new
DataflowJobAlreadyUpdatedException with the specified
DataflowPipelineJob and message.
- DataflowJobCancelledException(DataflowPipelineJob, String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobCancelledException
-
Create a new
DataflowJobAlreadyUpdatedException with the specified
DataflowPipelineJob, message, and cause.
- DataflowJobException - Exception in com.google.cloud.dataflow.sdk.runners
-
- DataflowJobExecutionException - Exception in com.google.cloud.dataflow.sdk.runners
-
- DataflowJobUpdatedException - Exception in com.google.cloud.dataflow.sdk.runners
-
- DataflowJobUpdatedException(DataflowPipelineJob, String, DataflowPipelineJob) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobUpdatedException
-
- DataflowJobUpdatedException(DataflowPipelineJob, String, DataflowPipelineJob, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobUpdatedException
-
- DataflowPipeline - Class in com.google.cloud.dataflow.sdk.runners
-
- DataflowPipelineDebugOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Internal.
- DataflowPipelineDebugOptions.DataflowClientFactory - Class in com.google.cloud.dataflow.sdk.options
-
Returns the default Dataflow client built from the passed in PipelineOptions.
- DataflowPipelineDebugOptions.PathValidatorFactory - Class in com.google.cloud.dataflow.sdk.options
-
- DataflowPipelineDebugOptions.StagerFactory - Class in com.google.cloud.dataflow.sdk.options
-
- DataflowPipelineJob - Class in com.google.cloud.dataflow.sdk.runners
-
- DataflowPipelineJob(String, String, Dataflow, DataflowAggregatorTransforms) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
-
Constructs the job.
- DataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
-
- DataflowPipelineOptions.JobNameFactory - Class in com.google.cloud.dataflow.sdk.options
-
- DataflowPipelineRegistrar - Class in com.google.cloud.dataflow.sdk.runners
-
- DataflowPipelineRegistrar.Options - Class in com.google.cloud.dataflow.sdk.runners
-
- DataflowPipelineRegistrar.Runner - Class in com.google.cloud.dataflow.sdk.runners
-
- DataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners
-
A
PipelineRunner that executes the operations in the
pipeline by first translating them to the Dataflow representation
using the
DataflowPipelineTranslator and then submitting
them to a Dataflow service for execution.
- DataflowPipelineRunner(DataflowPipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
-
- DataflowPipelineRunner.StreamingPubsubIOWrite<T> - Class in com.google.cloud.dataflow.sdk.runners
-
Specialized implementation for
PubsubIO.Write for the
Dataflow runner in streaming mode.
- DataflowPipelineRunnerHooks - Class in com.google.cloud.dataflow.sdk.runners
-
An instance of this class can be passed to the
DataflowPipelineRunner to add user defined hooks to be
invoked at various times during pipeline execution.
- DataflowPipelineRunnerHooks() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunnerHooks
-
- DataflowPipelineTranslator - Class in com.google.cloud.dataflow.sdk.runners
-
- DataflowPipelineTranslator.JobSpecification - Class in com.google.cloud.dataflow.sdk.runners
-
The result of a job translation.
- DataflowPipelineTranslator.TransformTranslator<TransformT extends PTransform> - Interface in com.google.cloud.dataflow.sdk.runners
-
- DataflowPipelineTranslator.TranslationContext - Interface in com.google.cloud.dataflow.sdk.runners
-
- DataflowPipelineWorkerPoolOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Options that are used to configure the Dataflow pipeline worker pool.
- DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType - Enum in com.google.cloud.dataflow.sdk.options
-
Type of autoscaling algorithm to use.
- DataflowPipelineWorkerPoolOptions.TeardownPolicy - Enum in com.google.cloud.dataflow.sdk.options
-
The policy for tearing down the workers spun up by the service.
- DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory - Class in com.google.cloud.dataflow.sdk.options
-
Returns the default Docker container image that executes Dataflow worker harness, residing in
Google Container Registry.
- DataflowProfilingAgentConfiguration() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions.DataflowProfilingAgentConfiguration
-
- DataflowProfilingOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Options for controlling profiling of pipeline execution.
- DataflowProfilingOptions.DataflowProfilingAgentConfiguration - Class in com.google.cloud.dataflow.sdk.options
-
Configuration the for profiling agent.
- DataflowServiceException - Exception in com.google.cloud.dataflow.sdk.runners
-
Signals there was an error retrieving information about a job from the Cloud Dataflow Service.
- DataflowWorkerHarnessOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Options that are used exclusively within the Dataflow worker harness.
- DataflowWorkerLoggingOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Options that are used to control logging configuration on the Dataflow worker.
- DataflowWorkerLoggingOptions.Level - Enum in com.google.cloud.dataflow.sdk.options
-
The set of log levels that can be used on the Dataflow worker.
- DataflowWorkerLoggingOptions.WorkerLogLevelOverrides - Class in com.google.cloud.dataflow.sdk.options
-
Defines a log level override for a specific class, package, or name.
- DATASTORE_BATCH_UPDATE_LIMIT - Static variable in class com.google.cloud.dataflow.sdk.io.DatastoreIO
-
Datastore has a limit of 500 mutations per batch operation, so we flush
changes to Datastore every 500 entities.
- DatastoreIO - Class in com.google.cloud.dataflow.sdk.io
-
- DatastoreIO() - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO
-
- DatastoreIO.DatastoreReader - Class in com.google.cloud.dataflow.sdk.io
-
- DatastoreIO.Sink - Class in com.google.cloud.dataflow.sdk.io
-
- DatastoreIO.Source - Class in com.google.cloud.dataflow.sdk.io
-
A
Source that reads the result rows of a Datastore query as
Entity objects.
- DatastoreReader(DatastoreIO.Source, Datastore) - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
-
Returns a DatastoreReader with Source and Datastore object set.
- days(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
-
Returns a
WindowFn that windows elements into periods measured by days.
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
-
- decode(InputStream, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
-
Decodes a value of type T from the given input stream in
the given context.
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
-
Deprecated.
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow.Coder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
-
- decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
-
- decodePane(byte) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
- decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
-
Builds an instance of
IterableT, this coder's associated
Iterable-like
subtype, from a list of decoded elements.
- decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
-
- decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
-
Builds an instance of
IterableT, this coder's associated
Iterable-like
subtype, from a list of decoded elements.
- decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.ListCoder
-
- decodeToIterable(List<T>) - Method in class com.google.cloud.dataflow.sdk.coders.SetCoder
-
Builds an instance of
IterableT, this coder's associated
Iterable-like
subtype, from a list of decoded elements.
- Default - Annotation Type in com.google.cloud.dataflow.sdk.options
-
Default represents a set of annotations that can be used to annotate getter properties
on
PipelineOptions with information representing the default value to be returned
if no value is specified.
- Default.Boolean - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified boolean primitive value.
- Default.Byte - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified byte primitive value.
- Default.Character - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified char primitive value.
- Default.Class - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified
Class value.
- Default.Double - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified double primitive value.
- Default.Enum - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified enum.
- Default.Float - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified float primitive value.
- Default.InstanceFactory - Annotation Type in com.google.cloud.dataflow.sdk.options
-
- Default.Integer - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified int primitive value.
- Default.Long - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified long primitive value.
- Default.Short - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified short primitive value.
- Default.String - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This represents that the default of the option is the specified
String
value.
- DEFAULT_HOST - Static variable in class com.google.cloud.dataflow.sdk.io.DatastoreIO
-
- DEFAULT_MAX_NUM_ELEMENTS - Static variable in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
-
The cost (in time and space) to compute quantiles to a given
accuracy is a function of the total number of elements in the
data set.
- DEFAULT_PUBSUB_CODER - Static variable in class com.google.cloud.dataflow.sdk.io.PubsubIO
-
The default
Coder used to translate to/from Cloud Pub/Sub messages.
- DEFAULT_TEXT_CODER - Static variable in class com.google.cloud.dataflow.sdk.io.TextIO
-
The default coder, which returns each line of the input file as a string.
- DefaultCoder - Annotation Type in com.google.cloud.dataflow.sdk.coders
-
The
DefaultCoder annotation
specifies a default
Coder class to handle encoding and decoding
instances of the annotated class.
- DefaultProjectFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcpOptions.DefaultProjectFactory
-
- Defaults() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
-
- DefaultTrigger<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A trigger that is equivalent to Repeatedly.forever(AfterWatermark.pastEndOfWindow()).
- defaultValue() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
-
Returns the default value when there are no values added to the accumulator.
- defaultValue() - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
-
Returns the default value when there are no values added to the accumulator.
- defaultValue() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext
-
- defaultValue() - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
-
Returns the default value of this transform, or null if there isn't one.
- DefaultValueFactory<T> - Interface in com.google.cloud.dataflow.sdk.options
-
- DELAYED_UNTIL_TAG - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- DelegateCoder<T,IntermediateT> - Class in com.google.cloud.dataflow.sdk.coders
-
A
DelegateCoder<T, IntermediateT> wraps a
Coder for
IntermediateT and
encodes/decodes values of type
T by converting
to/from
IntermediateT and then encoding/decoding using the underlying
Coder<IntermediateT>.
- DelegateCoder(Coder<IntermediateT>, DelegateCoder.CodingFunction<T, IntermediateT>, DelegateCoder.CodingFunction<IntermediateT, T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.DelegateCoder
-
- DelegateCoder.CodingFunction<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.coders
-
- deleteTimer(Instant, TimeDomain) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
-
Removes the timer set in this trigger context for the given
Instant
and
TimeDomain.
- dependsOnlyOnEarliestInputTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
-
Returns true if the result of combination of many output timestamps actually depends
only on the earliest.
- dependsOnlyOnEarliestInputTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn
-
Returns true if the result of combination of many output timestamps actually depends
only on the earliest.
- dependsOnlyOnEarliestInputTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
-
Returns true if the result of combination of many output timestamps actually depends
only on the earliest.
- dependsOnlyOnWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults
-
Returns true if the result does not depend on what outputs were combined but only
the window they are in.
- dependsOnlyOnWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn
-
Returns true if the result does not depend on what outputs were combined but only
the window they are in.
- DependsOnlyOnWindow() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
-
- dependsOnlyOnWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow
-
Returns true if the result does not depend on what outputs were combined but only
the window they are in.
- Description - Annotation Type in com.google.cloud.dataflow.sdk.options
-
Descriptions are used to generate human readable output when the --help
command is specified.
- detectClassPathResourcesToStage(ClassLoader) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
-
Attempts to detect all the resources the class loader has access to.
- DeterministicStandardCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
-
- DeterministicStandardCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.DeterministicStandardCoder
-
- DIRECTORY_CONTAINER - Static variable in class com.google.cloud.dataflow.sdk.io.ShardNameTemplate
-
Shard is a file within a directory.
- DirectPipeline - Class in com.google.cloud.dataflow.sdk.runners
-
- DirectPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
-
- DirectPipelineRegistrar - Class in com.google.cloud.dataflow.sdk.runners
-
- DirectPipelineRegistrar.Options - Class in com.google.cloud.dataflow.sdk.runners
-
- DirectPipelineRegistrar.Runner - Class in com.google.cloud.dataflow.sdk.runners
-
- DirectPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners
-
Executes the operations in the pipeline directly, in this process, without
any optimization.
- DirectPipelineRunner.EvaluationContext - Interface in com.google.cloud.dataflow.sdk.runners
-
The interface provided to registered callbacks for interacting
with the
DirectPipelineRunner, including reading and writing the
values of
PCollections and
PCollectionViews.
- DirectPipelineRunner.EvaluationResults - Interface in com.google.cloud.dataflow.sdk.runners
-
The interface provided to registered callbacks for interacting
with the
DirectPipelineRunner, including reading and writing the
values of
PCollections and
PCollectionViews.
- DirectPipelineRunner.TestCombineDoFn<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.runners
-
The implementation may split the
Combine.KeyedCombineFn into ADD, MERGE and EXTRACT phases (
see
com.google.cloud.dataflow.sdk.runners.worker.CombineValuesFn).
- DirectPipelineRunner.TransformEvaluator<TransformT extends PTransform> - Interface in com.google.cloud.dataflow.sdk.runners
-
An evaluator of a PTransform.
- DirectPipelineRunner.ValueWithMetadata<V> - Class in com.google.cloud.dataflow.sdk.runners
-
An immutable (value, timestamp) pair, along with other metadata necessary
for the implementation of DirectPipelineRunner.
- discardingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
Returns a new Window PTransform that uses the registered WindowFn and
Triggering behavior, and that discards elements in a pane after they are triggered.
- discardingFiredPanes() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
-
Returns a new Window PTransform that uses the registered WindowFn and
Triggering behavior, and that discards elements in a pane after they are triggered.
- discardingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
-
Returns a new Window PTransform that uses the registered WindowFn and
Triggering behavior, and that discards elements in a pane after they are triggered.
- DisplayData - Class in com.google.cloud.dataflow.sdk.transforms.display
-
Static display metadata associated with a pipeline component.
- DisplayData.Builder - Interface in com.google.cloud.dataflow.sdk.transforms.display
-
Utility to build up display metadata from a component and its included
subcomponents.
- DisplayData.Identifier - Class in com.google.cloud.dataflow.sdk.transforms.display
-
Unique identifier for a display metadata item within a component.
- DisplayData.Item - Class in com.google.cloud.dataflow.sdk.transforms.display
-
A display metadata item.
- DisplayData.ItemBuilder - Interface in com.google.cloud.dataflow.sdk.transforms.display
-
Utility to append optional fields to display metadata, or register additional display metadata
items.
- DoFn<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
The argument to
ParDo providing the code to use to process
elements of the input
PCollection.
- DoFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn
-
- DoFn.Context - Class in com.google.cloud.dataflow.sdk.transforms
-
Information accessible to all methods in this DoFn.
- DoFn.ProcessContext - Class in com.google.cloud.dataflow.sdk.transforms
-
- DoFn.RequiresWindowAccess - Interface in com.google.cloud.dataflow.sdk.transforms
-
- DoFnReflector - Class in com.google.cloud.dataflow.sdk.transforms
-
Utility implementing the necessary reflection for working with
DoFnWithContexts.
- DoFnReflector() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
-
- DoFnTester<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
A harness for unit-testing a
DoFn.
- DoFnTester.OutputElementWithTimestamp<OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
Holder for an OutputElement along with its associated timestamp.
- DoFnWithContext<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
The argument to
ParDo providing the code to use to process
elements of the input
PCollection.
- DoFnWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
-
- DoFnWithContext.Context - Class in com.google.cloud.dataflow.sdk.transforms
-
Information accessible to all methods in this DoFnWithContext.
- DoFnWithContext.ExtraContextFactory<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
-
Interface for runner implementors to provide implementations of extra context information.
- DoFnWithContext.FinishBundle - Annotation Type in com.google.cloud.dataflow.sdk.transforms
-
Annotation for the method to use to prepare an instance for processing a batch of elements.
- DoFnWithContext.ProcessContext - Class in com.google.cloud.dataflow.sdk.transforms
-
- DoFnWithContext.ProcessElement - Annotation Type in com.google.cloud.dataflow.sdk.transforms
-
Annotation for the method to use for processing elements.
- DoFnWithContext.StartBundle - Annotation Type in com.google.cloud.dataflow.sdk.transforms
-
Annotation for the method to use to prepare an instance for processing a batch of elements.
- DoubleCoder - Class in com.google.cloud.dataflow.sdk.coders
-
- doublesGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
-
Returns a PTransform that takes an input PCollection<Double> and returns a
PCollection<Double> whose contents is the maximum of the input PCollection's
elements, or Double.NEGATIVE_INFINITY if there are no elements.
- doublesGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
-
Returns a PTransform that takes an input PCollection<Double> and returns a
PCollection<Double> whose contents is the minimum of the input PCollection's
elements, or Double.POSITIVE_INFINITY if there are no elements.
- doublesGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
-
Returns a PTransform that takes an input
PCollection<Double> and returns a
PCollection<Double> whose contents is the sum of the
input PCollection's elements, or
0 if there are no elements.
- doublesPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
-
Returns a PTransform that takes an input PCollection<KV<K, Double>> and returns
a PCollection<KV<K, Double>> that contains an output element mapping each distinct key
in the input PCollection to the maximum of the values associated with that key in the
input PCollection.
- doublesPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
-
Returns a PTransform that takes an input PCollection<KV<K, Double>> and returns
a PCollection<KV<K, Double>> that contains an output element mapping each distinct key
in the input PCollection to the minimum of the values associated with that key in the
input PCollection.
- doublesPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
-
Returns a PTransform that takes an input
PCollection<KV<K, Double>> and returns a
PCollection<KV<K, Double>> that contains an output
element mapping each distinct key in the input
PCollection to the sum of the values associated with
that key in the input PCollection.
- DurationCoder - Class in com.google.cloud.dataflow.sdk.coders
-
- fewKeys() - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
-
Returns whether it groups just few keys.
- FileBasedReader(FileBasedSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
-
Subclasses should not perform IO operations at the constructor.
- FileBasedSink<T> - Class in com.google.cloud.dataflow.sdk.io
-
Abstract
Sink for file-based output.
- FileBasedSink(String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink
-
Construct a FileBasedSink with the given base output filename and extension.
- FileBasedSink(String, String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink
-
Construct a FileBasedSink with the given base output filename, extension, and file naming
template.
- FileBasedSink.FileBasedWriteOperation<T> - Class in com.google.cloud.dataflow.sdk.io
-
- FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention - Enum in com.google.cloud.dataflow.sdk.io
-
Options for handling of temporary output files.
- FileBasedSink.FileBasedWriter<T> - Class in com.google.cloud.dataflow.sdk.io
-
- FileBasedSink.FileResult - Class in com.google.cloud.dataflow.sdk.io
-
Result of a single bundle write.
- FileBasedSource<T> - Class in com.google.cloud.dataflow.sdk.io
-
A common base class for all file-based
Sources.
- FileBasedSource(String, long) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
Create a FileBaseSource based on a file or a file pattern specification.
- FileBasedSource(String, long, long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
Create a FileBasedSource based on a single file.
- FileBasedSource.FileBasedReader<T> - Class in com.google.cloud.dataflow.sdk.io
-
A
reader that implements code common to readers of
FileBasedSources.
- FileBasedSource.Mode - Enum in com.google.cloud.dataflow.sdk.io
-
A given FileBasedSource represents a file resource of one of these types.
- FileBasedWriteOperation(FileBasedSink<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Construct a FileBasedWriteOperation using the same base filename for both temporary and
output files.
- FileBasedWriteOperation(FileBasedSink<T>, String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Construct a FileBasedWriteOperation.
- FileBasedWriteOperation(FileBasedSink<T>, String, FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Create a new FileBasedWriteOperation.
- FileBasedWriter(FileBasedSink.FileBasedWriteOperation<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
-
Construct a new FileBasedWriter with a base filename.
- fileNamingTemplate - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink
-
Naming template for output files.
- FileResult(String) - Constructor for class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileResult
-
- Filter<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
PTransforms for filtering from a PCollection the
elements satisfying a predicate, or satisfying an inequality with
a given value based on the elements' natural ordering.
- finalize(Iterable<FileBasedSink.FileResult>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Finalizes writing by copying temporary output files to their final location and optionally
removing temporary files.
- finalize(Iterable<WriteT>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
-
Given an Iterable of results from bundle writes, performs finalization after writing and
closes the sink.
- finalizeCheckpoint() - Method in class com.google.cloud.dataflow.sdk.io.CountingSource.CounterMark
-
- finalizeCheckpoint() - Method in interface com.google.cloud.dataflow.sdk.io.UnboundedSource.CheckpointMark
-
Perform any finalization that needs to happen after a bundle of data read from
the source has been processed and committed.
- finishBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
-
Finishes processing this batch of elements.
- finishBundle() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
- finishBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
-
- finishedInAllMergingWindows() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergingTriggerInfo
-
Return true if the trigger is finished in all windows being merged.
- finishedInAnyMergingWindow() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergingTriggerInfo
-
Return true if the trigger is finished in any window being merged.
- finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Finish specifying a transform.
- finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
-
- finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PBegin
-
- finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
- finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
-
- finishSpecifying() - Method in interface com.google.cloud.dataflow.sdk.values.PInput
-
After building, finalizes this
PInput to make it ready for
being used as an input to a
PTransform.
- finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
-
- finishSpecifying() - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
-
After building, finalizes this
PValue to make it ready for
running.
- finishSpecifyingOutput() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
- finishSpecifyingOutput() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
-
- finishSpecifyingOutput() - Method in interface com.google.cloud.dataflow.sdk.values.POutput
-
As part of applying the producing
PTransform, finalizes this
output to make it ready for being used as an input and for running.
- finishSpecifyingOutput() - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
-
- firstOf(CoderProvider...) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderProviders
-
Returns a
CoderProvider that consults each of the provider
coderProviders
and returns the first
Coder provided.
- firstUnfinishedSubTrigger() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
-
Returns the first unfinished sub-trigger.
- fixedSizeGlobally(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample
-
Returns a PTransform that takes a PCollection<T>,
selects sampleSize elements, uniformly at random, and returns a
PCollection<Iterable<T>> containing the selected elements.
- fixedSizePerKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample
-
Returns a PTransform that takes an input
PCollection<KV<K, V>> and returns a
PCollection<KV<K, Iterable<V>>> that contains an output
element mapping each distinct key in the input
PCollection to a sample of sampleSize values
associated with that key in the input PCollection, taken
uniformly at random.
- FixedWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A
WindowFn that windows values into fixed-size timestamp-based windows.
- FlatMapElements<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
PTransforms for mapping a simple function that returns iterables over the elements of a
PCollection and merging the results.
- FlatMapElements.MissingOutputTypeDescriptor<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
- Flatten - Class in com.google.cloud.dataflow.sdk.transforms
-
Flatten<T> takes multiple PCollection<T>s bundled
into a PCollectionList<T> and returns a single
PCollection<T> containing all the elements in all the input
PCollections.
- Flatten() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Flatten
-
- Flatten.FlattenIterables<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
FlattenIterables<T> takes a PCollection<Iterable<T>> and returns a
PCollection<T> that contains all the elements from each iterable.
- Flatten.FlattenPCollectionList<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
- FlattenIterables() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenIterables
-
- forCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderFactories
-
- forever(Trigger<W>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
-
Create a composite trigger that repeatedly executes the trigger toRepeat, firing each
time it fires and ignoring any indications to finish.
- forKey(K, Coder<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn
-
- forKey(K, Coder<K>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.PerKeyCombineFn
-
- forKey(K, Coder<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext
-
- forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
-
Create an OnElementContext for executing the given trigger.
- forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
-
Create an OnMergeContext for executing the given trigger.
- forTrigger(ExecutableTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
-
Create a sub-context for the given sub-trigger.
- from(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
-
Returns a new
PTransform that's like this one but
that reads from the file(s) with the given name or pattern.
- from(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
-
Returns a
PTransform that reads from the file(s)
with the given name or pattern.
- from(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
Creates an
AvroSource that reads from the given file name or pattern ("glob").
- from(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
Returns a copy of this transform that reads from the specified table.
- from(TableReference) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
Returns a copy of this transform that reads from the specified table.
- from(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
-
Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or
"[dataset_id].[table_id]" for tables within the current project.
- from(TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
-
- from(FileBasedSource<T>) - Static method in class com.google.cloud.dataflow.sdk.io.CompressedSource
-
Creates a CompressedSource from an underlying FileBasedSource.
- from(BoundedSource<T>) - Method in class com.google.cloud.dataflow.sdk.io.Read.Builder
-
Returns a new Read.Bounded PTransform reading from the given
BoundedSource.
- from(UnboundedSource<T, ?>) - Method in class com.google.cloud.dataflow.sdk.io.Read.Builder
-
Returns a new Read.Unbounded PTransform reading from the given
UnboundedSource.
- from(BoundedSource<T>) - Static method in class com.google.cloud.dataflow.sdk.io.Read
-
Returns a new Read.Bounded PTransform reading from the given
BoundedSource.
- from(UnboundedSource<T, ?>) - Static method in class com.google.cloud.dataflow.sdk.io.Read
-
Returns a new Read.Unbounded PTransform reading from the given
UnboundedSource.
- from(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
-
Returns a new transform for reading from text files that's like this one but
that reads from the file(s) with the given name or pattern.
- from(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
-
Returns a transform for reading text files that reads from the file(s)
with the given filename or filename pattern.
- from(String) - Static method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
Creates an XmlSource for a single XML file or a set of XML files defined by a Java "glob" file
pattern.
- from(Map<String, String>) - Static method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
-
Expects a map keyed by logger Names with values representing Levels.
- from(HasDisplayData) - Static method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData
-
- fromArgs(String[]) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
-
Sets the command line arguments to parse when constructing the
PipelineOptions.
- fromArgs(String[]) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
Sets the command line arguments to parse when constructing the
PipelineOptions.
- fromIterableInternal(Iterable<WindowedValue<?>>) - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
-
For internal use only.
- fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
-
Constructs a runner from the provided options.
- fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
-
Construct a runner from the provided options.
- fromOptions(DataflowPipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
-
Constructs a translator from the provided options.
- fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
-
Constructs a DirectPipelineRunner from the given options.
- fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner
-
Constructs a runner from the provided options.
- fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
-
Constructs a runner from the provided options.
- fromOptions(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
-
- fromPath(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription
-
Creates a class representing a Pub/Sub subscription from the specified subscription path.
- fromPath(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic
-
Creates a class representing a Cloud Pub/Sub topic from the specified topic path.
- fromQuery(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
Returns a copy of this transform that reads the results of the specified query.
- fromQuery(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
-
Reads results received after executing the given query.
- fromStaticMethods(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderFactories
-
Creates a
CoderFactory built from particular static methods of a class that
implements
Coder.
- fromStaticMethods(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CoderProviders
-
Creates a
CoderProvider built from particular static methods of a class that
implements
Coder.
- GcpOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Options used to configure Google Cloud Platform project and credentials.
- GcpOptions.CredentialDirFactory - Class in com.google.cloud.dataflow.sdk.options
-
Returns the default credential directory of ${user.home}/.store/data-flow.
- GcpOptions.DefaultProjectFactory - Class in com.google.cloud.dataflow.sdk.options
-
Attempts to infer the default project based upon the environment this application
is executing within.
- GcpOptions.GcpUserCredentialsFactory - Class in com.google.cloud.dataflow.sdk.options
-
Attempts to load the GCP credentials.
- GcpUserCredentialsFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcpOptions.GcpUserCredentialsFactory
-
- GcsOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Options used to configure Google Cloud Storage.
- GcsOptions.ExecutorServiceFactory - Class in com.google.cloud.dataflow.sdk.options
-
- generateDestinationFilenames(int) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Generate output bundle filenames.
- generateInitialSplits(int, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
-
Returns a list of UnboundedSource objects representing the instances of this source
that should be used when executing the workflow.
- get(TupleTag<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.CoCombineResult
-
Returns the value represented by the given
TupleTag.
- get(int) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
Returns the
PCollection at the given index (origin zero).
- get(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
-
- get(int) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
-
Returns the
TupleTag at the given index (origin zero).
- getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
-
- getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn
-
- getAccumulatorCoder(CoderRegistry, Coder<Double>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
-
- getAccumulatorCoder(CoderRegistry, Coder<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
-
- getAccumulatorCoder(CoderRegistry, Coder<Integer>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
-
- getAccumulatorCoder(CoderRegistry, Coder<Long>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
-
- getAccumulatorCoder(CoderRegistry, Coder<InputT>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
-
Returns the Coder to use for accumulator AccumT
values, or null if it is not able to be inferred.
- getAccumulatorCoder(CoderRegistry, Coder<K>, Coder<InputT>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.PerKeyCombineFn
-
Returns the Coder to use for accumulator AccumT
values, or null if it is not able to be inferred.
- getAccumulatorCoder(CoderRegistry, Coder<DataT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
-
- getAccumulatorCoder(CoderRegistry, Coder<DataT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
-
- getAccumulatorCoder(CoderRegistry, Coder<K>, Coder<DataT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
-
- getAccumulatorCoder(CoderRegistry, Coder<K>, Coder<DataT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
-
- getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
-
- getAccumulatorCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
-
- getAddCounterMutator() - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
Returns a mutator, which can be used to add additional counters to
this EvaluationContext.
- getAggregators(DoFn<?, ?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.AggregatorRetriever
-
- getAggregatorSteps() - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorPipelineExtractor
-
- getAggregatorValue(Aggregator<?, AggregateT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
- getAggregatorValues(Aggregator<?, T>) - Method in interface com.google.cloud.dataflow.sdk.PipelineResult
-
Retrieves the current value of the provided
Aggregator.
- getAggregatorValues(Aggregator<?, OutputT>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
-
- getAlgorithm() - Method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
-
Returns the string representation of this type.
- getAll(TupleTag<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
-
Returns the values from the table represented by the given
TupleTag<V> as an Iterable<V> (which may be empty if there
are no results).
- getAll() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
- getAll() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
-
- getAll() - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
-
- getAllowedEncodings() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
-
- getAllowedEncodings() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
-
- getAllowedEncodings() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- getAllowedTimestampSkew() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
-
- getAllowedTimestampSkew() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
-
- getAllowedTimestampSkew() - Method in class com.google.cloud.dataflow.sdk.transforms.WithTimestamps
-
Returns the allowed timestamp skew duration, which is the maximum
duration that timestamps can be shifted backwards from the timestamp of the input element.
- getApiRootUrl() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
The root URL for the Dataflow API.
- getAppliedFn(CoderRegistry, Coder<? extends KV<K, ? extends Iterable<InputT>>>, WindowingStrategy<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
-
- getAppName() - Method in interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions
-
Name of application, for display purposes.
- getArgumentTypes(Method) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Returns a list of argument types for the given method, which must
be a part of the class.
- getAuthorizationServerEncodedUrl() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
The authorization server URL to use for OAuth 2 authentication.
- getAutoscalingAlgorithm() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
[Experimental] The autoscaling algorithm to use for the workerpool.
- getBaseOutputFilename() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink
-
Returns the base output filename for this file based sink.
- getBigtableOptions() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
-
Returns the Google Cloud Bigtable cluster being read from, and other parameters.
- getBigtableOptions() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
-
Returns the Google Cloud Bigtable cluster being written to, and other parameters.
- getBytes() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
-
Returns a newly-allocated
byte[] representing this
ByteKey.
- getBytesPerOffset() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
Returns approximately how many bytes of data correspond to a single offset in this source.
- getCause() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
-
Returns the reason that this WindowFn is invalid.
- getChannelFactory() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
-
- getCheckpointMark() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
-
- getCheckpointMarkCoder() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource
-
Returns a
Coder for encoding and decoding the checkpoints for this source, or
null if the checkpoints do not need to be durably committed.
- getClasses() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Returns a set of
TypeDescriptors, one for each
superclass (including this class).
- getCoder(TypeDescriptor<T>) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderProvider
-
Provides a coder for a given class, if possible.
- getCoder(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
- getCoder() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
-
Returns the coder used to encode/decode the intermediate values produced/consumed by the
coding functions of this DelegateCoder.
- getCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
- getCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
- getCoder() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
- getCoder() - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
-
Gets the coder, which may yet be absent.
- getCoder() - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
-
Gets the coder, which may yet be absent.
- getCoder() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
- getCoder() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
Returns the
Coder used by this
PCollection to encode and decode
the values stored in it.
- getCoder() - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
-
Returns the
Coder used by this
TypedPValue to encode and decode
the values stored in it.
- getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.AtomicCoder
-
- getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
- getCoderArguments() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
-
If this is a Coder for a parameterized type, returns the
list of Coders being used for each of the parameters, or
returns null if this cannot be done or this is not a
parameterized type.
- getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
-
- getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
- getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
-
If this is a Coder for a parameterized type, returns the
list of Coders being used for each of the parameters, or
returns null if this cannot be done or this is not a
parameterized type.
- getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
-
- getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
-
- getCoderArguments() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
-
- getCoderInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
-
For internal use only.
- getCoderRegistry() - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- getCoGbkResultSchema() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
-
- getCollection() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple.TaggedKeyedPCollection
-
Returns the underlying PCollection of this TaggedKeyedPCollection.
- getCombineFn() - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator
-
- getCombineFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView
-
- getComponents() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
Returns the list of
Coders that are components of this
Coder.
- getComponents() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
-
- getComponentType() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Returns the component type if this type is an array type,
otherwise returns null.
- getCompressionType() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
-
- getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
-
- getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
-
- getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
-
- getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
-
- getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
-
- getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
-
- getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
-
- getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
-
- getContinuationTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
Return a trigger to use after a GroupByKey to preserve the
intention of this trigger.
- getContinuationTrigger(List<Trigger<W>>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
- getContinuationTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnceTrigger
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxDoubleFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxIntegerFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxLongFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinDoubleFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinIntegerFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinLongFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
-
- getCounter(String) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
-
- getCreateDisposition() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns the create disposition.
- getCredentialDir() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
Directory for storing dataflow credentials after execution of the OAuth 2 webflow.
- getCredentialFactoryClass() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
The class of the credential factory that should be created and used to create
credentials.
- getCredentialId() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
This option controls which credential store to use when creating the credentials
using the OAuth 2 webflow.
- getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
-
- getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
-
Gets the current record from the delegate reader.
- getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
-
- getCurrent() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
-
- getCurrent() - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
-
Returns the last TransformTreeNode on the stack.
- getCurrentBlock() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
-
- getCurrentBlock() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
-
- getCurrentBlockOffset() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
-
- getCurrentBlockOffset() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
-
Returns the largest offset such that starting to read from that offset includes the current
block.
- getCurrentBlockSize() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
-
- getCurrentBlockSize() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
-
Returns the size of the current block in bytes as it is represented in the underlying file,
if possible.
- getCurrentOffset() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
-
- getCurrentOffset() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
-
Returns the delegate reader's current offset in the decompressed input.
- getCurrentOffset() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
-
- getCurrentRecord() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
-
Returns the current record.
- getCurrentRecordId() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
-
Returns a unique identifier for the current record.
- getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
-
- getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
-
Returns a Source describing the same input that this Reader currently reads
(including items already read).
- getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
-
- getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
-
- getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
-
- getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
-
Returns a Source describing the same input that this Reader currently reads
(including items already read).
- getCurrentSource() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
-
- getCurrentTime(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
To complete an implementation, return the desired time from the TriggerContext.
- getCurrentTime(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
-
- getCurrentTimestamp() - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
-
By default, returns the minimum possible timestamp.
- getCurrentTimestamp() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
-
Returns the timestamp associated with the current data item.
- getDataflowClient() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
An instance of the Dataflow client.
- getDataflowClient() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
-
Get the Cloud Dataflow API Client used by this job.
- getDataflowEndpoint() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
Dataflow endpoint to use.
- getDataflowJobFile() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
The path to write the translated Dataflow job specification out to
at job submission time.
- getDataset() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- getDayOfMonth() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
-
- getDayOfMonth() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- getDebuggee() - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
-
The Cloud Debugger debuggee to associate with.
- getDefaultCoder(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
Returns the
Coder to use by default for values of the given type.
- getDefaultCoder(TypeDescriptor<OutputT>, TypeDescriptor<InputT>, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
Returns the
Coder to use by default for values of the given type, where the given input
type uses the given
Coder.
- getDefaultCoder(Class<? extends T>, Class<T>, Map<Type, ? extends Coder<?>>, TypeVariable<?>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
Returns the
Coder to use for the specified type parameter specialization of the
subclass, given
Coders to use for all other type parameters (if any).
- getDefaultCoder(T) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
Returns the
Coder to use for the provided example value, if it can be determined.
- getDefaultCoder(Class<T>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
Returns the
Coder to use by default for values of the given class.
- getDefaultCoders(Class<? extends T>, Class<T>, Map<Type, ? extends Coder<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
- getDefaultOutputCoder(SerializableFunction<InputT, OutputT>, Coder<InputT>) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
Returns the
Coder to use on elements produced by this function, given the
Coder
used for its input elements.
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
-
Returns the delegate source's default output coder.
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.Source
-
Returns the default Coder to use for the data read from this source.
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
- getDefaultOutputCoder(CoderRegistry, Coder<Double>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
-
- getDefaultOutputCoder(CoderRegistry, Coder<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
-
- getDefaultOutputCoder(CoderRegistry, Coder<Integer>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
-
- getDefaultOutputCoder(CoderRegistry, Coder<Long>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
-
- getDefaultOutputCoder(PCollection<? extends KV<K, ? extends Iterable<InputT>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
-
- getDefaultOutputCoder(CoderRegistry, Coder<InputT>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
-
Returns the Coder to use by default for output
OutputT values, or null if it is not able to be inferred.
- getDefaultOutputCoder(CoderRegistry, Coder<K>, Coder<InputT>) - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.PerKeyCombineFn
-
Returns the Coder to use by default for output
OutputT values, or null if it is not able to be inferred.
- getDefaultOutputCoder(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
-
- getDefaultOutputCoder(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Filter
-
- getDefaultOutputCoder(PCollectionList<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenPCollectionList
-
- getDefaultOutputCoder(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
-
- getDefaultOutputCoder(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
-
- getDefaultOutputCoder(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- getDefaultOutputCoder(PCollection<? extends InputT>, TypedPValue<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- getDefaultOutputCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
-
Returns the default Coder to use for the output of this
single-output PTransform.
- getDefaultOutputCoder(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
-
Returns the default Coder to use for the output of this
single-output PTransform when applied to the given input.
- getDefaultOutputCoder(InputT, TypedPValue<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
-
Returns the default Coder to use for the given output of
this single-output PTransform when applied to the given input.
- getDefaultOutputCoder(CoderRegistry, Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn
-
- getDefaultOutputCoder(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
- getDefaultWorkerLogLevel() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
-
This option controls the default log level of all loggers without a log level override.
- getDirectPipelineRunnerRandomSeed() - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
-
- getDiskSizeGb() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
Remote worker disk size, in gigabytes, or 0 to use the default size.
- getDoFnClass(DoFn<?, ?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
-
- getDumpHeapOnOOM() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
If true, save a heap dump before killing a thread or process which is GC
thrashing or out of memory.
- getElemCoder() - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
-
- getElements() - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
-
- getEnableCloudDebugger() - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
-
Whether to enable the Cloud Debugger snapshot agent for the current job.
- getEnableProfilingAgent() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions
-
- getEnclosingNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Returns the enclosing composite transform node, or null if there is none.
- getEncodedElementByteSize(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
-
Returns the size in bytes of the encoded value using this coder.
- getEncodedElementByteSize(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
-
Returns the size in bytes of the encoded value using this coder.
- getEncodedElementByteSize(byte[], Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
-
- getEncodedElementByteSize(Byte, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
-
Returns the size in bytes of the encoded value using this coder.
- getEncodedElementByteSize(ByteString, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
-
- getEncodedElementByteSize(Double, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
-
Returns the size in bytes of the encoded value using this coder.
- getEncodedElementByteSize(DatastoreV1.Entity, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
-
- getEncodedElementByteSize(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
-
Overridden to short-circuit the default StandardCoder behavior of encoding and
counting the bytes.
- getEncodedElementByteSize(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
Returns the size in bytes of the encoded value using this coder.
- getEncodedElementByteSize(String, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder
-
Returns the size in bytes of the encoded value using this coder.
- getEncodedElementByteSize(TableRow, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
-
- getEncodedElementByteSize(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
-
- getEncodedElementByteSize(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
-
- getEncodedElementByteSize(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
-
- getEncodedElementByteSize(Void, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
-
- getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
The encoding identifier is designed to support evolution as per the design of Avro
In order to use this class effectively, carefully read the Avro
documentation at
Schema Resolution
to ensure that the old and new schema
match.
- getEncodingId() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
-
- getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
-
- getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
-
- getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
-
- getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
-
Deprecated.
The encoding identifier is designed to support evolution as per the design of Protocol
Buffers.
- getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
-
The encoding identifier is designed to support evolution as per the design of Protocol
Buffers.
- getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
-
- getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- getEncodingId() - Method in class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
-
The encoding id is the fully qualified name of the encoded/decoded class.
- getEndKey() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
-
- getEndOffset() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
Returns the specified ending offset of the source.
- getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
-
An estimate of the total size (in bytes) of the data that would be read from this source.
- getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
- getEstimatedSizeBytes(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
- getExecutorService() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
-
The ExecutorService instance to use to create threads, can be overridden to specify an
ExecutorService that is compatible with the users environment.
- getExpandedOutputs() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Returns the transform outputs, in expanded form.
- getExperiments() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
The list of backend experiments to enable.
- getExtensionRegistry() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
-
Returns the
ExtensionRegistry listing all known Protocol Buffers extension messages
to
T registered with this
ProtoCoder.
- getFallbackCoderProvider() - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
- getFanout() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView
-
- getFilename() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileResult
-
- getFilenamePrefix() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
- getFilenamePrefix() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
- getFilenameSuffix() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
- getFilenameSuffix() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
- getFileOrPatternSpec() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
- getFilepattern() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
-
- getFilepattern() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
-
- getFilesToStage() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
List of local files to make available to workers.
- getFinishedMergingWindows() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.MergingTriggerInfo
-
Return the merging windows in which the trigger is finished.
- getFlattenResults() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
Returns true/false if result flattening is enabled/disabled, or null if not applicable.
- getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
-
Returns the KeyedCombineFn used by this Combine operation.
- getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
-
- getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
-
- getFn() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
-
- getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
-
Returns a value in [0, 1] representing approximately what fraction of the
current source this reader has read so far, or
null if such
an estimate is not available.
- getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
-
- getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
-
- getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
-
- getFractionConsumed() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
-
- getFractionConsumed() - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
-
- getFractionOfBlockConsumed() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block
-
Returns the fraction of the block already consumed, if possible, as a value in
[0, 1].
- getFullName(PTransform<?, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Returns the full name of the currently being translated transform.
- getFullName() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
- getFullName() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
-
- getFullNameForTesting(PTransform<?, ?>) - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- getGapDuration() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
-
- getGcpCredential() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
The credential instance that should be used to authenticate against GCP services.
- getGcsEndpoint() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
-
GCS endpoint to use.
- getGcsUploadBufferSizeBytes() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
-
The buffer size (in bytes) to use when uploading files to GCS.
- getGcsUtil() - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
-
The GcsUtil instance that should be used to communicate with Google Cloud Storage.
- getGoogleApiTrace() - Method in interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions
-
This option enables tracing of API calls to Google services used within the
Dataflow SDK.
- getHost() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- getId() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
-
Returns the id of this TupleTag.
- getIdLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
- getIdLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
- getIncompatibleGlobalWindowErrorMessage() - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn
-
Returns the error message for not supported default values in Combine.globally().
- getIncompatibleGlobalWindowErrorMessage() - Method in class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
-
- getIndex(TupleTag<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
-
Returns the index for the given tuple tag, if the tag is present in this
schema, -1 if it isn't.
- getIndex() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
The zero-based index of this trigger firing that produced this pane.
- getInput(PTransform<InputT, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Returns the input of the currently being translated transform.
- getInput(PTransform<InputT, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
Returns the input of the currently being processed transform.
- getInput() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Returns the transform input, in unexpanded form.
- getInput() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
-
- getInputs() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Returns a mapping of inputs to the producing nodes for all inputs to
the transform.
- getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
-
Returns a
TypeDescriptor capturing what is known statically
about the input type of this
DoFn instance's most-derived
class.
- getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
-
Returns a
TypeDescriptor capturing what is known statically
about the input type of this
DoFnWithContext instance's most-derived
class.
- getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
-
- getInputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.SimpleFunction
-
Returns a
TypeDescriptor capturing what is known statically
about the input type of this
DoFn instance's most-derived
class.
- getInputValueCoder(Coder<KV<K, V>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
-
Returns the Coder of the values of the input to this transform.
- getInsertDefault() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView
-
- getInstanceComponents(T) - Static method in class com.google.cloud.dataflow.sdk.coders.AtomicCoder
-
Returns a list of values contained in the provided example
value, one per type parameter.
- getInstanceComponents(Object) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderFactory
-
Returns a list of objects contained in value, one per
type argument, or null if none can be determined.
- getInstanceComponents(Collection<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.CollectionCoder
-
Returns the first element in this collection if it is non-empty,
otherwise returns null.
- getInstanceComponents(Iterable<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableCoder
-
Returns the first element in this iterable if it is non-empty,
otherwise returns null.
- getInstanceComponents(KV<K, V>) - Static method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
- getInstanceComponents(List<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.ListCoder
-
Returns the first element in this list if it is non-empty,
otherwise returns null.
- getInstanceComponents(Map<K, V>) - Static method in class com.google.cloud.dataflow.sdk.coders.MapCoder
-
Returns the key and value for an arbitrary element of this map,
if it is non-empty, otherwise returns null.
- getInstanceComponents(Set<T>) - Static method in class com.google.cloud.dataflow.sdk.coders.SetCoder
-
Returns the first element in this set if it is non-empty,
otherwise returns null.
- getInstanceComponents(TimestampedValue<T>) - Static method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
-
- getInstanceComponentsHelper(IterableT) - Static method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
-
Returns the first element in the iterable-like exampleValue if it is non-empty,
otherwise returns null.
- getInterfaces() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Returns a set of
TypeDescriptors, one for each
interface implemented by this class.
- getJAXBClass() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder
-
- getJob() - Method in exception com.google.cloud.dataflow.sdk.runners.DataflowJobException
-
Returns the failed job.
- getJob() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.JobSpecification
-
- getJobId() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
-
The identity of the Dataflow job.
- getJobId() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
-
Get the id of this job.
- getJobMessageOutput() - Method in interface com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions
-
Output stream for job status messages.
- getJobName() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
-
The Dataflow job name is used as an idempotence key within the Dataflow service.
- getKey() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
-
Returns the key associated with this element.
- getKey() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Identifier
-
- getKey() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
-
- getKey() - Method in class com.google.cloud.dataflow.sdk.values.KV
-
Returns the key of this
KV.
- getKeyCoder() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
- getKeyCoder() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
-
- getKeyCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
-
- getKeyedCollections() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
-
- getKindString() - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded
-
- getKindString() - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
-
- getKindString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
-
- getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
-
- getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
-
Returns the name to use by default for this PTransform
(not including the names of any enclosing PTransforms).
- getKindString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
- getKindString() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
-
- getLabel() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
-
Retrieve the optional label for an item.
- getLastEmitted() - Method in class com.google.cloud.dataflow.sdk.io.CountingSource.CounterMark
-
Returns the last value emitted by the reader.
- getLinkUrl() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
-
Retrieve the optional link URL for an item.
- getMainOutputTag() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- getMaxConditionCost() - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
-
The maximum cost (as a ratio of CPU time) allowed for evaluating conditional snapshots.
- getMaxEndOffset(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
- getMaxEndOffset(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
Returns the actual ending offset of the current source.
- getMaxNumRecords() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
- getMaxNumWorkers() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
The maximum number of workers to use for the workerpool.
- getMaxReadTime() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
- getMessage() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
-
- getMessageType() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
-
Returns the Protocol Buffers
Message type this
ProtoCoder supports.
- getMinBundleSize() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
Returns the minimum bundle size that should be used when splitting the source into sub-sources.
- getMode() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
- getMonthOfYear() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- getName() - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator
-
Returns the name of the Aggregator.
- getName() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
-
Returns the transform name.
- getName() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
- getName() - Method in interface com.google.cloud.dataflow.sdk.values.PValue
-
Returns the name of this
PValue.
- getName() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
-
- getNamespace() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- getNamespace() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Identifier
-
- getNamespace() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
-
- getNetwork() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- getNonSpeculativeIndex() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
The zero-based index of this trigger firing among non-speculative panes.
- getNumber() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
-
- getNumber() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
-
- getNumber() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- getNumberOfWorkerHarnessThreads() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
Number of threads to use on the Dataflow worker harness.
- getNumShards() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
- getNumShards() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
- getNumWorkers() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
Number of workers to use when executing the Dataflow job.
- getOffset() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
-
- getOffset() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
-
- getOnly(TupleTag<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
-
If there is a singleton value for the given tag, returns it.
- getOnly(TupleTag<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
-
If there is a singleton value for the given tag, returns it.
- getOptions() - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- getOriginalWindowFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
-
Returns the original windowFn that this InvalidWindows replaced.
- getOutName(int) - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
-
If this TupleTag is tagging output outputIndex of
a PTransform, returns the name that should be used by
default for the output.
- getOutput(PTransform<?, OutputT>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Returns the output of the currently being translated transform.
- getOutput(PTransform<?, OutputT>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
Returns the output of the currently being processed transform.
- getOutput() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Returns the transform output, in unexpanded form.
- getOutput() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
-
- getOutputStrategyInternal(WindowingStrategy<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
- getOutputTime(Instant, GlobalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
-
- getOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
-
- getOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
-
- getOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
-
- getOutputTimeFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
-
- getOutputTimeFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
-
Ensures that later sliding windows have an output time that is past the end of earlier windows.
- getOutputTimeFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
-
Provides a default implementation for WindowingStrategy.getOutputTimeFn().
- getOutputType() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn
-
Returns a
TypeDescriptor capturing what is known statically
about the output type of this
CombineFn instance's
most-derived class.
- getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
-
Returns a
TypeDescriptor capturing what is known statically
about the output type of this
DoFn instance's
most-derived class.
- getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext
-
Returns a
TypeDescriptor capturing what is known statically
about the output type of this
DoFnWithContext instance's
most-derived class.
- getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
-
- getOutputTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.transforms.SimpleFunction
-
Returns a
TypeDescriptor capturing what is known statically
about the output type of this
DoFn instance's
most-derived class.
- getOverriddenTransform() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
-
- getOverrideWindmillBinary() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
Custom windmill_main binary to use with the streaming runner.
- getPathValidator() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
The path validator instance that should be used to validate paths.
- getPathValidatorClass() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
The class of the validator that should be created and used to validate paths.
- getPCollection(PCollection<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
-
Retrieves the value of the given PCollection.
- getPCollectionList(PCollectionList<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
-
Retrieves the values of each PCollection in the given
PCollectionList.
- getPCollectionValuesWithMetadata(PCollection<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
Retrieves the value of the given PCollection, along with element metadata
such as timestamps and windows.
- getPCollectionView(PCollectionView<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
-
- getPCollectionWindowedValues(PCollection<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults
-
Retrieves the windowed value of the given PCollection.
- getPeriod() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
-
- getPipeline() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
-
- getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.PBegin
-
- getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
- getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple
-
- getPipeline() - Method in interface com.google.cloud.dataflow.sdk.values.PInput
-
- getPipeline() - Method in interface com.google.cloud.dataflow.sdk.values.POutput
-
- getPipeline() - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
-
- getPipelineOptions() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptionsRegistrar
-
- getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Options
-
- getPipelineOptions() - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext
-
Returns the configured pipeline options.
- getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Options
-
- getPipelineOptions() - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
Returns the configured pipeline options.
- getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
-
- getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.Context
-
Returns the
PipelineOptions specified with the
PipelineRunner
invoking this
KeyedCombineFn.
- getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
-
Returns the
PipelineOptions specified with the
PipelineRunner
invoking this
DoFn.
- getPipelineOptions() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
-
Returns the
PipelineOptions specified with the
PipelineRunner
invoking this
DoFnWithContext.
- getPipelineRunners() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRegistrar.Runner
-
- getPipelineRunners() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRegistrar.Runner
-
- getPipelineRunners() - Method in interface com.google.cloud.dataflow.sdk.runners.PipelineRunnerRegistrar
-
- getPositionForFractionConsumed(double) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
-
Returns a position P such that the range [start, P) represents approximately
the given fraction of the range [start, end).
- getProducingTransformInternal() - Method in class com.google.cloud.dataflow.sdk.values.POutputValueBase
-
- getProducingTransformInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PValue
-
- getProfilingAgentConfiguration() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions
-
- getProject() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
-
- getProject() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
Project id to use when launching jobs.
- getProjectId() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
-
Get the project this job exists in.
- getPubsubRootUrl() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
Root URL for use with the Pubsub API.
- getQuery() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
Returns the query to be read, or null if reading from a table instead.
- getQuery() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- getRawType() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
- getReason() - Method in exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
-
- getReasons() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.NonDeterministicException
-
- getRecordClass() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
- getRecordElement() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
- getRecordType() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
-
- getRegisteredOptions() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
- getReplacedByJob() - Method in exception com.google.cloud.dataflow.sdk.runners.DataflowJobUpdatedException
-
The new job that replaces the job terminated with this exception.
- getReplacedByJob() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
-
- getRootCause() - Method in exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException
-
- getRootElement() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
- getRunner() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
-
The pipeline runner that will be used to execute the pipeline.
- getRunner() - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- getRunner() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
-
- getRunner() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
-
- getSchema() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
Returns the schema used by this coder.
- getSchema() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
-
- getSchema() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
- getSchema() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
- getSchema() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns the table schema.
- getSchema() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
-
- getSecretsFile() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
This option controls which file to use when attempting to create the credentials
using the OAuth 2 webflow.
- getServiceAccountKeyfile() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
This option controls which file to use when attempting to create the credentials using the
service account method.
- getServiceAccountName() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
This option controls which service account to use when attempting to create the credentials
using the service account method.
- getShardNameTemplate() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns the current shard name template string.
- getShardNameTemplate() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
Returns the current shard name template string.
- getShardTemplate() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
- getShardTemplate() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
- getShortValue() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
-
Return the optional short value for an item.
- getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
-
- getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
-
Returns the side inputs used by this Combine operation.
- getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
-
- getSideInputs() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
-
- getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
-
- getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
-
- getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
-
- getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
-
Return the earliest window that contains the end of the main-input window.
- getSideInputWindow(BoundedWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
-
Returns the window of the side input corresponding to the given window of
the main input.
- getSideOutputTags() - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- getSink() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Returns the FileBasedSink for this write operation.
- getSink() - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
-
Returns the Sink that this write operation writes to.
- getSink() - Method in class com.google.cloud.dataflow.sdk.io.Write.Bound
-
Returns the
Sink associated with this PTransform.
- getSink() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriteOperation
-
Return the XmlSink.Bound for this write operation.
- getSize() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
-
- getSize() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
-
- getSource() - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded
-
Returns the BoundedSource used to create this Read PTransform.
- getSource() - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
-
Returns the UnboundedSource used to create this Read PTransform.
- getSplitBacklogBytes() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
-
Returns the size of the backlog of unread data in the underlying data source represented by
this split of this source.
- getStableUniqueNames() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
-
Whether to check for stable unique names on each transform.
- getStager() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
The resource stager instance that should be used to stage resources.
- getStagerClass() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
The class responsible for staging resources to be accessible by workers
during job execution.
- getStagingLocation() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
-
GCS path for staging local files, e.g.
- getStartDate() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
-
- getStartDate() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
-
- getStartDate() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- getStartKey() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
-
- getStartOffset() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
Returns the starting offset of the source.
- getStartPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
-
- getStartPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
-
- getStartPosition() - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
-
Returns the starting position of the current range, inclusive.
- getState() - Method in interface com.google.cloud.dataflow.sdk.PipelineResult
-
Retrieves the current state of the pipeline execution.
- getState() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
-
- getStepName(PTransform<?, ?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
Gets the step name for this transform.
- getStepNames() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.JobSpecification
-
Returns the mapping of
AppliedPTransforms to the internal step
name for that
AppliedPTransform.
- getStopPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
-
- getStopPosition() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
-
- getStopPosition() - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
-
Returns the ending position of the current range, exclusive.
- getSubnetwork() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- getSubscription() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
- getSupertype(Class<? super T>) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Returns the generic form of a supertype.
- getTable() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
Returns the table to write, or null if reading from a query instead.
- getTable() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns the table reference, or null if a .
- getTableId() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
-
Returns the table being read from.
- getTableId() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
-
Returns the table being written to.
- getTag(int) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
-
Returns the tuple tag at the given index.
- getTagInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
-
A unique identifier, for internal use.
- getTeardownPolicy() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
The teardown policy for the VMs.
- getTeardownPolicyName() - Method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.TeardownPolicy
-
- getTempDatasetId() - Method in interface com.google.cloud.dataflow.sdk.options.BigQueryOptions
-
- getTempLocation() - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
-
A pipeline level default location for storing temporary files.
- getTimestamp() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
-
Returns the timestamp associated with this element.
- getTimestamp() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
-
- getTimestampLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
- getTimestampLabel() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
- getTimeZone() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
-
- getTimeZone() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
-
- getTimeZone() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- getTiming() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
Return true if this is the last pane that will be produced in the associated window.
- getTokenServerUrl() - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
The token server URL to use for OAuth 2 authentication.
- getTopic() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
- getTopic() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
- getTotalBacklogBytes() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
-
Returns the size of the backlog of unread data in the underlying data source represented by
all splits of this source.
- getTotalValue(Combine.CombineFn<T, ?, T>) - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorValues
-
- getTransform() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Returns the transform associated with this transform node.
- getTransform() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
-
- getTransformEvaluator(Class<TransformT>) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
-
Returns the TransformEvaluator to use for instances of the
specified PTransform class, or null if none registered.
- getTransformNameMapping() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
Mapping of old PTranform names to new ones, specified as JSON
{"oldName":"newName",...}.
- getTransformTranslator(Class<TransformT>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
-
- getTranslator() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
-
Returns the DataflowPipelineTranslator associated with this object.
- getTupleTag() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple.TaggedKeyedPCollection
-
Returns the TupleTag of this TaggedKeyedPCollection.
- getTupleTagList() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
-
Returns the TupleTagList tuple associated with this schema.
- getType() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
Returns the type this coder encodes/decodes.
- getType() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
- getType() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
-
Retrieve the DisplayData.Type of display metadata.
- getType() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
- getTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
-
Returns a TypeDescriptor capturing what is known statically
about the type of this TupleTag instance's most-derived
class.
- getTypeDescriptor() - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
-
- getTypeParameter(String) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
- getUnionTag() - Method in class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
-
- getUpdate() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- getUpdate() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
-
Whether to update the currently running pipeline with the same name as this one.
- getValidate() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
Returns true if table validation is enabled.
- getValidate() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns true if table validation is enabled.
- getValue() - Method in class com.google.cloud.dataflow.sdk.coders.StructuralByteArray
-
- getValue() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
-
Returns an immutable
ByteString representing this
ByteKey.
- getValue() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
-
Returns the value associated with this element.
- getValue() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
-
Retrieve the value of the metadata item.
- getValue() - Method in class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
-
- getValue() - Method in class com.google.cloud.dataflow.sdk.values.KV
-
Returns the value of this
KV.
- getValue() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
-
- getValueCoder() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
- getValueCoder() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
-
- getValues() - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorValues
-
Get the values of the
Aggregator at all steps it was used.
- getValuesAtSteps() - Method in class com.google.cloud.dataflow.sdk.runners.AggregatorValues
-
Get the values of the
Aggregator by the user name at each step it was used.
- getView() - Method in class com.google.cloud.dataflow.sdk.transforms.View.CreatePCollectionView
-
- getWatermark() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
-
Returns a timestamp before or at the timestamps of all future elements read by this reader.
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
-
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
-
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
-
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
-
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
-
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
-
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
-
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
-
- getWatermarkThatGuaranteesFiring(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
Returns a bound in watermark time by which this trigger would have fired at least once
for a given window had there been input data.
- getWindowedValue() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
-
Returns the WindowedValue associated with this element.
- getWindowFn() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
- getWindowingStrategy() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
- getWindowingStrategyInternal() - Method in interface com.google.cloud.dataflow.sdk.values.PCollectionView
-
For internal use only.
- getWindows() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
-
Returns the collection of windows this element has been placed into.
- getWorkerCacheMb() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
-
The size of the worker's in-memory cache, in megabytes.
- getWorkerDiskType() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
Specifies what type of persistent disk should be used.
- getWorkerHarnessContainerImage() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
Docker container image that executes Dataflow worker harness, residing in Google Container
Registry.
- getWorkerId() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
-
The identity of the worker running this pipeline.
- getWorkerLogLevelOverrides() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
-
This option controls the log levels for specifically named loggers.
- getWorkerMachineType() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
Machine type to create Dataflow worker VMs as.
- getWriteDisposition() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns the write disposition.
- getWriteOperation() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
-
Return the FileBasedWriteOperation that this Writer belongs to.
- getWriteOperation() - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
-
Returns the write operation this writer belongs to.
- getWriteOperation() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
-
Return the XmlWriteOperation this write belongs to.
- getWriterResultCoder() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
- getWriterResultCoder() - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
-
Returns a coder for the writer result type.
- getZone() - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- globally(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
-
Returns a PTransform that takes a PCollection<T>
and returns a PCollection<List<T>> whose single value is a
List of the approximate N-tiles of the elements
of the input PCollection.
- globally(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
-
- globally(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
-
Returns a PTransform that takes a PCollection<T>
and returns a PCollection<Long> containing a single value
that is an estimate of the number of distinct elements in the
input PCollection.
- globally(double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
-
- globally(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
-
Returns a
Combine.Globally PTransform
that uses the given
SerializableFunction to combine all
the elements in each window of the input
PCollection into a
single value in the output
PCollection.
- globally(CombineFnBase.GlobalCombineFn<? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
-
Returns a
Combine.Globally PTransform
that uses the given
GloballyCombineFn to combine all
the elements in each window of the input
PCollection into a
single value in the output
PCollection.
- globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Count
-
- globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
-
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> whose contents is the maximum according to the natural ordering of T
of the input PCollection's elements, or null if there are no elements.
- globally(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
-
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> whose contents is the maximum of the input PCollection's elements, or
null if there are no elements.
- globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Mean
-
Returns a PTransform that takes an input
PCollection<NumT> and returns a
PCollection<Double> whose contents is the mean of the
input PCollection's elements, or
0 if there are no elements.
- globally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
-
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> whose contents is the minimum according to the natural ordering of T
of the input PCollection's elements, or null if there are no elements.
- globally(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
-
Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> whose contents is the minimum of the input PCollection's elements, or
null if there are no elements.
- GlobalWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
The default window into which all data is placed (via
GlobalWindows).
- GlobalWindow.Coder - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
- GlobalWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
Default
WindowFn that assigns all data to the same window.
- GlobalWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
-
- GoogleApiDebugOptions - Interface in com.google.cloud.dataflow.sdk.options
-
These options configure debug settings for Google API clients created within the Dataflow SDK.
- GoogleApiDebugOptions.GoogleApiTracer - Class in com.google.cloud.dataflow.sdk.options
-
- GoogleApiTracer() - Constructor for class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
-
- greaterThan(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
-
Returns a PTransform that takes an input
PCollection<T> and returns a PCollection<T> with
elements that are greater than a given value, based on the
elements' natural ordering.
- greaterThanEq(T) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter
-
Returns a PTransform that takes an input
PCollection<T> and returns a PCollection<T> with
elements that are greater than or equal to a given value, based on
the elements' natural ordering.
- GroupAlsoByWindow(WindowingStrategy<?, ?>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupAlsoByWindow
-
- GroupByKey<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
-
GroupByKey<K, V> takes a PCollection<KV<K, V>>,
groups the values by key and windows, and returns a
PCollection<KV<K, Iterable<V>>> representing a map from
each distinct key and window of the input PCollection to an
Iterable over all the values associated with that key in
the input per window.
- GroupByKey.GroupAlsoByWindow<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
-
Helper transform that takes a collection of timestamp-ordered
values associated with each key, groups the values by window,
combines windows as needed, and for each window in each key,
outputs a collection of key/value-list pairs implicitly assigned
to the window and with the timestamp derived from that window.
- GroupByKey.GroupByKeyOnly<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
-
Primitive helper transform that groups by key only, ignoring any
window assignments.
- GroupByKey.ReifyTimestampsAndWindows<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
-
Helper transform that makes timestamps and window assignments
explicit in the value part of each key/value pair.
- GroupByKey.SortValuesByTimestamp<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
-
Helper transform that sorts the values associated with each key
by timestamp.
- GroupByKeyOnly() - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly
-
- groupedValues(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
-
Returns a
Combine.GroupedValues
PTransform that takes a
PCollection of
KVs where a key maps to an
Iterable of values, e.g.,
the result of a
GroupByKey, then uses the given
SerializableFunction to combine all the values associated
with a key, ignoring the key.
- groupedValues(CombineFnBase.GlobalCombineFn<? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
-
Returns a
Combine.GroupedValues
PTransform that takes a
PCollection of
KVs where a key maps to an
Iterable of values, e.g.,
the result of a
GroupByKey, then uses the given
CombineFn to combine all the values associated with a
key, ignoring the key.
- groupedValues(CombineFnBase.PerKeyCombineFn<? super K, ? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
-
Returns a
Combine.GroupedValues
PTransform that takes a
PCollection of
KVs where a key maps to an
Iterable of values, e.g.,
the result of a
GroupByKey, then uses the given
KeyedCombineFn to combine all the values associated with
each key.
- identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn
-
Returns the identity element of this operation, i.e.
- identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn
-
Returns the value that should be used for the combine of the empty set.
- identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn
-
Returns the identity element of this operation, i.e.
- identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn
-
Returns the identity element of this operation, i.e.
- identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn
-
- identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn
-
- identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
-
- identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
-
- identity() - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
-
- IDENTITY - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- idLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
Returns a transform that's like this one but that reads unique message IDs
from the given message attribute.
- idLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
-
Creates and returns a transform for reading from Cloud Pub/Sub where unique record
identifiers are expected to be provided as Pub/Sub message attributes.
- idLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
Returns a new transform that's like this one but that publishes unique record IDs
to a message attribute with the specified name.
- idLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
-
Creates a transform that writes to Pub/Sub, adding each record's unique identifier to the
published messages in an attribute with the specified name.
- in(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PBegin
-
- in(Pipeline) - Static method in class com.google.cloud.dataflow.sdk.values.PDone
-
- include(HasDisplayData) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder
-
Include display metadata from the specified subcomponent.
- INDEX_OF_MAX - Static variable in class com.google.cloud.dataflow.sdk.io.ShardNameTemplate
-
Shard name containing the index and max.
- initialize(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Initialization of the sink.
- initialize(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
-
Performs initialization before writing to the sink.
- initialize(AbstractGoogleClientRequest<?>) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer
-
- inOrder(Trigger<W>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
-
Returns an AfterEach Trigger with the given subtriggers.
- INSTANCE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow.Coder
-
- INSTANCE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindow
-
- INSTANCE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
-
- InstantCoder - Class in com.google.cloud.dataflow.sdk.coders
-
A
Coder for joda
Instant that encodes it as a big endian
Long
shifted such that lexicographic ordering of the bytes corresponds to chronological order.
- integersGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
-
Returns a PTransform that takes an input PCollection<Integer> and returns a
PCollection<Integer> whose contents is the maximum of the input PCollection's
elements, or Integer.MIN_VALUE if there are no elements.
- integersGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
-
Returns a PTransform that takes an input PCollection<Integer> and returns a
PCollection<Integer> whose contents is a single value that is the minimum of the input
PCollection's elements, or Integer.MAX_VALUE if there are no elements.
- integersGlobally() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
-
Returns a PTransform that takes an input
PCollection<Integer> and returns a
PCollection<Integer> whose contents is the sum of the
input PCollection's elements, or
0 if there are no elements.
- integersPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
-
Returns a PTransform that takes an input PCollection<KV<K, Integer>> and
returns a PCollection<KV<K, Integer>> that contains an output element mapping each
distinct key in the input PCollection to the maximum of the values associated with that
key in the input PCollection.
- integersPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
-
Returns a PTransform that takes an input PCollection<KV<K, Integer>> and
returns a PCollection<KV<K, Integer>> that contains an output element mapping each
distinct key in the input PCollection to the minimum of the values associated with that
key in the input PCollection.
- integersPerKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Sum
-
Returns a PTransform that takes an input
PCollection<KV<K, Integer>> and returns a
PCollection<KV<K, Integer>> that contains an output
element mapping each distinct key in the input
PCollection to the sum of the values associated with
that key in the input PCollection.
- interpolateKey(double) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
-
Returns a
ByteKey key such that
[startKey, key) represents
approximately the specified fraction of the range
[startKey, endKey).
- intersects(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
Returns whether this window intersects the given window.
- IntervalWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
- IntervalWindow(Instant, Instant) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
Creates a new IntervalWindow that represents the half-open time
interval [start, end).
- IntervalWindow(Instant, ReadableDuration) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
- into(WindowFn<? super T, ?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
-
Creates a
Window PTransform that uses the given
WindowFn to window the data.
- into(WindowFn<? super T, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
-
Returns a new
Window PTransform that's like this
transform but that will use the given
WindowFn, and that has
its input and output types bound.
- IntraBundleParallelization - Class in com.google.cloud.dataflow.sdk.transforms
-
Provides multi-threading of
DoFns, using threaded execution to
process multiple elements concurrently within a bundle.
- IntraBundleParallelization() - Constructor for class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization
-
- IntraBundleParallelization.Bound<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
A PTransform that, when applied to a PCollection<InputT>,
invokes a user-specified DoFn<InputT, OutputT> on all its elements,
with all its outputs collected into an output
PCollection<OutputT>.
- IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
A multi-threaded DoFn wrapper.
- IntraBundleParallelization.Unbound - Class in com.google.cloud.dataflow.sdk.transforms
-
An incomplete IntraBundleParallelization transform, with unbound input/output types.
- InvalidWindows<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A
WindowFn that represents an invalid pipeline state.
- InvalidWindows(String, WindowFn<?, W>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
-
- is(T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
-
- isArray() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Returns true if this type is known to be an array type.
- isAtSplitPoint() - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader
-
Returns true if the reader is at a split point.
- isAtSplitPoint() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
-
Returns true only for the first record; compressed sources cannot be split.
- isAtSplitPoint() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
-
- isBounded() - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
- isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
-
- isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
-
- isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
-
- isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
-
- isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
-
- isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
-
- isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
-
InvalidWindows objects with the same originalWindowFn are compatible.
- isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
-
- isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
-
- isCompatible(Trigger<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
Returns whether this performs the same triggering as the given Trigger.
- isCompatible(WindowFn<?, ?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
-
Returns whether this performs the same merging as the given
WindowFn.
- isCompositeNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Returns true if this node represents a composite transform that does not perform
processing of its own, but merely encapsulates a sub-pipeline (which may be empty).
- isDisjoint(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
Returns whether this window is disjoint from the given window.
- isEmpty() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
-
Returns
true if the
byte[] backing this
ByteKey is of length 0.
- isEmpty() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
-
- isEmpty() - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple
-
- isEqualTo(T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
-
Checks that the value of this SingletonAssert's view is equal
to the expected value.
- isFinished() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
-
Returns true if the current trigger is marked finished.
- isFinished(int) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
-
Return true if the given subtrigger is marked finished.
- isFinishedSpecifyingInternal() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
-
Returns whether this
PValueBase has been finalized, and
its core properties, e.g., name, can no longer be changed.
- isFirst() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
Return true if this is the first pane produced for the associated window.
- isLast() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
Return true if this is the last pane that will be produced in the associated window.
- isMerging() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
-
Returns true if the windowing strategy of the current PCollection is a merging
WindowFn.
- isNonMerging() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.NonMergingWindowFn
-
- isNonMerging() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
-
Returns true if this WindowFn never needs to merge any windows.
- isRegisterByteSizeObserverCheap(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder
-
- isRegisterByteSizeObserverCheap(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder
-
- isRegisterByteSizeObserverCheap(byte[], Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
-
- isRegisterByteSizeObserverCheap(Byte, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
-
- isRegisterByteSizeObserverCheap(ByteString, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder
-
- isRegisterByteSizeObserverCheap(T, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
-
- isRegisterByteSizeObserverCheap(Double, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
-
- isRegisterByteSizeObserverCheap(ReadableDuration, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder
-
- isRegisterByteSizeObserverCheap(Instant, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder
-
- isRegisterByteSizeObserverCheap(IterableT, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
-
- isRegisterByteSizeObserverCheap(KV<K, V>, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
Returns whether both keyCoder and valueCoder are considered not expensive.
- isRegisterByteSizeObserverCheap(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
-
NullableCoder is cheap if valueCoder is cheap.
- isRegisterByteSizeObserverCheap(T, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- isRegisterByteSizeObserverCheap(Integer, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder
-
- isRegisterByteSizeObserverCheap(Long, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder
-
- isRegisterByteSizeObserverCheap(Void, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder
-
- isRootNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
- isSplittable() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
-
Determines whether a single file represented by this source is splittable.
- isSplittable() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
Determines whether a file represented by this source is can be split into bundles.
- isStreaming() - Method in interface com.google.cloud.dataflow.sdk.options.StreamingOptions
-
Set to true if running a streaming pipeline.
- isSubtypeOf(TypeDescriptor<?>) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Return true if this type is a subtype of the given type.
- isSupertypeOf(TypeDescriptor<?>) - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Returns true if this type is assignable from the given type.
- isTerminal() - Method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
-
- isTestEncodability() - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
-
Controls whether the runner should ensure that all of the elements of
every
PCollection can be encoded using the appropriate
Coder.
- isTestSerializability() - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
-
Controls whether the runner should ensure that all of the elements of
the pipeline, such as DoFns, can be serialized.
- isTestUnorderedness() - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
-
Controls whether the runner should randomize the order of each
PCollection.
- isUnknown() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
Return true if there is no timing information for the current
PaneInfo.
- isWholeStream - Variable in class com.google.cloud.dataflow.sdk.coders.Coder.Context
-
Whether the encoded or decoded value fills the remainder of the
output or input (resp.) record/stream contents.
- items() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData
-
- IterableAssert(PTransform<PBegin, PCollectionView<Iterable<T>>>, Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
-
- IterableCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
-
- IterableCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.IterableCoder
-
- IterableLikeCoder<T,IterableT extends Iterable<T>> - Class in com.google.cloud.dataflow.sdk.coders
-
An abstract base class with functionality for assembling a
Coder for a class that implements
Iterable.
- IterableLikeCoder(Coder<T>, String) - Constructor for class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
-
- iterables() - Static method in class com.google.cloud.dataflow.sdk.transforms.Flatten
-
Returns a PTransform that takes a PCollection<Iterable<T>>
and returns a PCollection<T> containing all the elements from
all the Iterables.
- pane() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
-
Returns information about the pane within this window into which the
input element has been assigned.
- pane() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
-
Returns information about the pane within this window into which the
input element has been assigned.
- PaneInfo - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
Provides information about the pane an element belongs to.
- PaneInfo.PaneInfoCoder - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A Coder for encoding PaneInfo instances.
- PaneInfo.Timing - Enum in com.google.cloud.dataflow.sdk.transforms.windowing
-
Enumerates the possibilities for the timing of this pane firing related to the
input and output watermarks for its computation.
- PaneInfoCoder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.PaneInfoCoder
-
- ParDo - Class in com.google.cloud.dataflow.sdk.transforms
-
ParDo is the core element-wise transform in Google Cloud
Dataflow, invoking a user-specified function on each of the elements of the input
PCollection to produce zero or more output elements, all
of which are collected into the output
PCollection.
- ParDo() - Constructor for class com.google.cloud.dataflow.sdk.transforms.ParDo
-
- ParDo.Bound<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
A
PTransform that, when applied to a
PCollection<InputT>,
invokes a user-specified
DoFn<InputT, OutputT> on all its elements,
with all its outputs collected into an output
PCollection<OutputT>.
- ParDo.BoundMulti<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
A
PTransform that, when applied to a
PCollection<InputT>, invokes a user-specified
DoFn<InputT, OutputT> on all its elements, which can emit elements
to any of the
PTransform's main and side output
PCollections, which are bundled into a result
PCollectionTuple.
- ParDo.Unbound - Class in com.google.cloud.dataflow.sdk.transforms
-
An incomplete
ParDo transform, with unbound input/output types.
- ParDo.UnboundMulti<OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
An incomplete multi-output
ParDo transform, with unbound
input type.
- parseTableSpec(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO
-
Parse a table specification in the form
"[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]".
- Partition<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
Partition takes a PCollection<T> and a
PartitionFn, uses the PartitionFn to split the
elements of the input PCollection into N partitions, and
returns a PCollectionList<T> that bundles N
PCollection<T>s containing the split elements.
- Partition.PartitionFn<T> - Interface in com.google.cloud.dataflow.sdk.transforms
-
A function object that chooses an output partition for an element.
- partitionFor(T, int) - Method in interface com.google.cloud.dataflow.sdk.transforms.Partition.PartitionFn
-
Chooses the partition into which to put the given element.
- PartitioningWindowFn<T,W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A
WindowFn that places each value into exactly one window based on its timestamp and
never merges windows.
- PartitioningWindowFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn
-
- pastEndOfWindow() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark
-
Creates a trigger that fires when the watermark passes the end of the window.
- pastFirstElementInPane() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
-
Creates a trigger that fires when the current processing time passes the processing time
at which this trigger saw the first element in a pane.
- PathValidatorFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.PathValidatorFactory
-
- PBegin - Class in com.google.cloud.dataflow.sdk.values
-
- PBegin(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.PBegin
-
- PCollection<T> - Class in com.google.cloud.dataflow.sdk.values
-
- PCollection.IsBounded - Enum in com.google.cloud.dataflow.sdk.values
-
The enumeration of cases for whether a
PCollection is bounded.
- PCollectionList<T> - Class in com.google.cloud.dataflow.sdk.values
-
- pCollections() - Static method in class com.google.cloud.dataflow.sdk.transforms.Flatten
-
- PCollectionTuple - Class in com.google.cloud.dataflow.sdk.values
-
- PCollectionView<T> - Interface in com.google.cloud.dataflow.sdk.values
-
- PDone - Class in com.google.cloud.dataflow.sdk.values
-
PDone is the output of a
PTransform that has a trivial result,
such as a
Write.
- peekOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Returns the elements output so far to the main output.
- peekOutputElementsWithTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Returns the elements output so far to the main output with associated timestamps.
- peekSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Returns the elements output so far to the side output with the
given tag.
- perElement() - Static method in class com.google.cloud.dataflow.sdk.transforms.Count
-
- PerElement() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Count.PerElement
-
- perKey(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
-
Returns a PTransform that takes a
PCollection<KV<K, V>> and returns a
PCollection<KV<K, List<V>>> that contains an output
element mapping each distinct key in the input
PCollection to a List of the approximate
N-tiles of the values associated with that key in the
input PCollection.
- perKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles
-
- perKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
-
Returns a PTransform that takes a
PCollection<KV<K, V>> and returns a
PCollection<KV<K, Long>> that contains an output element
mapping each distinct key in the input PCollection to an
estimate of the number of distinct values associated with that
key in the input PCollection.
- perKey(double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique
-
- perKey(SerializableFunction<Iterable<V>, V>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
-
Returns a
Combine.PerKey PTransform that
first groups its input
PCollection of
KVs by keys and
windows, then invokes the given function on each of the values lists to
produce a combined value, and then returns a
PCollection
of
KVs mapping each distinct key to its combined value for each
window.
- perKey(CombineFnBase.GlobalCombineFn<? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
-
Returns a
Combine.PerKey PTransform that
first groups its input
PCollection of
KVs by keys and
windows, then invokes the given function on each of the values lists to
produce a combined value, and then returns a
PCollection
of
KVs mapping each distinct key to its combined value for each
window.
- perKey(CombineFnBase.PerKeyCombineFn<? super K, ? super InputT, ?, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Combine
-
Returns a
Combine.PerKey PTransform that
first groups its input
PCollection of
KVs by keys and
windows, then invokes the given function on each of the key/values-lists
pairs to produce a combined value, and then returns a
PCollection of
KVs mapping each distinct key to
its combined value for each window.
- perKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Count
-
- perKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
-
Returns a PTransform that takes an input PCollection<KV<K, T>> and returns a
PCollection<KV<K, T>> that contains an output element mapping each distinct key in the
input PCollection to the maximum according to the natural ordering of T of the
values associated with that key in the input PCollection.
- perKey(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Max
-
Returns a PTransform that takes an input PCollection<KV<K, T>> and returns a
PCollection<KV<K, T>> that contains one output element per key mapping each
to the maximum of the values associated with that key in the input PCollection.
- perKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Mean
-
Returns a PTransform that takes an input
PCollection<KV<K, N>> and returns a
PCollection<KV<K, Double>> that contains an output
element mapping each distinct key in the input
PCollection to the mean of the values associated with
that key in the input PCollection.
- perKey() - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
-
Returns a PTransform that takes an input PCollection<KV<K, T>> and returns a
PCollection<KV<K, T>> that contains an output element mapping each distinct key in the
input PCollection to the minimum according to the natural ordering of T of the
values associated with that key in the input PCollection.
- perKey(ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Min
-
Returns a PTransform that takes an input PCollection<KV<K, T>> and returns a
PCollection<KV<K, T>> that contains one output element per key mapping each
to the minimum of the values associated with that key in the input PCollection.
- perKey(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
-
Returns a PTransform that takes an input
PCollection<KV<K, V>> and returns a
PCollection<KV<K, List<V>>> that contains an output
element mapping each distinct key in the input
PCollection to the largest count values
associated with that key in the input
PCollection<KV<K, V>>, in decreasing order, sorted using
the given Comparator<V>.
- PInput - Interface in com.google.cloud.dataflow.sdk.values
-
The interface for things that might be input to a
PTransform.
- Pipeline - Class in com.google.cloud.dataflow.sdk
-
- Pipeline(PipelineRunner<?>) - Constructor for class com.google.cloud.dataflow.sdk.Pipeline
-
- Pipeline(PipelineRunner<?>, PipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.Pipeline
-
- Pipeline.PipelineExecutionException - Exception in com.google.cloud.dataflow.sdk
-
Thrown during execution of a
Pipeline, whenever user code within that
Pipeline throws an exception.
- Pipeline.PipelineVisitor - Interface in com.google.cloud.dataflow.sdk
-
- PipelineExecutionException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.Pipeline.PipelineExecutionException
-
- PipelineOptions - Interface in com.google.cloud.dataflow.sdk.options
-
PipelineOptions are used to configure Pipelines.
- PipelineOptions.CheckEnabled - Enum in com.google.cloud.dataflow.sdk.options
-
Enumeration of the possible states for a given check.
- PipelineOptionsFactory - Class in com.google.cloud.dataflow.sdk.options
-
- PipelineOptionsFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
- PipelineOptionsFactory.Builder - Class in com.google.cloud.dataflow.sdk.options
-
- PipelineOptionsRegistrar - Interface in com.google.cloud.dataflow.sdk.options
-
- PipelineOptionsValidator - Class in com.google.cloud.dataflow.sdk.options
-
- PipelineOptionsValidator() - Constructor for class com.google.cloud.dataflow.sdk.options.PipelineOptionsValidator
-
- PipelineResult - Interface in com.google.cloud.dataflow.sdk
-
- PipelineResult.State - Enum in com.google.cloud.dataflow.sdk
-
Named constants for common values for the job state.
- PipelineRunner<ResultT extends PipelineResult> - Class in com.google.cloud.dataflow.sdk.runners
-
- PipelineRunner() - Constructor for class com.google.cloud.dataflow.sdk.runners.PipelineRunner
-
- PipelineRunnerRegistrar - Interface in com.google.cloud.dataflow.sdk.runners
-
- plusDelayOf(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
Adds some delay to the original target time.
- popNode() - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
-
Removes the last TransformTreeNode from the stack.
- populateDisplayData(DisplayData.Builder) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.HasDisplayData
-
Register display metadata for the given transform or component.
- populateDisplayData(DisplayData.Builder) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
-
Register display metadata for the given transform or component.
- populateDisplayData(DisplayData.Builder) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
-
Register display metadata for the given transform or component.
- populateDisplayData(DisplayData.Builder) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
-
Register display metadata for the given transform or component.
- POutput - Interface in com.google.cloud.dataflow.sdk.values
-
The interface for things that might be output from a
PTransform.
- POutputValueBase - Class in com.google.cloud.dataflow.sdk.values
-
- POutputValueBase(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.POutputValueBase
-
- POutputValueBase() - Constructor for class com.google.cloud.dataflow.sdk.values.POutputValueBase
-
No-arg constructor for Java serialization only.
- prefetchOnElement(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- prefetchOnElement(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
- prefetchOnFire(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
- prefetchOnMerge(MergingStateAccessor<?, W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- prefetchOnMerge(MergingStateAccessor<?, W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
-
- prefetchOnMerge(MergingStateAccessor<?, W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
- prefetchShouldFire(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- prefetchShouldFire(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
-
- prefetchShouldFire(StateAccessor<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
- prepareWrite(WritableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
-
Called with the channel that a subclass will write its header, footer, and values to.
- prepareWrite(WritableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
-
Creates the output stream that elements will be written to.
- printHelp(PrintStream) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
Outputs the set of registered options with the PipelineOptionsFactory
with a description for each one if available to the output stream.
- printHelp(PrintStream, Class<? extends PipelineOptions>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
Outputs the set of options available to be set for the passed in
PipelineOptions
interface.
- processBatch(Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
- processBatch(InputT...) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
A convenience method for testing
DoFns with bundles of elements.
- ProcessContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
-
- ProcessContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
-
- processElement(DoFn<KV<K, Iterable<InputT>>, KV<K, OutputT>>.ProcessContext) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
-
- processElement(DoFn<InputT, OutputT>.ProcessContext) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
-
Processes one input element.
- processElement(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
- processElement(DoFn<InputT, OutputT>.ProcessContext) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
-
- producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
- producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
-
Whether this source is known to produce key/value pairs sorted by lexicographic order on
the bytes of the encoded key.
- producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
-
Returns whether the delegate source produces sorted keys.
- producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- producesSortedKeys(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
- PROJECT_ID_REGEXP - Static variable in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
-
Project IDs must contain lowercase letters, digits, or dashes.
- Proto2Coder<T extends com.google.protobuf.Message> - Class in com.google.cloud.dataflow.sdk.coders
-
- ProtoCoder<T extends com.google.protobuf.Message> - Class in com.google.cloud.dataflow.sdk.coders.protobuf
-
A
Coder using Google Protocol Buffers binary format.
- PROVIDER - Static variable in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
- PROVIDER - Static variable in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
-
- PTransform<InputT extends PInput,OutputT extends POutput> - Class in com.google.cloud.dataflow.sdk.transforms
-
A
PTransform<InputT, OutputT> is an operation that takes an
InputT (some subtype of
PInput) and produces an
OutputT (some subtype of
POutput).
- PTransform() - Constructor for class com.google.cloud.dataflow.sdk.transforms.PTransform
-
- PTransform(String) - Constructor for class com.google.cloud.dataflow.sdk.transforms.PTransform
-
- PubsubIO - Class in com.google.cloud.dataflow.sdk.io
-
Read and Write
PTransforms for Cloud Pub/Sub streams.
- PubsubIO.PubsubSubscription - Class in com.google.cloud.dataflow.sdk.io
-
Class representing a Cloud Pub/Sub Subscription.
- PubsubIO.PubsubTopic - Class in com.google.cloud.dataflow.sdk.io
-
Class representing a Cloud Pub/Sub Topic.
- PubsubIO.Read - Class in com.google.cloud.dataflow.sdk.io
-
- PubsubIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
-
A
PTransform that reads from a Cloud Pub/Sub source and returns
a unbounded
PCollection containing the items from the stream.
- PubsubIO.Write - Class in com.google.cloud.dataflow.sdk.io
-
- PubsubIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
-
- pushNode(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
-
Add a TransformTreeNode to the stack.
- PValue - Interface in com.google.cloud.dataflow.sdk.values
-
The interface for values that can be input to and output from
PTransforms.
- PValueBase - Class in com.google.cloud.dataflow.sdk.values
-
A
PValueBase is an abstract base class that provides
sensible default implementations for methods of
PValue.
- PValueBase(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.PValueBase
-
- PValueBase() - Constructor for class com.google.cloud.dataflow.sdk.values.PValueBase
-
No-arg constructor for Java serialization only.
- Sample - Class in com.google.cloud.dataflow.sdk.transforms
-
PTransforms for taking samples of the elements in a
PCollection, or samples of the values associated with each
key in a PCollection of KVs.
- Sample() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sample
-
- Sample.FixedSizedSampleFn<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
CombineFn that computes a fixed-size sample of a
collection of values.
- Sample.SampleAny<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
A
PTransform that takes a
PCollection<T> and a limit, and
produces a new
PCollection<T> containing up to limit
elements of the input
PCollection.
- satisfies(SerializableFunction<Iterable<T>, Void>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
-
- satisfies(DataflowAssert.AssertRelation<Iterable<T>, Iterable<T>>, Iterable<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
-
- satisfies(SerializableFunction<T, Void>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
-
- satisfies(DataflowAssert.AssertRelation<T, T>, T) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
-
Applies an AssertRelation to check the provided relation against the
value of this assert and the provided expected value.
- SerializableCoder<T extends Serializable> - Class in com.google.cloud.dataflow.sdk.coders
-
- SerializableCoder(Class<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.SerializableCoder
-
- SerializableComparator<T> - Interface in com.google.cloud.dataflow.sdk.transforms
-
A Comparator that is also Serializable.
- SerializableFunction<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms
-
A function that computes an output value of type
OutputT from an input value of type
InputT and is
Serializable.
- Sessions - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
- set(long...) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
-
- SET_PROJECT_FROM_OPTIONS_WARNING - Static variable in class com.google.cloud.dataflow.sdk.io.BigQueryIO
-
- setApiRootUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setAppName(String) - Method in interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions
-
- setAuthorizationServerEncodedUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setAutoscalingAlgorithm(DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- SetCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
-
- SetCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.SetCoder
-
- setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert
-
Sets the coder to use for elements of type T, as needed for internal purposes.
- setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
-
Sets the coder to use for elements of type T, as needed
for internal purposes.
- setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
Sets the
Coder used by this
PCollection to encode and decode the
values stored in it.
- setCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
-
Sets the
Coder used by this
TypedPValue to encode and decode the
values stored in it.
- setCoderRegistry(CoderRegistry) - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- setCredentialDir(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setCredentialFactoryClass(Class<? extends CredentialFactory>) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setCredentialId(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setDataflowClient(Dataflow) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setDataflowEndpoint(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setDataflowJobFile(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setDebuggee(Debuggee) - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
-
- setDefaultWorkerLogLevel(DataflowWorkerLoggingOptions.Level) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
-
- setDirectPipelineRunnerRandomSeed(Long) - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
-
- setDiskSizeGb(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setDumpHeapOnOOM(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setEnableCloudDebugger(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
-
- setEnableProfilingAgent(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions
-
- setExecutorService(ExecutorService) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
-
- setExperiments(List<String>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setFallbackCoderProvider(CoderProvider) - Method in class com.google.cloud.dataflow.sdk.coders.CoderRegistry
-
- setFilesToStage(List<String>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setFinished(boolean) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
-
Sets the finished bit for the current trigger.
- setFinished(boolean, int) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
-
Sets the finished bit for the given sub-trigger.
- setGcpCredential(Credential) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setGcsEndpoint(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
-
- setGcsUploadBufferSizeBytes(Integer) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
-
- setGcsUtil(GcsUtil) - Method in interface com.google.cloud.dataflow.sdk.options.GcsOptions
-
- setGoogleApiTrace(GoogleApiDebugOptions.GoogleApiTracer) - Method in interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions
-
- setHooks(DataflowPipelineRunnerHooks) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
-
Sets callbacks to invoke during execution.
- setHooks(DataflowPipelineRunnerHooks) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
-
Sets callbacks to invoke during execution see DataflowPipelineRunnerHooks.
- setIsBoundedInternal(PCollection.IsBounded) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
- setJobId(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
-
- setJobMessageOutput(PrintStream) - Method in interface com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions
-
- setJobName(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
-
- setMaxConditionCost(double) - Method in interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions
-
- setMaxNumWorkers(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setName(String) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
- setName(String) - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
-
- setNetwork(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setNumberOfWorkerHarnessThreads(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setNumWorkers(int) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setOutput(TransformTreeNode, POutput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
-
Sets the output of a transform node.
- setOutput(POutput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Adds an output to the transform node.
- setOverrideWindmillBinary(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setPathValidator(PathValidator) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setPathValidatorClass(Class<? extends PathValidator>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setPCollection(PCollection<T>, List<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
Shorthand for setting the value of a PCollection where the elements do not have
timestamps or windows.
- setPCollectionValuesWithMetadata(PCollection<T>, List<DirectPipelineRunner.ValueWithMetadata<T>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
Sets the value of the given PCollection, where each element also has a timestamp
and collection of windows.
- setPCollectionView(PCollectionView<T>, Iterable<WindowedValue<ElemT>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
- setPCollectionWindowedValue(PCollection<T>, List<WindowedValue<T>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationContext
-
Sets the value of the given PCollection, where each element also has a timestamp
and collection of windows.
- setProfilingAgentConfiguration(DataflowProfilingOptions.DataflowProfilingAgentConfiguration) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions
-
- setProject(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
-
- setProject(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setPubsubRootUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setRunner(Class<? extends PipelineRunner<?>>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
-
- setSecretsFile(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setServiceAccountKeyfile(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setServiceAccountName(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setSideInput(PCollectionView<?>, Iterable<WindowedValue<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
- setSideInputInGlobalWindow(PCollectionView<?>, Iterable<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
- setSideInputs(Map<PCollectionView<?>, Iterable<WindowedValue<?>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Registers the tuple of values of the side input
PCollectionViews to
pass to the
DoFn under test.
- setSideOutputTags(TupleTagList) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Registers the list of TupleTags that can be used by the
DoFn under test to output to side output
PCollections.
- setStableUniqueNames(PipelineOptions.CheckEnabled) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
-
- setStager(Stager) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setStagerClass(Class<? extends Stager>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setStagingLocation(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
-
- setStreaming(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.StreamingOptions
-
- setSubnetwork(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setTeardownPolicy(DataflowPipelineWorkerPoolOptions.TeardownPolicy) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setTempDatasetId(String) - Method in interface com.google.cloud.dataflow.sdk.options.BigQueryOptions
-
- setTempLocation(String) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions
-
- setTestEncodability(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
-
- setTestSerializability(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
-
- setTestUnorderedness(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DirectPipelineOptions
-
- setTimer(Instant, TimeDomain) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnElementContext
-
Sets a timer to fire when the watermark or processing time is beyond the given timestamp.
- setTimer(Instant, TimeDomain) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
-
Sets a timer to fire when the watermark or processing time is beyond the given timestamp.
- setTokenServerUrl(String) - Method in interface com.google.cloud.dataflow.sdk.options.GcpOptions
-
- setTransformNameMapping(Map<String, String>) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
- setTypeDescriptorInternal(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
- setTypeDescriptorInternal(TypeDescriptor<T>) - Method in class com.google.cloud.dataflow.sdk.values.TypedPValue
-
- setUpdate(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions
-
Deprecated.
- setUpdate(boolean) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions
-
- setupDelegateAggregators() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
-
Sets up
Aggregators created by the
DoFn so they are
usable within this context.
- setWindowingStrategyInternal(WindowingStrategy<?, ?>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection
-
- setWorkerCacheMb(Integer) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
-
- setWorkerDiskType(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setWorkerHarnessContainerImage(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setWorkerId(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerHarnessOptions
-
- setWorkerLogLevelOverrides(DataflowWorkerLoggingOptions.WorkerLogLevelOverrides) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions
-
- setWorkerMachineType(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- setZone(String) - Method in interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions
-
- ShardNameTemplate - Class in com.google.cloud.dataflow.sdk.io
-
Standard shard naming templates.
- ShardNameTemplate() - Constructor for class com.google.cloud.dataflow.sdk.io.ShardNameTemplate
-
- shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterAll
-
Returns true if the current state of the trigger indicates that its condition
is satisfied and it is ready to fire.
- shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
- shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterEach
-
- shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterFirst
-
- shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
-
- shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
-
- shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger
-
- shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Repeatedly
-
- shouldFire(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
Returns true if the current state of the trigger indicates that its condition
is satisfied and it is ready to fire.
- sideInput(PCollectionView<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.Context
-
Returns the value of the side input for the window corresponding to the
window of the main input element.
- sideInput(PCollectionView<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
-
Returns the value of the side input for the window corresponding to the
window of the main input element.
- sideInput(PCollectionView<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
-
Returns the value of the side input.
- sideOutput(TupleTag<T>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
-
Adds the given element to the side output PCollection with the
given tag.
- sideOutput(TupleTag<T>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
-
Adds the given element to the side output PCollection with the
given tag.
- sideOutputWithTimestamp(TupleTag<T>, T, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context
-
Adds the given element to the specified side output PCollection,
with the given timestamp.
- sideOutputWithTimestamp(TupleTag<T>, T, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context
-
Adds the given element to the specified side output PCollection,
with the given timestamp.
- SimpleCombineFn(SerializableFunction<Iterable<V>, V>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.SimpleCombineFn
-
Deprecated.
- SimpleFunction<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms
-
- SimpleFunction() - Constructor for class com.google.cloud.dataflow.sdk.transforms.SimpleFunction
-
- SingletonAssert(DataflowAssert.CreateActual<?, T>, Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.testing.DataflowAssert.SingletonAssert
-
- sink() - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
-
- Sink(String, String) - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
-
Constructs a Sink with given host and dataset.
- sink - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
The Sink that this WriteOperation will write to.
- Sink<T> - Class in com.google.cloud.dataflow.sdk.io
-
A
Sink represents a resource that can be written to using the
Write transform.
- Sink() - Constructor for class com.google.cloud.dataflow.sdk.io.Sink
-
- Sink.WriteOperation<T,WriteT> - Class in com.google.cloud.dataflow.sdk.io
-
- Sink.Writer<T,WriteT> - Class in com.google.cloud.dataflow.sdk.io
-
A Writer writes a bundle of elements from a PCollection to a sink.
- size() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
-
Returns the number of columns for this schema.
- size() - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList
-
- size() - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList
-
Returns the number of TupleTags in this TupleTagList.
- SlidingWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A
WindowFn that windows values into possibly overlapping fixed-size
timestamp-based windows.
- smallest(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
-
Returns a PTransform that takes an input
PCollection<T> and returns a PCollection<List<T>> with a
single element containing the smallest count elements of the input
PCollection<T>, in increasing order, sorted according to
their natural order.
- Smallest() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Top.Smallest
-
- smallestPerKey(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.Top
-
Returns a PTransform that takes an input
PCollection<KV<K, V>> and returns a
PCollection<KV<K, List<V>>> that contains an output
element mapping each distinct key in the input
PCollection to the smallest count values
associated with that key in the input
PCollection<KV<K, V>>, in increasing order, sorted
according to their natural order.
- SortValuesByTimestamp() - Constructor for class com.google.cloud.dataflow.sdk.transforms.GroupByKey.SortValuesByTimestamp
-
- source() - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
-
- Source<T> - Class in com.google.cloud.dataflow.sdk.io
-
Base class for defining input formats and creating a Source for reading the input.
- Source() - Constructor for class com.google.cloud.dataflow.sdk.io.Source
-
- Source.Reader<T> - Class in com.google.cloud.dataflow.sdk.io
-
The interface that readers of custom input sources must implement.
- SourceTestUtils - Class in com.google.cloud.dataflow.sdk.testing
-
Helper functions and test harnesses for checking correctness of
Source
implementations.
- SourceTestUtils() - Constructor for class com.google.cloud.dataflow.sdk.testing.SourceTestUtils
-
- SourceTestUtils.ExpectedSplitOutcome - Enum in com.google.cloud.dataflow.sdk.testing
-
- span(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
Returns the minimal window that includes both this window and
the given window.
- split(int) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
-
Returns a list of up to
numSplits + 1 ByteKeys in ascending order,
where the keys have been interpolated to form roughly equal sub-ranges of this
ByteKeyRange, assuming a uniform distribution of keys within this range.
- splitAtFraction(double) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
-
Tells the reader to narrow the range of the input it's going to read and give up
the remainder, so that the new range would contain approximately the given
fraction of the amount of data in the current range.
- splitAtFraction(double) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
-
- splitAtFraction(double) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
-
- splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource
-
Splits the source into bundles of approximately desiredBundleSizeBytes.
- splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
- splitIntoBundles(long, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
- StagerFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.StagerFactory
-
- StandardCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
-
- StandardCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- StandardOutputFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions.StandardOutputFactory
-
- start() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader
-
- start() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
-
- start() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader
-
Initializes the reader and advances the reader to the first record.
- start() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader
-
Initializes the reader and advances the reader to the first record.
- start() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
Returns the start of this window, inclusive.
- startBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn
-
Prepares this DoFn instance for processing a batch of elements.
- startBundle() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
- startBundle(DoFn<InputT, OutputT>.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.MultiThreadedIntraBundleProcessingDoFn
-
- startImpl() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
-
- startImpl() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader
-
- startReading(ReadableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader
-
Starts reading from the provided channel.
- startReading(ReadableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader
-
Creates a decompressing channel from the input channel and passes it to its delegate reader's
FileBasedReader#startReading(ReadableByteChannel).
- startReading(ReadableByteChannel) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader
-
Performs any initialization of the subclass of FileBasedReader that involves IO
operations.
- state() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
-
- state() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
-
Returns the interface for accessing persistent state.
- STREAMING_WORKER_HARNESS_CONTAINER_IMAGE - Static variable in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
-
- StreamingOptions - Interface in com.google.cloud.dataflow.sdk.options
-
Options used to configure streaming.
- StreamingPubsubIOWrite(DataflowPipelineRunner, PubsubIO.Write.Bound<T>) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.StreamingPubsubIOWrite
-
Builds an instance of this class from the overridden transform.
- StringDelegateCoder<T> - Class in com.google.cloud.dataflow.sdk.coders
-
A
Coder that wraps a
Coder<String>
and encodes/decodes values via string representations.
- StringDelegateCoder(Class<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
-
- StringUtf8Coder - Class in com.google.cloud.dataflow.sdk.coders
-
- StructuralByteArray - Class in com.google.cloud.dataflow.sdk.coders
-
A wrapper around a byte[] that uses structural, value-based
equality rather than byte[]'s normal object identity.
- StructuralByteArray(byte[]) - Constructor for class com.google.cloud.dataflow.sdk.coders.StructuralByteArray
-
- structuralValue(byte[]) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder
-
Returns an object with an Object.equals() method that represents structural equality
on the argument.
- structuralValue(T) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
-
Returns an object with an Object.equals() method that represents structural equality
on the argument.
- structuralValue(T) - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
-
Returns an object with an Object.equals() method that represents structural equality
on the argument.
- structuralValue(KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
- structuralValue(T) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
-
- structuralValue(T) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- structuralValueConsistentWithEquals(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- structuralValueConsistentWithEqualsInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties
-
- subscription(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
Returns a transform that's like this one but reading from the
given subscription.
- subscription(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
-
Creates and returns a transform for reading from a specific Cloud Pub/Sub subscription.
- subTrigger(int) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
-
Access the executable version of the specified sub-trigger.
- subTriggers - Variable in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
- subTriggers() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
- subTriggers() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo
-
Access the executable versions of the sub-triggers of the current trigger.
- Sum - Class in com.google.cloud.dataflow.sdk.transforms
-
PTransforms for computing the sum of the elements in a
PCollection, or the sum of the values associated with
each key in a PCollection of KVs.
- Sum.SumDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms
-
- Sum.SumIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms
-
- Sum.SumLongFn - Class in com.google.cloud.dataflow.sdk.transforms
-
- SumDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn
-
- SumIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn
-
- SumLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn
-
- supportsDynamicWorkRebalancing() - Method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
-
Returns whether a
TestPipeline supports dynamic work rebalancing, and thus tests
of dynamic work rebalancing are expected to pass.
- TableRowJsonCoder - Class in com.google.cloud.dataflow.sdk.coders
-
A
Coder that encodes BigQuery
TableRow objects in their native JSON format.
- TaggedKeyedPCollection(TupleTag<V>, PCollection<KV<K, V>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple.TaggedKeyedPCollection
-
- takeOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Returns the elements output so far to the main output.
- takeOutputElementsWithTimestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Returns the elements output so far to the main output with associated timestamps.
- takeSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester
-
Returns the elements output so far to the side output with the given tag.
- TEMPORARY_FILENAME_SEPARATOR - Static variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Name separator for temporary files.
- temporaryFileRetention - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation
-
Option to keep or remove temporary output files.
- TestCombineDoFn(PerKeyCombineFnRunner<? super K, ? super InputT, AccumT, OutputT>, Coder<AccumT>, boolean, Random) - Constructor for class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn
-
- TestDataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.testing
-
- TestDataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.testing
-
- testingPipelineOptions() - Static method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
-
- TestPipeline - Class in com.google.cloud.dataflow.sdk.testing
-
A creator of test pipelines that can be used inside of tests that can be
configured to run locally or against the live service.
- TextIO - Class in com.google.cloud.dataflow.sdk.io
-
- TextIO.CompressionType - Enum in com.google.cloud.dataflow.sdk.io
-
Possible text file compression types.
- TextIO.Read - Class in com.google.cloud.dataflow.sdk.io
-
A
PTransform that reads from a text file (or multiple text
files matching a pattern) and returns a
PCollection containing
the decoding of each of the lines of the text file(s).
- TextIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
-
A
PTransform that reads from one or more text files and returns a bounded
PCollection containing one element for each line of the input files.
- TextIO.Write - Class in com.google.cloud.dataflow.sdk.io
-
A
PTransform that writes a
PCollection to text file (or
multiple text files matching a sharding pattern), with each
element of the input collection encoded into its own line.
- TextIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
-
A PTransform that writes a bounded PCollection to a text file (or
multiple text files matching a sharding pattern), with each
PCollection element being encoded into its own line.
- TextualIntegerCoder - Class in com.google.cloud.dataflow.sdk.coders
-
A
Coder that encodes
Integer Integers as the ASCII bytes of
their textual, decimal, representation.
- TextualIntegerCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder
-
- that(PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
-
- thatIterable(PCollectionView<Iterable<T>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
-
- thatMap(PCollection<KV<K, V>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
-
- thatMultimap(PCollection<KV<K, V>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
-
- thatSingleton(PCollection<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
-
- thatSingletonIterable(PCollection<? extends Iterable<T>>) - Static method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert
-
- timestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
-
Returns the timestamp of the input element.
- timestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ProcessContext
-
Returns the timestamp of the input element.
- timestamp() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
-
Returns the timestamp of the current element.
- TIMESTAMP_MAX_VALUE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
-
- TIMESTAMP_MIN_VALUE - Static variable in class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow
-
- timestamped(Iterable<TimestampedValue<T>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
-
- timestamped(TimestampedValue<T>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
-
- timestamped(Iterable<T>, Iterable<Long>) - Static method in class com.google.cloud.dataflow.sdk.transforms.Create
-
Returns a new root transform that produces a
PCollection containing
the specified elements with the specified timestamps.
- TimestampedValue<V> - Class in com.google.cloud.dataflow.sdk.values
-
An immutable pair of a value and a timestamp.
- TimestampedValue(V, Instant) - Constructor for class com.google.cloud.dataflow.sdk.values.TimestampedValue
-
- TimestampedValue.TimestampedValueCoder<T> - Class in com.google.cloud.dataflow.sdk.values
-
- timestampLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
Returns a transform that's like this one but that reads message timestamps
from the given message attribute.
- timestampLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
-
Creates and returns a transform reading from Cloud Pub/Sub where record timestamps are
expected to be provided as Pub/Sub message attributes.
- timestampLabel(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
Returns a new transform that's like this one but that publishes record timestamps
to a message attribute with the specified name.
- timestampLabel(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
-
Creates a transform that writes to Pub/Sub, adds each record's timestamp to the published
messages in an attribute with the specified name.
- timestampMappers - Variable in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement
-
A list of timestampMappers m1, m2, m3, ...
- to(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns a new
PTransform that's like this one but
that writes to the file(s) with the given filename prefix.
- to(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
-
Returns a
PTransform that writes to the file(s)
with the given prefix.
- to(String) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns a copy of this write transformation, but writing to the specified table.
- to(TableReference) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns a copy of this write transformation, but writing to the specified table.
- to(SerializableFunction<BoundedWindow, String>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns a copy of this write transformation, but using the specified function to determine
which table to write to for each window.
- to(String) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
-
Creates a write transformation for the given table specification.
- to(TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
-
Creates a write transformation for the given table.
- to(SerializableFunction<BoundedWindow, String>) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
-
Creates a write transformation from a function that maps windows to table specifications.
- to(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
Returns a transform for writing to text files that's like this one but
that writes to the file(s) with the given filename prefix.
- to(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
-
Returns a transform for writing to text files that writes to the file(s)
with the given prefix.
- to(Sink<T>) - Static method in class com.google.cloud.dataflow.sdk.io.Write
-
Creates a Write transform that writes to the given Sink.
- toDoFn(DoFnWithContext<InputT, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnReflector
-
- toFilenamePrefix(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
-
Returns an XmlSink that writes to files with the given prefix.
- Top - Class in com.google.cloud.dataflow.sdk.transforms
-
PTransforms for finding the largest (or smallest) set
of elements in a PCollection, or the largest (or smallest)
set of values associated with each key in a PCollection of
KVs.
- Top.Largest<T extends Comparable<? super T>> - Class in com.google.cloud.dataflow.sdk.transforms
-
A Serializable Comparator that that uses the compared elements' natural
ordering.
- Top.Smallest<T extends Comparable<? super T>> - Class in com.google.cloud.dataflow.sdk.transforms
-
Serializable Comparator that that uses the reverse of the compared elements'
natural ordering.
- Top.TopCombineFn<T,ComparatorT extends Comparator<T> & Serializable> - Class in com.google.cloud.dataflow.sdk.transforms
-
CombineFn for Top transforms that combines a
bunch of Ts into a single count-long
List<T>, using compareFn to choose the largest
Ts.
- TopCombineFn(int, ComparatorT) - Constructor for class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn
-
- topic(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
Returns a transform that's like this one but that reads from the specified topic.
- topic(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
-
Creates and returns a transform for reading from a Cloud Pub/Sub topic.
- topic(String) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
Returns a new transform that's like this one but that writes to the specified
topic.
- topic(String) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
-
Creates a transform that publishes to the specified topic.
- toString() - Method in class com.google.cloud.dataflow.sdk.coders.Coder.Context
-
- toString() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
-
- toString() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- toString() - Method in class com.google.cloud.dataflow.sdk.coders.StringDelegateCoder
-
- toString() - Method in class com.google.cloud.dataflow.sdk.coders.StructuralByteArray
-
- toString() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
-
- toString() - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
-
- toString() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- toString() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
- toString() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
- toString() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey
-
- toString() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
-
- toString() - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
-
- toString() - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
-
- toString() - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- toString() - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner
-
- toString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline
-
- toString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner
-
- toString() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
-
- toString() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline
-
- toString() - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
-
- toString() - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner
-
- toString() - Method in class com.google.cloud.dataflow.sdk.testing.TestPipeline
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.AppliedPTransform
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Identifier
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Item
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.join.RawUnionValue
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterProcessingTime
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo
-
- toString() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
- toString() - Method in class com.google.cloud.dataflow.sdk.values.KV
-
- toString() - Method in class com.google.cloud.dataflow.sdk.values.PValueBase
-
- toString() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue
-
- toString() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag
-
- toString() - Method in class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
- toTableReference(SerializableFunction<BoundedWindow, TableReference>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns a copy of this write transformation, but using the specified function to determine
which table to write to for each window.
- toTableReference(SerializableFunction<BoundedWindow, TableReference>) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
-
Creates a write transformation from a function that maps windows to
TableReference
objects.
- toTableSpec(TableReference) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO
-
- TransformHierarchy - Class in com.google.cloud.dataflow.sdk.runners
-
Captures information about a collection of transformations and their
associated
PValues.
- TransformHierarchy() - Constructor for class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
-
Create a TransformHierarchy containing a root node.
- transforms - Variable in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
-
- TransformTreeNode - Class in com.google.cloud.dataflow.sdk.runners
-
Provides internal tracking of transform relationships with helper methods
for initialization and ordered visitation.
- TransformTreeNode(TransformTreeNode, PTransform<?, ?>, String, PInput) - Constructor for class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Creates a new TransformTreeNode with the given parent and transform.
- translate(TransformT, DataflowPipelineTranslator.TranslationContext) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TransformTranslator
-
- translate(Pipeline, DataflowPipelineRunner, List<DataflowPackage>) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator
-
Translates a
Pipeline into a
JobSpecification.
- traverseTopologically(Pipeline.PipelineVisitor) - Method in class com.google.cloud.dataflow.sdk.Pipeline
-
- Trigger<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
Triggers control when the elements for a specific key and window are output.
- Trigger(List<Trigger<W>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger
-
- trigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.OnMergeContext
-
- trigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
-
Returns the interface for accessing trigger info.
- Trigger.MergingTriggerInfo<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
-
Interact with properties of the trigger being executed, with extensions to deal with the
merging windows.
- Trigger.OnceTrigger<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
Triggers that are guaranteed to fire at most once should extend from this, rather
than the general
Trigger class to indicate that behavior.
- Trigger.OnElementContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
- Trigger.OnMergeContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
- Trigger.TriggerContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
Information accessible to all operational hooks in this Trigger.
- Trigger.TriggerInfo<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
-
Interface for accessing information about the trigger being executed and other triggers in the
same tree.
- TriggerBuilder<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing
-
Anything that can be used to create an instance of a Trigger implements this interface.
- TriggerContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
-
- triggering(TriggerBuilder<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
Sets a non-default trigger for this Window PTransform.
- triggering(TriggerBuilder<?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
-
Sets a non-default trigger for this Window PTransform.
- triggering(TriggerBuilder<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
-
Sets a non-default trigger for this Window PTransform.
- tryReturnRecordAt(boolean, ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
-
- tryReturnRecordAt(boolean, Long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
-
- tryReturnRecordAt(boolean, long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
-
- tryReturnRecordAt(boolean, PositionT) - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
-
Atomically determines whether a record at the given position can be returned and updates
internal state.
- trySplitAtPosition(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRangeTracker
-
- trySplitAtPosition(Long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
-
- trySplitAtPosition(long) - Method in class com.google.cloud.dataflow.sdk.io.range.OffsetRangeTracker
-
- trySplitAtPosition(PositionT) - Method in interface com.google.cloud.dataflow.sdk.io.range.RangeTracker
-
- TupleTag<V> - Class in com.google.cloud.dataflow.sdk.values
-
- TupleTag() - Constructor for class com.google.cloud.dataflow.sdk.values.TupleTag
-
Constructs a new TupleTag, with a fresh unique id.
- TupleTag(String) - Constructor for class com.google.cloud.dataflow.sdk.values.TupleTag
-
Constructs a new TupleTag with the given id.
- TupleTagList - Class in com.google.cloud.dataflow.sdk.values
-
- TypeDescriptor<T> - Class in com.google.cloud.dataflow.sdk.values
-
A description of a Java type, including actual generic parameters where possible.
- TypeDescriptor() - Constructor for class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
- TypeDescriptor(Object) - Constructor for class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Creates a
TypeDescriptor representing the type parameter
T, which should
resolve to a concrete type in the context of the class
clazz.
- TypeDescriptor(Class<?>) - Constructor for class com.google.cloud.dataflow.sdk.values.TypeDescriptor
-
Creates a
TypeDescriptor representing the type parameter
T, which should resolve to a concrete type in the context
of the class
clazz.
- TypedPValue<T> - Class in com.google.cloud.dataflow.sdk.values
-
A
TypedPValue<T> is the abstract base class of things that
store some number of values of type
T.
- TypedPValue(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.values.TypedPValue
-
- validate() - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
- validate(PInput) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
- validate(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
-
- validate(PCollection<KV<ByteString, Iterable<Mutation>>>) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
-
- validate() - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
-
Validates that the delegate source is a valid source and that the channel factory is not null.
- validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
-
Ensures the host and dataset are set.
- validate() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink
-
Perform pipeline-construction-time validation.
- validate() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource
-
- validate() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource
-
- validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink
-
Ensures that the sink is valid and can be written to before the write operation begins.
- validate() - Method in class com.google.cloud.dataflow.sdk.io.Source
-
Checks that this source is valid, before it can be used in a pipeline.
- validate(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
-
Validates that the root element, class to bind to a JAXB context, and filenamePrefix have
been set and that the class can be bound in a JAXB context.
- validate() - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
- validate(Class<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsValidator
-
Validates that the passed
PipelineOptions conforms to all the validation criteria from
the passed in interface.
- validate(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey
-
- validate(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform
-
Called before invoking apply (which may be intercepted by the runner) to
verify this transform is fully specified and applicable to the specified
input.
- validate(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsIterable
-
- validate(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsList
-
- validate(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap
-
- validate(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMultimap
-
- validate(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
-
- validate(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
- validateGetOutputTimestamp(WindowFn<T, W>, long) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
-
Assigns the given
timestamp to windows using the specified
windowFn, and
verifies that result of
windowFn.getOutputTime for later windows
(as defined by
maxTimestamp won't prevent the watermark from passing the end of earlier
windows.
- validateGetOutputTimestamps(WindowFn<T, W>, OutputTimeFn<? super W>, List<List<Long>>) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
-
Verifies that later-ending merged windows from any of the timestamps hold up output of
earlier-ending windows, using the provided
WindowFn and
OutputTimeFn.
- validateNonInterferingOutputTimes(WindowFn<T, W>, long) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
-
Assigns the given timestamp to windows using the specified windowFn, and
verifies that result of windowFn.getOutputTimestamp for each window is within the
proper bound.
- Validation - Annotation Type in com.google.cloud.dataflow.sdk.options
-
- Validation.Required - Annotation Type in com.google.cloud.dataflow.sdk.options
-
This criteria specifies that the value must be not null.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.annotations.Experimental.Kind
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException.ReasonCode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.CreateDisposition
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.WriteDisposition
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSource.Mode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.io.TextIO.CompressionType
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.TeardownPolicy
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.Level
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.options.PipelineOptions.CheckEnabled
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.testing.SourceTestUtils.ExpectedSplitOutcome
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.Timing
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Window.ClosingBehavior
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded
-
Returns the enum constant of this type with the specified name.
- values() - Static method in enum com.google.cloud.dataflow.sdk.annotations.Experimental.Kind
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException.ReasonCode
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.CreateDisposition
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.WriteDisposition
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.io.FileBasedSource.Mode
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.io.TextIO.CompressionType
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.TeardownPolicy
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.Level
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.options.PipelineOptions.CheckEnabled
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.PipelineResult.State
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values - Variable in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
-
- values() - Static method in enum com.google.cloud.dataflow.sdk.testing.SourceTestUtils.ExpectedSplitOutcome
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- Values<V> - Class in com.google.cloud.dataflow.sdk.transforms
-
Values<V> takes a PCollection of KV<K, V>s and
returns a PCollection<V> of the values.
- values() - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo.Timing
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.transforms.windowing.Window.ClosingBehavior
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- VarIntCoder - Class in com.google.cloud.dataflow.sdk.coders
-
- VarLongCoder - Class in com.google.cloud.dataflow.sdk.coders
-
A
Coder that encodes
Longs using between 1 and 10 bytes.
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder
-
- verifyDeterministic() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.DeterministicStandardCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.ListCoder
-
List sizes are always known, so ListIterable may be deterministic while
the general IterableLikeCoder is not.
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder
-
NullableCoder is deterministic if the nested Coder is.
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.SetCoder
-
- verifyDeterministic(String, Iterable<Coder<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- verifyDeterministic(String, Coder<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder
-
- verifyDeterministic() - Method in class com.google.cloud.dataflow.sdk.values.TimestampedValue.TimestampedValueCoder
-
- via(SerializableFunction<InputT, ? extends Iterable<OutputT>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements
-
For a
SerializableFunction<InputT, ? extends Iterable<OutputT>> fn,
returns a
PTransform that applies
fn to every element of the input
PCollection<InputT> and outputs all of the elements to the output
PCollection<OutputT>.
- via(SimpleFunction<InputT, ? extends Iterable<OutputT>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements
-
For a
SimpleFunction<InputT, ? extends Iterable<OutputT>> fn,
return a
PTransform that applies
fn to every element of the input
PCollection<InputT> and outputs all of the elements to the output
PCollection<OutputT>.
- via(SerializableFunction<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.MapElements
-
For a SerializableFunction<InputT, OutputT> fn and output type descriptor,
returns a PTransform that takes an input PCollection<InputT> and returns
a PCollection<OutputT> containing fn.apply(v) for every element v in
the input.
- via(SimpleFunction<InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.MapElements
-
For a SimpleFunction<InputT, OutputT> fn, returns a PTransform that
takes an input PCollection<InputT> and returns a PCollection<OutputT>
containing fn.apply(v) for every element v in the input.
- View - Class in com.google.cloud.dataflow.sdk.transforms
-
- View.AsIterable<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
Not intended for direct use by pipeline authors; public only so a
PipelineRunner may
override its behavior.
- View.AsList<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
Not intended for direct use by pipeline authors; public only so a
PipelineRunner may
override its behavior.
- View.AsMap<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
-
Not intended for direct use by pipeline authors; public only so a
PipelineRunner may
override its behavior.
- View.AsMultimap<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
-
Not intended for direct use by pipeline authors; public only so a
PipelineRunner may
override its behavior.
- View.AsSingleton<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
Not intended for direct use by pipeline authors; public only so a
PipelineRunner may
override its behavior.
- View.CreatePCollectionView<ElemT,ViewT> - Class in com.google.cloud.dataflow.sdk.transforms
-
- visit(Pipeline.PipelineVisitor, Set<PValue>) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy
-
Visits all nodes in the transform hierarchy, in transitive order.
- visit(Pipeline.PipelineVisitor, Set<PValue>) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode
-
Visit the transform node.
- visitTransform(TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
-
Called for each primitive transform after all of its topological predecessors
and inputs have been visited.
- visitTransform(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
-
- visitValue(PValue, TransformTreeNode) - Method in interface com.google.cloud.dataflow.sdk.Pipeline.PipelineVisitor
-
Called for each value after the transform that produced the value has been
visited.
- visitValue(PValue, TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.RecordingPipelineVisitor
-
- VoidCoder - Class in com.google.cloud.dataflow.sdk.coders
-
- waitToFinish(long, TimeUnit, MonitoringUtil.JobMessagesHandler) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob
-
Waits for the job to finish and return the final status.
- weeks(int, int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows
-
Returns a
WindowFn that windows elements into periods measured by weeks.
- window() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
-
Returns the window into which the input element has been assigned.
- window() - Method in interface com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ExtraContextFactory
-
- window() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext
-
The window that the current context is executing in.
- Window - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
Window logically divides up or groups the elements of a
PCollection into finite windows according to a
WindowFn.
- Window() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.Window
-
- Window.Bound<T> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
A PTransform that windows the elements of a PCollection<T>,
into finite windows according to a user-specified WindowFn.
- Window.ClosingBehavior - Enum in com.google.cloud.dataflow.sdk.transforms.windowing
-
Specifies the conditions under which a final pane will be created when a window is permanently
closed.
- Window.Remerge<T> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
PTransform that does not change assigned windows, but will cause
windows to be merged again as part of the next
GroupByKey.
- Window.Unbound - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
An incomplete Window transform, with unbound input/output type.
- windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
-
- windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
-
- windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
-
- windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows
-
- windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows
-
- windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
-
- windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
-
- windowCoder() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
-
Returns the
Coder used for serializing the windows used
by this windowFn.
- WindowFn<T,W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
The argument to the
Window transform used to assign elements into
windows and to determine how windows are merged.
- WindowFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn
-
- WindowFn.AssignContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
- WindowFn.MergeContext - Class in com.google.cloud.dataflow.sdk.transforms.windowing
-
- WindowFnTestUtils - Class in com.google.cloud.dataflow.sdk.testing
-
- WindowFnTestUtils() - Constructor for class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils
-
- windowingInternals() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.ProcessContext
-
Returns the process context to use for implementing windowing.
- windowingInternals() - Method in interface com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.ExtraContextFactory
-
Construct the
WindowingInternals to use within a
DoFnWithContext that
needs it.
- windows() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext
-
Returns the windows the current element was in, prior to this
WindowFn being called.
- windows() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.MergeContext
-
Returns the current set of windows.
- with(SimpleFunction<DataT, InputT>, Combine.CombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeCombineFnBuilder
-
- with(SimpleFunction<DataT, InputT>, CombineWithContext.CombineFnWithContext<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeCombineFnBuilder
-
- with(SimpleFunction<DataT, InputT>, Combine.CombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
-
- with(SimpleFunction<DataT, InputT>, CombineWithContext.CombineFnWithContext<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn
-
- with(SimpleFunction<DataT, InputT>, CombineFnBase.GlobalCombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext
-
- with(SimpleFunction<DataT, InputT>, Combine.KeyedCombineFn<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
-
- with(SimpleFunction<DataT, InputT>, CombineWithContext.KeyedCombineFnWithContext<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
-
- with(SimpleFunction<DataT, InputT>, Combine.CombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
-
- with(SimpleFunction<DataT, InputT>, CombineWithContext.CombineFnWithContext<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn
-
- with(SimpleFunction<DataT, InputT>, CombineFnBase.PerKeyCombineFn<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
-
- with(SimpleFunction<DataT, InputT>, CombineFnBase.GlobalCombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext
-
- with(SimpleFunction<DataT, InputT>, Combine.KeyedCombineFn<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
-
- with(SimpleFunction<DataT, InputT>, CombineWithContext.KeyedCombineFnWithContext<K, InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
-
- with(SimpleFunction<DataT, InputT>, Combine.CombineFn<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
-
- with(SimpleFunction<DataT, InputT>, CombineWithContext.CombineFnWithContext<InputT, ?, OutputT>, TupleTag<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder
-
- withAllowedLateness(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
Override the amount of lateness allowed for data elements in the pipeline.
- withAllowedLateness(Duration, Window.ClosingBehavior) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
Override the amount of lateness allowed for data elements in the pipeline.
- withAllowedLateness(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
-
Override the amount of lateness allowed for data elements in the pipeline.
- withAllowedLateness(Duration, Window.ClosingBehavior) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound
-
Override the amount of lateness allowed for data elements in the pipeline.
- withAllowedLateness(Duration) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window
-
Override the amount of lateness allowed for data elements in the pipeline.
- withAllowedTimestampSkew(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.WithTimestamps
-
Return a new WithTimestamps like this one with updated allowed timestamp skew, which is the
maximum duration that timestamps can be shifted backward.
- withBigtableOptions(BigtableOptions) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
-
Returns a new
BigtableIO.Read that will read from the Cloud Bigtable cluster
indicated by the given options, and using any other specified customizations.
- withBigtableOptions(BigtableOptions.Builder) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
-
Returns a new
BigtableIO.Read that will read from the Cloud Bigtable cluster
indicated by the given options, and using any other specified customizations.
- withBigtableOptions(BigtableOptions) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
-
Returns a new
BigtableIO.Write that will write to the Cloud Bigtable cluster
indicated by the given options, and using any other specified customizations.
- withBigtableOptions(BigtableOptions.Builder) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
-
Returns a new
BigtableIO.Write that will write to the Cloud Bigtable cluster
indicated by the given options, and using any other specified customizations.
- withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound
-
Returns a transform that's like this one but that uses the given
Coder to decode each record into a value of type
X.
- withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read
-
Creates and returns a transform for reading from Cloud Pub/Sub that uses the given
Coder to decode Pub/Sub messages into a value of type
T.
- withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound
-
Returns a new transform that's like this one
but that uses the given
Coder to encode each of
the elements of the input
PCollection into an
output record.
- withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write
-
Creates a transform that uses the given
Coder to encode each of the
elements of the input collection into an output message.
- withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
-
Returns a new transform for reading from text files that's like this one but
that uses the given
Coder to decode each of the
lines of the file into a value of type
X.
- withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
-
Returns a transform for reading text files that uses the given
Coder<T> to decode each of the lines of the file into a
value of type T.
- withCoder(Coder<X>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
Returns a transform for writing to text files that's like this one
but that uses the given
Coder to encode each of
the elements of the input
PCollection into an
output text line.
- withCoder(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
-
Returns a transform for writing to text files that uses the given
Coder to encode each of the elements of the input
PCollection into an output text line.
- withCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.TimestampedValues
-
Returns a
Create.TimestampedValues PTransform like this one that uses the given
Coder<T> to decode each of the objects into a
value of type
T.
- withCoder(Coder<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values
-
Returns a
Create.Values PTransform like this one that uses the given
Coder<T> to decode each of the objects into a
value of type
T.
- withCompressionType(TextIO.CompressionType) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
-
Returns a new transform for reading from text files that's like this one but
reads from input sources using the specified compression type.
- withCompressionType(TextIO.CompressionType) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
-
Returns a transform for reading text files that decompresses all input files
using the specified compression type.
- withCreateDisposition(BigQueryIO.Write.CreateDisposition) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns a copy of this write transformation, but using the specified create disposition.
- withCreateDisposition(BigQueryIO.Write.CreateDisposition) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
-
Creates a write transformation with the specified options for creating the table.
- withDataset(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
-
Returns a
Sink that is like this one, but will write to the specified dataset.
- withDataset(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- withDecompression(CompressedSource.DecompressingChannelFactory) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource
-
- withDefaultValue(T) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton
-
Default value to return for windows with no value in them.
- withEarlyFirings(Trigger.OnceTrigger<W>) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.AfterWatermarkLate
-
Creates a new Trigger like the this, except that it fires repeatedly whenever
the given Trigger fires before the watermark has passed the end of the window.
- withEarlyFirings(Trigger.OnceTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
-
Creates a new Trigger like the this, except that it fires repeatedly whenever
the given Trigger fires before the watermark has passed the end of the window.
- withEncodabilityTesting(boolean) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
-
Enable runtime testing to verify that all values can be encoded.
- withEndKey(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
-
Returns new
ByteKeyRange like this one, but with the specified end key.
- withEpsilon(double) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
-
Returns an ApproximateQuantilesCombineFn that's like
this one except that it uses the specified epsilon
value.
- withExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
-
Deprecated.
Produces a Proto2Coder like this one, but with the extensions from
the given classes registered.
- withExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder
-
Deprecated.
- withExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
-
Returns a
ProtoCoder like this one, but with the extensions from the given classes
registered.
- withExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder
-
- withFanout(int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
-
Returns a
PTransform identical to this, but that uses an intermediate node
to combine parts of the data to reduce load on the final global combine step.
- withGapDuration(Duration) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions
-
Creates a
Sessions WindowFn with the specified gap duration.
- withHost(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink
-
Returns a
Sink that is like this one, but will use the given host.
- withHost(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- withHotKeyFanout(SerializableFunction<? super K, Integer>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
-
If a single key has disproportionately many values, it may become a
bottleneck, especially in streaming mode.
- withHotKeyFanout(int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
-
- withKey(Object) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
-
Returns a new ValueWithMetadata with the implicit key associated
with this value set.
- WithKeys<K,V> - Class in com.google.cloud.dataflow.sdk.transforms
-
WithKeys<K, V> takes a PCollection<V>, and either a
constant key of type K or a function from V to
K, and returns a PCollection<KV<K, V>>, where each
of the values in the input PCollection has been paired with
either the constant key or a key computed from the value.
- withKeyType(TypeDescriptor<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithKeys
-
Return a
WithKeys that is like this one with the specified key type descriptor.
- withLabel(String) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.ItemBuilder
-
Add a human-readable label to describe the most-recently added metadata field.
- withLateFirings(Trigger.OnceTrigger<W>) - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.AfterWatermarkEarly
-
Creates a new Trigger like the this, except that it fires repeatedly whenever
the given Trigger fires after the watermark has passed the end of the window.
- withLateFirings(Trigger.OnceTrigger<W>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterWatermark.FromEndOfWindow
-
Creates a new Trigger like the this, except that it fires repeatedly whenever
the given Trigger fires after the watermark has passed the end of the window.
- withLinkUrl(String) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.ItemBuilder
-
Add a link URL to the most-recently added display metadata.
- withMaxInputSize(long) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn
-
Returns an ApproximateQuantilesCombineFn that's like
this one except that it uses the specified maxNumElements
value.
- withMaxNumRecords(long) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput
-
- withMaxNumRecords(long) - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
-
Returns a new
BoundedReadFromUnboundedSource that reads a bounded amount
of data from the given
UnboundedSource.
- withMaxParallelism(int) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound
-
- withMaxParallelism(int) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Unbound
-
- withMaxParallelism(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization
-
- withMaxReadTime(Duration) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput
-
- withMaxReadTime(Duration) - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded
-
Returns a new
BoundedReadFromUnboundedSource that reads a bounded amount
of data from the given
UnboundedSource.
- withMinBundleSize(long) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
Returns an
AvroSource that's like this one but uses the supplied minimum bundle size.
- withMinBundleSize(long) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
Sets a parameter minBundleSize for the minimum bundle size of the source.
- withNamespace(String) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
- withNumShards(int) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns a new
PTransform that's like this one but
that uses the provided shard count.
- withNumShards(int) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
-
Returns a
PTransform that uses the provided shard count.
- withNumShards(int) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
Returns a transform for writing to text files that's like this one but
that uses the provided shard count.
- withNumShards(int) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
-
Returns a transform for writing to text files that uses the provided shard count.
- withOffset(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows
-
Partitions the timestamp space into half-open intervals of the form
[N * size + offset, (N + 1) * size + offset),
where 0 is the epoch.
- withOffset(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows
-
Assigns timestamps into half-open intervals of the form
[N * period + offset, N * period + offset + size).
- withoutDefaults() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
-
Returns a
PTransform identical to this, but that does not attempt to
provide a default value in the case of empty input.
- withOutputTags(TupleTag<OutputT>, TupleTagList) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
-
- withOutputTags(TupleTag<OutputT>, TupleTagList) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
-
Returns a new multi-output
ParDo transform that's like
this transform but with the specified main and side output
tags.
- withOutputTags(TupleTag<OutputT>, TupleTagList) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
-
Creates a multi-output
ParDo PTransform whose
output
PCollections will be referenced using the given main
output and side output tags.
- withOutputTimeFn(OutputTimeFn<?>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound
-
(Experimental) Override the default
OutputTimeFn, to control
the output timestamp of values output from a
GroupByKey operation.
- withOutputType(TypeDescriptor<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements.MissingOutputTypeDescriptor
-
- withOutputType(TypeDescriptor<OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.MapElements.MissingOutputTypeDescriptor
-
- withoutResultFlattening() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
- withoutSharding() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns a new
PTransform that's like this one but
that forces a single file as output.
- withoutSharding() - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
-
Returns a
PTransform that forces a single file as
output.
- withoutSharding() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
Returns a transform for writing to text files that's like this one but
that forces a single file as output.
- withoutSharding() - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
-
Returns a transform for writing to text files that forces a single file as
output.
- withoutStrictParsing() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
-
During parsing of the arguments, we will skip over improperly formatted and unknown
arguments.
- withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
-
Returns a new
PTransform that's like this one but
that has GCS input path validation on pipeline creation disabled.
- withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
-
Returns a
PTransform that reads Avro file(s)
that has GCS path validation on pipeline creation disabled.
- withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns a new
PTransform that's like this one but
that has GCS output path validation on pipeline creation disabled.
- withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
-
Returns a
PTransform that writes Avro file(s) that has GCS path validation on
pipeline creation disabled.
- withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound
-
Disable table validation.
- withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read
-
Disables BigQuery table validation, which is enabled by default.
- withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns a copy of this write transformation, but without BigQuery table validation.
- withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
-
Creates a write transformation with BigQuery table validation disabled.
- withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound
-
Returns a new transform for reading from text files that's like this one but
that has GCS path validation on pipeline creation disabled.
- withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Read
-
Returns a transform for reading text files that has GCS path validation on
pipeline creation disabled.
- withoutValidation() - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
Returns a transform for writing to text files that's like this one but
that has GCS output path validation on pipeline creation disabled.
- withoutValidation() - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
-
Returns a transform for writing to text files that has GCS path validation on
pipeline creation disabled.
- withQuery(DatastoreV1.Query) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source
-
Returns a new
Source that reads the results of the specified query.
- withRecordClass(Class<T>) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
Sets a JAXB annotated class that can be populated using a record of the provided XML file.
- withRecordElement(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
Sets name of the record element of the XML document.
- withRepresentativeType(TypeDescriptor<IdT>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates.WithRepresentativeValues
-
Return a
WithRepresentativeValues PTransform that is like this one, but with
the specified output type descriptor.
- withRepresentativeValueFn(SerializableFunction<T, IdT>) - Static method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates
-
Returns a RemoveDuplicates<T, IdT> PTransform.
- withRootElement(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound
-
Returns an XmlSink that writes XML files with an enclosing root element of the
supplied name.
- withRootElement(String) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource
-
Sets name of the root element of the XML document.
- withRowFilter(RowFilter) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
-
Returns a new
BigtableIO.Read that will filter the rows read from Cloud Bigtable
using the given row filter.
- withSchema(Class<X>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
-
Returns a new
PTransform that's like this one but
that reads Avro file(s) containing records whose type is the
specified Avro-generated class.
- withSchema(Schema) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
-
Returns a new
PTransform that's like this one but
that reads Avro file(s) containing records of the specified schema.
- withSchema(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound
-
Returns a new
PTransform that's like this one but
that reads Avro file(s) containing records of the specified schema
in a JSON-encoded string form.
- withSchema(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
-
Returns a
PTransform that reads Avro file(s)
containing records whose type is the specified Avro-generated class.
- withSchema(Schema) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
-
Returns a
PTransform that reads Avro file(s)
containing records of the specified schema.
- withSchema(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read
-
Returns a
PTransform that reads Avro file(s)
containing records of the specified schema in a JSON-encoded
string form.
- withSchema(Class<X>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns a new
PTransform that's like this one but
that writes to Avro file(s) containing records whose type is the
specified Avro-generated class.
- withSchema(Schema) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns a new
PTransform that's like this one but
that writes to Avro file(s) containing records of the specified
schema.
- withSchema(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns a new
PTransform that's like this one but
that writes to Avro file(s) containing records of the specified
schema in a JSON-encoded string form.
- withSchema(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
-
Returns a
PTransform that writes Avro file(s)
containing records whose type is the specified Avro-generated class.
- withSchema(Schema) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
-
Returns a
PTransform that writes Avro file(s)
containing records of the specified schema.
- withSchema(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
-
Returns a
PTransform that writes Avro file(s)
containing records of the specified schema in a JSON-encoded
string form.
- withSchema(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
Returns an
AvroSource that's like this one but reads files containing records that
conform to the given schema.
- withSchema(Schema) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
Returns an
AvroSource that's like this one but reads files containing records that
conform to the given schema.
- withSchema(Class<X>) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource
-
Returns an
AvroSource that's like this one but reads files containing records of the
type of the given class.
- withSchema(TableSchema) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns a copy of this write transformation, but using the specified schema for rows
to be written.
- withSchema(TableSchema) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
-
Creates a write transformation with the specified schema to use in table creation.
- withSerializabilityTesting(boolean) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
-
Enable runtime testing to verify that all functions and
Coder
instances can be serialized.
- withShardNameTemplate(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns a new
PTransform that's like this one but
that uses the given shard name template.
- withShardNameTemplate(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
-
Returns a
PTransform that uses the given shard name
template.
- withShardNameTemplate(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
Returns a transform for writing to text files that's like this one but
that uses the given shard name template.
- withShardNameTemplate(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
-
Returns a transform for writing to text files that uses the given shard name
template.
- withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally
-
- withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues
-
- withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey
-
- withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
-
- withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound
-
- withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti
-
- withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
-
Returns a new
ParDo transform that's like this
transform but with the specified additional side inputs.
- withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Unbound
-
Returns a new
ParDo transform that is like this
transform but with the specified additional side inputs.
- withSideInputs(PCollectionView<?>...) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
-
Returns a new multi-output
ParDo transform that's like
this transform but with the specified side inputs.
- withSideInputs(Iterable<? extends PCollectionView<?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.UnboundMulti
-
Returns a new multi-output
ParDo transform that's like
this transform but with the specified additional side inputs.
- withSideInputs(PCollectionView<?>...) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
-
- withSideInputs(Iterable<? extends PCollectionView<?>>) - Static method in class com.google.cloud.dataflow.sdk.transforms.ParDo
-
Creates a
ParDo with the given side inputs.
- withSingletonValues() - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap
-
- withStartingDay(int, int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
-
- withStartingMonth(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
-
- withStartingYear(int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- withStartKey(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange
-
Returns new
ByteKeyRange like this one, but with the specified start key.
- withSuffix(String) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound
-
Returns a new
PTransform that's like this one but
that writes to the file(s) with the given filename suffix.
- withSuffix(String) - Static method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write
-
Returns a
PTransform that writes to the file(s) with the
given filename suffix.
- withSuffix(String) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound
-
Returns a transform for writing to text files that that's like this one but
that writes to the file(s) with the given filename suffix.
- withSuffix(String) - Static method in class com.google.cloud.dataflow.sdk.io.TextIO.Write
-
Returns a transform for writing to text files that appends the specified suffix
to the created files.
- withTableId(String) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read
-
- withTableId(String) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write
-
- withTimestampFn(SerializableFunction<Long, Instant>) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput
-
- WithTimestamps<T> - Class in com.google.cloud.dataflow.sdk.transforms
-
- withTimeZone(DateTimeZone) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows
-
- withTimeZone(DateTimeZone) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows
-
- withTimeZone(DateTimeZone) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows
-
- withUnorderednessTesting(boolean) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner
-
Enable runtime testing to verify that functions do not depend on order
of the elements.
- withValidation() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder
-
After creation we will validate that
PipelineOptions conforms to all the
validation criteria from
<T>.
- withValidation() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory
-
After creation we will validate that <T> conforms to all the
validation criteria.
- withValue(T) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.ValueWithMetadata
-
Returns a new ValueWithMetadata that is a copy of this one, but with
a different value.
- withWriteDisposition(BigQueryIO.Write.WriteDisposition) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound
-
Returns a copy of this write transformation, but using the specified write disposition.
- withWriteDisposition(BigQueryIO.Write.WriteDisposition) - Static method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write
-
Creates a write transformation with the specified options for writing to the table.
- WorkerHarnessContainerImageFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory
-
- WorkerLogLevelOverrides() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
-
- write() - Static method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO
-
- write(T) - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer
-
Called for each value in the bundle.
- Write() - Constructor for class com.google.cloud.dataflow.sdk.io.TextIO.Write
-
- Write - Class in com.google.cloud.dataflow.sdk.io
-
- Write() - Constructor for class com.google.cloud.dataflow.sdk.io.Write
-
- write() - Static method in class com.google.cloud.dataflow.sdk.io.XmlSink
-
Returns a builder for an XmlSink.
- write(T) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
-
Writes a value to the stream.
- Write - Class in com.google.cloud.dataflow.sdk.transforms
-
Deprecated.
- Write() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Write
-
Deprecated.
- Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io
-
- writeFooter() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
-
Writes footer at the end of output files.
- writeFooter() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
-
Writes the root element closing tag.
- writeHeader() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter
-
Writes header at the beginning of output files.
- writeHeader() - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriter
-
Writes the root element opening tag.
- writeOf(Class<T>, String, String) - Static method in class com.google.cloud.dataflow.sdk.io.XmlSink
-
Returns an XmlSink that writes objects as XML entities.
- WriteOperation() - Constructor for class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation
-
- Writer() - Constructor for class com.google.cloud.dataflow.sdk.io.Sink.Writer
-
- writeTo(String) - Static method in class com.google.cloud.dataflow.sdk.io.DatastoreIO
-
Returns a new
Write transform that will write to a
Sink.