com.google.cloud.dataflow.sdk.transforms

Class View

java.lang.Object

com.google.cloud.dataflow.sdk.transforms.View

public class View
extends Object

Transforms for creating PCollectionViews from PCollections, for consuming the contents of those PCollections as side inputs to ParDo transforms. These transforms support viewing a PCollection as a single value, an iterable, a map, or a multimap.

For a PCollection that contains a single value of type T per window, such as the output of Combine.globally(com.google.cloud.dataflow.sdk.transforms.SerializableFunction<java.lang.Iterable<V>, V>), use asSingleton() to prepare it for use as a side input:

 PCollectionView<T> output = someOtherPCollection
     .apply(Combine.globally(...))
     .apply(View.asSingleton());
 
 

To iterate over an entire window of a PCollection via side input, use asIterable():

 PCollectionView<Iterable<T>> output =
     somePCollection.apply(View.asIterable());
 
 

To access a PCollection of KV<K, V> as a Map<K, Iterable<V>> side input, use asMap():

 PCollectionView<Map<K, Iterable<V>> output =
     somePCollection.apply(View.asMap());
 
 

If a PCollection of KV<K, V> is known to have a single value for each key, then use View.AsMultimap#withSingletonValues View.asMap().withSingletonValues() to view it as a Map<K, V>:

 PCollectionView<Map<K, V> output =
     somePCollection.apply(View.asMap().withSingletonValues());
 
 

See ParDo.withSideInputs(com.google.cloud.dataflow.sdk.values.PCollectionView<?>...) for details on how to access this variable inside a ParDo over another PCollection.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	View.AsIterable<T> A PTransform that produces a PCollectionView of a singleton PCollection yielding the single element it contains.
static class	View.AsMultimap<K,V> A PTransform that produces a PCollectionView of a keyed PCollection yielding a map of keys to all associated values.
static class	View.AsSingleton<T> A PTransform that produces a PCollectionView of a singleton PCollection yielding the single element it contains.
static class	View.AsSingletonMap<K,InputT,OutputT> A PTransform that produces a PCollectionView of a keyed PCollection yielding a map of keys to a single associated values.
static class	View.CreatePCollectionView<ElemT,ViewT> Creates a primitive PCollectionView.

Method Summary

Methods

Modifier and Type	Method and Description
static <T> View.AsIterable<T>	asIterable() Returns a View.AsIterable that takes a PCollection as input and produces a PCollectionView of the values, to be consumed as an iterable side input.
static <T> PTransform<PCollection<T>,PCollectionView<List<T>>>	asList() Returns a transform that takes a PCollection and returns a List containing all of its elements, to be consumed as a side input.
static <K,V> View.AsMultimap<K,V>	asMap() Returns an View.AsMultimap that takes a PCollection as input and produces a PCollectionView of the values to be consumed as a Map<K, Iterable<V>> side input.
static <T> View.AsSingleton<T>	asSingleton() Returns a View.AsSingleton transform that takes a singleton PCollection as input and produces a PCollectionView of the single value, to be consumed as a side input.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

asSingleton

public static <T> View.AsSingleton<T> asSingleton()

Returns a View.AsSingleton transform that takes a singleton PCollection as input and produces a PCollectionView of the single value, to be consumed as a side input.

If the input PCollection is empty, throws NoSuchElementException in the consuming DoFn.

If the input PCollection contains more than one element, throws IllegalArgumentException in the consuming DoFn.

asList

public static <T> PTransform<PCollection<T>,PCollectionView<List<T>>> asList()

Returns a transform that takes a PCollection and returns a List containing all of its elements, to be consumed as a side input.

The resulting list is required to fit in memory.

asIterable

public static <T> View.AsIterable<T> asIterable()

Returns a View.AsIterable that takes a PCollection as input and produces a PCollectionView of the values, to be consumed as an iterable side input. The values of this Iterable may not be cached; if that behavior is desired, use asList().

asMap

public static <K,V> View.AsMultimap<K,V> asMap()

Returns an View.AsMultimap that takes a PCollection as input and produces a PCollectionView of the values to be consumed as a Map<K, Iterable<V>> side input.

Currently, the resulting map is required to fit into memory.