T - the type of the elements of all the PCollections in this listpublic class PCollectionList<T> extends java.lang.Object implements PInput, POutput
PCollectionList<T> is an immutable list of homogeneously
typed PCollection<T>s. A PCollectionList is used, for
instance, as the input to
Flatten or the
output of
Partition.
PCollectionLists can be created and accessed like follows:
PCollection<String> pc1 = ...;
PCollection<String> pc2 = ...;
PCollection<String> pc3 = ...;
// Create a PCollectionList with three PCollections:
PCollectionList<String> pcs = PCollectionList.of(pc1).and(pc2).and(pc3);
// Create an empty PCollectionList:
Pipeline p = ...;
PCollectionList<String> pcs2 = PCollectionList.<String>empty(p);
// Get PCollections out of a PCollectionList, by index (origin 0):
PCollection<String> pcX = pcs.get(1);
PCollection<String> pcY = pcs.get(0);
PCollection<String> pcZ = pcs.get(2);
// Get a list of all PCollections in a PCollectionList:
List<PCollection<String>> allPcs = pcs.getAll();
| Modifier and Type | Method and Description |
|---|---|
PCollectionList<T> |
and(java.lang.Iterable<PCollection<T>> pcs)
Returns a new PCollectionList that has all the PCollections of
this PCollectionList plus the given PCollections appended to the end,
in order.
|
PCollectionList<T> |
and(PCollection<T> pc)
Returns a new PCollectionList that has all the PCollections of
this PCollectionList plus the given PCollection appended to the end.
|
<Output extends POutput> |
apply(PTransform<PCollectionList<T>,Output> t)
Applies the given PTransform to this input
PCollectionList<T>,
and returns the PTransform's Output. |
static <T> PCollectionList<T> |
empty(Pipeline pipeline)
Returns an empty PCollectionList that is part of the given Pipeline.
|
java.util.Collection<? extends PValue> |
expand()
Expands this PInput into a list of its component input PValues.
|
void |
finishSpecifying()
After building, finalizes this PInput to make it ready for
being used as an input to a PTransform.
|
void |
finishSpecifyingOutput()
As part of finishing the producing
PTransform, finalizes this
PTransform output to make it ready for being used as an input and
for running. |
PCollection<T> |
get(int index)
Returns the PCollection at the given index (origin zero).
|
java.util.List<PCollection<T>> |
getAll()
Returns an immutable List of all the PCollections in this PCollectionList.
|
Pipeline |
getPipeline()
Returns the owning Pipeline of this PInput.
|
static <T> PCollectionList<T> |
of(java.lang.Iterable<PCollection<T>> pcs)
Returns a PCollectionList containing the given PCollections, in order.
|
static <T> PCollectionList<T> |
of(PCollection<T> pc)
Returns a singleton PCollectionList containing the given PCollection.
|
void |
recordAsOutput(Pipeline pipeline,
PTransform<?,?> transform)
Records that this
POutput is an output of the given
PTransform in the given Pipeline. |
int |
size()
Returns the number of PCollections in this PCollectionList.
|
public static <T> PCollectionList<T> empty(Pipeline pipeline)
Longer PCollectionLists can be created by calling
and(com.google.cloud.dataflow.sdk.values.PCollection<T>) on the result.
public static <T> PCollectionList<T> of(PCollection<T> pc)
Longer PCollectionLists can be created by calling
and(com.google.cloud.dataflow.sdk.values.PCollection<T>) on the result.
public static <T> PCollectionList<T> of(java.lang.Iterable<PCollection<T>> pcs)
The argument list cannot be empty.
All the PCollections in the resulting PCollectionList must be part of the same Pipeline.
Longer PCollectionLists can be created by calling
and(com.google.cloud.dataflow.sdk.values.PCollection<T>) on the result.
public PCollectionList<T> and(PCollection<T> pc)
All the PCollections in the resulting PCollectionList must be part of the same Pipeline.
public PCollectionList<T> and(java.lang.Iterable<PCollection<T>> pcs)
All the PCollections in the resulting PCollectionList must be part of the same Pipeline.
public int size()
public PCollection<T> get(int index)
[0..size()-1].public java.util.List<PCollection<T>> getAll()
public <Output extends POutput> Output apply(PTransform<PCollectionList<T>,Output> t)
PCollectionList<T>,
and returns the PTransform's Output.public Pipeline getPipeline()
PInputgetPipeline in interface PInputpublic java.util.Collection<? extends PValue> expand()
PInputA PValue expands to itself.
A tuple or list of PValues (e.g., PCollectionTuple, and PCollectionList) expands to its component PValues.
Not intended to be invoked directly by user code.
public void recordAsOutput(Pipeline pipeline, PTransform<?,?> transform)
POutputPOutput is an output of the given
PTransform in the given Pipeline.
Should expand this POutput and invoke
POutput.recordAsOutput(Pipeline, com.google.cloud.dataflow.sdk.transforms.PTransform)
on each component output PValue.
Automatically invoked as part of applying a
PTransform. Not to be invoked directly by user code.
recordAsOutput in interface POutputpublic void finishSpecifying()
PInputAfter building, finalizes this PInput to make it ready for being used as an input to a PTransform.
Automatically invoked whenever apply() is invoked on
this PInput, so users do not normally call this explicitly.
finishSpecifying in interface PInputpublic void finishSpecifyingOutput()
POutputPTransform, finalizes this
PTransform output to make it ready for being used as an input and
for running.
This includes ensuring that all PCollections
have Coders specified or defaulted.
Automatically invoked whenever this POutput is used
as a PInput to another PTransform, or if never
used as a PInput, when Pipeline.run() is called, so
users do not normally call this explicitly.
finishSpecifyingOutput in interface POutput