public class PubsubUnboundedSink<T> extends PTransform<PCollection<T>,PDone>
GroupByKey
followed by a ParDo
which
publishes as a side effect. (In the future we want to design and switch to a custom
UnboundedSink
implementation so as to gain access to system watermark and
end-of-pipeline cleanup.)
DoFn
instances
to execute concurrently and hide latency.
NOTE: This is not the implementation used when running on the Google Cloud Dataflow service.
name
Constructor and Description |
---|
PubsubUnboundedSink(com.google.cloud.dataflow.sdk.util.PubsubClient.PubsubClientFactory pubsubFactory,
com.google.cloud.dataflow.sdk.util.PubsubClient.TopicPath topic,
Coder<T> elementCoder,
String timestampLabel,
String idLabel,
int numShards) |
Modifier and Type | Method and Description |
---|---|
PDone |
apply(PCollection<T> input)
Applies this
PTransform on the given InputT , and returns its
Output . |
Coder<T> |
getElementCoder() |
String |
getIdLabel() |
String |
getTimestampLabel() |
com.google.cloud.dataflow.sdk.util.PubsubClient.TopicPath |
getTopic() |
getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate
public com.google.cloud.dataflow.sdk.util.PubsubClient.TopicPath getTopic()
public PDone apply(PCollection<T> input)
PTransform
PTransform
on the given InputT
, and returns its
Output
.
Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
The default implementation throws an exception. A derived class must
either implement apply, or else each runner must supply a custom
implementation via
PipelineRunner.apply(com.google.cloud.dataflow.sdk.transforms.PTransform<InputT, OutputT>, InputT)
.
apply
in class PTransform<PCollection<T>,PDone>