T
- the type of the elements of the input PCollectionpublic static class TextIO.Write.Bound<T> extends PTransform<PCollection<T>,PDone>
name
Modifier and Type | Method and Description |
---|---|
PDone |
apply(PCollection<T> input)
Applies this
PTransform on the given InputT , and returns its
Output . |
Coder<T> |
getCoder() |
protected Coder<Void> |
getDefaultOutputCoder()
Returns the default
Coder to use for the output of this
single-output PTransform . |
String |
getFilenamePrefix() |
String |
getFilenameSuffix() |
String |
getFooter() |
String |
getHeader() |
int |
getNumShards() |
String |
getShardNameTemplate()
Returns the current shard name template string.
|
String |
getShardTemplate() |
TextIO.Write.Bound<T> |
named(String name)
Returns a transform for writing to text files that's like this one but
with the given step name.
|
boolean |
needsValidation() |
void |
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.
|
TextIO.Write.Bound<T> |
to(String filenamePrefix)
Returns a transform for writing to text files that's like this one but
that writes to the file(s) with the given filename prefix.
|
<X> TextIO.Write.Bound<X> |
withCoder(Coder<X> coder)
Returns a transform for writing to text files that's like this one
but that uses the given
Coder to encode each of
the elements of the input PCollection into an
output text line. |
TextIO.Write.Bound<T> |
withFooter(String footer)
Returns a transform for writing to text files that adds a footer string to the files
it writes.
|
TextIO.Write.Bound<T> |
withHeader(String header)
Returns a transform for writing to text files that adds a header string to the files
it writes.
|
TextIO.Write.Bound<T> |
withNumShards(int numShards)
Returns a transform for writing to text files that's like this one but
that uses the provided shard count.
|
TextIO.Write.Bound<T> |
withoutSharding()
Returns a transform for writing to text files that's like this one but
that forces a single file as output.
|
TextIO.Write.Bound<T> |
withoutValidation()
Returns a transform for writing to text files that's like this one but
that has GCS output path validation on pipeline creation disabled.
|
TextIO.Write.Bound<T> |
withShardNameTemplate(String shardTemplate)
Returns a transform for writing to text files that's like this one but
that uses the given shard name template.
|
TextIO.Write.Bound<T> |
withSuffix(String nameExtension)
Returns a transform for writing to text files that that's like this one but
that writes to the file(s) with the given filename suffix.
|
getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, toString, validate
public TextIO.Write.Bound<T> named(String name)
Does not modify this object.
public TextIO.Write.Bound<T> to(String filenamePrefix)
See Write.to(String)
for more information.
Does not modify this object.
public TextIO.Write.Bound<T> withSuffix(String nameExtension)
Does not modify this object.
ShardNameTemplate
public TextIO.Write.Bound<T> withNumShards(int numShards)
Constraining the number of shards is likely to reduce the performance of a pipeline. Setting this value is not recommended unless you require a specific number of output files.
Does not modify this object.
numShards
- the number of shards to use, or 0 to let the system
decide.ShardNameTemplate
public TextIO.Write.Bound<T> withShardNameTemplate(String shardTemplate)
Does not modify this object.
ShardNameTemplate
public TextIO.Write.Bound<T> withoutSharding()
Constraining the number of shards is likely to reduce the performance of a pipeline. Using this setting is not recommended unless you truly require a single output file.
This is a shortcut for
.withNumShards(1).withShardNameTemplate("")
Does not modify this object.
public <X> TextIO.Write.Bound<X> withCoder(Coder<X> coder)
Coder
to encode each of
the elements of the input PCollection
into an
output text line. Does not modify this object.X
- the type of the elements of the input PCollection
public TextIO.Write.Bound<T> withHeader(@Nullable String header)
A null
value will clear any previously configured header.
Does not modify this object.
header
- the string to be added as file headerpublic TextIO.Write.Bound<T> withFooter(@Nullable String footer)
A null
value will clear any previously configured footer.
Does not modify this object.
footer
- the string to be added as file footerpublic TextIO.Write.Bound<T> withoutValidation()
This can be useful in the case where the GCS output location does not exist at the pipeline creation time, but is expected to be available at execution time.
Does not modify this object.
public PDone apply(PCollection<T> input)
PTransform
PTransform
on the given InputT
, and returns its
Output
.
Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
The default implementation throws an exception. A derived class must
either implement apply, or else each runner must supply a custom
implementation via
PipelineRunner.apply(com.google.cloud.dataflow.sdk.transforms.PTransform<InputT, OutputT>, InputT)
.
apply
in class PTransform<PCollection<T>,PDone>
public void populateDisplayData(DisplayData.Builder builder)
PTransform
populateDisplayData(DisplayData.Builder)
is invoked by Pipeline runners to collect
display data via DisplayData.from(HasDisplayData)
. Implementations may call
super.populateDisplayData(builder)
in order to register display data in the current
namespace, but should otherwise use subcomponent.populateDisplayData(builder)
to use
the namespace of the subcomponent.
By default, does not register any display data. Implementors may override this method to provide their own display data.
populateDisplayData
in interface HasDisplayData
populateDisplayData
in class PTransform<PCollection<T>,PDone>
builder
- The builder to populate with display data.HasDisplayData
public String getShardNameTemplate()
protected Coder<Void> getDefaultOutputCoder()
PTransform
Coder
to use for the output of this
single-output PTransform
.
By default, always throws
getDefaultOutputCoder
in class PTransform<PCollection<T>,PDone>
public String getFilenamePrefix()
public String getShardTemplate()
public int getNumShards()
public String getFilenameSuffix()
public boolean needsValidation()