public abstract static class SpannerIO.Write extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<com.google.cloud.spanner.Mutation>,SpannerWriteResult>
PTransform
that writes Mutation
objects to Google Cloud Spanner.SpannerIO
,
Serialized FormConstructor and Description |
---|
Write() |
Modifier and Type | Method and Description |
---|---|
SpannerWriteResult |
expand(org.apache.beam.sdk.values.PCollection<com.google.cloud.spanner.Mutation> input) |
SpannerIO.WriteGrouped |
grouped()
Same transform but can be applied to
PCollection of MutationGroup . |
void |
populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder) |
SpannerIO.Write |
withBatchSizeBytes(long batchSizeBytes)
Specifies the batch size limit (max number of bytes mutated per batch).
|
SpannerIO.Write |
withCommitDeadline(org.joda.time.Duration commitDeadline)
Specifies the deadline for the Commit API call.
|
SpannerIO.Write |
withDatabaseId(java.lang.String databaseId)
Specifies the Cloud Spanner database.
|
SpannerIO.Write |
withDatabaseId(org.apache.beam.sdk.options.ValueProvider<java.lang.String> databaseId)
Specifies the Cloud Spanner database.
|
SpannerIO.Write |
withEmulatorHost(java.lang.String emulatorHost) |
SpannerIO.Write |
withEmulatorHost(org.apache.beam.sdk.options.ValueProvider<java.lang.String> emulatorHost)
Specifies the Cloud Spanner emulator host.
|
SpannerIO.Write |
withFailureMode(SpannerIO.FailureMode failureMode)
Specifies failure mode.
|
SpannerIO.Write |
withGroupingFactor(int groupingFactor)
Specifies the multiple of max mutation (in terms of both bytes per batch and cells per batch)
that is used to select a set of mutations to sort by key for batching.
|
SpannerIO.Write |
withHost(java.lang.String host)
Specifies the Cloud Spanner host.
|
SpannerIO.Write |
withHost(org.apache.beam.sdk.options.ValueProvider<java.lang.String> host)
Specifies the Cloud Spanner host.
|
SpannerIO.Write |
withInstanceId(java.lang.String instanceId)
Specifies the Cloud Spanner instance.
|
SpannerIO.Write |
withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String> instanceId)
Specifies the Cloud Spanner instance.
|
SpannerIO.Write |
withMaxCumulativeBackoff(org.joda.time.Duration maxCumulativeBackoff)
Specifies the maximum cumulative backoff time when retrying after DEADLINE_EXCEEDED errors.
|
SpannerIO.Write |
withMaxNumMutations(long maxNumMutations)
Specifies the cell mutation limit (maximum number of mutated cells per batch).
|
SpannerIO.Write |
withMaxNumRows(long maxNumRows)
Specifies the row mutation limit (maximum number of mutated rows per batch).
|
SpannerIO.Write |
withProjectId(java.lang.String projectId)
Specifies the Cloud Spanner project.
|
SpannerIO.Write |
withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String> projectId)
Specifies the Cloud Spanner project.
|
SpannerIO.Write |
withSchemaReadySignal(org.apache.beam.sdk.values.PCollection<?> signal)
Specifies an optional input PCollection that can be used as the signal for
Wait.OnSignal to indicate when the database schema is ready to be read. |
SpannerIO.Write |
withSpannerConfig(SpannerConfig spannerConfig)
Specifies the Cloud Spanner configuration.
|
public SpannerIO.Write withSpannerConfig(SpannerConfig spannerConfig)
public SpannerIO.Write withProjectId(java.lang.String projectId)
public SpannerIO.Write withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String> projectId)
public SpannerIO.Write withInstanceId(java.lang.String instanceId)
public SpannerIO.Write withInstanceId(org.apache.beam.sdk.options.ValueProvider<java.lang.String> instanceId)
public SpannerIO.Write withDatabaseId(java.lang.String databaseId)
public SpannerIO.Write withDatabaseId(org.apache.beam.sdk.options.ValueProvider<java.lang.String> databaseId)
public SpannerIO.Write withHost(org.apache.beam.sdk.options.ValueProvider<java.lang.String> host)
public SpannerIO.Write withHost(java.lang.String host)
public SpannerIO.Write withEmulatorHost(org.apache.beam.sdk.options.ValueProvider<java.lang.String> emulatorHost)
public SpannerIO.Write withEmulatorHost(java.lang.String emulatorHost)
public SpannerIO.Write withCommitDeadline(org.joda.time.Duration commitDeadline)
withMaxCumulativeBackoff(Duration)
is
reached. DEADLINE_EXCEEDED errors are reported with logging and counters.public SpannerIO.Write withMaxCumulativeBackoff(org.joda.time.Duration maxCumulativeBackoff)
If the mutations still have not been written after this time, they are treated as a
failure, and handled according to the setting of #withFailureMode(FailureMode)
.
public SpannerIO.WriteGrouped grouped()
PCollection
of MutationGroup
.public SpannerIO.Write withBatchSizeBytes(long batchSizeBytes)
public SpannerIO.Write withFailureMode(SpannerIO.FailureMode failureMode)
SpannerIO.FailureMode.FAIL_FAST
mode is selected by default.public SpannerIO.Write withMaxNumMutations(long maxNumMutations)
public SpannerIO.Write withMaxNumRows(long maxNumRows)
public SpannerIO.Write withSchemaReadySignal(org.apache.beam.sdk.values.PCollection<?> signal)
Wait.OnSignal
to indicate when the database schema is ready to be read.
To be used when the database schema is created by another section of the pipeline, this
causes this transform to wait until the signal PCollection
has been closed before
reading the schema from the database.
Wait.OnSignal
public SpannerIO.Write withGroupingFactor(int groupingFactor)
public SpannerWriteResult expand(org.apache.beam.sdk.values.PCollection<com.google.cloud.spanner.Mutation> input)
expand
in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<com.google.cloud.spanner.Mutation>,SpannerWriteResult>
public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
populateDisplayData
in interface org.apache.beam.sdk.transforms.display.HasDisplayData
populateDisplayData
in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<com.google.cloud.spanner.Mutation>,SpannerWriteResult>