Class StorageApiWriteUnshardedRecords<DestinationT,ElementT>
- java.lang.Object
-
- org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,StorageApiWritePayload>>,org.apache.beam.sdk.values.PCollectionTuple>
-
- org.apache.beam.sdk.io.gcp.bigquery.StorageApiWriteUnshardedRecords<DestinationT,ElementT>
-
- All Implemented Interfaces:
java.io.Serializable
,org.apache.beam.sdk.transforms.display.HasDisplayData
public class StorageApiWriteUnshardedRecords<DestinationT,ElementT> extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,StorageApiWritePayload>>,org.apache.beam.sdk.values.PCollectionTuple>
Write records to the Storage API using a standard batch approach. PENDING streams are used, which do not become visible until they are finalized and committed. Each input bundle to the DoFn creates a stream per output table, appends all records in the bundle to the stream, and schedules a finalize/commit operation at the end.- See Also:
- Serialized Form
-
-
Constructor Summary
Constructors Constructor Description StorageApiWriteUnshardedRecords(org.apache.beam.sdk.io.gcp.bigquery.StorageApiDynamicDestinations<ElementT,DestinationT> dynamicDestinations, BigQueryServices bqServices, org.apache.beam.sdk.values.TupleTag<BigQueryStorageApiInsertError> failedRowsTag, org.apache.beam.sdk.coders.Coder<BigQueryStorageApiInsertError> failedRowsCoder)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description org.apache.beam.sdk.values.PCollectionTuple
expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,StorageApiWritePayload>> input)
-
-
-
Constructor Detail
-
StorageApiWriteUnshardedRecords
public StorageApiWriteUnshardedRecords(org.apache.beam.sdk.io.gcp.bigquery.StorageApiDynamicDestinations<ElementT,DestinationT> dynamicDestinations, BigQueryServices bqServices, org.apache.beam.sdk.values.TupleTag<BigQueryStorageApiInsertError> failedRowsTag, org.apache.beam.sdk.coders.Coder<BigQueryStorageApiInsertError> failedRowsCoder)
-
-
Method Detail
-
expand
public org.apache.beam.sdk.values.PCollectionTuple expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,StorageApiWritePayload>> input)
- Specified by:
expand
in classorg.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,StorageApiWritePayload>>,org.apache.beam.sdk.values.PCollectionTuple>
-
-