@Internal
@AutoService(value=org.apache.beam.sdk.schemas.io.SchemaIOProvider.class)
public class BigQuerySchemaIOProvider
extends java.lang.Object
implements org.apache.beam.sdk.schemas.io.SchemaIOProvider
SchemaIOProvider
for reading and writing to BigQuery with BigQueryIO
. For a description of configuration options and other defaults, see configurationSchema()
.Constructor and Description |
---|
BigQuerySchemaIOProvider() |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.schemas.Schema |
configurationSchema()
Returns the expected schema of the configuration object.
|
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySchemaIOProvider.BigQuerySchemaIO |
from(java.lang.String location,
org.apache.beam.sdk.values.Row configuration,
@Nullable org.apache.beam.sdk.schemas.Schema dataSchema)
Produces a SchemaIO given a String representing the data's location, the schema of the data
that resides there, and some IO-specific configuration object.
|
java.lang.String |
identifier()
Returns an id that uniquely represents this IO.
|
org.apache.beam.sdk.values.PCollection.IsBounded |
isBounded()
Indicates whether the PCollections produced by this transform will contain a bounded or
unbounded number of elements.
|
boolean |
requiresDataSchema()
Indicates whether this transform requires a specified data schema.
|
public java.lang.String identifier()
identifier
in interface org.apache.beam.sdk.schemas.io.SchemaIOProvider
public org.apache.beam.sdk.schemas.Schema configurationSchema()
BigQueryHelpers.parseTableSpec(java.lang.String)
. Used as an input
to BigQueryIO.TypedRead.from(String)
or BigQueryIO.Write.to(String)
.
BigQueryIO.TypedRead.fromQuery(String)
.
BigQueryIO.TypedRead.withQueryLocation(String)
.
BigQueryIO.Write.CreateDisposition
. Used as an input to BigQueryIO.Write.withCreateDisposition(BigQueryIO.Write.CreateDisposition)
.
BigQueryIO.TypedRead.withMethod(BigQueryIO.TypedRead.Method)
. Defaults to EXPORT, since
that is the only method that currently offers Beam Schema support.
BigQueryIO.Write.withMethod(BigQueryIO.Write.Method)
.
Currently defaults to STORAGE_WRITE_API.
configurationSchema
in interface org.apache.beam.sdk.schemas.io.SchemaIOProvider
public org.apache.beam.sdk.io.gcp.bigquery.BigQuerySchemaIOProvider.BigQuerySchemaIO from(java.lang.String location, org.apache.beam.sdk.values.Row configuration, @Nullable org.apache.beam.sdk.schemas.Schema dataSchema)
For BigQuery IO, only the configuration object is used. Location and data schema have no effect. Specifying a table and dataset is done through appropriate fields in the configuration object, and the data schema is automatically generated from either the input PCollection or schema of the BigQuery table.
from
in interface org.apache.beam.sdk.schemas.io.SchemaIOProvider
public boolean requiresDataSchema()
requiresDataSchema
in interface org.apache.beam.sdk.schemas.io.SchemaIOProvider
public org.apache.beam.sdk.values.PCollection.IsBounded isBounded()
isBounded
in interface org.apache.beam.sdk.schemas.io.SchemaIOProvider