com.google.cloud.dataflow.sdk.util

Class BigQueryTableInserter

java.lang.Object

com.google.cloud.dataflow.sdk.util.BigQueryTableInserter

public class BigQueryTableInserter
extends Object

Inserts rows into BigQuery.

Constructor Summary

Constructors

Constructor and Description
BigQueryTableInserter(com.google.api.services.bigquery.Bigquery client, com.google.api.services.bigquery.model.TableReference ref) Constructs a new row inserter.
BigQueryTableInserter(com.google.api.services.bigquery.Bigquery client, com.google.api.services.bigquery.model.TableReference ref, int maxRowsPerBatch) Constructs a new row inserter.

Method Summary

Modifier and Type	Method and Description
com.google.api.services.bigquery.model.Table	getOrCreateTable(BigQueryIO.Write.WriteDisposition writeDisposition, BigQueryIO.Write.CreateDisposition createDisposition, com.google.api.services.bigquery.model.TableSchema schema) Retrieves or creates the table.
void	insertAll(List<com.google.api.services.bigquery.model.TableRow> rowList) Insert all rows from the given list.
void	insertAll(List<com.google.api.services.bigquery.model.TableRow> rowList, List<String> insertIdList) Insert all rows from the given list using specified insertIds if not null.
boolean	isEmpty() Checks if a table is empty.
com.google.api.services.bigquery.model.Table	tryCreateTable(com.google.api.services.bigquery.model.TableSchema schema) Tries to create the BigQuery table.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BigQueryTableInserter

public BigQueryTableInserter(com.google.api.services.bigquery.Bigquery client,
                             com.google.api.services.bigquery.model.TableReference ref)

Constructs a new row inserter.

Parameters:

client - a BigQuery client

ref - identifies the table to insert into

BigQueryTableInserter

public BigQueryTableInserter(com.google.api.services.bigquery.Bigquery client,
                             com.google.api.services.bigquery.model.TableReference ref,
                             int maxRowsPerBatch)

Constructs a new row inserter.

Parameters:

client - a BigQuery client

ref - identifies the table to insert into

Method Detail

insertAll

public void insertAll(List<com.google.api.services.bigquery.model.TableRow> rowList)
               throws IOException

Insert all rows from the given list.

Throws:

IOException

insertAll

public void insertAll(List<com.google.api.services.bigquery.model.TableRow> rowList,
                      @Nullable
                      List<String> insertIdList)
               throws IOException

Insert all rows from the given list using specified insertIds if not null.

Throws:

IOException

getOrCreateTable

public com.google.api.services.bigquery.model.Table getOrCreateTable(BigQueryIO.Write.WriteDisposition writeDisposition,
                                                                     BigQueryIO.Write.CreateDisposition createDisposition,
                                                                     @Nullable
                                                                     com.google.api.services.bigquery.model.TableSchema schema)
                                                              throws IOException

Retrieves or creates the table.

The table is checked to conform to insertion requirements as specified by WriteDisposition and CreateDisposition.

If table truncation is requested (WriteDisposition.WRITE_TRUNCATE), then this will re-create the table if necessary to ensure it is empty.

If an empty table is required (WriteDisposition.WRITE_EMPTY), then this will fail if the table exists and is not empty.

When constructing a table, a TableSchema must be available. If a schema is provided, then it will be used. If no schema is provided, but an existing table is being cleared (WRITE_TRUNCATE option above), then the existing schema will be re-used. If no schema is available, then an IOException is thrown.

Throws:

IOException

isEmpty

public boolean isEmpty()
                throws IOException

Checks if a table is empty.

Throws:

IOException

tryCreateTable

@Nullable
public com.google.api.services.bigquery.model.Table tryCreateTable(com.google.api.services.bigquery.model.TableSchema schema)
                                                                      throws IOException

Tries to create the BigQuery table. If a table with the same name already exists in the dataset, the table creation fails, and the function returns null. In such a case, the existing table doesn't necessarily have the same schema as specified by the parameter.

Parameters:

schema - Schema of the new BigQuery table.

Returns:

The newly created BigQuery table information, or null if the table with the same name already exists.

Throws:

IOException - if other error than already existing table occurs.