public interface BigQuery extends Service<BigQueryOptions>
Modifier and Type | Interface and Description |
---|---|
static class |
BigQuery.DatasetDeleteOption
Class for specifying dataset delete options.
|
static class |
BigQuery.DatasetField
Fields of a BigQuery Dataset resource.
|
static class |
BigQuery.DatasetListOption
Class for specifying dataset list options.
|
static class |
BigQuery.DatasetOption
Class for specifying dataset get, create and update options.
|
static class |
BigQuery.JobField
Fields of a BigQuery Job resource.
|
static class |
BigQuery.JobListOption
Class for specifying job list options.
|
static class |
BigQuery.JobOption
Class for specifying table get and create options.
|
static class |
BigQuery.QueryOption |
static class |
BigQuery.QueryResultsOption
Class for specifying query results options.
|
static class |
BigQuery.TableDataListOption
Class for specifying table data list options.
|
static class |
BigQuery.TableField
Fields of a BigQuery Table resource.
|
static class |
BigQuery.TableListOption
Class for specifying table list options.
|
static class |
BigQuery.TableOption
Class for specifying table get, create and update options.
|
Modifier and Type | Method and Description |
---|---|
boolean |
cancel(JobId jobId)
Sends a job cancel request.
|
boolean |
cancel(String jobId)
Sends a job cancel request.
|
Dataset |
create(DatasetInfo datasetInfo,
BigQuery.DatasetOption... options)
Creates a new dataset.
|
Job |
create(JobInfo jobInfo,
BigQuery.JobOption... options)
Creates a new job.
|
Table |
create(TableInfo tableInfo,
BigQuery.TableOption... options)
Creates a new table.
|
boolean |
delete(DatasetId datasetId,
BigQuery.DatasetDeleteOption... options)
Deletes the requested dataset.
|
boolean |
delete(String datasetId,
BigQuery.DatasetDeleteOption... options)
Deletes the requested dataset.
|
boolean |
delete(String datasetId,
String tableId)
Deletes the requested table.
|
boolean |
delete(TableId tableId)
Deletes the requested table.
|
Dataset |
getDataset(DatasetId datasetId,
BigQuery.DatasetOption... options)
Returns the requested dataset or
null if not found. |
Dataset |
getDataset(String datasetId,
BigQuery.DatasetOption... options)
Returns the requested dataset or
null if not found. |
Job |
getJob(JobId jobId,
BigQuery.JobOption... options)
Returns the requested job or
null if not found. |
Job |
getJob(String jobId,
BigQuery.JobOption... options)
Returns the requested job or
null if not found. |
QueryResponse |
getQueryResults(JobId jobId,
BigQuery.QueryResultsOption... options)
Returns results of the query associated with the provided job.
|
Table |
getTable(String datasetId,
String tableId,
BigQuery.TableOption... options)
Returns the requested table or
null if not found. |
Table |
getTable(TableId tableId,
BigQuery.TableOption... options)
Returns the requested table or
null if not found. |
InsertAllResponse |
insertAll(InsertAllRequest request)
Sends an insert all request.
|
com.google.api.gax.paging.Page<Dataset> |
listDatasets(BigQuery.DatasetListOption... options)
Lists the project's datasets.
|
com.google.api.gax.paging.Page<Dataset> |
listDatasets(String projectId,
BigQuery.DatasetListOption... options)
Lists the datasets in the provided project.
|
com.google.api.gax.paging.Page<Job> |
listJobs(BigQuery.JobListOption... options)
Lists the jobs.
|
com.google.api.gax.paging.Page<FieldValueList> |
listTableData(String datasetId,
String tableId,
BigQuery.TableDataListOption... options)
Lists the table's rows.
|
com.google.api.gax.paging.Page<FieldValueList> |
listTableData(TableId tableId,
BigQuery.TableDataListOption... options)
Lists the table's rows.
|
com.google.api.gax.paging.Page<Table> |
listTables(DatasetId datasetId,
BigQuery.TableListOption... options)
Lists the tables in the dataset.
|
com.google.api.gax.paging.Page<Table> |
listTables(String datasetId,
BigQuery.TableListOption... options)
Lists the tables in the dataset.
|
QueryResponse |
query(QueryJobConfiguration configuration,
BigQuery.QueryOption... options)
Runs the query associated with the request, using an internally-generated random JobId.
|
QueryResponse |
query(QueryJobConfiguration configuration,
JobId jobId,
BigQuery.QueryOption... options)
Runs the query associated with the request, using the given job id.
|
Dataset |
update(DatasetInfo datasetInfo,
BigQuery.DatasetOption... options)
Updates dataset information.
|
Table |
update(TableInfo tableInfo,
BigQuery.TableOption... options)
Updates table information.
|
TableDataWriteChannel |
writer(WriteChannelConfiguration writeChannelConfiguration)
Returns a channel to write data to be inserted into a BigQuery table.
|
getOptions
Dataset create(DatasetInfo datasetInfo, BigQuery.DatasetOption... options)
Example of creating a dataset.
String datasetName = "my_dataset_name";
Dataset dataset = null;
DatasetInfo datasetInfo = DatasetInfo.newBuilder(datasetName).build();
try {
// the dataset was created
dataset = bigquery.create(datasetInfo);
} catch (BigQueryException e) {
// the dataset was not created
}
BigQueryException
- upon failureTable create(TableInfo tableInfo, BigQuery.TableOption... options)
Example of creating a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
String fieldName = "string_field";
TableId tableId = TableId.of(datasetName, tableName);
// Table field definition
Field field = Field.of(fieldName, Field.Type.string());
// Table schema definition
Schema schema = Schema.of(field);
TableDefinition tableDefinition = StandardTableDefinition.of(schema);
TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
Table table = bigquery.create(tableInfo);
BigQueryException
- upon failureJob create(JobInfo jobInfo, BigQuery.JobOption... options)
Example of creating a query job.
String query = "SELECT field FROM my_dataset_name.my_table_name";
Job job = null;
JobConfiguration jobConfiguration = QueryJobConfiguration.of(query);
JobInfo jobInfo = JobInfo.of(jobConfiguration);
try {
job = bigquery.create(jobInfo);
} catch (BigQueryException e) {
// the job was not created
}
BigQueryException
- upon failureDataset getDataset(String datasetId, BigQuery.DatasetOption... options)
null
if not found.
Example of getting a dataset.
String datasetName = "my_dataset";
Dataset dataset = bigquery.getDataset(datasetName);
BigQueryException
- upon failureDataset getDataset(DatasetId datasetId, BigQuery.DatasetOption... options)
null
if not found.
Example of getting a dataset.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
DatasetId datasetId = DatasetId.of(projectId, datasetName);
Dataset dataset = bigquery.getDataset(datasetId);
BigQueryException
- upon failurecom.google.api.gax.paging.Page<Dataset> listDatasets(BigQuery.DatasetListOption... options)
DatasetInfo.getDatasetId()
, DatasetInfo.getFriendlyName()
and
DatasetInfo.getGeneratedId()
). To get complete information use either
getDataset(String, DatasetOption...)
or
getDataset(DatasetId, DatasetOption...)
.
Example of listing datasets, specifying the page size.
Page<Dataset> datasets = bigquery.listDatasets(DatasetListOption.pageSize(100));
for (Dataset dataset : datasets.iterateAll()) {
// do something with the dataset
}
BigQueryException
- upon failurecom.google.api.gax.paging.Page<Dataset> listDatasets(String projectId, BigQuery.DatasetListOption... options)
DatasetInfo.getDatasetId()
, DatasetInfo.getFriendlyName()
and
DatasetInfo.getGeneratedId()
). To get complete information use either
getDataset(String, DatasetOption...)
or
getDataset(DatasetId, DatasetOption...)
.
Example of listing datasets in a project, specifying the page size.
String projectId = "my_project_id";
Page<Dataset> datasets = bigquery.listDatasets(projectId, DatasetListOption.pageSize(100));
for (Dataset dataset : datasets.iterateAll()) {
// do something with the dataset
}
BigQueryException
- upon failureboolean delete(String datasetId, BigQuery.DatasetDeleteOption... options)
Example of deleting a dataset from its id, even if non-empty.
String datasetName = "my_dataset_name";
Boolean deleted = bigquery.delete(datasetName, DatasetDeleteOption.deleteContents());
if (deleted) {
// the dataset was deleted
} else {
// the dataset was not found
}
true
if dataset was deleted, false
if it was not foundBigQueryException
- upon failureboolean delete(DatasetId datasetId, BigQuery.DatasetDeleteOption... options)
Example of deleting a dataset, even if non-empty.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
DatasetId datasetId = DatasetId.of(projectId, datasetName);
Boolean deleted = bigquery.delete(datasetId, DatasetDeleteOption.deleteContents());
if (deleted) {
// the dataset was deleted
} else {
// the dataset was not found
}
true
if dataset was deleted, false
if it was not foundBigQueryException
- upon failureboolean delete(String datasetId, String tableId)
Example of deleting a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
Boolean deleted = bigquery.delete(datasetName, tableName);
if (deleted) {
// the table was deleted
} else {
// the table was not found
}
true
if table was deleted, false
if it was not foundBigQueryException
- upon failureboolean delete(TableId tableId)
Example of deleting a table.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
TableId tableId = TableId.of(projectId, datasetName, tableName);
Boolean deleted = bigquery.delete(tableId);
if (deleted) {
// the table was deleted
} else {
// the table was not found
}
true
if table was deleted, false
if it was not foundBigQueryException
- upon failureDataset update(DatasetInfo datasetInfo, BigQuery.DatasetOption... options)
Example of updating a dataset by changing its friendly name.
String datasetName = "my_dataset_name";
String newFriendlyName = "some_new_friendly_name";
Dataset oldDataset = bigquery.getDataset(datasetName);
DatasetInfo datasetInfo = oldDataset.toBuilder().setFriendlyName(newFriendlyName).build();
Dataset newDataset = bigquery.update(datasetInfo);
BigQueryException
- upon failureTable update(TableInfo tableInfo, BigQuery.TableOption... options)
Example of updating a table by changing its friendly name.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
String newFriendlyName = "new_friendly_name";
Table oldTable = bigquery.getTable(datasetName, tableName);
TableInfo tableInfo = oldTable.toBuilder().setFriendlyName(newFriendlyName).build();
Table newTable = bigquery.update(tableInfo);
BigQueryException
- upon failureTable getTable(String datasetId, String tableId, BigQuery.TableOption... options)
null
if not found.
Example of getting a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
Table table = bigquery.getTable(datasetName, tableName);
BigQueryException
- upon failureTable getTable(TableId tableId, BigQuery.TableOption... options)
null
if not found.
Example of getting a table.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
TableId tableId = TableId.of(projectId, datasetName, tableName);
Table table = bigquery.getTable(tableId);
BigQueryException
- upon failurecom.google.api.gax.paging.Page<Table> listTables(String datasetId, BigQuery.TableListOption... options)
TableInfo.getTableId()
, TableInfo.getFriendlyName()
, TableInfo.getGeneratedId()
and type,
which is part of TableInfo.getDefinition()
). To get complete information use either
getTable(TableId, TableOption...)
or
getTable(String, String, TableOption...)
.
Example of listing the tables in a dataset, specifying the page size.
String datasetName = "my_dataset_name";
Page<Table> tables = bigquery.listTables(datasetName, TableListOption.pageSize(100));
for (Table table : tables.iterateAll()) {
// do something with the table
}
BigQueryException
- upon failurecom.google.api.gax.paging.Page<Table> listTables(DatasetId datasetId, BigQuery.TableListOption... options)
TableInfo.getTableId()
, TableInfo.getFriendlyName()
, TableInfo.getGeneratedId()
and type, which is part of TableInfo.getDefinition()
). To get complete information use either
getTable(TableId, TableOption...)
or
getTable(String, String, TableOption...)
.
Example of listing the tables in a dataset.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
DatasetId datasetId = DatasetId.of(projectId, datasetName);
Page<Table> tables = bigquery.listTables(datasetId, TableListOption.pageSize(100));
for (Table table : tables.iterateAll()) {
// do something with the table
}
BigQueryException
- upon failureInsertAllResponse insertAll(InsertAllRequest request)
Example of inserting rows into a table without running a load job.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
TableId tableId = TableId.of(datasetName, tableName);
// Values of the row to insert
Map<String, Object> rowContent = new HashMap<>();
rowContent.put("booleanField", true);
// Bytes are passed in base64
rowContent.put("bytesField", "Cg0NDg0="); // 0xA, 0xD, 0xD, 0xE, 0xD in base64
// Records are passed as a map
Map<String, Object> recordsContent = new HashMap<>();
recordsContent.put("stringField", "Hello, World!");
rowContent.put("recordField", recordsContent);
InsertAllResponse response = bigquery.insertAll(InsertAllRequest.newBuilder(tableId)
.addRow("rowId", rowContent)
// More rows can be added in the same RPC by invoking .addRow() on the builder
.build());
if (response.hasErrors()) {
// If any of the insertions failed, this lets you inspect the errors
for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
// inspect row error
}
}
BigQueryException
- upon failurecom.google.api.gax.paging.Page<FieldValueList> listTableData(String datasetId, String tableId, BigQuery.TableDataListOption... options)
Example of listing table rows, specifying the page size.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
Page<FieldValueList> tableData =
bigquery.listTableData(datasetName, tableName, TableDataListOption.pageSize(100));
for (FieldValueList row : tableData.iterateAll()) {
// do something with the row
}
BigQueryException
- upon failurecom.google.api.gax.paging.Page<FieldValueList> listTableData(TableId tableId, BigQuery.TableDataListOption... options)
Example of listing table rows, specifying the page size.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
TableId tableIdObject = TableId.of(datasetName, tableName);
Page<FieldValueList> tableData =
bigquery.listTableData(tableIdObject, TableDataListOption.pageSize(100));
for (FieldValueList row : rowIterator.hasNext()) {
// do something with the row
}
BigQueryException
- upon failureJob getJob(String jobId, BigQuery.JobOption... options)
null
if not found.
Example of getting a job.
String jobName = "my_job_name";
Job job = bigquery.getJob(jobName);
if (job == null) {
// job was not found
}
BigQueryException
- upon failureJob getJob(JobId jobId, BigQuery.JobOption... options)
null
if not found.
Example of getting a job.
String jobName = "my_job_name";
JobId jobIdObject = JobId.of(jobName);
Job job = bigquery.getJob(jobIdObject);
if (job == null) {
// job was not found
}
BigQueryException
- upon failurecom.google.api.gax.paging.Page<Job> listJobs(BigQuery.JobListOption... options)
Example of listing jobs, specifying the page size.
Page<Job> jobs = bigquery.listJobs(JobListOption.pageSize(100));
for (Job job : jobs.iterateAll()) {
// do something with the job
}
BigQueryException
- upon failureboolean cancel(String jobId)
getJob(JobId, JobOption...)
or
getJob(String, JobOption...)
).
Example of cancelling a job.
String jobName = "my_job_name";
boolean success = bigquery.cancel(jobName);
if (success) {
// job was cancelled
} else {
// job was not found
}
true
if cancel was requested successfully, false
if the job was not
foundBigQueryException
- upon failureboolean cancel(JobId jobId)
getJob(JobId, JobOption...)
or
getJob(String, JobOption...)
).
Example of cancelling a job.
String jobName = "my_job_name";
JobId jobId = JobId.of(jobName);
boolean success = bigquery.cancel(jobId);
if (success) {
// job was cancelled
} else {
// job was not found
}
true
if cancel was requested successfully, false
if the job was not
foundBigQueryException
- upon failureQueryResponse query(QueryJobConfiguration configuration, BigQuery.QueryOption... options) throws InterruptedException
Example of running a query.
String query = "SELECT distinct(corpus) FROM `bigquery-public-data.samples.shakespeare`";
QueryJobConfiguration queryConfig = QueryJobConfiguration.of(query);
// To run the legacy syntax queries use the following code instead:
// String query = "SELECT unique(corpus) FROM [bigquery-public-data:samples.shakespeare]"
// QueryJobConfiguration queryConfig =
// QueryJobConfiguration.newBuilder(query).setUseLegacySql(true).build();
QueryResponse response = bigquery.query(queryConfig);
if (response.hasErrors()) {
// handle errors
}
QueryResult result = response.getResult();
for (FieldValueList row : result.iterateAll()) {
// do something with the data
}
Example of running a query with query parameters.
String query =
"SELECT distinct(corpus) FROM `bigquery-public-data.samples.shakespeare` where word_count > ?";
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query)
.addPositionalParameter(QueryParameterValue.int64(5))
.build();
QueryResponse response = bigquery.query(queryConfig);
if (response.hasErrors()) {
// handle errors
}
QueryResult result = response.getResult();
for (FieldValueList row : result.iterateAll()) {
// do something with the data
}
BigQueryException
- upon failureInterruptedException
- if the current thread gets interrupted while waiting for the query
to completeQueryResponse query(QueryJobConfiguration configuration, JobId jobId, BigQuery.QueryOption... options) throws InterruptedException
See query(QueryJobConfiguration, QueryOption...)
for examples on populating a
QueryJobConfiguration
.
The recommended way to create a randomly generated JobId is the following:
JobId jobId = JobId.of();
For a user specified job id with an optional prefix use the following:
JobId jobId = JobId.of("my_prefix-my_unique_job_id");
BigQueryException
- upon failureInterruptedException
- if the current thread gets interrupted while waiting for the query
to completeQueryResponse getQueryResults(JobId jobId, BigQuery.QueryResultsOption... options)
Example of getting the results of query.
String query = "SELECT distinct(corpus) FROM `bigquery-public-data.samples.shakespeare`";
QueryJobConfiguration queryConfig = QueryJobConfiguration.of(query);
QueryResponse response = bigquery.query(queryConfig);
// Wait for things to finish
while (!response.jobCompleted()) {
Thread.sleep(1000);
response = bigquery.getQueryResults(response.getJobId());
}
if (response.hasErrors()) {
// handle errors
}
QueryResult result = response.getResult();
Iterator<FieldValueList> rowIterator = result.iterateAll();
for (FieldValueList row : result.iterateAll()) {
// do something with the data
}
BigQueryException
- upon failureTableDataWriteChannel writer(WriteChannelConfiguration writeChannelConfiguration)
WriteChannelConfiguration
parameter.
Example of creating a channel with which to write to a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
String csvData = "StringValue1\nStringValue2\n";
TableId tableId = TableId.of(datasetName, tableName);
WriteChannelConfiguration writeChannelConfiguration =
WriteChannelConfiguration.newBuilder(tableId)
.setFormatOptions(FormatOptions.csv())
.build();
TableDataWriteChannel writer = bigquery.writer(writeChannelConfiguration);
// Write data to writer
try {
writer.write(ByteBuffer.wrap(csvData.getBytes(Charsets.UTF_8)));
} finally {
writer.close();
}
// Get load job
Job job = writer.getJob();
job = job.waitFor();
LoadStatistics stats = job.getStatistics();
return stats.getOutputRows();
Example of writing a local file to a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
Path csvPath = FileSystems.getDefault().getPath(".", "my-data.csv");
TableId tableId = TableId.of(datasetName, tableName);
WriteChannelConfiguration writeChannelConfiguration =
WriteChannelConfiguration.newBuilder(tableId)
.setFormatOptions(FormatOptions.csv())
.build();
TableDataWriteChannel writer = bigquery.writer(writeChannelConfiguration);
// Write data to writer
try (OutputStream stream = Channels.newOutputStream(writer)) {
Files.copy(csvPath, stream);
}
// Get load job
Job job = writer.getJob();
job = job.waitFor();
LoadStatistics stats = job.getStatistics();
return stats.getOutputRows();
BigQueryException
- upon failureCopyright © 2017 Google. All rights reserved.