com.google.cloud.dataflow.sdk.runners.worker

Class BigQueryReader

java.lang.Object

java.util.Observable

com.google.cloud.dataflow.sdk.util.common.worker.Reader<WindowedValue<com.google.api.services.bigquery.model.TableRow>>

com.google.cloud.dataflow.sdk.runners.worker.BigQueryReader

public class BigQueryReader
extends Reader<WindowedValue<com.google.api.services.bigquery.model.TableRow>>

A source that reads a BigQuery table and yields TableRow objects.

The source is a wrapper over the BigQueryTableRowIterator class, which issues a query for all rows of a table and then iterates over the result. There is no support for progress reporting because the source is used only in situations where the entire table must be read by each worker (i.e. the source is used as a side input).

Nested Class Summary

Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Reader

Reader.AbstractReaderIterator<T>, Reader.DynamicSplitRequest, Reader.DynamicSplitResult, Reader.DynamicSplitResultWithPosition, Reader.Position, Reader.Progress, Reader.ReaderIterator<T>

Field Summary

Fields inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Reader

stateSampler, stateSamplerOperationName

Constructor Summary

Constructors

Constructor and Description
BigQueryReader(BigQueryOptions bigQueryOptions, String query, String projectId)
BigQueryReader(BigQueryOptions bigQueryOptions, com.google.api.services.bigquery.model.TableReference tableRef) Builds a BigQuery source using pipeline options to instantiate a Bigquery client.
BigQueryReader(com.google.api.services.bigquery.Bigquery bigQueryClient, String query, String projectId)
BigQueryReader(com.google.api.services.bigquery.Bigquery bigQueryClient, com.google.api.services.bigquery.model.TableReference tableRef) Builds a BigQueryReader directly using a BigQuery client.

Method Summary

Modifier and Type	Method and Description
Reader.ReaderIterator<WindowedValue<com.google.api.services.bigquery.model.TableRow>>	iterator() Returns a ReaderIterator that allows reading from this source.

Methods inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Reader

notifyElementRead, setStateSamplerAndOperationName, supportsRestart

Methods inherited from class java.util.Observable

addObserver, clearChanged, countObservers, deleteObserver, deleteObservers, hasChanged, notifyObservers, notifyObservers, setChanged

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BigQueryReader

public BigQueryReader(BigQueryOptions bigQueryOptions,
                      com.google.api.services.bigquery.model.TableReference tableRef)

Builds a BigQuery source using pipeline options to instantiate a Bigquery client.

BigQueryReader

public BigQueryReader(BigQueryOptions bigQueryOptions,
                      String query,
                      String projectId)

BigQueryReader

public BigQueryReader(com.google.api.services.bigquery.Bigquery bigQueryClient,
                      com.google.api.services.bigquery.model.TableReference tableRef)

Builds a BigQueryReader directly using a BigQuery client.

BigQueryReader

public BigQueryReader(com.google.api.services.bigquery.Bigquery bigQueryClient,
                      String query,
                      String projectId)

Method Detail

iterator

public Reader.ReaderIterator<WindowedValue<com.google.api.services.bigquery.model.TableRow>> iterator()
                                                                                               throws IOException

Description copied from class: Reader

Returns a ReaderIterator that allows reading from this source.

Specified by:

iterator in class Reader<WindowedValue<com.google.api.services.bigquery.model.TableRow>>

Throws:

IOException