Get an SCollection for a BigQuery SELECT query.
Get an SCollection for a BigQuery SELECT query.
Both Legacy SQL and
Standard SQL dialects are
supported. By default the query dialect will be automatically detected. To override this
behavior, start the query string with #legacysql
or #standardsql
.
Get an SCollection for a BigQuery table.
Get an SCollection for a BigQuery table.
Get an SCollection for a BigQuery TableRow JSON file.
Get a typed SCollection for a BigQuery SELECT query or table.
Get a typed SCollection for a BigQuery SELECT query or table.
Note that T
must be annotated with
BigQueryType.fromSchema,
BigQueryType.fromTable,
BigQueryType.fromQuery, or
BigQueryType.toTable.
By default the source (table or query) specified in the annotation will be used, but it can
be overridden with the newSource
parameter. For example:
@BigQueryType.fromTable("publicdata:samples.gsod") class Row // Read from [publicdata:samples.gsod] as specified in the annotation. sc.typedBigQuery[Row]() // Read from [myproject:samples.gsod] instead. sc.typedBigQuery[Row]("myproject:samples.gsod") // Read from a query instead. sc.typedBigQuery[Row]("SELECT * FROM [publicdata:samples.gsod] LIMIT 1000")
Both Legacy SQL and
Standard SQL dialects are
supported. By default the query dialect will be automatically detected. To override this
behavior, start the query string with #legacysql
or #standardsql
.
Enhanced version of ScioContext with BigQuery methods.