ScioContextOps

Instance Constructors

new ScioContextOps(self: ScioContext)

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
Any
final def ##(): Int

Definition Classes
Any
final def ==(arg0: Any): Boolean

Definition Classes
Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def bigQuerySelect(sqlQuery: String, flattenResults: Boolean = ...): SCollection[TableRow]

Get an SCollection for a BigQuery SELECT query.
Get an SCollection for a BigQuery SELECT query. Both Legacy SQL and Standard SQL dialects are supported. By default the query dialect will be automatically detected. To override this behavior, start the query string with #legacysql or #standardsql.
def bigQueryStorage(table: Table, selectedFields: List[String] = Nil, rowRestriction: String = null): SCollection[TableRow]

Get an SCollection for a BigQuery table using the storage API.
Get an SCollection for a BigQuery table using the storage API.
selectedFields
names of the fields in the table that should be read. If empty, all fields will be read. If the specified field is a nested field, all the sub-fields in the field will be selected.
rowRestriction
SQL text filtering statement, similar ti a WHERE clause in a query. Currently, we support combinations of predicates that are a comparison between a column and a constant value in SQL statement. Aggregates are not supported. For example:
```
"a > DATE '2014-09-27' AND (b > 5 AND c LIKE 'date')"
```
def bigQueryTable(table: Table): SCollection[TableRow]

Get an SCollection for a BigQuery table.
def getClass(): Class[_ <: AnyVal]

Definition Classes
AnyVal → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def tableRowJsonFile(path: String): SCollection[TableRow]

Get an SCollection for a BigQuery TableRow JSON file.
def toString(): String

Definition Classes
Any
def typedBigQuery[T <: HasAnnotation](newSource: String = null)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]

Get a typed SCollection for a BigQuery SELECT query, table or storage.
Get a typed SCollection for a BigQuery SELECT query, table or storage.
Note that T must be annotated with BigQueryType.fromSchema, BigQueryType.fromStorage, BigQueryType.fromTable, BigQueryType.fromQuery, or BigQueryType.toTable.
By default the source (table or query) specified in the annotation will be used, but it can be overridden with the newSource parameter. For example:
```
@BigQueryType.fromTable("publicdata:samples.gsod")
class Row

// Read from [publicdata:samples.gsod] as specified in the annotation.
sc.typedBigQuery[Row]()

// Read from [myproject:samples.gsod] instead.
sc.typedBigQuery[Row]("myproject:samples.gsod")

// Read from a query instead.
sc.typedBigQuery[Row]("SELECT * FROM [publicdata:samples.gsod] LIMIT 1000")
```
Both Legacy SQL and Standard SQL dialects are supported. By default the query dialect will be automatically detected. To override this behavior, start the query string with #legacysql or #standardsql.
def typedBigQueryStorage[T <: HasAnnotation](newSource: String = null, rowRestriction: String = null)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]

Get a typed SCollection for a BigQuery storage API.
Get a typed SCollection for a BigQuery storage API.
Note that T must be annotated with BigQueryType.fromStorage.
Similar to typedBigQuery but allows rowRestriction to be overridden.

Deprecated Value Members

def bigQueryTable(tableSpec: String): SCollection[TableRow]

Get an SCollection for a BigQuery table.
Get an SCollection for a BigQuery table.

Annotations
@deprecated
Deprecated
(Since version Scio 0.8) this method will be removed; use bigQueryTable(Table.Spec(table)) instead
def bigQueryTable(table: TableReference): SCollection[TableRow]

Get an SCollection for a BigQuery table.
Get an SCollection for a BigQuery table.

Annotations
@deprecated
Deprecated
(Since version Scio 0.8) this method will be removed; use bigQueryTable(Table.Ref(table)) instead

Related Doc: package syntax

final class ScioContextOps extends AnyVal

Instance Constructors

new ScioContextOps(self: ScioContext)

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def bigQuerySelect(sqlQuery: String, flattenResults: Boolean = ...): SCollection[TableRow]

def bigQueryStorage(table: Table, selectedFields: List[String] = Nil, rowRestriction: String = null): SCollection[TableRow]

def bigQueryTable(table: Table): SCollection[TableRow]

def getClass(): Class[_ <: AnyVal]

final def isInstanceOf[T0]: Boolean

def tableRowJsonFile(path: String): SCollection[TableRow]

def toString(): String

def typedBigQuery[T <: HasAnnotation](newSource: String = null)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]

def typedBigQueryStorage[T <: HasAnnotation](newSource: String = null, rowRestriction: String = null)(implicit arg0: ClassTag[T], arg1: scala.reflect.api.JavaUniverse.TypeTag[T], arg2: Coder[T]): SCollection[T]

Deprecated Value Members

def bigQueryTable(tableSpec: String): SCollection[TableRow]

def bigQueryTable(table: TableReference): SCollection[TableRow]

Inherited from AnyVal

Inherited from Any

Ungrouped