SCollectionTypedOps

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   Any

2. final def ##(): Int

   

   Definition Classes

   Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   Any

4. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

5. def getClass(): Class[_ <: AnyVal]

   

   Definition Classes

   AnyVal → Any

6. final def isInstanceOf[T0]: Boolean

   

   Definition Classes

   Any

7. def saveAsTypedBigQuery(tableSpec: String, writeDisposition: WriteDisposition = ..., createDisposition: CreateDisposition = ..., timePartitioning: TimePartitioning = ...)(implicit tt: scala.reflect.api.JavaUniverse.TypeTag[T], ct: ClassTag[T], coder: Coder[T]): ClosedTap[T]

   

   Save this SCollection as a BigQuery table.

   Save this SCollection as a BigQuery table. Note that element type T must be annotated with BigQueryType.

   This could be a complete case class with BigQueryType.toTable. For example:

   @BigQueryType.toTable
   case class Result(name: String, score: Double)
   
   val p: SCollection[Result] = // process data and convert elements to Result
   p.saveAsTypedBigQuery("myproject:mydataset.mytable")

   It could also be an empty class with schema from BigQueryType.fromSchema, BigQueryType.fromTable, or BigQueryType.fromQuery. For example:

   @BigQueryType.fromTable("publicdata:samples.gsod")
   class Row
   
   sc.typedBigQuery[Row]()
     .sample(withReplacement = false, fraction = 0.1)
     .saveAsTypedBigQuery("myproject:samples.gsod")

8. def saveAsTypedBigQuery(table: TableReference, writeDisposition: WriteDisposition, createDisposition: CreateDisposition, timePartitioning: TimePartitioning)(implicit tt: scala.reflect.api.JavaUniverse.TypeTag[T], ct: ClassTag[T], coder: Coder[T]): ClosedTap[T]

   

   Save this SCollection as a BigQuery table.

   Save this SCollection as a BigQuery table. Note that element type T must be a case class annotated with BigQueryType.toTable.

9. def toString(): String

   

   Definition Classes

   Any