package io
- Alphabetic
- Public
- All
Type Members
-
final
case class
BinaryIO(path: String) extends ScioIO[Array[Byte]] with Product with Serializable
A ScioIO class for writing raw bytes to files.
A ScioIO class for writing raw bytes to files. Like TextIO, but without newline delimiters and operating over Array[Byte] instead of String.
- path
a path to write to.
- final case class ClosedTap[T] extends Product with Serializable
-
final
case class
CustomIO[T](id: String) extends TestIO[T] with Product with Serializable
Special version of ScioIO for use with ScioContext.customInput and SCollection.saveAsCustomOutput.
- final class EmptyTapOf[A] extends TapT[A]
-
final
case class
ReadIO[T](id: String) extends TestIO[T] with Product with Serializable
Special version of ScioIO for use with SCollection.readAll and SCollection.readAllBytes.
-
trait
ScioIO[T] extends AnyRef
Base trait for all Read/Write IO classes.
Base trait for all Read/Write IO classes. Every IO connector must implement this. This trait has two abstract implicit methods #read, #write that need to be implemented in every subtype. Look at the com.spotify.scio.io.TextIO subclass for a reference implementation. IO connectors can choose to override #readTest and #writeTest if custom test logic is necessary.
-
trait
Tap[T] extends Serializable
Placeholder to an external data set that can either be load into memory as an iterator or opened in a new ScioContext as an SCollection.
-
class
TapNotAvailableException extends Exception
Exception for when a tap is not available.
- final class TapOf[A] extends TapT[A]
- sealed trait TapT[A] extends Serializable
-
trait
Taps extends AnyRef
Utility for managing
Future[Tap[T]]
s. -
trait
TestIO[T] extends ScioIO[T]
Base trait for ScioIO without business logic, for stubbing mock data with
JobTest
. - final case class TextIO(path: String) extends ScioIO[String] with Product with Serializable
-
final
case class
TextTap(path: String, params: ReadParam) extends Tap[String] with Product with Serializable
Tap for text files on local file system or GCS.
- final case class UnsupportedTap[T](msg: String) extends Tap[T] with Product with Serializable
Value Members
- object BinaryIO extends Serializable
- object EmptyTap extends Tap[Nothing] with Product with Serializable
- object EmptyTapOf extends Serializable
- object MaterializeTap extends Serializable
- object ScioIO
- object TapOf extends Serializable
- object TapT extends Serializable
-
object
Taps
Companion object for Taps.
- object TapsSysProps extends SysProps
- object TextIO extends Serializable