hdfs

Main package for HDFS APIs. Import all.

import com.spotify.scio.hdfs._

Note that Hadoop configuration files, e.g. core-site.xml, hdfs-site.xml, must be present. They can be packaged in src/main/resources directory.

Linear Supertypes

AnyRef, Any

case class HdfsAvroTap[T](path: String, schema: Schema = null)(implicit evidence$4: ClassTag[T]) extends Tap[T] with Product with Serializable

Tap for Avro files on HDFS.
implicit class HdfsSCollection[T] extends AnyRef

Enhanced version of SCollection with HDFS methods.
implicit class HdfsScioContext extends AnyRef

Enhanced version of ScioContext with HDFS methods.
implicit class HdfsTaps extends AnyRef

Enhanced version of com.spotify.scio.io.Taps with HDFS methods.
case class HdfsTextTap(path: String) extends Tap[String] with Product with Serializable

Tap for text files on HDFS.