partition

Type Members

1. abstract class PartitionedOutputFormat[P, K, V] extends FileOutputFormat[P, (K, V)]

   This format creates a new record writer for each different path that's generated by the partition function Each record writer defines a specific OutputCommitter that will define a different work directory for a given key.

2. case class PartitionedSink[P, K, V, B](subsink: DataSink[K, V, B], format: Class[_ <: PartitionedOutputFormat[P, K, V]], path: String, partition: (P) ⇒ String, overwrite: Boolean = false, check: (Path, Boolean, ScoobiConfiguration) ⇒ Unit = ..., compression: Option[Compression] = scala.None) extends DataSink[P, (K, V), (P, B)] with Product with Serializable