hbase

Type Members

class Admin extends AnyRef

Definition Classes
HBaseUtils
trait DefaultReads extends AnyRef
trait DefaultWrites extends AnyRef
trait Formats[A] extends Reads[A] with Writes[A]
class HBaseConfig extends Serializable

Wrapper for HBaseConfiguration
sealed abstract class HBaseDeleteHelpers extends AnyRef
final class HBaseDeleteRDD[K, Q] extends HBaseDeleteHelpers with Serializable
final class HBaseDeleteRDDKey[K] extends HBaseDeleteHelpers with Serializable
final class HBaseDeleteRDDSimple[K, Q] extends HBaseDeleteHelpers with Serializable
trait HBaseDeleteSupport extends AnyRef

Adds implicit methods to RDD[K], RDD[(K, Set[Q])], RDD[(K, Set[(Q, Long)])], RDD[(K, Map[String, Set[Q]])], RDD[(K, Map[String, Set[(Q, Long)]])], to delete HBase tables.
trait HBaseReadSupport extends AnyRef

Adds implicit methods to SparkContext to read from HBase sources.
final class HBaseSC extends Serializable
trait HBaseUtils extends AnyRef

Utilities for dealing with HBase tables
sealed abstract class HBaseWriteHelpers extends AnyRef
final class HBaseWriteRDD[K, Q, V] extends HBaseWriteHelpers with Serializable
final class HBaseWriteRDDFixed[K, V] extends HBaseWriteHelpers with Serializable
final class HBaseWriteRDDSimple[K, Q, V] extends HBaseWriteHelpers with Serializable
trait HBaseWriteSupport extends AnyRef

Adds implicit methods to RDD[(K, Map[Q, V])], RDD[(K, Seq[V])], RDD[(K, Map[String, Map[Q, V]])] RDD[(K, Map[Q, (V, Long)])], RDD[(K, Seq[(V, Long)])], RDD[(K, Map[String, Map[Q, (V, Long)]])] to write to HBase tables.
final class HFileRDD[K, Q, C, A, V] extends HFileRDDHelper
final class HFileRDDFixed[K, C, A, V] extends HFileRDDHelper
sealed abstract class HFileRDDHelper extends Serializable
final class HFileRDDSimple[K, Q, C, A, V] extends HFileRDDHelper
trait HFileSupport extends AnyRef
trait Reads[A] extends Serializable

An instance of Reads[A] testifies that Array[Byte] can be converted to A.
trait Writes[A] extends Serializable

An instance of Writes[A] testifies that A can be converted to Array[Byte].

Value Members

object Admin

Definition Classes
HBaseUtils
object HBaseConfig extends Serializable

Factories to generate HBaseConfig instances.
Factories to generate HBaseConfig instances.
We can generate an HBaseConfig - from an existing HBaseConfiguration - from a sequence of String key/value pairs - from an existing object with a rootdir and quorum members
The two latter cases are provided for simplicity (ideally a client should not have to deal with the native HBase API).
The last constructor contains the minimum information to be able to read and write to the HBase cluster. It can be used in tandem with a case class containing job configuration.
implicit val booleanReader: Reads[Boolean]

Definition Classes
DefaultReads
implicit val booleanWriter: Writes[Boolean]

Definition Classes
DefaultWrites
implicit val byteArrayReader: Reads[Array[Byte]]

Definition Classes
DefaultReads
implicit val byteArrayWriter: Writes[Array[Byte]]

Definition Classes
DefaultWrites
implicit lazy val cellKeyOrdering: CellKeyOrdering

Definition Classes
HFileSupport
implicit lazy val cellKeyTSOrdering: CellKeyTSOrdering

Definition Classes
HFileSupport
def computeSplits[K](rdd: RDD[K], regionsCount: Int)(implicit arg0: ClassTag[K], ord: Ordering[K]): Seq[K]

Given a RDD of keys and the number of requested table's regions, returns an array of keys that are start keys of the table's regions.
Given a RDD of keys and the number of requested table's regions, returns an array of keys that are start keys of the table's regions. The array length is regionsCount-1 since the start key of the first region is not needed (since it does not determine a split)
rdd
RDD of strings representing row keys
regionsCount
number of regions
returns
a sorted sequence of start keys

Definition Classes
HBaseUtils
def createJob(table: String, conf: Configuration): Job

Attributes
protected
Definition Classes
HBaseUtils
implicit def deleteHBaseRDD[K, Q](rdd: RDD[(K, Map[String, Set[Q]])])(implicit arg0: Writes[K], arg1: Writes[Q]): HBaseDeleteRDD[K, Q]

Definition Classes
HBaseDeleteSupport
implicit def deleteHBaseRDDKey[K](rdd: RDD[K])(implicit arg0: Writes[K]): HBaseDeleteRDDKey[K]

Definition Classes
HBaseDeleteSupport
implicit def deleteHBaseRDDSimple[K, Q](rdd: RDD[(K, Set[Q])])(implicit arg0: Writes[K], arg1: Writes[Q]): HBaseDeleteRDDSimple[K, Q]

Definition Classes
HBaseDeleteSupport
implicit def deleteHBaseRDDSimpleT[K, Q](rdd: RDD[(K, Set[(Q, Long)])])(implicit arg0: Writes[K], arg1: Writes[Q]): HBaseDeleteRDDSimple[K, (Q, Long)]

Definition Classes
HBaseDeleteSupport
implicit def deleteHBaseRDDT[K, Q](rdd: RDD[(K, Map[String, Set[(Q, Long)]])])(implicit arg0: Writes[K], arg1: Writes[Q]): HBaseDeleteRDD[K, (Q, Long)]

Definition Classes
HBaseDeleteSupport
implicit val doubleReader: Reads[Double]

Definition Classes
DefaultReads
implicit val doubleWriter: Writes[Double]

Definition Classes
DefaultWrites
implicit val floatReader: Reads[Float]

Definition Classes
DefaultReads
implicit val floatWriter: Writes[Float]

Definition Classes
DefaultWrites
implicit val intReader: Reads[Int]

Definition Classes
DefaultReads
implicit val intWriter: Writes[Int]

Definition Classes
DefaultWrites
implicit val jsonReader: Reads[JValue]

Definition Classes
DefaultReads
implicit val jsonWriter: Writes[JValue]

Definition Classes
DefaultWrites
implicit val longReader: Reads[Long]

Definition Classes
DefaultReads
implicit val longWriter: Writes[Long]

Definition Classes
DefaultWrites
implicit val shortReader: Reads[Short]

Definition Classes
DefaultReads
implicit val shortWriter: Writes[Short]

Definition Classes
DefaultWrites
implicit val stringReader: Reads[String]

Definition Classes
DefaultReads
implicit val stringWriter: Writes[String]

Definition Classes
DefaultWrites
implicit def toHBaseRDD[K, Q, V](rdd: RDD[(K, Map[String, Map[Q, V]])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: Writes[V]): HBaseWriteRDD[K, Q, V]

Definition Classes
HBaseWriteSupport
implicit def toHBaseRDDFixed[K, V](rdd: RDD[(K, Seq[V])])(implicit arg0: Writes[K], arg1: Writes[V]): HBaseWriteRDDFixed[K, V]

Definition Classes
HBaseWriteSupport
implicit def toHBaseRDDFixedTS[K, V](rdd: RDD[(K, Seq[(V, Long)])])(implicit arg0: Writes[K], arg1: Writes[V]): HBaseWriteRDDFixed[K, (V, Long)]

Definition Classes
HBaseWriteSupport
implicit def toHBaseRDDSimple[K, Q, V](rdd: RDD[(K, Map[Q, V])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: Writes[V]): HBaseWriteRDDSimple[K, Q, V]

Definition Classes
HBaseWriteSupport
implicit def toHBaseRDDSimpleTS[K, Q, V](rdd: RDD[(K, Map[Q, (V, Long)])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: Writes[V]): HBaseWriteRDDSimple[K, Q, (V, Long)]

Definition Classes
HBaseWriteSupport
implicit def toHBaseRDDT[K, Q, V](rdd: RDD[(K, Map[String, Map[Q, (V, Long)]])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: Writes[V]): HBaseWriteRDD[K, Q, (V, Long)]

Definition Classes
HBaseWriteSupport
implicit def toHBaseSC(sc: SparkContext): HBaseSC

Definition Classes
HBaseReadSupport
implicit def toHFileRDD[K, Q, A](rdd: RDD[(K, Map[String, Map[Q, A]])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: ClassTag[A], writer: Writes[A]): HFileRDD[K, Q, CellKey, A, A]

Definition Classes
HFileSupport
implicit def toHFileRDDFixed[K, A](rdd: RDD[(K, Seq[A])])(implicit arg0: Writes[K], arg1: ClassTag[A], writer: Writes[A]): HFileRDDFixed[K, CellKey, A, A]

Definition Classes
HFileSupport
implicit def toHFileRDDFixedTS[K, A](rdd: RDD[(K, Seq[(A, Long)])])(implicit arg0: Writes[K], arg1: ClassTag[A], writer: Writes[A]): HFileRDDFixed[K, CellKeyTS, (A, Long), A]

Definition Classes
HFileSupport
implicit def toHFileRDDSimple[K, Q, A](rdd: RDD[(K, Map[Q, A])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: ClassTag[A], writer: Writes[A]): HFileRDDSimple[K, Q, CellKey, A, A]

Definition Classes
HFileSupport
implicit def toHFileRDDSimpleTS[K, Q, A](rdd: RDD[(K, Map[Q, (A, Long)])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: ClassTag[A], writer: Writes[A]): HFileRDDSimple[K, Q, CellKeyTS, (A, Long), A]

Definition Classes
HFileSupport
implicit def toHFileRDDTS[K, Q, A](rdd: RDD[(K, Map[String, Map[Q, (A, Long)]])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: ClassTag[A], writer: Writes[A]): HFileRDD[K, Q, CellKeyTS, (A, Long), A]

Definition Classes
HFileSupport

package hbase

Type Members

class Admin extends AnyRef

trait DefaultReads extends AnyRef

trait DefaultWrites extends AnyRef

trait Formats[A] extends Reads[A] with Writes[A]

class HBaseConfig extends Serializable

sealed abstract class HBaseDeleteHelpers extends AnyRef

final class HBaseDeleteRDD[K, Q] extends HBaseDeleteHelpers with Serializable

final class HBaseDeleteRDDKey[K] extends HBaseDeleteHelpers with Serializable

final class HBaseDeleteRDDSimple[K, Q] extends HBaseDeleteHelpers with Serializable

trait HBaseDeleteSupport extends AnyRef

trait HBaseReadSupport extends AnyRef

final class HBaseSC extends Serializable

trait HBaseUtils extends AnyRef

sealed abstract class HBaseWriteHelpers extends AnyRef

final class HBaseWriteRDD[K, Q, V] extends HBaseWriteHelpers with Serializable

final class HBaseWriteRDDFixed[K, V] extends HBaseWriteHelpers with Serializable

final class HBaseWriteRDDSimple[K, Q, V] extends HBaseWriteHelpers with Serializable

trait HBaseWriteSupport extends AnyRef

final class HFileRDD[K, Q, C, A, V] extends HFileRDDHelper

final class HFileRDDFixed[K, C, A, V] extends HFileRDDHelper

sealed abstract class HFileRDDHelper extends Serializable

final class HFileRDDSimple[K, Q, C, A, V] extends HFileRDDHelper

trait HFileSupport extends AnyRef

trait Reads[A] extends Serializable

trait Writes[A] extends Serializable

Value Members

object Admin

object HBaseConfig extends Serializable

implicit val booleanReader: Reads[Boolean]

implicit val booleanWriter: Writes[Boolean]

implicit val byteArrayReader: Reads[Array[Byte]]

implicit val byteArrayWriter: Writes[Array[Byte]]

implicit lazy val cellKeyOrdering: CellKeyOrdering

implicit lazy val cellKeyTSOrdering: CellKeyTSOrdering

def computeSplits[K](rdd: RDD[K], regionsCount: Int)(implicit arg0: ClassTag[K], ord: Ordering[K]): Seq[K]

def createJob(table: String, conf: Configuration): Job

implicit def deleteHBaseRDD[K, Q](rdd: RDD[(K, Map[String, Set[Q]])])(implicit arg0: Writes[K], arg1: Writes[Q]): HBaseDeleteRDD[K, Q]

implicit def deleteHBaseRDDKey[K](rdd: RDD[K])(implicit arg0: Writes[K]): HBaseDeleteRDDKey[K]

implicit def deleteHBaseRDDSimple[K, Q](rdd: RDD[(K, Set[Q])])(implicit arg0: Writes[K], arg1: Writes[Q]): HBaseDeleteRDDSimple[K, Q]

implicit def deleteHBaseRDDSimpleT[K, Q](rdd: RDD[(K, Set[(Q, Long)])])(implicit arg0: Writes[K], arg1: Writes[Q]): HBaseDeleteRDDSimple[K, (Q, Long)]

implicit def deleteHBaseRDDT[K, Q](rdd: RDD[(K, Map[String, Set[(Q, Long)]])])(implicit arg0: Writes[K], arg1: Writes[Q]): HBaseDeleteRDD[K, (Q, Long)]

implicit val doubleReader: Reads[Double]

implicit val doubleWriter: Writes[Double]

implicit val floatReader: Reads[Float]

implicit val floatWriter: Writes[Float]

implicit val intReader: Reads[Int]

implicit val intWriter: Writes[Int]

implicit val jsonReader: Reads[JValue]

implicit val jsonWriter: Writes[JValue]

implicit val longReader: Reads[Long]

implicit val longWriter: Writes[Long]

implicit val shortReader: Reads[Short]

implicit val shortWriter: Writes[Short]

implicit val stringReader: Reads[String]

implicit val stringWriter: Writes[String]

implicit def toHBaseRDD[K, Q, V](rdd: RDD[(K, Map[String, Map[Q, V]])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: Writes[V]): HBaseWriteRDD[K, Q, V]

implicit def toHBaseRDDFixed[K, V](rdd: RDD[(K, Seq[V])])(implicit arg0: Writes[K], arg1: Writes[V]): HBaseWriteRDDFixed[K, V]

implicit def toHBaseRDDFixedTS[K, V](rdd: RDD[(K, Seq[(V, Long)])])(implicit arg0: Writes[K], arg1: Writes[V]): HBaseWriteRDDFixed[K, (V, Long)]

implicit def toHBaseRDDSimple[K, Q, V](rdd: RDD[(K, Map[Q, V])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: Writes[V]): HBaseWriteRDDSimple[K, Q, V]

implicit def toHBaseRDDSimpleTS[K, Q, V](rdd: RDD[(K, Map[Q, (V, Long)])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: Writes[V]): HBaseWriteRDDSimple[K, Q, (V, Long)]

implicit def toHBaseRDDT[K, Q, V](rdd: RDD[(K, Map[String, Map[Q, (V, Long)]])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: Writes[V]): HBaseWriteRDD[K, Q, (V, Long)]

implicit def toHBaseSC(sc: SparkContext): HBaseSC

implicit def toHFileRDD[K, Q, A](rdd: RDD[(K, Map[String, Map[Q, A]])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: ClassTag[A], writer: Writes[A]): HFileRDD[K, Q, CellKey, A, A]

implicit def toHFileRDDFixed[K, A](rdd: RDD[(K, Seq[A])])(implicit arg0: Writes[K], arg1: ClassTag[A], writer: Writes[A]): HFileRDDFixed[K, CellKey, A, A]

implicit def toHFileRDDFixedTS[K, A](rdd: RDD[(K, Seq[(A, Long)])])(implicit arg0: Writes[K], arg1: ClassTag[A], writer: Writes[A]): HFileRDDFixed[K, CellKeyTS, (A, Long), A]

implicit def toHFileRDDSimple[K, Q, A](rdd: RDD[(K, Map[Q, A])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: ClassTag[A], writer: Writes[A]): HFileRDDSimple[K, Q, CellKey, A, A]

implicit def toHFileRDDSimpleTS[K, Q, A](rdd: RDD[(K, Map[Q, (A, Long)])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: ClassTag[A], writer: Writes[A]): HFileRDDSimple[K, Q, CellKeyTS, (A, Long), A]

implicit def toHFileRDDTS[K, Q, A](rdd: RDD[(K, Map[String, Map[Q, (A, Long)]])])(implicit arg0: Writes[K], arg1: Writes[Q], arg2: ClassTag[A], writer: Writes[A]): HFileRDD[K, Q, CellKeyTS, (A, Long), A]

Inherited from HBaseUtils

Inherited from HFileSupport

Inherited from HBaseDeleteSupport

Inherited from DefaultReads

Inherited from HBaseReadSupport

Inherited from DefaultWrites

Inherited from HBaseWriteSupport

Inherited from AnyRef

Inherited from Any