partitioner

Type Members

1. case class CassandraPartition(index: Int, endpoints: Iterable[InetAddress], tokenRanges: Iterable[CqlTokenRange], rowCount: Long) extends EndpointPartition with Product with Serializable

   

   Metadata describing Cassandra table partition processed by a single Spark task.

   Metadata describing Cassandra table partition processed by a single Spark task. Beware the term "partition" is overloaded. Here, in the context of Spark, it means an arbitrary collection of rows that can be processed locally on a single Cassandra cluster node. A CassandraPartition typically contains multiple CQL partitions, i.e. rows identified by different values of the CQL partitioning key.

   index

   identifier of the partition, used internally by Spark

   endpoints

   which nodes the data partition is located on

   tokenRanges

   token ranges determining the row set to be fetched

   rowCount

   estimated total row count in a partition

2. class CassandraPartitionedRDD[T] extends RDD[T]

   

   RDD created by repartitionByCassandraReplica with preferred locations mapping to the CassandraReplicas each partition was created for.

3. class CassandraRDDPartitioner[V, T <: Token[V]] extends AnyRef

   

   Creates CassandraPartitions for given Cassandra table

4. case class CqlTokenRange(cql: String, values: Any*) extends Product with Serializable

   

   Stores a CQL WHERE predicate matching a range of tokens.

5. class DataSizeEstimates[V, T <: Token[V]] extends Logging

   

   Estimates amount of data in the Cassandra table.

   Estimates amount of data in the Cassandra table. Takes token range size estimates from the system.size_estimates table, available since Cassandra 2.1.5.

6. trait EndpointPartition extends Partition

   

7. class Murmur3PartitionerTokenRangeSplitter extends TokenRangeSplitter[Long, LongToken]

   

   Fast token range splitter assuming that data are spread out evenly in the whole range.

8. class NodeAddresses extends Serializable

   

   Looks up listen address of a cluster node given its RPC address.

   Looks up listen address of a cluster node given its RPC address. Uses system.peers table as the source of information. If such information for a node is missing, it assumes its listen address equals its RPC address

9. class RandomPartitionerTokenRangeSplitter extends TokenRangeSplitter[BigInt, BigIntToken]

   

   Fast token range splitter assuming that data are spread out evenly in the whole range.

10. case class ReplicaPartition(index: Int, endpoints: Set[InetAddress]) extends EndpointPartition with Product with Serializable

    

11. class ReplicaPartitioner extends Partitioner

    

    The replica partitioner will work on an RDD which is keyed on sets of InetAddresses representing Cassandra Hosts .

    The replica partitioner will work on an RDD which is keyed on sets of InetAddresses representing Cassandra Hosts . It will group keys which share a common IP address into partitionsPerReplicaSet Partitions.

12. class TokenRangeClusterer[V, T <: Token[V]] extends AnyRef

    

    Divides a set of token ranges into groups containing not more than maxRowCountPerGroup rows and not more than maxGroupSize token ranges.

    Divides a set of token ranges into groups containing not more than maxRowCountPerGroup rows and not more than maxGroupSize token ranges. Each group will form a single CassandraRDDPartition.

    The algorithm is as follows: 1. Sort token ranges by endpoints lexicographically. 2. Take the highest possible number of token ranges from the beginning of the list, such that their sum of rowCounts does not exceed maxRowCountPerGroup and they all contain at least one common endpoint. If it is not possible, take at least one item. Those token ranges will make a group. 3. Repeat the previous step until no more token ranges left.

13. trait TokenRangeSplitter[V, T <: Token[V]] extends AnyRef

    

    Splits a token range into smaller sub-ranges, each with the desired approximate number of rows.