Allows to execute SQL queries against Cassandra and access results as
SchemaRDD
collections.
Allows to execute SQL queries against Cassandra and access results as
SchemaRDD
collections. Predicate pushdown to Cassandra is supported.
Example:
import com.datastax.spark.connector._ val sparkMasterHost = "127.0.0.1" val cassandraHost = "127.0.0.1" // Tell Spark the address of one Cassandra node: val conf = new SparkConf(true).set("spark.cassandra.connection.host", cassandraHost) // Connect to the Spark cluster: val sc = new SparkContext("spark://" + sparkMasterHost + ":7077", "example", conf) // Create CassandraSQLContext: val cc = new CassandraSQLContext(sc) // Execute SQL query: val rdd = cc.sql("SELECT * FROM keyspace.table ...")
Store data source options
Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider.
Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider. It's used internally by Spark SQL to create Relation for a table which specifies the Cassandra data source e.g.
CREATE TEMPORARY TABLE tmpTable USING org.apache.spark.sql.cassandra OPTIONS ( table "table", keyspace "keyspace", cluster "test_cluster", pushdown "true", spark_cassandra_input_page_row_size "10", spark_cassandra_output_consistency_level "ONE", spark_cassandra_connection_timeout_ms "1000" )
A unified API for predicates, used by PredicatePushDown.
A unified API for predicates, used by PredicatePushDown.
Keeps all the Spark-specific stuff out of PredicatePushDown
It is also easy to plug-in custom predicate implementations for unit-testing.
Determines which filter predicates can be pushed down to Cassandra.
Determines which filter predicates can be pushed down to Cassandra.
The list of predicates to be pushed down is available in predicatesToPushDown
property.
The list of predicates that cannot be pushed down is available in predicatesToPreserve
property.
Store table name, keyspace name and option cluster name, keyspace is equivalent to database
Convert Cassandra data type to Catalyst data type
Provides PredicateOps
adapters for Expression and Filter classes