connector

Type Members

class DataFrameFunctions extends Logging

Provides Ryft-specific methods on org.apache.spark.sql.DataFrame
implicit class FormatImplicits extends AnyRef

Represents implicit conversions for com.ryft.spark.connector.query.value.RyftQueryValue DSL
case class FullSubParams(value: Value, operator: RyftValueOperator, format: Format) extends Product with Serializable
implicit class FullSubParamsImplicits extends AnyRef
class PairRDDFunctions[K] extends Logging

Provides Ryft-specific methods on pair org.apache.spark.rdd.RDD
class RDDFunctions[T] extends Logging

Provides Ryft-specific methods on org.apache.spark.rdd.RDD
implicit class RyftDataFrameReader extends AnyRef

Adds a method, ryft, to DataFrameReader that allows you to read ryft files using the DataFileReader
implicit class RyftQueryImplicits extends AnyRef

Allows to use com.ryft.spark.connector.query.RecordQuery DSL
class SparkContextFunctions extends Logging

Provides Ryft-specific methods on SparkContext
implicit class ValueImplicits extends AnyRef

Value Members

package config
package domain

Provides classes to describe query data model.
Provides classes to describe query data model.
Overview
Here are some essential enums and classes.
com.ryft.spark.connector.domain.LogicalOperator - Logical operators (and, or, xor) used by com.ryft.spark.connector.query.RyftQuery.
com.ryft.spark.connector.domain.Action - Represents action on RyftOne and corresponding REST endpoint. Currently supoprted Search and Count.
com.ryft.spark.connector.domain.RyftQueryOptions - And important class representing REST Search query parameters, like fuzziness, surrounding, list of fields and number of nodes to use for search.
package exception

Provides classes to represent connector specific exceptions.
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
def log: Logger

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logName: String

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
package partitioner
package preferred

package query

Provides classes to represent different RyftOne queries.

Overview

The main class to use is com.ryft.spark.connector.query.RecordQuery, as so

import com.ryft.spark.connector._
import com.ryft.spark.connector.domain.{recordField, contains}
import com.ryft.spark.connector.query.RecordQuery

val query = RecordQuery(recordField("Date"), contains, "04/14/2015")
                   .and(recordField("Arrest"), contains, "false")

For raw text searches com.ryft.spark.connector.query.SimpleQuery class should be used, as so

import com.ryft.spark.connector._
import com.ryft.spark.connector.query.SimpleQuery

val query = SimpleQuery("James Bond")

package rdd

Provides classes for RyftOne connector specific RDD operations.
package rest

package sql

Provides classes for DataFrames and SparkSQL support.

Overview

These classes are not intended to be used directly but should be accessed via com.ryft.spark.connector.RyftDataFrameReader

For example:

import org.apache.spark.sql.SQLContext
import org.apache.spark.sql.types._
import org.apache.spark.{SparkConf, SparkContext}
import com.ryft.spark.connector._

// Definition for the DataFrame setup
val schema = StructType(Seq(
StructField("Arrest", BooleanType), StructField("Beat", IntegerType),
StructField("Block", StringType), StructField("CaseNumber", StringType),
StructField("_index", StructType(Seq(
  StructField("file", StringType), StructField("offset", StringType),
  StructField("length", IntegerType), StructField("fuzziness", ByteType)))
 )
))

sqlContext.read.ryft(schema, "*.crimestat", "crime_table")

implicit def toDataFrameFunctions(df: DataFrame): DataFrameFunctions
implicit def toListRyftQueryValue(values: List[Any]): List[RyftQueryValue]
implicit def toPairRDDFunctions[K](rdd: RDD[(K, Array[Byte])]): PairRDDFunctions[K]
implicit def toRDDFunctions[T](rdd: RDD[T]): RDDFunctions[T]
implicit def toSparkContextFunctions(sc: SparkContext): SparkContextFunctions
package util

package connector

Type Members

class DataFrameFunctions extends Logging

implicit class FormatImplicits extends AnyRef

case class FullSubParams(value: Value, operator: RyftValueOperator, format: Format) extends Product with Serializable

implicit class FullSubParamsImplicits extends AnyRef

class PairRDDFunctions[K] extends Logging

class RDDFunctions[T] extends Logging

implicit class RyftDataFrameReader extends AnyRef

implicit class RyftQueryImplicits extends AnyRef

class SparkContextFunctions extends Logging

implicit class ValueImplicits extends AnyRef

Value Members

package config

package domain

Overview

package exception

def isTraceEnabled(): Boolean

def log: Logger

def logDebug(msg: ⇒ String, throwable: Throwable): Unit

def logDebug(msg: ⇒ String): Unit

def logError(msg: ⇒ String, throwable: Throwable): Unit

def logError(msg: ⇒ String): Unit

def logInfo(msg: ⇒ String, throwable: Throwable): Unit

def logInfo(msg: ⇒ String): Unit

def logName: String

def logTrace(msg: ⇒ String, throwable: Throwable): Unit

def logTrace(msg: ⇒ String): Unit

def logWarning(msg: ⇒ String, throwable: Throwable): Unit

def logWarning(msg: ⇒ String): Unit

package partitioner

package preferred

package query

Overview

package rdd

package rest

package sql

Overview

implicit def toDataFrameFunctions(df: DataFrame): DataFrameFunctions

implicit def toListRyftQueryValue(values: List[Any]): List[RyftQueryValue]

implicit def toPairRDDFunctions[K](rdd: RDD[(K, Array[Byte])]): PairRDDFunctions[K]

implicit def toRDDFunctions[T](rdd: RDD[T]): RDDFunctions[T]

implicit def toSparkContextFunctions(sc: SparkContext): SparkContextFunctions

package util

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped