DataFrame

Type Members

class GroupedView extends AnyRef
class TypedView[T] extends Iterable[T]

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def addColumn(expr: Expr): DataFrame

Add a column to the DataFrame, evaluating to 'expr' at each individual row index.
Add a column to the DataFrame, evaluating to 'expr' at each individual row index. Use the 'as' method on Expr to give the column a name.
expr
the Expr to evaluate as the new column
returns
new DataFrame
def apply(columnNames: String*): DataFrame

Select a subset of columns from this DataFrame.
Select a subset of columns from this DataFrame.
columnNames
names of columns to select
returns
new DataFrame
def apply(range: Range): DataFrame

Retrieve a subset of rows from this DataFrame based on range of indices.
Retrieve a subset of rows from this DataFrame based on range of indices.
range
range of row indices to retrieve
returns
new DataFrame
def apply(index: Int): Seq[Any]

Retrieve a single row by index.
Retrieve a single row by index.
index
row index
returns
row as a sequence of values
def as[T](implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): TypedView[T]

Typecast this DataFrame to a TypedView of the type parameter 'T'.
Typecast this DataFrame to a TypedView of the type parameter 'T'. All columns in this DataFrame will have to be accounted for in the given type. A DataFrame with multiple columns will have its rows represented as tuples of the individual types of these columns.
T
the type of a row in this DataFrame
returns
TypedView on the contents of this DataFrame
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def explode(expr: Expr): DataFrame

Explode this DataFrame on the given expression, flattening its contents and repeating all other cells on the row for every element in the sequence.
Explode this DataFrame on the given expression, flattening its contents and repeating all other cells on the row for every element in the sequence. The given Expr must evaluate to a list type. Use the 'as' method on Expr to name the flattened column.
expr
the Expr to explode on
returns
new DataFrame
def filter(expr: Expr): DataFrame

Retrieve a subset of rows from this DataFrame based on the boolean evaluation of the given expression.
Retrieve a subset of rows from this DataFrame based on the boolean evaluation of the given expression.
expr
the Expr to evaluate, if 'true' the given row will appear in the output
returns
new DataFrame
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def groupBy(keyExprs: Expr*): GroupedView

Partition this DataFrame into groups, defined by the given set of expressions.
Partition this DataFrame into groups, defined by the given set of expressions. The evaluation of each of the 'keyExprs' will appear as a column in the output.
keyExprs
the list of com.audienceproject.crossbow.expr.Expr that will evaluate to the keys of the groups
returns
GroupedView on this DataFrame
def hashCode(): Int

Definition Classes
AnyRef → Any
def isEmpty: Boolean
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def iterator: Iterator[Seq[Any]]
def join(other: DataFrame, joinExpr: Expr, joinType: JoinType = JoinType.Inner): DataFrame

Join this DataFrame on another DataFrame, with the key evaluated by 'joinExpr'.
Join this DataFrame on another DataFrame, with the key evaluated by 'joinExpr'. The resulting DataFrame will contain all the columns of this DataFrame and the other, where the column names of the other will be prepended with "#".
other
DataFrame to join with this one
joinExpr
Expr to evaluate as join key
joinType
JoinType as one of Inner, FullOuter, LeftOuter or RightOuter
returns
new DataFrame

Note
'joinExpr' must evaluate to a type with a natural ordering
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val numColumns: Int
def printSchema(): Unit
def removeColumns(columnNames: String*): DataFrame

Remove one or more columns from the DataFrame.
Remove one or more columns from the DataFrame.
columnNames
the names of the columns to remove
returns
new DataFrame
def renameColumns(toNewName: (String) ⇒ String): DataFrame

Rename the columns of this DataFrame by applying the given function.
Rename the columns of this DataFrame by applying the given function.
toNewName
function to map over the names of the columns
returns
new DataFrame
def renameColumns(newNames: String*): DataFrame

Rename the columns of this DataFrame.
Rename the columns of this DataFrame.
newNames
list of new names for each column of this DataFrame
returns
new DataFrame
val rowCount: Int
val schema: Schema
def select(exprs: Expr*): DataFrame

Map over this DataFrame, selecting a set of expressions which will become the columns of a new DataFrame.
Map over this DataFrame, selecting a set of expressions which will become the columns of a new DataFrame. Use the 'as' method on Expr to give names to the new columns. An expression which is only a column accessor will inherit the accessed column's name (unless it is renamed).
exprs
the list of Expr to evaluate as a new DataFrame
returns
new DataFrame
def sortBy(expr: Expr, givenOrderings: Order*): DataFrame

Sort this DataFrame by the evaluation of 'expr'.
Sort this DataFrame by the evaluation of 'expr'. If a natural ordering exists on this value, it will be used. User-defined orderings on other types or for overwriting the natural orderings with an explicit ordering can be supplied through the 'givenOrderings' argument.
expr
the Expr to evaluate as a sort key
givenOrderings
explicit Order to use on the sort key, or list of Order if the key is a tuple
returns
new DataFrame
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
def union(other: DataFrame): DataFrame

Union this DataFrame with another DataFrame.
Union this DataFrame with another DataFrame. Columns will be matched by name, and if matched they must have the same type. Columns that are not present in one or the other DataFrame will contain null-values in the output for the rows of the DataFrame in which the column was not present.
other
DataFrame to union with this one
returns
new DataFrame
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Docs: object DataFrame | package crossbow

class DataFrame extends AnyRef

Type Members

class GroupedView extends AnyRef

class TypedView[T] extends Iterable[T]

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

def addColumn(expr: Expr): DataFrame

def apply(columnNames: String*): DataFrame

def apply(range: Range): DataFrame

def apply(index: Int): Seq[Any]

def as[T](implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): TypedView[T]

final def asInstanceOf[T0]: T0

def clone(): AnyRef

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def explode(expr: Expr): DataFrame

def filter(expr: Expr): DataFrame

def finalize(): Unit

final def getClass(): Class[_]

def groupBy(keyExprs: Expr*): GroupedView

def hashCode(): Int

def isEmpty: Boolean

final def isInstanceOf[T0]: Boolean

def iterator: Iterator[Seq[Any]]

def join(other: DataFrame, joinExpr: Expr, joinType: JoinType = JoinType.Inner): DataFrame

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

val numColumns: Int

def printSchema(): Unit

def removeColumns(columnNames: String*): DataFrame

def renameColumns(toNewName: (String) ⇒ String): DataFrame

def renameColumns(newNames: String*): DataFrame

val rowCount: Int

val schema: Schema

def select(exprs: Expr*): DataFrame

def sortBy(expr: Expr, givenOrderings: Order*): DataFrame

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

def union(other: DataFrame): DataFrame

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from AnyRef

Inherited from Any

Ungrouped