A B C D E F G H I M N O P R S T U

A

asJavaRDD() - Method in class org.hpccsystems.spark.HpccRDD

Wrap this RDD as a JavaRDD so the Java API can be used.

B

buildScan(String[], Filter[]) - Method in class org.hpccsystems.spark.datasource.HpccRelation

C

clusterName - Variable in class org.hpccsystems.spark.datasource.HpccOptions

compression - Variable in class org.hpccsystems.spark.datasource.HpccOptions

compute(Partition, TaskContext) - Method in class org.hpccsystems.spark.HpccRDD

connectionInfo - Variable in class org.hpccsystems.spark.datasource.HpccOptions

construct(Object[]) - Method in class org.hpccsystems.spark.PySparkFieldConstructor

construct(Object[]) - Method in class org.hpccsystems.spark.RowConstructor

ConvertToHPCCFileFilterString(Filter) - Static method in class org.hpccsystems.spark.FileFilterConverter

CovertToHPCCFileFilter(Filter[]) - Static method in class org.hpccsystems.spark.FileFilterConverter

createRelation(SQLContext, Map<String, String>) - Method in class org.hpccsystems.spark.datasource.HpccRelationProvider

createRelation(SQLContext, SaveMode, Map<String, String>, Dataset<Row>) - Method in class org.hpccsystems.spark.datasource.HpccRelationProvider

D

DEFAULT_CONNECTION_TIMEOUT - Static variable in class org.hpccsystems.spark.HpccRDD

E

expirySeconds - Variable in class org.hpccsystems.spark.datasource.HpccOptions

F

FileFilterConverter - Class in org.hpccsystems.spark

FileFilterConverter() - Constructor for class org.hpccsystems.spark.FileFilterConverter

fileName - Variable in class org.hpccsystems.spark.datasource.HpccOptions

filePartLimit - Variable in class org.hpccsystems.spark.datasource.HpccOptions

filterString - Variable in class org.hpccsystems.spark.datasource.HpccOptions

finalizeRecord() - Method in class org.hpccsystems.spark.GenericRowRecordBuilder

G

GenericRowRecordAccessor - Class in org.hpccsystems.spark

GenericRowRecordAccessor(StructType) - Constructor for class org.hpccsystems.spark.GenericRowRecordAccessor

GenericRowRecordAccessor(FieldDef) - Constructor for class org.hpccsystems.spark.GenericRowRecordAccessor

GenericRowRecordBuilder - Class in org.hpccsystems.spark

GenericRowRecordBuilder(FieldDef) - Constructor for class org.hpccsystems.spark.GenericRowRecordBuilder

getChildRecordAccessor(int) - Method in class org.hpccsystems.spark.GenericRowRecordAccessor

getChildRecordBuilder(int) - Method in class org.hpccsystems.spark.GenericRowRecordBuilder

getDataframe(SparkSession) - Method in class org.hpccsystems.spark.HpccFile

Make a Spark Dataframe (Dataset) of THOR data available.

getFieldDefinition(int) - Method in class org.hpccsystems.spark.GenericRowRecordAccessor

getFieldValue(int) - Method in class org.hpccsystems.spark.GenericRowRecordAccessor

getFilePartRecordLimit() - Method in class org.hpccsystems.spark.HpccFile

Returns the current file part record limit

getName() - Method in class org.hpccsystems.spark.PySparkField

getNumFields() - Method in class org.hpccsystems.spark.GenericRowRecordAccessor

getPartitions() - Method in class org.hpccsystems.spark.HpccRDD

getPreferredLocations(Partition) - Method in class org.hpccsystems.spark.HpccRDD

getRDD() - Method in class org.hpccsystems.spark.HpccFile

Make a Spark Resilient Distributed Dataset (RDD) that provides access to THOR based datasets.

getRDD(SparkContext) - Method in class org.hpccsystems.spark.HpccFile

Make a Spark Resilient Distributed Dataset (RDD) that provides access to THOR based datasets.

getRecordDefinition() - Method in class org.hpccsystems.spark.GenericRowRecordBuilder

getValue() - Method in class org.hpccsystems.spark.PySparkField

H

HpccFile - Class in org.hpccsystems.spark

Access to file content on a collection of one or more HPCC clusters.

HpccFile(String, Connection) - Constructor for class org.hpccsystems.spark.HpccFile

Constructor for the HpccFile.

HpccFile(String, String, String, String) - Constructor for class org.hpccsystems.spark.HpccFile

Constructor for the HpccFile.

HpccFile(String, Connection, String, String, RemapInfo, int, String) - Constructor for class org.hpccsystems.spark.HpccFile

Constructor for the HpccFile.

HpccFileWriter - Class in org.hpccsystems.spark

HpccFileWriter(Connection) - Constructor for class org.hpccsystems.spark.HpccFileWriter

HpccFileWriter(String, String, String) - Constructor for class org.hpccsystems.spark.HpccFileWriter

HpccFileWriter Constructor Attempts to open a connection to the specified HPCC cluster and validates the user.

HpccOptions - Class in org.hpccsystems.spark.datasource

HpccOptions(TreeMap<String, String>) - Constructor for class org.hpccsystems.spark.datasource.HpccOptions

HpccRDD - Class in org.hpccsystems.spark

The implementation of the RDD

HpccRDD(SparkContext, DataPartition[], FieldDef) - Constructor for class org.hpccsystems.spark.HpccRDD

HpccRDD(SparkContext, DataPartition[], FieldDef, FieldDef) - Constructor for class org.hpccsystems.spark.HpccRDD

HpccRDD(SparkContext, DataPartition[], FieldDef, FieldDef, int, int) - Constructor for class org.hpccsystems.spark.HpccRDD

HpccRelation - Class in org.hpccsystems.spark.datasource

HpccRelation(SQLContext, HpccOptions) - Constructor for class org.hpccsystems.spark.datasource.HpccRelation

HpccRelationProvider - Class in org.hpccsystems.spark.datasource

HpccRelationProvider() - Constructor for class org.hpccsystems.spark.datasource.HpccRelationProvider

I

inferSchema(List<PySparkField>) - Method in class org.hpccsystems.spark.HpccFileWriter

Generates an inferred schema based on an example Map of FieldNames -> Example Field Objects.

M

makeMLLibLabeledPoint(String, String[]) - Method in class org.hpccsystems.spark.HpccRDD

Transform to an RDD of labeled points for MLLib supervised learning.

makeMLLibVector(String[]) - Method in class org.hpccsystems.spark.HpccRDD

Transform to mllib.linalg.Vectors for ML Lib machine learning.

N

needConversion() - Method in class org.hpccsystems.spark.datasource.HpccRelation

O

org.hpccsystems.spark - package org.hpccsystems.spark

Spark access to data residing in an HPCC environment.

org.hpccsystems.spark.datasource - package org.hpccsystems.spark.datasource

P

projectList - Variable in class org.hpccsystems.spark.datasource.HpccOptions

PySparkField - Class in org.hpccsystems.spark

PySparkField(String, Object) - Constructor for class org.hpccsystems.spark.PySparkField

PySparkFieldConstructor - Class in org.hpccsystems.spark

PySparkFieldConstructor() - Constructor for class org.hpccsystems.spark.PySparkFieldConstructor

R

RowConstructor - Class in org.hpccsystems.spark

RowConstructor() - Constructor for class org.hpccsystems.spark.RowConstructor

S

saveToHPCC(RDD<Row>, String, String) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster.

saveToHPCC(StructType, RDD<Row>, String, String) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster.

saveToHPCC(JavaRDD<Row>, String, String) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster.

saveToHPCC(StructType, JavaRDD<Row>, String, String) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster.

saveToHPCC(RDD<Row>, String, String, CompressionAlgorithm, boolean) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster Note: PySpark datasets can be written to HPCC by first calling inferSchema to generate a valid Java Schema and converting the PySpark RDD to a JavaRDD via the _py2java() helper

saveToHPCC(StructType, RDD<Row>, String, String, CompressionAlgorithm, boolean) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster Note: PySpark datasets can be written to HPCC by first calling inferSchema to generate a valid Java Schema and converting the PySpark RDD to a JavaRDD via the _py2java() helper

saveToHPCC(JavaRDD<Row>, String, String, CompressionAlgorithm, boolean) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster Note: PySpark datasets can be written to HPCC by first calling inferSchema to generate a valid Java Schema and converting the PySpark RDD to a JavaRDD via the _py2java() helper

saveToHPCC(StructType, JavaRDD<Row>, String, String, CompressionAlgorithm, boolean) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster Note: PySpark datasets can be written to HPCC by first calling inferSchema to generate a valid Java Schema and converting the PySpark RDD to a JavaRDD via the _py2java() helper

saveToHPCC(SparkContext, RDD<Row>, String, String) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster.

saveToHPCC(SparkContext, JavaRDD<Row>, String, String) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster.

saveToHPCC(SparkContext, RDD<Row>, String, String, CompressionAlgorithm, boolean) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster Note: PySpark datasets can be written to HPCC by first calling inferSchema to generate a valid Java Schema and converting the PySpark RDD to a JavaRDD via the _py2java() helper

saveToHPCC(SparkContext, StructType, JavaRDD<Row>, String, String, CompressionAlgorithm, boolean) - Method in class org.hpccsystems.spark.HpccFileWriter

Saves the provided RDD to the specified file within the specified cluster Note: PySpark datasets can be written to HPCC by first calling inferSchema to generate a valid Java Schema and converting the PySpark RDD to a JavaRDD via the _py2java() helper

schema() - Method in class org.hpccsystems.spark.datasource.HpccRelation

setFieldValue(int, Object) - Method in class org.hpccsystems.spark.GenericRowRecordBuilder

setFilePartRecordLimit(int) - Method in class org.hpccsystems.spark.HpccFile

Set file part record limit

setRecord(Object) - Method in class org.hpccsystems.spark.GenericRowRecordAccessor

setRecordDefinition(FieldDef) - Method in class org.hpccsystems.spark.GenericRowRecordBuilder

shortName() - Method in class org.hpccsystems.spark.datasource.HpccRelationProvider

sizeInBytes() - Method in class org.hpccsystems.spark.datasource.HpccRelation

SparkSchemaTranslator - Class in org.hpccsystems.spark

SparkSchemaTranslator() - Constructor for class org.hpccsystems.spark.SparkSchemaTranslator

sqlContext() - Method in class org.hpccsystems.spark.datasource.HpccRelation

startRecord() - Method in class org.hpccsystems.spark.GenericRowRecordBuilder

T

toHPCCRecordDef(StructType) - Static method in class org.hpccsystems.spark.SparkSchemaTranslator

toSparkSchema(FieldDef) - Static method in class org.hpccsystems.spark.SparkSchemaTranslator

translate a FieldDef into a StructField object of the schema

U

unhandledFilters(Filter[]) - Method in class org.hpccsystems.spark.datasource.HpccRelation

A B C D E F G H I M N O P R S T U