This project enables HPCC Systems / Spark interoperability.

The DataAccess project contains the classes which expose distributed streaming of HPCC based data via Spark constructs. In addition, the HPCC data is exposed as a Dataframe for the convenience of the Spark developer.
    

Packages

Package	Description
net.razorvine.pickle	Java implementation of Python's pickle serialization protocol.
net.razorvine.pickle.objects	Object constructors and other utility classes for the pickle package.
net.razorvine.pyro	Lightweight implementation of the client side Pyro protocol.
net.razorvine.pyro.serializer
org.apache.spark	Core Spark classes in Scala.
org.apache.spark.api.java	Spark Java programming APIs.
org.apache.spark.api.java.function	Set of interfaces to represent functions in Spark's Java API.
org.apache.spark.broadcast	Spark's broadcast variables, used to broadcast immutable datasets to all nodes.
org.apache.spark.executor	Package for executor components used with various cluster managers.
org.apache.spark.io	IO codecs used for compression.
org.apache.spark.memory
org.apache.spark.rdd	Provides implementation's of various RDDs.
org.apache.spark.scheduler	Spark's DAG scheduler.
org.apache.spark.serializer	Pluggable serializers for RDD and shuffle data.
org.apache.spark.shuffle.sort
org.apache.spark.sql
org.apache.spark.sql.api.java	Allows the execution of relational queries, including those expressed in SQL using Spark.
org.apache.spark.sql.execution
org.apache.spark.sql.execution.columnar
org.apache.spark.sql.execution.datasources
org.apache.spark.sql.execution.datasources.orc
org.apache.spark.sql.execution.datasources.parquet
org.apache.spark.sql.execution.streaming
org.apache.spark.sql.execution.vectorized
org.apache.spark.sql.expressions.javalang
org.apache.spark.sql.internal	All classes in this package are considered an internal API to Spark and are subject to change between minor releases.
org.apache.spark.sql.sources.v2
org.apache.spark.sql.sources.v2.reader
org.apache.spark.sql.sources.v2.reader.partitioning
org.apache.spark.sql.sources.v2.reader.streaming
org.apache.spark.sql.sources.v2.writer
org.apache.spark.sql.sources.v2.writer.streaming
org.apache.spark.sql.streaming
org.apache.spark.sql.vectorized
org.apache.spark.status.api.v1
org.apache.spark.storage
org.apache.spark.unsafe.map
org.apache.spark.util	Spark utilities.
org.apache.spark.util.collection
org.apache.spark.util.collection.unsafe.sort
org.apache.spark.util.random	Utilities for random number generation.
org.hpccsystems.commons.annotations	Used internally to annotate test (junit) types and thereby control execution of annotated test suites.
org.hpccsystems.commons.benchmarking	Provides generic support of performance metric gathering, calculating, and reporting.
org.hpccsystems.commons.ecl	Provides basic HPCC Systems ECL language construct support.
org.hpccsystems.commons.errors	Provides support for common HPCC Systems error handling and interpretation.
org.hpccsystems.commons.filter	Provides simple SQL-like filter support, most commonly to be translated into HPCC Systems filter syntax.
org.hpccsystems.commons.network	Provides basic local network address resolution in support of HPCC Systems connectivity.
org.hpccsystems.commons.utils	Provides useful utilities in support of Java client interaction with HPCC Systems instances.
org.hpccsystems.dfs.client	Provides efficient distributed data ingestion and extraction.
org.hpccsystems.dfs.cluster	Provides target cluster node address mapping between private/public networks.
org.hpccsystems.generated	Contains dynamically generated class populated with static build-time variables.
org.hpccsystems.spark	Provides access to data residing in HPCC Systems or Spark environments.
org.hpccsystems.spark.datasource	Provides mechanism to stream HPCC Systems data via Spark Relation.
org.hpccsystems.ws.client	Provides several HPCC Systems client libraries used to actuate a variety of actions on a target HPCC Systems instance.
org.hpccsystems.ws.client.antlr	Used internally to parse various formats of ECL record definitions.
org.hpccsystems.ws.client.extended	Provides HPCC Systems client libraries used to actuate non-standard HPCC Web Service actions.
org.hpccsystems.ws.client.gen.axis2.filespray.v1_20
org.hpccsystems.ws.client.gen.axis2.wsattributes.v1_21
org.hpccsystems.ws.client.gen.axis2.wscodesign.v1_01
org.hpccsystems.ws.client.gen.axis2.wsdfu.v1_39
org.hpccsystems.ws.client.gen.axis2.wsdfu.v1_56
org.hpccsystems.ws.client.gen.axis2.wsdfu.v1_57
org.hpccsystems.ws.client.gen.axis2.wsdfuxref.v1_02
org.hpccsystems.ws.client.gen.axis2.wsfileio.v1_00
org.hpccsystems.ws.client.gen.axis2.wspackageprocess.v1_04
org.hpccsystems.ws.client.gen.axis2.wsresources.v1_00
org.hpccsystems.ws.client.gen.axis2.wssmc.v1_23
org.hpccsystems.ws.client.gen.axis2.wssql.v3_06
org.hpccsystems.ws.client.gen.axis2.wsstore.v1_02
org.hpccsystems.ws.client.gen.axis2.wstopology.v1_30
org.hpccsystems.ws.client.gen.axis2.wsworkunits.v1_56
org.hpccsystems.ws.client.gen.axis2.wsworkunits.v1_58
org.hpccsystems.ws.client.gen.axis2.wsworkunits.v1_62
org.hpccsystems.ws.client.gen.axis2.wsworkunits.v1_69
org.hpccsystems.ws.client.gen.axis2.wsworkunits.v1_74
org.hpccsystems.ws.client.gen.axis2.wsworkunits.v1_81
org.hpccsystems.ws.client.platform	Provides local representation of target HPCC Systems platform.
org.hpccsystems.ws.client.platform.test	Used internally as base class for remote tests.
org.hpccsystems.ws.client.utils	Useful utilities for users of the HPCCWsClient libraries.
org.hpccsystems.ws.client.wrappers	Layer of abstraction between caller and HPCC Systems Web Service interface.
org.hpccsystems.ws.client.wrappers.gen.filespray
org.hpccsystems.ws.client.wrappers.gen.wscodesign
org.hpccsystems.ws.client.wrappers.gen.wsdfu
org.hpccsystems.ws.client.wrappers.gen.wsdfuxref
org.hpccsystems.ws.client.wrappers.gen.wspackageprocess
org.hpccsystems.ws.client.wrappers.gen.wsresources
org.hpccsystems.ws.client.wrappers.gen.wssmc
org.hpccsystems.ws.client.wrappers.gen.wssql
org.hpccsystems.ws.client.wrappers.gen.wstopology
org.hpccsystems.ws.client.wrappers.wsdfu
org.hpccsystems.ws.client.wrappers.wsworkunits