examples
Usage: BroadcastTest [slices] [numElem] [blockSize]
Simple test for reading and writing to a distributed file system.
Prints out environmental information, sleeps, and then exits.
Prints out environmental information, sleeps, and then exits. Made to test driver submission in the standalone scheduler.
Usage: GroupByTest [numMappers] [numKVPairs] [KeySize] [numReducers]
Alternating least squares matrix factorization.
Alternating least squares matrix factorization.
This is an example implementation for learning how to use Spark. For more conventional use, please refer to org.apache.spark.ml.recommendation.ALS.
Logistic regression based classification.
Logistic regression based classification.
This is an example implementation for learning how to use Spark. For more conventional use, please refer to org.apache.spark.ml.classification.LogisticRegression.
K-means clustering.
K-means clustering.
This is an example implementation for learning how to use Spark. For more conventional use, please refer to org.apache.spark.ml.clustering.KMeans.
Logistic regression based classification.
Logistic regression based classification.
This is an example implementation for learning how to use Spark. For more conventional use, please refer to org.apache.spark.ml.classification.LogisticRegression.
Executes a roll up-style query against Apache logs.
Executes a roll up-style query against Apache logs.
Usage: LogQuery [logFile]
Usage: MultiBroadcastTest [slices] [numElem]
Usage: SimpleSkewedGroupByTest [numMappers] [numKVPairs] [valSize] [numReducers] [ratio]
Usage: GroupByTest [numMappers] [numKVPairs] [KeySize] [numReducers]
Alternating least squares matrix factorization.
Alternating least squares matrix factorization.
This is an example implementation for learning how to use Spark. For more conventional use, please refer to org.apache.spark.ml.recommendation.ALS.
Logistic regression based classification.
Logistic regression based classification.
This is an example implementation for learning how to use Spark. For more conventional use, please refer to org.apache.spark.ml.classification.LogisticRegression.
K-means clustering.
K-means clustering.
This is an example implementation for learning how to use Spark. For more conventional use, please refer to org.apache.spark.ml.clustering.KMeans.
Logistic regression based classification.
Logistic regression based classification. Usage: SparkLR [slices]
This is an example implementation for learning how to use Spark. For more conventional use, please refer to org.apache.spark.ml.classification.LogisticRegression.
Computes the PageRank of URLs from an input file.
Computes the PageRank of URLs from an input file. Input file should be in format of: URL neighbor URL URL neighbor URL URL neighbor URL ... where URL and their neighbors are separated by space(s).
This is an example implementation for learning how to use Spark. For more conventional use, please refer to org.apache.spark.graphx.lib.PageRank
Computes an approximation to pi
Transitive closure on a graph.
Simple test for reading and writing to a distributed file system. This example does the following: