jobserver

class BroadcastPersister[T] extends NamedObjectPersister[NamedBroadcast[T]]

implementation of a NamedObjectPersister for Broadcast objects
class DataFramePersister extends NamedObjectPersister[NamedDataFrame]

implementation of a NamedObjectPersister for DataFrame objects
class JSessionTestJob extends JSessionJob[Array[Row]]
class JSessionTestLoaderJob extends JSessionJob[Long]
class JStreamingTestJob extends JStreamingJob[Integer]
case class NamedBroadcast[T](broadcast: Broadcast[T]) extends NamedObject with Product with Serializable

wrapper for named objects of type Broadcast
case class NamedDataFrame(df: DataFrame, forceComputation: Boolean, storageLevel: StorageLevel) extends NamedObject with Product with Serializable

wrapper for named objects of type DataFrame
class NamedObjectsTestJob extends SparkJob with NamedObjectSupport

A test job for named objects
trait SparkSessionJob extends jobserver.api.SparkJobBase
trait SparkStreamingJob extends SparkJobBase

Defines a Job that runs on a StreamingContext, note that these jobs are usually long running jobs and there's (yet) no way in Spark Job Server to query the status of these jobs.

object KMeansExample extends SparkJob with NamedRddSupport

A Spark job example that implements the SparkJob trait and can be submitted to the job server.
A Spark job example that implements the SparkJob trait and can be submitted to the job server.
Set the config with the sentence to split or count: input.string = "adsfasdf asdkf safksf a sdfa"
validate() returns SparkJobInvalid if there is no input.string
object NamedObjectsTestJobConfig
object SessionLoaderTestJob extends SparkSessionJob

A test job that accepts a SparkSession.
A test job that accepts a SparkSession. Initializes some dummy data into a table, reads it back out, and returns a count. Will create Hive metastore at job-server/metastore_db if Hive isn't configured.
object SessionTestJob extends SparkSessionJob

This job simply runs the Hive SQL in the config.
object StreamingTaskFailedTestJob extends SparkStreamingJob
object StreamingTestJob extends SparkStreamingJob

Annotations
@VisibleForTesting()
package context
package japi
package python