This is the entry point for a Spark Job Server to execute Spark jobs.
This is the entry point for a Spark Job Server to execute Spark jobs. This function should create or reuse RDDs and return the result at the end, which the Job Server will cache or display.
a SparkContext or similar for the job. May be reused across jobs.
the Typesafe Config object passed into the job request
the job result
This method is called by the job server to allow jobs to validate their input and reject invalid job requests.
This method is called by the job server to allow jobs to validate their input and reject invalid job requests. If SparkJobInvalid is returned, then the job server returns 400 to the user. NOTE: this method should return very quickly. If it responds slowly then the job server may time out trying to start this job.
either SparkJobValid or SparkJobInvalid
Defines a Job that runs on a StreamingContext, note that these jobs are usually long running jobs and there's (yet) no way in Spark Job Server to query the status of these jobs.