com.edmunds.rest.databricks.service

Interface JobService

All Known Implementing Classes:

JobServiceImpl

public interface JobService

A wrapper around the Databricks Job API.

See Also:

https://docs.databricks.com/api/latest/jobs.html

Method Summary

Modifier and Type	Method and Description
String	buildRunJobRestUrl(long jobId, long numberInJob) Formatting databricks job rest url.
void	cancelRun(long runId) Cancels a run.
long	createJob(JobSettingsDTO jobSettingsDTO) Creates job from settings DTO.
void	deleteJob(long jobId) Deletes the job and sends an email to the addresses specified in email_notifications.
void	deleteJob(String jobName) Deletes a job with a given name.
JobDTO	getJob(long jobId) Retrieves information about a single job.
JobDTO	getJobByName(String jobName) Will try and get a job by name.
JobDTO	getJobByName(String jobName, boolean failOnMultipleJobs) Will try and get a job by name.
String	getJobLink(long jobId) Produces the URL of a job given job id.
List<JobDTO>	getJobsByName(String jobName) Retrieves all jobs matching a name exactly.
List<JobDTO>	getJobsByRegex(Pattern regex) Retrieves all jobs with a name matching a given regex.
RunDTO	getRun(long runId) Retrieves the metadata of a run.
JobsDTO	listAllJobs() Returns a list of all jobs that are active.
RunsDTO	listRuns(Long jobId, Boolean activeOnly, Integer offset, Integer limit) Lists runs from most recently started to least.
void	reset(long jobId, JobSettingsDTO jobSettings) "resets" or "edits" a job definition.
RunNowDTO	runJobNow(long jobId) Runs the job now.
RunNowDTO	runJobNow(long jobId, Map<String,String> notebookParams) Runs the job now.
RunNowDTO	runJobNow(long jobId, RunParametersDTO params) Runs the job now.
void	upsertJob(JobSettingsDTO jobSettingsDTO, boolean failOnDuplicateJobNames) Given a job settings DTO object it will: - try to find the id by name.

Method Detail

createJob

long createJob(JobSettingsDTO jobSettingsDTO)
        throws IOException,
               DatabricksRestException

Creates job from settings DTO.

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#create

deleteJob

void deleteJob(long jobId)
        throws IOException,
               DatabricksRestException

Deletes the job and sends an email to the addresses specified in email_notifications.

Parameters:

jobId - The canonical identifier of the job to delete

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#delete

deleteJob

void deleteJob(String jobName)
        throws IOException,
               DatabricksRestException

Deletes a job with a given name. Will fail if multiple jobs exist.

Parameters:

jobName - - the job name to delete.

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#delete

getJob

JobDTO getJob(long jobId)
       throws IOException,
              DatabricksRestException

Retrieves information about a single job.

Parameters:

jobId - The canonical identifier of the job to retrieve information about

Returns:

POJO of the Job information

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#get

getJobsByName

List<JobDTO> getJobsByName(String jobName)
                    throws IOException,
                           DatabricksRestException

Retrieves all jobs matching a name exactly.

Parameters:

jobName - - the full name of the job

Returns:

- all jobs that match.

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

getJobByName

JobDTO getJobByName(String jobName)
             throws IOException,
                    DatabricksRestException

Will try and get a job by name. If more then one job exists, it will fail. If no job exists it will return null. Uses a list behind the scenes, so is O(n) with number of jobs.

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

getJobByName

JobDTO getJobByName(String jobName,
                    boolean failOnMultipleJobs)
             throws IOException,
                    DatabricksRestException

Will try and get a job by name. If more then one job exists and failOnMultipleJobs set to false, it will return the first job created. Uses a list behind the scenes, so is O(n) with number of jobs.

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

getJobsByRegex

List<JobDTO> getJobsByRegex(Pattern regex)
                     throws IOException,
                            DatabricksRestException

Retrieves all jobs with a name matching a given regex. Uses a list behind the scenes, so is O(n) with number of jobs.

Parameters:

regex - - the regex to earch for

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

listAllJobs

JobsDTO listAllJobs()
             throws IOException,
                    DatabricksRestException

Returns a list of all jobs that are active.

Returns:

A POJO of the Jobs

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

getJobLink

String getJobLink(long jobId)

Produces the URL of a job given job id.

Parameters:

jobId - The canonical identifier of the job to retrieve information about

Returns:

URL of the job

runJobNow

RunNowDTO runJobNow(long jobId)
             throws DatabricksRestException,
                    IOException

Runs the job now.

Parameters:

jobId - The job to run

Returns:

Returns the run_id and number_in_run of the triggered run

Throws:

DatabricksRestException

IOException

See Also:

https://docs.databricks.com/api/latest/jobs.html#run-now

runJobNow

RunNowDTO runJobNow(long jobId,
                    Map<String,String> notebookParams)
             throws DatabricksRestException,
                    IOException

Runs the job now. With specific notebook params.

Parameters:

jobId - The job to run

notebookParams - A map from keys to values for jobs with notebook task

Returns:

Returns the run_id and number_in_run of the triggered run

Throws:

DatabricksRestException

IOException

See Also:

https://docs.databricks.com/api/latest/jobs.html#run-now

runJobNow

RunNowDTO runJobNow(long jobId,
                    RunParametersDTO params)
             throws DatabricksRestException,
                    IOException

Runs the job now. With specific run parameters.

Parameters:

jobId - the job id to run

params - the run parameters

Returns:

the run now object

Throws:

DatabricksRestException - any db specific exceptions

IOException - any other exceptions

See Also:

https://docs.databricks.com/api/latest/jobs.html#run-now

listRuns

RunsDTO listRuns(Long jobId,
                 Boolean activeOnly,
                 Integer offset,
                 Integer limit)
          throws DatabricksRestException,
                 IOException

Lists runs from most recently started to least.

Parameters:

jobId - The job for which to list runs. If omitted, the Jobs service will list runs from all jobs

activeOnly - If true, lists active runs only; otherwise, lists both active and inactive runs

offset - The offset of the first run to return, relative to the most recent run. The default value is 20

limit - The number of runs to return. This value should be greater than 0 and less than 1000

Returns:

Returns RunsDTO containing an array of runs and a boolean indicating if there are more jobs that haven't been included

Throws:

DatabricksRestException

IOException

See Also:

https://docs.databricks.com/api/latest/jobs.html#runs-list

getRun

RunDTO getRun(long runId)
       throws DatabricksRestException,
              IOException

Retrieves the metadata of a run.

Parameters:

runId - The canonical identifier of the run for which to retrieve the metadata

Returns:

Returns the metadata of the specified run

Throws:

DatabricksRestException

IOException

See Also:

https://docs.databricks.com/api/latest/jobs.html#runs-get

cancelRun

void cancelRun(long runId)
        throws IOException,
               DatabricksRestException

Cancels a run. The run is canceled asynchronously, so when this request completes, the run may still be running. The run will be terminated shortly.

Parameters:

runId - The desired run to cancel

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#runs-cancel

reset

void reset(long jobId,
           JobSettingsDTO jobSettings)
    throws IOException,
           DatabricksRestException

"resets" or "edits" a job definition.

Parameters:

jobId - the job to edit

jobSettings - the settings to change the job to

Throws:

IOException - any other errors

DatabricksRestException - any specific db errors

See Also:

https://docs.databricks.com/api/latest/jobs.html#reset

buildRunJobRestUrl

String buildRunJobRestUrl(long jobId,
                          long numberInJob)

Formatting databricks job rest url. ex> https://something.cloud.databricks.com/#job/123/run/123

upsertJob

void upsertJob(JobSettingsDTO jobSettingsDTO,
               boolean failOnDuplicateJobNames)
        throws IOException,
               DatabricksRestException

Given a job settings DTO object it will: - try to find the id by name. If multiple exist, it will fail if the duplicate job name flag is set. Else it will update all jobs found. - create the job if it doesn't exist - reset the job if it does exist. Uses a combination of If job doesn't exist:

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#create If job exists:, https://docs.databricks.com/api/latest/jobs.html#reset