com.edmunds.rest.databricks.service

Class JobServiceImpl

java.lang.Object

com.edmunds.rest.databricks.service.DatabricksService

com.edmunds.rest.databricks.service.JobServiceImpl

All Implemented Interfaces:

JobService

public class JobServiceImpl
extends DatabricksService
implements JobService

The implementation of JobService.

Field Summary

Fields inherited from class com.edmunds.rest.databricks.service.DatabricksService

client, mapper

Constructor Summary

Constructors

Constructor and Description
JobServiceImpl(DatabricksRestClient client)

Method Summary

Modifier and Type	Method and Description
String	buildRunJobRestUrl(long jobId, long numberInJob) Formatting databricks job rest url.
void	cancelRun(long runId) Cancels a run.
long	createJob(JobSettingsDTO jobSettingsDTO) Creates job from settings DTO.
void	deleteJob(long jobId) Deletes the job and sends an email to the addresses specified in email_notifications.
void	deleteJob(String jobName) Deletes a job with a given name.
JobDTO	getJob(long jobId) Retrieves information about a single job.
JobDTO	getJobByName(String jobName) Will try and get a job by name.
JobDTO	getJobByName(String jobName, boolean failOnMultipleJobs) Will try and get a job by name.
String	getJobLink(long jobId) Produces the URL of a job given job id.
List<JobDTO>	getJobsByName(String jobName) Retrieves all jobs matching a name exactly.
List<JobDTO>	getJobsByRegex(Pattern regex) Retrieves all jobs with a name matching a given regex.
RunDTO	getRun(long runId) Retrieves the metadata of a run.
JobsDTO	listAllJobs() Returns a list of all jobs that are active.
RunsDTO	listRuns(Long jobId, Boolean activeOnly, Integer offset, Integer limit) Lists runs from most recently started to least.
void	reset(long jobId, JobSettingsDTO jobSettings) "resets" or "edits" a job definition.
RunNowDTO	runJobNow(long jobId) Runs the job now.
RunNowDTO	runJobNow(long jobId, Map<String,String> notebookParams) Runs the job now.
RunNowDTO	runJobNow(long jobId, RunParametersDTO params) Runs the job now.
void	upsertJob(JobSettingsDTO jobSettingsDTO, boolean failOnDuplicateJobNames) Given a job settings DTO object it will: - try to find the id by name.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JobServiceImpl

public JobServiceImpl(DatabricksRestClient client)

Method Detail

createJob

public long createJob(JobSettingsDTO jobSettingsDTO)
               throws IOException,
                      DatabricksRestException

Description copied from interface: JobService

Creates job from settings DTO.

Specified by:

createJob in interface JobService

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#create

deleteJob

public void deleteJob(long jobId)
               throws IOException,
                      DatabricksRestException

Description copied from interface: JobService

Deletes the job and sends an email to the addresses specified in email_notifications.

Specified by:

deleteJob in interface JobService

Parameters:

jobId - The canonical identifier of the job to delete

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#delete

deleteJob

public void deleteJob(String jobName)
               throws IOException,
                      DatabricksRestException

Description copied from interface: JobService

Deletes a job with a given name. Will fail if multiple jobs exist.

Specified by:

deleteJob in interface JobService

Parameters:

jobName - - the job name to delete.

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#delete

getJob

public JobDTO getJob(long jobId)
              throws IOException,
                     DatabricksRestException

Description copied from interface: JobService

Retrieves information about a single job.

Specified by:

getJob in interface JobService

Parameters:

jobId - The canonical identifier of the job to retrieve information about

Returns:

POJO of the Job information

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#get

getJobsByName

public List<JobDTO> getJobsByName(String jobName)
                           throws IOException,
                                  DatabricksRestException

Description copied from interface: JobService

Retrieves all jobs matching a name exactly.

Specified by:

getJobsByName in interface JobService

Parameters:

jobName - - the full name of the job

Returns:

- all jobs that match.

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

getJobByName

public JobDTO getJobByName(String jobName)
                    throws IOException,
                           DatabricksRestException

Description copied from interface: JobService

Will try and get a job by name. If more then one job exists, it will fail. If no job exists it will return null. Uses a list behind the scenes, so is O(n) with number of jobs.

Specified by:

getJobByName in interface JobService

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

getJobByName

public JobDTO getJobByName(String jobName,
                           boolean failOnMultipleJobs)
                    throws IOException,
                           DatabricksRestException

Description copied from interface: JobService

Will try and get a job by name. If more then one job exists and failOnMultipleJobs set to false, it will return the first job created. Uses a list behind the scenes, so is O(n) with number of jobs.

Specified by:

getJobByName in interface JobService

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

getJobsByRegex

public List<JobDTO> getJobsByRegex(Pattern regex)
                            throws IOException,
                                   DatabricksRestException

Description copied from interface: JobService

Retrieves all jobs with a name matching a given regex. Uses a list behind the scenes, so is O(n) with number of jobs.

Specified by:

getJobsByRegex in interface JobService

Parameters:

regex - - the regex to earch for

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

listAllJobs

public JobsDTO listAllJobs()
                    throws DatabricksRestException,
                           IOException

Description copied from interface: JobService

Returns a list of all jobs that are active.

Specified by:

listAllJobs in interface JobService

Returns:

A POJO of the Jobs

Throws:

DatabricksRestException

IOException

See Also:

https://docs.databricks.com/api/latest/jobs.html#list

getJobLink

public String getJobLink(long jobId)

Description copied from interface: JobService

Produces the URL of a job given job id.

Specified by:

getJobLink in interface JobService

Parameters:

jobId - The canonical identifier of the job to retrieve information about

Returns:

URL of the job

runJobNow

public RunNowDTO runJobNow(long jobId)
                    throws DatabricksRestException,
                           IOException

Description copied from interface: JobService

Runs the job now.

Specified by:

runJobNow in interface JobService

Parameters:

jobId - The job to run

Returns:

Returns the run_id and number_in_run of the triggered run

Throws:

DatabricksRestException

IOException

See Also:

https://docs.databricks.com/api/latest/jobs.html#run-now

runJobNow

public RunNowDTO runJobNow(long jobId,
                           Map<String,String> notebookParams)
                    throws DatabricksRestException,
                           IOException

Description copied from interface: JobService

Runs the job now. With specific notebook params.

Specified by:

runJobNow in interface JobService

Parameters:

jobId - The job to run

notebookParams - A map from keys to values for jobs with notebook task

Returns:

Returns the run_id and number_in_run of the triggered run

Throws:

DatabricksRestException

IOException

See Also:

https://docs.databricks.com/api/latest/jobs.html#run-now

runJobNow

public RunNowDTO runJobNow(long jobId,
                           RunParametersDTO params)
                    throws DatabricksRestException,
                           IOException

Description copied from interface: JobService

Runs the job now. With specific run parameters.

Specified by:

runJobNow in interface JobService

Parameters:

jobId - the job id to run

params - the run parameters

Returns:

the run now object

Throws:

DatabricksRestException - any db specific exceptions

IOException - any other exceptions

See Also:

https://docs.databricks.com/api/latest/jobs.html#run-now

listRuns

public RunsDTO listRuns(Long jobId,
                        Boolean activeOnly,
                        Integer offset,
                        Integer limit)
                 throws DatabricksRestException,
                        IOException

Description copied from interface: JobService

Lists runs from most recently started to least.

Specified by:

listRuns in interface JobService

Parameters:

jobId - The job for which to list runs. If omitted, the Jobs service will list runs from all jobs

activeOnly - If true, lists active runs only; otherwise, lists both active and inactive runs

offset - The offset of the first run to return, relative to the most recent run. The default value is 20

limit - The number of runs to return. This value should be greater than 0 and less than 1000

Returns:

Returns RunsDTO containing an array of runs and a boolean indicating if there are more jobs that haven't been included

Throws:

DatabricksRestException

IOException

See Also:

https://docs.databricks.com/api/latest/jobs.html#runs-list

getRun

public RunDTO getRun(long runId)
              throws DatabricksRestException,
                     IOException

Description copied from interface: JobService

Retrieves the metadata of a run.

Specified by:

getRun in interface JobService

Parameters:

runId - The canonical identifier of the run for which to retrieve the metadata

Returns:

Returns the metadata of the specified run

Throws:

DatabricksRestException

IOException

See Also:

https://docs.databricks.com/api/latest/jobs.html#runs-get

cancelRun

public void cancelRun(long runId)
               throws IOException,
                      DatabricksRestException

Description copied from interface: JobService

Cancels a run. The run is canceled asynchronously, so when this request completes, the run may still be running. The run will be terminated shortly.

Specified by:

cancelRun in interface JobService

Parameters:

runId - The desired run to cancel

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#runs-cancel

reset

public void reset(long jobId,
                  JobSettingsDTO jobSettings)
           throws IOException,
                  DatabricksRestException

Description copied from interface: JobService

"resets" or "edits" a job definition.

Specified by:

reset in interface JobService

Parameters:

jobId - the job to edit

jobSettings - the settings to change the job to

Throws:

IOException - any other errors

DatabricksRestException - any specific db errors

See Also:

https://docs.databricks.com/api/latest/jobs.html#reset

buildRunJobRestUrl

public String buildRunJobRestUrl(long jobId,
                                 long numberInJob)

Description copied from interface: JobService

Formatting databricks job rest url. ex> https://something.cloud.databricks.com/#job/123/run/123

Specified by:

buildRunJobRestUrl in interface JobService

upsertJob

public void upsertJob(JobSettingsDTO jobSettingsDTO,
                      boolean failOnDuplicateJobNames)
               throws IOException,
                      DatabricksRestException

Description copied from interface: JobService

Given a job settings DTO object it will: - try to find the id by name. If multiple exist, it will fail if the duplicate job name flag is set. Else it will update all jobs found. - create the job if it doesn't exist - reset the job if it does exist. Uses a combination of If job doesn't exist:

Specified by:

upsertJob in interface JobService

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/jobs.html#create If job exists:, https://docs.databricks.com/api/latest/jobs.html#reset