Package com.databricks.sdk.service.pipelines

Class PipelinesAPI

java.lang.Object

com.databricks.sdk.service.pipelines.PipelinesAPI

@Generated
public class PipelinesAPI
extends Object

The Delta Live Tables API allows you to create, edit, delete, start, and view details about pipelines.

Delta Live Tables is a framework for building reliable, maintainable, and testable data processing pipelines. You define the transformations to perform on your data, and Delta Live Tables manages task orchestration, cluster management, monitoring, data quality, and error handling.

Instead of defining your data pipelines using a series of separate Apache Spark tasks, Delta Live Tables manages how your data is transformed based on a target schema you define for each processing step. You can also enforce data quality with Delta Live Tables expectations. Expectations allow you to define expected data quality and specify how to handle records that fail those expectations.

Constructor Summary

Constructors

Constructor	Description
PipelinesAPI(ApiClient apiClient)	Regular-use constructor
PipelinesAPI(PipelinesService mock)	Constructor for mocks

Method Summary

Modifier and Type	Method	Description
CreatePipelineResponse	create(CreatePipeline request)	Create a pipeline.
void	delete(DeletePipelineRequest request)	Delete a pipeline.
void	delete(String pipelineId)
GetPipelineResponse	get(GetPipelineRequest request)	Get a pipeline.
GetPipelineResponse	get(String pipelineId)
GetUpdateResponse	getUpdate(GetUpdateRequest request)	Get a pipeline update.
GetUpdateResponse	getUpdate(String pipelineId, String updateId)
PipelinesService	impl()
Iterable<PipelineEvent>	listPipelineEvents(ListPipelineEventsRequest request)	List pipeline events.
Iterable<PipelineEvent>	listPipelineEvents(String pipelineId)
Iterable<PipelineStateInfo>	listPipelines(ListPipelinesRequest request)	List pipelines.
ListUpdatesResponse	listUpdates(ListUpdatesRequest request)	List pipeline updates.
ListUpdatesResponse	listUpdates(String pipelineId)
Wait<GetPipelineResponse,Void>	reset(ResetRequest request)	Reset a pipeline.
Wait<GetPipelineResponse,Void>	reset(String pipelineId)
StartUpdateResponse	startUpdate(StartUpdate request)	Queue a pipeline update.
StartUpdateResponse	startUpdate(String pipelineId)
Wait<GetPipelineResponse,Void>	stop(StopRequest request)	Stop a pipeline.
Wait<GetPipelineResponse,Void>	stop(String pipelineId)
void	update(EditPipeline request)	Edit a pipeline.
void	update(String pipelineId)
GetPipelineResponse	waitGetPipelineIdle(String pipelineId)
GetPipelineResponse	waitGetPipelineIdle(String pipelineId, Duration timeout, Consumer<GetPipelineResponse> callback)
GetPipelineResponse	waitGetPipelineRunning(String pipelineId)
GetPipelineResponse	waitGetPipelineRunning(String pipelineId, Duration timeout, Consumer<GetPipelineResponse> callback)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PipelinesAPI

public PipelinesAPI(ApiClient apiClient)

Regular-use constructor

PipelinesAPI

public PipelinesAPI(PipelinesService mock)

Constructor for mocks

Method Detail

waitGetPipelineIdle

public GetPipelineResponse waitGetPipelineIdle(String pipelineId)
                                        throws TimeoutException

Throws:

TimeoutException

waitGetPipelineIdle

public GetPipelineResponse waitGetPipelineIdle(String pipelineId,
                                               Duration timeout,
                                               Consumer<GetPipelineResponse> callback)
                                        throws TimeoutException

Throws:

TimeoutException

waitGetPipelineRunning

public GetPipelineResponse waitGetPipelineRunning(String pipelineId)
                                           throws TimeoutException

Throws:

TimeoutException

waitGetPipelineRunning

public GetPipelineResponse waitGetPipelineRunning(String pipelineId,
                                                  Duration timeout,
                                                  Consumer<GetPipelineResponse> callback)
                                           throws TimeoutException

Throws:

TimeoutException

create

public CreatePipelineResponse create(CreatePipeline request)

Create a pipeline.

Creates a new data processing pipeline based on the requested configuration. If successful, this method returns the ID of the new pipeline.

delete

public void delete(String pipelineId)

delete

public void delete(DeletePipelineRequest request)

Delete a pipeline.

Deletes a pipeline.

get

public GetPipelineResponse get(String pipelineId)

get

public GetPipelineResponse get(GetPipelineRequest request)

Get a pipeline.

getUpdate

public GetUpdateResponse getUpdate(String pipelineId,
                                   String updateId)

getUpdate

public GetUpdateResponse getUpdate(GetUpdateRequest request)

Get a pipeline update.

Gets an update from an active pipeline.

listPipelineEvents

public Iterable<PipelineEvent> listPipelineEvents(String pipelineId)

listPipelineEvents

public Iterable<PipelineEvent> listPipelineEvents(ListPipelineEventsRequest request)

List pipeline events.

Retrieves events for a pipeline.

listPipelines

public Iterable<PipelineStateInfo> listPipelines(ListPipelinesRequest request)

List pipelines.

Lists pipelines defined in the Delta Live Tables system.

listUpdates

public ListUpdatesResponse listUpdates(String pipelineId)

listUpdates

public ListUpdatesResponse listUpdates(ListUpdatesRequest request)

List pipeline updates.

List updates for an active pipeline.

reset

public Wait<GetPipelineResponse,Void> reset(String pipelineId)

reset

public Wait<GetPipelineResponse,Void> reset(ResetRequest request)

Reset a pipeline.

Resets a pipeline.

startUpdate

public StartUpdateResponse startUpdate(String pipelineId)

startUpdate

public StartUpdateResponse startUpdate(StartUpdate request)

Queue a pipeline update.

Starts or queues a pipeline update.

stop

public Wait<GetPipelineResponse,Void> stop(String pipelineId)

stop

public Wait<GetPipelineResponse,Void> stop(StopRequest request)

Stop a pipeline.

Stops a pipeline.

update

public void update(String pipelineId)

update

public void update(EditPipeline request)

Edit a pipeline.

Updates a pipeline with the supplied configuration.

impl

public PipelinesService impl()