Interface ClustersService
-
@Generated public interface ClustersService
The Clusters API allows you to create, start, edit, list, terminate, and delete clusters.Databricks maps cluster node instance types to compute units known as DBUs. See the instance type pricing page for a list of the supported instance types and their corresponding DBUs.
A Databricks cluster is a set of computation resources and configurations on which you run data engineering, data science, and data analytics workloads, such as production ETL pipelines, streaming analytics, ad-hoc analytics, and machine learning.
You run these workloads as a set of commands in a notebook or as an automated job. Databricks makes a distinction between all-purpose clusters and job clusters. You use all-purpose clusters to analyze data collaboratively using interactive notebooks. You use job clusters to run fast and robust automated jobs.
You can create an all-purpose cluster using the UI, CLI, or REST API. You can manually terminate and restart an all-purpose cluster. Multiple users can share such clusters to do collaborative interactive analysis.
IMPORTANT: Databricks retains cluster configuration information for up to 200 all-purpose clusters terminated in the last 30 days and up to 30 job clusters recently terminated by the job scheduler. To keep an all-purpose cluster configuration even after it has been terminated for more than 30 days, an administrator can pin a cluster to the cluster list.
This is the high-level interface, that contains generated methods.
Evolving: this interface is under development. Method signatures may change.
-
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description void
changeOwner(ChangeClusterOwner changeClusterOwner)
Change cluster owner.CreateClusterResponse
create(CreateCluster createCluster)
Create new cluster.void
delete(DeleteCluster deleteCluster)
Terminate cluster.void
edit(EditCluster editCluster)
Update cluster configuration.GetEventsResponse
events(GetEvents getEvents)
List cluster activity events.ClusterDetails
get(GetClusterRequest getClusterRequest)
Get cluster info.ListClustersResponse
list(ListClustersRequest listClustersRequest)
List all clusters.ListNodeTypesResponse
listNodeTypes()
List node types.ListAvailableZonesResponse
listZones()
List availability zones.void
permanentDelete(PermanentDeleteCluster permanentDeleteCluster)
Permanently delete cluster.void
pin(PinCluster pinCluster)
Pin cluster.void
resize(ResizeCluster resizeCluster)
Resize cluster.void
restart(RestartCluster restartCluster)
Restart cluster.GetSparkVersionsResponse
sparkVersions()
List available Spark versions.void
start(StartCluster startCluster)
Start terminated cluster.void
unpin(UnpinCluster unpinCluster)
Unpin cluster.
-
-
-
Method Detail
-
changeOwner
void changeOwner(ChangeClusterOwner changeClusterOwner)
Change cluster owner.Change the owner of the cluster. You must be an admin to perform this operation.
-
create
CreateClusterResponse create(CreateCluster createCluster)
Create new cluster.Creates a new Spark cluster. This method will acquire new instances from the cloud provider if necessary. Note: Databricks may not be able to acquire some of the requested nodes, due to cloud provider limitations (account limits, spot price, etc.) or transient network issues.
If Databricks acquires at least 85% of the requested on-demand nodes, cluster creation will succeed. Otherwise the cluster will terminate with an informative error message.
-
delete
void delete(DeleteCluster deleteCluster)
Terminate cluster.Terminates the Spark cluster with the specified ID. The cluster is removed asynchronously. Once the termination has completed, the cluster will be in a `TERMINATED` state. If the cluster is already in a `TERMINATING` or `TERMINATED` state, nothing will happen.
-
edit
void edit(EditCluster editCluster)
Update cluster configuration.Updates the configuration of a cluster to match the provided attributes and size. A cluster can be updated if it is in a `RUNNING` or `TERMINATED` state.
If a cluster is updated while in a `RUNNING` state, it will be restarted so that the new attributes can take effect.
If a cluster is updated while in a `TERMINATED` state, it will remain `TERMINATED`. The next time it is started using the `clusters/start` API, the new attributes will take effect. Any attempt to update a cluster in any other state will be rejected with an `INVALID_STATE` error code.
Clusters created by the Databricks Jobs service cannot be edited.
-
events
GetEventsResponse events(GetEvents getEvents)
List cluster activity events.Retrieves a list of events about the activity of a cluster. This API is paginated. If there are more events to read, the response includes all the nparameters necessary to request the next page of events.
-
get
ClusterDetails get(GetClusterRequest getClusterRequest)
Get cluster info.Retrieves the information for a cluster given its identifier. Clusters can be described while they are running, or up to 60 days after they are terminated.
-
list
ListClustersResponse list(ListClustersRequest listClustersRequest)
List all clusters.Return information about all pinned clusters, active clusters, up to 200 of the most recently terminated all-purpose clusters in the past 30 days, and up to 30 of the most recently terminated job clusters in the past 30 days.
For example, if there is 1 pinned cluster, 4 active clusters, 45 terminated all-purpose clusters in the past 30 days, and 50 terminated job clusters in the past 30 days, then this API returns the 1 pinned cluster, 4 active clusters, all 45 terminated all-purpose clusters, and the 30 most recently terminated job clusters.
-
listNodeTypes
ListNodeTypesResponse listNodeTypes()
List node types.Returns a list of supported Spark node types. These node types can be used to launch a cluster.
-
listZones
ListAvailableZonesResponse listZones()
List availability zones.Returns a list of availability zones where clusters can be created in (For example, us-west-2a). These zones can be used to launch a cluster.
-
permanentDelete
void permanentDelete(PermanentDeleteCluster permanentDeleteCluster)
Permanently delete cluster.Permanently deletes a Spark cluster. This cluster is terminated and resources are asynchronously removed.
In addition, users will no longer see permanently deleted clusters in the cluster list, and API users can no longer perform any action on permanently deleted clusters.
-
pin
void pin(PinCluster pinCluster)
Pin cluster.Pinning a cluster ensures that the cluster will always be returned by the ListClusters API. Pinning a cluster that is already pinned will have no effect. This API can only be called by workspace admins.
-
resize
void resize(ResizeCluster resizeCluster)
Resize cluster.Resizes a cluster to have a desired number of workers. This will fail unless the cluster is in a `RUNNING` state.
-
restart
void restart(RestartCluster restartCluster)
Restart cluster.Restarts a Spark cluster with the supplied ID. If the cluster is not currently in a `RUNNING` state, nothing will happen.
-
sparkVersions
GetSparkVersionsResponse sparkVersions()
List available Spark versions.Returns the list of available Spark versions. These versions can be used to launch a cluster.
-
start
void start(StartCluster startCluster)
Start terminated cluster.Starts a terminated Spark cluster with the supplied ID. This works similar to `createCluster` except:
* The previous cluster id and attributes are preserved. * The cluster starts with the last specified cluster size. * If the previous cluster was an autoscaling cluster, the current cluster starts with the minimum number of nodes. * If the cluster is not currently in a `TERMINATED` state, nothing will happen. * Clusters launched to run a job cannot be started.
-
unpin
void unpin(UnpinCluster unpinCluster)
Unpin cluster.Unpinning a cluster will allow the cluster to eventually be removed from the ListClusters API. Unpinning a cluster that is not pinned will have no effect. This API can only be called by workspace admins.
-
-