Package org.apache.flink.client.program.rest

Class RestClusterClient<T>

    • Constructor Summary

      Constructors 
      Constructor Description
      RestClusterClient​(org.apache.flink.configuration.Configuration config, T clusterId)  
      RestClusterClient​(org.apache.flink.configuration.Configuration config, T clusterId, org.apache.flink.runtime.highavailability.ClientHighAvailabilityServicesFactory factory)  

    • Constructor Detail

      • RestClusterClient

        public RestClusterClient​(org.apache.flink.configuration.Configuration config,
                         T clusterId)
                  throws Exception
        Throws:
        Exception

      • RestClusterClient

        public RestClusterClient​(org.apache.flink.configuration.Configuration config,
                         T clusterId,
                         org.apache.flink.runtime.highavailability.ClientHighAvailabilityServicesFactory factory)
                  throws Exception
        Throws:
        Exception

    • Method Detail

      • getFlinkConfiguration

        public org.apache.flink.configuration.Configuration getFlinkConfiguration()
        Description copied from interface: ClusterClient
        Return the Flink configuration object.
        Specified by:
        getFlinkConfiguration in interface ClusterClient<T>
        Returns:
        The Flink configuration object

      • getJobDetails

        public CompletableFuture<org.apache.flink.runtime.rest.messages.job.JobDetailsInfo> getJobDetails​(org.apache.flink.api.common.JobID jobId)
        Requests the job details.
        Parameters:
        jobId - The job id
        Returns:
        Job details

      • getJobStatus

        public CompletableFuture<org.apache.flink.api.common.JobStatus> getJobStatus​(org.apache.flink.api.common.JobID jobId)
        Description copied from interface: ClusterClient
        Requests the JobStatus of the job with the given JobID.
        Specified by:
        getJobStatus in interface ClusterClient<T>

      • requestJobResult

        public CompletableFuture<org.apache.flink.runtime.jobmaster.JobResult> requestJobResult​(@Nonnull
                                                                                        org.apache.flink.api.common.JobID jobId)
        Requests the JobResult for the given JobID. The method retries multiple times to poll the JobResult before giving up.
        Specified by:
        requestJobResult in interface ClusterClient<T>
        Parameters:
        jobId - specifying the job for which to retrieve the JobResult
        Returns:
        Future which is completed with the JobResult once the job has completed or with a failure if the JobResult could not be retrieved.

      • submitJob

        public CompletableFuture<org.apache.flink.api.common.JobID> submitJob​(@Nonnull
                                                                      org.apache.flink.streaming.api.graph.ExecutionPlan executionPlan)
        Description copied from interface: ClusterClient
        Submit the given ExecutionPlan to the cluster.
        Specified by:
        submitJob in interface ClusterClient<T>
        Parameters:
        executionPlan - to submit
        Returns:
        JobID of the submitted job

      • cancel

        public CompletableFuture<org.apache.flink.runtime.messages.Acknowledge> cancel​(org.apache.flink.api.common.JobID jobID)
        Description copied from interface: ClusterClient
        Cancels a job identified by the job id.
        Specified by:
        cancel in interface ClusterClient<T>
        Parameters:
        jobID - the job id

      • stopWithSavepoint

        public CompletableFuture<String> stopWithSavepoint​(org.apache.flink.api.common.JobID jobId,
                                                   boolean advanceToEndOfTime,
                                                   @Nullable
                                                   String savepointDirectory,
                                                   org.apache.flink.core.execution.SavepointFormatType formatType)
        Description copied from interface: ClusterClient
        Stops a program on Flink cluster whose job-manager is configured in this client's configuration. Stopping works only for streaming programs. Be aware, that the program might continue to run for a while after sending the stop command, because after sources stopped to emit data all operators need to finish processing.
        Specified by:
        stopWithSavepoint in interface ClusterClient<T>
        Parameters:
        jobId - the job ID of the streaming program to stop
        advanceToEndOfTime - flag indicating if the source should inject a MAX_WATERMARK in the pipeline
        savepointDirectory - directory the savepoint should be written to
        formatType - a binary format of the savepoint
        Returns:
        a CompletableFuture containing the path where the savepoint is located

      • stopWithDetachedSavepoint

        public CompletableFuture<String> stopWithDetachedSavepoint​(org.apache.flink.api.common.JobID jobId,
                                                           boolean advanceToEndOfTime,
                                                           @Nullable
                                                           String savepointDirectory,
                                                           org.apache.flink.core.execution.SavepointFormatType formatType)
        Description copied from interface: ClusterClient
        Stops a program on Flink cluster whose job-manager is configured in this client's configuration. Stopping works only for streaming programs. Be aware, that the program might continue to run for a while after sending the stop command, because after sources stopped to emit data all operators need to finish processing.
        Specified by:
        stopWithDetachedSavepoint in interface ClusterClient<T>
        Parameters:
        jobId - the job ID of the streaming program to stop
        advanceToEndOfTime - flag indicating if the source should inject a MAX_WATERMARK in the pipeline
        savepointDirectory - directory the savepoint should be written to
        formatType - a binary format of the savepoint
        Returns:
        the savepoint trigger id

      • cancelWithSavepoint

        public CompletableFuture<String> cancelWithSavepoint​(org.apache.flink.api.common.JobID jobId,
                                                     @Nullable
                                                     String savepointDirectory,
                                                     org.apache.flink.core.execution.SavepointFormatType formatType)
        Description copied from interface: ClusterClient
        Cancels a job identified by the job id and triggers a savepoint.
        Specified by:
        cancelWithSavepoint in interface ClusterClient<T>
        Parameters:
        jobId - the job id
        savepointDirectory - directory the savepoint should be written to
        formatType - a binary format of the savepoint
        Returns:
        future of path where the savepoint is located

      • triggerSavepoint

        public CompletableFuture<String> triggerSavepoint​(org.apache.flink.api.common.JobID jobId,
                                                  @Nullable
                                                  String savepointDirectory,
                                                  org.apache.flink.core.execution.SavepointFormatType formatType)
        Description copied from interface: ClusterClient
        Triggers a savepoint for the job identified by the job id. The savepoint will be written to the given savepoint directory, or CheckpointingOptions.SAVEPOINT_DIRECTORY if it is null.
        Specified by:
        triggerSavepoint in interface ClusterClient<T>
        Parameters:
        jobId - job id
        savepointDirectory - directory the savepoint should be written to
        formatType - a binary format of the savepoint
        Returns:
        path future where the savepoint is located

      • triggerCheckpoint

        public CompletableFuture<Long> triggerCheckpoint​(org.apache.flink.api.common.JobID jobId,
                                                 org.apache.flink.core.execution.CheckpointType checkpointType)
        Description copied from interface: ClusterClient
        Triggers a checkpoint for the job identified by the job id. The checkpoint will be written to the checkpoint directory for the job.
        Specified by:
        triggerCheckpoint in interface ClusterClient<T>
        Parameters:
        jobId - job id
        checkpointType - the checkpoint type (configured / full / incremental)

      • triggerDetachedSavepoint

        public CompletableFuture<String> triggerDetachedSavepoint​(org.apache.flink.api.common.JobID jobId,
                                                          @Nullable
                                                          String savepointDirectory,
                                                          org.apache.flink.core.execution.SavepointFormatType formatType)
        Description copied from interface: ClusterClient
        Triggers a detached savepoint for the job identified by the job id. The savepoint will be written to the given savepoint directory, or CheckpointingOptions.SAVEPOINT_DIRECTORY if it is null. Notice that: the detached savepoint will return with a savepoint trigger id instead of the path future, that means the client will return very quickly.
        Specified by:
        triggerDetachedSavepoint in interface ClusterClient<T>
        Parameters:
        jobId - job id
        savepointDirectory - directory the savepoint should be written to
        formatType - a binary format of the savepoint
        Returns:
        the savepoint trigger id

      • sendCoordinationRequest

        public CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse> sendCoordinationRequest​(org.apache.flink.api.common.JobID jobId,
                                                                                                                       String operatorUid,
                                                                                                                       org.apache.flink.runtime.operators.coordination.CoordinationRequest request)
        Description copied from interface: ClusterClient
        Sends out a request to a specified coordinator and return the response.

        On the client side, a unique operatorUid must be defined to identify an operator. Otherwise, the query cannot be executed correctly. Note that we use operatorUid instead of operatorID because the latter is an internal runtime concept that cannot be recognized by the client.

        Specified by:
        sendCoordinationRequest in interface ClusterClient<T>
        Parameters:
        jobId - specifies the job which the coordinator belongs to
        operatorUid - specifies which coordinator to receive the request
        request - the request to send
        Returns:
        the response from the coordinator

      • stopWithSavepoint

        public CompletableFuture<String> stopWithSavepoint​(org.apache.flink.api.common.JobID jobId,
                                                   boolean advanceToEndOfTime,
                                                   @Nullable
                                                   String savepointDirectory,
                                                   org.apache.flink.core.execution.SavepointFormatType formatType,
                                                   boolean isDetachedMode)

      • getAccumulators

        public CompletableFuture<Map<String,​Object>> getAccumulators​(org.apache.flink.api.common.JobID jobID,
                                                                   ClassLoader loader)
        Description copied from interface: ClusterClient
        Requests and returns the accumulators for the given job identifier. Accumulators can be requested while a is running or after it has finished.
        Specified by:
        getAccumulators in interface ClusterClient<T>
        Parameters:
        jobID - The job identifier of a job.
        loader - The class loader for deserializing the accumulator results.
        Returns:
        A Map containing the accumulator's name and its value.

      • listJobs

        public CompletableFuture<Collection<org.apache.flink.runtime.client.JobStatusMessage>> listJobs()
        Description copied from interface: ClusterClient
        Lists the currently running and finished jobs on the cluster.
        Specified by:
        listJobs in interface ClusterClient<T>
        Returns:
        future collection of running and finished jobs

      • getClusterId

        public T getClusterId()
        Description copied from interface: ClusterClient
        Returns the cluster id identifying the cluster to which the client is connected.
        Specified by:
        getClusterId in interface ClusterClient<T>
        Returns:
        cluster id of the connected cluster

      • disposeSavepoint

        public CompletableFuture<org.apache.flink.runtime.messages.Acknowledge> disposeSavepoint​(String savepointPath)
        Description copied from interface: ClusterClient
        Dispose the savepoint under the given path.
        Specified by:
        disposeSavepoint in interface ClusterClient<T>
        Parameters:
        savepointPath - path to the savepoint to be disposed
        Returns:
        acknowledge future of the dispose action

      • invalidateClusterDataset

        public CompletableFuture<Void> invalidateClusterDataset​(org.apache.flink.util.AbstractID clusterDatasetId)
        Description copied from interface: ClusterClient
        Invalidate the cached intermediate dataset with the given id.
        Specified by:
        invalidateClusterDataset in interface ClusterClient<T>
        Parameters:
        clusterDatasetId - id of the cluster dataset to be invalidated.
        Returns:
        Future which will be completed when the cached dataset is invalidated.

      • reportHeartbeat

        public CompletableFuture<Void> reportHeartbeat​(org.apache.flink.api.common.JobID jobId,
                                               long expiredTimestamp)
        Description copied from interface: ClusterClient
        The client reports the heartbeat to the dispatcher for aliveness.
        Specified by:
        reportHeartbeat in interface ClusterClient<T>
        Parameters:
        jobId - The jobId for the client and the job.
        Returns:

      • shutDownCluster

        public void shutDownCluster()
        Description copied from interface: ClusterClient
        Shut down the cluster that this client communicate with.
        Specified by:
        shutDownCluster in interface ClusterClient<T>

      • updateJobResourceRequirements

        public CompletableFuture<org.apache.flink.runtime.messages.Acknowledge> updateJobResourceRequirements​(org.apache.flink.api.common.JobID jobId,
                                                                                                      org.apache.flink.runtime.jobgraph.JobResourceRequirements jobResourceRequirements)
        Update JobResourceRequirements of a given job.
        Parameters:
        jobId - jobId specifies the job for which to change the resource requirements
        jobResourceRequirements - new resource requirements for the provided job
        Returns:
        Future which is completed upon successful operation.

      • getClusterOverview

        public CompletableFuture<org.apache.flink.runtime.rest.handler.legacy.messages.ClusterOverviewWithVersion> getClusterOverview()
        Get an overview of the Flink cluster.
        Returns:
        Future with the cluster overview.

      • sendRequest

        @VisibleForTesting
public <M extends org.apache.flink.runtime.rest.messages.MessageHeaders<R,​P,​U>,​U extends org.apache.flink.runtime.rest.messages.MessageParameters,​R extends org.apache.flink.runtime.rest.messages.RequestBody,​P extends org.apache.flink.runtime.rest.messages.ResponseBody> CompletableFuture<P> sendRequest​(M messageHeaders,
                                                                                                                                                                                                                                                                                                                                             U messageParameters,
                                                                                                                                                                                                                                                                                                                                             R request)