Package org.apache.flink.runtime.highavailability

Class AbstractHaServices

    • Field Detail

      • logger

        protected final org.slf4j.Logger logger

      • ioExecutor

        protected final Executor ioExecutor
        The executor to run external IO operations on.

      • configuration

        protected final org.apache.flink.configuration.Configuration configuration
        The runtime configuration.

    • Method Detail

      • getJobManagerLeaderRetriever

        public LeaderRetrievalService getJobManagerLeaderRetriever​(org.apache.flink.api.common.JobID jobID)
        Description copied from interface: HighAvailabilityServices
        Gets the leader retriever for the job JobMaster which is responsible for the given job.
        Specified by:
        getJobManagerLeaderRetriever in interface HighAvailabilityServices
        Parameters:
        jobID - The identifier of the job.
        Returns:
        Leader retrieval service to retrieve the job manager for the given job

      • getJobManagerLeaderRetriever

        public LeaderRetrievalService getJobManagerLeaderRetriever​(org.apache.flink.api.common.JobID jobID,
                                                           String defaultJobManagerAddress)
        Description copied from interface: HighAvailabilityServices
        Gets the leader retriever for the job JobMaster which is responsible for the given job.
        Specified by:
        getJobManagerLeaderRetriever in interface HighAvailabilityServices
        Parameters:
        jobID - The identifier of the job.
        defaultJobManagerAddress - JobManager address which will be returned by a static leader retrieval service.
        Returns:
        Leader retrieval service to retrieve the job manager for the given job

      • close

        public void close()
           throws Exception
        Description copied from interface: HighAvailabilityServices
        Closes the high availability services, releasing all resources.

        This method does not delete or clean up any data stored in external stores (file systems, ZooKeeper, etc). Another instance of the high availability services will be able to recover the job.

        If an exception occurs during closing services, this method will attempt to continue closing other services and report exceptions only after all services have been attempted to be closed.

        Specified by:
        close in interface AutoCloseable
        Specified by:
        close in interface HighAvailabilityServices
        Throws:
        Exception - Thrown, if an exception occurred while closing these services.

      • cleanupAllData

        public void cleanupAllData()
                    throws Exception
        Description copied from interface: HighAvailabilityServices
        Deletes all data stored by high availability services in external stores.

        After this method was called, any job or session that was managed by these high availability services will be unrecoverable.

        If an exception occurs during cleanup, this method will attempt to continue the cleanup and report exceptions only after all cleanup steps have been attempted.

        Specified by:
        cleanupAllData in interface HighAvailabilityServices
        Throws:
        Exception - if an error occurred while cleaning up data stored by them.

      • createLeaderRetrievalService

        protected abstract LeaderRetrievalService createLeaderRetrievalService​(String leaderName)
        Create leader retrieval service with specified leaderName.
        Parameters:
        leaderName - ConfigMap name in Kubernetes or child node path in Zookeeper.
        Returns:
        Return LeaderRetrievalService using Zookeeper or Kubernetes.

      • createCheckpointRecoveryFactory

        protected abstract CheckpointRecoveryFactory createCheckpointRecoveryFactory()
                                                                      throws Exception
        Create the checkpoint recovery factory for the job manager.
        Returns:
        Checkpoint recovery factory
        Throws:
        Exception

      • createExecutionPlanStore

        protected abstract ExecutionPlanStore createExecutionPlanStore()
                                                        throws Exception
        Create the submitted execution plan store for the job manager.
        Returns:
        Submitted execution plan store
        Throws:
        Exception - if the submitted execution plan store could not be created

      • internalClose

        protected abstract void internalClose()
                               throws Exception
        Closes the components which is used for external operations(e.g. Zookeeper Client, Kubernetes Client).
        Throws:
        Exception - if the close operation failed

      • internalCleanup

        protected abstract void internalCleanup()
                                 throws Exception
        Clean up the meta data in the distributed system(e.g. Zookeeper, Kubernetes ConfigMap).

        If an exception occurs during internal cleanup, we will continue the cleanup in cleanupAllData() and report exceptions only after all cleanup steps have been attempted.

        Throws:
        Exception - when do the cleanup operation on external storage.

      • internalCleanupJobData

        protected abstract void internalCleanupJobData​(org.apache.flink.api.common.JobID jobID)
                                        throws Exception
        Clean up the meta data in the distributed system(e.g. Zookeeper, Kubernetes ConfigMap) for the specified Job. Method implementations need to be thread-safe.
        Parameters:
        jobID - The identifier of the job to cleanup.
        Throws:
        Exception - when do the cleanup operation on external storage.

      • getLeaderPathForResourceManager

        protected abstract String getLeaderPathForResourceManager()
        Get the leader path for ResourceManager.
        Returns:
        Return the ResourceManager leader name. It is ConfigMap name in Kubernetes or child node path in Zookeeper.

      • getLeaderPathForDispatcher

        protected abstract String getLeaderPathForDispatcher()
        Get the leader path for Dispatcher.
        Returns:
        Return the Dispatcher leader name. It is ConfigMap name in Kubernetes or child node path in Zookeeper.

      • getLeaderPathForJobManager

        protected abstract String getLeaderPathForJobManager​(org.apache.flink.api.common.JobID jobID)
        Get the leader path for specific JobManager.
        Parameters:
        jobID - job id
        Returns:
        Return the JobManager leader name for specified job id. It is ConfigMap name in Kubernetes or child node path in Zookeeper.

      • getLeaderPathForRestServer

        protected abstract String getLeaderPathForRestServer()
        Get the leader path for RestServer.
        Returns:
        Return the RestServer leader name. It is ConfigMap name in Kubernetes or child node path in Zookeeper.