Package org.apache.flink.runtime.util

Class ZooKeeperUtils

  • public class ZooKeeperUtils
extends Object
    Class containing helper functions to interact with ZooKeeper.

    • Field Detail

      • HA_STORAGE_SUBMITTED_EXECUTION_PLAN_PREFIX

        public static final String HA_STORAGE_SUBMITTED_EXECUTION_PLAN_PREFIX
        The prefix of the submitted job graph file.
        See Also:
        Constant Field Values

      • HA_STORAGE_COMPLETED_CHECKPOINT

        public static final String HA_STORAGE_COMPLETED_CHECKPOINT
        The prefix of the completed checkpoint file.
        See Also:
        Constant Field Values

      • RESOURCE_MANAGER_NODE

        public static final String RESOURCE_MANAGER_NODE
        The prefix of the resource manager node.
        See Also:
        Constant Field Values

    • Method Detail

      • getLeaderPathForJob

        public static String getLeaderPathForJob​(org.apache.flink.api.common.JobID jobId)

      • getJobsPath

        public static String getJobsPath()

      • getCheckpointIdCounterPath

        public static String getCheckpointIdCounterPath()

      • getLeaderPath

        public static String getLeaderPath()

      • getDispatcherNode

        public static String getDispatcherNode()

      • getResourceManagerNode

        public static String getResourceManagerNode()

      • getRestServerNode

        public static String getRestServerNode()

      • getLeaderLatchPath

        public static String getLeaderLatchPath()

      • getLeaderPath

        public static String getLeaderPath​(String suffix)

      • generateConnectionInformationPath

        public static String generateConnectionInformationPath​(String path)

      • isConnectionInfoPath

        public static boolean isConnectionInfoPath​(String path)

      • generateLeaderLatchPath

        public static String generateLeaderLatchPath​(String path)

      • startCuratorFramework

        public static CuratorFrameworkWithUnhandledErrorListener startCuratorFramework​(org.apache.flink.configuration.Configuration configuration,
                                                                               org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler)
        Starts a CuratorFramework instance and connects it to the given ZooKeeper quorum.
        Parameters:
        configuration - Configuration object containing the configuration values
        fatalErrorHandler - FatalErrorHandler fatalErrorHandler to handle unexpected errors of CuratorFramework
        Returns:
        CuratorFrameworkWithUnhandledErrorListener instance

      • startCuratorFramework

        @VisibleForTesting
public static CuratorFrameworkWithUnhandledErrorListener startCuratorFramework​(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFrameworkFactory.Builder builder,
                                                                               org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler)
        Starts a CuratorFramework instance and connects it to the given ZooKeeper quorum from a builder.
        Parameters:
        builder - CuratorFrameworkFactory.Builder A builder for curatorFramework.
        fatalErrorHandler - FatalErrorHandler fatalErrorHandler to handle unexpected errors of CuratorFramework
        Returns:
        CuratorFrameworkWithUnhandledErrorListener instance

      • isZooKeeperRecoveryMode

        public static boolean isZooKeeperRecoveryMode​(org.apache.flink.configuration.Configuration flinkConf)
        Returns whether HighAvailabilityMode.ZOOKEEPER is configured.

      • getZooKeeperEnsemble

        public static String getZooKeeperEnsemble​(org.apache.flink.configuration.Configuration flinkConf)
                                   throws org.apache.flink.configuration.IllegalConfigurationException
        Returns the configured ZooKeeper quorum (and removes whitespace, because ZooKeeper does not tolerate it).
        Throws:
        org.apache.flink.configuration.IllegalConfigurationException

      • createLeaderRetrievalDriverFactory

        public static ZooKeeperLeaderRetrievalDriverFactory createLeaderRetrievalDriverFactory​(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework client,
                                                                                       String path,
                                                                                       org.apache.flink.configuration.Configuration configuration)
        Creates a LeaderRetrievalDriverFactory implemented by ZooKeeper.
        Parameters:
        client - The CuratorFramework ZooKeeper client to use
        path - The path for the leader zNode
        configuration - configuration for further config options
        Returns:
        LeaderRetrievalDriverFactory instance.

      • writeLeaderInformationToZooKeeper

        public static void writeLeaderInformationToZooKeeper​(LeaderInformation leaderInformation,
                                                     org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework curatorFramework,
                                                     BooleanSupplier hasLeadershipCheck,
                                                     String connectionInformationPath)
                                              throws Exception
        Throws:
        Exception

      • createCompletedCheckpoints

        public static CompletedCheckpointStore createCompletedCheckpoints​(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework client,
                                                                  org.apache.flink.configuration.Configuration configuration,
                                                                  int maxNumberOfCheckpointsToRetain,
                                                                  SharedStateRegistryFactory sharedStateRegistryFactory,
                                                                  Executor ioExecutor,
                                                                  Executor executor,
                                                                  org.apache.flink.core.execution.RecoveryClaimMode recoveryClaimMode)
                                                           throws Exception
        Creates a DefaultCompletedCheckpointStore instance with ZooKeeperStateHandleStore.
        Parameters:
        client - The CuratorFramework ZooKeeper client to use
        configuration - Configuration object
        maxNumberOfCheckpointsToRetain - The maximum number of checkpoints to retain
        executor - to run ZooKeeper callbacks
        recoveryClaimMode - the mode in which the job is being restored
        Returns:
        DefaultCompletedCheckpointStore instance
        Throws:
        Exception - if the completed checkpoint store cannot be created

      • getPathForJob

        public static String getPathForJob​(org.apache.flink.api.common.JobID jobId)
        Returns the JobID as a String (with leading slash).

      • generateZookeeperPath

        public static String generateZookeeperPath​(String... paths)
        Creates a ZooKeeper path of the form "/a/b/.../z".

      • splitZooKeeperPath

        public static String[] splitZooKeeperPath​(String path)
        Splits the given ZooKeeper path into its parts.
        Parameters:
        path - path to split
        Returns:
        splited path

      • trimStartingSlash

        public static String trimStartingSlash​(String path)

      • useNamespaceAndEnsurePath

        public static org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework useNamespaceAndEnsurePath​(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework client,
                                                                                                                       String path)
                                                                                                                throws Exception
        Returns a facade of the client that uses the specified namespace, and ensures that all nodes in the path exist.
        Parameters:
        client - ZK client
        path - the new namespace
        Returns:
        ZK Client that uses the new namespace
        Throws:
        Exception - ZK errors

      • createTreeCache

        public static org.apache.flink.shaded.curator5.org.apache.curator.framework.recipes.cache.TreeCache createTreeCache​(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework client,
                                                                                                                    String pathToNode,
                                                                                                                    org.apache.flink.util.function.RunnableWithException nodeChangeCallback)
        Creates a TreeCache that only observes a specific node.
        Parameters:
        client - ZK client
        pathToNode - full path of the node to observe
        nodeChangeCallback - callback to run if the node has changed
        Returns:
        tree cache

      • createTreeCache

        public static org.apache.flink.shaded.curator5.org.apache.curator.framework.recipes.cache.TreeCache createTreeCache​(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework client,
                                                                                                                    String pathToNode,
                                                                                                                    org.apache.flink.shaded.curator5.org.apache.curator.framework.recipes.cache.TreeCacheSelector selector)

      • deleteZNode

        public static void deleteZNode​(org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFramework curatorFramework,
                               String path)
                        throws Exception
        Throws:
        Exception