Package org.apache.druid.frame.util

Class DurableStorageUtils

  • public class DurableStorageUtils
extends Object
    Helper class that fetches the directory and file names corresponding to file location

    • Constructor Detail

      • DurableStorageUtils

        public DurableStorageUtils()

    • Method Detail

      • getControllerDirectory

        public static String getControllerDirectory​(String controllerTaskId)

      • getWorkerOutputSuccessFilePath

        public static String getWorkerOutputSuccessFilePath​(String controllerTaskId,
                                                    int stageNumber,
                                                    int workerNumber)

      • getQueryResultsSuccessFilePath

        public static String getQueryResultsSuccessFilePath​(String controllerTaskId,
                                                    int stageNumber,
                                                    int workerNumber)

      • getTaskIdOutputsFolderName

        public static String getTaskIdOutputsFolderName​(String controllerTaskId,
                                                int stageNumber,
                                                int workerNumber,
                                                String taskId)
        Fetches the directory location where a particular worker will store the partition files corresponding to the stage number, and it's task id

      • getQueryResultsForTaskIdFolderName

        public static String getQueryResultsForTaskIdFolderName​(String controllerTaskId,
                                                        int stageNumber,
                                                        int workerNumber,
                                                        String taskId)

      • getPartitionOutputsFileNameWithPathForPartition

        public static String getPartitionOutputsFileNameWithPathForPartition​(String controllerTaskId,
                                                                     int stageNumber,
                                                                     int workerNumber,
                                                                     String taskId,
                                                                     int partitionNumber)
        Fetches the file location where a particular worker writes the data corresponding to a particular stage and partition

      • getQueryResultsFileNameWithPathForPartition

        public static String getQueryResultsFileNameWithPathForPartition​(String controllerTaskId,
                                                                 int stageNumber,
                                                                 int workerNumber,
                                                                 String taskId,
                                                                 int partitionNumber)

      • getOutputsFileNameForPath

        public static String getOutputsFileNameForPath​(String controllerTaskId,
                                               int stageNumber,
                                               int workerNumber,
                                               String taskId,
                                               String path)
        Fetches the file location where a particular worker writes the data corresponding to a particular stage and a custom path name

      • getNextDirNameWithPrefixFromPath

        @Nullable
public static String getNextDirNameWithPrefixFromPath​(String path)
        Tries to parse out the most top level directory from the path. Returns null if there is no such directory.

        For eg:
        • for input path controller_query_id/task/123 the function will return controller_query_id
        • for input path abcd, the function will return abcd
        • for input path null, the function will return null

      • isQueryResultFileActive

        public static boolean isQueryResultFileActive​(String path,
                                              Set<String> knownTasks)
        Tries to parse out the controller taskID from the query results path, and checks if the taskID is present in the set of known tasks. Returns true if the set contains the taskId. Returns false if taskId could not be parsed or if the set does not contain the taskId.

        For eg:
        • for path controller_query_id/task/123 the function will return false
        • for path query-result/controller_query_id/results.json, the function will return true
          • if the controller_query_id is in known tasks
        • for path query-result/controller_query_id/results.json, the function will return false
          • if the controller_query_id is not in known tasks
        • for path null, the function will return false