org.apache.drill.exec.util

Class DrillFileSystemUtil

java.lang.Object

org.apache.drill.exec.util.DrillFileSystemUtil

public class DrillFileSystemUtil
extends Object

In Drill file system all directories and files that start with dot or underscore is ignored. This helper class that delegates all work to list directory and file statuses to FileSystemUtil class, only adding Drill file system filter first.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.fs.PathFilter	DRILL_SYSTEM_FILTER Path filter that skips all files and folders that start with dot or underscore.

Constructor Summary

Constructors

Constructor and Description
DrillFileSystemUtil()

Method Summary

Modifier and Type	Method and Description
static org.apache.hadoop.fs.Path	createPathSafe(String path) Safely creates Hadoop Path for null and empty String paths
static List<org.apache.hadoop.fs.FileStatus>	listAll(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listAllSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listDirectories(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listFilesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DRILL_SYSTEM_FILTER

public static final org.apache.hadoop.fs.PathFilter DRILL_SYSTEM_FILTER

Path filter that skips all files and folders that start with dot or underscore.

Constructor Detail

DrillFileSystemUtil

public DrillFileSystemUtil()

Method Detail

listDirectories

public static List<org.apache.hadoop.fs.FileStatus> listDirectories(org.apache.hadoop.fs.FileSystem fs,
                                                                    org.apache.hadoop.fs.Path path,
                                                                    boolean recursive,
                                                                    org.apache.hadoop.fs.PathFilter... filters)
                                                             throws IOException

Returns statuses of all directories present in given path applying custom filters if present. Directories that start with dot or underscore are skipped. Will also include nested directories if recursive flag is set to true.

Parameters:

fs - current file system

path - path to directory

recursive - true if nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching directory statuses

Throws:

IOException

listDirectoriesSafe

public static List<org.apache.hadoop.fs.FileStatus> listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs,
                                                                        org.apache.hadoop.fs.Path path,
                                                                        boolean recursive,
                                                                        org.apache.hadoop.fs.PathFilter... filters)

Returns statuses of all directories present in given path applying custom filters if present. Directories that start with dot or underscore are skipped. Will include nested directories if recursive flag is set to true. Will ignore all exceptions during listing if any.

Parameters:

fs - current file system

path - path to directory

recursive - true if nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching directory statuses

listFiles

public static List<org.apache.hadoop.fs.FileStatus> listFiles(org.apache.hadoop.fs.FileSystem fs,
                                                              org.apache.hadoop.fs.Path path,
                                                              boolean recursive,
                                                              org.apache.hadoop.fs.PathFilter... filters)
                                                       throws IOException

Returns statuses of all files present in given path applying custom filters if present. Files and nested directories that start with dot or underscore are skipped. Will also include files from nested directories if recursive flag is set to true.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if files in nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching file statuses

Throws:

IOException

listFilesSafe

public static List<org.apache.hadoop.fs.FileStatus> listFilesSafe(org.apache.hadoop.fs.FileSystem fs,
                                                                  org.apache.hadoop.fs.Path path,
                                                                  boolean recursive,
                                                                  org.apache.hadoop.fs.PathFilter... filters)

Returns statuses of all files present in given path applying custom filters if present. Files and nested directories that start with dot or underscore are skipped. Will include files from nested directories if recursive flag is set to true. Will ignore all exceptions during listing if any.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if files in nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching file statuses

listAll

public static List<org.apache.hadoop.fs.FileStatus> listAll(org.apache.hadoop.fs.FileSystem fs,
                                                            org.apache.hadoop.fs.Path path,
                                                            boolean recursive,
                                                            org.apache.hadoop.fs.PathFilter... filters)
                                                     throws IOException

Returns statuses of all directories and files present in given path applying custom filters if present. Directories and files that start with dot or underscore are skipped. Will also include nested directories and their files if recursive flag is set to true.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if nested directories and their files should be included

filters - list of custom filters (optional)

Returns:

list of matching directory and file statuses

Throws:

IOException

listAllSafe

public static List<org.apache.hadoop.fs.FileStatus> listAllSafe(org.apache.hadoop.fs.FileSystem fs,
                                                                org.apache.hadoop.fs.Path path,
                                                                boolean recursive,
                                                                org.apache.hadoop.fs.PathFilter... filters)

Returns statuses of all directories and files present in given path applying custom filters if present. Directories and files that start with dot or underscore are skipped. Will include nested directories and their files if recursive flag is set to true. Will ignore all exceptions during listing if any.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if nested directories and their files should be included

filters - list of custom filters (optional)

Returns:

list of matching directory and file statuses

createPathSafe

public static org.apache.hadoop.fs.Path createPathSafe(String path)

Safely creates Hadoop Path for null and empty String paths

Parameters:

path - String path, which can be null or empty

Returns:

Hadoop Path. Root - for empty or null path