public class HarFileSystem extends FileSystem
FileSystem.Statistics
Modifier and Type | Field and Description |
---|---|
static int |
METADATA_CACHE_ENTRIES_DEFAULT |
static String |
METADATA_CACHE_ENTRIES_KEY |
static int |
VERSION |
DEFAULT_FS, FS_DEFAULT_NAME_KEY, SHUTDOWN_HOOK_PRIORITY, statistics, TRASH_PREFIX, USER_HOME_PREFIX
Constructor and Description |
---|
HarFileSystem()
public construction of harfilesystem
|
HarFileSystem(FileSystem fs)
Constructor to create a HarFileSystem with an
underlying filesystem.
|
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
append(Path f)
Append to an existing file (optional operation).
|
FSDataOutputStream |
append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStreamBuilder |
appendFile(Path path)
Create a Builder to append a file.
|
protected URI |
canonicalizeUri(URI uri)
Canonicalize the given URI.
|
protected void |
checkPath(Path path)
Check that a Path belongs to this FileSystem.
|
void |
close()
Close this FileSystem instance.
|
void |
completeLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
void |
copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path[] srcs,
Path dst)
The src files are on the local disk.
|
void |
copyFromLocalFile(boolean delSrc,
boolean overwrite,
Path src,
Path dst)
not implemented.
|
void |
copyToLocalFile(boolean delSrc,
Path src,
Path dst)
copies the file in the har filesystem to a local file.
|
FSDataOutputStream |
create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStreamBuilder |
createFile(Path path)
Create a new FSDataOutputStreamBuilder for the file with path.
|
FSDataOutputStream |
createNonRecursive(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
boolean |
delete(Path f,
boolean recursive)
Not implemented.
|
protected URI |
getCanonicalUri()
Used for delegation token related functionality.
|
FileSystem[] |
getChildFileSystems()
Used for delegation token related functionality.
|
Configuration |
getConf()
Return the configuration used by this object.
|
long |
getDefaultBlockSize()
Return the number of bytes that large input files should be optimally
be split into to minimize I/O time.
|
long |
getDefaultBlockSize(Path f)
Return the number of bytes that large input files should be optimally
be split into to minimize I/O time.
|
short |
getDefaultReplication()
Get the default replication.
|
short |
getDefaultReplication(Path f)
Get the default replication for a path.
|
BlockLocation[] |
getFileBlockLocations(FileStatus file,
long start,
long len)
Get block locations from the underlying fs and fix their
offsets and lengths.
|
FileChecksum |
getFileChecksum(Path f,
long length)
Get the checksum of a file, from the beginning of the file till the
specific length.
|
FileStatus |
getFileStatus(Path f)
return the filestatus of files in har archive.
|
static int |
getHarHash(Path p)
the hash of the path p inside the filesystem
|
int |
getHarVersion() |
Path |
getHomeDirectory()
return the top level archive path.
|
Path |
getInitialWorkingDirectory()
Note: with the new FileContext class, getWorkingDirectory()
will be removed.
|
String |
getScheme()
Return the protocol scheme for the FileSystem.
|
FsServerDefaults |
getServerDefaults()
Return a set of server default configuration values.
|
FsServerDefaults |
getServerDefaults(Path f)
Return a set of server default configuration values.
|
FsStatus |
getStatus(Path p)
Returns a status object describing the use and capacity of the
filesystem.
|
URI |
getUri()
Returns the uri of this filesystem.
|
long |
getUsed()
Return the total size of all files in the filesystem.
|
long |
getUsed(Path path)
Return the total size of all files from a specified path.
|
Path |
getWorkingDirectory()
return the top level archive.
|
void |
initialize(URI name,
Configuration conf)
Initialize a Har filesystem per har archive.
|
FileStatus[] |
listStatus(Path f)
liststatus returns the children of a directory
after looking up the index files.
|
Path |
makeQualified(Path path)
Qualify a path to one which uses this FileSystem and, if relative,
made absolute.
|
boolean |
mkdirs(Path f,
FsPermission permission)
not implemented.
|
FSDataInputStream |
open(Path f,
int bufferSize)
Returns a har input stream which fakes end of
file.
|
boolean |
rename(Path src,
Path dst)
Renames Path src to Path dst.
|
Path |
resolvePath(Path p)
Return the fully-qualified path of path, resolving the path
through any symlinks or mount point.
|
void |
setOwner(Path p,
String username,
String groupname)
not implemented.
|
void |
setPermission(Path p,
FsPermission permission)
Not implemented.
|
boolean |
setReplication(Path src,
short replication)
Not implemented.
|
void |
setTimes(Path p,
long mtime,
long atime)
Set access time of a file.
|
void |
setWorkingDirectory(Path newDir)
Set the current working directory for the given FileSystem.
|
Path |
startLocalOutput(Path fsOutputFile,
Path tmpLocalFile)
not implemented.
|
boolean |
truncate(Path f,
long newLength)
Not implemented.
|
addDelegationTokens, append, areSymlinksEnabled, cancelDeleteOnExit, clearStatistics, closeAll, closeAllForUGI, concat, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createNonRecursive, createSnapshot, createSnapshot, createSymlink, delete, deleteOnExit, deleteSnapshot, enableSymlinks, exists, fixRelativePart, get, get, get, getAclStatus, getAllStatistics, getAllStoragePolicies, getBlockSize, getCanonicalServiceName, getContentSummary, getDefaultPort, getDefaultUri, getFileBlockLocations, getFileChecksum, getFileLinkStatus, getFileSystemClass, getFSofPath, getGlobalStorageStatistics, getLength, getLinkTarget, getLocal, getName, getNamed, getQuotaUsage, getReplication, getStatistics, getStatistics, getStatus, getStoragePolicy, getStorageStatistics, getTrashRoot, getTrashRoots, getXAttr, getXAttrs, getXAttrs, globStatus, globStatus, isDirectory, isFile, listCorruptFileBlocks, listFiles, listLocatedStatus, listLocatedStatus, listStatus, listStatus, listStatus, listStatusIterator, listXAttrs, mkdirs, mkdirs, modifyAclEntries, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, removeAcl, removeAclEntries, removeDefaultAcl, removeXAttr, rename, renameSnapshot, resolveLink, setAcl, setDefaultUri, setDefaultUri, setStoragePolicy, setVerifyChecksum, setWriteChecksum, setXAttr, setXAttr, supportsSymlinks, unsetStoragePolicy
setConf
public static final String METADATA_CACHE_ENTRIES_KEY
public static final int METADATA_CACHE_ENTRIES_DEFAULT
public static final int VERSION
public HarFileSystem()
public HarFileSystem(FileSystem fs)
fs
- underlying file systempublic String getScheme()
getScheme
in class FileSystem
har
public void initialize(URI name, Configuration conf) throws IOException
initialize
in class FileSystem
name
- a URI whose authority section names the host, port, etc.
for this FileSystemconf
- the configurationIOException
- on any failure to initialize this instance.public Configuration getConf()
Configurable
getConf
in interface Configurable
getConf
in class Configured
public int getHarVersion() throws IOException
IOException
public Path getWorkingDirectory()
getWorkingDirectory
in class FileSystem
public Path getInitialWorkingDirectory()
FileSystem
getInitialWorkingDirectory
in class FileSystem
public FsStatus getStatus(Path p) throws IOException
FileSystem
getStatus
in class FileSystem
p
- Path for which status should be obtained. null means
the default partition.IOException
- see specific implementationprotected URI getCanonicalUri()
getCanonicalUri
in class FileSystem
FileSystem.canonicalizeUri(URI)
protected URI canonicalizeUri(URI uri)
FileSystem
FileSystem.getDefaultPort()
returns a
default port.canonicalizeUri
in class FileSystem
NetUtils.getCanonicalUri(URI, int)
public URI getUri()
getUri
in class FileSystem
protected void checkPath(Path path)
FileSystem
checkPath
in class FileSystem
path
- to checkpublic Path resolvePath(Path p) throws IOException
FileSystem
resolvePath
in class FileSystem
p
- path to be resolvedFileNotFoundException
- if the path is not presentIOException
- for any other errorpublic Path makeQualified(Path path)
FileSystem
makeQualified
in class FileSystem
path
- to qualify.Path.makeQualified(URI, Path)
public BlockLocation[] getFileBlockLocations(FileStatus file, long start, long len) throws IOException
getFileBlockLocations
in class FileSystem
file
- the input file status to get block locationsstart
- the start of the desired range in the contained filelen
- the length of the desired rangeIOException
public static int getHarHash(Path p)
p
- the path in the harfilesystempublic FileStatus getFileStatus(Path f) throws IOException
getFileStatus
in class FileSystem
f
- the path in har filesystemIOException
FileNotFoundException
- when the path does not existpublic FileChecksum getFileChecksum(Path f, long length)
FileSystem
getFileChecksum
in class FileSystem
f
- The file pathlength
- The length of the file range for checksum calculationpublic FSDataInputStream open(Path f, int bufferSize) throws IOException
open
in class FileSystem
f
- the file name to openbufferSize
- the size of the buffer to be used.IOException
- IO failurepublic FileSystem[] getChildFileSystems()
public FSDataOutputStream create(Path f, FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress) throws IOException
FileSystem
create
in class FileSystem
f
- the file name to openpermission
- file permissionoverwrite
- if a file with this name already exists, then if true,
the file will be overwritten, and if false an error will be thrown.bufferSize
- the size of the buffer to be used.replication
- required block replication for the file.blockSize
- block sizeprogress
- the progress reporterIOException
- IO failureFileSystem.setPermission(Path, FsPermission)
public FSDataOutputStream createNonRecursive(Path f, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress) throws IOException
FileSystem
createNonRecursive
in class FileSystem
f
- the file name to openoverwrite
- if a file with this name already exists, then if true,
the file will be overwritten, and if false an error will be thrown.bufferSize
- the size of the buffer to be used.replication
- required block replication for the file.blockSize
- block sizeprogress
- the progress reporterIOException
- IO failureFileSystem.setPermission(Path, FsPermission)
public FSDataOutputStream append(Path f, int bufferSize, Progressable progress) throws IOException
FileSystem
append
in class FileSystem
f
- the existing file to be appended.bufferSize
- the size of the buffer to be used.progress
- for reporting progress if it is not null.IOException
- IO failurepublic void close() throws IOException
FileSystem
FileSystem.deleteOnExit(Path)
, and remove this FS instance
from the cache, if cached.
After this operation, the outcome of any method call on this FileSystem
instance, or any input/output stream created by it is undefined.close
in interface Closeable
close
in interface AutoCloseable
close
in class FileSystem
IOException
- IO failurepublic boolean setReplication(Path src, short replication) throws IOException
setReplication
in class FileSystem
src
- file namereplication
- new replicationIOException
public boolean rename(Path src, Path dst) throws IOException
FileSystem
rename
in class FileSystem
src
- path to be renameddst
- new path after renameIOException
- on failurepublic FSDataOutputStream append(Path f) throws IOException
FileSystem
append(f, getConf().getInt(IO_FILE_BUFFER_SIZE_KEY,
IO_FILE_BUFFER_SIZE_DEFAULT), null)
append
in class FileSystem
f
- the existing file to be appended.IOException
- IO failurepublic boolean truncate(Path f, long newLength) throws IOException
truncate
in class FileSystem
f
- The path to the file to be truncatednewLength
- The size the file is to be truncated totrue
if the file has been truncated to the desired
newLength
and is immediately available to be reused for
write operations such as append
, or
false
if a background process of adjusting the length of
the last block has been started, and clients should wait for it to
complete before proceeding with further file updates.IOException
- IO failurepublic boolean delete(Path f, boolean recursive) throws IOException
delete
in class FileSystem
f
- the path to delete.recursive
- if path is a directory and set to
true, the directory is deleted else throws an exception. In
case of a file the recursive can be set to either true or false.IOException
- IO failurepublic FileStatus[] listStatus(Path f) throws IOException
listStatus
in class FileSystem
f
- given pathFileNotFoundException
- when the path does not existIOException
- see specific implementationpublic Path getHomeDirectory()
getHomeDirectory
in class FileSystem
public void setWorkingDirectory(Path newDir)
FileSystem
setWorkingDirectory
in class FileSystem
newDir
- Path of new working directorypublic boolean mkdirs(Path f, FsPermission permission) throws IOException
mkdirs
in class FileSystem
f
- path to createpermission
- to apply to fIOException
- IO failurepublic void copyFromLocalFile(boolean delSrc, boolean overwrite, Path src, Path dst) throws IOException
copyFromLocalFile
in class FileSystem
delSrc
- whether to delete the srcoverwrite
- whether to overwrite an existing filesrc
- pathdst
- pathIOException
- IO failurepublic void copyFromLocalFile(boolean delSrc, boolean overwrite, Path[] srcs, Path dst) throws IOException
FileSystem
copyFromLocalFile
in class FileSystem
delSrc
- whether to delete the srcoverwrite
- whether to overwrite an existing filesrcs
- array of paths which are sourcedst
- pathIOException
- IO failurepublic void copyToLocalFile(boolean delSrc, Path src, Path dst) throws IOException
copyToLocalFile
in class FileSystem
delSrc
- whether to delete the srcsrc
- path src file in the remote filesystemdst
- path local destinationIOException
- IO failurepublic Path startLocalOutput(Path fsOutputFile, Path tmpLocalFile) throws IOException
startLocalOutput
in class FileSystem
fsOutputFile
- path of output filetmpLocalFile
- path of local tmp fileIOException
- IO failurepublic void completeLocalOutput(Path fsOutputFile, Path tmpLocalFile) throws IOException
completeLocalOutput
in class FileSystem
fsOutputFile
- path of output filetmpLocalFile
- path to local tmp fileIOException
- IO failurepublic void setOwner(Path p, String username, String groupname) throws IOException
setOwner
in class FileSystem
p
- The pathusername
- If it is null, the original username remains unchanged.groupname
- If it is null, the original groupname remains unchanged.IOException
- IO failurepublic void setTimes(Path p, long mtime, long atime) throws IOException
FileSystem
setTimes
in class FileSystem
p
- The pathmtime
- Set the modification time of this file.
The number of milliseconds since Jan 1, 1970.
A value of -1 means that this call should not set modification time.atime
- Set the access time of this file.
The number of milliseconds since Jan 1, 1970.
A value of -1 means that this call should not set access time.IOException
- IO failurepublic void setPermission(Path p, FsPermission permission) throws IOException
setPermission
in class FileSystem
p
- The pathpermission
- permissionIOException
- IO failurepublic FsServerDefaults getServerDefaults() throws IOException
FileSystem
getServerDefaults
in class FileSystem
IOException
- IO failurepublic FsServerDefaults getServerDefaults(Path f) throws IOException
FileSystem
getServerDefaults
in class FileSystem
f
- path is used to identify an FS since an FS could have
another FS that it could be delegating the call toIOException
- IO failurepublic long getUsed() throws IOException
FileSystem
getUsed
in class FileSystem
IOException
- IO failurepublic long getUsed(Path path) throws IOException
getUsed
in class FileSystem
IOException
- IO failurepublic long getDefaultBlockSize()
FileSystem
getDefaultBlockSize
in class FileSystem
public long getDefaultBlockSize(Path f)
FileSystem
getDefaultBlockSize
in class FileSystem
f
- path of filepublic short getDefaultReplication()
FileSystem
getDefaultReplication
in class FileSystem
public short getDefaultReplication(Path f)
FileSystem
getDefaultReplication
in class FileSystem
f
- of the filepublic FSDataOutputStreamBuilder createFile(Path path)
FileSystem
createFile
in class FileSystem
path
- file pathpublic FSDataOutputStreamBuilder appendFile(Path path)
FileSystem
appendFile
in class FileSystem
path
- file path.FSDataOutputStreamBuilder
to build file append request.Copyright © 2019 Apache Software Foundation. All Rights Reserved.