Package org.apache.beam.sdk.io

Class ClassLoaderFileSystem

    • Method Detail

      • match

        protected java.util.List<MatchResult> match​(java.util.List<java.lang.String> specs)
                                     throws java.io.IOException
        Description copied from class: FileSystem
        This is the entry point to convert user-provided specs to ResourceIds. Callers should use FileSystem.match(java.util.List<java.lang.String>) to resolve users specs ambiguities before calling other methods.

        Implementation should handle the following ambiguities of a user-provided spec:

        1. spec could be a glob or a uri. FileSystem.match(java.util.List<java.lang.String>) should be able to tell and choose efficient implementations.
        2. The user-provided spec might refer to files or directories. It is common that users that wish to indicate a directory will omit the trailing /, such as in a spec of "/tmp/dir". The FileSystem should be able to recognize a directory with the trailing / omitted, but should always return a correct FileSystem (e.g., "/tmp/dir/" inside the returned MatchResult.

        All FileSystem implementations should support glob in the final hierarchical path component of FileSystem. This allows SDK libraries to construct file system agnostic spec. FileSystems can support additional patterns for user-provided specs.

        Specified by:
        match in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>
        Returns:
        List<MatchResult> in the same order of the input specs.
        Throws:
        java.io.IOException - if all specs failed to match due to issues like: network connection, authorization. Exception for individual spec need to be deferred until callers retrieve metadata with MatchResult.metadata().

      • copy

        protected void copy​(java.util.List<ClassLoaderFileSystem.ClassLoaderResourceId> srcResourceIds,
                    java.util.List<ClassLoaderFileSystem.ClassLoaderResourceId> destResourceIds)
             throws java.io.IOException
        Description copied from class: FileSystem
        Copies a List of file-like resources from one location to another.

        The number of source resources must equal the number of destination resources. Destination resources will be created recursively.

        Specified by:
        copy in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>
        Parameters:
        srcResourceIds - the references of the source resources
        destResourceIds - the references of the destination resources
        Throws:
        java.io.FileNotFoundException - if the source resources are missing. When copy throws, each resource might or might not be copied. In such scenarios, callers can use match() to determine the state of the resources.
        java.io.IOException

      • rename

        protected void rename​(java.util.List<ClassLoaderFileSystem.ClassLoaderResourceId> srcResourceIds,
                      java.util.List<ClassLoaderFileSystem.ClassLoaderResourceId> destResourceIds,
                      MoveOptions... moveOptions)
               throws java.io.IOException
        Description copied from class: FileSystem
        Renames a List of file-like resources from one location to another.

        The number of source resources must equal the number of destination resources. Destination resources will be created recursively.

        Specified by:
        rename in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>
        Parameters:
        srcResourceIds - the references of the source resources
        destResourceIds - the references of the destination resources
        moveOptions - move options specifying handling of error conditions
        Throws:
        java.io.FileNotFoundException - if the source resources are missing. When rename throws, the state of the resources is unknown but safe: for every (source, destination) pair of resources, the following are possible: a) source exists, b) destination exists, c) source and destination both exist. Thus no data is lost, however, duplicated resource are possible. In such scenarios, callers can use match() to determine the state of the resource.
        java.io.IOException

      • delete

        protected void delete​(java.util.Collection<ClassLoaderFileSystem.ClassLoaderResourceId> resourceIds)
               throws java.io.IOException
        Description copied from class: FileSystem
        Deletes a collection of resources.
        Specified by:
        delete in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>
        Parameters:
        resourceIds - the references of the resources to delete.
        Throws:
        java.io.FileNotFoundException - if resources are missing. When delete throws, each resource might or might not be deleted. In such scenarios, callers can use match() to determine the state of the resources.
        java.io.IOException

      • matchNewResource

        protected ClassLoaderFileSystem.ClassLoaderResourceId matchNewResource​(java.lang.String path,
                                                                       boolean isDirectory)
        Description copied from class: FileSystem
        Returns a new ResourceId for this filesystem that represents the named resource. The user supplies both the resource spec and whether it is a directory.

        The supplied singleResourceSpec is expected to be in a proper format, including any necessary escaping, for this FileSystem.

        This function may throw an IllegalArgumentException if given an invalid argument, such as when the specified singleResourceSpec is not a valid resource name.

        Specified by:
        matchNewResource in class FileSystem<ClassLoaderFileSystem.ClassLoaderResourceId>