org.apache.flink.runtime.blob

Class FileSystemBlobStore

java.lang.Object

org.apache.flink.runtime.blob.FileSystemBlobStore

All Implemented Interfaces:

Closeable, AutoCloseable, BlobStore, BlobStoreService, BlobView

public class FileSystemBlobStore
extends Object
implements BlobStoreService

Blob store backed by FileSystem.

This is used in addition to the local blob storage for high availability.

Constructor Summary

Constructors

Constructor and Description
FileSystemBlobStore(FileSystem fileSystem, String storagePath)

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	closeAndCleanupAllData() Closes and cleans up the store.
void	delete(BlobKey blobKey) Tries to delete a blob from storage.
void	delete(JobID jobId, String key) Tries to delete a blob from storage.
void	deleteAll(JobID jobId) Tries to delete all blobs for the given job from storage.
void	get(BlobKey blobKey, File localFile) Copies a blob to a local file.
void	get(JobID jobId, String key, File localFile) Copies a blob to a local file.
void	put(File localFile, BlobKey blobKey) Copies the local file to the blob store.
void	put(File localFile, JobID jobId, String key) Copies a local file to the blob store.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileSystemBlobStore

public FileSystemBlobStore(FileSystem fileSystem,
                   String storagePath)
                    throws IOException

Throws:

IOException

Method Detail

put

public void put(File localFile,
       BlobKey blobKey)
         throws IOException

Description copied from interface: BlobStore

Copies the local file to the blob store.

Specified by:

put in interface BlobStore

Parameters:

localFile - The file to copy

blobKey - The ID for the file in the blob store

Throws:

IOException - If the copy fails

put

public void put(File localFile,
       JobID jobId,
       String key)
         throws IOException

Description copied from interface: BlobStore

Copies a local file to the blob store.

The job ID and key make up a composite key for the file.

Specified by:

put in interface BlobStore

Parameters:

localFile - The file to copy

jobId - The JobID part of ID for the file in the blob store

key - The String part of ID for the file in the blob store

Throws:

IOException - If the copy fails

get

public void get(BlobKey blobKey,
       File localFile)
         throws IOException

Description copied from interface: BlobView

Copies a blob to a local file.

Specified by:

get in interface BlobView

Parameters:

blobKey - The blob ID

localFile - The local file to copy to

Throws:

IOException - If the copy fails

get

public void get(JobID jobId,
       String key,
       File localFile)
         throws IOException

Description copied from interface: BlobView

Copies a blob to a local file.

Specified by:

get in interface BlobView

Parameters:

jobId - The JobID part of ID for the blob

key - The String part of ID for the blob

localFile - The local file to copy to

Throws:

IOException - If the copy fails

delete

public void delete(BlobKey blobKey)

Description copied from interface: BlobStore

Tries to delete a blob from storage.

NOTE: This also tries to delete any created directories if empty.

Specified by:

delete in interface BlobStore

Parameters:

blobKey - The blob ID

delete

public void delete(JobID jobId,
          String key)

Description copied from interface: BlobStore

Tries to delete a blob from storage.

NOTE: This also tries to delete any created directories if empty.

Specified by:

delete in interface BlobStore

Parameters:

jobId - The JobID part of ID for the blob

key - The String part of ID for the blob

deleteAll

public void deleteAll(JobID jobId)

Description copied from interface: BlobStore

Tries to delete all blobs for the given job from storage.

NOTE: This also tries to delete any created directories if empty.

Specified by:

deleteAll in interface BlobStore

Parameters:

jobId - The JobID part of all blobs to delete

closeAndCleanupAllData

public void closeAndCleanupAllData()

Description copied from interface: BlobStoreService

Closes and cleans up the store. This entails the deletion of all blobs.

Specified by:

closeAndCleanupAllData in interface BlobStoreService

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException