Package alluxio.worker.block
Interface BlockStore
-
- All Superinterfaces:
java.lang.AutoCloseable
,java.io.Closeable
,SessionCleanable
public interface BlockStore extends java.io.Closeable, SessionCleanable
An abstraction of block store on worker.
-
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description void
abortBlock(long sessionId, long blockId)
Aborts a temporary block.void
accessBlock(long sessionId, long blockId)
Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access.void
commitBlock(long sessionId, long blockId, boolean pinOnCreate)
Commits a block to Alluxio managed space.java.lang.String
createBlock(long sessionId, long blockId, int tier, CreateBlockOptions createBlockOptions)
Creates a block in Alluxio managed space.BlockReader
createBlockReader(long sessionId, long blockId, long lockId)
Creates a reader of an existing block to read data from this block.BlockReader
createBlockReader(long sessionId, long blockId, long offset, boolean positionShort, alluxio.proto.dataserver.Protocol.OpenUfsBlockOptions options)
Creates the block reader to read from Alluxio block or UFS block.BlockWriter
createBlockWriter(long sessionId, long blockId)
Creates aBlockWriter
for an existing temporary block which is already created bycreateBlock(long, long, int, alluxio.worker.block.CreateBlockOptions)
.BlockReader
createUfsBlockReader(long sessionId, long blockId, long offset, boolean positionShort, alluxio.proto.dataserver.Protocol.OpenUfsBlockOptions options)
Creates a block reader to read a UFS block starting from given block offset.BlockMeta
getBlockMeta(long sessionId, long blockId, long lockId)
Gets the metadata of a specific block from local storage.BlockStoreMeta
getBlockStoreMeta()
Gets the metadata of the entire store in a snapshot.BlockStoreMeta
getBlockStoreMetaFull()
Similar asgetBlockStoreMeta()
except that this includes more information about the block store (e.g.BlockWriter
getBlockWriter(long sessionId, long blockId)
Creates a writer to write data to a temp block.java.util.Optional<TempBlockMeta>
getTempBlockMeta(long blockId)
Gets the temp metadata of a specific block from local storage.java.util.Optional<BlockMeta>
getVolatileBlockMeta(long blockId)
Gets the metadata of a block given its block id or empty if block does not exist.boolean
hasBlockMeta(long blockId)
Checks if the storage has a given block.boolean
hasTempBlockMeta(long blockId)
Checks if the storage has a given temp block.default void
initialize()
Initialize the block store.java.util.concurrent.CompletableFuture<java.util.List<alluxio.grpc.BlockStatus>>
load(java.util.List<alluxio.grpc.Block> fileBlocks, alluxio.grpc.UfsReadOptions options)
Load blocks into alluxio.long
lockBlock(long sessionId, long blockId)
Locks an existing block and guards subsequent reads on this block.long
lockBlockNoException(long sessionId, long blockId)
Locks an existing block and guards subsequent reads on this block.void
moveBlock(long sessionId, long blockId, AllocateOptions moveOptions)
Moves an existing block to a new location.java.util.Optional<BlockLock>
pinBlock(long sessionId, long blockId)
Pins the block indicating subsequent access.void
registerBlockStoreEventListener(BlockStoreEventListener listener)
Registers aBlockStoreEventListener
to this block store.void
removeBlock(long sessionId, long blockId)
Removes an existing block.void
removeInaccessibleStorage()
Remove Storage directories that are no longer accessible.void
requestSpace(long sessionId, long blockId, long additionalBytes)
Requests to increase the size of a temp block.void
unlockBlock(long lockId)
Releases an acquired block lock based on a lockId (returned bylockBlock(long, long)
.boolean
unlockBlock(long sessionId, long blockId)
Releases an acquired block lock based on a session id and block id.void
unpinBlock(BlockLock lock)
Unpins an accessed block based on the id (returned bypinBlock(long, long)
).void
updatePinnedInodes(java.util.Set<java.lang.Long> inodes)
Update the pinned inodes.-
Methods inherited from interface alluxio.worker.SessionCleanable
cleanupSession
-
-
-
-
Method Detail
-
initialize
default void initialize()
Initialize the block store.
-
abortBlock
void abortBlock(long sessionId, long blockId)
Aborts a temporary block. The metadata of this block will not be added, its data will be deleted and the space will be reclaimed. Since a temp block is "private" to the writer, this requires no previously acquired lock.- Parameters:
sessionId
- the id of the sessionblockId
- the id of a temp block
-
accessBlock
void accessBlock(long sessionId, long blockId)
Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access. //TODO(beinan): looks like we should not expose this method except the test- Parameters:
sessionId
- the id of the session to access a blockblockId
- the id of an accessed block
-
commitBlock
void commitBlock(long sessionId, long blockId, boolean pinOnCreate)
Commits a block to Alluxio managed space. The block must be temporary. The block will not be persisted or accessible before commitBlock succeeds.- Parameters:
sessionId
- the id of the clientblockId
- the id of the block to commitpinOnCreate
- whether to pin block on create
-
createBlock
java.lang.String createBlock(long sessionId, long blockId, int tier, CreateBlockOptions createBlockOptions)
Creates a block in Alluxio managed space. CallscreateBlockWriter(long, long)
to get a writer for writing to the block. The block will be temporary until it is committed bycommitBlock(long, long, boolean)
. Throws anIllegalArgumentException
if the location does not belong to tiered storage.- Parameters:
sessionId
- the id of the clientblockId
- the id of the block to createtier
- the tier to place the new block inBlockStoreLocation.ANY_TIER
for any tiercreateBlockOptions
- the createBlockOptions- Returns:
- a string representing the path to the local file
-
createBlockReader
BlockReader createBlockReader(long sessionId, long blockId, long offset, boolean positionShort, alluxio.proto.dataserver.Protocol.OpenUfsBlockOptions options) throws java.io.IOException
Creates the block reader to read from Alluxio block or UFS block. Owner of this block reader must close it or lock will leak.- Parameters:
sessionId
- the client session IDblockId
- the ID of the UFS block to readoffset
- the offset within the blockpositionShort
- whether the operation is using positioned read to a small buffer sizeoptions
- the options- Returns:
- a block reader to read data from
- Throws:
java.io.IOException
- if it fails to get block reader
-
createBlockReader
BlockReader createBlockReader(long sessionId, long blockId, long lockId) throws BlockDoesNotExistException, java.io.IOException
Creates a reader of an existing block to read data from this block.This operation requires the lock id returned by a previously acquired
lockBlock(long, long)
.- Parameters:
sessionId
- the id of the session to get the readerblockId
- the id of an existing blocklockId
- the id of the lock returned bylockBlock(long, long)
- Returns:
- a
BlockReader
instance on this block - Throws:
BlockDoesNotExistException
- if lockId is not foundjava.io.IOException
-
createUfsBlockReader
BlockReader createUfsBlockReader(long sessionId, long blockId, long offset, boolean positionShort, alluxio.proto.dataserver.Protocol.OpenUfsBlockOptions options) throws java.io.IOException
Creates a block reader to read a UFS block starting from given block offset. Owner of this block reader must close it to cleanup state.- Parameters:
sessionId
- the client session IDblockId
- the ID of the UFS block to readoffset
- the offset within the blockpositionShort
- whether the operation is using positioned read to a small buffer sizeoptions
- the options- Returns:
- the block reader instance
- Throws:
java.io.IOException
- if it fails to get block reader
-
createBlockWriter
BlockWriter createBlockWriter(long sessionId, long blockId) throws java.io.IOException
Creates aBlockWriter
for an existing temporary block which is already created bycreateBlock(long, long, int, alluxio.worker.block.CreateBlockOptions)
.- Parameters:
sessionId
- the id of the clientblockId
- the id of the block to be opened for writing- Returns:
- the block writer for the local block file
- Throws:
java.io.IOException
-
getBlockStoreMeta
BlockStoreMeta getBlockStoreMeta()
Gets the metadata of the entire store in a snapshot. There is no guarantee the state will be consistent with the snapshot after this method is called. This function should be cheap since it is called for every block.- Returns:
- store metadata
-
getBlockStoreMetaFull
BlockStoreMeta getBlockStoreMetaFull()
Similar asgetBlockStoreMeta()
except that this includes more information about the block store (e.g. blockId list). This is an expensive operation.- Returns:
- full store metadata
-
getTempBlockMeta
java.util.Optional<TempBlockMeta> getTempBlockMeta(long blockId)
Gets the temp metadata of a specific block from local storage.- Parameters:
blockId
- the id of the block- Returns:
- metadata of the block if the temp block exists
-
hasBlockMeta
boolean hasBlockMeta(long blockId)
Checks if the storage has a given block.- Parameters:
blockId
- the block id- Returns:
- true if the block is contained, false otherwise
-
hasTempBlockMeta
boolean hasTempBlockMeta(long blockId)
Checks if the storage has a given temp block.- Parameters:
blockId
- the temp block id- Returns:
- true if the block is contained, false otherwise
-
getVolatileBlockMeta
java.util.Optional<BlockMeta> getVolatileBlockMeta(long blockId)
Gets the metadata of a block given its block id or empty if block does not exist. This method does not require a lock id so the block is possible to be moved or removed after it returns.- Parameters:
blockId
- the block id- Returns:
- metadata of the block
-
moveBlock
void moveBlock(long sessionId, long blockId, AllocateOptions moveOptions) throws java.io.IOException
Moves an existing block to a new location.- Parameters:
sessionId
- the id of the session to move a blockblockId
- the id of an existing blockmoveOptions
- the options for move- Throws:
java.io.IOException
-
pinBlock
java.util.Optional<BlockLock> pinBlock(long sessionId, long blockId)
Pins the block indicating subsequent access.- Parameters:
sessionId
- the id of the session to lock this blockblockId
- the id of the block to lock- Returns:
- a lock of block to conveniently unpin the block later, or empty if the block does not exist
-
unpinBlock
void unpinBlock(BlockLock lock)
Unpins an accessed block based on the id (returned bypinBlock(long, long)
).- Parameters:
lock
- the lock returned bypinBlock(long, long)
-
updatePinnedInodes
void updatePinnedInodes(java.util.Set<java.lang.Long> inodes)
Update the pinned inodes.- Parameters:
inodes
- a set of inodes that are currently pinned
-
registerBlockStoreEventListener
void registerBlockStoreEventListener(BlockStoreEventListener listener)
Registers aBlockStoreEventListener
to this block store.- Parameters:
listener
- the listener to those events
-
removeBlock
void removeBlock(long sessionId, long blockId) throws java.io.IOException
Removes an existing block. If the block can not be found in this store.- Parameters:
sessionId
- the id of the session to remove a blockblockId
- the id of an existing block- Throws:
java.io.IOException
-
removeInaccessibleStorage
void removeInaccessibleStorage()
Remove Storage directories that are no longer accessible.
-
requestSpace
void requestSpace(long sessionId, long blockId, long additionalBytes)
Requests to increase the size of a temp block. Since a temp block is "private" to the writer client, this operation requires no previously acquired lock.- Parameters:
sessionId
- the id of the session to request spaceblockId
- the id of the temp blockadditionalBytes
- the amount of more space to request in bytes, never be less than 0
-
load
java.util.concurrent.CompletableFuture<java.util.List<alluxio.grpc.BlockStatus>> load(java.util.List<alluxio.grpc.Block> fileBlocks, alluxio.grpc.UfsReadOptions options)
Load blocks into alluxio.- Parameters:
fileBlocks
- list of fileBlocks, one file blocks contains blocks belong to one fileoptions
- read ufs options- Returns:
- future of load status for failed blocks
-
getBlockMeta
BlockMeta getBlockMeta(long sessionId, long blockId, long lockId) throws BlockDoesNotExistException, InvalidWorkerStateException
Gets the metadata of a specific block from local storage.This method requires the lock id returned by a previously acquired
lockBlock(long, long)
.- Parameters:
sessionId
- the id of the session to get this fileblockId
- the id of the blocklockId
- the id of the lock- Returns:
- metadata of the block
- Throws:
BlockDoesNotExistException
- if the block id can not be found in committed blocks or lockId can not be foundInvalidWorkerStateException
- if session id or block id is not the same as that in the LockRecord of lockId
-
lockBlock
long lockBlock(long sessionId, long blockId) throws BlockDoesNotExistException
Locks an existing block and guards subsequent reads on this block.- Parameters:
sessionId
- the id of the session to lock this blockblockId
- the id of the block to lock- Returns:
- the lock id (non-negative) if the lock is acquired successfully
- Throws:
BlockDoesNotExistException
- if block id can not be found, for example, evicted already
-
lockBlockNoException
long lockBlockNoException(long sessionId, long blockId)
Locks an existing block and guards subsequent reads on this block. If the lock fails, returnBlockLockManager#INVALID_LOCK_ID
.- Parameters:
sessionId
- the id of the session to lock this blockblockId
- the id of the block to lock- Returns:
- the lock id (non-negative) that uniquely identifies the lock obtained or
BlockLockManager#INVALID_LOCK_ID
if it failed to lock
-
unlockBlock
void unlockBlock(long lockId) throws BlockDoesNotExistException
Releases an acquired block lock based on a lockId (returned bylockBlock(long, long)
.- Parameters:
lockId
- the id of the lock returned bylockBlock(long, long)
- Throws:
BlockDoesNotExistException
- if lockId can not be found
-
unlockBlock
boolean unlockBlock(long sessionId, long blockId)
Releases an acquired block lock based on a session id and block id. TODO(calvin): temporary, will be removed after changing client side code.- Parameters:
sessionId
- the id of the session to lock this blockblockId
- the id of the block to lock- Returns:
- false if it fails to unlock due to the lock is not found
-
getBlockWriter
BlockWriter getBlockWriter(long sessionId, long blockId) throws BlockDoesNotExistException, BlockAlreadyExistsException, InvalidWorkerStateException, java.io.IOException
Creates a writer to write data to a temp block. Since the temp block is "private" to the writer, this operation requires no previously acquired lock.- Parameters:
sessionId
- the id of the session to get the writerblockId
- the id of the temp block- Returns:
- a
BlockWriter
instance on this block - Throws:
BlockDoesNotExistException
- if the block can not be foundBlockAlreadyExistsException
- if a committed block with the same ID existsInvalidWorkerStateException
- if the worker state is invalidjava.io.IOException
-
-