Class BlobStoreRepository
- java.lang.Object
-
- org.elasticsearch.common.component.AbstractLifecycleComponent
-
- org.elasticsearch.repositories.blobstore.BlobStoreRepository
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
,LifecycleComponent
,Releasable
,Repository
- Direct Known Subclasses:
FsRepository
public abstract class BlobStoreRepository extends AbstractLifecycleComponent implements Repository
BlobStore - based implementation of Snapshot RepositoryThis repository works with any
BlobStore
implementation. The blobStore could be (and preferred) lazy initialized increateBlobStore()
.BlobStoreRepository maintains the following structure in the blob store
STORE_ROOT |- index-N - JSON serialized {@link RepositoryData} containing a list of all snapshot ids and the indices belonging to | each snapshot, N is the generation of the file |- index.latest - contains the numeric value of the latest generation of the index file (i.e. N from above) |- incompatible-snapshots - list of all snapshot ids that are no longer compatible with the current version of the cluster |- snap-20131010.dat - SMILE serialized {@link SnapshotInfo} for snapshot "20131010" |- meta-20131010.dat - SMILE serialized {@link MetaData} for snapshot "20131010" (includes only global metadata) |- snap-20131011.dat - SMILE serialized {@link SnapshotInfo} for snapshot "20131011" |- meta-20131011.dat - SMILE serialized {@link MetaData} for snapshot "20131011" ..... |- indices/ - data for all indices |- Ac1342-B_x/ - data for index "foo" which was assigned the unique id of Ac1342-B_x in the repository | |- meta-20131010.dat - JSON Serialized {@link IndexMetaData} for index "foo" | |- 0/ - data for shard "0" of index "foo" | | |- __1 \ (files with numeric names were created by older ES versions) | | |- __2 | | | |- __VPO5oDMVT5y4Akv8T_AO_A |- files from different segments see snap-* for their mappings to real segment files | | |- __1gbJy18wS_2kv1qI7FgKuQ | | | |- __R8JvZAHlSMyMXyZc2SS8Zg / | | ..... | | |- snap-20131010.dat - SMILE serialized {@link BlobStoreIndexShardSnapshot} for snapshot "20131010" | | |- snap-20131011.dat - SMILE serialized {@link BlobStoreIndexShardSnapshot} for snapshot "20131011" | | |- index-123 - SMILE serialized {@link BlobStoreIndexShardSnapshots} for the shard | | | |- 1/ - data for shard "1" of index "foo" | | |- __1 | | ..... | | | |-2/ | ...... | |- 1xB0D8_B3y/ - data for index "bar" which was assigned the unique id of 1xB0D8_B3y in the repository ......
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.elasticsearch.repositories.Repository
Repository.Factory
-
-
Field Summary
Fields Modifier and Type Field Description protected RepositoryMetaData
metadata
protected NamedXContentRegistry
namedXContentRegistry
-
Fields inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent
lifecycle
-
-
Constructor Summary
Constructors Modifier Constructor Description protected
BlobStoreRepository(RepositoryMetaData metadata, Settings settings, NamedXContentRegistry namedXContentRegistry)
Constructs new BlobStoreRepository
-
Method Summary
Modifier and Type Method Description protected void
assertSnapshotOrGenericThread()
protected abstract BlobPath
basePath()
Returns base path of the repositoryprotected BlobContainer
blobContainer()
maintains single lazy instance ofBlobContainer
protected BlobStore
blobStore()
maintains single lazy instance ofBlobStore
protected ByteSizeValue
chunkSize()
Returns data file chunk size.protected abstract BlobStore
createBlobStore()
Creates new BlobStore to read and write data.void
deleteSnapshot(SnapshotId snapshotId, long repositoryStateId)
Deletes snapshotprotected void
doClose()
protected void
doStart()
protected void
doStop()
void
endVerification(java.lang.String seed)
Called at the end of repository verification process.SnapshotInfo
finalizeSnapshot(SnapshotId snapshotId, java.util.List<IndexId> indices, long startTime, java.lang.String failure, int totalShards, java.util.List<SnapshotShardFailure> shardFailures, long repositoryStateId, boolean includeGlobalState)
Finalizes snapshotting processprotected BlobStore
getBlobStore()
RepositoryMetaData
getMetadata()
Returns metadata about this repository.RepositoryData
getRepositoryData()
Returns aRepositoryData
to describe the data in the repository, including the snapshots and the indices across all snapshots found in the repository.long
getRestoreThrottleTimeInNanos()
Returns restore throttle time in nanosecondsIndexShardSnapshotStatus
getShardSnapshotStatus(SnapshotId snapshotId, Version version, IndexId indexId, ShardId shardId)
Retrieve shard snapshot status for the stored snapshotMetaData
getSnapshotGlobalMetaData(SnapshotId snapshotId)
Returns global metadata associated with the snapshot.IndexMetaData
getSnapshotIndexMetaData(SnapshotId snapshotId, IndexId index)
Returns the index metadata associated with the snapshot.SnapshotInfo
getSnapshotInfo(SnapshotId snapshotId)
Reads snapshot description from repository.long
getSnapshotThrottleTimeInNanos()
Returns snapshot throttle time in nanosecondsvoid
initializeSnapshot(SnapshotId snapshotId, java.util.List<IndexId> indices, MetaData clusterMetaData)
Starts snapshotting processprotected boolean
isCompress()
Returns true if metadata and snapshot files should be compressedboolean
isReadOnly()
Returns true if the repository supports only read operationsvoid
restoreShard(IndexShard shard, SnapshotId snapshotId, Version version, IndexId indexId, ShardId snapshotShardId, RecoveryState recoveryState)
Restores snapshot of the shard.void
snapshotShard(IndexShard shard, Store store, SnapshotId snapshotId, IndexId indexId, org.apache.lucene.index.IndexCommit snapshotIndexCommit, IndexShardSnapshotStatus snapshotStatus)
Creates a snapshot of the shard based on the index commit point.java.lang.String
startVerification()
Verifies repository on the master node and returns the verification token.static java.lang.String
testBlobPrefix(java.lang.String seed)
java.lang.String
toString()
void
verify(java.lang.String seed, DiscoveryNode localNode)
Verifies repository settings on data node.protected void
writeIndexGen(RepositoryData repositoryData, long repositoryStateId)
-
Methods inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent
addLifecycleListener, close, lifecycleState, removeLifecycleListener, start, stop
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.elasticsearch.common.component.LifecycleComponent
addLifecycleListener, lifecycleState, removeLifecycleListener, start, stop
-
Methods inherited from interface org.elasticsearch.common.lease.Releasable
close
-
-
-
-
Field Detail
-
metadata
protected final RepositoryMetaData metadata
-
namedXContentRegistry
protected final NamedXContentRegistry namedXContentRegistry
-
-
Constructor Detail
-
BlobStoreRepository
protected BlobStoreRepository(RepositoryMetaData metadata, Settings settings, NamedXContentRegistry namedXContentRegistry)
Constructs new BlobStoreRepository- Parameters:
metadata
- The metadata for this repository including name and settingssettings
- Settings for the node this repository object is created on
-
-
Method Detail
-
doStart
protected void doStart()
- Specified by:
doStart
in classAbstractLifecycleComponent
-
doStop
protected void doStop()
- Specified by:
doStop
in classAbstractLifecycleComponent
-
doClose
protected void doClose()
- Specified by:
doClose
in classAbstractLifecycleComponent
-
getBlobStore
protected BlobStore getBlobStore()
-
blobContainer
protected BlobContainer blobContainer()
maintains single lazy instance ofBlobContainer
-
createBlobStore
protected abstract BlobStore createBlobStore() throws java.lang.Exception
Creates new BlobStore to read and write data.- Throws:
java.lang.Exception
-
basePath
protected abstract BlobPath basePath()
Returns base path of the repository
-
isCompress
protected boolean isCompress()
Returns true if metadata and snapshot files should be compressed- Returns:
- true if compression is needed
-
chunkSize
protected ByteSizeValue chunkSize()
Returns data file chunk size.This method should return null if no chunking is needed.
- Returns:
- chunk size
-
getMetadata
public RepositoryMetaData getMetadata()
Description copied from interface:Repository
Returns metadata about this repository.- Specified by:
getMetadata
in interfaceRepository
-
initializeSnapshot
public void initializeSnapshot(SnapshotId snapshotId, java.util.List<IndexId> indices, MetaData clusterMetaData)
Description copied from interface:Repository
Starts snapshotting process- Specified by:
initializeSnapshot
in interfaceRepository
- Parameters:
snapshotId
- snapshot idindices
- list of indices to be snapshottedclusterMetaData
- cluster metadata
-
deleteSnapshot
public void deleteSnapshot(SnapshotId snapshotId, long repositoryStateId)
Description copied from interface:Repository
Deletes snapshot- Specified by:
deleteSnapshot
in interfaceRepository
- Parameters:
snapshotId
- snapshot idrepositoryStateId
- the unique id identifying the state of the repository when the snapshot deletion began
-
finalizeSnapshot
public SnapshotInfo finalizeSnapshot(SnapshotId snapshotId, java.util.List<IndexId> indices, long startTime, java.lang.String failure, int totalShards, java.util.List<SnapshotShardFailure> shardFailures, long repositoryStateId, boolean includeGlobalState)
Finalizes snapshotting processThis method is called on master after all shards are snapshotted.
- Specified by:
finalizeSnapshot
in interfaceRepository
- Parameters:
snapshotId
- snapshot idindices
- list of indices in the snapshotstartTime
- start time of the snapshotfailure
- global failure reason or nulltotalShards
- total number of shardsshardFailures
- list of shard failuresrepositoryStateId
- the unique id identifying the state of the repository when the snapshot beganincludeGlobalState
- include cluster global state- Returns:
- snapshot description
-
getSnapshotInfo
public SnapshotInfo getSnapshotInfo(SnapshotId snapshotId)
Description copied from interface:Repository
Reads snapshot description from repository.- Specified by:
getSnapshotInfo
in interfaceRepository
- Parameters:
snapshotId
- snapshot id- Returns:
- information about snapshot
-
getSnapshotGlobalMetaData
public MetaData getSnapshotGlobalMetaData(SnapshotId snapshotId)
Description copied from interface:Repository
Returns global metadata associated with the snapshot.- Specified by:
getSnapshotGlobalMetaData
in interfaceRepository
- Parameters:
snapshotId
- the snapshot id to load the global metadata from- Returns:
- the global metadata about the snapshot
-
getSnapshotIndexMetaData
public IndexMetaData getSnapshotIndexMetaData(SnapshotId snapshotId, IndexId index) throws java.io.IOException
Description copied from interface:Repository
Returns the index metadata associated with the snapshot.- Specified by:
getSnapshotIndexMetaData
in interfaceRepository
- Parameters:
snapshotId
- the snapshot id to load the index metadata fromindex
- theIndexId
to load the metadata from- Returns:
- the index metadata about the given index for the given snapshot
- Throws:
java.io.IOException
-
getSnapshotThrottleTimeInNanos
public long getSnapshotThrottleTimeInNanos()
Description copied from interface:Repository
Returns snapshot throttle time in nanoseconds- Specified by:
getSnapshotThrottleTimeInNanos
in interfaceRepository
-
getRestoreThrottleTimeInNanos
public long getRestoreThrottleTimeInNanos()
Description copied from interface:Repository
Returns restore throttle time in nanoseconds- Specified by:
getRestoreThrottleTimeInNanos
in interfaceRepository
-
assertSnapshotOrGenericThread
protected void assertSnapshotOrGenericThread()
-
startVerification
public java.lang.String startVerification()
Description copied from interface:Repository
Verifies repository on the master node and returns the verification token.If the verification token is not null, it's passed to all data nodes for verification. If it's null - no additional verification is required
- Specified by:
startVerification
in interfaceRepository
- Returns:
- verification token that should be passed to all Index Shard Repositories for additional verification or null
-
endVerification
public void endVerification(java.lang.String seed)
Description copied from interface:Repository
Called at the end of repository verification process.This method should perform all necessary cleanup of the temporary files created in the repository
- Specified by:
endVerification
in interfaceRepository
- Parameters:
seed
- verification request generated byRepository.startVerification()
command
-
getRepositoryData
public RepositoryData getRepositoryData()
Description copied from interface:Repository
Returns aRepositoryData
to describe the data in the repository, including the snapshots and the indices across all snapshots found in the repository. Throws aRepositoryException
if there was an error in reading the data.- Specified by:
getRepositoryData
in interfaceRepository
-
testBlobPrefix
public static java.lang.String testBlobPrefix(java.lang.String seed)
-
isReadOnly
public boolean isReadOnly()
Description copied from interface:Repository
Returns true if the repository supports only read operations- Specified by:
isReadOnly
in interfaceRepository
- Returns:
- true if the repository is read/only
-
writeIndexGen
protected void writeIndexGen(RepositoryData repositoryData, long repositoryStateId) throws java.io.IOException
- Throws:
java.io.IOException
-
snapshotShard
public void snapshotShard(IndexShard shard, Store store, SnapshotId snapshotId, IndexId indexId, org.apache.lucene.index.IndexCommit snapshotIndexCommit, IndexShardSnapshotStatus snapshotStatus)
Description copied from interface:Repository
Creates a snapshot of the shard based on the index commit point.The index commit point can be obtained by using
Engine.acquireLastIndexCommit(boolean)
method. Repository implementations shouldn't release the snapshot index commit point. It is done by the method caller.As snapshot process progresses, implementation of this method should update
IndexShardSnapshotStatus
object and checkIndexShardSnapshotStatus.isAborted()
to see if the snapshot process should be aborted.- Specified by:
snapshotShard
in interfaceRepository
- Parameters:
shard
- shard to be snapshottedstore
- store to be snapshottedsnapshotId
- snapshot idindexId
- id for the index being snapshottedsnapshotIndexCommit
- commit pointsnapshotStatus
- snapshot status
-
restoreShard
public void restoreShard(IndexShard shard, SnapshotId snapshotId, Version version, IndexId indexId, ShardId snapshotShardId, RecoveryState recoveryState)
Description copied from interface:Repository
Restores snapshot of the shard.The index can be renamed on restore, hence different
shardId
andsnapshotShardId
are supplied.- Specified by:
restoreShard
in interfaceRepository
- Parameters:
shard
- the shard to restore the index intosnapshotId
- snapshot idversion
- version of elasticsearch that created this snapshotindexId
- id of the index in the repository from which the restore is occurringsnapshotShardId
- shard id (in the snapshot)recoveryState
- recovery state
-
getShardSnapshotStatus
public IndexShardSnapshotStatus getShardSnapshotStatus(SnapshotId snapshotId, Version version, IndexId indexId, ShardId shardId)
Description copied from interface:Repository
Retrieve shard snapshot status for the stored snapshot- Specified by:
getShardSnapshotStatus
in interfaceRepository
- Parameters:
snapshotId
- snapshot idversion
- version of elasticsearch that created this snapshotindexId
- the snapshotted index id for the shard to get status forshardId
- shard id- Returns:
- snapshot status
-
verify
public void verify(java.lang.String seed, DiscoveryNode localNode)
Description copied from interface:Repository
Verifies repository settings on data node.- Specified by:
verify
in interfaceRepository
- Parameters:
seed
- value returned byRepository.startVerification()
localNode
- the local node information, for inclusion in verification errors
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
-