Package org.elasticsearch.indices
Class IndicesService
- java.lang.Object
-
- org.elasticsearch.common.component.AbstractLifecycleComponent
-
- org.elasticsearch.indices.IndicesService
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
,java.lang.Iterable<IndexService>
,LifecycleComponent
,Releasable
,IndexService.ShardStoreDeleter
,IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
public class IndicesService extends AbstractLifecycleComponent implements IndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>, IndexService.ShardStoreDeleter
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
IndicesService.ShardDeletionCheckResult
result type returned bysignaling different reasons why a shard can / cannot be deleted
-
Nested classes/interfaces inherited from interface org.elasticsearch.indices.cluster.IndicesClusterStateService.AllocatedIndices
IndicesClusterStateService.AllocatedIndices.IndexRemovalReason
-
-
Field Summary
Fields Modifier and Type Field Description static Setting<TimeValue>
INDICES_CACHE_CLEAN_INTERVAL_SETTING
static java.lang.String
INDICES_SHARDS_CLOSED_TIMEOUT
-
Fields inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent
lifecycle
-
-
Constructor Summary
Constructors Constructor Description IndicesService(Settings settings, PluginsService pluginsService, NodeEnvironment nodeEnv, NamedXContentRegistry xContentRegistry, AnalysisRegistry analysisRegistry, IndexNameExpressionResolver indexNameExpressionResolver, MapperRegistry mapperRegistry, NamedWriteableRegistry namedWriteableRegistry, ThreadPool threadPool, IndexScopedSettings indexScopedSettings, CircuitBreakerService circuitBreakerService, BigArrays bigArrays, ScriptService scriptService, Client client, MetaStateService metaStateService, java.util.Collection<java.util.function.Function<IndexSettings,java.util.Optional<EngineFactory>>> engineFactoryProviders, java.util.Map<java.lang.String,java.util.function.Function<IndexSettings,IndexStore>> indexStoreFactories)
-
Method Summary
Modifier and Type Method Description void
addPendingDelete(Index index, IndexSettings settings)
Adds a pending delete for the given index.void
addPendingDelete(ShardId shardId, IndexSettings settings)
Adds a pending delete for the given index shard.AliasFilter
buildAliasFilter(ClusterState state, java.lang.String index, java.util.Set<java.lang.String> resolvedExpressions)
boolean
canCache(ShardSearchRequest request, SearchContext context)
Can the shard request be cached at all?boolean
canDeleteIndexContents(Index index, IndexSettings indexSettings)
This method returns true if the current node is allowed to delete the given index.IndicesService.ShardDeletionCheckResult
canDeleteShardContent(ShardId shardId, IndexSettings indexSettings)
ReturnsShardDeletionCheckResult
signaling whether the shards content for the given shard can be deleted.static java.util.Optional<java.lang.String>
checkShardLimit(int newShards, ClusterState state, DeprecationLogger deprecationLogger)
Checks to see if an operation can be performed without taking the cluster over the cluster-wide shard limit.void
clearIndexShardCache(ShardId shardId, boolean queryCache, boolean fieldDataCache, boolean requestCache, java.lang.String... fields)
Clears the caches for the given shard id if the shard is still allocated on this nodeIndexService
createIndex(IndexMetaData indexMetaData, java.util.List<IndexEventListener> builtInListeners)
Creates a newIndexService
for the given metadata.MapperService
createIndexMapperService(IndexMetaData indexMetaData)
creates a new mapper service for the given index, in order to do administrative work like mapping updates.IndexShard
createShard(ShardRouting shardRouting, RecoveryState recoveryState, PeerRecoveryTargetService recoveryTargetService, PeerRecoveryTargetService.RecoveryListener recoveryListener, RepositoriesService repositoriesService, java.util.function.Consumer<IndexShard.ShardFailure> onShardFailure, java.util.function.Consumer<ShardId> globalCheckpointSyncer, RetentionLeaseSyncer retentionLeaseSyncer)
Creates a shard for the specified shard routing and starts recovery.void
deleteShardStore(java.lang.String reason, ShardLock lock, IndexSettings indexSettings)
Deletes the shard with an already acquired shard lock.void
deleteShardStore(java.lang.String reason, ShardId shardId, ClusterState clusterState)
This method deletes the shard contents on disk for the given shard ID.void
deleteUnassignedIndex(java.lang.String reason, IndexMetaData metaData, ClusterState clusterState)
Deletes an index that is not assigned to this node.protected void
doClose()
protected void
doStart()
protected void
doStop()
AnalysisRegistry
getAnalysis()
CircuitBreakerService
getCircuitBreakerService()
java.util.function.Function<java.lang.String,java.util.function.Predicate<java.lang.String>>
getFieldFilter()
Returns a function which given an index name, returns a predicate which fields must match in order to be returned by get mappings, get index, get field mappings and field capabilities API.IndicesFieldDataCache
getIndicesFieldDataCache()
IndicesQueryCache
getIndicesQueryCache()
QueryRewriteContext
getRewriteContext(java.util.function.LongSupplier nowInMillis)
Returns a newQueryRewriteContext
with the givennow
providerByteSizeValue
getTotalIndexingBufferBytes()
boolean
hasIndex(Index index)
boolean
hasUncompletedPendingDeletes()
Checks if all pending deletes have completed.IndexService
indexService(Index index)
Returns an IndexService for the specified index if exists otherwise returnsnull
.IndexService
indexServiceSafe(Index index)
Returns an IndexService for the specified index if exists otherwise aIndexNotFoundException
is thrown.boolean
isMetaDataField(java.lang.String field)
Returns true if the provided field is a registered metadata field (including ones registered via plugins), false otherwise.java.util.Iterator<IndexService>
iterator()
void
loadIntoContext(ShardSearchRequest request, SearchContext context, QueryPhase queryPhase)
Loads the cache result, computing it if needed by executing the query phase and otherwise deserializing the cached value into thecontext's query result
.void
processPendingDeletes(Index index, IndexSettings indexSettings, TimeValue timeout)
Processes all pending deletes for the given index.void
removeIndex(Index index, IndicesClusterStateService.AllocatedIndices.IndexRemovalReason reason, java.lang.String extraInfo)
Removes the given index from this service and releases all associated resources.NodeIndicesStats
stats(boolean includePrevious)
Returns the node stats indices stats.NodeIndicesStats
stats(boolean includePrevious, CommonStatsFlags flags)
IndexMetaData
verifyIndexIsDeleted(Index index, ClusterState clusterState)
Verify that the contents on disk for the given index is deleted; if not, delete the contents.void
verifyIndexMetadata(IndexMetaData metaData, IndexMetaData metaDataUpdate)
This method verifies that the givenmetaData
holds sane values to create anIndexService
.-
Methods inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent
addLifecycleListener, close, lifecycleState, removeLifecycleListener, start, stop
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.elasticsearch.indices.cluster.IndicesClusterStateService.AllocatedIndices
getShardOrNull
-
-
-
-
Field Detail
-
INDICES_SHARDS_CLOSED_TIMEOUT
public static final java.lang.String INDICES_SHARDS_CLOSED_TIMEOUT
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
IndicesService
public IndicesService(Settings settings, PluginsService pluginsService, NodeEnvironment nodeEnv, NamedXContentRegistry xContentRegistry, AnalysisRegistry analysisRegistry, IndexNameExpressionResolver indexNameExpressionResolver, MapperRegistry mapperRegistry, NamedWriteableRegistry namedWriteableRegistry, ThreadPool threadPool, IndexScopedSettings indexScopedSettings, CircuitBreakerService circuitBreakerService, BigArrays bigArrays, ScriptService scriptService, Client client, MetaStateService metaStateService, java.util.Collection<java.util.function.Function<IndexSettings,java.util.Optional<EngineFactory>>> engineFactoryProviders, java.util.Map<java.lang.String,java.util.function.Function<IndexSettings,IndexStore>> indexStoreFactories)
-
-
Method Detail
-
doStart
protected void doStart()
- Specified by:
doStart
in classAbstractLifecycleComponent
-
doStop
protected void doStop()
- Specified by:
doStop
in classAbstractLifecycleComponent
-
doClose
protected void doClose()
- Specified by:
doClose
in classAbstractLifecycleComponent
-
stats
public NodeIndicesStats stats(boolean includePrevious)
Returns the node stats indices stats. TheincludePrevious
flag controls if old shards stats will be aggregated as well (only for relevant stats, such as refresh and indexing, not for docs/store).
-
stats
public NodeIndicesStats stats(boolean includePrevious, CommonStatsFlags flags)
-
iterator
public java.util.Iterator<IndexService> iterator()
- Specified by:
iterator
in interfacejava.lang.Iterable<IndexService>
-
hasIndex
public boolean hasIndex(Index index)
-
indexService
@Nullable public IndexService indexService(Index index)
Returns an IndexService for the specified index if exists otherwise returnsnull
.- Specified by:
indexService
in interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
-
indexServiceSafe
public IndexService indexServiceSafe(Index index)
Returns an IndexService for the specified index if exists otherwise aIndexNotFoundException
is thrown.
-
createIndex
public IndexService createIndex(IndexMetaData indexMetaData, java.util.List<IndexEventListener> builtInListeners) throws java.io.IOException
Creates a newIndexService
for the given metadata.- Specified by:
createIndex
in interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
- Parameters:
indexMetaData
- the index metadata to create the index forbuiltInListeners
- a list of built-in lifecycleIndexEventListener
that should should be used along side with the per-index listeners- Throws:
ResourceAlreadyExistsException
- if the index already exists.java.io.IOException
-
createIndexMapperService
public MapperService createIndexMapperService(IndexMetaData indexMetaData) throws java.io.IOException
creates a new mapper service for the given index, in order to do administrative work like mapping updates. This *should not* be used for document parsing. Doing so will result in an exception. Note: the returnedMapperService
should be closed when unneeded.- Throws:
java.io.IOException
-
verifyIndexMetadata
public void verifyIndexMetadata(IndexMetaData metaData, IndexMetaData metaDataUpdate) throws java.io.IOException
This method verifies that the givenmetaData
holds sane values to create anIndexService
. This method tries to update the meta data of the createdIndexService
if the givenmetaDataUpdate
is different from the givenmetaData
. This method will throw an exception if the creation or the update fails. The createdIndexService
will not be registered and will be closed immediately.- Throws:
java.io.IOException
-
createShard
public IndexShard createShard(ShardRouting shardRouting, RecoveryState recoveryState, PeerRecoveryTargetService recoveryTargetService, PeerRecoveryTargetService.RecoveryListener recoveryListener, RepositoriesService repositoriesService, java.util.function.Consumer<IndexShard.ShardFailure> onShardFailure, java.util.function.Consumer<ShardId> globalCheckpointSyncer, RetentionLeaseSyncer retentionLeaseSyncer) throws java.io.IOException
Description copied from interface:IndicesClusterStateService.AllocatedIndices
Creates a shard for the specified shard routing and starts recovery.- Specified by:
createShard
in interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
- Parameters:
shardRouting
- the shard routingrecoveryState
- the recovery staterecoveryTargetService
- recovery service for the targetrecoveryListener
- a callback when recovery changes state (finishes or fails)repositoriesService
- service responsible for snapshot/restoreonShardFailure
- a callback when this shard failsglobalCheckpointSyncer
- a callback when this shard syncs the global checkpointretentionLeaseSyncer
- a callback when this shard syncs retention leases- Returns:
- a new shard
- Throws:
java.io.IOException
- if an I/O exception occurs when creating the shard
-
removeIndex
public void removeIndex(Index index, IndicesClusterStateService.AllocatedIndices.IndexRemovalReason reason, java.lang.String extraInfo)
Description copied from interface:IndicesClusterStateService.AllocatedIndices
Removes the given index from this service and releases all associated resources. Persistent parts of the index like the shards files, state and transaction logs are kept around in the case of a disaster recovery.- Specified by:
removeIndex
in interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
- Parameters:
index
- the index to removereason
- the reason to remove the indexextraInfo
- extra information that will be used for logging and reporting
-
getIndicesFieldDataCache
public IndicesFieldDataCache getIndicesFieldDataCache()
-
getCircuitBreakerService
public CircuitBreakerService getCircuitBreakerService()
-
getIndicesQueryCache
public IndicesQueryCache getIndicesQueryCache()
-
deleteUnassignedIndex
public void deleteUnassignedIndex(java.lang.String reason, IndexMetaData metaData, ClusterState clusterState)
Deletes an index that is not assigned to this node. This method cleans up all disk folders relating to the index but does not deal with in-memory structures. For those callremoveIndex(Index, IndexRemovalReason, String)
- Specified by:
deleteUnassignedIndex
in interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
-
deleteShardStore
public void deleteShardStore(java.lang.String reason, ShardLock lock, IndexSettings indexSettings) throws java.io.IOException
Deletes the shard with an already acquired shard lock.- Specified by:
deleteShardStore
in interfaceIndexService.ShardStoreDeleter
- Parameters:
reason
- the reason for the shard deletionlock
- the lock of the shard to deleteindexSettings
- the shards index settings.- Throws:
java.io.IOException
- if an IOException occurs
-
deleteShardStore
public void deleteShardStore(java.lang.String reason, ShardId shardId, ClusterState clusterState) throws java.io.IOException, ShardLockObtainFailedException
This method deletes the shard contents on disk for the given shard ID. This method will fail if the shard deleting is prevented bycanDeleteShardContent(ShardId, IndexSettings)
of if the shards lock can not be acquired. On data nodes, if the deleted shard is the last shard folder in its index, the method will attempt to remove the index folder as well.- Parameters:
reason
- the reason for the shard deletionshardId
- the shards ID to deleteclusterState
- . This is required to access the indexes settings etc.- Throws:
java.io.IOException
- if an IOException occursShardLockObtainFailedException
-
canDeleteIndexContents
public boolean canDeleteIndexContents(Index index, IndexSettings indexSettings)
This method returns true if the current node is allowed to delete the given index. This is the case if the index is deleted in the metadata or there is no allocation on the local node and the index isn't on a shared file system.- Parameters:
index
-Index
to check whether deletion is allowedindexSettings
-IndexSettings
for the given index- Returns:
- true if the index can be deleted on this node
-
verifyIndexIsDeleted
@Nullable public IndexMetaData verifyIndexIsDeleted(Index index, ClusterState clusterState)
Verify that the contents on disk for the given index is deleted; if not, delete the contents. This method assumes that an index is already deleted in the cluster state and/or explicitly through index tombstones.- Specified by:
verifyIndexIsDeleted
in interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
- Parameters:
index
-Index
to make sure its deleted from diskclusterState
-ClusterState
to ensure the index is not part of it- Returns:
- IndexMetaData for the index loaded from disk
-
canDeleteShardContent
public IndicesService.ShardDeletionCheckResult canDeleteShardContent(ShardId shardId, IndexSettings indexSettings)
ReturnsShardDeletionCheckResult
signaling whether the shards content for the given shard can be deleted.- Parameters:
shardId
- the shard to delete.indexSettings
- the shards's relevantIndexSettings
. This is required to access the indexes settings etc.
-
addPendingDelete
public void addPendingDelete(ShardId shardId, IndexSettings settings)
Adds a pending delete for the given index shard.- Specified by:
addPendingDelete
in interfaceIndexService.ShardStoreDeleter
-
addPendingDelete
public void addPendingDelete(Index index, IndexSettings settings)
Adds a pending delete for the given index.
-
processPendingDeletes
public void processPendingDeletes(Index index, IndexSettings indexSettings, TimeValue timeout) throws java.io.IOException, java.lang.InterruptedException, ShardLockObtainFailedException
Processes all pending deletes for the given index. This method will acquire all locks for the given index and will process all pending deletes for this index. Pending deletes might occur if the OS doesn't allow deletion of files because they are used by a different process ie. on Windows where files might still be open by a virus scanner. On a shared filesystem a replica might not have been closed when the primary is deleted causing problems on delete calls so we schedule there deletes later.- Specified by:
processPendingDeletes
in interfaceIndicesClusterStateService.AllocatedIndices<IndexShard,IndexService>
- Parameters:
index
- the index to process the pending deletes fortimeout
- the timeout used for processing pending deletes- Throws:
java.io.IOException
java.lang.InterruptedException
ShardLockObtainFailedException
-
hasUncompletedPendingDeletes
public boolean hasUncompletedPendingDeletes()
Checks if all pending deletes have completed. Used by tests to ensure we don't check directory contents while deletion still ongoing. * The reason is that, on Windows, browsing the directory contents can interfere with the deletion process and delay it unnecessarily.
-
getAnalysis
public AnalysisRegistry getAnalysis()
-
canCache
public boolean canCache(ShardSearchRequest request, SearchContext context)
Can the shard request be cached at all?
-
loadIntoContext
public void loadIntoContext(ShardSearchRequest request, SearchContext context, QueryPhase queryPhase) throws java.lang.Exception
Loads the cache result, computing it if needed by executing the query phase and otherwise deserializing the cached value into thecontext's query result
. The combination of load + compute allows to have a single load operation that will cause other requests with the same key to wait till its loaded an reuse the same cache.- Throws:
java.lang.Exception
-
getTotalIndexingBufferBytes
public ByteSizeValue getTotalIndexingBufferBytes()
-
buildAliasFilter
public AliasFilter buildAliasFilter(ClusterState state, java.lang.String index, java.util.Set<java.lang.String> resolvedExpressions)
-
getRewriteContext
public QueryRewriteContext getRewriteContext(java.util.function.LongSupplier nowInMillis)
Returns a newQueryRewriteContext
with the givennow
provider
-
clearIndexShardCache
public void clearIndexShardCache(ShardId shardId, boolean queryCache, boolean fieldDataCache, boolean requestCache, java.lang.String... fields)
Clears the caches for the given shard id if the shard is still allocated on this node
-
getFieldFilter
public java.util.function.Function<java.lang.String,java.util.function.Predicate<java.lang.String>> getFieldFilter()
Returns a function which given an index name, returns a predicate which fields must match in order to be returned by get mappings, get index, get field mappings and field capabilities API. Useful to filter the fields that such API return. The predicate receives the the field name as input argument. In case multiple plugins register a field filter throughMapperPlugin.getFieldFilter()
, only fields that match all the registered filters will be returned by get mappings, get index, get field mappings and field capabilities API.
-
isMetaDataField
public boolean isMetaDataField(java.lang.String field)
Returns true if the provided field is a registered metadata field (including ones registered via plugins), false otherwise.
-
checkShardLimit
public static java.util.Optional<java.lang.String> checkShardLimit(int newShards, ClusterState state, DeprecationLogger deprecationLogger)
Checks to see if an operation can be performed without taking the cluster over the cluster-wide shard limit. Adds a deprecation warning or returns an error message as appropriate- Parameters:
newShards
- The number of shards to be added by this operationstate
- The current cluster statedeprecationLogger
- The logger to use for deprecation warnings- Returns:
- If present, an error message to be given as the reason for failing an operation. If empty, a sign that the operation is valid.
-
-