Package org.elasticsearch.env

Class NodeEnvironment

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable
    public final class NodeEnvironment
extends java.lang.Object
implements java.io.Closeable
    A component that holds all data paths for a single node.

    • Field Detail

      • MAX_LOCAL_STORAGE_NODES_SETTING

        public static final Setting<java.lang.Integer> MAX_LOCAL_STORAGE_NODES_SETTING
        Maximum number of data nodes that should run in an environment.

      • NODE_ID_SEED_SETTING

        public static final Setting<java.lang.Long> NODE_ID_SEED_SETTING
        Seed for determining a persisted unique uuid of this node. If the node has already a persisted uuid on disk, this seed will be ignored and the uuid from disk will be reused.

      • ENABLE_LUCENE_SEGMENT_INFOS_TRACE_SETTING

        public static final Setting<java.lang.Boolean> ENABLE_LUCENE_SEGMENT_INFOS_TRACE_SETTING
        If true the [verbose] SegmentInfos.infoStream logging is sent to System.out.

      • NODE_LOCK_FILENAME

        public static final java.lang.String NODE_LOCK_FILENAME
        See Also:
        Constant Field Values

    • Constructor Detail

      • NodeEnvironment

        public NodeEnvironment​(Settings settings,
                       Environment environment,
                       java.util.function.Consumer<java.lang.String> nodeIdConsumer)
                throws java.io.IOException
        Setup the environment.
        Parameters:
        settings - settings from elasticsearch.yml
        nodeIdConsumer - called as soon as the node id is available to the node name in log messages if it wasn't loaded from elasticsearch.yml
        Throws:
        java.io.IOException

    • Method Detail

      • resolveNodePath

        public static java.nio.file.Path resolveNodePath​(java.nio.file.Path path,
                                                 int nodeLockId)
        Resolve a specific nodes/{node.id} path for the specified path and node lock id.
        Parameters:
        path - the path
        nodeLockId - the node lock id
        Returns:
        the resolved path

      • generateNodeId

        public static java.lang.String generateNodeId​(Settings settings)

      • deleteShardDirectorySafe

        public void deleteShardDirectorySafe​(ShardId shardId,
                                     IndexSettings indexSettings)
                              throws java.io.IOException,
                                     ShardLockObtainFailedException
        Deletes a shard data directory iff the shards locks were successfully acquired.
        Parameters:
        shardId - the id of the shard to delete to delete
        Throws:
        java.io.IOException - if an IOException occurs
        ShardLockObtainFailedException

      • acquireFSLockForPaths

        public static void acquireFSLockForPaths​(IndexSettings indexSettings,
                                         java.nio.file.Path... shardPaths)
                                  throws java.io.IOException
        Acquires, then releases, all write.lock files in the given shard paths. The "write.lock" file is assumed to be under the shard path's "index" directory as used by Elasticsearch.
        Throws:
        org.apache.lucene.store.LockObtainFailedException - if any of the locks could not be acquired
        java.io.IOException

      • deleteShardDirectoryUnderLock

        public void deleteShardDirectoryUnderLock​(ShardLock lock,
                                          IndexSettings indexSettings)
                                   throws java.io.IOException
        Deletes a shard data directory. Note: this method assumes that the shard lock is acquired. This method will also attempt to acquire the write locks for the shard's paths before deleting the data, but this is best effort, as the lock is released before the deletion happens in order to allow the folder to be deleted
        Parameters:
        lock - the shards lock
        Throws:
        java.io.IOException - if an IOException occurs
        ElasticsearchException - if the write.lock is not acquirable

      • deleteIndexDirectorySafe

        public void deleteIndexDirectorySafe​(Index index,
                                     long lockTimeoutMS,
                                     IndexSettings indexSettings)
                              throws java.io.IOException,
                                     ShardLockObtainFailedException
        Deletes an indexes data directory recursively iff all of the indexes shards locks were successfully acquired. If any of the indexes shard directories can't be locked non of the shards will be deleted
        Parameters:
        index - the index to delete
        lockTimeoutMS - how long to wait for acquiring the indices shard locks
        indexSettings - settings for the index being deleted
        Throws:
        java.io.IOException - if any of the shards data directories can't be locked or deleted
        ShardLockObtainFailedException

      • deleteIndexDirectoryUnderLock

        public void deleteIndexDirectoryUnderLock​(Index index,
                                          IndexSettings indexSettings)
                                   throws java.io.IOException
        Deletes an indexes data directory recursively. Note: this method assumes that the shard lock is acquired
        Parameters:
        index - the index to delete
        indexSettings - settings for the index being deleted
        Throws:
        java.io.IOException

      • lockAllForIndex

        public java.util.List<ShardLock> lockAllForIndex​(Index index,
                                                 IndexSettings settings,
                                                 long lockTimeoutMS)
                                          throws java.io.IOException,
                                                 ShardLockObtainFailedException
        Tries to lock all local shards for the given index. If any of the shard locks can't be acquired a ShardLockObtainFailedException is thrown and all previously acquired locks are released.
        Parameters:
        index - the index to lock shards for
        lockTimeoutMS - how long to wait for acquiring the indices shard locks
        Returns:
        the ShardLock instances for this index.
        Throws:
        java.io.IOException - if an IOException occurs.
        ShardLockObtainFailedException

      • shardLock

        public ShardLock shardLock​(ShardId id)
                    throws ShardLockObtainFailedException
        Tries to lock the given shards ID. A shard lock is required to perform any kind of write operation on a shards data directory like deleting files, creating a new index writer or recover from a different shard instance into it. If the shard lock can not be acquired a ShardLockObtainFailedException is thrown. Note: this method will return immediately if the lock can't be acquired.
        Parameters:
        id - the shard ID to lock
        Returns:
        the shard lock. Call ShardLock.close() to release the lock
        Throws:
        ShardLockObtainFailedException

      • shardLock

        public ShardLock shardLock​(ShardId shardId,
                           long lockTimeoutMS)
                    throws ShardLockObtainFailedException
        Tries to lock the given shards ID. A shard lock is required to perform any kind of write operation on a shards data directory like deleting files, creating a new index writer or recover from a different shard instance into it. If the shard lock can not be acquired a ShardLockObtainFailedException is thrown
        Parameters:
        shardId - the shard ID to lock
        lockTimeoutMS - the lock timeout in milliseconds
        Returns:
        the shard lock. Call ShardLock.close() to release the lock
        Throws:
        ShardLockObtainFailedException

      • lockedShards

        public java.util.Set<ShardId> lockedShards()
        Returns all currently lock shards. Note: the shard ids return do not contain a valid Index UUID

      • hasNodeFile

        public boolean hasNodeFile()

      • nodeDataPaths

        public java.nio.file.Path[] nodeDataPaths()
        Returns an array of all of the nodes data locations.
        Throws:
        java.lang.IllegalStateException - if the node is not configured to store local locations

      • sharedDataPath

        public java.nio.file.Path sharedDataPath()
        Returns shared data path for this node environment

      • nodeId

        public java.lang.String nodeId()
        returns the unique uuid describing this node. The uuid is persistent in the data folder of this node and remains across restarts.

      • getNodeLockId

        public int getNodeLockId()

      • indexPaths

        public java.nio.file.Path[] indexPaths​(Index index)
        Returns all index paths.

      • availableIndexFolders

        public java.util.Set<java.lang.String> availableIndexFolders()
                                                      throws java.io.IOException
        Returns all folder names in ${data.paths}/nodes/{node.id}/indices folder
        Throws:
        java.io.IOException

      • availableIndexFolders

        public java.util.Set<java.lang.String> availableIndexFolders​(java.util.function.Predicate<java.lang.String> excludeIndexPathIdsPredicate)
                                                      throws java.io.IOException
        Returns folder names in ${data.paths}/nodes/{node.id}/indices folder that don't match the given predicate.
        Parameters:
        excludeIndexPathIdsPredicate - folder names to exclude
        Throws:
        java.io.IOException

      • availableIndexFoldersForPath

        public java.util.Set<java.lang.String> availableIndexFoldersForPath​(NodeEnvironment.NodePath nodePath)
                                                             throws java.io.IOException
        Return all directory names in the nodes/{node.id}/indices directory for the given node path.
        Parameters:
        nodePath - the path
        Returns:
        all directories that could be indices for the given node path.
        Throws:
        java.io.IOException - if an I/O exception occurs traversing the filesystem

      • availableIndexFoldersForPath

        public java.util.Set<java.lang.String> availableIndexFoldersForPath​(NodeEnvironment.NodePath nodePath,
                                                                    java.util.function.Predicate<java.lang.String> excludeIndexPathIdsPredicate)
                                                             throws java.io.IOException
        Return directory names in the nodes/{node.id}/indices directory for the given node path that don't match the given predicate.
        Parameters:
        nodePath - the path
        excludeIndexPathIdsPredicate - folder names to exclude
        Returns:
        all directories that could be indices for the given node path.
        Throws:
        java.io.IOException - if an I/O exception occurs traversing the filesystem

      • resolveIndexFolder

        public java.nio.file.Path[] resolveIndexFolder​(java.lang.String indexFolderName)
                                        throws java.io.IOException
        Resolves all existing paths to indexFolderName in ${data.paths}/nodes/{node.id}/indices
        Throws:
        java.io.IOException

      • findAllShardIds

        public java.util.Set<ShardId> findAllShardIds​(Index index)
                                       throws java.io.IOException
        Tries to find all allocated shards for the given index on the current node. NOTE: This methods is prone to race-conditions on the filesystem layer since it might not see directories created concurrently or while it's traversing.
        Parameters:
        index - the index to filter shards
        Returns:
        a set of shard IDs
        Throws:
        java.io.IOException - if an IOException occurs

      • shardCountPerPath

        public java.util.Map<NodeEnvironment.NodePath,​java.lang.Long> shardCountPerPath​(Index index)
                                                                               throws java.io.IOException
        Find all the shards for this index, returning a map of the NodePath to the number of shards on that path
        Parameters:
        index - the index by which to filter shards
        Returns:
        a map of NodePath to count of the shards for the index on that path
        Throws:
        java.io.IOException - if an IOException occurs

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable

      • ensureAtomicMoveSupported

        public void ensureAtomicMoveSupported()
                               throws java.io.IOException
        This method tries to write an empty file and moves it using an atomic move operation. This method throws an IllegalStateException if this operation is not supported by the filesystem. This test is executed on each of the data directories. This method cleans up all files even in the case of an error.
        Throws:
        java.io.IOException

      • resolveBaseCustomLocation

        public java.nio.file.Path resolveBaseCustomLocation​(IndexSettings indexSettings)
        Resolve the custom path for a index's shard. Uses the IndexMetaData.SETTING_DATA_PATH setting to determine the root path for the index.
        Parameters:
        indexSettings - settings for the index

      • resolveBaseCustomLocation

        public static java.nio.file.Path resolveBaseCustomLocation​(IndexSettings indexSettings,
                                                           java.nio.file.Path sharedDataPath,
                                                           int nodeLockId)
        Resolve the custom path for a index's shard. Uses the IndexMetaData.SETTING_DATA_PATH setting to determine the root path for the index.
        Parameters:
        indexSettings - settings for the index

      • resolveCustomLocation

        public java.nio.file.Path resolveCustomLocation​(IndexSettings indexSettings,
                                                ShardId shardId)
        Resolve the custom path for a index's shard. Uses the IndexMetaData.SETTING_DATA_PATH setting to determine the root path for the index.
        Parameters:
        indexSettings - settings for the index
        shardId - shard to resolve the path to

      • resolveCustomLocation

        public static java.nio.file.Path resolveCustomLocation​(IndexSettings indexSettings,
                                                       ShardId shardId,
                                                       java.nio.file.Path sharedDataPath,
                                                       int nodeLockId)

      • shardStatePathToDataPath

        public static java.nio.file.Path shardStatePathToDataPath​(java.nio.file.Path shardPath)
        Returns the NodePath.path for this shard.