Package org.elasticsearch.index.shard

Class IndexShard

    • Field Detail

      • shardRouting

        protected volatile ShardRouting shardRouting

      • currentEngineReference

        protected final java.util.concurrent.atomic.AtomicReference<Engine> currentEngineReference

    • Method Detail

      • getThreadPool

        public ThreadPool getThreadPool()

      • store

        public Store store()

      • getIndexSort

        public org.apache.lucene.search.Sort getIndexSort()
        Return the sort order of this index, or null if the index has no sort.

      • getPendingPrimaryTerm

        public long getPendingPrimaryTerm()
        USE THIS METHOD WITH CARE! Returns the primary term the index shard is supposed to be on. In case of primary promotion or when a replica learns about a new term due to a new primary, the term that's exposed here will not be the term that the shard internally uses to assign to operations. The shard will auto-correct its internal operation term, but this might take time. See IndexMetaData.primaryTerm(int)

      • getOperationPrimaryTerm

        public long getOperationPrimaryTerm()
        Returns the primary term that is currently being used to assign to operations

      • getQueryCachingPolicy

        public org.apache.lucene.search.QueryCachingPolicy getQueryCachingPolicy()

      • updateShardState

        public void updateShardState​(ShardRouting newRouting,
                             long newPrimaryTerm,
                             java.util.function.BiConsumer<IndexShard,​ActionListener<PrimaryReplicaSyncer.ResyncTask>> primaryReplicaSyncer,
                             long applyingClusterStateVersion,
                             java.util.Set<java.lang.String> inSyncAllocationIds,
                             IndexShardRoutingTable routingTable,
                             java.util.Set<java.lang.String> pre60AllocationIds)
                      throws java.io.IOException
        Description copied from interface: IndicesClusterStateService.Shard
        Updates the shard state based on an incoming cluster state: - Updates and persists the new routing value. - Updates the primary term if this shard is a primary. - Updates the allocation ids that are tracked by the shard if it is a primary. See ReplicationTracker.updateFromMaster(long, Set, IndexShardRoutingTable, Set) for details.
        Specified by:
        updateShardState in interface IndicesClusterStateService.Shard
        Parameters:
        newRouting - the new routing entry
        newPrimaryTerm - the new primary term
        primaryReplicaSyncer - the primary-replica resync action to trigger when a term is increased on a primary
        applyingClusterStateVersion - the cluster state version being applied when updating the allocation IDs from the master
        inSyncAllocationIds - the allocation ids of the currently in-sync shard copies
        routingTable - the shard routing table
        Throws:
        java.io.IOException - if shard state could not be persisted

      • relocated

        public void relocated​(java.util.function.Consumer<ReplicationTracker.PrimaryContext> consumer)
               throws IllegalIndexShardStateException,
                      java.lang.InterruptedException
        Completes the relocation. Operations are blocked and current operations are drained before changing state to relocated. The provided Runnable is executed after all operations are successfully blocked.
        Parameters:
        consumer - a Runnable that is executed after operations are blocked
        Throws:
        IllegalIndexShardStateException - if the shard is not relocating due to concurrent cancellation
        java.lang.InterruptedException - if blocking operations is interrupted

      • applyIndexOperationOnPrimary

        public Engine.IndexResult applyIndexOperationOnPrimary​(long version,
                                                       VersionType versionType,
                                                       SourceToParse sourceToParse,
                                                       long ifSeqNo,
                                                       long ifPrimaryTerm,
                                                       long autoGeneratedTimestamp,
                                                       boolean isRetry)
                                                throws java.io.IOException
        Throws:
        java.io.IOException

      • applyIndexOperationOnReplica

        public Engine.IndexResult applyIndexOperationOnReplica​(long seqNo,
                                                       long version,
                                                       VersionType versionType,
                                                       long autoGeneratedTimeStamp,
                                                       boolean isRetry,
                                                       SourceToParse sourceToParse)
                                                throws java.io.IOException
        Throws:
        java.io.IOException

      • markSeqNoAsNoop

        public Engine.NoOpResult markSeqNoAsNoop​(long seqNo,
                                         java.lang.String reason)
                                  throws java.io.IOException
        Throws:
        java.io.IOException

      • getFailedIndexResult

        public Engine.IndexResult getFailedIndexResult​(java.lang.Exception e,
                                               long version)

      • getFailedDeleteResult

        public Engine.DeleteResult getFailedDeleteResult​(java.lang.Exception e,
                                                 long version)

      • applyDeleteOperationOnPrimary

        public Engine.DeleteResult applyDeleteOperationOnPrimary​(long version,
                                                         java.lang.String type,
                                                         java.lang.String id,
                                                         VersionType versionType,
                                                         long ifSeqNo,
                                                         long ifPrimaryTerm)
                                                  throws java.io.IOException
        Throws:
        java.io.IOException

      • applyDeleteOperationOnReplica

        public Engine.DeleteResult applyDeleteOperationOnReplica​(long seqNo,
                                                         long version,
                                                         java.lang.String type,
                                                         java.lang.String id,
                                                         VersionType versionType)
                                                  throws java.io.IOException
        Throws:
        java.io.IOException

      • refresh

        public void refresh​(java.lang.String source)
        Writes all indexing changes to disk and opens a new searcher reflecting all changes. This can throw AlreadyClosedException.

      • getWritingBytes

        public long getWritingBytes()
        Returns how many bytes we are currently moving from heap to disk

      • commitStats

        public CommitStats commitStats()
        Returns:
        CommitStats
        Throws:
        org.apache.lucene.store.AlreadyClosedException - if shard is closed

      • seqNoStats

        public SeqNoStats seqNoStats()
        Returns:
        SeqNoStats
        Throws:
        org.apache.lucene.store.AlreadyClosedException - if shard is closed

      • indexingStats

        public IndexingStats indexingStats​(java.lang.String... types)

      • searchStats

        public SearchStats searchStats​(java.lang.String... groups)

      • segmentStats

        public SegmentsStats segmentStats​(boolean includeSegmentFileSizes)

      • fieldDataStats

        public FieldDataStats fieldDataStats​(java.lang.String... fields)

      • completionStats

        public CompletionStats completionStats​(java.lang.String... fields)

      • flush

        public Engine.CommitId flush​(FlushRequest request)
        Executes the given flush request against the engine.
        Parameters:
        request - the flush request
        Returns:
        the commit ID

      • trimTranslog

        public void trimTranslog()
        checks and removes translog files that no longer need to be retained. See TranslogDeletionPolicy for details

      • forceMerge

        public void forceMerge​(ForceMergeRequest forceMerge)
                throws java.io.IOException
        Throws:
        java.io.IOException

      • upgrade

        public org.apache.lucene.util.Version upgrade​(UpgradeRequest upgrade)
                                       throws java.io.IOException
        Upgrades the shard to the current version of Lucene and returns the minimum segment version
        Throws:
        java.io.IOException

      • minimumCompatibleVersion

        public org.apache.lucene.util.Version minimumCompatibleVersion()

      • acquireLastIndexCommit

        public Engine.IndexCommitRef acquireLastIndexCommit​(boolean flushFirst)
                                             throws EngineException
        Creates a new IndexCommit snapshot from the currently running engine. All resources referenced by this commit won't be freed until the commit / snapshot is closed.
        Parameters:
        flushFirst - true if the index should first be flushed to disk / a low level lucene commit should be executed
        Throws:
        EngineException

      • acquireSafeIndexCommit

        public Engine.IndexCommitRef acquireSafeIndexCommit()
                                             throws EngineException
        Snapshots the most recent safe index commit from the currently running engine. All index files referenced by this index commit won't be freed until the commit/snapshot is closed.
        Throws:
        EngineException

      • snapshotStoreMetadata

        public Store.MetadataSnapshot snapshotStoreMetadata()
                                             throws java.io.IOException
        gets a Store.MetadataSnapshot for the current directory. This method is safe to call in all lifecycle of the index shard, without having to worry about the current state of the engine and concurrent flushes.
        Throws:
        org.apache.lucene.index.IndexNotFoundException - if no index is found in the current directory
        org.apache.lucene.index.CorruptIndexException - if the lucene index is corrupted. This can be caused by a checksum mismatch or an unexpected exception when opening the index reading the segments file.
        org.apache.lucene.index.IndexFormatTooOldException - if the lucene index is too old to be opened.
        org.apache.lucene.index.IndexFormatTooNewException - if the lucene index is too new to be opened.
        java.io.FileNotFoundException - if one or more files referenced by a commit are not present.
        java.nio.file.NoSuchFileException - if one or more files referenced by a commit are not present.
        java.io.IOException

      • failShard

        public void failShard​(java.lang.String reason,
                      @Nullable
                      java.lang.Exception e)
        Fails the shard and marks the shard store as corrupted if e is caused by index corruption

      • acquireSearcher

        public Engine.Searcher acquireSearcher​(java.lang.String source)

      • close

        public void close​(java.lang.String reason,
                  boolean flushEngine)
           throws java.io.IOException
        Throws:
        java.io.IOException

      • prepareForIndexRecovery

        public void prepareForIndexRecovery()
        called before starting to copy index files over

      • trimOperationOfPreviousPrimaryTerms

        public void trimOperationOfPreviousPrimaryTerms​(long aboveSeqNo)

      • getMaxSeenAutoIdTimestamp

        public long getMaxSeenAutoIdTimestamp()
        Returns the maximum auto_id_timestamp of all append-only requests have been processed by this shard or the auto_id_timestamp received from the primary via updateMaxUnsafeAutoIdTimestamp(long) at the beginning of a peer-recovery or a primary-replica resync.
        See Also:
        updateMaxUnsafeAutoIdTimestamp(long)

      • updateMaxUnsafeAutoIdTimestamp

        public void updateMaxUnsafeAutoIdTimestamp​(long maxSeenAutoIdTimestampFromPrimary)
        Since operations stored in soft-deletes do not have max_auto_id_timestamp, the primary has to propagate its max_auto_id_timestamp (via getMaxSeenAutoIdTimestamp() of all processed append-only requests to replicas at the beginning of a peer-recovery or a primary-replica resync to force a replica to disable optimization for all append-only requests which are replicated via replication while its retry variants are replicated via recovery without auto_id_timestamp.

        Without this force-update, a replica can generate duplicate documents (for the same id) if it first receives a retry append-only (without timestamp) via recovery, then an original append-only (with timestamp) via replication.

      • openEngineAndRecoverFromTranslog

        public void openEngineAndRecoverFromTranslog()
                                      throws java.io.IOException
        opens the engine on top of the existing lucene engine and translog. Operations from the translog will be replayed to bring lucene up to date.
        Throws:
        java.io.IOException

      • openEngineAndSkipTranslogRecovery

        public void openEngineAndSkipTranslogRecovery()
                                       throws java.io.IOException
        Opens the engine on top of the existing lucene engine and translog. The translog is kept but its operations won't be replayed.
        Throws:
        java.io.IOException

      • onNewEngine

        protected void onNewEngine​(Engine newEngine)

      • performRecoveryRestart

        public void performRecoveryRestart()
                            throws java.io.IOException
        called if recovery has to be restarted after network error / delay **
        Throws:
        java.io.IOException

      • recoveryStats

        public RecoveryStats recoveryStats()
        returns stats about ongoing recoveries, both source and target

      • finalizeRecovery

        public void finalizeRecovery()
        perform the last stages of recovery once all translog operations are done. note that you should still call postRecovery(String).

      • ignoreRecoveryAttempt

        public boolean ignoreRecoveryAttempt()
        Returns true if this shard can ignore a recovery attempt made to it (since the already doing/done it)

      • isReadAllowed

        public boolean isReadAllowed()
        returns true if the IndexShardState allows reading

      • getIndexBufferRAMBytesUsed

        public long getIndexBufferRAMBytesUsed()
        Returns number of heap bytes used by the indexing buffer for this shard, or 0 if the shard is closed

      • addShardFailureCallback

        public void addShardFailureCallback​(java.util.function.Consumer<IndexShard.ShardFailure> onShardFailure)

      • checkIdle

        public void checkIdle​(long inactiveTimeNS)
        Called by IndexingMemoryController to check whether more than inactiveTimeNS has passed since the last indexing operation, and notify listeners that we are now inactive so e.g. sync'd flush can happen.

      • isActive

        public boolean isActive()

      • recoverFromLocalShards

        public boolean recoverFromLocalShards​(java.util.function.BiConsumer<java.lang.String,​MappingMetaData> mappingUpdateConsumer,
                                      java.util.List<IndexShard> localShards)
                               throws java.io.IOException
        Throws:
        java.io.IOException

      • recoverFromStore

        public boolean recoverFromStore()

      • restoreFromRepository

        public boolean restoreFromRepository​(Repository repository)

      • onSettingsChanged

        public void onSettingsChanged()

      • acquireRetentionLock

        public java.io.Closeable acquireRetentionLock()
        Acquires a lock on the translog files and Lucene soft-deleted documents to prevent them from being trimmed

      • estimateNumberOfHistoryOperations

        public int estimateNumberOfHistoryOperations​(java.lang.String source,
                                             long startingSeqNo)
                                      throws java.io.IOException
        Returns the estimated number of history operations whose seq# at least the provided seq# in this shard.
        Throws:
        java.io.IOException

      • getHistoryOperations

        public Translog.Snapshot getHistoryOperations​(java.lang.String source,
                                              long startingSeqNo)
                                       throws java.io.IOException
        Creates a new history snapshot for reading operations since the provided starting seqno (inclusive). The returned snapshot can be retrieved from either Lucene index or translog files.
        Throws:
        java.io.IOException

      • hasCompleteHistoryOperations

        public boolean hasCompleteHistoryOperations​(java.lang.String source,
                                            long startingSeqNo)
                                     throws java.io.IOException
        Checks if we have a completed history of operations since the given starting seqno (inclusive). This method should be called after acquiring the retention lock; See acquireRetentionLock()
        Throws:
        java.io.IOException

      • getMinRetainedSeqNo

        public long getMinRetainedSeqNo()
        Gets the minimum retained sequence number for this engine.
        Returns:
        the minimum retained sequence number

      • newChangesSnapshot

        public Translog.Snapshot newChangesSnapshot​(java.lang.String source,
                                            long fromSeqNo,
                                            long toSeqNo,
                                            boolean requiredFullRange)
                                     throws java.io.IOException
        Creates a new changes snapshot for reading operations whose seq_no are between fromSeqNo(inclusive) and toSeqNo(inclusive). The caller has to close the returned snapshot after finishing the reading.
        Parameters:
        source - the source of the request
        fromSeqNo - the from seq_no (inclusive) to read
        toSeqNo - the to seq_no (inclusive) to read
        requiredFullRange - if true then Translog.Snapshot.next() will throw IllegalStateException if any operation between fromSeqNo and toSeqNo is missing. This parameter should be only enabled when the entire requesting range is below the global checkpoint.
        Throws:
        java.io.IOException

      • segments

        public java.util.List<Segment> segments​(boolean verbose)

      • flushAndCloseEngine

        public void flushAndCloseEngine()
                         throws java.io.IOException
        Throws:
        java.io.IOException

      • getHistoryUUID

        public java.lang.String getHistoryUUID()

      • activateThrottling

        public void activateThrottling()

      • deactivateThrottling

        public void deactivateThrottling()

      • writeIndexingBuffer

        public void writeIndexingBuffer()
        Called when our shard is using too much heap and should move buffered indexed/deleted documents to disk.

      • updateLocalCheckpointForShard

        public void updateLocalCheckpointForShard​(java.lang.String allocationId,
                                          long checkpoint)
        Notifies the service to update the local checkpoint for the shard with the provided allocation ID. See ReplicationTracker.updateLocalCheckpoint(String, long) for details.
        Parameters:
        allocationId - the allocation ID of the shard to update the local checkpoint for
        checkpoint - the local checkpoint for the shard

      • updateGlobalCheckpointForShard

        public void updateGlobalCheckpointForShard​(java.lang.String allocationId,
                                           long globalCheckpoint)
        Update the local knowledge of the global checkpoint for the specified allocation ID.
        Parameters:
        allocationId - the allocation ID to update the global checkpoint for
        globalCheckpoint - the global checkpoint

      • addGlobalCheckpointListener

        public void addGlobalCheckpointListener​(long waitingForGlobalCheckpoint,
                                        GlobalCheckpointListeners.GlobalCheckpointListener listener,
                                        TimeValue timeout)
        Add a global checkpoint listener. If the global checkpoint is equal to or above the global checkpoint the listener is waiting for, then the listener will be notified immediately via an executor (so possibly not on the current thread). If the specified timeout elapses before the listener is notified, the listener will be notified with an TimeoutException. A caller may pass null to specify no timeout.
        Parameters:
        waitingForGlobalCheckpoint - the global checkpoint the listener is waiting for
        listener - the listener
        timeout - the timeout

      • getRetentionLeases

        public RetentionLeases getRetentionLeases()
        Get all retention leases tracked on this shard.
        Returns:
        the retention leases

      • getRetentionLeases

        public Tuple<java.lang.Boolean,​RetentionLeases> getRetentionLeases​(boolean expireLeases)
        If the expire leases parameter is false, gets all retention leases tracked on this shard and otherwise first calculates expiration of existing retention leases, and then gets all non-expired retention leases tracked on this shard. Note that only the primary shard calculates which leases are expired, and if any have expired, syncs the retention leases to any replicas. If the expire leases parameter is true, this replication tracker must be in primary mode.
        Returns:
        a tuple indicating whether or not any retention leases were expired, and the non-expired retention leases

      • addRetentionLease

        public RetentionLease addRetentionLease​(java.lang.String id,
                                        long retainingSequenceNumber,
                                        java.lang.String source,
                                        ActionListener<ReplicationResponse> listener)
        Adds a new retention lease.
        Parameters:
        id - the identifier of the retention lease
        retainingSequenceNumber - the retaining sequence number
        source - the source of the retention lease
        listener - the callback when the retention lease is successfully added and synced to replicas
        Returns:
        the new retention lease
        Throws:
        java.lang.IllegalArgumentException - if the specified retention lease already exists

      • renewRetentionLease

        public RetentionLease renewRetentionLease​(java.lang.String id,
                                          long retainingSequenceNumber,
                                          java.lang.String source)
        Renews an existing retention lease.
        Parameters:
        id - the identifier of the retention lease
        retainingSequenceNumber - the retaining sequence number
        source - the source of the retention lease
        Returns:
        the renewed retention lease
        Throws:
        java.lang.IllegalArgumentException - if the specified retention lease does not exist

      • removeRetentionLease

        public void removeRetentionLease​(java.lang.String id,
                                 ActionListener<ReplicationResponse> listener)
        Removes an existing retention lease.
        Parameters:
        id - the identifier of the retention lease
        listener - the callback when the retention lease is successfully removed and synced to replicas

      • updateRetentionLeasesOnReplica

        public void updateRetentionLeasesOnReplica​(RetentionLeases retentionLeases)
        Updates retention leases on a replica.
        Parameters:
        retentionLeases - the retention leases

      • loadRetentionLeases

        public RetentionLeases loadRetentionLeases()
                                    throws java.io.IOException
        Loads the latest retention leases from their dedicated state file.
        Returns:
        the retention leases
        Throws:
        java.io.IOException - if an I/O exception occurs reading the retention leases

      • persistRetentionLeases

        public void persistRetentionLeases()
                            throws java.io.IOException
        Persists the current retention leases to their dedicated state file.
        Throws:
        java.io.IOException - if an exception occurs writing the state file

      • assertRetentionLeasesPersisted

        public boolean assertRetentionLeasesPersisted()
                                       throws java.io.IOException
        Throws:
        java.io.IOException

      • syncRetentionLeases

        public void syncRetentionLeases()
        Syncs the current retention leases to all replicas.

      • initiateTracking

        public void initiateTracking​(java.lang.String allocationId)
        Called when the recovery process for a shard has opened the engine on the target shard. Ensures that the right data structures have been set up locally to track local checkpoint information for the shard and that the shard is added to the replication group.
        Parameters:
        allocationId - the allocation ID of the shard for which recovery was initiated

      • markAllocationIdAsInSync

        public void markAllocationIdAsInSync​(java.lang.String allocationId,
                                     long localCheckpoint)
                              throws java.lang.InterruptedException
        Marks the shard with the provided allocation ID as in-sync with the primary shard. See ReplicationTracker.markAllocationIdAsInSync(String, long) for additional details.
        Parameters:
        allocationId - the allocation ID of the shard to mark as in-sync
        localCheckpoint - the current local checkpoint on the shard
        Throws:
        java.lang.InterruptedException

      • getLocalCheckpoint

        public long getLocalCheckpoint()
        Returns the local checkpoint for the shard.
        Returns:
        the local checkpoint

      • getGlobalCheckpoint

        public long getGlobalCheckpoint()
        Returns the global checkpoint for the shard.
        Returns:
        the global checkpoint

      • getLastSyncedGlobalCheckpoint

        public long getLastSyncedGlobalCheckpoint()
        Returns the latest global checkpoint value that has been persisted in the underlying storage (i.e. translog's checkpoint)

      • getInSyncGlobalCheckpoints

        public com.carrotsearch.hppc.ObjectLongMap<java.lang.String> getInSyncGlobalCheckpoints()
        Get the local knowledge of the global checkpoints for all in-sync allocation IDs.
        Returns:
        a map from allocation ID to the local knowledge of the global checkpoint for that allocation ID

      • maybeSyncGlobalCheckpoint

        public void maybeSyncGlobalCheckpoint​(java.lang.String reason)
        Syncs the global checkpoint to the replicas if the global checkpoint on at least one replica is behind the global checkpoint on the primary.

      • getReplicationGroup

        public ReplicationGroup getReplicationGroup()
        Returns the current replication group for the shard.
        Returns:
        the replication group

      • updateGlobalCheckpointOnReplica

        public void updateGlobalCheckpointOnReplica​(long globalCheckpoint,
                                            java.lang.String reason)
        Updates the global checkpoint on a replica shard after it has been updated by the primary.
        Parameters:
        globalCheckpoint - the global checkpoint
        reason - the reason the global checkpoint was updated

      • activateWithPrimaryContext

        public void activateWithPrimaryContext​(ReplicationTracker.PrimaryContext primaryContext)
        Updates the known allocation IDs and the local checkpoints for the corresponding allocations from a primary relocation source.
        Parameters:
        primaryContext - the sequence number context

      • pendingInSync

        public boolean pendingInSync()
        Check if there are any recoveries pending in-sync.
        Returns:
        true if there is at least one shard pending in-sync, otherwise false

      • noopUpdate

        public void noopUpdate​(java.lang.String type)
        Should be called for each no-op update operation to increment relevant statistics.
        Parameters:
        type - the doc type of the update

      • getEngineOrNull

        protected Engine getEngineOrNull()
        NOTE: returns null if engine is not yet started (e.g. recovery phase 1, copying over index files, is still running), or if engine is closed.

      • isRelocatedPrimary

        public boolean isRelocatedPrimary()
        Returns whether the shard is a relocated primary, i.e. not in charge anymore of replicating changes (see ReplicationTracker).

      • acquirePrimaryOperationPermit

        public void acquirePrimaryOperationPermit​(ActionListener<Releasable> onPermitAcquired,
                                          java.lang.String executorOnDelay,
                                          java.lang.Object debugInfo)
        Acquire a primary operation permit whenever the shard is ready for indexing. If a permit is directly available, the provided ActionListener will be called on the calling thread. During relocation hand-off, permit acquisition can be delayed. The provided ActionListener will then be called using the provided executor.
        Parameters:
        debugInfo - an extra information that can be useful when tracing an unreleased permit. When assertions are enabled the tracing will capture the supplied object's Object.toString() value. Otherwise the object isn't used

      • acquireAllPrimaryOperationsPermits

        public void acquireAllPrimaryOperationsPermits​(ActionListener<Releasable> onPermitAcquired,
                                               TimeValue timeout)
        Acquire all primary operation permits. Once all permits are acquired, the provided ActionListener is called. It is the responsibility of the caller to close the Releasable.

      • runUnderPrimaryPermit

        public void runUnderPrimaryPermit​(java.lang.Runnable runnable,
                                  java.util.function.Consumer<java.lang.Exception> onFailure,
                                  java.lang.String executorOnDelay,
                                  java.lang.Object debugInfo)
        Runs the specified runnable under a permit and otherwise calling back the specified failure callback. This method is really a convenience for acquirePrimaryOperationPermit(ActionListener, String, Object) where the listener equates to try-with-resources closing the releasable after executing the runnable on successfully acquiring the permit, an otherwise calling back the failure callback.
        Parameters:
        runnable - the runnable to execute under permit
        onFailure - the callback on failure
        executorOnDelay - the executor to execute the runnable on if permit acquisition is blocked
        debugInfo - debug info

      • acquireReplicaOperationPermit

        public void acquireReplicaOperationPermit​(long opPrimaryTerm,
                                          long globalCheckpoint,
                                          long maxSeqNoOfUpdatesOrDeletes,
                                          ActionListener<Releasable> onPermitAcquired,
                                          java.lang.String executorOnDelay,
                                          java.lang.Object debugInfo)
        Acquire a replica operation permit whenever the shard is ready for indexing (see acquirePrimaryOperationPermit(ActionListener, String, Object)). If the given primary term is lower than then one in shardRouting, the ActionListener.onFailure(Exception) method of the provided listener is invoked with an IllegalStateException. If permit acquisition is delayed, the listener will be invoked on the executor with the specified name.
        Parameters:
        opPrimaryTerm - the operation primary term
        globalCheckpoint - the global checkpoint associated with the request
        maxSeqNoOfUpdatesOrDeletes - the max seq_no of updates (index operations overwrite Lucene) or deletes captured on the primary after this replication request was executed on it (see getMaxSeqNoOfUpdatesOrDeletes()
        onPermitAcquired - the listener for permit acquisition
        executorOnDelay - the name of the executor to invoke the listener on if permit acquisition is delayed
        debugInfo - an extra information that can be useful when tracing an unreleased permit. When assertions are enabled the tracing will capture the supplied object's Object.toString() value. Otherwise the object isn't used

      • acquireAllReplicaOperationsPermits

        public void acquireAllReplicaOperationsPermits​(long opPrimaryTerm,
                                               long globalCheckpoint,
                                               long maxSeqNoOfUpdatesOrDeletes,
                                               ActionListener<Releasable> onPermitAcquired,
                                               TimeValue timeout)
        Acquire all replica operation permits whenever the shard is ready for indexing (see acquireAllPrimaryOperationsPermits(ActionListener, TimeValue). If the given primary term is lower than then one in shardRouting, the ActionListener.onFailure(Exception) method of the provided listener is invoked with an IllegalStateException.
        Parameters:
        opPrimaryTerm - the operation primary term
        globalCheckpoint - the global checkpoint associated with the request
        maxSeqNoOfUpdatesOrDeletes - the max seq_no of updates (index operations overwrite Lucene) or deletes captured on the primary after this replication request was executed on it (see getMaxSeqNoOfUpdatesOrDeletes()
        onPermitAcquired - the listener for permit acquisition
        timeout - the maximum time to wait for the in-flight operations block

      • getActiveOperationsCount

        public int getActiveOperationsCount()

      • getActiveOperations

        public java.util.List<java.lang.String> getActiveOperations()
        Returns:
        a list of describing each permit that wasn't released yet. The description consist of the debugInfo supplied when the permit was acquired plus a stack traces that was captured when the permit was request.

      • sync

        public final void sync​(Translog.Location location,
                       java.util.function.Consumer<java.lang.Exception> syncListener)
        Syncs the given location with the underlying storage unless already synced. This method might return immediately without actually fsyncing the location until the sync listener is called. Yet, unless there is already another thread fsyncing the transaction log the caller thread will be hijacked to run the fsync for all pending fsync operations. This method allows indexing threads to continue indexing without blocking on fsync calls. We ensure that there is only one thread blocking on the sync an all others can continue indexing. NOTE: if the syncListener throws an exception when it's processed the exception will only be logged. Users should make sure that the listener handles all exception cases internally.

      • sync

        public void sync()
          throws java.io.IOException
        Throws:
        java.io.IOException

      • isSyncNeeded

        public boolean isSyncNeeded()
        Checks if the underlying storage sync is required.

      • getTranslogDurability

        public Translog.Durability getTranslogDurability()
        Returns the current translog durability mode

      • afterWriteOperation

        public void afterWriteOperation()
        Schedules a flush or translog generation roll if needed but will not schedule more than one concurrently. The operation will be executed asynchronously on the flush thread pool.

      • isRefreshNeeded

        public boolean isRefreshNeeded()
        Returns true iff one or more changes to the engine are not visible to via the current searcher *or* there are pending refresh listeners. Otherwise false.
        Throws:
        org.apache.lucene.store.AlreadyClosedException - if the engine or internal indexwriter in the engine is already closed

      • addRefreshListener

        public void addRefreshListener​(Translog.Location location,
                               java.util.function.Consumer<java.lang.Boolean> listener)
        Add a listener for refreshes.
        Parameters:
        location - the location to listen for
        listener - for the refresh. Called with true if registering the listener ran it out of slots and forced a refresh. Called with false otherwise.

      • getMaxSeqNoOfUpdatesOrDeletes

        public long getMaxSeqNoOfUpdatesOrDeletes()
        Returns the maximum sequence number of either update or delete operations have been processed in this shard or the sequence number from advanceMaxSeqNoOfUpdatesOrDeletes(long). An index request is considered as an update operation if it overwrites the existing documents in Lucene index with the same document id.

        The primary captures this value after executes a replication request, then transfers it to a replica before executing that replication request on a replica.

      • advanceMaxSeqNoOfUpdatesOrDeletes

        public void advanceMaxSeqNoOfUpdatesOrDeletes​(long seqNo)
        A replica calls this method to advance the max_seq_no_of_updates marker of its engine to at least the max_seq_no_of_updates value (piggybacked in a replication request) that it receives from its primary before executing that replication request. The receiving value is at least as high as the max_seq_no_of_updates on the primary was when any of the operations of that replication request were processed on it.

        A replica shard also calls this method to bootstrap the max_seq_no_of_updates marker with the value that it received from the primary in peer-recovery, before it replays remote translog operations from the primary. The receiving value is at least as high as the max_seq_no_of_updates on the primary was when any of these operations were processed on it.

        These transfers guarantee that every index/delete operation when executing on a replica engine will observe this marker a value which is at least the value of the max_seq_no_of_updates marker on the primary after that operation was executed on the primary.

        See Also:
        acquireReplicaOperationPermit(long, long, long, ActionListener, String, Object), RecoveryTarget.indexTranslogOperations(List, int, long, long, RetentionLeases, ActionListener)

      • verifyShardBeforeIndexClosing

        public void verifyShardBeforeIndexClosing()
                                   throws java.lang.IllegalStateException
        Performs the pre-closing checks on the IndexShard.
        Throws:
        java.lang.IllegalStateException - if the sanity checks failed