Package org.opensearch.indices

Class IndexingMemoryController

java.lang.Object

org.opensearch.indices.IndexingMemoryController

All Implemented Interfaces:

Closeable, AutoCloseable, IndexingOperationListener

public class IndexingMemoryController
extends Object
implements IndexingOperationListener, Closeable

Memory controller for indexing operations

Opensearch.internal:

Nested Class Summary

Nested classes/interfaces inherited from interface org.opensearch.index.shard.IndexingOperationListener

IndexingOperationListener.CompositeListener

Field Summary

Fields

Modifier and Type	Field	Description
static final Setting<org.opensearch.core.common.unit.ByteSizeValue>	INDEX_BUFFER_SIZE_SETTING	How much heap (% or bytes) we will share across all actively indexing shards on this node (default: 10%).
static final Setting<org.opensearch.core.common.unit.ByteSizeValue>	MAX_INDEX_BUFFER_SIZE_SETTING	Only applies when indices.memory.index_buffer_size is a %, to set a ceiling on the actual size in bytes (default: not set).
static final Setting<org.opensearch.core.common.unit.ByteSizeValue>	MIN_INDEX_BUFFER_SIZE_SETTING	Only applies when indices.memory.index_buffer_size is a %, to set a floor on the actual size in bytes (default: 48 MB).
static final Setting<org.opensearch.common.unit.TimeValue>	SHARD_INACTIVE_TIME_SETTING	If we see no indexing operations after this much time for a given shard, we consider that shard inactive (default: 5 minutes).
static final Setting<org.opensearch.common.unit.TimeValue>	SHARD_MEMORY_INTERVAL_TIME_SETTING	How frequently we check indexing memory usage (default: 5 seconds).

Method Summary

Modifier and Type	Method	Description
protected void	activateThrottling(IndexShard shard)	Asks this shard to throttle indexing to one thread
protected List<IndexShard>	availableShards()
protected void	checkIdle(IndexShard shard, long inactiveTimeNS)	ask this shard to check now whether it is inactive, and reduces its indexing buffer if so.
void	close()
protected void	deactivateThrottling(IndexShard shard)	Asks this shard to stop throttling indexing to one thread
protected long	getIndexBufferRAMBytesUsed(IndexShard shard)	returns how much heap this shard is using for its indexing buffer
protected long	getShardWritingBytes(IndexShard shard)	returns how many bytes this shard is currently writing to disk
void	postDelete(org.opensearch.core.index.shard.ShardId shardId, Engine.Delete delete, Engine.DeleteResult result)	Called after the delete operation occurred.
void	postIndex(org.opensearch.core.index.shard.ShardId shardId, Engine.Index index, Engine.IndexResult result)	Called after the indexing operation occurred.
protected Scheduler.Cancellable	scheduleTask(ThreadPool threadPool)
protected void	writeIndexingBufferAsync(IndexShard shard)	ask this shard to refresh, in the background, to free up heap

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.opensearch.index.shard.IndexingOperationListener

postDelete, postIndex, preDelete, preIndex

Field Details

INDEX_BUFFER_SIZE_SETTING

public static final Setting<org.opensearch.core.common.unit.ByteSizeValue> INDEX_BUFFER_SIZE_SETTING

How much heap (% or bytes) we will share across all actively indexing shards on this node (default: 10%).

MIN_INDEX_BUFFER_SIZE_SETTING

public static final Setting<org.opensearch.core.common.unit.ByteSizeValue> MIN_INDEX_BUFFER_SIZE_SETTING

Only applies when indices.memory.index_buffer_size is a %, to set a floor on the actual size in bytes (default: 48 MB).

MAX_INDEX_BUFFER_SIZE_SETTING

public static final Setting<org.opensearch.core.common.unit.ByteSizeValue> MAX_INDEX_BUFFER_SIZE_SETTING

Only applies when indices.memory.index_buffer_size is a %, to set a ceiling on the actual size in bytes (default: not set).

SHARD_INACTIVE_TIME_SETTING

public static final Setting<org.opensearch.common.unit.TimeValue> SHARD_INACTIVE_TIME_SETTING

If we see no indexing operations after this much time for a given shard, we consider that shard inactive (default: 5 minutes).

SHARD_MEMORY_INTERVAL_TIME_SETTING

public static final Setting<org.opensearch.common.unit.TimeValue> SHARD_MEMORY_INTERVAL_TIME_SETTING

How frequently we check indexing memory usage (default: 5 seconds).

Method Details

scheduleTask

protected Scheduler.Cancellable scheduleTask(ThreadPool threadPool)

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

availableShards

protected List<IndexShard> availableShards()

getIndexBufferRAMBytesUsed

protected long getIndexBufferRAMBytesUsed(IndexShard shard)

returns how much heap this shard is using for its indexing buffer

getShardWritingBytes

protected long getShardWritingBytes(IndexShard shard)

returns how many bytes this shard is currently writing to disk

writeIndexingBufferAsync

protected void writeIndexingBufferAsync(IndexShard shard)

ask this shard to refresh, in the background, to free up heap

activateThrottling

protected void activateThrottling(IndexShard shard)

Asks this shard to throttle indexing to one thread

deactivateThrottling

protected void deactivateThrottling(IndexShard shard)

Asks this shard to stop throttling indexing to one thread

postIndex

public void postIndex(org.opensearch.core.index.shard.ShardId shardId,
 Engine.Index index,
 Engine.IndexResult result)

Description copied from interface: IndexingOperationListener

Called after the indexing operation occurred. Note that this is also called when indexing a document did not succeed due to document related failures. See IndexingOperationListener.postIndex(ShardId, Engine.Index, Exception) for engine level failures

Specified by:

postIndex in interface IndexingOperationListener

postDelete

public void postDelete(org.opensearch.core.index.shard.ShardId shardId,
 Engine.Delete delete,
 Engine.DeleteResult result)

Description copied from interface: IndexingOperationListener

Called after the delete operation occurred. Note that this is also called when deleting a document did not succeed due to document related failures. See IndexingOperationListener.postDelete(ShardId, Engine.Delete, Exception) for engine level failures

Specified by:

postDelete in interface IndexingOperationListener

checkIdle

protected void checkIdle(IndexShard shard,
 long inactiveTimeNS)

ask this shard to check now whether it is inactive, and reduces its indexing buffer if so.