Package org.opensearch.index.store.remote.file

Class OnDemandBlockSnapshotIndexInput

java.lang.Object
org.apache.lucene.store.DataInput
org.apache.lucene.store.IndexInput
org.opensearch.index.store.remote.file.OnDemandBlockSnapshotIndexInput
All Implemented Interfaces:
Closeable, AutoCloseable, Cloneable, org.apache.lucene.store.RandomAccessInput
public class OnDemandBlockSnapshotIndexInput extends org.apache.lucene.store.IndexInput
This is an implementation of OnDemandBlockIndexInput where this class provides the main IndexInput using shard snapshot files.
This class rely on TransferManager to really fetch the snapshot files from the remote blob store and maybe cache them
Opensearch.internal:

  • Field Details

    • fileInfo

      protected final BlobStoreIndexShardSnapshot.FileInfo fileInfo
      FileInfo contains snapshot metadata references for this IndexInput

    • directory

      protected final org.apache.lucene.store.FSDirectory directory
      Underlying lucene directory to open blocks and for caching

    • fileName

      protected final String fileName
      File Name

    • partSize

      protected final long partSize
      Maximum size in bytes of snapshot file parts.

    • originalFileSize

      protected final long originalFileSize
      Size of the file, larger than length if it's a slice

    • CLEANER_THREAD_NAME_PREFIX

      public static final String CLEANER_THREAD_NAME_PREFIX
      See Also:

    • offset

      protected final long offset
      Start offset of the virtual file : non-zero in the slice case

    • length

      protected final long length
      Length of the virtual file, smaller than actual file size if it's a slice

    • isClone

      protected final boolean isClone
      Whether this index input is a clone or otherwise the root file before slicing

    • blockSizeShift

      protected final int blockSizeShift
      Variables used for block calculation and fetching. blockSize must be a power of two, and is defined as 2^blockShiftSize. blockMask is defined as blockSize - 1 and is used to calculate the offset within a block.

    • blockSize

      protected final int blockSize

    • blockMask

      protected final int blockMask

  • Constructor Details

  • Method Details

    • buildSlice

      protected OnDemandBlockSnapshotIndexInput buildSlice(String sliceDescription, long offset, long length)
      Builds the actual sliced IndexInput (may apply extra offset in subclasses).

    • fetchBlock

      protected org.apache.lucene.store.IndexInput fetchBlock(int blockId) throws IOException
      Given a blockId, fetch it's IndexInput which might be partial/split/cloned one
      Parameters:
      blockId - to fetch for
      Returns:
      fetched IndexInput
      Throws:
      IOException

    • getBlobParts

      protected List<BlobFetchRequest.BlobPart> getBlobParts(long blockStart, long blockEnd)
      Returns list of blob parts/chunks in a file for a given block.

    • clone

      public OnDemandBlockSnapshotIndexInput clone()

    • getActualBlockSize

      protected long getActualBlockSize(int blockId)

    • slice

      public org.apache.lucene.store.IndexInput slice(String sliceDescription, long offset, long length) throws IOException
      Specified by:
      slice in class org.apache.lucene.store.IndexInput
      Throws:
      IOException

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Specified by:
      close in class org.apache.lucene.store.IndexInput
      Throws:
      IOException

    • getFilePointer

      public long getFilePointer()
      Specified by:
      getFilePointer in class org.apache.lucene.store.IndexInput

    • length

      public long length()
      Specified by:
      length in interface org.apache.lucene.store.RandomAccessInput
      Specified by:
      length in class org.apache.lucene.store.IndexInput

    • readByte

      public byte readByte() throws IOException
      Specified by:
      readByte in class org.apache.lucene.store.DataInput
      Throws:
      IOException

    • readShort

      public short readShort() throws IOException
      Overrides:
      readShort in class org.apache.lucene.store.DataInput
      Throws:
      IOException

    • readInt

      public int readInt() throws IOException
      Overrides:
      readInt in class org.apache.lucene.store.DataInput
      Throws:
      IOException

    • readLong

      public long readLong() throws IOException
      Overrides:
      readLong in class org.apache.lucene.store.DataInput
      Throws:
      IOException

    • readVInt

      public final int readVInt() throws IOException
      Overrides:
      readVInt in class org.apache.lucene.store.DataInput
      Throws:
      IOException

    • readVLong

      public final long readVLong() throws IOException
      Overrides:
      readVLong in class org.apache.lucene.store.DataInput
      Throws:
      IOException

    • seek

      public void seek(long pos) throws IOException
      Specified by:
      seek in class org.apache.lucene.store.IndexInput
      Throws:
      IOException

    • readByte

      public final byte readByte(long pos) throws IOException
      Specified by:
      readByte in interface org.apache.lucene.store.RandomAccessInput
      Throws:
      IOException

    • readShort

      public short readShort(long pos) throws IOException
      Specified by:
      readShort in interface org.apache.lucene.store.RandomAccessInput
      Throws:
      IOException

    • readInt

      public int readInt(long pos) throws IOException
      Specified by:
      readInt in interface org.apache.lucene.store.RandomAccessInput
      Throws:
      IOException

    • readLong

      public long readLong(long pos) throws IOException
      Specified by:
      readLong in interface org.apache.lucene.store.RandomAccessInput
      Throws:
      IOException

    • readBytes

      public final void readBytes(byte[] b, int offset, int len) throws IOException
      Specified by:
      readBytes in class org.apache.lucene.store.DataInput
      Throws:
      IOException

    • cloneBlock

      protected void cloneBlock(org.opensearch.index.store.remote.file.OnDemandBlockIndexInput other)

    • getBlock

      protected int getBlock(long pos)

    • getBlockOffset

      protected int getBlockOffset(long pos)

    • getBlockStart

      protected long getBlockStart(int blockId)

    • currentBlockStart

      protected long currentBlockStart()

    • currentBlockPosition

      protected int currentBlockPosition()

    • builder

      public static org.opensearch.index.store.remote.file.OnDemandBlockIndexInput.Builder builder()