net.openhft.chronicle.bytes

Class MappedBytes

java.lang.Object

net.openhft.chronicle.core.io.AbstractReferenceCounted

net.openhft.chronicle.bytes.AbstractBytes<Void>

net.openhft.chronicle.bytes.MappedBytes

All Implemented Interfaces:

Closeable, Appendable, AutoCloseable, CharSequence, Bytes<Void>, BytesIn<Void>, BytesOut<Void>, BytesPrepender<Bytes<Void>>, BytesStore<Bytes<Void>,Void>, ByteStringAppender<Bytes<Void>>, ByteStringParser<Bytes<Void>>, HexDumpBytesDescription<BytesOut<Void>>, HasUncheckedRandomDataInput, RandomDataInput, RandomDataOutput<Bytes<Void>>, DecimalAppender, StreamingCommon<Bytes<Void>>, StreamingDataInput<Bytes<Void>>, StreamingDataOutput<Bytes<Void>>, net.openhft.chronicle.core.io.ManagedCloseable, net.openhft.chronicle.core.io.QueryCloseable, net.openhft.chronicle.core.io.ReferenceCounted, net.openhft.chronicle.core.io.ReferenceCountedTracer, net.openhft.chronicle.core.io.ReferenceOwner, net.openhft.chronicle.core.io.SingleThreadedChecked, net.openhft.chronicle.core.io.Syncable

Direct Known Subclasses:

CommonMappedBytes

public abstract class MappedBytes
extends AbstractBytes<Void>
implements net.openhft.chronicle.core.io.Closeable, net.openhft.chronicle.core.io.ManagedCloseable, net.openhft.chronicle.core.io.Syncable

A specialized implementation of AbstractBytes that wraps memory-mapped data for efficient random file access.

Memory is grouped in chunks of 64 MB by default. The chunk size can be significantly increased if the OS supports sparse files via the OS.isSparseFileSupported() method, e.g. blockSize(512 << 30).

Only the most recently accessed memory chunk is reserved, and the previous chunk is released. For random access, a chunk can be manually reserved by obtaining the bytesStore() and using reserve(owner) on it. However, it is crucial to call release(owner) on the same BytesStore before closing the file to avoid memory leaks.

Several factory methods are provided to create different types of MappedBytes, including single mapped bytes and chunked mapped bytes, with optional settings for read-only mode and chunk overlap size.

Note: MappedBytes, like all Bytes, are single-threaded. Also, it is recommended to ensure the mapped file is reserved before using MappedBytes.

See Also:

BytesStore, MappedFile, AbstractBytes

Field Summary

Fields

Modifier and Type	Field and Description
protected static boolean	TRACE

Fields inherited from class net.openhft.chronicle.bytes.AbstractBytes

bytesStore, isPresent, readPosition, writeLimit

Fields inherited from class net.openhft.chronicle.core.io.AbstractReferenceCounted

referenceCounted, WARN_COUNT, WARN_NS

Fields inherited from interface net.openhft.chronicle.bytes.Bytes

DEFAULT_BYTE_BUFFER_CAPACITY, MAX_CAPACITY, MAX_HEAP_CAPACITY

Fields inherited from interface net.openhft.chronicle.bytes.StreamingDataOutput

JAVA9_STRING_CODER_LATIN, JAVA9_STRING_CODER_UTF16

Fields inherited from interface net.openhft.chronicle.core.io.SingleThreadedChecked

DISABLE_SINGLE_THREADED_CHECK

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	MappedBytes()
protected	MappedBytes(String name)

Method Summary

Modifier and Type	Method and Description
@NotNull Bytes<Void>	bytesForRead() Provides a bytes object for read operations.
@NotNull Bytes<Void>	bytesForWrite() Provides a bytes object for write operations.
abstract void	chunkCount(long[] chunkCount) Updates the number of chunks in the mapped file.
abstract boolean	isBackingFileReadOnly() Checks if the backing file is read-only.
static @NotNull MappedBytes	mappedBytes(@NotNull File file, long chunkSize) Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.
static @NotNull MappedBytes	mappedBytes(@NotNull File file, long chunkSize, long overlapSize) Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.
static @NotNull MappedBytes	mappedBytes(@NotNull File file, long chunkSize, long overlapSize, boolean readOnly)
static @NotNull MappedBytes	mappedBytes(@NotNull File file, long chunkSize, long overlapSize, int pageSize, boolean readOnly) Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.
static @NotNull MappedBytes	mappedBytes(@NotNull MappedFile rw) Create a MappedBytes for a MappedFile
static @NotNull MappedBytes	mappedBytes(@NotNull String filename, long chunkSize) Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.
abstract MappedFile	mappedFile() Retrieves the mapped file.
static @NotNull MappedBytes	readOnly(@NotNull File file) Creates a MappedBytes instance that wraps a read-only memory-mapped file.
boolean	sharedMemory() Checks if the bytes are stored in shared memory.
static @NotNull MappedBytes	singleMappedBytes(@NotNull File file, long capacity) Creates a MappedBytes instance that wraps a single memory-mapped file.
static @NotNull MappedBytes	singleMappedBytes(@NotNull File file, long capacity, boolean readOnly) Creates a MappedBytes instance that wraps a single memory-mapped file.
static @NotNull MappedBytes	singleMappedBytes(@NotNull String filename, long capacity) Creates a MappedBytes instance that wraps a single memory-mapped file.
void	sync() Ensures that any modifications to this MappedBytes instance are written to the storage device containing the mapped file.

Methods inherited from class net.openhft.chronicle.bytes.AbstractBytes

acquireUncheckedInput, addressForRead, addressForWrite, addressForWritePosition, append, append, append, append, append, append, appendAndReturnLength, byteCheckSum, byteCheckSum, byteCheckSum, bytesStore, bytesStore, canReadDirect, canWriteDirect, capacity, clear, clearAndPad, compact, compareAndSwapInt, compareAndSwapLong, copyTo, decimaliser, decimaliser, equals, fpAppend0, fpAppend0, hashCode, isDirectMemory, isElastic, isImmutableEmptyByteStore, lastDecimalPlaces, lastDecimalPlaces, lastNumberHadDigits, lastNumberHadDigits, length, lenient, lenient, move, nativeRead, nativeWrite, peekUnsignedByte, peekUnsignedByte, performRelease, prewrite, prewrite, prewriteByte, prewriteInt, prewriteLong, prewriteOffsetPositionMoved, prewriteShort, read, read, readByte, readByte, readCheckOffset, readDouble, readDouble, readFloat, readFloat, readInt, readInt, readLimit, readLimit, readLong, readLong, readOffsetPositionMoved, readPosition, readPosition, readPositionForHeader, readShort, readShort, readSkip, readUnsignedByte, readUnsignedByte, readVolatileByte, readVolatileInt, readVolatileInt, readVolatileLong, readVolatileLong, readVolatileShort, realCapacity, safeCopySize, start, startsWith, testAndSetInt, toString, uncheckedReadSkipBackOne, uncheckedReadSkipOne, uncheckedReadUnsignedByte, uncheckedWritePosition, underlyingObject, write, write, write, write, write, write, write8bit, write8bit, write8bit, write8bit, writeBoolean, writeByte, writeByte, writeCheckOffset, writeDouble, writeDouble, writeDoubleAndInt, writeFloat, writeFloat, writeInt, writeInt, writeIntAdv, writeLimit, writeLimit, writeLong, writeLong, writeLongAdv, writeOffsetPositionMoved, writeOffsetPositionMoved, writeOrderedInt, writeOrderedInt, writeOrderedLong, writeOrderedLong, writePosition, writePosition, writeShort, writeShort, writeSkip, writeSome, writeVolatileByte, writeVolatileInt, writeVolatileLong, writeVolatileShort

Methods inherited from class net.openhft.chronicle.core.io.AbstractReferenceCounted

addReferenceChangeListener, assertReferencesReleased, backgroundPerformRelease, canReleaseInBackground, createdHere, disableReferenceTracing, enableReferenceTracing, refCount, referenceCountedUnmonitored, referenceId, release, releaseLast, removeReferenceChangeListener, reserve, reserveTransfer, singleThreadedCheckDisabled, singleThreadedCheckReset, threadSafetyCheck, throwExceptionIfNotReleased, throwExceptionIfReleased, tryReserve, unmonitor, warnAndReleaseIfNotReleased

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface net.openhft.chronicle.core.io.ManagedCloseable

createdHere, throwExceptionIfClosed, warnAndCloseIfNotClosed

Methods inherited from interface net.openhft.chronicle.core.io.Closeable

close, closeQuietly, closeQuietly

Methods inherited from interface net.openhft.chronicle.core.io.QueryCloseable

isClosed, isClosing

Methods inherited from interface net.openhft.chronicle.core.io.Syncable

syncIfAvailable

Methods inherited from interface net.openhft.chronicle.bytes.Bytes

allocateDirect, allocateDirect, allocateElasticDirect, allocateElasticDirect, allocateElasticOnHeap, allocateElasticOnHeap, copy, copyTo, copyTo, directFrom, elasticByteBuffer, elasticByteBuffer, elasticByteBuffer, elasticHeapByteBuffer, elasticHeapByteBuffer, empty, ensureCapacity, forFieldGroup, from, from, fromDirect, fromHexString, indexOf, indexOf, isClear, isEqual, readMarshallableLength16, readWithLength, readWrite, safeLimit, toHexString, toHexString, toHexString, toString, toString, toString, unchecked, unchecked, unwrite, valueOf, wrapForRead, wrapForRead, wrapForWrite, wrapForWrite, write, writeMarshallableLength16

Methods inherited from interface net.openhft.chronicle.bytes.BytesStore

addAndGetDouble, addAndGetDoubleNotAtomic, addAndGetFloat, addAndGetFloatNotAtomic, addAndGetInt, addAndGetIntNotAtomic, addAndGetLong, addAndGetShortNotAtomic, addAndGetUnsignedByteNotAtomic, charAt, cipher, cipher, compareAndSwapDouble, compareAndSwapFloat, contentEquals, elasticByteBuffer, endsWith, equalBytes, follow, forFields, from, hash, inside, inside, isEmpty, isEqual, lazyNativeBytesStoreWithFixedCapacity, nativePointer, nativeStore, nativeStoreFrom, nativeStoreWithFixedCapacity, startsWith, subSequence, to8bitString, toDebugString, toDebugString, wrap, wrap, wrap, writeMaxInt, writeMaxLong, zeroOut

Methods inherited from interface net.openhft.chronicle.bytes.RandomDataOutput

append, append, write, write, writeBoolean, writeByte, writeInt24, writeOrderedDouble, writeOrderedFloat, writeUnsignedByte, writeUnsignedInt, writeUnsignedShort, writeUtf8, writeUtf8Limited, writeVolatileDouble, writeVolatileFloat

Methods inherited from interface java.lang.CharSequence

chars, codePoints

Methods inherited from interface net.openhft.chronicle.bytes.BytesIn

bytesMethodReader, bytesMethodReaderBuilder, readObject

Methods inherited from interface net.openhft.chronicle.bytes.RandomDataInput

canReadDirect, compareUtf8, copyTo, fastHash, findByte, parseLong, peekVolatileInt, printable, readBoolean, readIncompleteLong, readUnsignedInt, readUnsignedInt24, readUnsignedShort, readUtf8, readUtf8Limited, readUtf8Limited, readVolatileDouble, readVolatileFloat, subBytes, toByteArray, toTemporaryDirectByteBuffer

Methods inherited from interface net.openhft.chronicle.bytes.ByteStringParser

parse8bit, parse8bit, parse8bit, parse8bit, parse8bit, parseBigDecimal, parseBoolean, parseBoolean, parseDouble, parseFlexibleLong, parseFloat, parseInt, parseLong, parseLongDecimal, parseUtf8, parseUtf8, parseUtf8, reader, skipTo

Methods inherited from interface net.openhft.chronicle.bytes.StreamingDataInput

inputStream, parseHexLong, parseUtf8, parseUtf8, rawReadByte, rawReadInt, rawReadLong, read, read, read, read, read, read8bit, read8bit, read8bit, readBigDecimal, readBigInteger, readBoolean, readChar, readEnum, readHistogram, readIncompleteLong, readInt24, readLimitToCapacity, readPositionRemaining, readPositionUnlimited, readStopBit, readStopBitChar, readStopBitDecimal, readStopBitDouble, readUnsignedInt, readUnsignedInt24, readUnsignedShort, readUtf8, readUtf8, readUtf8, readUtf8, readWithLength, readWithLength, readWithLength0, unsafeRead, unsafeReadObject, unsafeReadObject

Methods inherited from interface net.openhft.chronicle.bytes.BytesOut

bytesMethodWriter, writeObject

Methods inherited from interface net.openhft.chronicle.bytes.ByteStringAppender

append, append, append, append, append, append8bit, append8bit, append8bit, append8bit, append8bit, appendBase, appendBase16, appendBase16, appendDateMillis, appendDecimal, appendTimeMillis, writer

Methods inherited from interface net.openhft.chronicle.bytes.StreamingDataOutput

appendUtf8, appendUtf8, appendUtf8, appendUtf8, appendUtf8, appendUtf8, copyFrom, outputStream, rawWriteByte, rawWriteInt, rawWriteLong, unsafeWrite, unsafeWriteObject, unsafeWriteObject, write, write, write, write, write, write8bit, write8bit, write8bit, writeBigDecimal, writeBigInteger, writeChar, writeEnum, writeHistogram, writeInt24, writePositionForHeader, writePositionRemaining, writeStopBit, writeStopBit, writeStopBit, writeStopBitDecimal, writeUnsignedByte, writeUnsignedInt, writeUnsignedInt24, writeUnsignedShort, writeUtf8, writeUtf8, writeWithLength

Methods inherited from interface net.openhft.chronicle.bytes.BytesPrepender

prepend

Methods inherited from interface net.openhft.chronicle.bytes.HexDumpBytesDescription

adjustHexDumpIndentation, retainedHexDumpDescription, writeHexDumpDescription

Methods inherited from interface net.openhft.chronicle.core.io.SingleThreadedChecked

singleThreadedCheckDisabled, singleThreadedCheckReset

Methods inherited from interface net.openhft.chronicle.core.io.ReferenceCountedTracer

onReleased

Field Detail

TRACE

protected static final boolean TRACE

Constructor Detail

MappedBytes

protected MappedBytes()
               throws net.openhft.chronicle.core.io.ClosedIllegalStateException,
                      net.openhft.chronicle.core.io.ThreadingIllegalStateException

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException

net.openhft.chronicle.core.io.ThreadingIllegalStateException

MappedBytes

protected MappedBytes(String name)
               throws net.openhft.chronicle.core.io.ClosedIllegalStateException,
                      net.openhft.chronicle.core.io.ThreadingIllegalStateException

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException

net.openhft.chronicle.core.io.ThreadingIllegalStateException

Method Detail

singleMappedBytes

@NotNull
public static @NotNull MappedBytes singleMappedBytes(@NotNull
                                                              @NotNull String filename,
                                                              long capacity)
                                                       throws FileNotFoundException,
                                                              IllegalStateException

Creates a MappedBytes instance that wraps a single memory-mapped file.

Parameters:

filename - The name of the file to be memory-mapped.

capacity - The maximum number of bytes that can be read from or written to the mapped file.

Returns:

A new MappedBytes instance.

Throws:

FileNotFoundException - If the file does not exist.

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way

IllegalStateException

singleMappedBytes

@NotNull
public static @NotNull MappedBytes singleMappedBytes(@NotNull
                                                              @NotNull File file,
                                                              long capacity)
                                                       throws FileNotFoundException

Creates a MappedBytes instance that wraps a single memory-mapped file.

Parameters:

file - The name of the file to be memory-mapped.

capacity - The maximum number of bytes that can be read from or written to the mapped file.

Returns:

A new MappedBytes instance.

Throws:

FileNotFoundException - If the file does not exist.

singleMappedBytes

@NotNull
public static @NotNull MappedBytes singleMappedBytes(@NotNull
                                                              @NotNull File file,
                                                              long capacity,
                                                              boolean readOnly)
                                                       throws FileNotFoundException

Creates a MappedBytes instance that wraps a single memory-mapped file.

Parameters:

file - The name of the file to be memory-mapped.

capacity - The maximum number of bytes that can be read from or written to the mapped file.

readOnly - read only is true, read-write if false

Returns:

A new MappedBytes instance.

Throws:

FileNotFoundException - If the file does not exist.

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull String filename,
                                                        long chunkSize)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.

Parameters:

filename - The name of the file to be memory-mapped.

chunkSize - The size of each chunk in bytes.

Returns:

A new MappedBytes instance.

Throws:

FileNotFoundException - If the file does not exist.

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull File file,
                                                        long chunkSize)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.

Parameters:

file - The name of the file to be memory-mapped.

chunkSize - The size of each chunk in bytes.

Returns:

A new MappedBytes instance.

Throws:

FileNotFoundException - If the file does not exist.

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull File file,
                                                        long chunkSize,
                                                        long overlapSize)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.

Parameters:

file - The name of the file to be memory-mapped.

chunkSize - The size of each chunk in bytes.

overlapSize - The size of overlap of chunks in bytes.

Returns:

A new MappedBytes instance.

Throws:

FileNotFoundException - If the file does not exist.

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull File file,
                                                        long chunkSize,
                                                        long overlapSize,
                                                        int pageSize,
                                                        boolean readOnly)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.

Parameters:

file - The name of the file to be memory-mapped.

chunkSize - The size of each chunk in bytes.

overlapSize - The size of overlap of chunks in bytes.

pageSize - The custom page size in bytes.

readOnly - read only is true, read-write if false

Returns:

A new MappedBytes instance.

Throws:

FileNotFoundException - If the file does not exist.

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull File file,
                                                        long chunkSize,
                                                        long overlapSize,
                                                        boolean readOnly)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Throws:

FileNotFoundException

net.openhft.chronicle.core.io.ClosedIllegalStateException

See Also:

mappedBytes(File, long, long, int, boolean)

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull MappedFile rw)
                                                 throws net.openhft.chronicle.core.io.ClosedIllegalStateException

Create a MappedBytes for a MappedFile

Parameters:

rw - MappedFile to use

Returns:

the MappedBytes

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way.

readOnly

@NotNull
public static @NotNull MappedBytes readOnly(@NotNull
                                                     @NotNull File file)
                                              throws FileNotFoundException,
                                                     net.openhft.chronicle.core.io.ClosedIllegalStateException,
                                                     net.openhft.chronicle.core.io.ThreadingIllegalStateException

Creates a MappedBytes instance that wraps a read-only memory-mapped file.

Parameters:

file - The file to be memory-mapped in read-only mode.

Returns:

A new MappedBytes instance.

Throws:

FileNotFoundException - If the file does not exist.

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way

isBackingFileReadOnly

public abstract boolean isBackingFileReadOnly()

Checks if the backing file is read-only.

Returns:

true if the backing file is read-only, false otherwise.

sharedMemory

public boolean sharedMemory()

Checks if the bytes are stored in shared memory.

Specified by:

sharedMemory in interface Bytes<Void>

Returns:

true if the bytes are stored in shared memory, false otherwise.

chunkCount

public abstract void chunkCount(long[] chunkCount)

Updates the number of chunks in the mapped file.

Parameters:

chunkCount - The new number of chunks.

mappedFile

public abstract MappedFile mappedFile()

Retrieves the mapped file.

Returns:

the MappedFile instance.

sync

public void sync()

Ensures that any modifications to this MappedBytes instance are written to the storage device containing the mapped file.

Specified by:

sync in interface net.openhft.chronicle.core.io.Syncable

bytesForRead

@NotNull
public @NotNull Bytes<Void> bytesForRead()
                                           throws net.openhft.chronicle.core.io.ClosedIllegalStateException

Provides a bytes object for read operations. This object is backed by the current MappedBytes instance.

Specified by:

bytesForRead in interface Bytes<Void>

Specified by:

bytesForRead in interface BytesStore<Bytes<Void>,Void>

Returns:

a Bytes instance for read operations.

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way.

bytesForWrite

@NotNull
public @NotNull Bytes<Void> bytesForWrite()
                                            throws net.openhft.chronicle.core.io.ClosedIllegalStateException

Provides a bytes object for write operations. This object is backed by the current MappedBytes instance.

Specified by:

bytesForWrite in interface Bytes<Void>

Specified by:

bytesForWrite in interface BytesStore<Bytes<Void>,Void>

Returns:

a Bytes instance for write operations.

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way.