Package org.nustaq.offheap
Class FSTBinaryOffheapMap
java.lang.Object
org.nustaq.offheap.FSTBinaryOffheapMap
- Direct Known Subclasses:
FSTCodedOffheapMap
Baseclass of offheap maps.
FST OffHeap Maps enable to store key/value pairs in offheap memory. Additionally it provides
and iterator interface for all values. In order to also iterate keys, add each key to its value object,
as this OffHeap map does not support iteration of keys out of the box.
See subclasses for directly applicable classes (E.g. FSTStringOffheapMap)
The base is a generic bytesource to bytesource map. Note that key should be as short as possible (4-20 bytes) and
should have their most modified digit at the last character of their value.
e.g. [0,0,0,0,123,44] where '44' changes with each new key. Else on-heap memory consumption will grow.
Performance of lookup degrades with growing key size.
In case entries are updated frequently with values of different size, avoid fragementation
by adding extra space to each entry. Override getEntryLengthForContentLength for this.
-
Nested Class Summary
-
Field Summary
Modifier and TypeFieldDescriptionprotected MallocBytezAllocator
protected long
static final int
static final int
protected Bytez
static final int
protected FreeList
static final long
protected OffHeapByteTree
static final int
protected int
protected String
static final long
protected Bytez
protected int
protected int
-
Constructor Summary
ConstructorDescriptionFSTBinaryOffheapMap
(int keyLen, long sizeMemBytes, int numberOfElems) FSTBinaryOffheapMap
(String mappedFile, int keyLen, long sizeMemBytes, int numberOfElems) -
Method Summary
Modifier and TypeMethodDescriptionprotected long
addEntry
(ByteSource key, ByteSource value) protected void
addToFreeList
(long offset) protected void
decElems()
void
protected void
finalize()
void
free()
getBinary
(ByteSource key) get an entry. the returned ByteSource must be processed immediately as it will be reused internally on next get Warning: Concurrent Modification (e.g. add remove elements during iteration) is NOT SUPPORTED and NOT CHECKED.int
protected int
getContentLenFromHeader
(long off) protected int
getEntryLengthForContentLength
(long lengthOfEntry) called upon add, allows to reserve extra space for later growth per entrylong
protected int
protected int
getLenFromHeader
(long off) int
getSize()
long
protected void
incElems()
protected void
init
(int keyLen, long sizeMemBytes, int numberOfElems) protected void
initFromFile
(String file, int keyLen, long sizeMemBytes, int numberOfElems) static void
printBinaryKey
(ByteSource key) void
putBinary
(ByteSource key, ByteSource value) void
removeBinary
(ByteSource key) remove the key from the binary mapprotected void
removeEntry
(long offset) void
resizeStore
(long required, long maxgrowbytes) PRIVILEGED method.protected void
setEntry
(long off, int entryLen, ByteSource value) protected void
writeEntryHeader
(long offset, int entryLen, int contentLen, boolean removed)
-
Field Details
-
MB
public static final long MB- See Also:
-
GB
public static final long GB- See Also:
-
CUSTOM_FILEHEADER_LEN
public static final int CUSTOM_FILEHEADER_LEN- See Also:
-
CORE_HEADER_LEN
public static final int CORE_HEADER_LEN- See Also:
-
FILE_HEADER_LEN
public static final int FILE_HEADER_LEN- See Also:
-
KEY_OFFSET_IN_HEADER
public static final int KEY_OFFSET_IN_HEADER- See Also:
-
index
-
memory
-
customHeader
-
alloc
-
numElem
protected int numElem -
keyLen
protected int keyLen -
bytezOffset
protected long bytezOffset -
freeList
-
mappedFile
-
mutationCount
protected int mutationCount
-
-
Constructor Details
-
Method Details
-
getCustomFileHeader
-
initFromFile
protected void initFromFile(String file, int keyLen, long sizeMemBytes, int numberOfElems) throws Exception - Throws:
Exception
-
init
protected void init(int keyLen, long sizeMemBytes, int numberOfElems) -
finalize
-
free
public void free() -
putBinary
-
removeEntry
protected void removeEntry(long offset) -
addToFreeList
protected void addToFreeList(long offset) -
setEntry
-
addEntry
-
resizeStore
public void resizeStore(long required, long maxgrowbytes) PRIVILEGED method. You gotta know what your doing here .. currently a very expensive operation .. frees everything, resize file and remap. Remapping involves rebuild of index.- Parameters:
required
-
-
getBinary
get an entry. the returned ByteSource must be processed immediately as it will be reused internally on next get Warning: Concurrent Modification (e.g. add remove elements during iteration) is NOT SUPPORTED and NOT CHECKED. Collect keys to change inside iteration and perform changes after iteration is finished.- Parameters:
key
-- Returns:
-
removeBinary
remove the key from the binary map- Parameters:
key
-
-
decElems
protected void decElems() -
incElems
protected void incElems() -
getEntryLengthForContentLength
protected int getEntryLengthForContentLength(long lengthOfEntry) called upon add, allows to reserve extra space for later growth per entry- Parameters:
lengthOfEntry
-- Returns:
-
writeEntryHeader
protected void writeEntryHeader(long offset, int entryLen, int contentLen, boolean removed) -
getHeaderLen
protected int getHeaderLen() -
getLenFromHeader
protected int getLenFromHeader(long off) -
getContentLenFromHeader
protected int getContentLenFromHeader(long off) -
binaryValues
-
printBinaryKey
-
binaryKeys
-
getFreeMem
public long getFreeMem() -
getUsedMem
public long getUsedMem() -
getCapacityMB
public int getCapacityMB() -
getSize
public int getSize() -
dumpIndexStats
public void dumpIndexStats() -
getFileName
-
main
- Throws:
Exception
-