Package org.neo4j.gds.core.utils.paged

Class HugeAtomicLongArray

java.lang.Object

org.neo4j.gds.core.utils.paged.HugeAtomicLongArray

All Implemented Interfaces:

HugeCursorSupport<long[]>

public abstract class HugeAtomicLongArray
extends java.lang.Object
implements HugeCursorSupport<long[]>

A long-indexable version of a AtomicLongArray that can contain more than 2 bn. elements.

It is implemented by paging of smaller long-arrays (long[][]) to support approx. 32k bn. elements. If the the provided size is small enough, an optimized view of a single long[] might be used.

• The array is of a fixed size and cannot grow or shrink dynamically.
• The array is not optimized for sparseness and has a large memory overhead if the values written to it are very sparse.
• The array does not support default values and returns the same default for unset values that a regular long[] does (0).
• It only supports a minimal subset of the atomic operations that AtomicLongArray provides.

Basic Usage

 
 AllocationTracker allocationTracker = ...;
 long arraySize = 42L;
 HugeAtomicLongArray array = HugeAtomicLongArray.newArray(arraySize, allocationTracker);
 array.set(13L, 37L);
 long value = array.get(13L);
 // value = 37L
 
 

Implementation is similar to the AtomicLongArray, based on sun.misc.Unsafe https://hg.openjdk.java.net/jdk/jdk13/file/9e0c80381e32/jdk/src/java.base/share/classes/java/util/concurrent/atomic/AtomicLongArray.java

Constructor Summary

Constructors

Constructor	Description
HugeAtomicLongArray()

Method Summary

Modifier and Type	Method	Description
LongNodeProperties	asNodeProperties()
abstract long	compareAndExchange(long index, long expect, long update)	Atomically sets the element at position index to the given updated value if the current value, referred to as the witness value, == the expected value.
abstract boolean	compareAndSet(long index, long expect, long update)	Atomically sets the element at position index to the given updated value if the current value == the expected value.
abstract void	copyTo(HugeAtomicLongArray dest, long length)
abstract long	get(long index)
abstract long	getAndAdd(long index, long delta)	Atomically adds the given delta to the value at the given index.
static long	memoryEstimation(long size)
static HugeAtomicLongArray	newArray(long size, AllocationTracker allocationTracker)	Creates a new array of the given size, tracking the memory requirements into the given AllocationTracker.
static HugeAtomicLongArray	newArray(long size, LongPageCreator pageFiller, AllocationTracker allocationTracker)	Creates a new array of the given size, tracking the memory requirements into the given AllocationTracker.
abstract HugeCursor<long[]>	newCursor()	Returns a new HugeCursor for this array.
abstract long	release()	Destroys the data, allowing the underlying storage arrays to be collected as garbage.
abstract void	set(long index, long value)	Sets the long value at the given index to the given value.
abstract void	setAll(long value)	Set all entries in the array to the given value.
abstract long	size()	Returns the length of this array.
abstract long	sizeOf()
abstract void	update(long index, java.util.function.LongUnaryOperator updateFunction)	Atomically updates the element at index index with the results of applying the given function, returning the updated value.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.neo4j.gds.core.utils.paged.HugeCursorSupport

initCursor, initCursor

Constructor Detail

HugeAtomicLongArray

public HugeAtomicLongArray()

Method Detail

newCursor

public abstract HugeCursor<long[]> newCursor()

Returns a new HugeCursor for this array. The cursor is not positioned and in an invalid state. To position the cursor you must call HugeCursorSupport.initCursor(HugeCursor) or HugeCursorSupport.initCursor(HugeCursor, long, long). Then the cursor needs to be put in a valid state by calling HugeCursor.next(). Obtaining a HugeCursor for an empty array (where HugeCursorSupport.size() returns 0) is undefined and might result in a NullPointerException or another RuntimeException.

Specified by:

newCursor in interface HugeCursorSupport<long[]>

get

public abstract long get(long index)

Returns:

the long value at the given index

Throws:

java.lang.ArrayIndexOutOfBoundsException - if the index is not within size()

getAndAdd

public abstract long getAndAdd(long index,
                               long delta)

Atomically adds the given delta to the value at the given index.

Parameters:

index - the index

delta - the value to add

Returns:

the previous value at index

set

public abstract void set(long index,
                         long value)

Sets the long value at the given index to the given value.

Throws:

java.lang.ArrayIndexOutOfBoundsException - if the index is not within size()

compareAndSet

public abstract boolean compareAndSet(long index,
                                      long expect,
                                      long update)

Atomically sets the element at position index to the given updated value if the current value == the expected value.

Parameters:

index - the index

expect - the expected value

update - the new value

Returns:

true if successful. False return indicates that the actual value was not equal to the expected value.

compareAndExchange

public abstract long compareAndExchange(long index,
                                        long expect,
                                        long update)

Atomically sets the element at position index to the given updated value if the current value, referred to as the witness value, == the expected value. This operation works as if implemented as

     if (this.compareAndSet(index, expect, update)) {
         return expect;
     } else {
         return this.get(index);
     }
 

The actual implementation is done with a single atomic operation so that the returned witness value is the value that was failing the update, not one that needs be read again after the failed update. This allows one to write CAS-loops in a different way, which removes one volatile read per loop iteration

     var oldValue = this.get(index);
     while (true) {
         var newValue = updateFunction(oldValue);
         var witnessValue = this.compareAndExchange(index, oldValue, newValue);
         if (witnessValue == oldValue) {
             // update successful
             break;
         }
         // update unsuccessful set, loop and try again.
         // Here we already have the updated witness value and don't need to issue
         // a new read
         oldValue = witnessValue;
     }
 

Parameters:

index - the index

expect - the expected value

update - the new value

Returns:

the result that is the witness value, which will be the same as the expected value if successful or the new current value if unsuccessful.

update

public abstract void update(long index,
                            java.util.function.LongUnaryOperator updateFunction)

Atomically updates the element at index index with the results of applying the given function, returning the updated value. The function should be side-effect-free, since it may be re-applied when attempted updates fail due to contention among threads.

Parameters:

index - the index

updateFunction - a side-effect-free function

size

public abstract long size()

Returns the length of this array.

If the size is greater than zero, the highest supported index is size() - 1

The behavior is identical to calling array.length on primitive arrays.

Specified by:

size in interface HugeCursorSupport<long[]>

sizeOf

public abstract long sizeOf()

Returns:

the amount of memory used by the instance of this array, in bytes. This should be the same as returned from release() without actually releasing the array.

setAll

public abstract void setAll(long value)

Set all entries in the array to the given value. This method is not atomic!

release

public abstract long release()

Destroys the data, allowing the underlying storage arrays to be collected as garbage. The array is unusable after calling this method and will throw NullPointerExceptions on virtually every method invocation.

Note that the data might not immediately collectible if there are still cursors alive that reference this array. You have to HugeCursor.close() every cursor instance as well.

The amount is not removed from the AllocationTracker that had been provided in the constructor.

Returns:

the amount of memory freed, in bytes.

asNodeProperties

public LongNodeProperties asNodeProperties()

copyTo

public abstract void copyTo(HugeAtomicLongArray dest,
                            long length)

newArray

public static HugeAtomicLongArray newArray(long size,
                                           AllocationTracker allocationTracker)

Creates a new array of the given size, tracking the memory requirements into the given AllocationTracker. The tracker is no longer referenced, as the arrays do not dynamically change their size.

newArray

public static HugeAtomicLongArray newArray(long size,
                                           LongPageCreator pageFiller,
                                           AllocationTracker allocationTracker)

Creates a new array of the given size, tracking the memory requirements into the given AllocationTracker. The tracker is no longer referenced, as the arrays do not dynamically change their size. The values are pre-calculated according to the semantics of Arrays.setAll(long[], java.util.function.IntToLongFunction)

memoryEstimation

public static long memoryEstimation(long size)