Package org.neo4j.gds.core.concurrency
Class ParallelUtil
- java.lang.Object
-
- org.neo4j.gds.core.concurrency.ParallelUtil
-
public final class ParallelUtil extends java.lang.Object
-
-
Field Summary
Fields Modifier and Type Field Description static int
DEFAULT_BATCH_SIZE
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static int
adjustedBatchSize(int nodeCount, int concurrency)
static int
adjustedBatchSize(int nodeCount, int concurrency, int minBatchSize)
static long
adjustedBatchSize(long nodeCount, int concurrency, long minBatchSize)
static long
adjustedBatchSize(long nodeCount, int concurrency, long minBatchSize, long maxBatchSize)
static long
adjustedBatchSize(long nodeCount, long batchSize)
static void
awaitTermination(java.util.Collection<java.util.concurrent.Future<?>> futures)
static boolean
canRunInParallel(@Nullable java.util.concurrent.ExecutorService executor)
static void
parallelForEachNode(long nodeCount, int concurrency, TerminationFlag terminationFlag, java.util.function.LongConsumer consumer)
static <P extends Partition>
voidparallelPartitionsConsume(RunWithConcurrency.Builder runnerBuilder, java.util.stream.Stream<P> partitions, java.util.function.Supplier<PartitionConsumer<P>> taskSupplier)
This method is useful, when |partitions| is greatly larger than concurrency as we only create a single consumer per thread.static <T extends java.util.stream.BaseStream<?,T>,R>
RparallelStream(T data, int concurrency, java.util.function.Function<T,R> fn)
Executes the given function in parallel on the givenBaseStream
, using a FJ pool of the requested size.static <T extends java.util.stream.BaseStream<?,T>>
voidparallelStreamConsume(T data, int concurrency, TerminationFlag terminationFlag, java.util.function.Consumer<T> consumer)
Executes the given function in parallel on the givenBaseStream
, using a FJ pool of the requested size.static void
readParallel(int concurrency, long size, java.util.concurrent.ExecutorService executor, BiLongConsumer task)
static void
run(java.lang.Runnable task, java.util.concurrent.ExecutorService executor)
Runs a single task and waits until it's finished.static java.util.Collection<java.util.concurrent.Future<?>>
run(java.util.Collection<? extends java.lang.Runnable> tasks, boolean allowSynchronousRun, java.util.concurrent.ExecutorService executor, @Nullable java.util.Collection<java.util.concurrent.Future<?>> futures)
static void
run(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
Runs a collection ofRunnable
s in parallel for their side-effects.static void
run(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)
static java.util.Collection<java.lang.Runnable>
tasks(int concurrency, java.util.function.IntFunction<? extends java.lang.Runnable> newTask)
static java.util.Collection<java.lang.Runnable>
tasks(int concurrency, java.util.function.Supplier<? extends java.lang.Runnable> newTask)
static int
threadCount(int batchSize, int elementCount)
static long
threadCount(long batchSize, long elementCount)
-
-
-
Field Detail
-
DEFAULT_BATCH_SIZE
public static final int DEFAULT_BATCH_SIZE
- See Also:
- Constant Field Values
-
-
Method Detail
-
parallelStream
public static <T extends java.util.stream.BaseStream<?,T>,R> R parallelStream(T data, int concurrency, java.util.function.Function<T,R> fn)
Executes the given function in parallel on the givenBaseStream
, using a FJ pool of the requested size. The concurrency value is assumed to already be validated towards the edition limitation.
-
parallelStreamConsume
public static <T extends java.util.stream.BaseStream<?,T>> void parallelStreamConsume(T data, int concurrency, TerminationFlag terminationFlag, java.util.function.Consumer<T> consumer)
Executes the given function in parallel on the givenBaseStream
, using a FJ pool of the requested size. The concurrency value is assumed to already be validated towards the edition limitation.
-
parallelForEachNode
public static void parallelForEachNode(long nodeCount, int concurrency, TerminationFlag terminationFlag, java.util.function.LongConsumer consumer)
-
parallelPartitionsConsume
public static <P extends Partition> void parallelPartitionsConsume(RunWithConcurrency.Builder runnerBuilder, java.util.stream.Stream<P> partitions, java.util.function.Supplier<PartitionConsumer<P>> taskSupplier)
This method is useful, when |partitions| is greatly larger than concurrency as we only create a single consumer per thread. Compared to parallelForEachNode, thread local state does not need to be resolved for each node but only per partition.
-
threadCount
public static int threadCount(int batchSize, int elementCount)
- Returns:
- the number of threads required to compute elementCount with the given batchSize
-
threadCount
public static long threadCount(long batchSize, long elementCount)
-
adjustedBatchSize
public static int adjustedBatchSize(int nodeCount, int concurrency, int minBatchSize)
- Returns:
- a batch size, so that
nodeCount
is equally divided byconcurrency
but no smaller thanminBatchSize
.
-
adjustedBatchSize
public static int adjustedBatchSize(int nodeCount, int concurrency)
- Returns:
- a batch size, so that
nodeCount
is equally divided byconcurrency
but no smaller thanDEFAULT_BATCH_SIZE
. - See Also:
adjustedBatchSize(int, int, int)
-
adjustedBatchSize
public static long adjustedBatchSize(long nodeCount, int concurrency, long minBatchSize)
- Returns:
- a batch size, so that
nodeCount
is equally divided byconcurrency
but no smaller thanminBatchSize
. - See Also:
adjustedBatchSize(int, int, int)
-
adjustedBatchSize
public static long adjustedBatchSize(long nodeCount, int concurrency, long minBatchSize, long maxBatchSize)
- Returns:
- a batch size, so that
nodeCount
is equally divided byconcurrency
but no smaller thanminBatchSize
and no larger thanmaxBatchSize
. - See Also:
adjustedBatchSize(long, int, long)
-
adjustedBatchSize
public static long adjustedBatchSize(long nodeCount, long batchSize)
- Returns:
- a batch size, that is
1) at least
batchSize
2) a power of two 3) dividesnodeCount
into int-sized chunks.
-
canRunInParallel
public static boolean canRunInParallel(@Nullable @Nullable java.util.concurrent.ExecutorService executor)
-
readParallel
public static void readParallel(int concurrency, long size, java.util.concurrent.ExecutorService executor, BiLongConsumer task)
-
tasks
public static java.util.Collection<java.lang.Runnable> tasks(int concurrency, java.util.function.Supplier<? extends java.lang.Runnable> newTask)
-
tasks
public static java.util.Collection<java.lang.Runnable> tasks(int concurrency, java.util.function.IntFunction<? extends java.lang.Runnable> newTask)
-
run
public static void run(java.lang.Runnable task, java.util.concurrent.ExecutorService executor)
Runs a single task and waits until it's finished.
-
run
public static void run(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
Runs a collection ofRunnable
s in parallel for their side-effects. The level of parallelism is defined by the given executor.This is similar to
ExecutorService.invokeAll(Collection)
, except that all Exceptions thrown by any task are chained together.
-
run
public static void run(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)
-
run
public static java.util.Collection<java.util.concurrent.Future<?>> run(java.util.Collection<? extends java.lang.Runnable> tasks, boolean allowSynchronousRun, java.util.concurrent.ExecutorService executor, @Nullable @Nullable java.util.Collection<java.util.concurrent.Future<?>> futures)
-
awaitTermination
public static void awaitTermination(java.util.Collection<java.util.concurrent.Future<?>> futures)
-
-