Class ParallelUtil
- java.lang.Object
-
- org.neo4j.gds.core.concurrency.ParallelUtil
-
public final class ParallelUtil extends java.lang.Object
-
-
Field Summary
Fields Modifier and Type Field Description static int
DEFAULT_BATCH_SIZE
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static int
adjustedBatchSize(int nodeCount, int concurrency)
static int
adjustedBatchSize(int nodeCount, int concurrency, int minBatchSize)
static long
adjustedBatchSize(long nodeCount, int concurrency, long minBatchSize)
static long
adjustedBatchSize(long nodeCount, int concurrency, long minBatchSize, long maxBatchSize)
static long
adjustedBatchSize(long nodeCount, long batchSize)
static void
awaitTermination(java.util.Collection<java.util.concurrent.Future<?>> futures)
static void
awaitTerminations(java.util.Queue<java.util.concurrent.Future<?>> futures)
static boolean
canRunInParallel(@Nullable java.util.concurrent.ExecutorService executor)
static java.util.concurrent.ForkJoinPool
getFJPoolWithConcurrency(int concurrency)
static void
parallelForEachNode(long nodeCount, int concurrency, java.util.function.LongConsumer consumer)
static void
parallelForEachNode(Graph graph, int concurrency, java.util.function.LongConsumer consumer)
static <T extends java.util.stream.BaseStream<?,T>,R>
RparallelStream(T data, int concurrency, java.util.function.Function<T,R> fn)
Executes the given function in parallel on the givenBaseStream
, using a FJ pool of the requested size.static <T extends java.util.stream.BaseStream<?,T>>
voidparallelStreamConsume(T data, int concurrency, java.util.function.Consumer<T> consumer)
Executes the given function in parallel on the givenBaseStream
, using a FJ pool of the requested size.static <T extends java.lang.Runnable>
voidreadParallel(int concurrency, int batchSize, BatchNodeIterable idMapping, java.util.concurrent.ExecutorService executor, HugeParallelGraphImporter<T> importer)
Executes read operations in parallel, based on the given batch size and executor.static void
readParallel(int concurrency, long size, java.util.concurrent.ExecutorService executor, BiLongConsumer task)
static void
run(java.lang.Runnable task, java.util.concurrent.ExecutorService executor)
Runs a single task and waits until it's finished.static java.util.Collection<java.util.concurrent.Future<?>>
run(java.util.Collection<? extends java.lang.Runnable> tasks, boolean allowSynchronousRun, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)
static void
run(java.util.Collection<? extends java.lang.Runnable> tasks, java.lang.Runnable selfTask, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)
static void
run(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
Runs a collection ofRunnable
s in parallel for their side-effects.static void
run(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)
static void
runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long maxRetries, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long maxRetries, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static void
runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.static java.util.Collection<java.lang.Runnable>
tasks(int concurrency, java.util.function.Function<java.lang.Integer,? extends java.lang.Runnable> newTask)
static java.util.Collection<java.lang.Runnable>
tasks(int concurrency, java.util.function.Supplier<? extends java.lang.Runnable> newTask)
static int
threadCount(int batchSize, int elementCount)
static long
threadCount(long batchSize, long elementCount)
-
-
-
Field Detail
-
DEFAULT_BATCH_SIZE
public static final int DEFAULT_BATCH_SIZE
- See Also:
- Constant Field Values
-
-
Method Detail
-
parallelStream
public static <T extends java.util.stream.BaseStream<?,T>,R> R parallelStream(T data, int concurrency, java.util.function.Function<T,R> fn)
Executes the given function in parallel on the givenBaseStream
, using a FJ pool of the requested size. The concurrency value is assumed to already be validated towards the edition limitation.
-
parallelStreamConsume
public static <T extends java.util.stream.BaseStream<?,T>> void parallelStreamConsume(T data, int concurrency, java.util.function.Consumer<T> consumer)
Executes the given function in parallel on the givenBaseStream
, using a FJ pool of the requested size. The concurrency value is assumed to already be validated towards the edition limitation.
-
parallelForEachNode
public static void parallelForEachNode(Graph graph, int concurrency, java.util.function.LongConsumer consumer)
-
parallelForEachNode
public static void parallelForEachNode(long nodeCount, int concurrency, java.util.function.LongConsumer consumer)
-
threadCount
public static int threadCount(int batchSize, int elementCount)
- Returns:
- the number of threads required to compute elementCount with the given batchSize
-
threadCount
public static long threadCount(long batchSize, long elementCount)
-
adjustedBatchSize
public static int adjustedBatchSize(int nodeCount, int concurrency, int minBatchSize)
- Returns:
- a batch size, so that
nodeCount
is equally divided byconcurrency
but no smaller thanminBatchSize
.
-
adjustedBatchSize
public static int adjustedBatchSize(int nodeCount, int concurrency)
- Returns:
- a batch size, so that
nodeCount
is equally divided byconcurrency
but no smaller thanDEFAULT_BATCH_SIZE
. - See Also:
adjustedBatchSize(int, int, int)
-
adjustedBatchSize
public static long adjustedBatchSize(long nodeCount, int concurrency, long minBatchSize)
- Returns:
- a batch size, so that
nodeCount
is equally divided byconcurrency
but no smaller thanDEFAULT_BATCH_SIZE
. - See Also:
adjustedBatchSize(int, int, int)
-
adjustedBatchSize
public static long adjustedBatchSize(long nodeCount, int concurrency, long minBatchSize, long maxBatchSize)
- Returns:
- a batch size, so that
nodeCount
is equally divided byconcurrency
but no smaller thanminBatchSize
and no larger thanmaxBatchSize
. - See Also:
adjustedBatchSize(long, int, long)
-
adjustedBatchSize
public static long adjustedBatchSize(long nodeCount, long batchSize)
- Returns:
- a batch size, that is
1) at least
batchSize
2) a power of two 3) dividesnodeCount
into int-sized chunks.
-
canRunInParallel
public static boolean canRunInParallel(@Nullable @Nullable java.util.concurrent.ExecutorService executor)
-
readParallel
public static <T extends java.lang.Runnable> void readParallel(int concurrency, int batchSize, BatchNodeIterable idMapping, java.util.concurrent.ExecutorService executor, HugeParallelGraphImporter<T> importer)
Executes read operations in parallel, based on the given batch size and executor.
-
readParallel
public static void readParallel(int concurrency, long size, java.util.concurrent.ExecutorService executor, BiLongConsumer task)
-
tasks
public static java.util.Collection<java.lang.Runnable> tasks(int concurrency, java.util.function.Supplier<? extends java.lang.Runnable> newTask)
-
tasks
public static java.util.Collection<java.lang.Runnable> tasks(int concurrency, java.util.function.Function<java.lang.Integer,? extends java.lang.Runnable> newTask)
-
run
public static void run(java.lang.Runnable task, java.util.concurrent.ExecutorService executor)
Runs a single task and waits until it's finished.
-
run
public static void run(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
Runs a collection ofRunnable
s in parallel for their side-effects. The level of parallelism is defined by the given executor.This is similar to
ExecutorService.invokeAll(Collection)
, except that all Exceptions thrown by any task are chained together.
-
run
public static void run(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)
-
run
public static java.util.Collection<java.util.concurrent.Future<?>> run(java.util.Collection<? extends java.lang.Runnable> tasks, boolean allowSynchronousRun, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)
-
run
public static void run(java.util.Collection<? extends java.lang.Runnable> tasks, java.lang.Runnable selfTask, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executeexecutor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executeexecutor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long maxRetries, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executeexecutor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long maxRetries, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executeexecutor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.The provided
terminationFlag
is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executeterminationFlag
- a flag to check periodically if the execution should be terminatedexecutor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.The provided
terminationFlag
is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executeterminationFlag
- a flag to check periodically if the execution should be terminatedexecutor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.If the pool is full, wait for
waitTime
timeUnit
s and retry submitting the tasks indefinitely.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executewaitTime
- how long to wait between retriestimeUnit
- the unit forwaitTime
executor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.If the pool is full, wait for
waitTime
timeUnit
s and retry submitting the tasks indefinitely.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executewaitTime
- how long to wait between retriestimeUnit
- the unit forwaitTime
executor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.If the pool is full, wait for
waitTime
timeUnit
s and retry submitting the tasks indefinitely.The provided
terminationFlag
is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executewaitTime
- how long to wait between retriestimeUnit
- the unit forwaitTime
terminationFlag
- a flag to check periodically if the execution should be terminatedexecutor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.If the pool is full, wait for
waitTime
timeUnit
s and retry submitting the tasks indefinitely.The provided
terminationFlag
is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executewaitTime
- how long to wait between retriestimeUnit
- the unit forwaitTime
terminationFlag
- a flag to check periodically if the execution should be terminatedexecutor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.If the pool is full, wait for
waitTime
timeUnit
s and retry submitting the tasks at mostmaxRetries
times.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executemaxRetries
- how many retries when submitting on a full pool before giving upwaitTime
- how long to wait between retriestimeUnit
- the unit forwaitTime
executor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.If the pool is full, wait for
waitTime
timeUnit
s and retry submitting the tasks at mostmaxRetries
times.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executemaxRetries
- how many retries when submitting on a full pool before giving upwaitTime
- how long to wait between retriestimeUnit
- the unit forwaitTime
executor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.If the pool is full, wait for
waitTime
timeUnit
s and retry submitting the tasks at mostmaxRetries
times.The provided
terminationFlag
is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executemaxRetries
- how many retries when submitting on a full pool before giving upwaitTime
- how long to wait between retriestimeUnit
- the unit forwaitTime
terminationFlag
- a flag to check periodically if the execution should be terminatedexecutor
- the executor to submit the tasks to
-
runWithConcurrency
public static void runWithConcurrency(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
Try to run all tasks for their side-effects using at mostconcurrency
threads at once.If the concurrency is 1 or less, or there is only a single task, or the provided
ExecutorService
has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.If the tasks are submitted to the
executor
, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is
interrupted
during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.We will try to submit tasks as long as no more than
concurrency
are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more thanconcurrency
tasks are running in the providedexecutor
.We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.
We will try to submit tasks as long as the
executor
can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.If the pool is full, wait for
waitTime
timeUnit
s and retry submitting the tasks at mostmaxRetries
times.The provided
terminationFlag
is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.- Parameters:
concurrency
- how many tasks should be run simultaneoustasks
- the tasks to executemaxRetries
- how many retries when submitting on a full pool before giving upwaitTime
- how long to wait between retriestimeUnit
- the unit forwaitTime
terminationFlag
- a flag to check periodically if the execution should be terminatedexecutor
- the executor to submit the tasks to
-
awaitTermination
public static void awaitTermination(java.util.Collection<java.util.concurrent.Future<?>> futures)
-
awaitTerminations
public static void awaitTerminations(java.util.Queue<java.util.concurrent.Future<?>> futures)
-
getFJPoolWithConcurrency
public static java.util.concurrent.ForkJoinPool getFJPoolWithConcurrency(int concurrency)
-
-