Package org.neo4j.gds.core.concurrency

Class ParallelUtil

  • public final class ParallelUtil
extends java.lang.Object

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static int adjustedBatchSize​(int nodeCount, int concurrency)  
      static int adjustedBatchSize​(int nodeCount, int concurrency, int minBatchSize)  
      static long adjustedBatchSize​(long nodeCount, int concurrency, long minBatchSize)  
      static long adjustedBatchSize​(long nodeCount, int concurrency, long minBatchSize, long maxBatchSize)  
      static long adjustedBatchSize​(long nodeCount, long batchSize)  
      static void awaitTermination​(java.util.Collection<java.util.concurrent.Future<?>> futures)  
      static void awaitTerminations​(java.util.Queue<java.util.concurrent.Future<?>> futures)  
      static boolean canRunInParallel​(@Nullable java.util.concurrent.ExecutorService executor)  
      static void parallelForEachNode​(long nodeCount, int concurrency, java.util.function.LongConsumer consumer)  
      static void parallelForEachNode​(Graph graph, int concurrency, java.util.function.LongConsumer consumer)  
      static <T extends java.util.stream.BaseStream<?,​T>,​R>
      R      		 parallelStream​(T data, int concurrency, java.util.function.Function<T,​R> fn)
      Executes the given function in parallel on the given BaseStream, using a FJ pool of the requested size.
      static <T extends java.util.stream.BaseStream<?,​T>>
      void      		 parallelStreamConsume​(T data, int concurrency, java.util.function.Consumer<T> consumer)
      Executes the given function in parallel on the given BaseStream, using a FJ pool of the requested size.
      static <T extends java.lang.Runnable>
      void      		 readParallel​(int concurrency, int batchSize, BatchNodeIterable idMap, java.util.concurrent.ExecutorService executor, HugeParallelGraphImporter<T> importer)
      Executes read operations in parallel, based on the given batch size and executor.
      static void readParallel​(int concurrency, long size, java.util.concurrent.ExecutorService executor, BiLongConsumer task)  
      static void run​(java.lang.Runnable task, java.util.concurrent.ExecutorService executor)
      Runs a single task and waits until it's finished.
      static java.util.Collection<java.util.concurrent.Future<?>> run​(java.util.Collection<? extends java.lang.Runnable> tasks, boolean allowSynchronousRun, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)  
      static void run​(java.util.Collection<? extends java.lang.Runnable> tasks, java.lang.Runnable selfTask, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)  
      static void run​(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
      Runs a collection of Runnables in parallel for their side-effects.
      static void run​(java.util.Collection<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor, java.util.Collection<java.util.concurrent.Future<?>> futures)  
      static void runWithConcurrency​(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long maxRetries, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.lang.Iterable<? extends java.lang.Runnable> tasks, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long maxRetries, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long maxRetries, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, long waitTime, java.util.concurrent.TimeUnit timeUnit, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static void runWithConcurrency​(int concurrency, java.util.Iterator<? extends java.lang.Runnable> tasks, TerminationFlag terminationFlag, java.util.concurrent.ExecutorService executor)
      Try to run all tasks for their side-effects using at most concurrency threads at once.
      static java.util.Collection<java.lang.Runnable> tasks​(int concurrency, java.util.function.Function<java.lang.Integer,​? extends java.lang.Runnable> newTask)  
      static java.util.Collection<java.lang.Runnable> tasks​(int concurrency, java.util.function.Supplier<? extends java.lang.Runnable> newTask)  
      static int threadCount​(int batchSize, int elementCount)  
      static long threadCount​(long batchSize, long elementCount)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • parallelStream

        public static <T extends java.util.stream.BaseStream<?,​T>,​R> R parallelStream​(T data,
                                                                                          int concurrency,
                                                                                          java.util.function.Function<T,​R> fn)
        Executes the given function in parallel on the given BaseStream, using a FJ pool of the requested size. The concurrency value is assumed to already be validated towards the edition limitation.

      • parallelStreamConsume

        public static <T extends java.util.stream.BaseStream<?,​T>> void parallelStreamConsume​(T data,
                                                                                            int concurrency,
                                                                                            java.util.function.Consumer<T> consumer)
        Executes the given function in parallel on the given BaseStream, using a FJ pool of the requested size. The concurrency value is assumed to already be validated towards the edition limitation.

      • parallelForEachNode

        public static void parallelForEachNode​(Graph graph,
                                       int concurrency,
                                       java.util.function.LongConsumer consumer)

      • parallelForEachNode

        public static void parallelForEachNode​(long nodeCount,
                                       int concurrency,
                                       java.util.function.LongConsumer consumer)

      • threadCount

        public static int threadCount​(int batchSize,
                              int elementCount)
        Returns:
        the number of threads required to compute elementCount with the given batchSize

      • threadCount

        public static long threadCount​(long batchSize,
                               long elementCount)

      • adjustedBatchSize

        public static int adjustedBatchSize​(int nodeCount,
                                    int concurrency,
                                    int minBatchSize)
        Returns:
        a batch size, so that nodeCount is equally divided by concurrency but no smaller than minBatchSize.

      • adjustedBatchSize

        public static int adjustedBatchSize​(int nodeCount,
                                    int concurrency)
        Returns:
        a batch size, so that nodeCount is equally divided by concurrency but no smaller than DEFAULT_BATCH_SIZE.
        See Also:
        adjustedBatchSize(int, int, int)

      • adjustedBatchSize

        public static long adjustedBatchSize​(long nodeCount,
                                     int concurrency,
                                     long minBatchSize)
        Returns:
        a batch size, so that nodeCount is equally divided by concurrency but no smaller than DEFAULT_BATCH_SIZE.
        See Also:
        adjustedBatchSize(int, int, int)

      • adjustedBatchSize

        public static long adjustedBatchSize​(long nodeCount,
                                     int concurrency,
                                     long minBatchSize,
                                     long maxBatchSize)
        Returns:
        a batch size, so that nodeCount is equally divided by concurrency but no smaller than minBatchSize and no larger than maxBatchSize.
        See Also:
        adjustedBatchSize(long, int, long)

      • adjustedBatchSize

        public static long adjustedBatchSize​(long nodeCount,
                                     long batchSize)
        Returns:
        a batch size, that is 1) at least batchSize 2) a power of two 3) divides nodeCount into int-sized chunks.

      • canRunInParallel

        public static boolean canRunInParallel​(@Nullable
                                       @Nullable java.util.concurrent.ExecutorService executor)

      • readParallel

        public static <T extends java.lang.Runnable> void readParallel​(int concurrency,
                                                               int batchSize,
                                                               BatchNodeIterable idMap,
                                                               java.util.concurrent.ExecutorService executor,
                                                               HugeParallelGraphImporter<T> importer)
        Executes read operations in parallel, based on the given batch size and executor.

      • readParallel

        public static void readParallel​(int concurrency,
                                long size,
                                java.util.concurrent.ExecutorService executor,
                                BiLongConsumer task)

      • tasks

        public static java.util.Collection<java.lang.Runnable> tasks​(int concurrency,
                                                             java.util.function.Supplier<? extends java.lang.Runnable> newTask)

      • tasks

        public static java.util.Collection<java.lang.Runnable> tasks​(int concurrency,
                                                             java.util.function.Function<java.lang.Integer,​? extends java.lang.Runnable> newTask)

      • run

        public static void run​(java.lang.Runnable task,
                       java.util.concurrent.ExecutorService executor)
        Runs a single task and waits until it's finished.

      • run

        public static void run​(java.util.Collection<? extends java.lang.Runnable> tasks,
                       java.util.concurrent.ExecutorService executor)
        Runs a collection of Runnables in parallel for their side-effects. The level of parallelism is defined by the given executor.

        This is similar to ExecutorService.invokeAll(Collection), except that all Exceptions thrown by any task are chained together.

      • run

        public static void run​(java.util.Collection<? extends java.lang.Runnable> tasks,
                       java.util.concurrent.ExecutorService executor,
                       java.util.Collection<java.util.concurrent.Future<?>> futures)

      • run

        public static java.util.Collection<java.util.concurrent.Future<?>> run​(java.util.Collection<? extends java.lang.Runnable> tasks,
                                                                       boolean allowSynchronousRun,
                                                                       java.util.concurrent.ExecutorService executor,
                                                                       java.util.Collection<java.util.concurrent.Future<?>> futures)

      • run

        public static void run​(java.util.Collection<? extends java.lang.Runnable> tasks,
                       java.lang.Runnable selfTask,
                       java.util.concurrent.ExecutorService executor,
                       java.util.Collection<java.util.concurrent.Future<?>> futures)

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.lang.Iterable<? extends java.lang.Runnable> tasks,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.util.Iterator<? extends java.lang.Runnable> tasks,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.lang.Iterable<? extends java.lang.Runnable> tasks,
                                      long maxRetries,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.util.Iterator<? extends java.lang.Runnable> tasks,
                                      long maxRetries,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.lang.Iterable<? extends java.lang.Runnable> tasks,
                                      TerminationFlag terminationFlag,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        The provided terminationFlag is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        terminationFlag - a flag to check periodically if the execution should be terminated
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.util.Iterator<? extends java.lang.Runnable> tasks,
                                      TerminationFlag terminationFlag,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        The provided terminationFlag is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        terminationFlag - a flag to check periodically if the execution should be terminated
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.lang.Iterable<? extends java.lang.Runnable> tasks,
                                      long waitTime,
                                      java.util.concurrent.TimeUnit timeUnit,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        If the pool is full, wait for waitTime timeUnits and retry submitting the tasks indefinitely.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        waitTime - how long to wait between retries
        timeUnit - the unit for waitTime
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.util.Iterator<? extends java.lang.Runnable> tasks,
                                      long waitTime,
                                      java.util.concurrent.TimeUnit timeUnit,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        If the pool is full, wait for waitTime timeUnits and retry submitting the tasks indefinitely.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        waitTime - how long to wait between retries
        timeUnit - the unit for waitTime
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.lang.Iterable<? extends java.lang.Runnable> tasks,
                                      long waitTime,
                                      java.util.concurrent.TimeUnit timeUnit,
                                      TerminationFlag terminationFlag,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        If the pool is full, wait for waitTime timeUnits and retry submitting the tasks indefinitely.

        The provided terminationFlag is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        waitTime - how long to wait between retries
        timeUnit - the unit for waitTime
        terminationFlag - a flag to check periodically if the execution should be terminated
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.util.Iterator<? extends java.lang.Runnable> tasks,
                                      long waitTime,
                                      java.util.concurrent.TimeUnit timeUnit,
                                      TerminationFlag terminationFlag,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        If the pool is full, wait for waitTime timeUnits and retry submitting the tasks indefinitely.

        The provided terminationFlag is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        waitTime - how long to wait between retries
        timeUnit - the unit for waitTime
        terminationFlag - a flag to check periodically if the execution should be terminated
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.lang.Iterable<? extends java.lang.Runnable> tasks,
                                      long maxRetries,
                                      long waitTime,
                                      java.util.concurrent.TimeUnit timeUnit,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        If the pool is full, wait for waitTime timeUnits and retry submitting the tasks at most maxRetries times.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        maxRetries - how many retries when submitting on a full pool before giving up
        waitTime - how long to wait between retries
        timeUnit - the unit for waitTime
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.util.Iterator<? extends java.lang.Runnable> tasks,
                                      long maxRetries,
                                      long waitTime,
                                      java.util.concurrent.TimeUnit timeUnit,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        If the pool is full, wait for waitTime timeUnits and retry submitting the tasks at most maxRetries times.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        maxRetries - how many retries when submitting on a full pool before giving up
        waitTime - how long to wait between retries
        timeUnit - the unit for waitTime
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.lang.Iterable<? extends java.lang.Runnable> tasks,
                                      long maxRetries,
                                      long waitTime,
                                      java.util.concurrent.TimeUnit timeUnit,
                                      TerminationFlag terminationFlag,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterable creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        If the pool is full, wait for waitTime timeUnits and retry submitting the tasks at most maxRetries times.

        The provided terminationFlag is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        maxRetries - how many retries when submitting on a full pool before giving up
        waitTime - how long to wait between retries
        timeUnit - the unit for waitTime
        terminationFlag - a flag to check periodically if the execution should be terminated
        executor - the executor to submit the tasks to

      • runWithConcurrency

        public static void runWithConcurrency​(int concurrency,
                                      java.util.Iterator<? extends java.lang.Runnable> tasks,
                                      long maxRetries,
                                      long waitTime,
                                      java.util.concurrent.TimeUnit timeUnit,
                                      TerminationFlag terminationFlag,
                                      java.util.concurrent.ExecutorService executor)
        Try to run all tasks for their side-effects using at most concurrency threads at once.

        If the concurrency is 1 or less, or there is only a single task, or the provided ExecutorService has terminated the tasks are run sequentially on the calling thread until all tasks are finished or the first Exception is thrown.

        If the tasks are submitted to the executor, it may happen that not all tasks are actually executed. If the provided iterator creates the tasks lazily upon iteration, not all elements might actually be created.

        The calling thread will be always blocked during the execution of the tasks and is not available for scheduling purposes. If the calling thread is interrupted during the execution, running tasks are cancelled and not-yet-started tasks are abandoned.

        We will try to submit tasks as long as no more than concurrency are already started and then continue to submit tasks one-by-one, after a previous tasks has finished, so that no more than concurrency tasks are running in the provided executor.

        We do not submit all tasks at-once into the worker queue to avoid creating a large amount of tasks and further support lazily created tasks. We can support thousands, even millions of tasks without resource exhaustion that way.

        We will try to submit tasks as long as the executor can directly start new tasks, that is, we want to avoid creating tasks and put them into the waiting queue if it may never be executed afterwards.

        If the pool is full, wait for waitTime timeUnits and retry submitting the tasks at most maxRetries times.

        The provided terminationFlag is checked before submitting new tasks and if it signals termination, running tasks are cancelled and not-yet-started tasks are abandoned.

        Parameters:
        concurrency - how many tasks should be run simultaneous
        tasks - the tasks to execute
        maxRetries - how many retries when submitting on a full pool before giving up
        waitTime - how long to wait between retries
        timeUnit - the unit for waitTime
        terminationFlag - a flag to check periodically if the execution should be terminated
        executor - the executor to submit the tasks to

      • awaitTermination

        public static void awaitTermination​(java.util.Collection<java.util.concurrent.Future<?>> futures)

      • awaitTerminations

        public static void awaitTerminations​(java.util.Queue<java.util.concurrent.Future<?>> futures)