A "parallel iterator" combining Scala’s parallel collections with an
underlying grouped iterator, processing the contents of each chunk in
parallel, and exposing the interface of a simple ungrouped iterator.
The goal is to read a chunk from the underlying grouped iterator,
and then to process each element inside the chunk in parallel via #map
and #flatMap, using the standard Scala parallel collections to speed up
the processing the passed function over the iterator.
ParIterator is typically constructed via the method Iterator#par() which
is added via the implicits in ParIterator.Implicits, for example:
scala> import com.timgroup.iterata.ParIterator.Implicits._
scala> val it = (1 to 100000).iterator.par().map(n => (n + 1, Thread.currentThread.getId))
scala> it.map(_._1).toSet.size
res2: Int = 8// addition was distributed over 8 threads
A
the type of each element
Linear Supertypes
AbstractIterator[A], Iterator[A], TraversableOnce[A], GenTraversableOnce[A], AnyRef, Any
A "parallel iterator" combining Scala’s parallel collections with an underlying grouped iterator, processing the contents of each chunk in parallel, and exposing the interface of a simple ungrouped iterator.
See: http://docs.scala-lang.org/overviews/parallel-collections/overview.html
The goal is to read a chunk from the underlying grouped iterator, and then to process each element inside the chunk in parallel via
#map
and#flatMap
, using the standard Scala parallel collections to speed up the processing the passed function over the iterator.ParIterator is typically constructed via the method
Iterator#par()
which is added via the implicits inParIterator.Implicits
, for example:the type of each element