object Broadcast
Provides mechanisms for broadcast distribution of elements to multiple streams.
- Source
- Broadcast.scala
- Alphabetic
- By Inheritance
- Broadcast
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
apply[F[_], O](minReady: Int)(implicit arg0: Concurrent[F]): Pipe[F, O, Stream[F, O]]
Allows elements of a stream to be broadcast to multiple workers.
Allows elements of a stream to be broadcast to multiple workers.
As the elements arrive, they are broadcast to all
workers
that have started evaluation before the element was pulled.Elements are pulled as chunks from the source and the next chunk is pulled when all workers are done with processing the current chunk. This behaviour may slow down processing of incoming chunks by faster workers. If this is not desired, consider using the
prefetch
andprefetchN
combinators on workers to compensate for slower workers.Often this combinator is used together with parJoin, such as :
Stream(1,2,3,4).broadcast.map { worker => worker.evalMap { o => IO.println("1:" + o.toString) } }.take(3).parJoinUnbounded.compile.drain.unsafeRunSync
Note that in the example, above the workers are not guaranteed to see all elements emitted. This is due to different subscription times of each worker and speed of emitting the elements by the source. If this is not desired, consider using
broacastThrough
andbroadcastTo
, which are built on top ofBroadcast.through
, as an alternative. They will hold on pulling from source if there are no workers ready.When
source
terminates, then the inner streams (workers) are terminated once all elements pulled fromsource
are processed by all workers. However, note that when thatsource
terminates, resulting stream will not terminate until the inner streams terminate.- minReady
specifies that broadcasting will hold off until at least
minReady
subscribers will be ready
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @HotSpotIntrinsicCandidate()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
through[F[_], O, O2](pipes: Pipe[F, O, O2]*)(implicit arg0: Concurrent[F]): Pipe[F, O, O2]
Like apply but instead of providing a stream of worker streams, it runs each inner stream through the supplied pipes.
Like apply but instead of providing a stream of worker streams, it runs each inner stream through the supplied pipes.
Supplied pipes are run concurrently with each other. Hence, the number of pipes determines concurrency. Also, this guarantees that each pipe will view all
O
pulled from source stream, unlikebroadcast
.Resulting values are collected and returned in a single stream of
O2
values.- pipes
pipes that will concurrently process the work
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
Deprecated Value Members
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] ) @Deprecated @deprecated
- Deprecated
(Since version ) see corresponding Javadoc for more information.