ZPipeline

zio.stream.ZPipeline
See theZPipeline companion object
final class ZPipeline[-Env, +Err, -In, +Out]

A ZPipeline[Env, Err, In, Out] is a polymorphic stream transformer. Pipelines accept a stream as input, and return the transformed stream as output.

Pipelines can be thought of as a recipe for calling a bunch of methods on a source stream, to yield a new (transformed) stream. A nice mental model is the following type alias:

type ZPipeline[Env, Err, In, Out] = ZStream[Env, Err, In] => ZStream[Env, Err, Out]

This encoding of a pipeline with a type alias is not used because it does not infer well. In its place, this trait captures the polymorphism inherent to many pipelines, which can therefore be more flexible about the environment and error types of the streams they transform.

There is no fundamental requirement for pipelines to exist, because everything pipelines do can be done directly on a stream. However, because pipelines separate the stream transformation from the source stream itself, it becomes possible to abstract over stream transformations at the level of values, creating, storing, and passing around reusable transformation pipelines that can be applied to many different streams.

The most common way to create a pipeline is to convert a sink into a pipeline (in general, transforming elements of a stream requires the power of a sink). However, the companion object has lots of other pipeline constructors based on the methods of stream.

Attributes

Companion:
object
Graph
Supertypes
class Object
trait Matchable
class Any
Self type
ZPipeline[Env, Err, In, Out]

Members list

Concise view

Value members

Concrete methods

def <<<[Env1 <: Env, Err1 >: Err, In2](that: => ZPipeline[Env1, Err1, In2, In])(implicit trace: Trace): ZPipeline[Env1, Err1, In2, Out]

Composes two pipelines into one pipeline, by first applying the transformation of the specified pipeline, and then applying the transformation of this pipeline.

Composes two pipelines into one pipeline, by first applying the transformation of the specified pipeline, and then applying the transformation of this pipeline.

Attributes

def >>>[Env1 <: Env, Err1 >: Err, Out2](that: => ZPipeline[Env1, Err1, Out, Out2])(implicit trace: Trace): ZPipeline[Env1, Err1, In, Out2]

Composes two pipelines into one pipeline, by first applying the transformation of this pipeline, and then applying the transformation of the specified pipeline.

Composes two pipelines into one pipeline, by first applying the transformation of this pipeline, and then applying the transformation of the specified pipeline.

Attributes

def >>>[Env1 <: Env, Err1 >: Err, Leftover, Out2](that: => ZSink[Env1, Err1, Out, Leftover, Out2])(implicit trace: Trace): ZSink[Env1, Err1, In, Leftover, Out2]

Compose this transducer with a sink, resulting in a sink that processes elements by piping them through this transducer and piping the results into the sink.

Compose this transducer with a sink, resulting in a sink that processes elements by piping them through this transducer and piping the results into the sink.

Attributes

def aggregateAsync[Env1 <: Env, Err1 >: Err, Out1 >: Out, Out2](sink: ZSink[Env1, Err1, Out1, Out1, Out2])(implicit trace: Trace): ZPipeline[Env1, Err1, In, Out2]

Aggregates elements of this stream using the provided sink for as long as the downstream operators on the stream are busy.

Aggregates elements of this stream using the provided sink for as long as the downstream operators on the stream are busy.

This operator divides the stream into two asynchronous "islands". Operators upstream of this operator run on one fiber, while downstream operators run on another. Whenever the downstream fiber is busy processing elements, the upstream fiber will feed elements into the sink until it signals completion.

Any sink can be used here, but see ZSink.foldWeightedZIO and ZSink.foldUntilZIO for sinks that cover the common usecases.

Attributes

def aggregateAsyncWithin[Env1 <: Env, Err1 >: Err, Out1 >: Out, Out2](sink: ZSink[Env1, Err1, Out1, Out1, Out2], schedule: Schedule[Env1, Option[Out2], Any])(implicit trace: Trace): ZPipeline[Env1, Err1, In, Out2]

Like aggregateAsyncWithinEither, but only returns the Right results.

Like aggregateAsyncWithinEither, but only returns the Right results.

Attributes

def aggregateAsyncWithinEither[Env1 <: Env, Err1 >: Err, Out1 >: Out, Out2, Out3](sink: ZSink[Env1, Err1, Out1, Out1, Out2], schedule: Schedule[Env1, Option[Out2], Out3])(implicit trace: Trace): ZPipeline[Env1, Err1, In, Either[Out3, Out2]]

Aggregates elements using the provided sink until it completes, or until the delay signalled by the schedule has passed.

Aggregates elements using the provided sink until it completes, or until the delay signalled by the schedule has passed.

This operator divides the stream into two asynchronous islands. Operators upstream of this operator run on one fiber, while downstream operators run on another. Elements will be aggregated by the sink until the downstream fiber pulls the aggregated value, or until the schedule's delay has passed.

Aggregated elements will be fed into the schedule to determine the delays between pulls.

Attributes

def andThen[Env1 <: Env, Err1 >: Err, Out2](that: => ZPipeline[Env1, Err1, Out, Out2])(implicit trace: Trace): ZPipeline[Env1, Err1, In, Out2]

A named version of the >>> operator.

A named version of the >>> operator.

Attributes

def apply[Env1 <: Env, Err1 >: Err](stream: => ZStream[Env1, Err1, In])(implicit trace: Trace): ZStream[Env1, Err1, Out]

Attach this pipeline to the given stream

Attach this pipeline to the given stream

Attributes

def changes(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Returns a new pipeline that only emits elements that are not equal to the previous element emitted, using natural equality to determine whether two elements are equal.

Returns a new pipeline that only emits elements that are not equal to the previous element emitted, using natural equality to determine whether two elements are equal.

Attributes

def changesWith(f: (Out, Out) => Boolean)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Returns a new pipeline that only emits elements that are not equal to the previous element emitted, using the specified function to determine whether two elements are equal.

Returns a new pipeline that only emits elements that are not equal to the previous element emitted, using the specified function to determine whether two elements are equal.

Attributes

def changesWithZIO(f: (Out, Out) => UIO[Boolean])(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Returns a new pipeline that only emits elements that are not equal to the previous element emitted, using the specified effectual function to determine whether two elements are equal.

Returns a new pipeline that only emits elements that are not equal to the previous element emitted, using the specified effectual function to determine whether two elements are equal.

Attributes

def chunks(implicit trace: Trace): ZPipeline[Env, Err, In, Chunk[Out]]

Exposes the underlying chunks of the stream as a stream of chunks of elements.

Exposes the underlying chunks of the stream as a stream of chunks of elements.

Attributes

def collect[Out2](pf: PartialFunction[Out, Out2])(implicit trace: Trace): ZPipeline[Env, Err, In, Out2]

Performs a filter and map in a single step.

Performs a filter and map in a single step.

Attributes

def collectLeft[A, B](implicit ev: Out <:< Either[A, B], trace: Trace): ZPipeline[Env, Err, In, A]

Filters any Right values.

Filters any Right values.

Attributes

def collectRight[A, B](implicit ev: Out <:< Either[A, B], trace: Trace): ZPipeline[Env, Err, In, B]

Filters any Left values.

Filters any Left values.

Attributes

def collectSome[Out2](implicit ev: Out <:< Option[Out2], trace: Trace): ZPipeline[Env, Err, In, Out2]

Filters any 'None' values.

Filters any 'None' values.

Attributes

def collectSuccess[Out2, L1](implicit ev: Out <:< Exit[L1, Out2], trace: Trace): ZPipeline[Env, Err, In, Out2]

Filters any Exit.Failure values.

Filters any Exit.Failure values.

Attributes

def collectWhile[Out2](pf: PartialFunction[Out, Out2])(implicit trace: Trace): ZPipeline[Env, Err, In, Out2]

Transforms all elements of the pipeline for as long as the specified partial function is defined.

Transforms all elements of the pipeline for as long as the specified partial function is defined.

Attributes

def collectWhileLeft[A, B](implicit ev: Out <:< Either[A, B], trace: Trace): ZPipeline[Env, Err, In, A]

Terminates the pipeline when encountering the first Right.

Terminates the pipeline when encountering the first Right.

Attributes

def collectWhileRight[A, B](implicit ev: Out <:< Either[A, B], trace: Trace): ZPipeline[Env, Err, In, B]

Terminates the pipeline when encountering the first Left.

Terminates the pipeline when encountering the first Left.

Attributes

def collectWhileSome[Out2](implicit ev: Out <:< Option[Out2], trace: Trace): ZPipeline[Env, Err, In, Out2]

Terminates the pipeline when encountering the first None.

Terminates the pipeline when encountering the first None.

Attributes

def collectWhileSuccess[Err2, Out2](implicit ev: Out <:< Exit[Err2, Out2], trace: Trace): ZPipeline[Env, Err, In, Out2]

Terminates the pipeline when encountering the first Exit.Failure.

Terminates the pipeline when encountering the first Exit.Failure.

Attributes

def collectWhileZIO[Env2 <: Env, Err2 >: Err, Out2](pf: PartialFunction[Out, ZIO[Env2, Err2, Out2]])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out2]

Effectfully transforms all elements of the pipeline for as long as the specified partial function is defined.

Effectfully transforms all elements of the pipeline for as long as the specified partial function is defined.

Attributes

def compose[Env1 <: Env, Err1 >: Err, In2](that: => ZPipeline[Env1, Err1, In2, In])(implicit trace: Trace): ZPipeline[Env1, Err1, In2, Out]

A named version of the <<< operator.

A named version of the <<< operator.

Attributes

def drain(implicit trace: Trace): ZPipeline[Env, Err, In, Nothing]

Converts this pipeline to a pipeline that executes its effects but emits no elements. Useful for sequencing effects using pipeline:

Converts this pipeline to a pipeline that executes its effects but emits no elements. Useful for sequencing effects using pipeline:

(Stream(1, 2, 3).tap(i => ZIO(println(i))) ++
 (Stream.fromZIO(ZIO(println("Done!"))) >>> ZPipeline.drain) ++
 Stream(4, 5, 6).tap(i => ZIO(println(i)))).run(Sink.drain)

Attributes

def drop(n: => Int)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Drops the specified number of elements from this stream.

Drops the specified number of elements from this stream.

Attributes

def dropRight(n: => Int)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Drops the last specified number of elements from this pipeline.

Drops the last specified number of elements from this pipeline.

Attributes

Note:

This combinator keeps n elements in memory. Be careful with big numbers.

def dropUntil(f: Out => Boolean)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Drops all elements of the pipeline until the specified predicate evaluates to true.

Drops all elements of the pipeline until the specified predicate evaluates to true.

Attributes

def dropUntilZIO[Env1 <: Env, Err1 >: Err](f: Out => ZIO[Env1, Err1, Boolean])(implicit trace: Trace): ZPipeline[Env1, Err1, In, Out]

Drops incoming elements until the effectful predicate p is satisfied.

Drops incoming elements until the effectful predicate p is satisfied.

Attributes

def dropWhile(f: Out => Boolean)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Drops all elements of the pipeline for as long as the specified predicate evaluates to true.

Drops all elements of the pipeline for as long as the specified predicate evaluates to true.

Attributes

def dropWhileZIO[Env1 <: Env, Err1 >: Err](f: Out => ZIO[Env1, Err1, Boolean])(implicit trace: Trace): ZPipeline[Env1, Err1, In, Out]

Drops incoming elements as long as the effectful predicate p is satisfied.

Drops incoming elements as long as the effectful predicate p is satisfied.

Attributes

def filter(f: Out => Boolean)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Filters the elements emitted by this pipeline using the provided function.

Filters the elements emitted by this pipeline using the provided function.

Attributes

def filterZIO[Env2 <: Env, Err2 >: Err](f: Out => ZIO[Env2, Err2, Boolean])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out]

Effectfully filters the elements emitted by this pipeline.

Effectfully filters the elements emitted by this pipeline.

Attributes

def flattenChunks[Out2](implicit ev: Out <:< Chunk[Out2], trace: Trace): ZPipeline[Env, Err, In, Out2]

Submerges the chunks carried by this pipeline into the pipeline's structure, while still preserving them.

Submerges the chunks carried by this pipeline into the pipeline's structure, while still preserving them.

Attributes

def flattenExit[Err2, Out2](implicit ev: Out <:< Exit[Err2, Out2], trace: Trace): ZPipeline[Env, Err2, In, Out2]

Flattens Exit values. Exit.Failure values translate to pipeline failures while Exit.Success values translate to stream elements.

Flattens Exit values. Exit.Failure values translate to pipeline failures while Exit.Success values translate to stream elements.

Attributes

def flattenIterables[Out2](implicit ev: Out <:< Iterable[Out2], trace: Trace): ZPipeline[Env, Err, In, Out2]

Submerges the iterables carried by this pipeline into the pipeline's structure, while still preserving them.

Submerges the iterables carried by this pipeline into the pipeline's structure, while still preserving them.

Attributes

def flattenTake[Err1 >: Err, Out2](implicit ev: Out <:< Take[Err1, Out2], trace: Trace): ZPipeline[Env, Err1, In, Out2]

Flattens take values.

Flattens take values.

Attributes

def grouped(chunkSize: => Int)(implicit trace: Trace): ZPipeline[Env, Err, In, Chunk[Out]]

Partitions the pipeline with specified chunkSize

Partitions the pipeline with specified chunkSize

Attributes

chunkSize

size of the chunk

def groupedWithin(chunkSize: => Int, within: => Duration)(implicit trace: Trace): ZPipeline[Env, Err, In, Chunk[Out]]

Partitions the stream with the specified chunkSize or until the specified duration has passed, whichever is satisfied first.

Partitions the stream with the specified chunkSize or until the specified duration has passed, whichever is satisfied first.

Attributes

def intersperse[Out2 >: Out](middle: => Out2)(implicit trace: Trace): ZPipeline[Env, Err, In, Out2]

Intersperse pipeline with provided element similar to List.mkString.

Intersperse pipeline with provided element similar to List.mkString.

Attributes

def intersperse[Out2 >: Out](start: => Out2, middle: => Out2, end: => Out2)(implicit trace: Trace): ZPipeline[Env, Err, In, Out2]

Intersperse and also add a prefix and a suffix

Intersperse and also add a prefix and a suffix

Attributes

def map[Out2](f: Out => Out2)(implicit trace: Trace): ZPipeline[Env, Err, In, Out2]

Transforms the elements of this pipeline using the supplied function.

Transforms the elements of this pipeline using the supplied function.

Attributes

def mapAccum[State, Out2](s: => State)(f: (State, Out) => (State, Out2))(implicit trace: Trace): ZPipeline[Env, Err, In, Out2]

Statefully maps over the elements of this pipeline to produce new elements.

Statefully maps over the elements of this pipeline to produce new elements.

Attributes

def mapAccumZIO[Env2 <: Env, Err2 >: Err, State, Out2](s: => State)(f: (State, Out) => ZIO[Env2, Err2, (State, Out2)])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out2]

Statefully and effectfully maps over the elements of this pipeline to produce new elements.

Statefully and effectfully maps over the elements of this pipeline to produce new elements.

Attributes

def mapChunks[Out2](f: Chunk[Out] => Chunk[Out2])(implicit trace: Trace): ZPipeline[Env, Err, In, Out2]

Transforms the chunks emitted by this stream.

Transforms the chunks emitted by this stream.

Attributes

def mapChunksZIO[Env2 <: Env, Err2 >: Err, Out2](f: Chunk[Out] => ZIO[Env2, Err2, Chunk[Out2]])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out2]

Creates a pipeline that maps chunks of elements with the specified effect.

Creates a pipeline that maps chunks of elements with the specified effect.

Attributes

def mapError[Err2](f: Err => Err2)(implicit trace: Trace): ZPipeline[Env, Err2, In, Out]

Transforms the errors emitted by this pipeline using f.

Transforms the errors emitted by this pipeline using f.

Attributes

def mapErrorCause[Err2](f: Cause[Err] => Cause[Err2])(implicit trace: Trace): ZPipeline[Env, Err2, In, Out]

A more powerful version of mapError which also surfaces the Cause of the channel failure

A more powerful version of mapError which also surfaces the Cause of the channel failure

Attributes

def mapStream[Env2 <: Env, Err2 >: Err, Out2](f: Out => ZStream[Env2, Err2, Out2])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out2]

Creates a pipeline that maps elements with the specified function that returns a stream.

Creates a pipeline that maps elements with the specified function that returns a stream.

Attributes

def mapZIO[Env2 <: Env, Err2 >: Err, Out2](f: Out => ZIO[Env2, Err2, Out2])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out2]

Creates a pipeline that maps elements with the specified effectful function.

Creates a pipeline that maps elements with the specified effectful function.

Attributes

def mapZIOPar[Env2 <: Env, Err2 >: Err, Out2](n: => Int)(f: Out => ZIO[Env2, Err2, Out2])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out2]

Maps over elements of the stream with the specified effectful function, executing up to n invocations of f concurrently. Transformed elements will be emitted in the original order.

Maps over elements of the stream with the specified effectful function, executing up to n invocations of f concurrently. Transformed elements will be emitted in the original order.

Attributes

Note:

This combinator destroys the chunking structure. It's recommended to use rechunk afterwards.

def mapZIOParUnordered[Env2 <: Env, Err2 >: Err, Out2](n: => Int)(f: Out => ZIO[Env2, Err2, Out2])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out2]

Maps over elements of the stream with the specified effectful function, executing up to n invocations of f concurrently. The element order is not enforced by this combinator, and elements may be reordered.

Maps over elements of the stream with the specified effectful function, executing up to n invocations of f concurrently. The element order is not enforced by this combinator, and elements may be reordered.

Attributes

def orDie(implicit ev: Err <:< Throwable, trace: Trace): ZPipeline[Env, Nothing, In, Out]

Translates pipeline failure into death of the fiber, making all failures unchecked and not a part of the type of the effect.

Translates pipeline failure into death of the fiber, making all failures unchecked and not a part of the type of the effect.

Attributes

def orDieWith(f: Err => Throwable)(implicit trace: Trace): ZPipeline[Env, Nothing, In, Out]

Keeps none of the errors, and terminates the fiber with them, using the specified function to convert the E into a Throwable.

Keeps none of the errors, and terminates the fiber with them, using the specified function to convert the E into a Throwable.

Attributes

def take(n: => Long)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Takes the specified number of elements from this pipeline.

Takes the specified number of elements from this pipeline.

Attributes

def takeUntil(f: Out => Boolean)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Takes all elements of the pipeline until the specified predicate evaluates to true.

Takes all elements of the pipeline until the specified predicate evaluates to true.

Attributes

def takeWhile(f: Out => Boolean)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Takes all elements of the pipeline for as long as the specified predicate evaluates to true.

Takes all elements of the pipeline for as long as the specified predicate evaluates to true.

Attributes

def tap[Env2 <: Env, Err2 >: Err](f: Out => ZIO[Env2, Err2, Any])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out]

Adds an effect to consumption of every element of the pipeline.

Adds an effect to consumption of every element of the pipeline.

Attributes

def throttleEnforce(units: Long, duration: => Duration, burst: => Long)(costFn: Chunk[Out] => Long)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Throttles the chunks of this pipeline according to the given bandwidth parameters using the token bucket algorithm. Allows for burst in the processing of elements by allowing the token bucket to accumulate tokens up to a units + burst threshold. Chunks that do not meet the bandwidth constraints are dropped. The weight of each chunk is determined by the costFn function.

Throttles the chunks of this pipeline according to the given bandwidth parameters using the token bucket algorithm. Allows for burst in the processing of elements by allowing the token bucket to accumulate tokens up to a units + burst threshold. Chunks that do not meet the bandwidth constraints are dropped. The weight of each chunk is determined by the costFn function.

Attributes

def throttleEnforceZIO[Env2 <: Env, Err2 >: Err](units: => Long, duration: => Duration, burst: => Long)(costFn: Chunk[Out] => ZIO[Env2, Err2, Long])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out]

Throttles the chunks of this pipeline according to the given bandwidth parameters using the token bucket algorithm. Allows for burst in the processing of elements by allowing the token bucket to accumulate tokens up to a units + burst threshold. Chunks that do not meet the bandwidth constraints are dropped. The weight of each chunk is determined by the costFn effectful function.

Throttles the chunks of this pipeline according to the given bandwidth parameters using the token bucket algorithm. Allows for burst in the processing of elements by allowing the token bucket to accumulate tokens up to a units + burst threshold. Chunks that do not meet the bandwidth constraints are dropped. The weight of each chunk is determined by the costFn effectful function.

Attributes

def throttleShape(units: => Long, duration: => Duration, burst: Long)(costFn: Chunk[Out] => Long)(implicit trace: Trace): ZPipeline[Env, Err, In, Out]

Delays the chunks of this pipeline according to the given bandwidth parameters using the token bucket algorithm. Allows for burst in the processing of elements by allowing the token bucket to accumulate tokens up to a units + burst threshold. The weight of each chunk is determined by the costFn function.

Delays the chunks of this pipeline according to the given bandwidth parameters using the token bucket algorithm. Allows for burst in the processing of elements by allowing the token bucket to accumulate tokens up to a units + burst threshold. The weight of each chunk is determined by the costFn function.

Attributes

def throttleShapeZIO[Env2 <: Env, Err2 >: Err](units: => Long, duration: => Duration, burst: => Long)(costFn: Chunk[Out] => ZIO[Env2, Err2, Long])(implicit trace: Trace): ZPipeline[Env2, Err2, In, Out]

Delays the chunks of this pipeline according to the given bandwidth parameters using the token bucket algorithm. Allows for burst in the processing of elements by allowing the token bucket to accumulate tokens up to a units + burst threshold. The weight of each chunk is determined by the costFn effectful function.

Delays the chunks of this pipeline according to the given bandwidth parameters using the token bucket algorithm. Allows for burst in the processing of elements by allowing the token bucket to accumulate tokens up to a units + burst threshold. The weight of each chunk is determined by the costFn effectful function.

Attributes

def toChannel: ZChannel[Env, ZNothing, Chunk[In], Any, Err, Chunk[Out], Any]

Converts this pipeline to its underlying channel

Converts this pipeline to its underlying channel

Attributes

def zipWithIndex(implicit trace: Trace): ZPipeline[Env, Err, In, (Out, Long)]

Zips this pipeline together with the index of elements.

Zips this pipeline together with the index of elements.

Attributes

def zipWithNext(implicit trace: Trace): ZPipeline[Env, Err, In, (Out, Option[Out])]
def zipWithPrevious(implicit trace: Trace): ZPipeline[Env, Err, In, (Option[Out], Out)]
def zipWithPreviousAndNext(implicit trace: Trace): ZPipeline[Env, Err, In, (Option[Out], Out, Option[Out])]

Concrete fields

val channel: ZChannel[Env, ZNothing, Chunk[In], Any, Err, Chunk[Out], Any]