fs2
package fs2
Type members
Classlikes
Strict, finite sequence of values that allows index-based random access of elements.
Chunk
s can be created from a variety of collection types using methods on the Chunk
companion(e.g.,
Chunk.array
, Chunk.seq
, Chunk.vector
).Chunks can be appended via the
chunks -- that is, there's no copying of the source chunks. For example,
returns a
an appended chunk is
has size 1, indexed lookup is
chunk elements to a single array backed chunk. Note
++
method. The returned chunk is a composite of the inputchunks -- that is, there's no copying of the source chunks. For example,
Chunk(1, 2) ++ Chunk(3, 4) ++ Chunk(5, 6)
returns a
Chunk.Queue(Chunk(1, 2), Chunk(3, 4), Chunk(5, 6))
. As a result, indexed based lookup ofan appended chunk is
O(number of underlying chunks)
. In the worse case, where each constituent chunkhas size 1, indexed lookup is
O(size)
. To restore O(1)
lookup, call compact
, which copies all the underlyingchunk elements to a single array backed chunk. Note
compact
requires a ClassTag
of the element type.Alternatively, a collection of chunks can be directly copied to a new array backed chunk via
Chunk.concat(chunks)
. Like compact
, Chunk.concat
requires a ClassTag
for the element type.Various subtypes of
-
-
-
Chunk
are exposed for efficiency reasons:-
Chunk.Singleton
-
Chunk.ArraySlice
-
Chunk.Queue
In particular, calling
access to the underlying backing array, along with an offset and length, referring to a slice
of that array.
.toArraySlice
on a chunk returns a Chunk.ArraySlice
, which providesaccess to the underlying backing array, along with an offset and length, referring to a slice
of that array.
- Companion
- object
Supports building a result of type
Out
from zero or more Chunk[A]
.This is similar to the standard library collection builders but optimized for
building a collection from a stream.
building a collection from a stream.
The companion object provides implicit conversions (methods starting with
which adapts various collections to the
supports
),which adapts various collections to the
Collector
trait.- Companion
- object
@implicitNotFound("Cannot find an implicit Compiler[F, G]. This typically means you need a Concurrent[F] in scope")
Provides compilation of a
Stream[F, O]
to a G[*]
.In the most common case,
are other common instantiations like
F = G = IO
or another "fully featured" effect type. However, thereare other common instantiations like
F = Pure, G = Id
, which allows compiling aStream[Pure, A]
in to pure values.For the common case where
super charged
-
- both
- only
Support for stream interruption requires compilation to an effect which has a
F = G
, the target
implicit constructor provides an instance ofCompiler[F, F]
-- target
requires a Compiler.Target[F]
instance. The Compiler.Target[F]
is asuper charged
MonadErrorThrow[F]
, providing additional capabilities needed for stream compilation.Compiler.Target[F]
instances are given for all F[_]
which have:-
Concurrent[F]
instances- both
MonadCancelThrow[F]
and Sync[F]
intances- only
Sync[F]
instancesSupport for stream interruption requires compilation to an effect which has a
Concurrent
instance.- Companion
- object
A purely functional data structure that describes a process. This process
may evaluate actions in an effect type F, emit any number of output values
of type O (or None), and may a) terminate with a single result of type R;
or b) terminate abnormally by raising (inside the effect
or c) terminate because it was cancelled by another process,
or d) not terminate.
may evaluate actions in an effect type F, emit any number of output values
of type O (or None), and may a) terminate with a single result of type R;
or b) terminate abnormally by raising (inside the effect
F
) an exception,or c) terminate because it was cancelled by another process,
or d) not terminate.
Like types from other effect libraries, pulls are pure and immutable values.
They preserve referential transparency.
They preserve referential transparency.
=== Chunking ===
The output values of a pull are emitted not one by one, but in chunks.
A
a pull
values, which can wrap unboxed byte arrays -- avoiding boxing/unboxing costs.
The
the individual element level. Generally, working at the chunk level will
result in better performance but at the cost of more complex implementations
A
Chunk
is an immutable sequence with constant-time indexed lookup. For example,a pull
p: Pull[F, Byte, R]
internally operates and emits Chunk[Byte]
values, which can wrap unboxed byte arrays -- avoiding boxing/unboxing costs.
The
Pull
API provides mechanisms for working at both the chunk level andthe individual element level. Generally, working at the chunk level will
result in better performance but at the cost of more complex implementations
A pull only emits non-empty chunks.
However, chunks are not merely an operational matter of efficiency. Each
pull is emitted from a chunk atomically, which is to say, any errors or
interruptions in a pull can only happen between chunks, not within a
chunk. For instance, if creating a new chunk of values fails (raises an
uncaught exception) while creating an intermediate value, then it fails
to create the entire chunk and previous values are discarded.
pull is emitted from a chunk atomically, which is to say, any errors or
interruptions in a pull can only happen between chunks, not within a
chunk. For instance, if creating a new chunk of values fails (raises an
uncaught exception) while creating an intermediate value, then it fails
to create the entire chunk and previous values are discarded.
=== Evaluation ===
Like other functional effect types (e.g.
describes a process or computation. It is not a running process nor a
handle for the result of a spawned, running process, like
cats.effect.IO
), a pulldescribes a process or computation. It is not a running process nor a
handle for the result of a spawned, running process, like
scala.concurrent.Future
.A pull can be converted to a stream and then compiled to an effectful value.
For a
monad instance of
The result of that
the pull, in the order it emits them, using a fold function. Depending on that
function, outputs may be collected into a list (or vector or array or ...),
combined together into a single value, or just discarded altogether (by draining
the pull).
For a
Pull[F, O, Unit]
, the result of compilation is a combination, via themonad instance of
F
, of all the actions in the effect F
present in the pull.The result of that
F
action is the result of combining the outputs emitted bythe pull, in the order it emits them, using a fold function. Depending on that
function, outputs may be collected into a list (or vector or array or ...),
combined together into a single value, or just discarded altogether (by draining
the pull).
Compilation is pull-based, rather than push-based (hence the name of the datatype).
It is the compilation process itself, that determines when the evaluation
of each single effect can proceed or is held back. Effects and outputs later
in the pull are not performed or emitted, unless and until the compilation
calls for them.
It is the compilation process itself, that determines when the evaluation
of each single effect can proceed or is held back. Effects and outputs later
in the pull are not performed or emitted, unless and until the compilation
calls for them.
=== Resource scoping ===
The effects in a
the execution of the pull, may be shared by several pulls, and must be
properly finalised when no longer needed, regardless of whether the pull completed
successfully or not. A pull tracks its resources using '''scopes''', which register
how many pulls are actively using each resource, and finalises resources when no
longer used.
Pull
may operate on resources, which must be retained duringthe execution of the pull, may be shared by several pulls, and must be
properly finalised when no longer needed, regardless of whether the pull completed
successfully or not. A pull tracks its resources using '''scopes''', which register
how many pulls are actively using each resource, and finalises resources when no
longer used.
Some operations of the
or resource boundaries.
Pull
API can be used to introduce new resource scopes,or resource boundaries.
=== Functional typeclasses ===
The
for
Pull
data structure is a "free" implementation of Monad
and has an instancefor
cats.effect.kernel.Sync
.For any types
F[_]
and O
, a Pull[F, O, *]
holds the following laws:-
pure >=> f == f
-
f >=> pure == f
-
(f >=> g) >=> h == f >=> (g >=> h)
wheref >=> g
is defined asa => a flatMap f flatMap g
-
handleErrorWith(raiseError(e))(f) == f(e)
- Type Params
- F[_]
-
the type of effect that can be performed by this pull.
An effect type ofNothing
, also known infs2
by the aliasPure
,
indicates that this pull perform no effectful actions.
Note:Nothing
is a polykinded type, so it can also be
applied as an argument to the type parameterF[_]
. - O
-
The outputs emitted by this Pull. An output type of
Nothing
means
that this pull does not emit any outputs. - R
-
The type of result returned by this Pull upon successful termination.
An output type ofNothing
indicates that this pull cannot terminate
successfully: it may fail, be cancelled, or never terminate.
- Companion
- object
@implicitNotFound("Cannot find an implicit value for RaiseThrowable[${F}]: an instance is available for any F which has an ApplicativeError[F, Throwable] instance or for F = Fallible. If getting this error for a non-specific F, try manually supplying the type parameter (e.g., Stream.raiseError[IO](t) instead of Stream.raiseError(t)). If getting this error when working with pure streams, use F = Fallible.")
Witnesses that
F
supports raising throwables.An instance of
is available for the uninhabited type
RaiseThrowable
is available for any F
which has anApplicativeError[F, Throwable]
instance. Alternatively, an instanceis available for the uninhabited type
Fallible
.- Companion
- object
A stream producing output of type
O
and which may evaluate F
effects.-
'''Purely functional''' a value of type
Stream[F, O]
describes an effectful computation.
A function that returns aStream[F, O]
builds a description of an effectful computation,
but does not perform them. The methods of theStream
class derive new descriptions from others.
This is similar to how effect types likecats.effect.IO
andmonix.Task
build descriptions of
computations. -
'''Pull''': to evaluate a stream, a consumer pulls its values from it, by repeatedly performing one pull step at a time.
Each step is aF
-effectful computation that may yield someO
values (or none), and a stream from which to continue pulling.
The consumer controls the evaluation of the stream, which effectful operations are performed, and when. -
'''Non-Strict''': stream evaluation only pulls from the stream a prefix large enough to compute its results.
Thus, although a stream may yield an unbounded number of values or, after successfully yielding several values,
either raise an error or hang up and never yield any value, the consumer need not reach those points of failure.
For the same reason, in general, no effect inF
is evaluated unless and until the consumer needs it. -
'''Abstract''': a stream needs not be a plain finite list of fixed effectful computations in F.
It can also represent an input or output connection through which data incrementally arrives.
It can represent an effectful computation, such as reading the system's time, that can be re-evaluated
as often as the consumer of the stream requires.
=== Special properties for streams ===
There are some special properties or cases of streams:
- A stream is '''finite''' if we can reach the end after a limited number of pull steps,
which may yield a finite number of values. It is '''empty''' if it terminates and yields no values.
- A '''singleton''' stream is a stream that ends after yielding one single value.
- A '''pure''' stream is one in which the
- A '''never''' stream is a stream that never terminates and never yields any value.
- A stream is '''finite''' if we can reach the end after a limited number of pull steps,
which may yield a finite number of values. It is '''empty''' if it terminates and yields no values.
- A '''singleton''' stream is a stream that ends after yielding one single value.
- A '''pure''' stream is one in which the
F
is Pure, which indicates that it evaluates no effects.- A '''never''' stream is a stream that never terminates and never yields any value.
== Pure Streams and operations ==
We can sometimes think of streams, naively, as lists of
This is particularly true for '''pure''' streams, which are instances of
We can convert every ''pure and finite'' stream into a
Also, we can convert pure ''infinite'' streams into instances of the
O
elements with F
-effects.This is particularly true for '''pure''' streams, which are instances of
Stream
which use the Pure effect type.We can convert every ''pure and finite'' stream into a
List[O]
using the .toList
method.Also, we can convert pure ''infinite'' streams into instances of the
Stream[O]
class from the Scala standard library.A method of the
in that their signature includes no type-class constraint (or implicit parameter) on the
Pure methods in
that applying the stream's method and converting the result to a list gets the same result as
first converting the stream to a list, and then applying list methods.
Stream
class is '''pure''' if it can be applied to pure streams. Such methods are identifiedin that their signature includes no type-class constraint (or implicit parameter) on the
F
method.Pure methods in
Stream[F, O]
can be projected ''naturally'' to methods in the List
class, which meansthat applying the stream's method and converting the result to a list gets the same result as
first converting the stream to a list, and then applying list methods.
Some methods that project directly to list are
There are other methods, like
but their stream counterparts return an (either empty or singleton) stream.
Other methods, like
map
, filter
, takeWhile
, etc.There are other methods, like
exists
or find
, that in the List
class they return a value or an Option
,but their stream counterparts return an (either empty or singleton) stream.
Other methods, like
zipWithPrevious
, have a more complicated but still pure translation to list methods.== Type-Class instances and laws of the Stream Operations ==
Laws (using infix syntax):
append
forms a monoid in conjunction with empty
:-
empty append s == s
and s append empty == s
.-
(s1 append s2) append s3 == s1 append (s2 append s3)
And
-
cons
is consistent with using ++
to prepend a single chunk:-
s.cons(c) == Stream.chunk(c) ++ s
Stream.raiseError
propagates until being caught by handleErrorWith
:-
Stream.raiseError(e) handleErrorWith h == h(e)
-
Stream.raiseError(e) ++ s == Stream.raiseError(e)
-
Stream.raiseError(e) flatMap f == Stream.raiseError(e)
Stream
forms a monad with emit
and flatMap
:-
Stream.emit >=> f == f
(left identity)-
f >=> Stream.emit === f
(right identity - note weaker equality notion here)-
(f >=> g) >=> h == f >=> (g >=> h)
(associativity)where
Stream.emit(a)
is defined as chunk(Chunk.singleton(a)) and
f >=> g is defined as
a => a flatMap f flatMap g`The monad is the list-style sequencing monad:
-
-
-
(a ++ b) flatMap f == (a flatMap f) ++ (b flatMap f)
-
Stream.empty flatMap f == Stream.empty
== Technical notes==
''Note:'' since the chunk structure of the stream is observable, and
the right identity law uses a weaker notion of equality,
normalizes both sides with respect to chunk structure:
s flatMap Stream.emit
produces a stream of singleton chunks,the right identity law uses a weaker notion of equality,
===
whichnormalizes both sides with respect to chunk structure:
(s1 === s2) = normalize(s1) == normalize(s2)
where
==
is full equality(
a == b
iff f(a)
is identical to f(b)
for all f
)normalize(s)
can be defined as s.flatMap(Stream.emit)
, which justproduces a singly-chunked stream from any input stream
s
.For instance, for a stream
- the result of
- the result of
The latter is using the definition of
s
and a function f: A => B
,- the result of
s.map(f)
is a Stream with the same chunking as the s
; wheras...- the result of
s.flatMap(x => S.emit(f(x)))
is a Stream structured as a sequence of singleton chunks.The latter is using the definition of
map
that is derived from the Monad
instance.This is not unlike equality for maps or sets, which is defined by which elements they contain,
not by how these are spread between a tree's branches or a hashtable buckets.
However, a
so two streams "equal" under that notion may give different results through this method.
not by how these are spread between a tree's branches or a hashtable buckets.
However, a
Stream
structure can be observed through the chunks
method,so two streams "equal" under that notion may give different results through this method.
''Note:'' For efficiency
chunk at a time and preserves chunk structure, which differs from
the
which would produce singleton chunk. In particular, if
chunked version will fail on the first ''chunk'' with an error, while
the unchunked version will fail on the first ''element'' with an error.
Exceptions in pure code like this are strongly discouraged.
[[Stream.map]]
function operates on an entirechunk at a time and preserves chunk structure, which differs from
the
map
derived from the monad (s map f == s flatMap (f andThen Stream.emit)
)which would produce singleton chunk. In particular, if
f
throws errors, thechunked version will fail on the first ''chunk'' with an error, while
the unchunked version will fail on the first ''element'' with an error.
Exceptions in pure code like this are strongly discouraged.
- Companion
- object
object text
Provides utilities for working with streams of text (e.g., encoding byte streams to strings).
Types
A stream transformation represented as a function from stream to stream.
Pipes are typically applied with the
through
operation on Stream
.A stream transformation that combines two streams in to a single stream,
represented as a function from two streams to a single stream.
represented as a function from two streams to a single stream.
Pipe2
s are typically applied with the through2
operation on Stream
.