streaming
Wraps 2 sinks and will automatically begin reading from the second only when the first is empty.
Wraps 2 sinks and will automatically begin reading from the second only when
the first is empty. The None from the first sink is never exposed. The
first error reported from either sink is propagated.
A Pipe is a callback-based data transport abstraction meant for handling streams.
A Pipe is a callback-based data transport abstraction meant for handling streams. It provides backpressure feedback for both the write and read ends.
Pipes are primarily a way to easily process incoming/outgoing streams and manage
backpressure. A Producer pushes items into a pipe and a consumer pulls them
out. Pulling is done through the use of a callback function which the Pipe
holds onto until an item is pushed. Each call to pull will only ever pull one
item out of the pipe, so generally the consumer enters a loop by calling pull
within the callback function.
Backpressure is handled differently for the producer and consumer. In effect,
the consumer is the "leader" in terms of backpressure, since the consumer must
always ask for more items. For the producer, the return value of push will
indicate if backpressure is occurring. When the pipe is "full", push returns
a Trigger, which the producer "fills" by supplying a callback function. This
function will be called once the backpressure has been alleviated and the pipe
can accept more items.
This is a special exception that Input/Output controllers look for when error handling pipes.
This is a special exception that Input/Output controllers look for when error handling pipes. In most cases they will log the error that terminated the pipe, but for this one exception, the failure will be silent. This is basically for situations where a certain amount of data is expected but for some reason the receiver decides to cancel for some business-logic reason.
A Sink is the write side of a pipe.
A Sink is the write side of a pipe. It allows you to push items to it, and will return whether or not it can accept more data. In the case where the pipe is full, the Sink will return a mutable Trigger and you can attach a callback to for when the pipe can receive more items
A Source is the read side of a pipe.
A Source is the read side of a pipe. You provide a handler for when an item is ready and the Source will call it. Note that if the underlying pipe has multiple items ready, onReady will only be called once. This is so that the consumer of the sink can implicitly apply backpressure by only pulling when it is able to
When a user attempts to push a value into a pipe, and the pipe either fills or was already full, a Trigger is returned in the PushResult.
When a user attempts to push a value into a pipe, and the pipe either fills or was already full, a Trigger is returned in the PushResult. This is essentially just a fillable callback function that is called when the pipe either becomes empty or is closed or terminated
Notice that when the trigger is executed we don't include any information about the state of the pipe. The handler can just try pushing again to determine if the pipe is dead or not.
Multiplexing is the process of combining multiple independant streams into a single stream.
Multiplexing is the process of combining multiple independant streams into a single stream. Each message in the multiplexed stream carries information about which sub-stream it originated from, so that eventually the stream can be demultiplexed back into the constituent sub-streams.
By multiplexing a "base" Sink[T], a new Sink[SubSource[K,T]] is
created, with each SubSource containing a Source[T]. When a
Source is pushed into the multiplexing sink, all messages pushed to
that source are routed into the base sink.
Likewise, demultiplexing a multiplexed Source[T] will create a
Source[SubSource[K,T]], with each Subsource being one of the
sub-streams being fed into the multiplexed source.