Creating Streams
In this section, we’ll explore various methods for creating Effect Stream
s. These methods will help you generate streams tailored to your needs.
You can create a pure stream by using the Stream.make
constructor. This constructor accepts a variable list of values as its arguments.
Sometimes, you may require a stream that doesn’t produce any values. In such cases, you can use Stream.empty
. This constructor creates a stream that remains empty.
If you need a stream that contains a single void
value, you can use Stream.void
. This constructor is handy when you want to represent a stream with a single event or signal.
To create a stream of integers within a specified range [min, max]
(including both endpoints, min
and max
), you can use Stream.range
. This is particularly useful for generating a stream of sequential numbers.
With Stream.iterate
, you can generate a stream by applying a function iteratively to an initial value. The initial value becomes the first element produced by the stream, followed by subsequent values produced by f(init)
, f(f(init))
, and so on.
Stream.scoped
is used to create a single-valued stream from a scoped resource. It can be handy when dealing with resources that require explicit acquisition, usage, and release.
Much like the Effect
data type, you can generate a Stream
using the fail
and succeed
functions:
You can construct a stream from a Chunk
like this:
Moreover, you can create a stream from multiple Chunk
s as well:
You can generate a stream from an Effect workflow by employing the Stream.fromEffect
constructor. For instance, consider the following stream, which generates a single random number:
This method allows you to seamlessly transform the output of an Effect into a stream, providing a straightforward way to work with asynchronous operations within your streams.
Imagine you have an asynchronous function that relies on callbacks. If you want to capture the results emitted by those callbacks as a stream, you can use the Stream.async
function. This function is designed to adapt functions that invoke their callbacks multiple times and emit the results as a stream.
Let’s break down how to use it in the following example:
The StreamEmit.Emit<R, E, A, void>
type represents an asynchronous callback that can be called multiple times. This callback takes a value of type Effect<Chunk<A>, Option<E>, R>
. Here’s what each of the possible outcomes means:
-
When the value provided to the callback results in a
Chunk<A>
upon success, it signifies that the specified elements should be emitted as part of the stream. -
If the value passed to the callback results in a failure with
Some<E>
, it indicates the termination of the stream with the specified error. -
When the value passed to the callback results in a failure with
None
, it serves as a signal for the end of the stream, essentially terminating it.
To put it simply, this type allows you to specify how your asynchronous callback interacts with the stream, determining when to emit elements, when to terminate with an error, or when to signal the end of the stream.
You can create a pure stream from an Iterable
of values using the Stream.fromIterable
constructor. It’s a straightforward way to convert a collection of values into a stream.
When you have an effect that produces a value of type Iterable
, you can employ the Stream.fromIterableEffect
constructor to generate a stream from that effect.
For instance, let’s say you have a database operation that retrieves a list of users. Since this operation involves effects, you can utilize Stream.fromIterableEffect
to convert the result into a Stream
:
This enables you to work seamlessly with effects and convert their results into streams for further processing.
Async iterables are another type of data source that can be converted into a stream. With the Stream.fromAsyncIterable
constructor, you can work with asynchronous data sources and handle potential errors gracefully.
In this code, we define an async iterable and then create a stream named stream
from it. Additionally, we provide an error handler function to manage any potential errors that may occur during the conversion.
You can create a stream that endlessly repeats a specific value using the Stream.repeatValue
constructor:
Stream.repeat
allows you to create a stream that repeats a specified stream’s content according to a schedule. This can be useful for generating recurring events or values.
Imagine you have an effectful API call, and you want to use the result of that call to create a stream. You can achieve this by creating a stream from the effect and repeating it indefinitely.
Here’s an example of generating a stream of random numbers:
You can repeatedly evaluate a given effect and terminate the stream based on specific conditions.
In this example, we’re draining an Iterator
to create a stream from it:
You can create a stream that emits void
values at specified intervals using the Stream.tick
constructor. This is useful for creating periodic events.
In functional programming, the concept of unfold
can be thought of as the counterpart to fold
.
With fold
, we process a data structure and produce a return value. For example, we can take an Array<number>
and calculate the sum of its elements.
On the other hand, unfold
represents an operation where we start with an initial value and generate a recursive data structure, adding one element at a time using a specified state function. For example, we can create a sequence of natural numbers starting from 1
and using the increment
function as the state function.
The Stream module includes an unfold
function defined as follows:
Here’s how it works:
- initialState. This is the initial state value.
- step. The state function
step
takes the current states
as input. If the result of this function isNone
, the stream ends. If it’sSome<[A, S]>
, the next element in the stream isA
, and the stateS
is updated for the next step process.
For example, let’s create a stream of natural numbers using Stream.unfold
:
Sometimes, we may need to perform effectful state transformations during the unfolding operation. This is where Stream.unfoldEffect
comes in handy. It allows us to work with effects while generating streams.
Here’s an example of creating an infinite stream of random 1
and -1
values using Stream.unfoldEffect
:
There are also similar operations like Stream.unfoldChunk
and Stream.unfoldChunkEffect
tailored for working with Chunk
data types.
Stream.paginate
is similar to Stream.unfold
but allows emitting values one step further.
For example, the following stream emits 0, 1, 2, 3
elements:
Here’s how it works:
- We start with an initial value of
0
. - The provided function takes the current value
n
and returns a tuple. The first element of the tuple is the value to emit (n
), and the second element determines whether to continue (Option.some(n + 1)
) or stop (Option.none()
).
There are also similar operations like Stream.paginateChunk
and Stream.paginateChunkEffect
tailored for working with Chunk
data types.
You might wonder about the difference between the unfold
and paginate
combinators and when to use one over the other. Let’s explore this by diving into an example.
Imagine we have a paginated API that provides a substantial amount of data in a paginated manner. When we make a request to this API, it returns a ResultPage
object containing the results for the current page and a flag indicating whether it’s the last page or if there’s more data to retrieve on the next page. Here’s a simplified representation of our API:
Our goal is to convert this paginated API into a stream of RowData
events. For our initial attempt, we might think that using the Stream.unfold
operation is the way to go:
However, this approach has a drawback, it doesn’t include the results from the last page. To work around this, we perform an extra API call to include those missing results:
While this approach works, it’s clear that Stream.unfold
isn’t the most friendly option for retrieving data from paginated APIs. It requires additional workarounds to include the results from the last page.
This is where Stream.paginate
comes to the rescue. It provides a more ergonomic way to convert a paginated API into an Effect stream. Let’s rewrite our solution using Stream.paginate
:
In Effect, there are two essential asynchronous messaging data types: Queue and PubSub. You can easily transform these data types into Stream
s by utilizing Stream.fromQueue
and Stream.fromPubSub
, respectively.
We can create a stream from a Schedule
that does not require any further input. The stream will emit an element for each value output from the schedule, continuing for as long as the schedule continues: