Skip to content

Scaling Voice AI at MasterClass with Effect & TypeScript

Jun 24, 2025

#3: Scaling Voice AI at MasterClass with Effect & TypeScript

In this episode Johannes Schickling had a conversation with David Golightly, Staff Engineer at MasterClass, to explore how his team built Cortex – a real-time voice AI orchestration layer that powers personalized conversations with celebrity instructors like Gordon Ramsay and Mark Cuban.

In this episode of Cause & Effect, Johannes Schickling had a conversation with David Golightly, Staff Engineer at MasterClass, to explore how his team built Cortex – a real-time voice AI orchestration layer that powers personalized conversations with celebrity instructors like Gordon Ramsay and Mark Cuban.

Song: Dosi & Aisake - Cruising [NCS Release]
Music provided by NoCopyrightSounds
Free Download/Stream: http://ncs.io/Cruising
Watch: http://ncs.lnk.to/CruisingAT/youtube

  • (00:00) - Intro & David’s background
  • (04:56) - Discovering Effect & early impressions
  • (08:32) - Why RxJS wasn’t enough for Cortex
  • (16:15) - MasterClass On Call
  • (19:10) - Building the orchestration layer
  • (25:30) - Incremental adoption of Effect at MasterClass
  • (31:43) - Text-to-speech component
  • (40:08) - Error handling, observability, open-telemetry
  • (01:01:20) - Looking ahead: Effect 4.0 & the future
  • (01:08:00) - Closing thoughts
Discuss this episode on Discord

Transcript

00:00When I started looking

00:01into Effect, I started seeing,

00:03Well, this has a lot of promise and

00:05potential if you use

00:07it in the right way.

00:08What I mean by that is you don't have to

00:10necessarily go from 0%

00:12to 100% overnight.

00:14You don't have to completely

00:16redesign the entire application

00:18to take advantage of some of the benefits

00:20that Effect has to offer.

00:22That's how I started, was incrementally

00:24adopting Effect in

00:26parts of the code base.

00:27And this was really

00:28what enabled it to work.

00:33Welcome to Cause & Effect,

00:35a podcast about the TypeScript library

00:37and ecosystem called Effect,

00:40helping engineers to build

00:41production-ready software.

00:43I'm your host, Johannes Schickling, and

00:44I've been building with

00:45Effect for over four years.

00:47With this podcast, I want to help others

00:49understand the powers and

00:51benefits of using Effect.

00:53In this episode, I'm talking to David

00:55Golightly, who's a Staff

00:56Engineer at MasterClass.

00:58In this conversation, we explore how

01:00David has built Cortex,

01:02MasterClass's AI voice chat system,

01:04leveraging Effect streams heavily

01:06to build a cutting-edge AI

01:08experience. Let's get into it.

01:10Hey David, so great to have you on the

01:12podcast. How are you doing?

01:14Doing great. Thanks for having me.

01:17I'm really excited. The two of us had the

01:19pleasure to now meet in person twice in

01:22the course of the last half a year.

01:24The first time we've met in person was at

01:26the Effect TypeScript

01:28Meetup in San Francisco.

01:30And then a couple of weeks ago, we had

01:33the pleasure to meet again in beautiful

01:35Italy in Livorno, where you've also given

01:38a talk about Cortex and what you're

01:41building at MasterClass.

01:43And it was such a pleasure to spend a

01:46bunch of time together and talk about all

01:48things TypeScript related beyond.

01:51You also have some beautiful music gear

01:53in the background. But today we're going

01:55to talk about how you're

01:57using Effect at MasterClass.

01:58So would you mind introducing yourself

02:01and sharing your background?

02:04Yeah, sure. My name is David Golightly. I

02:07am a Staff Engineer at MasterClass.

02:10And I've been in the industry building

02:12mostly web applications for

02:14about almost 18 years now.

02:18I've also dabbled in some mobile

02:20applications, some back end, some other

02:23embedded applications over the years.

02:25But my main focus has been first

02:27in the JavaScript world and then more

02:29recently in the TypeScript

02:30world for the last several years.

02:32My actual background, though, I don't

02:34have a Computer Science degree. My

02:36education is in music.

02:38And I grew up

02:40surrounded by pianos.

02:41My father is a piano technician and also

02:45a piano professor who plays

02:47classical, you know, everything you could

02:50think of classical music.

02:52He has played it. You know, he's played

02:54with orchestras and he's, you know,

02:56played solo concerts and chamber music.

02:58And so forth. So just steeped in

03:02classical music growing up in pianos and

03:04also in repairing pianos, which I found

03:07somehow translated

03:09into programming sort of.

03:12You're doing a lot of really repetitive

03:13tasks. You have 88 keys. You have 230

03:16some strings on a regular piano.

03:19So there's a lot of repetitive tasks and

03:21you quickly figure out ways to work more

03:24efficiently when

03:25you're doing piano repair.

03:27So later on, after I got my degree in

03:29music, I found that it wasn't well, well,

03:32music is a passion of mine and it's

03:34something that I still do.

03:35It's not something I want to try to make

03:37a career in. And so I quickly discovered

03:40that, well,

03:41programming is just right there.

03:43And I actually enjoy doing it somewhat.

03:45And so I devoted some energy into

03:47learning how to do that professionally.

03:50And, you know, all these

03:51years later, here I am.

03:52That is awesome. And such a funny

03:54parallel since we never talked about

03:57this, but my mother also

03:58happens to be a professor in piano.

04:01And so I likewise also grew up surrounded

04:05by pianos and played the piano for quite

04:09a while, but then also found myself more

04:12attractive to technology.

04:14But I think this is a fairly common

04:17overlap of people who had their start in

04:22music or a different kind of art and then

04:25gravitated also towards programming.

04:28And I think it's just a certain way of

04:30like how someone's brain works that

04:33really like there's a couple of people

04:36come to mind to have professional music

04:38background who are just like absolutely

04:40brilliant in the engineering field.

04:43So that's a very funny parallel. But

04:46keeping it a bit more focused on like

04:48Effect, what has brought you to Effect?

04:50How did you first learn about it? And

04:53what did you think about it when you

04:55saw it the first time?

04:56Yeah, so I was not necessarily going to

05:00adopt Effect for the work I was doing,

05:02say, maybe over a year ago,

05:05which was more React focused.

05:08It wasn't something that I was

05:08necessarily considering.

05:10You know, I think React is a

05:12great framework and it totally

05:13transformed how we do

05:16front end development.

05:17But it's not without its problems in

05:20terms of state management and so forth.

05:21But I was mostly happy with it for the

05:23front end and we try to

05:24keep things lightweight there.

05:26But then I got asked to work on this new

05:28project Cortex, which I'll

05:29talk about a little bit more.

05:31And that was going to be a server side

05:33application that was also not a

05:35conventional API server.

05:36Instead, it was managing a lot of async

05:38events, a lot of open WebSockets, more

05:42than one of like several open WebSocket

05:44WebSocket connections that

05:46it needed to then coordinate.

05:48So I was looking at a proof of

05:51concept that somebody else at our company

05:53had built that was really, you

05:55know, what we call async hell.

05:57That is just lots of callbacks, lots of

06:01event listeners passing

06:03around references to random things.

06:05And we're starting to try to build into

06:07it observability and error handling and,

06:11you know, more advanced interruption type

06:13features that involve coordinating the

06:15state of multiple WebSockets.

06:17And it just wasn't going to work.

06:18It was not going to scale.

06:20The implementation was just going to get

06:22increasingly tangled.

06:24And so that's when I started looking

06:26around for, well, what is

06:27the state of the art here?

06:28I had used RxJS in the past.

06:30And I think that, you know, for people

06:32who've used RxJS, when they see Effect,

06:34that's what their first thought is.

06:36Oh, this is RxJS.

06:37I've seen this before.

06:38You know, this is this is familiar to me.

06:39It's just another RxJS kind of knockoff.

06:42I didn't really find it to be the case,

06:44though, for several reasons.

06:46But I think that what really caught my

06:48eye with Effect was the

06:51really active community.

06:53Because when I'm looking to adopt a new

06:55framework that is going to form the

06:58backbone of how we build an application,

07:00I don't want to have to

07:01be inventing stuff out of whole cloth

07:03I don't want to have to

07:04resurrect things from a couple of

07:06examples that don't

07:07really apply to my application.

07:10I don't want to be in

07:12the dark and be on my own.

07:13And I especially don't want to ask my

07:15teammates to accept a framework where

07:18they don't have the proper

07:20documentation or the guidance.

07:22They don't know where to find help.

07:23And so really, I found that the community

07:25seems to be Effect's biggest asset.

07:28And just how helpful everybody is and how

07:30enthusiastic everybody is

07:32really ease the adoption process.

07:35That is awesome to hear.

07:36And that definitely also reflects my own

07:39perspective and my own reality.

07:41This is what drew me

07:42to Effect very early on.

07:44The community was much smaller at this

07:46point and I tried to play a positive role

07:49in like growing and

07:50forming that community.

07:52But it's also something I'm like super

07:54excited about and super

07:56proud of that we like together.

07:58Yeah, broad Effect to

08:00the point where it is today.

08:02And it attracts more

08:03like brilliant minds.

08:04And it's also interesting that Effect

08:08attracts a lot of experience engineers

08:10like yourself, but also new engineers

08:13that are drawn to

08:15building something great.

08:17And that community is such a compelling

08:19aspect of that as well.

08:21So maybe to linger on the RxJS part for a

08:25little bit since yes, quite a few people

08:28are kind of comparing Effect with RxJS.

08:33For folks who are familiar with RxJS but

08:35not yet with Effect, where would you say

08:38there are parallels and the commonalities

08:41and where would you say

08:43things are much more different?

08:45Well, in RxJS, the idea is

08:47it's very stream oriented.

08:50So you essentially are creating

08:52everything is an observable.

08:56It can emit events.

08:58And this is really useful actually.

09:00I think a lot of programming situations

09:02can be modeled in this way.

09:04You have especially the asynchronous

09:06stuff like really anything side-effectey

09:08is happening asynchronously.

09:11You can't necessarily expect it or

09:13predict it, but then you have to react to

09:15it and do something in response to it.

09:18You have user input,

09:21user loads the web page.

09:22You didn't know when or why

09:23they were going to do that.

09:24They click on a link or they decide to

09:27watch a video or fill out

09:28a form or what have you.

09:31This is ultimately the timing and

09:33coordination of these things

09:34and these inputs are driven by humans.

09:36And likewise, when you have systems where

09:38you have connections coming from third

09:40party services or you have other kinds of

09:43asynchronous behaviors where you have to

09:45connect users to each other.

09:49A lot of the hardest parts of programming

09:51happen when we have asynchronous behavior

09:55that we have to model.

09:56And this is why I think you look at

09:59paradigms like HTTP or more specifically

10:03like a model view controller that you see

10:05in Rails that is still so successful.

10:07And one of the things that that has

10:09largely done is allow us to architect

10:11applications around kind of one shot

10:13actions where you get essentially a

10:17function call via an HTTP request.

10:20And your application now has to implement

10:24the function call and respond with it.

10:27But it's everything that happens between

10:29the request and the response is largely

10:32typically serialized.

10:34It's sequential. It's

10:35not in parallel, right?

10:36Which is great. It makes things a lot

10:38more easy to reason about.

10:40You know, things happen in a sequence A,

10:42B, C, D, E and then you're done.

10:44But most of what we have to deal with,

10:47especially in building web interfaces or

10:50building applications like

10:51Cortex, is not like that.

10:53Because you have a lot of things that can

10:54happen and you might be in the middle of

10:56one thing and then something else happens

10:57that you didn't expect.

10:58And now you have to deal with that. And a

11:01lot of this also involves kind of keeping

11:03state updates going.

11:05This is why honestly, this is why React

11:07superseded jQuery is because they're

11:09really good at this.

11:10But getting back to your question about

11:12RxJS, RxJS is kind of an even more next

11:16level approach at this where you can

11:17model everything as an observable.

11:19You don't know when it's going to happen,

11:21but you know that you might receive one

11:23or more different kind of events.

11:25And then how do you combine events from

11:28one observable with events from another

11:30observable when they need to interact to

11:32produce some sort of output.

11:34And RxJS is really built

11:35around this sort of model.

11:38The thing is though, RxJS is also trying

11:41to be not just RxJS, but it's a reactive

11:45framework that is cross-platform.

11:48So you have the same APIs more or less

11:51that are designed for JavaScript or

11:56TypeScript that are then also ported over

11:58to Java or ported over to Swift or ported

12:01over to Kotlin or ported over to

12:03whatever other language or framework.

12:06And so I think there was an intentional

12:08desire on the part of the designers there

12:11to keep things language agnostic in their

12:14designs, which in my opinion is kind of a

12:17flaw because it means that it's

12:20yes, you can transfer that knowledge

12:21from one language to

12:23another pretty easily.

12:24If you know reactive programming, you can

12:26do it in any language.

12:27But it means that you're passing on a lot

12:30of the strengths of each language that

12:32you're in in order to make things as same

12:35as possible between every

12:38language that you're working in.

12:40Effect in my mind is taking kind of a

12:42very different approach, even though it's

12:44based on the ZIO, which

12:45is the Scala equivalent.

12:48It was the inspiration for Effect.

12:50My understanding is that now, Effect's

12:52design is pretty detached from needing to

12:55keep parity with ZIO.

12:57Effect is basically about TypeScript and

13:00getting really good at TypeScript, making

13:02TypeScript the best it can be.

13:05And there is no, as far as I know, I

13:08can't speak for the core team, but as far

13:11as I know, there's no attempt at saying,

13:13"Well, what if we make Effect

13:15for Java or Swift or Kotlin?"

13:17It's not about that. It's just about what

13:21is TypeScript good at and how can we be

13:23really good at that.

13:24So that's a really big point, I think,

13:26because with RxJS in TypeScript, it never

13:30felt like they really got how to make a

13:33TypeSafe reactive stream

13:35application really work.

13:37the types

13:37kind of sort of worked.

13:40As far as I recall, there wasn't really a

13:42great story around requirements

13:44management or even really error handling

13:47the way there is with Effect.

13:48And so it lacked a lot of the

13:50potential, it failed to take advantage

13:53of a lot of the potential that TypeScript

13:55has to offer in the way that Effect does.

13:58Does that answer your question?

14:00No, it does. And it

14:01reflects also my perspective.

14:03In fact, I've been using ReactiveCocoa

14:06back in the days, and I believe that was

14:09early on in Swift, and it was quite

14:12pleasant to use, but it was very

14:14deliberate for the purposes of streams.

14:19And there was sort

14:20of like a peak moment

14:22where it got really into it.

14:23But then I felt like a lot of impedance

14:26mismatch where everything wanted to be

14:29modeled as a stream, but

14:31not everything is a stream.

14:33Some things are just like a one-off

14:35thing, and you just do an HTTP request

14:39and you get it back.

14:41Ensure that might fail, and eventually

14:43you're just interested in one value, but

14:45you kind of model your tree

14:47tries, etc. as a stream, etc.

14:50So this was my first exposure

14:52really to this mindset, but it felt like

14:56the gravity was too hard on

14:59this very specific primitive.

15:02And this is where I would rather like

15:05create more distance between Effect and

15:07RxJS, because Effect is not trying to

15:11tell you, "Hey, everything is a stream."

15:13No. Effect gives you a stream abstraction

15:16that lets you use the stream abstraction

15:19when you have a stream.

15:20For example, when you use a WebSocket or

15:24when you use something else, maybe you

15:26want to read a file and use streams

15:29through the bytes, etc.

15:30This is where streams are a great use

15:32case, but if you just want to do a

15:35one-off thing that happens to be

15:37asynchronous, then you don't need to be

15:39forced that this is the

15:40stream mindset, but this is where Effect

15:43gives you different primitives.

15:44And I think this is kind of the huge leap

15:48beyond what RxJS gives you, and I think

15:51this was always like the big caveat for

15:53RxJS that not everyone buys into the

15:57"everything is a stream" mindset.

16:00And this is where Effect is much more

16:02applicable to any sort of circumstance

16:05where programming

16:06languages is applicable to.

16:08Very interesting to hear your perspective

16:11on this. You've been

16:12mentioning Cortex now a few times.

16:15So before we dive into how it's built

16:17specifically and how it leverages Effect,

16:20can you motivate what Cortex is and what

16:24role it plays within MasterClass?

16:27Right, yes. So Masterclass has been

16:30developing over the last year plus our

16:34own in-house voice AI chat that has got

16:38kind of a MasterClass twist to it,

16:39and that you're not talking with

16:41fictional, invented characters or

16:44anonymous service bots, but we got

16:48authorization from many of our

16:50instructors who are working closely with

16:52us to essentially clone

16:54them into an AI persona.

16:57This includes an LLM that is trained on

17:01their ideas, their writings, their public

17:05speaking, and it also

17:06includes a voice clone of them.

17:08And in some cases also, we have some

17:11chefs like Gordon Ramsay is being

17:14released soon, and we have a lot of his

17:16recipes that are included, and he can

17:18walk you through

17:18making one of these recipes.

17:20So it's a really interesting, interactive

17:23way that I think enables MasterClass

17:26instructors to make themselves really

17:29available to a much broader audience than

17:32they would be able to on

17:34their own as individuals.

17:36And so this has also presented an

17:38enormous technical challenge because I

17:40think when you are talking to an

17:43anonymous AI, well, you might

17:45be a little bit more forgiving.

17:47You know that it's a bot that you're

17:48speaking to on some level.

17:49But with these real people who are very

17:54invested in the integrity of their image

17:56and their public persona, the bar is much

18:00higher, I think, in terms

18:02of really representing that.

18:03And this is also like if you sign up and

18:05you try to interact with Gordon Ramsay,

18:06well, if you're a fan, how many times

18:08have you seen him on TV?

18:10You can watch the MasterClass courses,

18:12you know, it's also

18:12like more than an hour of content.

18:15You know what he sounds like, you

18:16know how he speaks, you know what he's

18:19passionate about, what he doesn't like.

18:21You know, this personality is something

18:24that you've probably learned already. And

18:25now we have to put that into an AI agent

18:28or an AI bot that you can talk to.

18:31And so I feel like the very similitude has

18:34to really be there. And so that's been

18:36really challenging just to kind of get

18:38through the uncanny valley stage of that.

18:41And so this is the background of what

18:43we're building. If you want to try it

18:45out, you can go to

18:46oncall.masterclass.com, all one word

18:51oncall, O-N-C-A-L-L, dot masterclass.com.

18:54And you can sign up and try it out.

18:57So this is the general product that we're

18:59building. And we have a whole LLM team,

19:01you know, a machine learning team that is

19:03working on getting the voice models and

19:05the LLM models really trained up to

19:08represent each instructor.

19:10And what I got brought on to do was

19:12essentially to create an orchestration

19:14layer because we have several different

19:17components in the pipeline in order to

19:21build this experience.

19:22First, we have a speech-to-text component

19:26where the end user's voice of microphone

19:30stream is sent to a speech-to-text

19:33service that then listens to the user's

19:36speech and derives transcripts

19:38from it of what they've said.

19:41And then at certain intervals when we

19:43believe that the user's done speaking and

19:45is ready to hear what the AI has to say,

19:47which is not a trivial problem, by the

19:49way, we then send that transcript to the

19:52LLM to generate a response.

19:55And then the LLM generates text response,

19:57and then we send that off to a text-to-speech

19:59service to generate the audio for

20:02that response in the instructor's voice

20:05and then send that back to the user.

20:06So we have several different

20:08asynchronous services, the

20:10speech-to-text and the text-to-speech

20:12components of which are WebSockets.

20:15And then the LLM is a HTTP streaming

20:19connection essentially wrapped in an

20:22async iterable in the JavaScript world.

20:25So this is coordinating

20:28between these different elements.

20:30But as you can see, there's kind of a

20:31step one, step two,

20:32step three to this process.

20:34What makes it kind of interesting is, as

20:36I mentioned, we don't know if

20:37the user is done talking yet.

20:39The user can kind of start

20:40talking again at any time.

20:42What we want to do is shut down anything

20:44the bot is doing at that point because

20:46the user is supreme.

20:48The user drives the conversation.

20:50We don't want the bot talking over the

20:52user when the user is

20:54trying to say something.

20:55And so it's really crucial that we are

20:58very responsive to that and stop whatever

21:01is in progress downstream, whether it's

21:04at the LLM stage or

21:05the text-to-speech stage.

21:06And so this is

21:07essentially what Cortex is doing.

21:08Cortex is a Node.js application that is

21:12connected to bi-WebSockets from our two

21:15clients that we have, Web and iOS

21:18clients, that connect

21:19to Cortex over WebSocket.

21:20And Cortex, in turn, provides WebSocket

21:24interface that abstracts these multiple

21:27services and receives the microphone

21:31audio stream from the user

21:32and emits back bot audio events.

21:35So from the client's perspective, the

21:38contract is simply send the user's

21:42microphone audio to Cortex and receive

21:44back the bot audio and then the clients

21:48have to actually play

21:49it through the speakers.

21:50And that's kind of the, you know, the

21:52whole the whole contract.

21:53There's a couple of other details in

21:55terms of, you know, if a tool call

21:58happens or, you know, if state change

22:01happens or something like that.

22:03But that's the main gist of what that

22:06WebSocket contract to

22:08Cortex looks like under the hood.

22:10However, Cortex has to orchestrate the

22:13various services that I talked about,

22:16execute tool calls and and do a whole

22:18bunch of other, you know, state

22:20management under the hood.

22:22And so that's what we built with

22:22Effect. It's 100 percent.

22:24I mean, 100 percent. But like it's it's

22:27pretty much the every piece of it is

22:29built in Effect or

22:30using Effect in some way.

22:32That's awesome. So you've mentioned that

22:34initially when you got into this project,

22:37you got a first like proof of concept and

22:39where it really like exposed you to all

22:41of the complexity that's for solving this

22:45actual product to

22:46implement this actual product.

22:48This complexity need to be tamed. Did you

22:51work off that initial proof of concept or

22:54did you rather roll

22:55something from scratch?

22:57And how did you go about that?

22:59That's an interesting question because

23:00I'm always trying to figure out how to.

23:04Like re-architect a

23:06piece of software, you know.

23:08It's something you don't want

23:09to have to do very often.

23:11Also, I don't know, actually, if it's

23:13me or just the stage of career I'm

23:16at and the kind of problems that I get

23:18brought in to help with.

23:19But I often find myself looking at a

23:21piece of software that is

23:22in need of re-architecting.

23:24I think it's largely because I was

23:26specifically asked to

23:28re-architect it in this situation.

23:30But the proof of concept was interesting

23:33because, you know, it was built by our

23:37project lead on call, who

23:39is now our VP of architecture, who's

23:42just a brilliant person who has no

23:44experience really with

23:45JavaScript or TypeScript.

23:47And asked chat GPT to help. I think it

23:50was chat GPT to help with building it and

23:53essentially just piece that all together

23:55from chat GPT prompts.

23:58There were a couple of things that came

24:00out of that that were, for example, a

24:02library that is like a private library

24:05that isn't really maintained.

24:06There wasn't really a

24:09top level architecture.

24:11you had individual pieces that

24:12were kind of worked in isolation, but

24:13then the glue between them became really

24:16convoluted and hard to follow.

24:19So so that that kind of became a problem.

24:21But it worked. You know, ultimately, this

24:23proof of concept kind

24:24of worked for a demo.

24:27It didn't handle interruptions very well.

24:30It didn't have great latency.

24:32I think in reality of like most

24:35production grade systems are actually let

24:39me take that back, probably not

24:40production grade, but applications,

24:43JavaScript applications that are running

24:44in production are probably

24:47That sort of state where is working well

24:51enough on the happy path, but all of like

24:54those higher level requirements that

24:57really make a end user experience superb

25:00that you can interrupt an AI model when

25:05when it's talking to you or that's more

25:07resilient to errors, etc.

25:09This is where you need to go beyond that

25:11convoluted everything

25:13kind of pieced together.

25:15And I think this is like a perfect point

25:17to to start with, like

25:20applying Effect to to go beyond that.

25:23That's sort of like a messy state.

25:27Yeah, this is why I wanted to

25:31tame the asynchronous

25:32behaviors in in Cortex.

25:35And I was looking for a package that

25:37would help me do that. And

25:38this is when I found Effect.

25:40And then at that point like I

25:42said, I had seen RxJS. and I had

25:45good experiences at

25:46first building RxJS

25:47But like kind of one of the common

25:49complaints is that, well, now, like you

25:52said, everything has to be a stream.

25:53Now you have to model

25:54everything in this way.

25:56And it really completely changes how you

25:58architect your software and you need a

26:01lot of buy in from your team if you're

26:02going to start doing this, because now

26:04everybody has to learn this.

26:05They can't just sit down and, you know,

26:07start coding on whatever part, you know,

26:09feature they're trying to do.

26:10They have to first learn this framework

26:11and that can be a big ask for folks.

26:15And so that's something that I wanted to

26:16be conscientious about.

26:18However, when I started

26:19looking into Effect, I started seeing,

26:22well, you know, this has a lot of promise

26:24and potential if you

26:25use it in the right way.

26:27And what I mean by that is like, you

26:29know, you don't have to necessarily go

26:32from zero percent to

26:33100 percent overnight.

26:34You don't have to completely redesign the

26:37entire application to take advantage of

26:39some of the benefits

26:41that Effect has to offer.

26:42That's kind of how I started, was

26:44incrementally adopting

26:45Effect in parts of the code base.

26:48And this was really what

26:49enabled it to work at all.

26:51If I had to say, hold on, I need to take

26:54this for a month and just completely

26:56build it from scratch and nothing's going

26:58to work and I can't deliver any new

26:59features until I'm done.

27:03That was not going to fly.

27:04I was in a position

27:05where I did not have to ask for

27:07permission to go live in a cave for a

27:10month or however long

27:11it was going to take.

27:12I always like avoid

27:13being in that situation.

27:15I don't think that's a good place for us

27:16to be in as software engineers.

27:18We need to be shipping like every day,

27:20shipping features, shipping bug fixes,

27:22shipping products constantly.

27:25And like we never have the luxury to just

27:28say, well, hold on, I'm just going to

27:30like spend a month not shipping, you

27:32know, to do this technical thing that I

27:34can't really explain

27:35why I have to do this.

27:36Right. Nobody, nobody likes that.

27:38So fortunately, Effect is not

27:41a framework that is like that.

27:43That's not what was

27:44asked of us to adopt Effect.

27:47Instead, it was you can start plugging in

27:50Effect judiciously here and there.

27:52And I started at the top level of the

27:54application around really around the

27:56WebSocket connection layer, around the

27:58HTTP layer and the application runtime.

28:02And that allowed us to kind of have a top

28:05level of error handling that immediately

28:09provided benefits for recovery and so on.

28:13But we didn't stop there.

28:15That was just the first step.

28:17We started kind of rebuilding different

28:20parts of the application.

28:21Some of them were like isolated behind a

28:24run promise or, you know, like a stream

28:26that was, yeah, emitting

28:28events into an event emitter.

28:30But it allowed us to kind of rebuild

28:32isolated parts of the application on

28:35their own where like selectively where we

28:37thought they had the most value to be

28:39converted into Effect.

28:40And often driven by product requirements.

28:44So when I wanted to implement

28:45interruptions, that was the reason to now

28:49rebuild the speech-to-text component in

28:52Effect, because it was going to be a lot

28:54more work to do that.

28:56If I was going to have to do it

28:58the old way, it was like, really, it

29:00wasn't like rebuild the whole thing

29:02in Effect, the entire application.

29:05Or just keep working the old way. It was

29:07really for each isolated piece.

29:09It was like, do I want to like add more

29:11spaghetti to this application in this

29:13component or can I just rebuild this

29:15piece in Effect and make it much more

29:18expressive and elegant while building the

29:20feature that I'm being asked to build.

29:22So it was much less of a refactoring

29:26process and more of a incrementally

29:29adopting it while also building and

29:31delivering features.

29:32I love that. And I think you've hit on a

29:34couple of really nice points here.

29:36One is that Effect, even though you get

29:40a lot of benefits once you have

29:43Effectified more and more of your

29:45application, you're going to find that

29:46you can delete a lot of code.

29:48Everything just fits

29:49nicely together in the same way.

29:51If you incrementally adopt React, let's

29:54say we're just in this transition of like

29:56transitioning a larger code base from

29:58jQuery, something else to React.

30:00You don't need to do that in one like

30:03long night, but you

30:04can do that step by step.

30:06And once you've reached a point where

30:08everything is in React, then you can

30:10delete a whole bunch of like glue code, a

30:13whole bunch of like things that bridge

30:15from the old way to the new way.

30:17But it is also totally fine that you go

30:20through this like transitionary phase.

30:22But the other thing that you've mentioned

30:24that I think is like a super great

30:26insight, which is like, how do you

30:28prioritize what to refactor with Effect

30:31when or when to rethink

30:33something and apply Effect?

30:35And this is so elegant that you can like

30:38go top down from like, hey, what is the

30:41thing that we want to improve for users?

30:43What is like the business outcome that we

30:46want to affect here?

30:48And then like looking at those, Effect

30:51typically has

30:52something in store for that.

30:54If you want to improve performance, if

30:56you want to improve reliability,

30:58resilience, error handling, whatever it

31:01might be, Effect typically

31:02has something in store for that.

31:05And then you can prioritize what to work

31:07on depending on your end user needs.

31:10And hearing here about the speech to text

31:13aspect and applying Effect for that,

31:17that sounds super interesting.

31:19So I want to hear a little bit more like

31:20zooming into from macro into a bit more

31:23micro into this component.

31:25Can you give a rough overview of like how

31:28that thing was built before?

31:31What you diagnosed the biggest things to

31:35improve where and which primitives of

31:38Effect did you use to

31:40ultimately implement it?

31:42Well, actually what I prefer to do is

31:44zoom in on the text-to-speech component.

31:47Oh, I'm sorry.

31:49I remember that wrong.

31:50But yes, no, it's okay.

31:51You remembered it right.

31:53It was you.

31:53I said speech-to-text earlier, but I feel

31:55like the text-to-speech component.

31:58This is the component that I also talked

31:59about in my Effect Days

32:01talk and did a deep dive on.

32:03I feel like the text-to-speech component

32:05was kind of like a real unlock, really an

32:09aha moment of like, wow, this is this is

32:12what Effect can do in

32:14terms of like saving

32:15a lot of complexity from your code base.

32:18As I mentioned, this is a

32:20component where like it's a WebSocket and

32:22we stream LLM thoughts to it.

32:26So an LLM response, as you might be

32:28familiar, it sends what are called chunks

32:32in a chat completion response.

32:34And the chunks are usually a token or two

32:36tokens and they're not

32:37complete words a lot of the time.

32:39So then we're like accumulating the

32:42chunks into basically what we call

32:44coherent thoughts and the coherent

32:47thoughts can then be sent to the text-to-

32:50speech component to

32:51generate the bot audio.

32:53However, if there's an interruption, we

32:55need to shut down the LLM and we also

32:57need to shut down the

32:58text-to-speech component so that we don't

33:01continue to generate more thoughts based

33:03on the previous thing that the user said

33:05before they continued talking.

33:07And now we want to start over and respond

33:09to the most recent thing

33:11that that user has said.

33:12So the text-to-speech component now it's

33:16a WebSocket connection.

33:17When you send it a

33:19coherent thought, that connection

33:22will then respond asynchronously with one

33:26or more different

33:28events that you might get.

33:30And we're basically just

33:31streaming those up to the client.

33:32But when there's

33:34an interruption, we need to actually shut

33:36down the WebSocket

33:36connection, close the connection.

33:38Abruptly, so we don't get any more

33:40messages from it and then reopen it.

33:43And then in that period of time, and it's

33:45not usually very long, it can just be

33:47like a hundred milliseconds or two

33:48hundred milliseconds where we're waiting

33:50for that WebSocket connection to open.

33:52We hope that

33:52we've created a connection, but we've not

33:54yet received the open

33:55event from the connection.

33:56And it's in that time that we were often

33:59getting the LLM was trying to send

34:02messages to it, but it was erroring

34:04because we were sending WebSocket

34:07messages out to a

34:08WebSocket that was not yet open.

34:10So we had to now queue those messages to

34:13wait for the open event from the

34:16WebSocket connections and then flush the

34:18queue when it was open.

34:20So as you can imagine, this created some

34:22code complexity in the pre-Effect

34:26implementation and it was something that

34:29Effect turned out to be actually very

34:31good at because these are the kinds of

34:32things that Effect has out of the box.

34:35In Effect we were able to replace the

34:39WebSocket handling code with the

34:41WebSocket utility from

34:43Effect from effect/platform APIs

34:47And that has kind of a magical property

34:51to it that you don't really ever have to

34:53think about when the WebSocket is closing

34:55and opening and you don't

34:57have to wait for an open event.

34:59What it essentially gives you is what is

35:02called in Effect a channel.

35:03And this became a primitive

35:05that I became curious about.

35:06It's something that I wish was a little

35:08bit more first class in the effect world.

35:11It's certainly used behind the scenes in

35:14a lot of things like stream and other

35:17APIs in the Effect world.

35:19But this is what you get essentially when

35:21you create a WebSocket connection using

35:23the effect/platform API.

35:24But then if you use the Effect stream

35:26operator pipe through channel, you now

35:28have a duplex stream, which is one where

35:32you can start listening to other streams.

35:35And then instead of doing like a run for

35:37each or run collector, whatever you're

35:40doing and you typically do it with a

35:41stream, you now are piping the events or

35:45the items in that stream out through the

35:48channel through the WebSocket connection.

35:50And then downstream from that pipe

35:52through channel, you are getting incoming

35:54events that are coming from that

35:56WebSocket connection that you can then

35:58emit further on down your application.

36:01So this is great. But this is also what

36:04it is also doing is this is abstracting

36:06the WebSocket lifecycle

36:08into the stream lifecycle.

36:09So if you emit an error

36:12upstream, it will close

36:14the WebSocket for you.

36:15And then if you have a stream.retry, it

36:18will reopen the WebSocket for you

36:19in the event of an error.

36:21And because the streams are pull based,

36:23you don't have to rely on queuing

36:26explicitly. You aren't going

36:28to miss any of your events.

36:30When the stream is reopened, it will

36:32start pulling those events

36:33and behaving as expected.

36:35So this really allowed us to

36:38abstract all of the sort of tangled like

36:39we had, I think, a promise that was a

36:42reference to a promise that was kept

36:44around and was

36:44awaited in different places.

36:45And it was it was a mess before. And now

36:49we had a single stream that was just a

36:52linear stream that where you had the

36:55stuff going out, going in the top of the

36:57stuff coming from the stream, coming out the bottom.

36:59And it became very easy to reason about

37:02what was happening. And you didn't really

37:03even have to think about the stream

37:05lifecycle at all. It was just you made an

37:07error when you want it to

37:09close and then just retry.

37:10The WebSocket

37:11connection is now handled by the stream

37:14lifecycle. So you can use

37:15the stream retry stream.

37:18will be shut down when the

37:21scope of the stream ends and the

37:24WebSocket will automatically closed.

37:26We also have a flush event that we send

37:30out to the text-to-speech

37:32service, which essentially says we're

37:34done sending you a new speech for now.

37:38So send us everything you got. And the

37:41peculiarity of this particular service is

37:44that they will then accept your flush

37:46event and they will

37:47promptly close the stream on you.

37:50That's not really what we wanted, but I

37:52don't know. They designed it this way.

37:53And I don't really have, you know,

37:55leverage to get them to redesign their

37:57entire API. I just have to

37:58work with what we have. Right.

37:59This is a lot of application development.

38:01You don't have the liberty to redesign

38:04every API that you're working with. You

38:07have to abstract it in some way. And so

38:09this is what we're having to do here. But

38:10the Effect stream

38:11primitives make it really easy.

38:14That sounds brilliant so far. And I think

38:16what is so nice about this is

38:18that it is A very clear and very

38:22intuitive from a user perspective what

38:25the system should do.

38:27And as users, we're like all familiar

38:30with when this goes wrong and how

38:33frustrating it is. Like if I'm talking to

38:36the AI, the AI goes off like in a wrong

38:39direction that I don't want.

38:41And I want to interrupt it.

38:42And it doesn't act on this interruption.

38:45You need to listen to it for another 20

38:48seconds until I finally need to repeat

38:50what I've just said.

38:52And all of those things, they need to be

38:54handled. And all of that is a lot of

38:56complexity. And if you leave that

38:59complexity unchecked, like you said, it

39:02was a mess. And I think that nicely

39:04reflects the majority of JavaScript

39:07applications that are out there or

39:09TypeScript applications.

39:10And I think it's really like this

39:13fine line to walk where you capture

39:17all of like the existential complexity

39:20that your use case requires and shaving

39:23off like all the accidental complexity

39:26and isolating this nicely.

39:28And even to a degree where you say like,

39:30okay, those services that you need to

39:32call, they're not in the ideal shape that

39:35you would like it to be. But then you can

39:37just like wrap it and create your own

39:40world that you're happy in and where you

39:43can model your application in a nice way.

39:45And yeah, I'm very happy to hear that

39:48effect streams and the various primitives

39:51that you've been using. You've been using

39:53the WebSocket abstraction that Effect

39:56gives you, and I

39:56suppose also queues, etc.

39:59That all of this has been so nicely

40:01fitting together to model your use case.

40:05So going a little bit beyond streams,

40:08which other aspects of Effect have you

40:11been using or which other kind of

40:13superpowers have you been getting out of

40:15Effect that have played a

40:17meaningful role in the application?

40:19Well, the error handling has been huge.

40:22Honestly, we modeled all of our possible

40:25errors. We have, I think, maybe up to 30

40:29or so errors that the system can emit

40:31that are domain specific tagged errors.

40:35And those are decorated with a specific

40:40error code and an error source, because

40:43one of the things that will often happen

40:46or that was happening originally, I

40:49think, was, oh, we got an error. The

40:51connection went down.

40:52Well, we got an error and something weird

40:55happened, and I don't know why, and now

40:57I'm in a weird state. Oh, we got an

40:59error, and the whole service just crashed

41:03or something like this, right?

41:04And even if you can just wrap everything

41:08in a try catch, best case scenario, you

41:11have some unrecoverable error, your

41:13connection goes down, and you don't know

41:16why. You just know, oops,

41:17bye, and then all of a sudden.

41:19And so it's frustrating for the user, and

41:23it's also frustrating for the rest of the

41:24team when they're trying to diagnose

41:26what's going on, which we spent a lot of

41:28time doing in our

41:29development workflow internally.

41:31And, you know, I'm a big fan of passing

41:35the buck, so I don't like things to be my

41:38fault. And I think

41:39we're all in that domain.

41:41I say this to joke, actually, I'm fine

41:44with taking responsibility if it's my

41:46fault, but I would rather things not go

41:48wrong because of the

41:49decisions that I made.

41:50A lot of times early on, it was like,

41:53oh, Cortex is down. Oh, Cortex emitted an

41:55error that I don't understand.

41:57And, you know, fair enough from a

41:58client's perspective or from a test

42:00engineer's perspective,

42:01that's what it seems like.

42:03But that doesn't really give you enough

42:05information to troubleshoot because most

42:08of the time it's not, you know, Cortex is

42:11just a hub. Cortex is just passing events

42:14from one service to another.

42:15It's not Cortex really as the source of

42:18the errors. Instead, what we see a lot of

42:22the time is, oh, one of our

42:23backend services went down.

42:25Oh, a backend service emitted something

42:26that we didn't expect. Oh, it's being

42:29slow or something like this.

42:32And now we're able to like create

42:34targeted pinpoint errors whenever a

42:37specific thing goes wrong

42:38somewhere in the system.

42:39And then those propagate up to our top

42:41level error handling. And so if we have

42:43something that's unrecoverable that

42:44happens, we can now close

42:47the connection like before.

42:48But now we can send up detailed

42:51information that allows our people to

42:54diagnose what's the

42:55problem. So it's not Cortex.

42:57It's like, oh, our speech-to-text service

43:00crashed or is out of memory or something.

43:03And so now we aren't able to create calls

43:07until we fix that

43:08piece of infrastructure.

43:09So that gives us a lot more information

43:11that kind of actually saves a lot of time

43:14debugging the system. It points you

43:16directly to where the source of the

43:18problem is instead of making

43:21you go on a debugging hunt.

43:23So that has been huge for us. The error

43:26handling has been extremely valuable.

43:28There are a lot of other errors that are

43:30recoverable, but we want

43:31it to log and report them.

43:33So the whole error handling story in

43:36Effect is fantastic, just surfacing when

43:40things can go wrong and

43:42forcing you to deal with it.

43:43It has also meant, interestingly, I feel

43:46like that, you know, like within Cortex,

43:50not every function is an effect.

43:51You know, not every single line of code

43:54is a yield star. There's a fair amount of

43:57just plain old, data manipulation

44:00that is happening throughout the code.

44:03It's data manipulation using

44:05functions that are

44:06synchronous and aren't going to throw.

44:11Right. You could have very high

44:12confidence that, you know, if you're

44:14trying to get an index of a thing from a

44:15string, you know, or

44:17whatever, you're not going to throw.

44:19you can do like a

44:20lot of just kind of conventional

44:22programming in areas that

44:23are kind of safe and sandboxed.

44:25And it doesn't mean that every single

44:27calculation needs to be done in an

44:30effect. It just gives you a safe place to

44:33do that kind of work without having to

44:36worry, you know, oh, if this throws or

44:38oh, if this does something asynchronous

44:39and I, you know, don't handle it, you

44:42know, don't await it or whatever.

44:44You know, usually those those kinds of

44:46cases are they get a lot of attention

44:48because we have to

44:50think about them so much.

44:52But that's not usually the most of the

44:55work that we're doing. Ideally, right?

44:57We're thinking about like, purely

44:59functional transformations of data from

45:02one state into another.

45:03taking the input from

45:05some kind of asynchronous effect and

45:07sending it out to

45:08some asynchronous effect.

45:09But like the actual program, the business

45:11logic is usually something that is like

45:13pretty, you know, step by step, you know,

45:16just just logic. Is usually when

45:18we're not interfacing with an external,

45:21you know, service or

45:22some kind of side effect.

45:24Then we can just write code like normal.

45:28You know, we don't have to model

45:29everything as a stream just to add up

45:32some numbers or something.

45:33Right. And I think that the super plain

45:36way how you put it just write code like

45:38normal. I think this is kind of the in a

45:42nutshell, the difference

45:43between Effect and RxJS

45:46Where in RxJS you need to do everything

45:49as a stream. And in Effect, you can write

45:52code like normal. And another aspect of

45:55writing code like normal

45:57is trading errors as values.

46:00This is we're all super used to just

46:04passing around and

46:05manipulating data. And somehow, we're

46:09kind of brainwashed into

46:11thinking that errors need to be like we

46:13need something we need.

46:14We're almost like paralyzed about like,

46:16how should we deal with errors? But if

46:19we're just trading errors as values as

46:22well, errors as data and passing them

46:24around and Effect just makes that easy by

46:26giving us a separate channel

46:28to deal with that error data.

46:30And then, like you say, like you don't

46:33want to you'd like to pass it along, then

46:35it's just data that you pass along. So I

46:38think that's just like code like it's

46:41normal. I think that is like one of

46:43Effect's superpowers.

46:45And closely related to errors is like

46:48having visibility into when errors happen

46:51when something doesn't go as expected. So

46:55and I think if I remember correctly, the

46:57telemetry part the observability part has

47:00also been a key aspect of

47:02building Cortex and operating it.

47:04So maybe can speak a little bit more to

47:07how you do observability and telemetry

47:10usually within MasterClass, particularly

47:14within JavaScript applications and how

47:17Effect has helped you to

47:20maybe improve that even further.

47:22Right. Yeah. So I sort of have to admit

47:25that we don't have an excellent story for

47:27the most part or before Cortex didn't

47:30have an excellent story about

47:31observability at MasterClass

47:33in the JavaScript world.

47:34We have a number of services.

47:36We have we have a Raygun. We have New

47:39Relic and we have Core Logics. We get our

47:42logs to we send our logs to and we have.

47:44So we have a bunch of observability

47:46services for things like video. We have a

47:48dedicated video monitoring service that

47:51we integrate and

47:53a few high value business.

47:55Applications like that. We want to keep

47:57an eye on, you know, error rates for

48:00people visiting our home page or things

48:01that are really

48:02indicate business traffic, business

48:05being impacted by

48:07some technical problem.

48:08However, usually that that amounts

48:11to like something

48:12that's easily reproducible.

48:13And easily fixable usually and there's a

48:16either some infrastructure that needs to

48:19be restarted or code change that needs to

48:21be rolled back or something like that.

48:23Cortex really represents a new level of

48:26complexity when it comes to understanding

48:28what's going on internally. And I think

48:29that a big reason for that is that it is

48:32not a one shot HTTP server type of

48:35application, but is instead

48:38You know, a stream of streams and is

48:41handling all of these asynchronous events

48:43that are passing through it. It's not

48:45directly doing much of any work. It's

48:47just literally in between all these

48:49services handing events

48:51from one service to another.

48:52So, as I mentioned before, when

48:54things go wrong, they're mostly not going

48:55wrong in Cortex. And likewise with

48:57observability

48:59where the system is spending time

49:01doing work, it's mostly not spending time

49:04inside of Cortex doing that work.

49:07Cortex is instead, waiting for

49:10events from other services. And so what

49:12we're interested in measuring is not

49:14really the conventional, I think,

49:16telemetry story when it comes

49:17to building tracing and spans.

49:20I think this is a story that is also

49:22baked into the default Effect telemetry

49:26story, right? When you have a Effect dot

49:29with span with a name, you know, and you

49:32and you and you wrap

49:33that around an Effect.

49:35Well, that's great. That ensures that

49:37that span or that Effect, the time that

49:40it executes is going to be represented by

49:43a span in your trace. And for most like

49:46one shot type of actions that you might

49:48perform, that works great,

49:51which is most of the time.

49:52If you're doing actual work within an

49:55effect, within a one shot effect, then

49:57that is the conventional way that you do

50:00telemetry. We're not really doing that at

50:02all in our telemetry

50:04implementation in Cortex.

50:05Instead, we're interested in measuring

50:07time between events that are coming from

50:10outside services. Cortex is the only

50:12place where we can really gather this

50:13information because it's the hub.

50:15But Cortex isn't sitting around. We don't

50:19have like an effect that is, you know,

50:21sleeping until it gets a certain

50:23notification from or an event from

50:25another stream.

50:26It wouldn't make any sense

50:27to build the application that way.

50:28And so it doesn't really make a lot of

50:30sense to build our telemetry that way.

50:33I suppose what we're doing with

50:35Open telemetry is a little unconventional

50:36in that a span doesn't represent Cortex

50:40doing that work. Instead, it represents

50:42usually really represents like the LLM is

50:44working or the text-to-speech

50:46service is working.

50:48And we're just waiting for that. But it's

50:49measured from Cortex, not from these

50:52other services. But because

50:53it's really all streams.

50:54What we have to go on isn't an effect

50:57that we're measuring. It is literally the

51:00time between two events in those streams

51:03that we're measuring. And so we really

51:05had to roll a lot of our

51:07own telemetry handling.

51:10But, you know, we were going to have to

51:11do this anyway,

51:12ultimately, because when you have.

51:14Let's say we're not using Effect. We're

51:16using the original approach, the

51:19non-Effect approach that is event

51:21emitters everywhere, you know, WebSocket

51:23event handlers and so forth.

51:25You get a transcription from the speech

51:28to text service and you want to start the

51:31span of time that it takes that you're

51:34measuring that it takes to.

51:36Generate a response

51:37to that transcription.

51:39Well, you can have a central hub where

51:42like maybe you're

51:42collecting all of these events.

51:44But you're starting that span in one

51:48event handler and then you're entering

51:49you're ending it in a different event

51:51handler for a different service.

51:53And so you need a place where you're

51:55holding on to those references might be a

51:57centralized location that is listening to

51:59all of these events.

52:00But then it becomes kind of tangled up

52:02because you're having to.

52:04Keep these references around and

52:06keep them alive, from one

52:08event handler to a

52:10completely different event handler.

52:11And this is an area where.

52:14Yeah, we had to roll some of our own code

52:15in Effect do it this way.

52:17But I feel like Effect kind of made it

52:19easier to do it this

52:20way anyway.

52:21Allowing us to have a kind of a

52:23connection level, long lived reference to

52:27these spans and then just manipulate

52:30spans in what is essentially a stream dot

52:34tap where we are listening to all of the

52:37incoming events and then just

52:38Starting and stopping them based on

52:40which events are occurring.

52:42It's not been perfect, honestly.

52:45It has been a little error prone

52:46sometimes and we've had to go in and kind

52:49of tweak things when we

52:53have unexpected behaviors.

52:55It's an area that has provided immense

52:57value for us, however.

52:59It's given us a lot of insight into what

53:01people are experiencing

53:02if people are experiencing.

53:04Really slow behaviors, slow responses.

53:08If people are experiencing the bot is

53:10talking over me or this sort of thing.

53:12If we have errors somewhere in the

53:14system, we can see exactly where and when

53:16that happened and in what service and in

53:18what part of that

53:19services work it happened.

53:20And we're able to trace, you know, what

53:24was the sequence of of chunks

53:26that were emitted by the LLM.

53:28You know, how long did it take

53:29for us to get that first

53:30chunk out of it out of the LLM.

53:32You know, comparing the user message in

53:35the bot response and you know, if the

53:37user interrupted the bot, how much of the

53:39bot speech did the user here?

53:42And so a lot of these questions that are

53:44really of interest to the business and

53:46also for us technically.

53:48When it comes to how on call is being

53:51used and how people are experiencing it

53:54are really answered by the telemetry that

53:57we've built using

53:58Opentelemetry and Effect.

54:00But,it's a very

54:02custom system that I I don't know that it

54:05has its optimal form yet.

54:08And I also don't know that is necessarily

54:10going to apply to

54:12everybody in the same way.

54:14I don't know. This is like I said, it's

54:15it's it's a very custom system that is

54:18built for our use case that will not

54:21apply in most conventional applications.

54:24But I think that's OK.

54:25There's always special cases.

54:28This makes sense. And, where

54:30Opentelemetry or the previous

54:32technologies that is based on Open

54:35Sensors and Open Tracing, I believe those

54:40were the two predecessor technologies

54:42that merged into Opentelemetry.

54:44Where they're historically coming from is

54:46like from distributed tracing. And that

54:49is typically in a

54:50microservice kind of architecture.

54:52We have one service request response

54:54style calling into another

54:56one, calling into another one.

54:58So I think where those systems or this

55:00technology shines historically, at least,

55:02is on a request response pattern where

55:06you just get that burned on charge that

55:08that you know from like a

55:09performance profiler in a single threaded

55:12environment or a multi-threaded

55:13environment, now we get it

55:14over like a network boundary.

55:16So this is where those shine. But going

55:19beyond that for different modalities,

55:21like for long running streams or I've

55:24been also experimenting with using

55:26OpenTelemetry in a front-end setting

55:28where like a front-end session, you don't

55:31have that request response, but you have

55:33a front-end session.

55:34For example, think about how you use

55:36social media. You might be doomscrolling

55:39for a very long time. So is your entire

55:42session, is that the trace that you have

55:45with possibly like thousands of spans?

55:49Or where do you make cut basically? How

55:51do you design your trace and your spans?

55:54I think that is still something that

55:55we're figuring out as an industry.

55:57And it's been cool to hear about your

56:01usage of it. And I think this also speaks

56:03to the flexibility of Effect. Yes,

56:06they're like default patterns make it

56:09really easy and kind of trivial to

56:11instrument your app out of the box.

56:13But if you want to instrument it in a way

56:15that's a little bit more specific to how

56:18you would like to give your trace

56:21meaning, that's possible as well.

56:23Maybe taking a step back here for folks

56:26who are curious about Opentelemetry and

56:29like generally see the value in

56:31observability, but maybe haven't taken

56:33that leap yet themselves instrumenting

56:36their app, which sort of guidance would

56:38you offer to people what to focus on,

56:41maybe what to initially leave out of the

56:44picture just to get going?

56:46Oh, that's a really good question. I feel

56:47like the answers are going to be really

56:49specific to your use case. And in the

56:53case of an application like Cortex,

56:55extremely custom. And we have spent a lot

56:58of time refining and iterating on our

57:02telemetry implementation. But most

57:04applications probably

57:05don't need that, honestly.

57:07I think, especially in the

57:09JavaScript world, there's both browser

57:11and Node based auto instrumentations that

57:15are available that do a lot out of the

57:17box. So I feel like a lot of what I would

57:21want to start with are the ends of the

57:23application when your code is calling out

57:26to another service or when you receive

57:28events from the user.

57:30Because that kind of determines the shape

57:32of that user session or that interaction

57:34that you might be

57:35interested in measuring.

57:37And then it will identify kind of

57:39hotspots of like, oh, the user waited a

57:42long time for this response or whatever,

57:44what's up with that? And then you can

57:46start to drill further.

57:48And then the other thing that I think is

57:49really super valuable is distributed

57:53tracing where you are propagating a trace

57:56across service boundaries. And sometimes

57:58this is just, you know, you're

58:00instrumenting in your browser application

58:02or in your iOS application.

58:04you're making calls out to your API service and

58:07you want to see what's going on in the

58:09API service as well during that time

58:11period. You're propagating the trace from

58:14your client to your server so that when

58:17you see them all together,

58:20you can kind of piece together.

58:22Oh, the client called out to the server

58:24and then the server made these database

58:25calls and you can see that all in one

58:27distributed trace. That's really super

58:29valuable. So just focusing on the ends,

58:33either incoming events or outgoing

58:36service calls, and then making sure you

58:39have your distributed trace propagation

58:41set up correctly. Those would be the top

58:43things I would recommend.

58:45Right. I agree. And I think the benefits

58:48of having a good observability story

58:50for your application and for your system

58:52is so manifold. Like it helps you with

58:56correctness to kind of understand like,

58:59oh, like something is not going well.

59:01That you're not just like

59:03completely in the dark and looking for

59:05the needle in the haystack, but that you

59:07actually have a great foundation to

59:10figure out what went wrong.

59:12That is the, I think the foundation where

59:14people start leaning on observability

59:17beyond just console log. But then also

59:21like doesn't just help you with making

59:22things correct or diagnosing when

59:26something was not correct,

59:27but also making it faster.

59:29Like otherwise you might just know like,

59:31okay, that API request has taken two

59:33seconds, but why? And sometimes there's

59:36like really counterintuitive situations.

59:38It's very simple and very

59:40easy to realize, oh,

59:42this is why it's so slow.

59:44And also speaking of AI, this will like

59:49be a perfect foundation to give an AI

59:52even more context what is going wrong and

59:55let the AI iterate further and help you

59:57make your app more

59:59reliable and more performant.

01:00:00One of the frontiers that we're going to

01:00:03be exploring, I think now that we've

01:00:05cracked the seal on observability is

01:00:08integrating it with our existing data

01:00:10analytics or our end user analytics that

01:00:14we are already collecting.

01:00:15In MasterClass we're a really good, robust data

01:00:17team that is, you know, while respecting

01:00:21like anonymity and user privacy is still

01:00:24really invested in

01:00:25understanding the user journey.

01:00:28What are people doing? You know, why are

01:00:30they there? What is enticing

01:00:31them? What's driving them away? And these

01:00:33sorts of questions that are really

01:00:35foundational to understanding the best

01:00:39way that we can

01:00:39deliver value to our users.

01:00:41Integrating this with a sort of Opentelemetry

01:00:43will give us even more insights

01:00:46of like, oh, did a user try to

01:00:49load a page, but then they bounced

01:00:52because it took too long to load and

01:00:54things like this that will give us an

01:00:57integration level between the sort of end user metrics

01:01:01that we've been using and also the technical

01:01:04implementations behind the scenes that

01:01:05are underpinning that user experience.

01:01:09I'm really looking forward to being able

01:01:11to explore that further. And I feel like

01:01:14it has tremendous

01:01:15potential to offer value.

01:01:17That sounds awesome. So speaking of a bit

01:01:20more forward looking perspectives

01:01:22I'm curious now that you've been part of

01:01:25the Effect community, I think at this

01:01:27point way beyond a year already and super

01:01:30impressive what you've been able to build

01:01:33with Effect in that short period of time.

01:01:35What are the things that you're looking

01:01:37forward most to when it

01:01:39comes to the Effect ecosystem?

01:01:42I'm really looking forward to seeing

01:01:43Effect 4.0 come out. That looks awesome.

01:01:47A lot of the improvements they've made to

01:01:49the bundle size and to the implementation

01:01:53complexity look really promising.

01:01:56And, you know, I really admire how

01:01:59responsive the Effect team has been to

01:02:03the community, to the needs of the

01:02:05community and really listening to

01:02:07feedback, incorporating it, iterating.

01:02:10That's really, I think, been vital to any

01:02:13platform like this, getting any traction.

01:02:16It's a very ambitious platform. It just

01:02:20has to be said, the design of it to

01:02:23encapsulate so much information about

01:02:26what's going on at an atomic level in

01:02:28your application, but then extending that

01:02:30out into really

01:02:31building whole frameworks.

01:02:34The HTTP API, the Schema, the tracing

01:02:38integration, the possibility that the

01:02:42database abstractions, the networking

01:02:45abstractions like WebSockets, file

01:02:47system, node runtime,

01:02:50there's React integrations.

01:02:52Really, you name it, there's just tons

01:02:55and tons of, and there's a whole bunch of

01:02:56stuff coming in down the pipeline,

01:02:58Cluster and AI integrations.

01:03:01The paradigm is really extensible and it

01:03:05has proven itself really robust and able

01:03:08to handle all these different scenarios.

01:03:10But now making all of those things all

01:03:13work, you know, that's

01:03:15an incredible challenge.

01:03:16I hope something that they

01:03:19succeed at. It's just so much.

01:03:22But I think that with this large and

01:03:25growing community, I think there will be

01:03:27people using every aspect of that.

01:03:30Probably not everybody is going to use

01:03:33every piece of the Effect

01:03:34ecosystem. I am not, certainly.

01:03:37And most people will only use a small

01:03:39sliver of it. And that's fine. That's all

01:03:41you really need to do.

01:03:43But when you have hundreds or thousands

01:03:45of engineers all building different kinds

01:03:48of applications with it, now you start to

01:03:50get a lot more signal on

01:03:52for this specific use case.

01:03:54Here's what I'm trying to do.

01:03:56Here's how I've approached it.

01:03:57And that information, feeding that

01:03:59information into the design of the

01:04:01platform is going to be tremendously

01:04:03valuable to making it more extensible.

01:04:07But anyway, I'm really interested in the

01:04:09Effect 4.0 developments that I think have

01:04:12come out of this exact kind of feedback

01:04:15integration and iteration.

01:04:17And also, I'm really excited about things

01:04:20like the, I think there was at the Effect

01:04:23Days, there was some, there was a demo

01:04:24that I think Mattia did.

01:04:26Mattia, who did about the error reporting

01:04:30and that was integrated into the editor

01:04:32when there's like type errors in Effect.

01:04:35Sometimes those can be cryptic and it's

01:04:38nice to see that they're working on

01:04:40making those a lot more

01:04:41human readable and friendly.

01:04:45It's nice to see the dev tools getting so

01:04:47much love. It's nice to see the

01:04:50documentation getting so much

01:04:51improvement, so often very thankless.

01:04:53Honestly, more than any specific

01:04:56technical feature of the API, I just love

01:05:00to see the developer

01:05:01experience getting some attention.

01:05:03And it makes things so much easier on

01:05:05everybody to start building really cool,

01:05:08ambitious things if you get really good,

01:05:13clear, understandable errors at build

01:05:16time or in your editor.

01:05:18And if you have really good debugging

01:05:21tools that let you understand what's

01:05:23happening, if you have really good

01:05:24documentation that has use cases and

01:05:26examples of all kinds of different types

01:05:30of patterns that you might

01:05:31want to take advantage of.

01:05:33This is the sort of, I think often unsexy

01:05:36work of building a framework like this

01:05:38that is so fundamental to people, like

01:05:41end users being able

01:05:42to get value out of it.

01:05:43And it seems like the Effect team is

01:05:45taking this seriously. And that to me,

01:05:48more than almost anything, is what gives

01:05:50me excitement and hope about

01:05:53the future of this framework.

01:05:54100%. I mean, I couldn't agree more.

01:05:57They're all super ambitious efforts on

01:06:01their own. And luckily, we have a small

01:06:05but super talented and absolutely

01:06:08brilliant core team of folks who are

01:06:11working on the various pieces.

01:06:13You've been mentioning the docs. We have

01:06:15Giulio working more or less full time on

01:06:18the docs as well as on the Effect Schema,

01:06:21which we could probably also fill an

01:06:23entire show on just talking about the

01:06:25Effect Schema, which I

01:06:26think you're also using.

01:06:27Oh, yeah. We're using Effect Schema all

01:06:29over the place. I didn't

01:06:30even talk about that. Yeah.

01:06:31Yeah. But then also the the dev tools.

01:06:35I'm super excited that

01:06:37Mattia will be joining.

01:06:39Well, at the time of this recording, a

01:06:42couple of days from now, but when the

01:06:43time comes and the recording airs, I

01:06:46think Mattia will have already started

01:06:48working full time on the dev tools and

01:06:50other pieces and improving

01:06:52the developer experience.

01:06:54And then at the core of all of it, Effect

01:06:564.0 and what it will enable. That's

01:06:59really been the distillation

01:07:01of all the feedback that we've gathered

01:07:03over the last couple of years.

01:07:04And I think really tackle some of the

01:07:07most common points of feedback or

01:07:09frustration head on. And I think it's

01:07:12really like a huge improvement over

01:07:15what's already really great and useful.

01:07:19So I'm fully on board with with

01:07:22everything you've you've shared. And

01:07:25yeah, I think I can't thank you enough

01:07:28for sharing all that what you've been

01:07:31building with Cortex at MasterClass.

01:07:34It's really, really impressive.

01:07:36What you've been able to build in such a

01:07:38short period of time speaks to

01:07:40experience, but also speaks to just like

01:07:43how capable the building blocks are. And

01:07:45like when like experience and ambition

01:07:48comes together with great materials.

01:07:50I think this is how we're going to get

01:07:51great experiences and great applications.

01:07:54So thank you so much for doing that work

01:07:57and sharing it.

01:07:58And thank you so much for coming on the show today.

01:08:01Well, thank you so much for having me.

01:08:03And I just you know, I have to say that

01:08:05so far, I think the on call product has

01:08:08been a huge success and Cortex has been

01:08:12very, very crucial part of that.

01:08:16And I feel like it would not have been

01:08:18possible without the Effect ecosystem and

01:08:21also the support of the Effect core team

01:08:23and the community to helping us get

01:08:27there. I think that has played a fundamental

01:08:29role. So thank you. I mean, to you,

01:08:32Johannes, and thank you to the broader

01:08:34Effect Team and the community for all of

01:08:37the support and assistance and enthusiasm

01:08:39and building this incredible framework.

01:08:40It has really been a game changer.

01:08:43That is awesome to

01:08:44hear. Thank you so much.

01:08:46Thank you.

01:08:48Thank you for listening to the

01:08:49Cause & Effect Podcast.

01:08:51If you've enjoyed this episode, please

01:08:53subscribe, leave a review

01:08:54and share it with your friends.

01:08:56If you haven't done so already, you can

01:08:58join our Discord community.

01:09:00And if you have any questions, feedback

01:09:02or suggestions about this episode or

01:09:04about Effect in general,

01:09:06don't hesitate to get in touch.

01:09:08See you in the next episode.