Cause & Effect Podcast

Play

All Episodes

Mar 7, 2025

#2: Scaling AI for Customer Support at Markprompt with Effect

Join us as we talk with Michael Fester from Markprompt about scaling AI-powered customer support with Effect, building reliable and high-performance infrastructure, and enhancing developer productivity in a fast-evolving AI landscape.

Episode Notes

Join us as we talk with Michael Fester from Markprompt about scaling AI-powered customer support with Effect, building reliable and high-performance infrastructure, and enhancing developer productivity in a fast-evolving AI landscape.

Effect is an ecosystem of tools to build production-grade software in TypeScript.

• Website & docs
• Community & support
• Follow us on X (Twitter)
• GitHub

Song: Dosi & Aisake - Cruising [NCS Release]
Music provided by NoCopyrightSounds
Free Download/Stream: http://ncs.io/Cruising
Watch: http://ncs.lnk.to/CruisingAT/youtube

• (00:00) - Welcome & guest introduction
• (01:54) - Michael’s journey: from academia to AI & Customer Support
• (03:49) - What is Markprompt? Overview & use cases
• (07:45) - Markprompt’s system architecture
• (10:22) - Challenges of running AI-powered support systems
• (13:20) - Improving reliability with Effect
• (16:41) - Technical architecture breakdown
• (19:51) - The public API server setup
• (23:50) - Ingestion engine
• (26:29) - Onboarding engineers to Effect
• (30:51) - Migrating the codebase to Effect
• (35:19) - Effect in production: the power of schema
• (39:02) - Migrating to Effect: challenges & key takeaways
• (41:45) - Effect brings out the best in us engineers
• (45:34) - The Future of AI infrastructure
• (50:18) - Closing remarks & thanks

Discuss this episode on Discord

Transcript

• 01:00:00
  As an engineer, you want to build
• 01:00:01
  features, you want to
• 01:00:01
  build like real robust systems.
• 01:00:04
  You want to spend ideally
• 01:00:05
  all your time doing that.
• 01:00:06
  You don't want to spend your time
• 01:00:08
  fighting the language,
• 01:00:09
  building something that, you
• 01:00:10
  know, the language was not meant for.
• 01:00:12
  To give you an example,
• 01:00:13
  like concurrency in TypeScript.
• 01:00:14
  You want a way to build a concurrent
• 01:00:17
  system where you're
• 01:00:18
  thinking about what is
• 01:00:19
  the system actually doing rather than
• 01:00:20
  like setting up all the
• 01:00:22
  boilerplate in order to
• 01:00:23
  have that system in place.
• 01:00:25
  So I would say Effect really
• 01:00:27
  gives you the right premise.
• 01:00:28
  So now that we've completed this
• 01:00:30
  migration, kind of feel
• 01:00:31
  like this is the end game feels
• 01:00:33
  like more stable and at least if we
• 01:00:35
  continue building our
• 01:00:36
  code base on TypeScript with
• 01:00:38
  Effect, it doesn't feel like we're going
• 01:00:39
  to spend many more like
• 01:00:41
  cycles, like refactoring
• 01:00:43
  the code.
• 01:00:47
  Welcome to Cause & Effect, a podcast
• 01:00:49
  about the TypeScript
• 01:00:50
  library and ecosystem called
• 01:00:51
  Effect, helping engineers to build
• 01:00:54
  production-ready software.
• 01:00:56
  I'm your host, Johannes Schickling, and I've
• 01:00:58
  been building with
• 01:00:58
  Effect for over four years.
• 01:01:00
  With this podcast, I want to help others
• 01:01:02
  understand the powers and
• 01:01:04
  benefits of using Effect.
• 01:01:06
  In this episode, I'm talking to Michael
• 01:01:08
  Fester, co-founder of
• 01:01:10
  Markprompt, an AI-based customer
• 01:01:12
  support platform.
• 01:01:13
  In this conversation, we explore how
• 01:01:16
  Markprompt uses Effect
• 01:01:17
  across its entire architecture,
• 01:01:20
  including API and RPC endpoints,
• 01:01:22
  long-running workflows, and their
• 01:01:24
  customer-facing product.
• 01:01:25
  Let's get into it.
• 01:01:28
  Hey, welcome Michael.
• 01:01:29
  So nice to have you on the
• 01:01:30
  Cause & Effect Podcast.
• 01:01:31
  How are you doing?
• 01:01:32
  I'm good.
• 01:01:33
  Thank you.
• 01:01:33
  I'm really happy to be here.
• 01:01:35
  It's been, I guess it's been a while that
• 01:01:36
  we're trying to
• 01:01:37
  organize this, but I'm very
• 01:01:39
  happy that we're doing it now.
• 01:01:41
  Yeah.
• 01:01:41
  I mean, you're super, super busy.
• 01:01:44
  You're based in San
• 01:01:45
  Francisco building an AI company.
• 01:01:48
  So I'm really, really happy that you took
• 01:01:50
  the time to share
• 01:01:51
  your story about Effect.
• 01:01:53
  Maybe you can share a little bit about
• 01:01:56
  yourself, your
• 01:01:56
  background and what are you doing?
• 01:01:59
  Yeah, sure.
• 01:02:00
  So I have a background.
• 01:02:02
  I started in academia.
• 01:02:03
  Actually, I was doing
• 01:02:04
  research in mathematics.
• 01:02:05
  Then I started getting into AI and I
• 01:02:07
  built my first AI
• 01:02:08
  company in Europe, specialized
• 01:02:10
  on device voice recognition.
• 01:02:12
  And we got acquired by Sonos.
• 01:02:14
  And since then I moved here to San
• 01:02:17
  Francisco for my next
• 01:02:19
  venture, which is we're building
• 01:02:22
  AI infrastructure for customer support.
• 01:02:25
  So this is all like posts,
• 01:02:27
  chat GPT, GPT 4 coming out.
• 01:02:30
  And we saw this, this incredible
• 01:02:31
  opportunity to start
• 01:02:32
  working in this space.
• 01:02:35
  That is awesome.
• 01:02:36
  And I highly encourage everyone who is
• 01:02:40
  like interested in AI,
• 01:02:41
  particularly is looking
• 01:02:43
  for a better customer support solution to
• 01:02:46
  check out Markprompt.
• 01:02:48
  Can you share about a few companies who
• 01:02:50
  are already using Markprompt?
• 01:02:51
  Yeah, sure.
• 01:02:52
  So we've been working very closely with
• 01:02:55
  some companies very
• 01:02:56
  early on in the process.
• 01:02:57
  Actually, that's really interesting.
• 01:02:58
  Vercel started using us as one of our
• 01:03:00
  first customers,
• 01:03:01
  actually ElevenLabs, Replicate.
• 01:03:03
  Things are just moving so fast.
• 01:03:05
  So our approach here is not so much of a,
• 01:03:08
  you know, we have a
• 01:03:09
  vision of how things will
• 01:03:10
  end up, right?
• 01:03:11
  It's very much like a learning process,
• 01:03:13
  working very, very
• 01:03:14
  closely with customers, hand in
• 01:03:16
  hand getting their feedback,
• 01:03:17
  understanding together, how
• 01:03:19
  this thing is evolving, how
• 01:03:21
  we can improve, you know, the behavior of
• 01:03:23
  the LLMs or how we can
• 01:03:24
  extend the use cases.
• 01:03:26
  So this has been a very different way, I
• 01:03:28
  would say, of building
• 01:03:29
  a company from the past,
• 01:03:30
  because it's so new to
• 01:03:32
  everyone in the space.
• 01:03:33
  And we're just like scratching the
• 01:03:35
  surface of the
• 01:03:35
  opportunities of building these systems
• 01:03:38
  for you know, down the line.
• 01:03:40
  That's awesome.
• 01:03:41
  So before we go into the technical parts,
• 01:03:44
  maybe you can share
• 01:03:45
  through the lens of one
• 01:03:47
  of your typical customers, what
• 01:03:49
  Markpompt does and which
• 01:03:51
  problems it solves for some
• 01:03:53
  of those customers.
• 01:03:54
  Sure.
• 01:03:55
  So there's multiple facets of this
• 01:03:57
  running a support
• 01:03:58
  operation is it's not a one size
• 01:04:01
  fits all.
• 01:04:02
  Every company has their own ways of, you
• 01:04:04
  know, the nature of
• 01:04:05
  customer issues, the way that
• 01:04:07
  they engage with customers,
• 01:04:09
  is it like direct contact?
• 01:04:11
  Is it, you know, is it mostly self serve?
• 01:04:14
  Technical challenges are very different
• 01:04:16
  from one company to the other.
• 01:04:18
  So we offer really an
• 01:04:20
  end to end solution.
• 01:04:21
  So there's multiple
• 01:04:23
  components in our system.
• 01:04:24
  It all starts with understanding your
• 01:04:26
  system of record, which is
• 01:04:28
  things like your underlying
• 01:04:30
  customer knowledge, your, you know, your
• 01:04:33
  documentation, your CRM, the API is that
• 01:04:37
  you use for automating
• 01:04:39
  parts of your support.
• 01:04:41
  And we connect all that together.
• 01:04:43
  And then we build applications that are
• 01:04:45
  either customer facing
• 01:04:46
  or agent facing so and use
• 01:04:49
  internally for the company.
• 01:04:51
  So it can be, you know, everything from,
• 01:04:52
  you know, your contact form powered by AI
• 01:04:55
  to your chatbot to your
• 01:04:58
  Slack and Discord bots, right?
• 01:05:00
  This is customer
• 01:05:01
  facing email order replies.
• 01:05:04
  So you send an email and the AI agent
• 01:05:08
  will start, you know,
• 01:05:09
  trying to resolve the issue
• 01:05:11
  for you.
• 01:05:12
  And it might be possible
• 01:05:13
  that it resolves it fully.
• 01:05:16
  And if not, it's going to ask you for all
• 01:05:17
  the questions that the team then needs to
• 01:05:19
  use once you know, the ticket
• 01:05:21
  hits the inbox internally.
• 01:05:23
  And then there's the all the features
• 01:05:25
  which are internal
• 01:05:26
  facing for the support
• 01:05:28
  team to be able to streamline their work.
• 01:05:31
  Because the truth is that customer
• 01:05:32
  support before AI involved
• 01:05:36
  a lot of essentially like
• 01:05:38
  robotic work, just answering the same
• 01:05:41
  questions over and over again.
• 01:05:42
  This is not super engaging
• 01:05:44
  and exciting just to have
• 01:05:45
  to answer the same
• 01:05:46
  question over and over again.
• 01:05:47
  So if you can start like taking that
• 01:05:49
  burden off of your
• 01:05:50
  shoulders and help you, you know,
• 01:05:52
  work on more like high leverage strategic
• 01:05:56
  questions, then, you
• 01:05:58
  know, we can really help
• 01:05:59
  streamline the support operation.
• 01:06:00
  what happens when you've put
• 01:06:01
  an AI or both like customer
• 01:06:03
  facing and agent facing, then you can
• 01:06:06
  start, you know,
• 01:06:07
  extracting a lot of intelligence,
• 01:06:09
  a lot of knowledge, because you're really
• 01:06:10
  at the nerve center
• 01:06:11
  in a company, you are,
• 01:06:13
  you know, the interface between your
• 01:06:15
  customers and your company, right?
• 01:06:18
  So you have so much like knowledge that
• 01:06:20
  often just gets siloed
• 01:06:21
  within the support team,
• 01:06:23
  right?
• 01:06:23
  It's it's actually very rare that support
• 01:06:25
  team reports to product, right?
• 01:06:28
  And I think that's a
• 01:06:29
  that's really a problem.
• 01:06:30
  But if we can start rooting
• 01:06:32
  out all this insight and
• 01:06:33
  knowledge, we can start
• 01:06:35
  having a really, really good
• 01:06:36
  understanding on, you know,
• 01:06:37
  product shortcomings, where
• 01:06:39
  the engineering team might,
• 01:06:40
  you know, put some effort, right?
• 01:06:43
  Because this is where your customers have
• 01:06:44
  been experiencing a
• 01:06:46
  lot of friction, right?
• 01:06:47
  So we can start generating these
• 01:06:49
  reporting and all sorts of
• 01:06:51
  other insights that can help
• 01:06:53
  you as a company
• 01:06:54
  streamline your operations,
• 01:06:56
  things like spotting knowledge
• 01:06:57
  gaps, right, helping you write, you know,
• 01:06:59
  keeping your your
• 01:07:00
  documentation up to date,
• 01:07:02
  these are all things that
• 01:07:02
  we can start also working on.
• 01:07:05
  So this concept of
• 01:07:05
  agent is very versatile.
• 01:07:08
  The agents can solve issues, they can,
• 01:07:10
  you know, answer emails,
• 01:07:11
  but they can also start
• 01:07:12
  being tasked of gathering
• 01:07:14
  insights and producing new
• 01:07:15
  knowledge and keeping your
• 01:07:16
  system of record,
• 01:07:17
  always up to date.
• 01:07:18
  So there's really a lot of
• 01:07:19
  things that need to happen.
• 01:07:20
  You cannot approach the
• 01:07:22
  problem just from one
• 01:07:23
  angle, you really want to tie
• 01:07:24
  the whole thing together, because this is
• 01:07:26
  where you start creating
• 01:07:27
  these positive feedback
• 01:07:28
  loops, so called data flywheels, right,
• 01:07:31
  that enable you to
• 01:07:32
  start deploying the solution
• 01:07:33
  and trust it that it's
• 01:07:34
  operating really well.
• 01:07:36
  So yeah, there's,
• 01:07:37
  there's a lot of things to it.
• 01:07:38
  That sounds fascinating, and also quite
• 01:07:41
  complex in terms of the
• 01:07:42
  different surface areas, etc.
• 01:07:45
  So maybe we can go
• 01:07:46
  slightly more technical.
• 01:07:47
  And before we go into all of the Effect's
• 01:07:49
  specifics, can you more from like an
• 01:07:52
  engineering perspective
• 01:07:53
  provide an overview over your system?
• 01:07:56
  And which different high level components
• 01:07:59
  it's composed out of?
• 01:08:01
  Sure.
• 01:08:02
  So this is a, you know, apart from the
• 01:08:04
  machine learning things,
• 01:08:06
  the LLM things, this is a
• 01:08:08
  classical, you know, setup.
• 01:08:09
  there's some servers, both
• 01:08:11
  like short, like serverless instances for
• 01:08:13
  API endpoints, then
• 01:08:14
  there's some long running
• 01:08:16
  servers to do things like, you know, pre
• 01:08:18
  processing data, post
• 01:08:21
  processing, interactions.
• 01:08:23
  So yeah, there's a there's a bunch of
• 01:08:25
  servers here, everything
• 01:08:25
  is in node, everything is
• 01:08:27
  or TypeScript.
• 01:08:29
  The front end is a React application.
• 01:08:32
  So both we have a dashboard, which is a V
• 01:08:35
  duplication, there is
• 01:08:37
  some integrations
• 01:08:38
  inside of Salesforce,
• 01:08:39
  Zendesk, Slack, and so on.
• 01:08:41
  So it's a classical full
• 01:08:43
  stack, React TypeScript application.
• 01:08:47
  So you've built quite an expansive system
• 01:08:49
  here and building all
• 01:08:51
  of that in TypeScript
• 01:08:52
  I think TypeScript is sort of notorious
• 01:08:54
  for only scratching the
• 01:08:55
  surface and only covering
• 01:08:57
  the happy path.
• 01:08:58
  There's a lot of things that can go wrong
• 01:09:00
  in all of those systems, integrating with
• 01:09:03
  different systems.
• 01:09:04
  All of those things I think have led you
• 01:09:06
  to take a more serious
• 01:09:08
  look at Effect and ultimately
• 01:09:10
  adopting it.
• 01:09:11
  So can you share that journey of going
• 01:09:14
  from a non-Effect code
• 01:09:16
  base to converting step
• 01:09:18
  by step to Effect?
• 01:09:20
  Sure, so obviously there's the common
• 01:09:22
  complexity increase that
• 01:09:24
  happens when you're building
• 01:09:25
  a product and putting it into production.
• 01:09:27
  So as you add features and so on, then
• 01:09:29
  you hit sort of a ceiling
• 01:09:31
  and then you start assessing
• 01:09:33
  the situation and then you might start
• 01:09:35
  refactoring and so on.
• 01:09:36
  And then next phase, you hit some other
• 01:09:39
  limitations and then you
• 01:09:41
  reassess again the situation.
• 01:09:44
  So that's just the common thing of
• 01:09:45
  working on an
• 01:09:46
  increasingly growing code base.
• 01:09:49
  But then there's LLMs and these are
• 01:09:51
  non-deterministic systems and it poses
• 01:09:54
  some completely new kinds
• 01:09:56
  of challenges.
• 01:09:56
  How do you monitor the behavior of the
• 01:09:58
  system, for instance?
• 01:10:00
  How do you deal with readable streams
• 01:10:02
  instead of just sending over a payload?
• 01:10:05
  This has the potential to making your
• 01:10:07
  code base more complex
• 01:10:08
  if you need to stream some
• 01:10:10
  tokens down to the customer, but you
• 01:10:12
  actually also need to
• 01:10:13
  store these chunks on your own
• 01:10:16
  for post-processing.
• 01:10:17
  How do you deal with that?
• 01:10:18
  TypeScript doesn't give you
• 01:10:19
  some great answers to this.
• 01:10:22
  How do you deal with reliability?
• 01:10:24
  OpenAI and Anthropic and so on, they offer
• 01:10:27
  a service which is in high demand.
• 01:10:31
  We are in a period right
• 01:10:33
  now with scarce resources.
• 01:10:35
  So these APIs, they go down and they
• 01:10:37
  actually go down more
• 01:10:38
  than you would think.
• 01:10:40
  And so how do you build fallback
• 01:10:42
  mechanisms so that, say that
• 01:10:44
  OpenAI is down as your main
• 01:10:46
  provider, your customers cannot be down.
• 01:10:48
  If OpenAI is down,
• 01:10:49
  that's not their problem.
• 01:10:50
  That's our problem.
• 01:10:52
  So how do you build a
• 01:10:53
  fallback system to Anthropic?
• 01:10:55
  What happens a few weeks ago when OpenAI
• 01:10:58
  was down for, I think,
• 01:11:00
  five hours and then the
• 01:11:01
  entire industry shifts towards Anthropic
• 01:11:04
  that just has this
• 01:11:05
  spike that they're unable
• 01:11:07
  to deal with?
• 01:11:08
  So they are experiencing outage or just
• 01:11:11
  very degraded performance.
• 01:11:13
  There's outage and then there's actually
• 01:11:15
  performance, the
• 01:11:16
  speed of tokens and so on.
• 01:11:19
  All that, there's no good answer, no
• 01:11:22
  clear best practice in
• 01:11:25
  vanilla TypeScript to deal
• 01:11:27
  with these things.
• 01:11:28
  And then there's the fact that we are a
• 01:11:29
  startup working in a
• 01:11:31
  fast-paced environment where
• 01:11:33
  things change all the time.
• 01:11:36
  And we want to be efficient and we want
• 01:11:37
  to be able to not deal
• 01:11:39
  with all the complexity
• 01:11:41
  and all the problems and down times and
• 01:11:42
  reliability, but we need to
• 01:11:44
  be effective in shipping new
• 01:11:46
  features to constantly
• 01:11:47
  meet up with the demand.
• 01:11:49
  the zero to one of
• 01:11:50
  building a company here.
• 01:11:51
  So there's multiple factors that really
• 01:11:54
  made us think hard about
• 01:11:55
  what would be the foundation
• 01:11:58
  for us to be able to operate really fast
• 01:12:00
  with a small set of
• 01:12:02
  resources from a startup and
• 01:12:04
  be a highly reliable solution because we
• 01:12:07
  are catering to larger
• 01:12:08
  companies, enterprise
• 01:12:10
  companies.
• 01:12:10
  So reliability is just really key.
• 01:12:13
  Obviously, I can talk a lot about the
• 01:12:15
  whole telemetry part of
• 01:12:18
  things, making sure everything
• 01:12:20
  is properly monitored and that we can
• 01:12:21
  actually spot when things
• 01:12:24
  go wrong before our customers
• 01:12:25
  even see it and then we can take action.
• 01:12:28
  So there's a bunch of things here.
• 01:12:29
  And Effect just came at a very good
• 01:12:31
  moment where we still had
• 01:12:33
  a sufficiently small code
• 01:12:36
  base that we could actually start
• 01:12:38
  seriously thinking about
• 01:12:39
  completely converting it to
• 01:12:40
  Effect.
• 01:12:41
  So the timing was really good.
• 01:12:43
  So you've mentioned in regards to
• 01:12:45
  adopting Effect for the
• 01:12:46
  sake of improving reliability,
• 01:12:49
  one specific use case was that Markbrom
• 01:12:52
  is using LLMs such as
• 01:12:54
  the products provided by
• 01:12:56
  OpenAI.
• 01:12:56
  And if they go down, you need to fall
• 01:12:58
  back to another system.
• 01:13:00
  And in case that system also goes down,
• 01:13:02
  fall back to yet another one.
• 01:13:04
  All of that is helping to improve the
• 01:13:07
  reliability of Markprompt.
• 01:13:10
  Which other challenges did you face
• 01:13:12
  similar to that one in
• 01:13:14
  regards to reliability and
• 01:13:15
  how did you improve
• 01:13:16
  reliability through Effect?
• 01:13:19
  I mean, there's lots of examples where
• 01:13:21
  Effect has helped us
• 01:13:23
  consolidate our system.
• 01:13:25
  One example is one day we woke up and we
• 01:13:28
  realized that we were under
• 01:13:29
  attack by a malicious hacker
• 01:13:32
  who was interacting with a public facing
• 01:13:35
  deployment of Markprompt and
• 01:13:37
  sending, I think, 20 million
• 01:13:40
  requests per hour.
• 01:13:43
  And this was across the globe, so it was
• 01:13:46
  kind of hard to deal with.
• 01:13:47
  So Ethan Niser, he spent the summer with
• 01:13:50
  us and he's been working
• 01:13:53
  on a lot of Effect-related
• 01:13:54
  things and on his first day,
• 01:13:56
  this attack started happening.
• 01:13:58
  And so he built this great rate limiting
• 01:14:02
  system in the middleware
• 01:14:03
  that is capable of looking
• 01:14:04
  at multiple facets of a payload.
• 01:14:08
  And the interesting thing here is that
• 01:14:09
  you cannot just rely
• 01:14:11
  on IP and fingerprinting
• 01:14:12
  and so on.
• 01:14:13
  You actually also want to start looking
• 01:14:14
  at the LLM payload itself.
• 01:14:16
  What kinds of messages is there, pattern
• 01:14:18
  that we can spot here in
• 01:14:20
  the nature of the requests?
• 01:14:22
  And so it's a multi-tiered rate limiting
• 01:14:24
  system that is very easy
• 01:14:25
  to automatically trigger
• 01:14:27
  and also manually trigger.
• 01:14:28
  So this is an example also where building
• 01:14:30
  with composability has just allowed us to
• 01:14:34
  build a fairly sophisticated
• 01:14:35
  system here, which just works.
• 01:14:38
  We are now very
• 01:14:39
  confident that it just works.
• 01:14:40
  And by the way, one of the reasons that
• 01:14:42
  we're confident that it
• 01:14:43
  works is that it's very
• 01:14:44
  easy to test because the whole dependency
• 01:14:46
  injection story is
• 01:14:47
  just such a natural thing
• 01:14:48
  to do.
• 01:14:49
  You just operate a provider and you can
• 01:14:51
  very easily simulate a
• 01:14:53
  situation that is hard to
• 01:14:55
  reproduce with other means.
• 01:14:56
  So dependency injection, I would say, is
• 01:14:59
  a huge factor to the
• 01:15:00
  reliability because we
• 01:15:01
  can restart testing all sorts of
• 01:15:05
  different combinations of
• 01:15:06
  non-happy paths and building
• 01:15:08
  that with just change or service that you
• 01:15:12
  provide is just very,
• 01:15:13
  very easy with Effect.
• 01:15:14
  Telemetry is obviously a thing.
• 01:15:16
  So I think that the whole testing story
• 01:15:19
  is not just about one thing.
• 01:15:22
  You want to do unit tests.
• 01:15:23
  You want to do integration tests with
• 01:15:25
  dependency injection.
• 01:15:26
  You also really want to have a full
• 01:15:28
  understanding of what's going
• 01:15:29
  on when you are in production.
• 01:15:31
  And so being able to have good traces,
• 01:15:33
  good logging of every
• 01:15:35
  component of your system
• 01:15:36
  is non-trivial.
• 01:15:38
  And so we rely heavily on telemetry and
• 01:15:41
  Effect allows us to just
• 01:15:43
  add spans everywhere in
• 01:15:45
  a very, very simple way.
• 01:15:46
  And it just naturally fits
• 01:15:48
  into the rest of the system.
• 01:15:50
  And this really has allowed us to have a
• 01:15:51
  very, very good
• 01:15:52
  understanding of failure scenarios
• 01:15:55
  before our customers even see them.
• 01:15:57
  So these are just general patterns that
• 01:15:59
  are very natural to do
• 01:16:01
  in Effect that we don't
• 01:16:02
  even think about because they actually
• 01:16:05
  don't incur any sort of
• 01:16:07
  additional thinking on our
• 01:16:08
  part.
• 01:16:09
  It's just adding a line
• 01:16:11
  and then it just works.
• 01:16:12
  Yeah, this makes a lot of sense.
• 01:16:14
  I can totally see how reliability is
• 01:16:17
  almost sort of like a emerging factor,
• 01:16:20
  like a second order effect that comes
• 01:16:22
  when you have a composable system,
• 01:16:25
  and when you have observability, when you
• 01:16:28
  can look into what's going on,
• 01:16:30
  when you can eat your
• 01:16:32
  vegetables, then you're healthy.
• 01:16:34
  And so
• 01:16:35
  Building a reliable system is going the
• 01:16:37
  extra mile on all of those things.
• 01:16:39
  This makes a lot of sense.
• 01:16:41
  Given that you built
• 01:16:42
  quite an expansive system,
• 01:16:44
  maybe we can try to break it down a
• 01:16:46
  little bit more with a technical lens on.
• 01:16:49
  So I'm sure that you have
• 01:16:51
  a client side application.
• 01:16:53
  You've mentioned that it's like a single
• 01:16:54
  page app built with Vite,
• 01:16:57
  but more on the backend side.
• 01:16:59
  Can you describe to me what are the
• 01:17:02
  different subcomponents of your system?
• 01:17:04
  Yeah, so in a very simple way, the main
• 01:17:07
  subcomponents here,
• 01:17:09
  there's the application server,
• 01:17:11
  which is essentially an RPC
• 01:17:12
  server that the client
• 01:17:14
  application is speaking with.
• 01:17:15
  Then we have a public API.
• 01:17:18
  So Markprompt is built as
• 01:17:19
  Stripe for customer support,
• 01:17:21
  in the sense that every part of the
• 01:17:23
  system can be
• 01:17:24
  interacted with via our APIs
• 01:17:26
  so that you can actually build deeper
• 01:17:28
  product experiences
• 01:17:29
  inside of your core product.
• 01:17:32
  So we have a public facing API server.
• 01:17:34
  Then we have the ingestion engine, which
• 01:17:37
  takes in all your systems data.
• 01:17:40
  So this can be public facing like a
• 01:17:41
  website that we would
• 01:17:42
  scrape on a regular basis.
• 01:17:44
  It could be GitHub repository or
• 01:17:47
  historical tickets from
• 01:17:49
  Zendesk and Jira and Confluence
• 01:17:51
  and databases from
• 01:17:53
  Salesforce and whatnot.
• 01:17:55
  So we do a bunch of things here.
• 01:17:57
  We import data.
• 01:17:58
  We chunk it up.
• 01:17:58
  We monitor the whole structure of
• 01:18:02
  documents through the AST
• 01:18:03
  and then we build
• 01:18:04
  embeddings for multi-levels.
• 01:18:06
  And then we store them
• 01:18:07
  in a vector database.
• 01:18:08
  And then we have all the
• 01:18:09
  application related servers,
• 01:18:11
  so things like Discord or
• 01:18:13
  Slack or Zendesk and so on,
• 01:18:15
  which are more like simple servers here.
• 01:18:18
  Got it.
• 01:18:18
  Yeah, that makes a lot of sense to break
• 01:18:20
  it up into those different pieces.
• 01:18:22
  I'd actually love to learn a
• 01:18:24
  little bit more about each,
• 01:18:25
  maybe starting with
• 01:18:26
  the application server.
• 01:18:28
  So you mentioned that you're
• 01:18:29
  using Effect RPC in that regard.
• 01:18:31
  What did you use before
• 01:18:33
  you were using Effect?
• 01:18:34
  And what are the qualities of using
• 01:18:37
  Effect for an
• 01:18:39
  application for an RPC server here?
• 01:18:41
  Yeah, sure.
• 01:18:43
  I mean, this before was
• 01:18:44
  just like simple API handlers.
• 01:18:46
  So you would have two
• 01:18:48
  completely decoupled systems.
• 01:18:51
  And now pretty much all the
• 01:18:53
  endpoints are in full RPC.
• 01:18:56
  It starts with a schema, actually.
• 01:18:58
  We define the input schema, the output
• 01:19:01
  schema, and the error schema
• 01:19:03
  for each of these endpoints.
• 01:19:05
  And then it trickles down all the way to
• 01:19:07
  the client application.
• 01:19:08
  So we have this.
• 01:19:09
  Ethan actually wrote this and I hope he's
• 01:19:10
  going to publish it.
• 01:19:11
  Sometimes soon, essentially React query,
• 01:19:13
  but wrapped around with Effect
• 01:19:15
  so that we have the full schema from the
• 01:19:17
  backend all the way to
• 01:19:18
  the client side code.
• 01:19:19
  And we have nothing more to deal with.
• 01:19:21
  So the whole thing is completely
• 01:19:22
  streamlined in terms of the
• 01:19:24
  communication between the client
• 01:19:26
  and the server.
• 01:19:28
  That makes a lot of sense.
• 01:19:29
  And so before you
• 01:19:30
  didn't have any schema layer,
• 01:19:32
  anything to express the kind of data
• 01:19:35
  you're sending across?
• 01:19:37
  No, we didn't actually.
• 01:19:39
  Got it.
• 01:19:39
  Well, I suppose that through the process
• 01:19:42
  of just modeling the schema,
• 01:19:44
  I'm sure that there were a couple of code
• 01:19:46
  paths that you didn't
• 01:19:47
  even account for before.
• 01:19:49
  So the--
• 01:19:50
  Absolutely.
• 01:19:51
  The public API server where you're
• 01:19:55
  exposing an open API spec, I suppose,
• 01:19:59
  how similar or different is that to
• 01:20:01
  your application server?
• 01:20:02
  Can you describe the public
• 01:20:03
  API server setup a bit more?
• 01:20:06
  Yeah, sure.
• 01:20:06
  So this is more of a
• 01:20:08
  standard HTTP server.
• 01:20:10
  This is actually using Effect HTTP.
• 01:20:12
  So again, we have the full--
• 01:20:13
  Everything starts with the Effect runtime
• 01:20:17
  taking over directly
• 01:20:18
  at the request level.
• 01:20:19
  So again, we can model
• 01:20:20
  everything very nicely via schemas.
• 01:20:23
  And one really nice thing of having
• 01:20:26
  everything defined by schemas is
• 01:20:28
  that we can, for instance, automatically
• 01:20:30
  generate the open API spec.
• 01:20:32
  So this is something that we now have
• 01:20:34
  every single endpoint is
• 01:20:35
  automatically documented
• 01:20:37
  just via the schema.
• 01:20:39
  Got it.
• 01:20:39
  How much overlap does your
• 01:20:41
  internal application server,
• 01:20:43
  that RPC server with
• 01:20:45
  the public API server,
• 01:20:47
  does it have a lot of overlap?
• 01:20:49
  Do you reuse a lot of code internally?
• 01:20:51
  Or are those rather
• 01:20:52
  fulfilling different purposes?
• 01:20:54
  They're fulfilling
• 01:20:55
  completely different purposes, yes.
• 01:20:58
  So yeah, I mean, these things are
• 01:21:00
  completely separate.
• 01:21:02
  So to give you one very important
• 01:21:04
  differentiator is that on the RPC server,
• 01:21:07
  we provide permissions.
• 01:21:10
  So you want to make sure that a given
• 01:21:12
  user can perform an action
• 01:21:14
  that they are allowed to.
• 01:21:16
  And so we have a very nice system that
• 01:21:19
  provides the level of
• 01:21:21
  permissions of a user.
• 01:21:23
  This is completely transparent.
• 01:21:25
  We actually used RLS before,
• 01:21:28
  role level security to
• 01:21:29
  enforce these policies.
• 01:21:31
  But there was a real performance impact.
• 01:21:33
  It was just slowing down the crazy,
• 01:21:34
  especially when you're dealing with
• 01:21:36
  hundreds of thousands of data points.
• 01:21:39
  So this had a real impact.
• 01:21:41
  So we got rid of that.
• 01:21:42
  And actually, everything
• 01:21:43
  now is modeled in code.
• 01:21:44
  So the permission, which I guess was
• 01:21:46
  always how things have
• 01:21:48
  been done without RLS.
• 01:21:52
  And so this is, again, an example where
• 01:21:54
  Effect at the type level,
• 01:21:56
  you can immediately spot whether this is
• 01:21:58
  a user that is allowed to
• 01:22:00
  perform a single action,
• 01:22:02
  because that service
• 01:22:04
  requires these sets of permissions.
• 01:22:06
  And so this is very,
• 01:22:07
  very nice to model as well.
• 01:22:09
  That sounds very interesting.
• 01:22:10
  So you've built yourself an internal DSL
• 01:22:13
  to model those permission rules.
• 01:22:16
  Can you explain a little bit more how
• 01:22:18
  that's implemented and how
• 01:22:20
  that can be imagined from
• 01:22:22
  like a type level
• 01:22:23
  perspective or from an API perspective?
• 01:22:26
  Yeah.
• 01:22:27
  I mean, so every user comes
• 01:22:28
  with a set of permissions, right?
• 01:22:30
  And these permissions
• 01:22:30
  can be modeled via types.
• 01:22:33
  And services can declare what level of
• 01:22:38
  permission they required in
• 01:22:40
  order to perform an action.
• 01:22:42
  And so we immediately know if within an
• 01:22:45
  RPC endpoint, we're using a service,
• 01:22:47
  but that the provided
• 01:22:48
  permissions don't match.
• 01:22:52
  Well, then we can flag it.
• 01:22:53
  And so we can guarantee that there's no
• 01:22:55
  code being shipped where
• 01:22:56
  there's an inconsistency here.
• 01:22:59
  I love it.
• 01:22:59
  That sounds very elegant.
• 01:23:00
  I would love to see some code for that.
• 01:23:03
  Could you imagine that there could be a
• 01:23:05
  more commonly reusable package for that?
• 01:23:08
  Or is that only something that makes
• 01:23:10
  sense within your code base?
• 01:23:13
  Good question.
• 01:23:15
  Obviously, we have been just crunching
• 01:23:18
  through this code and
• 01:23:19
  getting into production.
• 01:23:20
  So we haven't thought too much about how
• 01:23:23
  this would extend to a
• 01:23:24
  more generic use case.
• 01:23:27
  But I guess this could be
• 01:23:29
  a very interesting topic.
• 01:23:31
  Right.
• 01:23:31
  Well, anyone in the audience is happily
• 01:23:34
  invited to try to
• 01:23:35
  replicate this by themselves.
• 01:23:37
  Maybe that's even something that we
• 01:23:39
  could, as part of the Effect org, also
• 01:23:42
  look into providing.
• 01:23:43
  To hear you describing
• 01:23:44
  this use case in this DSL,
• 01:23:46
  that makes a lot of
• 01:23:47
  sense for a lot of use cases.
• 01:23:49
  So moving then to the ingestion engine,
• 01:23:53
  can you describe a little bit more of how
• 01:23:56
  that looks like from a
• 01:23:57
  lifecycle perspective?
• 01:23:59
  So you've mentioned the ingestion engine
• 01:24:01
  is mostly meant to bring in data from
• 01:24:03
  various data sources,
• 01:24:05
  whether it's
• 01:24:05
  Salesforce, whether it's Notion.
• 01:24:07
  Walk me through how that works and how
• 01:24:10
  does that work in regards to Effect?
• 01:24:13
  Do you use Effect streams for that?
• 01:24:15
  Yeah.
• 01:24:15
  How does that work?
• 01:24:17
  So it's still pretty bare bones here.
• 01:24:19
  This is something that we have been
• 01:24:20
  discussing also with the
• 01:24:21
  Effect team and the Effect cluster.
• 01:24:23
  This is where we will probably be going.
• 01:24:26
  So now we have our own system, which is
• 01:24:28
  basically just some
• 01:24:31
  long running node servers
• 01:24:32
  that are able to go and fetch the data.
• 01:24:36
  So that could be make a large request to
• 01:24:39
  an SQL database or starting a scraper
• 01:24:42
  that visits a website and when it's done
• 01:24:45
  and purged all the pages,
• 01:24:48
  it is then indexed in a
• 01:24:49
  temporary cache on our side.
• 01:24:52
  And then it goes into the post processing
• 01:24:53
  pipeline, where again, we obviously we
• 01:24:56
  parallelize all this
• 01:24:58
  because every page
• 01:24:59
  needs some post processing.
• 01:25:01
  So we chunk it up and we try to get a
• 01:25:05
  sense of the structure
• 01:25:06
  of the document, the AST,
• 01:25:08
  and then we build these embeddings for
• 01:25:10
  each levels in the AST and
• 01:25:12
  then we store it in a database.
• 01:25:15
  And so the main challenge here is that
• 01:25:17
  these are long running
• 01:25:18
  tasks which can time out,
• 01:25:20
  they can fail and whatnot.
• 01:25:24
  And so now we rely on external systems to
• 01:25:26
  do the whole workflow orchestration,
• 01:25:28
  but this is something where we are going
• 01:25:30
  to be investing some
• 01:25:30
  effort as well because
• 01:25:32
  Effect is the perfect solution here
• 01:25:35
  because we have the full
• 01:25:37
  sets of tools for dealing with
• 01:25:40
  these kinds of situations.
• 01:25:42
  Yeah, I'm really looking forward for
• 01:25:45
  Effect Cluster and Effect
• 01:25:46
  Workflows to mature further.
• 01:25:49
  I think there's more and more use cases,
• 01:25:52
  particularly when it
• 01:25:53
  comes to AI applications
• 01:25:55
  where you want to express
• 01:25:56
  things as long running workflows.
• 01:25:59
  And I think this is where Effect is
• 01:26:02
  really like such a perfect foundation.
• 01:26:05
  So yeah, I think we're going to see some
• 01:26:08
  very interesting updates
• 01:26:09
  throughout the year here.
• 01:26:11
  So stay tuned for that.
• 01:26:13
  Maybe changing gears slightly.
• 01:26:15
  Initially, you've built all of the
• 01:26:17
  system, I think mostly by yourself.
• 01:26:19
  This is where you started
• 01:26:21
  adopting Effect by yourself.
• 01:26:23
  and then later you onboarded other
• 01:26:25
  engineers who didn't have
• 01:26:27
  prior Effect experience.
• 01:26:29
  Can you share a bit more about that
• 01:26:31
  story, how you onboarded
• 01:26:33
  other engineers who didn't
• 01:26:34
  have Effect experience to become
• 01:26:36
  successful working on your code base?
• 01:26:39
  I mean, this is such
• 01:26:39
  an interesting question.
• 01:26:40
  Because you would think that Effect is
• 01:26:44
  mainly for a TypeScript audience.
• 01:26:47
  Web developers who are proficient now in
• 01:26:50
  TypeScript and realize they
• 01:26:52
  need to go to the next level
• 01:26:54
  in terms of building complex systems.
• 01:26:56
  My take here is actually that having a
• 01:27:00
  code base which is already
• 01:27:01
  fully Effect will actually
• 01:27:04
  lower the barrier to entry for people who
• 01:27:07
  have less experience with TypeScript.
• 01:27:10
  So we've seen that with Elliot, our
• 01:27:13
  founding engineer who has
• 01:27:14
  joined a few months ago,
• 01:27:15
  with very little TypeScript experience,
• 01:27:17
  was able to very quickly take
• 01:27:20
  it up and then be productive.
• 01:27:22
  We've run some trials with some
• 01:27:23
  candidates in the last few weeks with,
• 01:27:26
  again, little experience in
• 01:27:27
  TypeScript.
• 01:27:28
  The thing is that we are hiring a lot of
• 01:27:30
  machine learning engineers.
• 01:27:32
  And so they have typically already a lot
• 01:27:35
  of experience with more
• 01:27:36
  of the machine learning
• 01:27:37
  sets of tools like Python
• 01:27:39
  and very little on TypeScript.
• 01:27:41
  But what we have seen is onboarding them
• 01:27:44
  on our code base, they
• 01:27:45
  were actually able to be
• 01:27:46
  very productive, very early on.
• 01:27:48
  And the great thing is that there's not
• 01:27:50
  much room for bad
• 01:27:51
  practice or the kinds of things
• 01:27:53
  that you need to learn when you get
• 01:27:55
  started with a new language.
• 01:27:56
  So what I think is that actually
• 01:27:58
  Effect will
• 01:27:59
  allow them to be proficient and
• 01:28:02
  build very reliable
• 01:28:03
  code within a few weeks.
• 01:28:05
  If it had not been an Effect and more
• 01:28:07
  like vanilla TypeScript,
• 01:28:08
  yes, sure, there's fewer
• 01:28:09
  things to learn.
• 01:28:10
  It's kind of easy and so on.
• 01:28:12
  But you would build up this understanding
• 01:28:15
  and intuition for the
• 01:28:16
  language over the course of
• 01:28:17
  years.
• 01:28:18
  Right.
• 01:28:18
  And then within years, you'll be able to
• 01:28:19
  build very complex and
• 01:28:21
  reliable systems because you
• 01:28:23
  have been proficient in the language.
• 01:28:26
  So Effect gives you these guardrails that
• 01:28:29
  with the whole thing is
• 01:28:31
  in Effect, sure, you need
• 01:28:32
  to understand the
• 01:28:33
  whole catalog of things.
• 01:28:35
  And this you just take up
• 01:28:36
  as you have new use cases.
• 01:28:37
  But if you have a really good foundation,
• 01:28:39
  like the full RPC
• 01:28:41
  server, you basically just
• 01:28:43
  you start writing schema, right?
• 01:28:45
  You write a schema and then
• 01:28:46
  you fit it into the system.
• 01:28:48
  And then once it's there, it's done.
• 01:28:51
  You don't need to change it.
• 01:28:52
  You haven't done anything bad
• 01:28:54
  or anything that will break.
• 01:28:56
  Right.
• 01:28:56
  So my take on this is that actually it
• 01:28:59
  might actually have the
• 01:29:00
  opposite effect of what
• 01:29:01
  we would expect, which is to be able to
• 01:29:03
  onboard non TypeScript
• 01:29:06
  engineers much, much faster.
• 01:29:09
  Yeah, that definitely sounds
• 01:29:11
  a little bit counterintuitive.
• 01:29:12
  And the first time I heard about this
• 01:29:15
  anecdote, and you shared
• 01:29:16
  this privately with me before,
• 01:29:18
  and that person you've just mentioned,
• 01:29:21
  Elliot, gave a brilliant talk at the last
• 01:29:23
  Effect Meetup in San Francisco where he
• 01:29:26
  shared his first hand
• 01:29:28
  experience learning Effect.
• 01:29:30
  And yeah, I think that is a
• 01:29:32
  little bit counterintuitive.
• 01:29:33
  But when you think about it more, it
• 01:29:35
  makes a lot of sense because most
• 01:29:37
  experienced TypeScript
• 01:29:39
  engineers over years, they had to first
• 01:29:42
  learn how to do anything
• 01:29:44
  with TypeScript at all and then
• 01:29:46
  later learn to ignore like those 80 paths
• 01:29:49
  out of 100 how you could do
• 01:29:52
  something with TypeScript,
• 01:29:53
  but that will lead you
• 01:29:54
  down a dangerous road.
• 01:29:56
  Whereas Effect constrains you a little
• 01:29:58
  bit more and puts you on a
• 01:30:01
  path to success in a way.
• 01:30:03
  That's also like it has been my
• 01:30:04
  experience for I think for me
• 01:30:06
  coming more from TypeScript,
• 01:30:09
  I had to unlearn a lot of things how I
• 01:30:12
  used to do things in TypeScript before.
• 01:30:14
  But this is where for me, a good
• 01:30:17
  foundation to pick up Effect
• 01:30:19
  was actually dabbling in other
• 01:30:21
  languages such as Swift and Rust, where I
• 01:30:24
  already primed my mind more on more saner
• 01:30:29
  engineering practices and learning the
• 01:30:31
  right habits and
• 01:30:33
  embracing the right abstractions.
• 01:30:35
  And so I can totally see how someone who
• 01:30:38
  doesn't have prior TypeScript experience
• 01:30:40
  maybe experience in other languages,
• 01:30:42
  which have more appropriate primitives
• 01:30:45
  can be up and running with Effect in no
• 01:30:48
  time. That is amazing to hear.
• 01:30:50
  So can you walk me through a little bit
• 01:30:53
  more through the process
• 01:30:54
  how you went tactically
• 01:30:56
  about migrating the code base as it was
• 01:31:00
  before Effect one by one
• 01:31:02
  step at a time to migrating
• 01:31:03
  it to Effect. Rome wasn't conquered in a
• 01:31:06
  day, so wasn't your code
• 01:31:08
  base. So how did you go about it?
• 01:31:10
  We had some great help. And this is where
• 01:31:12
  I think the Effect community
• 01:31:13
  is such a wonderful community
• 01:31:15
  of people who are nice and helpful. You
• 01:31:19
  kicked it off with us. So we've been
• 01:31:22
  this for years. And immediately I saw the
• 01:31:25
  importance of this, but
• 01:31:27
  never found the time to
• 01:31:30
  actually learn how to do it in practice.
• 01:31:32
  I think this is also
• 01:31:33
  maybe related to the lack of
• 01:31:34
  documentation and concrete examples back
• 01:31:36
  in the day. How do you
• 01:31:37
  actually convert an API
• 01:31:38
  endpoint to Effect? How do you just do
• 01:31:41
  it? So I think this little
• 01:31:43
  catalyzer was really important,
• 01:31:45
  which allowed us, our team, to then get a
• 01:31:49
  sense of where we could
• 01:31:50
  start fitting it in. It was still
• 01:31:52
  like very, very new. This was earlier
• 01:31:53
  last year. And then we had the pleasure
• 01:31:57
  of having Ethan with
• 01:31:59
  us over the summer. And he wrote a lot of
• 01:32:01
  very important code for both
• 01:32:03
  the backend side of things,
• 01:32:05
  schema, the database, the React query
• 01:32:08
  part, which put us in a
• 01:32:10
  really good foundation.
• 01:32:11
  We had most of it basically sketched out.
• 01:32:14
  And then in September, we decided to
• 01:32:18
  do a big, big push and just get
• 01:32:20
  everything migrated. So this was like a
• 01:32:23
  bit slow in the beginning.
• 01:32:24
  And then very, very quickly, within a few
• 01:32:27
  weeks, we had the
• 01:32:28
  whole thing migrated over.
• 01:32:32
  Good. That makes a lot of sense. And
• 01:32:34
  happy to have acted as
• 01:32:36
  a little catalyst here,
• 01:32:38
  but you've picked up everything so
• 01:32:40
  quickly. So I think this was rather a
• 01:32:43
  little bit of giving you
• 01:32:45
  more courage to go this way. And I think
• 01:32:48
  you figured out quickly
• 01:32:50
  how to work with everything,
• 01:32:53
  and you migrated everything in no time. I
• 01:32:57
  would say, honestly,
• 01:32:59
  the important thing is the
• 01:33:00
  catalyzer here. Once you have the starter
• 01:33:03
  blocks, then it's quite easy to
• 01:33:05
  generalize. It's more
• 01:33:07
  like where do you start? And the
• 01:33:09
  documentation obviously is much better
• 01:33:10
  now. So I think this is
• 01:33:11
  a great thing. But I think this little
• 01:33:14
  initial nudge is
• 01:33:17
  what made the difference here.
• 01:33:19
  If you think about your system as sort of
• 01:33:22
  like a tree of different subsystems,
• 01:33:27
  you can go about converting an existing
• 01:33:29
  system to Effect in different ways.
• 01:33:31
  Either you go first for
• 01:33:33
  the leaves, and then you go up one step
• 01:33:36
  at a time until your entire
• 01:33:37
  program, your entire system is
• 01:33:39
  refactored with Effect. But you can also
• 01:33:41
  isolate specific sub nodes
• 01:33:44
  and from the outside call the
• 01:33:47
  effect and inside of it call other
• 01:33:50
  things that are not yet Effect with
• 01:33:52
  Effect.runPromise or
• 01:33:54
  effect dot promise to run some effects,
• 01:33:57
  some non-effect code yet.
• 01:34:00
  Did you find any particular
• 01:34:02
  topology adoption strategy for you to be
• 01:34:06
  working well? So did you go first
• 01:34:08
  all for the leaves? Or did you go
• 01:34:11
  wherever you wanted to
• 01:34:12
  refactor something anyway? How did you
• 01:34:14
  go about the decision process of when to
• 01:34:17
  refactor what with Effect?
• 01:34:19
  Well, yeah, it was the leaves in
• 01:34:20
  the beginning. And then it was the top
• 01:34:24
  down afterwards, because
• 01:34:26
  we just realized that this
• 01:34:27
  was what we needed. And we just needed as
• 01:34:29
  quickly as possible. So
• 01:34:31
  then there was a combined push
• 01:34:34
  from the whole team to just get the
• 01:34:36
  migration done as quickly as
• 01:34:37
  possible. So the RPC was written
• 01:34:40
  really like top down, right? Because,
• 01:34:43
  well top down in the
• 01:34:44
  sense that one route would be
• 01:34:47
  migrated to RPC, Effect RPC, then another
• 01:34:49
  one and so on. But for
• 01:34:52
  sure, this is a nice thing with
• 01:34:53
  Effect is that you can do it on the
• 01:34:55
  leave to start with and you're not
• 01:34:57
  touching anything else
• 01:34:58
  of your code base. But yeah, you
• 01:35:00
  definitely feel the urge to
• 01:35:01
  get the whole thing end to end.
• 01:35:05
  The runtime should be initiated at the
• 01:35:08
  very beginning of the
• 01:35:10
  thing, right? So that you have
• 01:35:11
  the full happy path, because you get to
• 01:35:14
  love the happy path here, right?
• 01:35:16
  Nice. So if I remember
• 01:35:18
  correctly, I think it's now been pretty
• 01:35:20
  much a year since you've been using
• 01:35:22
  Effect in production.
• 01:35:24
  looking back, what were some
• 01:35:26
  of the things that were
• 01:35:27
  like unexpectedly good with
• 01:35:30
  Effect and also some things that were
• 01:35:32
  more challenging that you thought or
• 01:35:33
  unexpected in some ways?
• 01:35:36
  I didn't expect the Schema package
• 01:35:39
  to be, you know, I didn't have any
• 01:35:42
  expectations that this
• 01:35:43
  was really a core thing. In fact, because
• 01:35:45
  this is really wonderful, how you can
• 01:35:49
  control both like the
• 01:35:50
  decoding part, the encoding part, how you
• 01:35:53
  can pipe everything
• 01:35:54
  together. This is one of the sort of,
• 01:36:00
  it shifts away how you think about a
• 01:36:01
  problem. Because usually,
• 01:36:03
  now when I think of a problem,
• 01:36:05
  I start thinking about what is the
• 01:36:06
  schema, right? It is very
• 01:36:07
  different from what is the actual
• 01:36:09
  application code. You redesign your
• 01:36:11
  schema, you think, well, how should it
• 01:36:13
  transform the data here?
• 01:36:16
  And then you work your way up to getting
• 01:36:18
  it all the way in your
• 01:36:21
  code. So I think that schema was
• 01:36:23
  a very, very positive,
• 01:36:24
  unexpected thing that came out of this.
• 01:36:27
  To give you an example,
• 01:36:29
  I always wanted to, again, in the quest
• 01:36:32
  for extreme lining and being
• 01:36:34
  as efficient as possible as a
• 01:36:36
  startup, things like documentation, I
• 01:36:39
  care a lot about documentation. And the
• 01:36:41
  more we can put that
• 01:36:42
  on like autopilot, the better, right? So
• 01:36:44
  the ability to take
• 01:36:46
  your public API endpoints,
• 01:36:47
  and the other schemas generate an open
• 01:36:49
  API spec is exactly the kind of things
• 01:36:52
  that you want, right?
• 01:36:54
  So this was a very, very nice discovery.
• 01:36:57
  Right. That makes a lot of
• 01:36:59
  sense. And I think that goes
• 01:37:00
  hand in hand with what I've also found in
• 01:37:04
  my development practices over the years,
• 01:37:06
  is that I always started to think more
• 01:37:10
  about the types of a system.
• 01:37:13
  That's like typically what I
• 01:37:13
  start out with. I just like described the
• 01:37:16
  types. But then I've also
• 01:37:18
  found myself thinking about,
• 01:37:20
  actually, this data of this particular
• 01:37:22
  type doesn't just live inside of this
• 01:37:25
  boundary of the system,
• 01:37:26
  but it needs to move across network
• 01:37:29
  boundaries, across threat boundaries,
• 01:37:31
  etc. And now you're,
• 01:37:33
  you need to serialize, deserialize that
• 01:37:35
  variable. And this is where
• 01:37:38
  schemas come in. So I ended up
• 01:37:40
  doing exactly the same as you. Like I
• 01:37:43
  start thinking about like,
• 01:37:44
  okay, what data am I dealing
• 01:37:46
  with here? And I modeled it as schemas
• 01:37:49
  and derived the types from it.
• 01:37:51
  And well, we see this here as
• 01:37:53
  like a big revelation turns out in other
• 01:37:56
  programming languages, that's what you
• 01:37:58
  always do. When you have
• 01:37:59
  like in Rust, etc. When you have like a
• 01:38:02
  struct definition, you can
• 01:38:03
  derive schemas from it. But
• 01:38:06
  I think for TypeScript, this is really
• 01:38:08
  for most engineers, that's
• 01:38:10
  something novel. And I think
• 01:38:12
  this is a pattern that should be embraced
• 01:38:14
  much more since I think
• 01:38:16
  default is you just ignore
• 01:38:18
  the existence of a schema and you JSON
• 01:38:21
  stringify everything. And then you're
• 01:38:23
  surprised what happens
• 01:38:25
  to your classes or to your dates. So
• 01:38:27
  yeah, that schema is such
• 01:38:29
  an integral part of Effect,
• 01:38:32
  I think is no coincidence. And I'm very
• 01:38:34
  happy that it gets so
• 01:38:36
  much attention for detail.
• 01:38:38
  Yeah, I mean, I would say it's not only
• 01:38:41
  the fact that you have like
• 01:38:42
  decoding and encoding and so
• 01:38:44
  on, it's the fact that it's so much in
• 01:38:48
  harmony with the rest of
• 01:38:49
  the whole like Effect way of
• 01:38:51
  doing things, right? So it just feels so,
• 01:38:55
  you know, an integral
• 01:38:57
  part of effect itself.
• 01:38:59
  Yeah, I fully agree. So in regards to the
• 01:39:04
  more challenging aspects,
• 01:39:06
  what were some of the challenges that you
• 01:39:08
  found working in an Effect code base?
• 01:39:11
  Sure. I would say it can give you a sense
• 01:39:14
  of, you know
• 01:39:17
  maybe a
• 01:39:17
  false sense of security.
• 01:39:19
  Because you think that everything
• 01:39:20
  is dealt with at the
• 01:39:21
  type level and you get no
• 01:39:23
  flagged of compile time errors and so on.
• 01:39:26
  And this is partly
• 01:39:28
  true. But I think it's very,
• 01:39:30
  very important. It's very important to
• 01:39:33
  understand that this is not
• 01:39:34
  the case. And it gives you
• 01:39:36
  really good like foundation, but you're
• 01:39:39
  not exempt of or die or catch all that
• 01:39:44
  was put at the end of
• 01:39:46
  your pipes. And so all the other errors
• 01:39:48
  in between are, you know, just you don't
• 01:39:52
  see them. And we had
• 01:39:52
  an example where we had this
• 01:39:55
  bug in our code base, and it
• 01:39:56
  was quite hard to figure out
• 01:39:59
  where it went wrong. So I think this is
• 01:40:01
  something that, you know,
• 01:40:02
  Effect is a very, very powerful
• 01:40:03
  tool. And it brings out the best in us,
• 01:40:08
  you know, engineers, I think.
• 01:40:10
  But it also needs to be
• 01:40:12
  dealt with knowing that, you know,
• 01:40:15
  it's not like a safety guard
• 01:40:17
  or anything, you can mess up
• 01:40:19
  and you can do things that are very hard
• 01:40:21
  to then investigate because
• 01:40:23
  you've built up this, this
• 01:40:24
  sense of everything is just working
• 01:40:26
  smoothly. So this is one thing I think
• 01:40:28
  that this is as important
• 01:40:30
  to keep in mind when you're building, you
• 01:40:32
  know, on top of Effect. I
• 01:40:34
  think another thing that it gets,
• 01:40:35
  a little bit hard to track is,
• 01:40:38
  dependency
• 01:40:39
  injection is just wonderful,
• 01:40:42
  in Effect. And I think that's, that's, I
• 01:40:43
  would mention this probably
• 01:40:44
  as the second most important,
• 01:40:46
  discovery or delight of
• 01:40:48
  working with Effect. But,
• 01:40:50
  you know, where do you put your
• 01:40:53
  dependency? Where is it in your code? How
• 01:40:55
  do you like chase the
• 01:40:56
  dependency, you know, the service
• 01:40:58
  that you provided, that is now affecting
• 01:41:01
  a completely different
• 01:41:02
  part of your code base,
• 01:41:03
  that might be, you
• 01:41:05
  know, some some tooling
• 01:41:06
  to build inside of the
• 01:41:07
  IDE itself, to be able to make it easy to
• 01:41:10
  spot the connection, right.
• 01:41:12
  But now we still like in this
• 01:41:13
  very barebone situation where it's kind
• 01:41:15
  of hard to understand
• 01:41:17
  exactly, you know, the hierarchy of
• 01:41:20
  your dependencies, right, and where they
• 01:41:23
  are actually being used
• 01:41:24
  in your code. So it can be
• 01:41:26
  a little bit hard to navigate sometimes.
• 01:41:28
  Right. Yeah, I found that
• 01:41:29
  myself as well in the project
• 01:41:31
  I'm working on. And it is actually
• 01:41:34
  something that we're thinking
• 01:41:35
  about, that we could possibly
• 01:41:37
  enhance through a IDE integration, etc.
• 01:41:41
  So stay, stay tuned for
• 01:41:44
  for that. You said something
• 01:41:46
  interesting just before, where you said
• 01:41:48
  effect brings out the
• 01:41:50
  best in us engineers. Can you
• 01:41:53
  elaborate on that? What do you mean by that
• 01:41:55
  Well, as an engineer,
• 01:41:58
  you want to build features,
• 01:41:59
  you want to build like real robust
• 01:42:00
  systems, you want to spend
• 01:42:03
  ideally all your time doing
• 01:42:05
  that. You don't want to spend your time
• 01:42:08
  fighting the language, fighting, you
• 01:42:10
  know, building something
• 01:42:11
  that you know, the language was not meant
• 01:42:13
  for, to give you an
• 01:42:14
  example, like concurrency and
• 01:42:15
  TypeScript, right, these are things that
• 01:42:17
  you know, you want a way to,
• 01:42:21
  you know, build a concurrent
• 01:42:22
  system, where you're thinking about what
• 01:42:25
  is the system actually doing
• 01:42:26
  rather than like setting up
• 01:42:28
  all the boilerplate in order to have that
• 01:42:30
  system in place. So I would
• 01:42:34
  say Effect really gives you
• 01:42:36
  the right premise. So one thing I'm
• 01:42:40
  thinking about is, now that we've
• 01:42:42
  completed this migration,
• 01:42:44
  the full migration, it kind of feels like
• 01:42:46
  this is the end game. I know
• 01:42:48
  this might sound a bit like
• 01:42:50
  naive or unrealistic, but it kind of
• 01:42:54
  feels like more stable.
• 01:42:56
  And at least if we continue
• 01:42:58
  building our code base on TypeScript with
• 01:43:00
  Effect, it doesn't feel
• 01:43:02
  like we're going to spend
• 01:43:04
  many more like cycles, like refactoring
• 01:43:06
  the code, right, it feels
• 01:43:09
  like this sort of timeless,
• 01:43:12
  like code base now now now it's, it's,
• 01:43:14
  it's there. Effect is
• 01:43:16
  going to evolve, our code base is
• 01:43:17
  going to evolve. It feels like this is
• 01:43:19
  going to be like adjustments
• 01:43:20
  and not like a full rewrite.
• 01:43:22
  And this is what I mean by, you know, it
• 01:43:24
  feels like kind of like
• 01:43:25
  building a house with some really
• 01:43:27
  good like walls and floor and ceiling and
• 01:43:29
  so on. It's just there now.
• 01:43:32
  And this is what we are all
• 01:43:34
  I guess, striving for as engineers is to
• 01:43:36
  have these, you know, the
• 01:43:38
  right tools, right, the right
• 01:43:39
  foundations, instead of having to fight,
• 01:43:42
  you know, all these like small things
• 01:43:44
  that degrade over time,
• 01:43:46
  and then need to revisit them. And you
• 01:43:48
  realize how silly you were
• 01:43:51
  writing this code, you know,
• 01:43:52
  last year kind of thing, it feels like
• 01:43:54
  more harmonious and stable
• 01:43:57
  Right. I love that. And
• 01:43:59
  I think that is also something that has
• 01:44:02
  really brought me to Effect.
• 01:44:04
  And to flip it on its head,
• 01:44:06
  something that I experienced so many
• 01:44:10
  times and have observed so many other
• 01:44:13
  teams as sort of like a
• 01:44:15
  malfunction of engineering teams and of
• 01:44:18
  projects that engineers are
• 01:44:20
  afraid to make changes in a
• 01:44:22
  project. And I've seen this time and time
• 01:44:25
  and time again. And when
• 01:44:27
  you're afraid to make changes to a
• 01:44:29
  project, particularly if the code was
• 01:44:31
  written by someone else, maybe
• 01:44:32
  the person is no longer there.
• 01:44:35
  That's terrible. Like if you're, if
• 01:44:36
  you're afraid to change something, you
• 01:44:38
  might not change it and
• 01:44:39
  you might not improve it. Or if you
• 01:44:41
  change it, you don't know
• 01:44:42
  about the consequences, etc.
• 01:44:45
  And effect allows you to, gives you the
• 01:44:48
  confidence to change something and you
• 01:44:51
  know what's going to
• 01:44:52
  happen. It even makes it fun to do so.
• 01:44:54
  And I think that's
• 01:44:55
  sort of like a twin to the
• 01:45:00
  aspect of composability, almost like the
• 01:45:03
  other side of the same
• 01:45:04
  coin, where if you can compose
• 01:45:07
  things together, you can decompose things
• 01:45:10
  that makes, that's, those
• 01:45:11
  are just aspects of changing
• 01:45:13
  something and changing not in a big bang
• 01:45:17
  way, but just moving things
• 01:45:19
  around almost feels like a,
• 01:45:21
  like a dance. We're getting a little bit
• 01:45:23
  too, too like pie in the sky
• 01:45:26
  here, but that's really like
• 01:45:27
  my reality as well, like working with
• 01:45:29
  Effect and one of the things
• 01:45:31
  that I like so much about it.
• 01:45:33
  And another thing here may be looking a
• 01:45:35
  little bit more into the
• 01:45:36
  future, but we're increasingly
• 01:45:37
  living in the future where we get, where
• 01:45:40
  we do a lot of our code
• 01:45:42
  development in an AI-assisted way.
• 01:45:45
  Now it's maybe no longer just us, us as
• 01:45:48
  engineers doing performing
• 01:45:50
  changes on our code, but it's now
• 01:45:52
  AI systems performing changes on our
• 01:45:54
  code. And this is where the same
• 01:45:56
  underlying dynamic is even more
• 01:45:58
  severe. If there is a system, if there is
• 01:46:02
  some actor here who's
• 01:46:04
  changing our code, who is maybe
• 01:46:07
  even less competent or less knowledgeable
• 01:46:10
  about this, then maybe even
• 01:46:12
  more scary to make changes
• 01:46:14
  to a system. And if the material here,
• 01:46:18
  the house that we're
• 01:46:18
  building that you've mentioned,
• 01:46:20
  is where it's inviting to have changes be
• 01:46:24
  made, that sets an even better foundation
• 01:46:27
  for those AI-assisted,
• 01:46:30
  for this AI-assisted future. So that's
• 01:46:33
  something I love thinking about.
• 01:46:35
  Yeah, I agree. I think the guardrails
• 01:46:41
  here are so important,
• 01:46:42
  right? You want to make sure that,
• 01:46:44
  whatever the LLM generates has as little
• 01:46:49
  unintended side effects as possible,
• 01:46:52
  because it could hallucinate some code.
• 01:46:55
  So if you can make sure that
• 01:46:57
  the blast radius is as tight,
• 01:46:59
  small as possible, then I think you have
• 01:47:01
  a really good foundation. Obviously, I
• 01:47:03
  don't know where this
• 01:47:04
  is heading. Maybe we're going to be
• 01:47:05
  completely wrong and that we're not even
• 01:47:07
  going to be thinking
• 01:47:07
  about code in the same way in the future.
• 01:47:12
  But it feels like it's a
• 01:47:13
  really good way of ensuring
• 01:47:16
  that whatever an LLM generates stays
• 01:47:20
  within the intended radius.
• 01:47:25
  Exactly. And I think just to round this
• 01:47:28
  part out, Effect gives
• 01:47:30
  you a middle ground between
• 01:47:34
  a natural language like English that
• 01:47:37
  might be more intuitive to
• 01:47:40
  describe what a system should do,
• 01:47:41
  but maybe less concise, as opposed to a
• 01:47:45
  type system or code,
• 01:47:47
  which can be more concise,
• 01:47:49
  but maybe less intuitive. And I feel
• 01:47:51
  Effect is the best of
• 01:47:52
  both worlds, where you can,
• 01:47:54
  through the help of a type system, and
• 01:47:56
  here in this case,
• 01:47:58
  TypeScript provides a marvelous type
• 01:48:00
  system that allows you to give you the
• 01:48:02
  correctness benefits that you wouldn't
• 01:48:04
  get from English, yet
• 01:48:06
  it gives you that more intuitive way to
• 01:48:09
  express and compose things. I feel like
• 01:48:13
  Effect is converging
• 01:48:14
  towards something that's the best of both
• 01:48:16
  worlds here, that is
• 01:48:18
  actually a wonderful substrate
• 01:48:20
  for AIs to also compose
• 01:48:22
  systems out of.
• 01:48:24
  Wish that the latest versions of LLMs
• 01:48:26
  have even better
• 01:48:27
  understanding of the latest developments
• 01:48:28
  in Effect. But I think you mentioned that
• 01:48:31
  you have a nice hack, which is to
• 01:48:33
  clone the Effect repository in cursor and
• 01:48:36
  then have it look at
• 01:48:38
  that while you're coding.
• 01:48:40
  Yeah, so that's actually a shout out to
• 01:48:42
  our friend Tim
• 01:48:44
  Suchanek. He taught me about
• 01:48:48
  this little trick, and I've since applied
• 01:48:50
  it many times also for other projects.
• 01:48:52
  I think we're currently in this uncanny
• 01:48:54
  valley, where LLMs always
• 01:48:57
  lag behind on the latest and
• 01:48:59
  greatest of actual information. The
• 01:49:02
  Effect docs, for example, in
• 01:49:04
  particular, they have really
• 01:49:06
  advanced a lot, got a lot better over the
• 01:49:09
  last couple of months. And
• 01:49:10
  those LLMs that we're using
• 01:49:13
  on a daily basis have not fully caught up
• 01:49:15
  to that level yet, and
• 01:49:17
  are also sometimes trained
• 01:49:19
  on old code snippets, different syntax,
• 01:49:22
  etc. And so when you ask
• 01:49:24
  them, you might get an outdated
• 01:49:25
  response. So what Tim is doing, and since
• 01:49:29
  he's taught me about
• 01:49:31
  this as well, is basically
• 01:49:33
  check out the Effect code base on your
• 01:49:36
  computer locally, open
• 01:49:38
  Cursor in it, and then
• 01:49:40
  Cursor uses the full Effect repo with
• 01:49:43
  all the examples, tests,
• 01:49:44
  etc. as the context. And then
• 01:49:46
  inside of that, you use the Cursor chat
• 01:49:49
  to ask your question, and
• 01:49:51
  you're going to get a really
• 01:49:52
  useful result. So I've been using this a
• 01:49:55
  bunch myself, but we're
• 01:49:57
  also working on some exciting
• 01:49:59
  improvements to the Effect docs, etc.,
• 01:50:02
  where we're trying to
• 01:50:04
  combine the latest and greatest
• 01:50:07
  documentation with some of our own AI
• 01:50:11
  developments as well. So stay tuned for
• 01:50:13
  that. You won't need
• 01:50:14
  those little tricks for too much longer.
• 01:50:18
  So Michael,
• 01:50:19
  thank you so much for taking
• 01:50:21
  out so much of your busy day to share
• 01:50:24
  about your experience and journey with
• 01:50:26
  Effect. Really greatly
• 01:50:28
  appreciate that you share all of that
• 01:50:31
  with the entire audience
• 01:50:32
  here. Before rounding this out,
• 01:50:35
  is there anything else you would like to
• 01:50:37
  share with the folks
• 01:50:38
  listening? I'm sure that you're
• 01:50:42
  very busy building a lot of things, but
• 01:50:44
  you're also interested in
• 01:50:46
  bringing on more engineers.
• 01:50:49
  Anyone in particular who
• 01:50:51
  should reach out to you?
• 01:50:53
  Yeah, so we are constantly hiring and we
• 01:50:57
  are obviously, it's a full TypeScript
• 01:51:01
  Effect code base, but we are also very
• 01:51:04
  much looking for
• 01:51:05
  engineers with an ML background.
• 01:51:07
  And we would warmly welcome them even if
• 01:51:11
  they don't have
• 01:51:12
  specific TypeScript experience.
• 01:51:14
  So yeah, if you're interested in working
• 01:51:18
  on such a code base, which I
• 01:51:20
  think is really, really nice
• 01:51:22
  and working on some really interesting
• 01:51:24
  LLM related topics,
• 01:51:27
  then absolutely reach out.
• 01:51:30
  Yeah, I couldn't give you a warmer
• 01:51:33
  recommendation to work at
• 01:51:35
  Markprompt. Michael, Marie,
• 01:51:38
  Elliot, all the folks working at
• 01:51:40
  Markprompt are A plus humans. If I
• 01:51:43
  wouldn't be doing all of the
• 01:51:45
  things I'm doing, I'd strongly consider
• 01:51:47
  working with you all as
• 01:51:49
  well. So whoever gets a chance
• 01:51:50
  to work with you, I think you're making a
• 01:51:53
  very smart decision. So
• 01:51:55
  now moving back my focus to
• 01:51:57
  you, Michael, thank you so much again for
• 01:52:00
  taking time, sharing all of your
• 01:52:03
  experience and all of
• 01:52:05
  those anecdotes with us. I greatly
• 01:52:06
  appreciate it. Well, thank
• 01:52:08
  you, Johannes. Always a pleasure
• 01:52:09
  to speak with you. Awesome.
• 01:52:12
  Thank you so much. Thank you.