Module of the Week - RcMap

On Effect Office Hours, people often ask me to “roast” their Effect codebases. This takes some nerve on my part, given that my own GitHub profile is public.

Rather than give anyone a reason to look at my code, let’s outsource the roasting. We’ll build “Roast My Repo”, a hypothetical app with AI reviewers trained to bring out your inner imposter syndrome.

Reference Counting

To gain access to your code, the app will clone your repository into a temporary folder. We will start with two AI reviewers: one examines your dependency graph and the other reviews your code. Both will need to access the same repository, so they will share a checkout instead of cloning the repository twice.

But if both review agents are sharing one checkout, how do we know when we can delete it?

We can track this with a count. Each reviewer adds one when it starts using the checkout and subtracts one when it’s done. When the count reaches zero, nobody needs the files anymore, so we can safely remove the directory.

Try connecting and disconnecting the reviewers below to see this in action.

This is called reference counting.

Why a Map?

So far, our reviewers have all been reviewing the same repository. But what happens when a user submits a different one?

Suppose we’re reviewing acme/api and acme/website. They need separate checkouts, each with its own reference count.

Select Reviewer A, then click acme/api to connect them. Do the same with Reviewer B and acme/website. Now click acme/website again to release Reviewer B’s checkout. Watch it disappear while Reviewer A’s checkout stays available.

We can keep track of these checkouts in a map, using the repository URL and commit as the key. Each entry holds a checkout and its reference count. Reviewers requesting the same key share that checkout. When its count reaches zero, we delete the checkout and remove the entry.

Select acme/api beneath Reviewer A to add it to the map. Select it beneath Reviewer B too: the reference count increases, but there’s still only one entry. Deselect it beneath both reviewers and watch the entry disappear.

Using a map allows us to access shared resources by key. Reference counting lets us automatically clean them up when they’re no longer in use. Effect’s RcMap combines these two ideas into a single data structure.

Let’s use RcMap to implement the checkout sharing we’ve just seen.

The Git Service

Defining the service

Let’s put our checkouts behind a Git service. Its checkout method takes a repository URL and desired commit and returns the path to a local checkout.

We’ll group the URL and commit in a Repository class. This is what we’ll pass to checkout and use as the key in our map:

1
import { Data } from "effect"
2

3
export class Repository extends Data.Class<{
4
  readonly url: string
5
  readonly commit: string
6
}> {
7
  get identifier(): string {
8
    return `${this.url}@${this.commit}`
9
  }
10
}

RcMap compares keys using Effect’s equality rules. Two separately created Repository values with the same URL and commit count as the same key. In Effect v4, plain objects also have structural equality by default, so we could use those instead, but Data.Class is useful here for grouping the fields and getter together.

Inside the Git service, we create the map using the RcMap.make constructor. We must define a lookup function, which will receives a Repository and returns an Effect that acquires the checkout.

For this example, we’ll create a temporary directory and log where the clone would go. A full implementation would run Git before returning the path.

1
import { Context, Data, Effect, FileSystem, Layer, RcMap, Scope } from "effect"
2
import { Repository } from "./repository.ts"
3

4
export class GitError extends Data.TaggedError("GitError")<{
5
  readonly repository: Repository
6
  readonly cause: unknown
7
}> {}
8

9
export interface GitService {
10
  readonly checkout: (
11
    repository: Repository,
12
  ) => Effect.Effect<string, GitError, Scope.Scope>
13
}
14

15
export class Git extends Context.Service<Git, GitService>()("Git", {
16
  make: Effect.gen(function* () {
17
    const fs = yield* FileSystem.FileSystem
18

19
    const checkouts = yield* RcMap.make({
20
      lookup: (repository: Repository) =>
21
        fs.makeTempDirectoryScoped({ prefix: repository.identifier }).pipe(
22
          Effect.tap((dir) => Effect.log(`Cloning to: ${dir}`)),
23
          Effect.mapError((cause) => new GitError({ repository, cause })),
24
        ),
25
    })
26

27
    return {
28
      checkout: (repository) => RcMap.get(checkouts, repository),
29
    }
30
  }),
31
}) {
32
  static readonly layer = Layer.effect(this, this.make)
33
}

Acquiring a checkout

Constructing the map doesn’t create any directories yet. When checkout is invoked, it delegates to RcMap.get. If the key isn’t in the map, RcMap runs the lookup function which will create the checkout and insert an entry into the map. Otherwise, the reviewer shares the existing checkout.

Notice that the checkout method provided by the Git service requires a Scope.

1
export interface GitService {
2
  readonly checkout: (
3
    repository: Repository,
4
  ) => Effect.Effect<string, GitError, Scope.Scope>
5
}

That’s how RcMap tracks how long each reviewer needs the files. When the Scope associated with a given call to checkout closes, the reference count for that map entry is decremented. When no references remain, the temporary directory is removed.

Running reviews

Let’s run two reviews against the same repository. Each review gets its own scope, so it can release the checkout as soon as it’s done.

1
import { NodeRuntime, NodeServices } from "@effect/platform-node"
2
import { Effect, Layer } from "effect"
3
import { Git } from "./git.ts"
4
import { Repository } from "./repository.ts"
5

6
declare const reviewDependencies: (path: string) => Effect.Effect<void, Error>
7
declare const reviewCode: (path: string) => Effect.Effect<void, Error>
8

9
const MainLayer = Git.layer.pipe(Layer.provide(NodeServices.layer))
10

11
const program = Effect.gen(function* () {
12
  const git = yield* Git
13

14
  const repository = new Repository({
15
    url: "https://github.com/acme/api.git",
16
    commit: "a1b2c3d",
17
  })
18

19
  const dependencyReview = git.checkout(repository).pipe(
20
    Effect.flatMap(reviewDependencies),
21
    Effect.scoped, // Close the scope after the review completes
22
  )
23

24
  const codeReview = git.checkout(repository).pipe(
25
    Effect.flatMap(reviewCode),
26
    Effect.scoped, // Close the scope after the review completes
27
  )
28

29
  // Run the reviews concurrently
30
  yield* Effect.all([dependencyReview, codeReview], {
31
    concurrency: "unbounded",
32
  })
33
})
34

35
program.pipe(Effect.provide(MainLayer), NodeRuntime.runMain)

Effect.all runs both reviews concurrently against the same repository, so if one is still acquiring the checkout, the other waits for it. Once it’s ready, both receive the same directory path.

Placing Effect.scoped after Effect.flatMap keeps the scope open while the review runs. When the review finishes, the scope closes and releases its reference to the checkout.

Say the dependency review finishes first. Its scope closes, taking the reference count from two to one. The code review carries on using the directory. When that finishes, the count reaches zero and RcMap removes the directory. Neither review needs to know what the other is doing.

Keeping checkouts alive

Both reviews finish, their associated Scopes are closed, and the checkout directory is deleted from the file system. A second later, another reviewer asks for the same repository. Now we have to re-clone the entire repository again.

This is pretty inefficient.

Luckily, RcMap provides us with the ability to keep entries around for a little while before cleaning them up. Let’s set the map’s idleTimeToLive to five seconds, giving another review a chance to reuse the same checkout before it’s deleted:

1
const checkouts =
2
  yield *
3
  RcMap.make({
4
    lookup: (repository: Repository) =>
5
      fs.makeTempDirectoryScoped({ prefix: repository.identifier }).pipe(
6
        Effect.tap((dir) => Effect.log(`Cloning to: ${dir}`)),
7
        Effect.mapError((cause) => new GitError({ repository, cause })),
8
      ),
9
    idleTimeToLive: "5 seconds",
10
  })

Now, when the reference count for a given checkout reaches zero, the checkout sticks around for five more seconds before being cleaned up.

Close all three scopes below to start the countdown, then click Get checkout before it reaches zero. You’ll get the same directory back. Close the new scope and wait five seconds to see it removed.

Reusing existing checkouts

Suppose we only want to run an optional review if there’s already a checkout for that repository. If there isn’t, we’ll skip the review. Calling get won’t work here: it creates a checkout when the key is missing. We need a way to request an existing entry without creating one.

RcMap.getOption does exactly that. It returns Some(path) for an existing checkout, or None if the key is missing. If the checkout is still being acquired, it waits for the path rather than returning None.

Try clicking Get existing below with an empty map. Then, click Get checkout followed by Get existing and observe the difference.

Extending the idle window

Suppose we added a scheduler that assigns reviews to a pool of AI workers. The dependency review has finished, but the code review is still queued, waiting for a free worker. Nothing is using the checkout, so its idle countdown has started. We’d like to keep those files around for the queued review rather than clone the repository again.

The scheduler can call RcMap.touch periodically to reset the idle countdown while the review waits for a worker:

1
yield * RcMap.touch(checkouts, repository)

Limiting checkouts

Keeping idle checkouts around saves us from cloning them again, but they still take up disk space. We can limit how many entries the map holds by setting the map’s capacity:

1
const checkouts =
2
  yield *
3
  RcMap.make({
4
    lookup: (repository: Repository) =>
5
      fs.makeTempDirectoryScoped({ prefix: repository.identifier }).pipe(
6
        Effect.tap((dir) => Effect.log(`Cloning to: ${dir}`)),
7
        Effect.mapError((cause) => new GitError({ repository, cause })),
8
      ),
9
    idleTimeToLive: "5 seconds",
10
    capacity: 10,
11
  })

Attempting to add a new entry to a map that is at capacity will fail with a Cause.ExceededCapacityError.

The map below has a capacity of two entries. Request acme/api and acme/website, then try acme/docs. The third request fails, but requesting acme/api again still works because it shares an existing entry.

With a capacity set, RcMap.get can fail with Cause.ExceededCapacityError when a new entry would exceed the limit. Our GitService.checkout signature needs to include that error unless we map it to our existing GitError:

1
import { Cause, Effect, Scope } from "effect"
2

3
export interface GitService {
4
  readonly checkout: (
5
    repository: Repository,
6
  ) => Effect.Effect<
7
    string,
8
    GitError | Cause.ExceededCapacityError,
9
    Scope.Scope
10
  >
11
}

I’d keep ExceededCapacityError in the signature so callers can decide what to do when the map is full, such as re-queueing the review or retrying after a delay.

Invalidating checkouts

Suppose a review runs a tool that modifies files in the checkout. The repository URL and commit haven’t changed, but the files on disk have. We’d like the next review to get a clean copy.

Inside our Git service, we can remove the entry from the map with RcMap.invalidate:

1
yield * RcMap.invalidate(checkouts, repository)

The next time get is invoked for that repository, the map’s lookup function will be called and a new checkout will be created.

But what happens to reviews that were using the checkout we invalidated? Let’s take a look.

Click Invalidate below, then Get checkout. Notice what happens to the scopes referencing the invalidated checkout.

Invalidated map entries keep their references, so in our case the old checkout will remain on the file system until the last Scope associated with it is closed.

Conclusion

Effect’s reference-counted data structures are exceptionally useful in situations where you have multiple consumers that all need shared access to the same scoped resource.

We focused on RcMap in this post, which manages shared resources by key. Effect also provides RcRef for when you only need to share a single resource.

We hope you enjoyed this edition of Module of the Week! Until the next one - Happy Effecting!