Execution Planning

Imagine that we’ve refactored our generateDadJoke program from our Getting Started guide. Now, instead of handling all errors internally, the code can fail with domain-specific issues like network interruptions or provider outages:

1
import type { 
import AiResponse
AiResponse, 
import Completions
Completions } from "@effect/ai"
2
import { 
import OpenAiCompletions
OpenAiCompletions } from "@effect/ai-openai"
3
import { 
import Data
Data, 
import Effect@since ― 2.0.0
@since ― 2.0.0
@since ― 2.0.0
Effect } from "effect"
4

5
class 
class NetworkError
NetworkError extends 
import Data
Data.
const TaggedError: <"NetworkError">(tag: "NetworkError") => new <A>(args: Equals<A, {}> extends true ? void : { readonly [P in keyof A as P extends "_tag" ? never : P]: A[P]; }) => YieldableError & {
    ...;
} & Readonly<...>@since ― 2.0.0
TaggedError("NetworkError") {}
6

7
class 
class ProviderOutage
ProviderOutage extends 
import Data
Data.
const TaggedError: <"ProviderOutage">(tag: "ProviderOutage") => new <A>(args: Equals<A, {}> extends true ? void : { readonly [P in keyof A as P extends "_tag" ? never : P]: A[P]; }) => YieldableError & {
    ...;
} & Readonly<...>@since ― 2.0.0
TaggedError("ProviderOutage") {}
8

9
declare const 
const generateDadJoke: Effect.Effect<AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>
generateDadJoke: 
import Effect@since ― 2.0.0
@since ― 2.0.0
@since ― 2.0.0
Effect.
interface Effect<out A, out E = never, out R = never>
The Effect interface defines a value that describes a workflow or job,
which can succeed or fail.
Details
The Effect interface represents a computation that can model a workflow
involving various types of operations, such as synchronous, asynchronous,
concurrent, and parallel interactions. It operates within a context of type
R, and the result can either be a success with a value of type A or a
failure with an error of type E. The Effect is designed to handle complex
interactions with external resources, offering advanced features such as
fiber-based concurrency, scheduling, interruption handling, and scalability.
This makes it suitable for tasks that require fine-grained control over
concurrency and error management.
To execute an Effect value, you need a Runtime, which provides the
environment necessary to run and manage the computation.
@since ― 2.0.0
@since ― 2.0.0
Effect<
10
  
import AiResponse
AiResponse.
class AiResponse@since ― 1.0.0
AiResponse,
11
  
class NetworkError
NetworkError | 
class ProviderOutage
ProviderOutage,
12
  
import Completions
Completions.
class Completions@since ― 1.0.0
@since ― 1.0.0
Completions
13
>
14

15
const 
const main: Effect.Effect<void, NetworkError | ProviderOutage, OpenAiClient | AiModels>
main = 
import Effect@since ― 2.0.0
@since ― 2.0.0
@since ― 2.0.0
Effect.
const gen: <YieldWrap<Effect.Effect<AiPlan<in Error, in out Provides, in out Requires>.Provider<Completions.Completions | Tokenizer>, never, OpenAiClient | AiModels>> | YieldWrap<...>, void>(f: (resume: Effect.Adapter) => Generator<...>) => Effect.Effect<...> (+1 overload)
Provides a way to write effectful code using generator functions, simplifying
control flow and error handling.
When to Use
Effect.gen allows you to write code that looks and behaves like synchronous
code, but it can handle asynchronous tasks, errors, and complex control flow
(like loops and conditions). It helps make asynchronous code more readable
and easier to manage.
The generator functions work similarly to async/await but with more
explicit control over the execution of effects. You can yield* values from
effects and return the final result at the end.
Example
import { Effect } from "effect"

const addServiceCharge = (amount: number) => amount + 1

const applyDiscount = (
  total: number,
  discountRate: number
): Effect.Effect<number, Error> =>
  discountRate === 0
    ? Effect.fail(new Error("Discount rate cannot be zero"))
    : Effect.succeed(total - (total * discountRate) / 100)

const fetchTransactionAmount = Effect.promise(() => Promise.resolve(100))

const fetchDiscountRate = Effect.promise(() => Promise.resolve(5))

export const program = Effect.gen(function* () {
  const transactionAmount = yield* fetchTransactionAmount
  const discountRate = yield* fetchDiscountRate
  const discountedAmount = yield* applyDiscount(
    transactionAmount,
    discountRate
  )
  const finalAmount = addServiceCharge(discountedAmount)
  return `Final amount to charge: ${finalAmount}`
})
@since ― 2.0.0
gen(function*() {
16
  const 
const gpt4o: AiPlan.Provider<Completions.Completions | Tokenizer>
gpt4o = yield* 
import OpenAiCompletions
OpenAiCompletions.
const model: (model: (string & {}) | OpenAiCompletions.Model, config?: Omit<OpenAiCompletions.Config.Service, "model">) => AiModel<Completions.Completions | Tokenizer, OpenAiClient>@since ― 1.0.0
model("gpt-4o")
17
  const 
const response: AiResponse.AiResponse
response = yield* 
const gpt4o: AiPlan.Provider<Completions.Completions | Tokenizer>
gpt4o.
AiPlan<in Error, in out Provides, in out Requires>.Provider<Completions | Tokenizer>.provide: <AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>(effect: Effect.Effect<AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>) => Effect.Effect<...>
provide(
const generateDadJoke: Effect.Effect<AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>
generateDadJoke)
18
})

This is fine, but what if we instead want to:

Retry the program a fixed number of times on NetworkErrors
Add some backoff delay between retries
Fallback to a different model provider if OpenAi is down

How can we accomplish such logic?

The AiPlan Type

The Effect AI integrations provide a robust method for creating structured execution plans for your LLM interactions through the AiPlan data type. Rather than making a single model call and hoping it succeeds, AiPlan lets you describe how to handle retries, fallbacks, and recoverable errors in a clear, declarative way.

This is especially useful when:

You want to fall back to a secondary model if the primary one is unavailable
You want to retry on transient errors (e.g. network failures)
You want to control timing between retry attempts

Think of an AiPlan as the blueprint for for an LLM interaction, with logic for when to keep trying to interact with one provider and when to switch providers.

interface AiPlan<Errors, Provides, Requires> {}

An AiPlan has three generic type parameters:

Errors: Any errors that can be handled during execution of the plan
Provides: The services that this plan can provide (e.g. Completions, Embeddings)
Requires: The services that this plan requires (e.g. OpenAiClient, AnthropicClient)

If you’ve used AiModel before (via OpenAiCompletions.model() or similar), you’ll find AiPlan familiar. In fact, an AiModel is in fact an AiPlan with just a single step.

This means you can start by writing your code with plain AiModels, and as your needs become more complex (e.g. adding retries, switching providers), you can upgrade to AiPlan without changing how the rest of your code works.

Defining a Primary Model

The primary entry point to building up an AiPlan is the AiPlan.fromModel constructor.

This method defines the primary model that you would like to use for an LLM interaction, as well as the rules for retrying it under specific conditions.

Use this when you want to:

Retry a model multiple times
Customize backoff timing between attempts
Decide whether to retry based on the error type

Example (Creating an AiPlan from an AiModel)

1
import { 
import AiPlan
AiPlan } from "@effect/ai"
13 collapsed lines
2
import type { 
import AiResponse
AiResponse, 
import Completions
Completions } from "@effect/ai"
3
import { 
import OpenAiCompletions
OpenAiCompletions } from "@effect/ai-openai"
4
import { 
import Data
Data, 
import Effect@since ― 2.0.0
@since ― 2.0.0
@since ― 2.0.0
Effect, 
import Schedule
Schedule } from "effect"
5

6
class 
class NetworkError
NetworkError extends 
import Data
Data.
const TaggedError: <"NetworkError">(tag: "NetworkError") => new <A>(args: Equals<A, {}> extends true ? void : { readonly [P in keyof A as P extends "_tag" ? never : P]: A[P]; }) => YieldableError & {
    ...;
} & Readonly<...>@since ― 2.0.0
TaggedError("NetworkError") {}
7

8
class 
class ProviderOutage
ProviderOutage extends 
import Data
Data.
const TaggedError: <"ProviderOutage">(tag: "ProviderOutage") => new <A>(args: Equals<A, {}> extends true ? void : { readonly [P in keyof A as P extends "_tag" ? never : P]: A[P]; }) => YieldableError & {
    ...;
} & Readonly<...>@since ― 2.0.0
TaggedError("ProviderOutage") {}
9

10
declare const 
const generateDadJoke: Effect.Effect<AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>
generateDadJoke: 
import Effect@since ― 2.0.0
@since ― 2.0.0
@since ― 2.0.0
Effect.
interface Effect<out A, out E = never, out R = never>
The Effect interface defines a value that describes a workflow or job,
which can succeed or fail.
Details
The Effect interface represents a computation that can model a workflow
involving various types of operations, such as synchronous, asynchronous,
concurrent, and parallel interactions. It operates within a context of type
R, and the result can either be a success with a value of type A or a
failure with an error of type E. The Effect is designed to handle complex
interactions with external resources, offering advanced features such as
fiber-based concurrency, scheduling, interruption handling, and scalability.
This makes it suitable for tasks that require fine-grained control over
concurrency and error management.
To execute an Effect value, you need a Runtime, which provides the
environment necessary to run and manage the computation.
@since ― 2.0.0
@since ― 2.0.0
Effect<
11
  
import AiResponse
AiResponse.
class AiResponse@since ― 1.0.0
AiResponse,
12
  
class NetworkError
NetworkError | 
class ProviderOutage
ProviderOutage,
13
  
import Completions
Completions.
class Completions@since ― 1.0.0
@since ― 1.0.0
Completions
14
>
15

16
const 
const DadJokePlan: AiPlan.AiPlan<NetworkError | ProviderOutage, Completions.Completions | Tokenizer, OpenAiClient>
DadJokePlan = 
import AiPlan
AiPlan.
const fromModel: <Completions.Completions | Tokenizer, OpenAiClient, NetworkError | ProviderOutage, Duration, unknown, never, never>(model: AiModel<...>, options?: {
    ...;
} | undefined) => AiPlan.AiPlan<...>@since ― 1.0.0
fromModel(
import OpenAiCompletions
OpenAiCompletions.
const model: (model: (string & {}) | OpenAiCompletions.Model, config?: Omit<OpenAiCompletions.Config.Service, "model">) => AiModel<Completions.Completions | Tokenizer, OpenAiClient>@since ― 1.0.0
model("gpt-4o"), {
17
  
attempts?: number | undefined
attempts: 3,
18
  
schedule?: Schedule.Schedule<Duration, unknown, never> | undefined
schedule: 
import Schedule
Schedule.
const exponential: (base: DurationInput, factor?: number) => Schedule.Schedule<Duration>
Creates a schedule that recurs indefinitely with exponentially increasing
delays.
Details
This schedule starts with an initial delay of base and increases the delay
exponentially on each repetition using the formula base * factor^n, where
n is the number of times the schedule has executed so far. If no factor
is provided, it defaults to 2, causing the delay to double after each
execution.
@since ― 2.0.0
exponential("100 millis", 1.5),
19
  
while?: ((error: NetworkError | ProviderOutage) => boolean | Effect.Effect<boolean, never, never>) | undefined
while: (
error: NetworkError | ProviderOutage
error: 
class NetworkError
NetworkError | 
class ProviderOutage
ProviderOutage) =>
20
    
error: NetworkError | ProviderOutage
error.
_tag: "NetworkError" | "ProviderOutage"
_tag === "NetworkError"
21
})
22

23
const 
const main: Effect.Effect<void, NetworkError | ProviderOutage, OpenAiClient | AiModels>
main = 
import Effect@since ― 2.0.0
@since ― 2.0.0
@since ― 2.0.0
Effect.
const gen: <YieldWrap<Effect.Effect<AiPlan.AiPlan<in Error, in out Provides, in out Requires>.Provider<Completions.Completions | Tokenizer>, never, OpenAiClient | AiModels>> | YieldWrap<...>, void>(f: (resume: Effect.Adapter) => Generator<...>) => Effect.Effect<...> (+1 overload)
Provides a way to write effectful code using generator functions, simplifying
control flow and error handling.
When to Use
Effect.gen allows you to write code that looks and behaves like synchronous
code, but it can handle asynchronous tasks, errors, and complex control flow
(like loops and conditions). It helps make asynchronous code more readable
and easier to manage.
The generator functions work similarly to async/await but with more
explicit control over the execution of effects. You can yield* values from
effects and return the final result at the end.
Example
import { Effect } from "effect"

const addServiceCharge = (amount: number) => amount + 1

const applyDiscount = (
  total: number,
  discountRate: number
): Effect.Effect<number, Error> =>
  discountRate === 0
    ? Effect.fail(new Error("Discount rate cannot be zero"))
    : Effect.succeed(total - (total * discountRate) / 100)

const fetchTransactionAmount = Effect.promise(() => Promise.resolve(100))

const fetchDiscountRate = Effect.promise(() => Promise.resolve(5))

export const program = Effect.gen(function* () {
  const transactionAmount = yield* fetchTransactionAmount
  const discountRate = yield* fetchDiscountRate
  const discountedAmount = yield* applyDiscount(
    transactionAmount,
    discountRate
  )
  const finalAmount = addServiceCharge(discountedAmount)
  return `Final amount to charge: ${finalAmount}`
})
@since ― 2.0.0
gen(function*() {
24
  const 
const plan: AiPlan.AiPlan.Provider<Completions.Completions | Tokenizer>
plan = yield* 
const DadJokePlan: AiPlan.AiPlan<NetworkError | ProviderOutage, Completions.Completions | Tokenizer, OpenAiClient>
DadJokePlan
25
  const 
const response: AiResponse.AiResponse
response = yield* 
const plan: AiPlan.AiPlan.Provider<Completions.Completions | Tokenizer>
plan.
AiPlan<in Error, in out Provides, in out Requires>.Provider<Completions | Tokenizer>.provide: <AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>(effect: Effect.Effect<AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>) => Effect.Effect<...>
provide(
const generateDadJoke: Effect.Effect<AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>
generateDadJoke)
26
})

This plan will:

Attempt to use OpenAi’s "gpt-4o" model up to 3 times
Wait with an exponential backoff between attempts (starting at 100ms)
Only re-attempt the call to OpenAi if the error is a NetworkError

Adding Fallback Models

To make your LLM interactions resilient to provider outages, you can define a fallback model to use via AiPlan.withFallback. This will allow the plan to automatically fallback to another model if the previous step in the execution plan fails.

Use this when:

You want to make your LLM interactions resilient to provider outages
You want to potentially have multiple fallback models

Example (Adding a Fallback to Anthropic from OpenAi)

1
import { 
import AiPlan
AiPlan } from "@effect/ai"
14 collapsed lines
2
import type { 
import AiResponse
AiResponse, 
import Completions
Completions } from "@effect/ai"
3
import { 
import AnthropicCompletions
AnthropicCompletions } from "@effect/ai-anthropic"
4
import { 
import OpenAiCompletions
OpenAiCompletions } from "@effect/ai-openai"
5
import { 
import Data
Data, 
import Effect@since ― 2.0.0
@since ― 2.0.0
@since ― 2.0.0
Effect, 
import Schedule
Schedule } from "effect"
6

7
class 
class NetworkError
NetworkError extends 
import Data
Data.
const TaggedError: <"NetworkError">(tag: "NetworkError") => new <A>(args: Equals<A, {}> extends true ? void : { readonly [P in keyof A as P extends "_tag" ? never : P]: A[P]; }) => YieldableError & {
    ...;
} & Readonly<...>@since ― 2.0.0
TaggedError("NetworkError") {}
8

9
class 
class ProviderOutage
ProviderOutage extends 
import Data
Data.
const TaggedError: <"ProviderOutage">(tag: "ProviderOutage") => new <A>(args: Equals<A, {}> extends true ? void : { readonly [P in keyof A as P extends "_tag" ? never : P]: A[P]; }) => YieldableError & {
    ...;
} & Readonly<...>@since ― 2.0.0
TaggedError("ProviderOutage") {}
10

11
declare const 
const generateDadJoke: Effect.Effect<AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>
generateDadJoke: 
import Effect@since ― 2.0.0
@since ― 2.0.0
@since ― 2.0.0
Effect.
interface Effect<out A, out E = never, out R = never>
The Effect interface defines a value that describes a workflow or job,
which can succeed or fail.
Details
The Effect interface represents a computation that can model a workflow
involving various types of operations, such as synchronous, asynchronous,
concurrent, and parallel interactions. It operates within a context of type
R, and the result can either be a success with a value of type A or a
failure with an error of type E. The Effect is designed to handle complex
interactions with external resources, offering advanced features such as
fiber-based concurrency, scheduling, interruption handling, and scalability.
This makes it suitable for tasks that require fine-grained control over
concurrency and error management.
To execute an Effect value, you need a Runtime, which provides the
environment necessary to run and manage the computation.
@since ― 2.0.0
@since ― 2.0.0
Effect<
12
  
import AiResponse
AiResponse.
class AiResponse@since ― 1.0.0
AiResponse,
13
  
class NetworkError
NetworkError | 
class ProviderOutage
ProviderOutage,
14
  
import Completions
Completions.
class Completions@since ― 1.0.0
@since ― 1.0.0
Completions
15
>
16

17
const 
const DadJokePlan: AiPlan.AiPlan<NetworkError | ProviderOutage, Completions.Completions | Tokenizer, OpenAiClient | AnthropicClient>
DadJokePlan = 
import AiPlan
AiPlan.
const fromModel: <Completions.Completions | Tokenizer, OpenAiClient, NetworkError | ProviderOutage, Duration, unknown, never, never>(model: AiModel<...>, options?: {
    ...;
} | undefined) => AiPlan.AiPlan<...>@since ― 1.0.0
fromModel(
import OpenAiCompletions
OpenAiCompletions.
const model: (model: (string & {}) | OpenAiCompletions.Model, config?: Omit<OpenAiCompletions.Config.Service, "model">) => AiModel<Completions.Completions | Tokenizer, OpenAiClient>@since ― 1.0.0
model("gpt-4o"), {
4 collapsed lines
18
  
attempts?: number | undefined
attempts: 3,
19
  
schedule?: Schedule.Schedule<Duration, unknown, never> | undefined
schedule: 
import Schedule
Schedule.
const exponential: (base: DurationInput, factor?: number) => Schedule.Schedule<Duration>
Creates a schedule that recurs indefinitely with exponentially increasing
delays.
Details
This schedule starts with an initial delay of base and increases the delay
exponentially on each repetition using the formula base * factor^n, where
n is the number of times the schedule has executed so far. If no factor
is provided, it defaults to 2, causing the delay to double after each
execution.
@since ― 2.0.0
exponential("100 millis", 1.5),
20
  
while?: ((error: NetworkError | ProviderOutage) => boolean | Effect.Effect<boolean, never, never>) | undefined
while: (
error: NetworkError | ProviderOutage
error: 
class NetworkError
NetworkError | 
class ProviderOutage
ProviderOutage) =>
21
    
error: NetworkError | ProviderOutage
error.
_tag: "NetworkError" | "ProviderOutage"
_tag === "NetworkError"
22
}).
Pipeable.pipe<AiPlan.AiPlan<NetworkError | ProviderOutage, Completions.Completions | Tokenizer, OpenAiClient>, AiPlan.AiPlan<...>>(this: AiPlan.AiPlan<...>, ab: (_: AiPlan.AiPlan<...>) => AiPlan.AiPlan<...>): AiPlan.AiPlan<...> (+21 overloads)
pipe(
23
  
import AiPlan
AiPlan.
const withFallback: <Completions.Completions | Tokenizer, Completions.Completions | Tokenizer, AnthropicClient, Duration, NetworkError | ProviderOutage, unknown, never, never>(options: {
    ...;
}) => <E, Requires>(self: AiPlan.AiPlan<...>) => AiPlan.AiPlan<...> (+1 overload)@since ― 1.0.0
withFallback({
24
    
model: AiModel<Completions.Completions | Tokenizer, AnthropicClient>
model: 
import AnthropicCompletions
AnthropicCompletions.
const model: (model: (string & {}) | AnthropicCompletions.Model, config?: Omit<AnthropicCompletions.Config.Service, "model">) => AiModel<Completions.Completions | Tokenizer, AnthropicClient>@since ― 1.0.0
model("claude-3-7-sonnet-latest"),
25
    
attempts?: number | undefined
attempts: 2,
26
    
schedule?: Schedule.Schedule<Duration, unknown, never> | undefined
schedule: 
import Schedule
Schedule.
const exponential: (base: DurationInput, factor?: number) => Schedule.Schedule<Duration>
Creates a schedule that recurs indefinitely with exponentially increasing
delays.
Details
This schedule starts with an initial delay of base and increases the delay
exponentially on each repetition using the formula base * factor^n, where
n is the number of times the schedule has executed so far. If no factor
is provided, it defaults to 2, causing the delay to double after each
execution.
@since ― 2.0.0
exponential("100 millis", 1.5),
27
    
while?: ((error: NetworkError | ProviderOutage) => boolean | Effect.Effect<boolean, never, never>) | undefined
while: (
error: NetworkError | ProviderOutage
error: 
class NetworkError
NetworkError | 
class ProviderOutage
ProviderOutage) =>
28
      
error: NetworkError | ProviderOutage
error.
_tag: "NetworkError" | "ProviderOutage"
_tag === "ProviderOutage"
29
  })
30
)
31

32
const 
const main: Effect.Effect<void, NetworkError | ProviderOutage, OpenAiClient | AiModels | AnthropicClient>
main = 
import Effect@since ― 2.0.0
@since ― 2.0.0
@since ― 2.0.0
Effect.
const gen: <YieldWrap<Effect.Effect<AiPlan.AiPlan<in Error, in out Provides, in out Requires>.Provider<Completions.Completions | Tokenizer>, never, OpenAiClient | AiModels | AnthropicClient>> | YieldWrap<...>, void>(f: (resume: Effect.Adapter) => Generator<...>) => Effect.Effect<...> (+1 overload)
Provides a way to write effectful code using generator functions, simplifying
control flow and error handling.
When to Use
Effect.gen allows you to write code that looks and behaves like synchronous
code, but it can handle asynchronous tasks, errors, and complex control flow
(like loops and conditions). It helps make asynchronous code more readable
and easier to manage.
The generator functions work similarly to async/await but with more
explicit control over the execution of effects. You can yield* values from
effects and return the final result at the end.
Example
import { Effect } from "effect"

const addServiceCharge = (amount: number) => amount + 1

const applyDiscount = (
  total: number,
  discountRate: number
): Effect.Effect<number, Error> =>
  discountRate === 0
    ? Effect.fail(new Error("Discount rate cannot be zero"))
    : Effect.succeed(total - (total * discountRate) / 100)

const fetchTransactionAmount = Effect.promise(() => Promise.resolve(100))

const fetchDiscountRate = Effect.promise(() => Promise.resolve(5))

export const program = Effect.gen(function* () {
  const transactionAmount = yield* fetchTransactionAmount
  const discountRate = yield* fetchDiscountRate
  const discountedAmount = yield* applyDiscount(
    transactionAmount,
    discountRate
  )
  const finalAmount = addServiceCharge(discountedAmount)
  return `Final amount to charge: ${finalAmount}`
})
@since ― 2.0.0
gen(function*() {
33
  const 
const plan: AiPlan.AiPlan.Provider<Completions.Completions | Tokenizer>
plan = yield* 
const DadJokePlan: AiPlan.AiPlan<NetworkError | ProviderOutage, Completions.Completions | Tokenizer, OpenAiClient | AnthropicClient>
DadJokePlan
34
  const 
const response: AiResponse.AiResponse
response = yield* 
const plan: AiPlan.AiPlan.Provider<Completions.Completions | Tokenizer>
plan.
AiPlan<in Error, in out Provides, in out Requires>.Provider<Completions | Tokenizer>.provide: <AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>(effect: Effect.Effect<AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>) => Effect.Effect<...>
provide(
const generateDadJoke: Effect.Effect<AiResponse.AiResponse, NetworkError | ProviderOutage, Completions.Completions>
generateDadJoke)
35
})

This plan will:

Fallback to Anthropic if the OpenAi step of the plan fails
Attempt to use Anthropic’s "claude-3-7-sonnet" model up to 2 times
Wait with an exponential backoff between attempts (starting at 100ms)
Only run and / or re-attempt the fallback if the error is a ProviderOutage

End-to-End Usage

The following is the complete program with the desired AiPlan fully implemented:

1
import { AiPlan } from "@effect/ai"
2
import type { AiResponse, Completions } from "@effect/ai"
3
import { AnthropicClient, AnthropicCompletions } from "@effect/ai-anthropic"
4
import { OpenAiClient, OpenAiCompletions } from "@effect/ai-openai"
5
import { NodeHttpClient } from "@effect/platform-node"
6
import { Data, Effect, Schedule } from "effect"
7

8
class NetworkError extends Data.TaggedError("NetworkError") {}
9

10
class ProviderOutage extends Data.TaggedError("ProviderOutage") {}
11

12
declare const generateDadJoke: Effect.Effect<
13
  AiResponse.AiResponse,
14
  NetworkError | ProviderOutage,
15
  Completions.Completions
16
>
17

18
const DadJokePlan = AiPlan.fromModel(OpenAiCompletions.model("gpt-4o"), {
19
  attempts: 3,
20
  schedule: Schedule.exponential("100 millis", 1.5),
21
  while: (error: NetworkError | ProviderOutage) =>
22
    error._tag === "NetworkError"
23
}).pipe(
24
  AiPlan.withFallback({
25
    model: AnthropicCompletions.model("claude-3-7-sonnet-latest"),
26
    attempts: 2,
27
    schedule: Schedule.exponential("100 millis", 1.5),
28
    while: (error: NetworkError | ProviderOutage) =>
29
      error._tag === "ProviderOutage"
30
  })
31
)
32

33
const main = Effect.gen(function*() {
34
  const plan = yield* DadJokePlan
35
  const response = yield* plan.provide(generateDadJoke)
36
})
37

38
const Anthropic = AnthropicClient.layerConfig({
39
  apiKey: Config.redacted("ANTHROPIC_API_KEY")
40
}).pipe(Layer.provide(NodeHttpClient.layerUndici))
41

42
const OpenAi = OpenAiClient.layerConfig({
43
  apiKey: Config.redacted("OPENAI_API_KEY")
44
}).pipe(Layer.provide(NodeHttpClient.layerUndici))
45

46
main.pipe(
47
  Effect.provide([Anthropic, OpenAi]),
48
  Effect.runPromise
49
)

Notes & Patterns

You can chain multiple AiPlan.withFallback calls together for more complex failover logic
Just like AiModel, AiPlan is re-usable and can be abstracted into shared logic if desired
Great for teams needing multi-provider resilience or predictable behavior under failure