Getting Started

In this getting started guide, we will demonstrate how to generate a simple text completion using an LLM provider (OpenAi) using the Effect AI integration packages.

We’ll walk through:

Writing provider-agnostic logic to interact with an LLM
Declaring the specific LLM model to use for the interaction
Using a provider integration to make the program executable

Installation

First, we will need to install the base @effect/ai package to gain access to the core AI abstractions. In addition, we will need to install at least one provider integration package (in this case @effect/ai-openai):

# Install the base package for the core abstractions (always required)
npm install @effect/ai

# Install one (or more) provider integrations
npm install @effect/ai-openai

# Also add the core Effect package (if not already installed)
npm install effect

# Install the base package for the core abstractions (always required)
pnpm add @effect/ai

# Install one (or more) provider integrations
pnpm add @effect/ai-openai

# Also add the core Effect package (if not already installed)
pnpm add effect

# Install the base package for the core abstractions (always required)
yarn add @effect/ai

# Install one (or more) provider integrations
yarn add @effect/ai-openai

# Also add the core Effect package (if not already installed)
yarn add effect

# Install the base package for the core abstractions (always required)
bun add @effect/ai

# Install one (or more) provider integrations
bun add @effect/ai-openai

# Also add the core Effect package (if not already installed)
bun add effect

Define an Interaction with a Language Model

First let’s define a simple interaction with a large language model (LLM):

Example (Using the LanguageModel Service to Generate a Dad Joke)

1
import { LanguageModel } from "@effect/ai"
2
import { Effect } from "effect"
3

4
// Using `LanguageModel` will add it to your program's requirements
5
//
6
//          ┌─── Effect<GenerateTextResponse<{}>, AiError, LanguageModel>
7
//          ▼
8
const generateDadJoke = Effect.gen(function* () {
9
  // Use the `LanguageModel` to generate some text
10
  const response = yield* LanguageModel.generateText({
11
    prompt: "Generate a dad joke",
12
  })
13
  // Log the generated text to the console
14
  console.log(response.text)
15
  // Return the response
16
  return response
17
})

Select a Provider

Next, we need to select which model provider we want to use:

Example (Using a Model Provider to Satisfy the LanguageModel Requirement)

1
import { OpenAiLanguageModel } from "@effect/ai-openai"
2
import { LanguageModel } from "@effect/ai"
3
import { Effect } from "effect"
4

7 collapsed lines
5
const generateDadJoke = Effect.gen(function* () {
6
  const response = yield* LanguageModel.generateText({
7
    prompt: "Generate a dad joke",
8
  })
9
  console.log(response.text)
10
  return response
11
})
12

13
// Create a `Model` which provides a concrete implementation of
14
// `LanguageModel` and requires an `OpenAiClient`
15
//
16
//      ┌─── Model<"openai", LanguageModel | ProviderName, OpenAiClient>
17
//      ▼
18
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
19

20
// Provide the `Model` to the program
21
//
22
//     ┌─── Effect<GenerateTextResponse<{}>, AiError, OpenAiClient>
23
//     ▼
24
const main = generateDadJoke.pipe(Effect.provide(Gpt4o))

Before moving on, it is important that we understand the purpose of the Model data type.

Understanding `Model`

The Model data type represents a provider-specific implementation of one or more services, such as LanguageModel or EmbeddingsModel. It is the primary way that you can plug a real large language model into your program.

export interface Model<ProviderName, Provides, Requires> {}

An Model has three generic type parameters:

ProviderName - the name of the large language model provider that will be used
Provides - the services this model will provide when built
Requires - the services this model will require to be built

This allows Effect to track which services the Model requires as well as which services the Model will provide.

Creating n `Model`

To create a Model, you can use the model-specific factory from one of Effect’s provider integration packages.

Example (Defining a Model to Interact with OpenAI)

import { OpenAiLanguageModel } from "@effect/ai-openai"

//      ┌─── Model<"openai", LanguageModel | ProviderName, OpenAiClient>
//      ▼
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

This creates a Model that:

Provides the ProviderName service, which allows introspection of the current provider in use by the program
Provides an OpenAI-specific implementation of the LanguageModel service using "gpt-4o"
Requires an OpenAiClient to be built

Providing a `Model`

Once you’ve created a Model, you can directly Effect.provide it to your Effect programs just like any other service:

1
import { OpenAiLanguageModel } from "@effect/ai-openai"
2
import { LanguageModel } from "@effect/ai"
3

4
//      ┌─── Model<"openai", LanguageModel | ProviderName, OpenAiClient>
5
//      ▼
6
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
7

8
//       ┌─── Effect<GenerateTextResponse<{}>, AiError, OpenAiClient>
9
//       ▼
10
const program = LanguageModel.generateText({
11
  prompt: "Generate a dad joke",
12
}).pipe(Effect.provide(Gpt4o))

Benefits of `Model`

There are several benefits to this approach:

Reusability

You can provide the same Model to as many programs as you like.

Example (Providing a Model to Multiple Programs)

1
import { OpenAiLanguageModel } from "@effect/ai-openai"
2
import { LanguageModel } from "@effect/ai"
3
import { Effect } from "effect"
4

7 collapsed lines
5
const generateDadJoke = Effect.gen(function* () {
6
  const response = yield* LanguageModel.generateText({
7
    prompt: "Generate a dad joke",
8
  })
9
  console.log(response.text)
10
  return response
11
})
12

13
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
14

15
const main = Effect.gen(function* () {
16
  // You can provide the `Model` individually to each
17
  // program, or to all of them at once (as we do here)
18
  const res1 = yield* generateDadJoke
19
  const res2 = yield* generateDadJoke
20
  const res3 = yield* generateDadJoke
21
}).pipe(Effect.provide(Gpt4o))

Flexibility

If we know that one model or provider performs better at a given task than another, we can freely mix and match models and providers together.

For example, if we know Anthropic’s Claude generates some really great dad jokes, we can mix it into our existing program with just a few lines of code:

Example (Mixing Multiple Providers and Models)

1
import { AnthropicLanguageModel } from "@effect/ai-anthropic"
2
import { OpenAiLanguageModel } from "@effect/ai-openai"
3
import { LanguageModel } from "@effect/ai"
4
import { Effect } from "effect"
5

7 collapsed lines
6
const generateDadJoke = Effect.gen(function* () {
7
  const response = yield* LanguageModel.generateText({
8
    prompt: "Generate a dad joke",
9
  })
10
  console.log(response.text)
11
  return response
12
})
13

14
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
15
const Claude37 = AnthropicLanguageModel.model("claude-3-7-sonnet-latest")
16

17
//      ┌─── Effect<void, AiError, AnthropicClient | OpenAiClient>
18
//      ▼
19
const main = Effect.gen(function* () {
20
  const res1 = yield* generateDadJoke
21
  const res2 = yield* generateDadJoke
22
  const res3 = yield* Effect.provide(generateDadJoke, Claude37)
23
}).pipe(Effect.provide(Gpt4o))

Because Effect performs type-level dependency tracking, we can see that an AnthropicClient is now required to make our program runnable.

Abstractability

An Model can also be yield*’ed to lift its dependencies into the calling Effect. This is particularly useful when creating services that depend on AI interactions, where you want to avoid leaking service-level dependencies into the service interface.

For example, in the code below the main program is only dependent upon the DadJokes service. All AI requirements are abstracted away into Layer composition.

Example (Abstracting LLM Interactions into a Service)

1
import { AnthropicLanguageModel } from "@effect/ai-anthropic"
2
import { OpenAiLanguageModel } from "@effect/ai-openai"
3
import { LanguageModel } from "@effect/ai"
4
import { Effect } from "effect"
5

6
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
7
const Claude37 = AnthropicLanguageModel.model("claude-3-7-sonnet-latest")
8

9
class DadJokes extends Effect.Service<DadJokes>()("app/DadJokes", {
10
  effect: Effect.gen(function* () {
11
    // Yielding the model will return a layer with no requirements
12
    //
13
    //     ┌─── Layer<LanguageModel | ProviderName>
14
    //     ▼
15
    const gpt = yield* Gpt4o
16
    const claude = yield* Claude37
17

7 collapsed lines
18
    const generateDadJoke = Effect.gen(function* () {
19
      const response = yield* LanguageModel.generateText({
20
        prompt: "Generate a dad joke",
21
      })
22
      console.log(response.text)
23
      return response
24
    })
25

26
    return {
27
      generateDadJoke: Effect.provide(generateDadJoke, gpt),
28
      generateBetterDadJoke: Effect.provide(generateDadJoke, claude),
29
    }
30
  }),
31
}) {}
32

33
// Programs which utilize the `DadJokes` service have no knowledge of
34
// any AI requirements
35
//
36
//     ┌─── Effect<void, AiError, DadJokes>
37
//     ▼
38
const main = Effect.gen(function* () {
39
  const dadJokes = yield* DadJokes
40
  const res1 = yield* dadJokes.generateDadJoke
41
  const res2 = yield* dadJokes.generateBetterDadJoke
42
})
43

44
// The AI requirements are abstracted away into `Layer` composition
45
//
46
//         ┌─── Layer<DadJokes, never, AnthropicClient | OpenAiClient>
47
//         ▼
48
DadJokes.Default

Create a Provider Client

To make our code executable, we must finish satisfying our program’s requirements.

Let’s take another look at our program from earlier:

1
import { OpenAiLanguageModel } from "@effect/ai-openai"
2
import { LanguageModel } from "@effect/ai"
3
import { Effect } from "effect"
4

7 collapsed lines
5
const generateDadJoke = Effect.gen(function* () {
6
  const response = yield* LanguageModel.generateText({
7
    prompt: "Generate a dad joke",
8
  })
9
  console.log(response.text)
10
  return response
11
})
12

13
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
14

15
//     ┌─── Effect<GenerateTextResponse<{}>, AiError, OpenAiClient>
16
//     ▼
17
const main = generateDadJoke.pipe(Effect.provide(Gpt4o))

We can see that our main program still requires us to provide an OpenAiClient.

Each of our provider integration packages exports a client module that can be used to construct a client for that provider.

Example (Creating a Client Layer for a Model Provider)

1
import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
2
import { LanguageModel } from "@effect/ai"
3
import { Config, Effect } from "effect"
4

13 collapsed lines
5
const generateDadJoke = Effect.gen(function* () {
6
  const response = yield* LanguageModel.generateText({
7
    prompt: "Generate a dad joke",
8
  })
9
  console.log(response.text)
10
  return response
11
})
12

13
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
14

15
const main = generateDadJoke.pipe(Effect.provide(Gpt4o))
16

17
// Create a `Layer` which produces an `OpenAiClient` and requires
18
// an `HttpClient`
19
//
20
//      ┌─── Layer<OpenAiClient, ConfigError, HttpClient>
21
//      ▼
22
const OpenAi = OpenAiClient.layerConfig({
23
  apiKey: Config.redacted("OPENAI_API_KEY"),
24
})

In the code above, we use the layerConfig constructor from the OpenAiClient module to create a Layer which will produce an OpenAiClient. The layerConfig constructor allows us to read in configuration variables using Effect’s configuration system.

The provider clients also have a dependency on an HttpClient implementation to avoid any platform dependencies. This way, you can provide whichever HttpClient implementation is most appropriate for the platform your code is running upon.

For example, if we know we are going to run this code in NodeJS, we can utilize the NodeHttpClient module from @effect/platform-node to provide an HttpClient implementation:

1
import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
2
import { LanguageModel } from "@effect/ai"
3
import { NodeHttpClient } from "@effect/platform-node"
4
import { Config, Effect, Layer } from "effect"
5

13 collapsed lines
6
const generateDadJoke = Effect.gen(function* () {
7
  const response = yield* LanguageModel.generateText({
8
    prompt: "Generate a dad joke",
9
  })
10
  console.log(response.text)
11
  return response
12
})
13

14
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
15

16
const main = generateDadJoke.pipe(Effect.provide(Gpt4o))
17

18
// Create a `Layer` which produces an `OpenAiClient` and requires
19
// an `HttpClient`
20
//
21
//      ┌─── Layer<OpenAiClient, ConfigError, HttpClient>
22
//      ▼
23
const OpenAi = OpenAiClient.layerConfig({
24
  apiKey: Config.redacted("OPENAI_API_KEY"),
25
})
26

27
// Provide a platform-specific implementation of `HttpClient` to our
28
// OpenAi layer
29
//
30
//        ┌─── Layer<OpenAiClient, ConfigError, never>
31
//        ▼
32
const OpenAiWithHttp = Layer.provide(OpenAi, NodeHttpClient.layerUndici)

Running the Program

Now that we have a Layer which provides us with an OpenAiClient, we’re ready to make our main program runnable.

Our final program looks like the following:

1
import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
2
import { LanguageModel } from "@effect/ai"
3
import { NodeHttpClient } from "@effect/platform-node"
4
import { Config, Effect, Layer } from "effect"
5

6
const generateDadJoke = Effect.gen(function* () {
7
  const response = yield* LanguageModel.generateText({
8
    prompt: "Generate a dad joke",
9
  })
10
  console.log(response.text)
11
  return response
12
})
13

14
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
15

16
const main = generateDadJoke.pipe(Effect.provide(Gpt4o))
17

18
const OpenAi = OpenAiClient.layerConfig({
19
  apiKey: Config.redacted("OPENAI_API_KEY"),
20
})
21

22
const OpenAiWithHttp = Layer.provide(OpenAi, NodeHttpClient.layerUndici)
23

24
main.pipe(Effect.provide(OpenAiWithHttp), Effect.runPromise)

Getting Started

Installation

Define an Interaction with a Language Model

Select a Provider

Understanding Model

Creating n Model

Providing a Model

Benefits of Model

Create a Provider Client

Running the Program

Understanding `Model`

Creating n `Model`

Providing a `Model`

Benefits of `Model`