Getting Started

In this getting started guide, we will demonstrate how to generate a simple text completion using an LLM provider (OpenAi) using the Effect AI integration packages.

We’ll walk through:

• Writing provider-agnostic logic to interact with an LLM
• Declaring the specific LLM model to use for the interaction
• Using a provider integration to make the program executable

Installation

First, we will need to install the base @effect/ai package to gain access to the core AI abstractions. In addition, we will need to install at least one provider integration package (in this case @effect/ai-openai):

npm

# Install the base package for the core abstractions (always required)
npm install @effect/ai

# Install one (or more) provider integrations
npm install @effect/ai-openai

# Also add the core Effect package (if not already installed)
npm install effect

pnpm

# Install the base package for the core abstractions (always required)
pnpm add @effect/ai

# Install one (or more) provider integrations
pnpm add @effect/ai-openai

# Also add the core Effect package (if not already installed)
pnpm add effect

Yarn

# Install the base package for the core abstractions (always required)
yarn add @effect/ai

# Install one (or more) provider integrations
yarn add @effect/ai-openai

# Also add the core Effect package (if not already installed)
yarn add effect

Bun

# Install the base package for the core abstractions (always required)
bun add @effect/ai

# Install one (or more) provider integrations
bun add @effect/ai-openai

# Also add the core Effect package (if not already installed)
bun add effect

Define an Interaction with a Language Model

First let’s define a simple interaction with a large language model (LLM):

Example (Using the AiLanguageModel Service to Generate a Dad Joke)

import { AiLanguageModel } from "@effect/ai"
import { Effect } from "effect"

// Using `AiLanguageModel` will add it to your program's requirements
//
//          ┌─── Effect<AiResponse, AiError, AiLanguageModel>
//          ▼
const generateDadJoke = Effect.gen(function*() {
  // Use the `AiLanguageModel` to generate some text
  const response = yield* AiLanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  // Log the generated text to the console
  console.log(response.text)
  // Return the response
  return response
})

Declarative LLM Interactions

Notice that the above code does not know or care which LLM provider (OpenAi, Anthropic, etc.) will be used. Instead, we focus on what we want to accomplish (i.e. our business logic), not how to accomplish it.

Select a Provider

Next, we need to select which model provider we want to use:

Example (Using a Model Provider to Satisfy the AiLanguageModel Requirement)

import { OpenAiLanguageModel } from "@effect/ai-openai"
import { AiLanguageModel } from "@effect/ai"
import { Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* AiLanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

// Create an `AiModel` which provides a concrete implementation of
// `AiLanguageModel` and requires an `OpenAiClient`
//
//      ┌─── AiModel<AiLanguageModel, OpenAiClient>
//      ▼
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

// Provide the `AiModel` to the program
//
//     ┌─── Effect<AiResponse, AiError, OpenAiClient>
//     ▼
const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

Before moving on, it is important that we understand the purpose of the AiModel data type.

Understanding AiModel

The AiModel data type represents a provider-specific implementation of one or more services, such as AiLanguageModel or AiEmbeddingsModel. It is the primary way that you can plug a real large language model into your program.

export interface AiModel<Provides, Requires> {}

An AiModel has two generic type parameters:

• Provides - the services this model will provide when built
• Requires - the services this model will require to be built

This allows Effect to track which services should be added to the requirements of the program that builds the AiModel, as well as which services the built AiModel can provide.

Creating an AiModel

To create an AiModel, you can use the model-specific factory from one of Effect’s provider integration packages.

Example (Defining an AiModel to Interact with OpenAi)

import { OpenAiLanguageModel } from "@effect/ai-openai"

//      ┌─── AiModel<AiLanguageModel, OpenAiClient>
//      ▼
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

This creates an AiModel that:

• Provides an OpenAi-specific implementation of the AiLanguageModel service using "gpt-4o"
• Requires an OpenAiClient to be built

Providing an AiModel

Once you’ve created an AiModel, you can directly Effect.provide it to your Effect programs just like any other service:

import { OpenAiLanguageModel } from "@effect/ai-openai"
import { AiLanguageModel } from "@effect/ai"

//      ┌─── AiModel<AiLanguageModel, OpenAiClient>
//      ▼
const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

//       ┌─── Effect<AiResponse, AiError, OpenAiClient>
//       ▼
const program = AiLanguageModel.generateText({
  prompt: "Generate a dad joke"
}).pipe(Effect.provide(Gpt4o))

Benefits of AiModel

There are several benefits to this approach:

Reusability

You can provide the same AiModel to as many programs as you like.

Example (Providing an AiModel to Multiple Programs)

import { OpenAiLanguageModel } from "@effect/ai-openai"
import { AiLanguageModel } from "@effect/ai"
import { Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* AiLanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

const main = Effect.gen(function*() {
  // You can provide the `AiModel` individually to each
  // program, or to all of them at once (as we do here)
  const res1 = yield* generateDadJoke
  const res2 = yield* generateDadJoke
  const res3 = yield* generateDadJoke
}).pipe(Effect.provide(Gpt4o))

Flexibility

If we know that one model or provider performs better at a given task than another, we can freely mix and match models and providers together.

For example, if we know Anthropic’s Claude generates some really great dad jokes, we can mix it into our existing program with just a few lines of code:

Example (Mixing Multiple Providers and Models)

import { AnthropicLanguageModel } from "@effect/ai-anthropic"
import { OpenAiLanguageModel } from "@effect/ai-openai"
import { AiLanguageModel } from "@effect/ai"
import { Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* AiLanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
const Claude37 = AnthropicLanguageModel.model("claude-3-7-sonnet-latest")

//      ┌─── Effect<void, AiError, AnthropicClient | OpenAiClient>
//      ▼
const main = Effect.gen(function*() {
  const res1 = yield* generateDadJoke
  const res2 = yield* generateDadJoke
  const res3 = yield* Effect.provide(generateDadJoke, Claude37)
}).pipe(Effect.provide(Gpt4o))

Because Effect performs type-level dependency tracking, we can see that an AnthropicClient must now also be provided to make our program runnable.

Abstractability

An AiModel can also be yield*’ed to lift its dependencies into the calling Effect. This is particularly useful when creating services that depend on AI interactions, where you want to avoid leaking service-level dependencies into the service interface.

For example, in the code below the main program is only dependent upon the DadJokes service. All AI requirements are abstracted away into Layer composition.

Example (Abstracting LLM Interactions into a Service)

import { AnthropicLanguageModel } from "@effect/ai-anthropic"
import { OpenAiLanguageModel } from "@effect/ai-openai"
import { AiLanguageModel } from "@effect/ai"
import { Effect } from "effect"

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")
const Claude37 = AnthropicLanguageModel.model("claude-3-7-sonnet-latest")

class DadJokes extends Effect.Service<DadJokes>()("app/DadJokes", {
  effect: Effect.gen(function*() {
    // Yielding the model will return a layer with no requirements
    //
    //     ┌─── Layer<AiLanguageModel>
    //     ▼
    const gpt = yield* Gpt4o
    const claude = yield* Claude37

    const generateDadJoke = Effect.gen(function*() {
      const response = yield* AiLanguageModel.generateText({
        prompt: "Generate a dad joke"
      })
      console.log(response.text)
      return response
    })

    return {
      generateDadJoke: Effect.provide(generateDadJoke, gpt),
      generateBetterDadJoke: Effect.provide(generateDadJoke, claude)
    }
  })
}) {}

// Programs which utilize the `DadJokes` service have no knowledge of
// any AI requirements
//
//     ┌─── Effect<void, AiError, DadJokes>
//     ▼
const main = Effect.gen(function*() {
  const dadJokes = yield* DadJokes
  const res1 = yield* dadJokes.generateDadJoke
  const res2 = yield* dadJokes.generateBetterDadJoke
})

// The AI requirements are abstracted away into `Layer` composition
//
//         ┌─── Layer<DadJokes, never, AnthropicClient | OpenAiClient>
//         ▼
DadJokes.Default

Create a Provider Client

To make our code executable, we must finish satisfying our program’s requirements.

Let’s take another look at our program from earlier:

import { OpenAiLanguageModel } from "@effect/ai-openai"
import { AiLanguageModel } from "@effect/ai"
import { Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* AiLanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

//     ┌─── Effect<AiResponse, AiError, OpenAiClient>
//     ▼
const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

We can see that our main program still requires us to provide an OpenAiClient.

Each of our provider integration packages exports a client module that can be used to construct a client for that provider.

Example (Creating a Client Layer for a Model Provider)

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { AiLanguageModel } from "@effect/ai"
import { Config, Effect } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* AiLanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

// Create a `Layer` which produces an `OpenAiClient` and requires
// an `HttpClient`
//
//      ┌─── Layer<OpenAiClient, ConfigError, HttpClient>
//      ▼
const OpenAi = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
})

In the code above, we use the layerConfig constructor from the OpenAiClient module to create a Layer which will produce an OpenAiClient. The layerConfig constructor allows us to read in configuration variables using Effect’s configuration system.

The provider clients also have a dependency on an HttpClient implementation to avoid any platform dependencies. This way, you can provide whichever HttpClient implementation is most appropriate for the platform your code is running upon.

For example, if we know we are going to run this code in NodeJS, we can utilize the NodeHttpClient module from @effect/platform-node to provide an HttpClient implementation:

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { AiLanguageModel } from "@effect/ai"
import { NodeHttpClient } from "@effect/platform-node"
import { Config, Effect, Layer } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* AiLanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

// Create a `Layer` which produces an `OpenAiClient` and requires
// an `HttpClient`
//
//      ┌─── Layer<OpenAiClient, ConfigError, HttpClient>
//      ▼
const OpenAi = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
})

// Provide a platform-specific implementation of `HttpClient` to our
// OpenAi layer
//
//        ┌─── Layer<OpenAiClient, ConfigError, never>
//        ▼
const OpenAiWithHttp = Layer.provide(OpenAi, NodeHttpClient.layerUndici)

Running the Program

Now that we have a Layer which provides us with an OpenAiClient, we’re ready to make our main program runnable.

Our final program looks like the following:

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { AiLanguageModel } from "@effect/ai"
import { NodeHttpClient } from "@effect/platform-node"
import { Config, Effect, Layer } from "effect"

const generateDadJoke = Effect.gen(function*() {
  const response = yield* AiLanguageModel.generateText({
    prompt: "Generate a dad joke"
  })
  console.log(response.text)
  return response
})

const Gpt4o = OpenAiLanguageModel.model("gpt-4o")

const main = generateDadJoke.pipe(
  Effect.provide(Gpt4o)
)

const OpenAi = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
})

const OpenAiWithHttp = Layer.provide(OpenAi, NodeHttpClient.layerUndici)

main.pipe(
  Effect.provide(OpenAiWithHttp),
  Effect.runPromise
)