Chat unstable

Stateful conversation sessions on top of a language model.

A Chat keeps Prompt history in a Ref and reuses it for text generation, streaming, and structured output. Each generation call combines the current history with the caller's new prompt, invokes the active language model, and appends the response parts back into history. Constructors create fresh sessions, seed sessions from prompts, restore exported history, or connect a chat to persistence.

14 exports Since v4.0.0 Source

Constructors

empty

unstable

Since v4.0.0

Creates a new Chat service with empty conversation history.

When to use

Use when you need to start a fresh chat session without initial context or system prompts.

Signature

declare const empty: Effect.Effect<Chat>

Example Creating an empty chat

import { Effect } from "effect"
import { Chat } from "effect/ai"

const freshChat = Effect.gen(function*() {
  const chat = yield* Chat.empty
  const history = yield* chat.export
  return (history as { content: ReadonlyArray<unknown> }).content.length
})

await Effect.runPromise(freshChat) // => 0

fromExport

unstable

Since v4.0.0

Creates a Chat service from previously exported chat data.

Details

Restores a chat session from structured data that was previously exported using the export method. Useful for persisting and restoring conversation state.

Signature

declare function fromExport(data: unknown): Effect<Chat, SchemaError>

Example Restoring chat data

import { Effect, Ref } from "effect"
import { Chat } from "effect/ai"

const restoreChat = Effect.gen(function*() {
  const originalChat = yield* Chat.fromPrompt([
    {
      role: "user",
      content: "Which library are we using?"
    },
    {
      role: "assistant",
      content: "The project uses Effect."
    }
  ])

  const exported = yield* originalChat.export
  const restoredChat = yield* Chat.fromExport(exported)
  const restoredHistory = yield* Ref.get(restoredChat.history)

  const restoredResponse = restoredHistory.content[1]
  if (restoredResponse?.role === "assistant") {
    const restoredText = restoredResponse.content[0]
    if (restoredText?.type === "text") {
      return {
        roles: restoredHistory.content.map((message) => message.role),
        text: restoredText.text
      }
    }
  }
  return undefined
})

await Effect.runPromise(restoreChat) // => { roles: ["user", "assistant"], text: "The project uses Effect." }

fromJson

unstable

Since v4.0.0

Creates a Chat service from previously exported JSON chat data.

Details

Restores a chat session from JSON string that was previously exported using the exportJson method. This is the most convenient way to persist and restore chat sessions to/from storage systems.

Signature

declare function fromJson(data: string): Effect<Chat, SchemaError>

Example Restoring chat history from JSON

import { Effect, Ref } from "effect"
import { Chat } from "effect/ai"

const restoreFromJson = Effect.gen(function*() {
  const original = yield* Chat.fromPrompt("Hello")
  const jsonData = yield* original.exportJson
  const restoredChat = yield* Chat.fromJson(jsonData)
  const history = yield* Ref.get(restoredChat.history)
  return history.content.length
})

await Effect.runPromise(restoreFromJson) // => 1

fromPrompt

unstable

Since v4.0.0

Creates a new Chat service from an initial prompt.

Details

This is the primary constructor for creating chat instances. It initializes a new conversation with the provided prompt as the starting context.

Signature

declare function fromPrompt(prompt: RawInput): Effect<Chat, never, never>

Example Creating a chat from a system prompt

import { Effect } from "effect"
import { Chat } from "effect/ai"

const chatWithSystemPrompt = Effect.gen(function*() {
  const chat = yield* Chat.fromPrompt([{
    role: "system",
    content: "You are a helpful assistant specialized in mathematics."
  }])

  const history = yield* chat.export
  return (history as { content: ReadonlyArray<unknown> }).content.length
})

await Effect.runPromise(chatWithSystemPrompt) // => 1

Example Restoring chat history from a prompt

import { Effect } from "effect"
import { Chat } from "effect/ai"

// Initialize with conversation history
const existingChat = Effect.gen(function*() {
  const chat = yield* Chat.fromPrompt([
    {
      role: "user",
      content: [{ type: "text", text: "What's the weather like?" }]
    },
    {
      role: "assistant",
      content: [{ type: "text", text: "I don't have access to weather data." }]
    },
    {
      role: "user",
      content: [{ type: "text", text: "Can you help me with coding?" }]
    }
  ])

  const history = yield* chat.export
  return (history as { content: ReadonlyArray<unknown> }).content.length
})

await Effect.runPromise(existingChat) // => 3

makePersisted

unstable

Since v4.0.0

Creates a new chat persistence service.

When to use

Use when you need programmatic persisted chat creation and retrieval backed by the current BackingPersistence.

Details

The provided store identifier will be used to indicate which "store" the backing persistence should load chats from.

See

layerPersisted for the Layer-based constructor

Signature

declare const makePersisted: (...args: [options: {
  readonly storeId: string;
}]) => Effect<Service, never, Scope | BackingPersistence>

Errors

ChatNotFoundError

unstable

Since v4.0.0

Represents an error that occurs when attempting to retrieve a persisted Chat that does not exist in the backing persistence store.

When to use

Use to represent a missing persisted conversation when lookup by id cannot find stored history.

Signature

declare class ChatNotFoundError extends {
  readonly _tag: "ChatNotFoundError";
  readonly chatId: string;
} & YieldableError<this> {
  constructor(...args: [props: {
    readonly _tag?: "ChatNotFoundError";
    readonly chatId: string;
  }, options?: MakeOptions]);
}

Layers

layerPersisted

unstable

Since v4.0.0

Creates a Layer for a new chat persistence service.

When to use

Use to provide Chat.Persistence from a configured BackingPersistence when your application needs persisted chat sessions backed by a named store.

Details

The provided store identifier will be used to indicate which "store" the backing persistence should load chats from.

See

makePersisted for the effect constructor when building the service directly instead of providing it as a layer

Signature

declare function layerPersisted(options: {
  readonly storeId: string;
}): Layer<Persistence, never, BackingPersistence>

Models

Chat

interface unstable

Since v4.0.0

Chat session with history, export, and generation operations.

See

Persisted for the persistence-backed extension

Signature

interface Chat {
  readonly "~effect/ai/Chat": "~effect/ai/Chat";
  readonly export: Effect<unknown, AiError>;
  readonly exportJson: Effect<string, AiError>;
  readonly generateObject: <ObjectEncoded extends Record<string, any>, ObjectSchema extends Encoder<ObjectEncoded, unknown>, Options extends NoExcessProperties<GenerateObjectOptions<any, ObjectSchema>, Options>>(options: Options & GenerateObjectOptions<ExtractTools<Options>, ObjectSchema>) => Effect<GenerateObjectResponse<ExtractTools<Options>, ObjectSchema["Type"], ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options> | ObjectSchema["DecodingServices"]>;
  readonly generateText: {
    <Options extends NoExcessProperties<GenerateTextOptions<{}>, Options>>(options: Options & {
      readonly toolkit?: undefined;
    } & GenerateTextOptions<{}>): Effect<GenerateTextResponse<{}, ExtractToolParametersModeWithoutToolkit<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
    <Tools extends Record<string, Any>, Options extends NoExcessProperties<GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }, Options>>(options: Options & GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }): Effect<GenerateTextResponse<Tools, ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
    <Options extends {
      readonly toolkit: WithHandler<any> | Effect<WithHandler<any>, never, any>;
    } & GenerateTextOptions<any> & Readonly<Record<Exclude<keyof Options, keyof GenerateTextOptions<any>>, never>>>(options: Options & GenerateTextOptions<ExtractTools<Options>> & {
      readonly toolkit: Options["toolkit"];
    }): Effect<GenerateTextResponse<ExtractTools<Options>, ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
  };
  readonly history: Ref<Prompt>;
  readonly streamText: {
    <Options extends NoExcessProperties<GenerateTextOptions<{}>, Options>>(options: Options & {
      readonly toolkit?: undefined;
    } & GenerateTextOptions<{}>): Stream<StreamPart<{}, ExtractToolParametersModeWithoutToolkit<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
    <Tools extends Record<string, Any>, Options extends NoExcessProperties<GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }, Options>>(options: Options & GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }): Stream<StreamPart<Tools, ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
    <Options extends {
      readonly toolkit: WithHandler<any> | Effect<WithHandler<any>, never, any>;
    } & GenerateTextOptions<any> & Readonly<Record<Exclude<keyof Options, keyof GenerateTextOptions<any>>, never>>>(options: Options & GenerateTextOptions<ExtractTools<Options>> & {
      readonly toolkit: Options["toolkit"];
    }): Stream<StreamPart<ExtractTools<Options>, ExtractToolParametersMode<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
  };
}

Persisted

interface unstable

Since v4.0.0

Represents a Chat that is backed by persistence.

Details

When calling a text generation method (e.g. generateText), the previous chat history as well as the relevent response parts will be saved to the backing persistence store.

Signature

interface Persisted extends Chat {
  readonly id: string;
  readonly save: Effect<void, PersistenceError | AiError>;
}

Other

Persistence

unstable

Since v4.0.0

Namespace containing the service contract for chat persistence.

Services

Chat

unstable

Since v4.0.0

Service key for stateful AI conversations.

Signature

declare const Chat: Context.Service<Chat, Chat>

Example Accessing the Chat service

import { Effect, Layer, Stream } from "effect"
import { Chat, LanguageModel } from "effect/ai"

const FakeLanguageModel = Layer.effect(
  LanguageModel.LanguageModel,
  LanguageModel.make({
    generateText: () =>
      Effect.succeed([{
        type: "text",
        text: "Quantum computers use quantum states to process information."
      }]),
    streamText: () => Stream.empty
  })
)

const ChatLayer = Layer.effect(Chat.Chat, Chat.empty)

const program = Effect.gen(function*() {
  const chat = yield* Chat.Chat
  const response = yield* chat.generateText({
    prompt: "Explain quantum computing in simple terms"
  })
  return response.text
})

await Effect.runPromise(
  program.pipe(Effect.provide(Layer.merge(ChatLayer, FakeLanguageModel)))
) // => "Quantum computers use quantum states to process information."

Persistence

unstable

Since v4.0.0

Service tag for persistence-backed AI conversation storage.

When to use

Use to provide the storage operations needed by persisted conversation sessions.

Signature

declare class Persistence extends Shape<"effect/ai/Chat/Persisted", Service, this> {
  constructor(_: never);
}

Type IDs

TypeId

unstable

Since v4.0.0

Brand for Chat implementations.

Signature

declare const TypeId: "~effect/ai/Chat"

TypeId

type unstable

Since v4.0.0

Brand type for Chat.

Signature

type TypeId = "~effect/ai/Chat"