LanguageModel

Defines the shared service for language model providers.

The LanguageModel service lets application code ask for generated text, streamed text, or structured output without depending on a specific provider. Requests can include tools, and the service can resolve tool calls while the model is generating a response. This module contains the service contract, request and response types, structured-output support, and the constructor used by provider packages to adapt their own generate and stream functions to the shared interface.

20 exports Added in v4.0.0 Source

Constructors

make

Added in v4.0.0 Source

Creates a LanguageModel service from provider-specific text generation and streaming implementations.

When to use

Use when you are implementing a provider adapter and need to expose the standard language-model service while keeping provider-specific request hooks behind it.

Details

The returned service implements generateText, generateObject, and streamText. It prepares ProviderOptions for each request, including the normalized prompt, tools, tool choice, response format, tracing span, and incremental response fields, before calling the supplied provider hook. Structured object generation uses the generateText hook and the configured codecTransformer, or defaultCodecTransformer when none is supplied.

Gotchas

Provider hooks must return encoded response parts that match the toolkit and response format prepared in ProviderOptions; invalid parts fail decoding as AiError.InvalidOutputError.

See

Service for the returned service contract
ProviderOptions for the normalized options passed to provider hooks
defaultCodecTransformer for the default structured-output schema transformer

Signature

declare const make: (params: {
  readonly codecTransformer?: CodecTransformer;
  readonly generateText: (options: ProviderOptions) => Effect.Effect<Array<Response.PartEncoded>, AiError.AiError, IdGenerator>;
  readonly streamText: (options: ProviderOptions) => Stream.Stream<Response.StreamPartEncoded, AiError.AiError, IdGenerator>;
}) => Effect.Effect<Service>

Generators

generateObject

Added in v4.0.0 Source

Generates a structured object from a schema using a language model.

Signature

declare function generateObject<ObjectEncoded extends Record<string, any>, StructuredOutputSchema extends Encoder<ObjectEncoded, unknown>, Options extends NoExcessProperties<GenerateObjectOptions<any, StructuredOutputSchema>, Options>>(options: Options & GenerateObjectOptions<ExtractTools<Options>, StructuredOutputSchema>): Effect<GenerateObjectResponse<ExtractTools<Options>, StructuredOutputSchema["Type"], ExtractEncodedToolParameters<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options> | StructuredOutputSchema["DecodingServices"]>

Example

(Generating an object)

import { Effect, Layer, Schema, Stream } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const EventSchema = Schema.Struct({
  title: Schema.String,
  date: Schema.String,
  location: Schema.String
})

const FakeLanguageModel = Layer.effect(
  LanguageModel.LanguageModel,
  LanguageModel.make({
    generateText: () =>
      Effect.succeed([{
        type: "text",
        text: '{"title":"Tech Conference","date":"March 15th","location":"San Francisco"}'
      }]),
    streamText: () => Stream.empty
  })
)

const program = LanguageModel.generateObject({
  prompt: "Extract event info: Tech Conference on March 15th in San Francisco",
  schema: EventSchema,
  objectName: "event"
}).pipe(
  Effect.map((response) => response.value),
  Effect.provide(FakeLanguageModel)
)

await Effect.runPromise(program) // => { title: "Tech Conference", date: "March 15th", location: "San Francisco" }

Models

GenerateObjectResponse

Added in v4.0.0 Source

Response class for structured object generation operations.

Signature

declare class GenerateObjectResponse<Tools extends Record<string, Tool.Any>, A, EncodedToolParameters extends boolean = false> extends GenerateTextResponse<Tools, EncodedToolParameters> {
  constructor<Tools extends Record<string, Any>, A, EncodedToolParameters extends boolean = false>(value: A, content: Array<Part<Tools, EncodedToolParameters>>);
  readonly value: A;
}

Example

(Inspecting an object response)

import { LanguageModel, Response } from "effect/unstable/ai"

const response = new LanguageModel.GenerateObjectResponse(
  { name: "John Doe", email: "john@example.com" },
  [Response.makePart("text", { text: '{"name":"John Doe","email":"john@example.com"}' })]
)

response.value // => { name: "John Doe", email: "john@example.com" }
response.text // => '{"name":"John Doe","email":"john@example.com"}'

GenerateTextResponse

Added in v4.0.0 Source

Response class for text generation operations, with accessors for extracting text, tool calls, usage information, and other response parts from generated content.

Signature

declare class GenerateTextResponse<Tools extends Record<string, Tool.Any>, EncodedToolParameters extends boolean = false> {
  constructor<Tools extends Record<string, Any>, EncodedToolParameters extends boolean = false>(content: Array<Part<Tools, EncodedToolParameters>>);
  readonly content: Array<Part<Tools, EncodedToolParameters>>;
  finishReason: "length" | "error" | "stop" | "content-filter" | "tool-calls" | "pause" | "other" | "unknown";
  reasoning: Array<ReasoningPart>;
  reasoningText: string | undefined;
  text: string;
  toolCalls: Array<ToolCallParts<Tools, EncodedToolParameters>>;
  toolResults: Array<ToolResultParts<Tools>>;
  usage: Usage;
}

Example

(Inspecting a text response)

import { LanguageModel, Response } from "effect/unstable/ai"

const response = new LanguageModel.GenerateTextResponse([
  Response.makePart("text", { text: "Plants convert light into energy." })
])

const result = [response.text, response.finishReason] // => ["Plants convert light into energy.", "unknown"]

Service interface

Added in v4.0.0 Source

The service interface for language model operations, defining the contract that all language model implementations must fulfill.

Signature

interface Service {
  readonly generateObject: <ObjectEncoded extends Record<string, any>, StructuredOutputSchema extends Encoder<ObjectEncoded, unknown>, Options extends NoExcessProperties<GenerateObjectOptions<any, StructuredOutputSchema>, Options>, Tools extends Record<string, Any> = {}>(options: Options & GenerateObjectOptions<Tools, StructuredOutputSchema>) => Effect<GenerateObjectResponse<Tools, StructuredOutputSchema["Type"], ExtractEncodedToolParameters<Options>>, ExtractError<Options>, ExtractServices<Options> | StructuredOutputSchema["DecodingServices"]>;
  readonly generateText: {
    <Options extends NoExcessProperties<GenerateTextOptionsWithoutToolkit, Options>>(options: Options & Omit<GenerateTextOptions<{}>, "toolkit"> & {
      readonly toolkit?: undefined;
    }): Effect<GenerateTextResponse<{}, false>, ExtractError<Options>, ExtractServices<Options>>;
    <Tools extends Record<string, Any>, Options extends NoExcessProperties<GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }, Options>>(options: Options & GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }): Effect<GenerateTextResponse<Tools, ExtractEncodedToolParameters<Options>>, ExtractError<Options>, ExtractServices<Options>>;
    <Options extends {
      readonly toolkit: WithHandler<any> | Effect<WithHandler<any>, never, any>;
    } & GenerateTextOptions<any> & Readonly<Record<Exclude<keyof Options, keyof GenerateTextOptions<any>>, never>>>(options: Options & GenerateTextOptions<ExtractTools<Options>> & {
      readonly toolkit: Options["toolkit"];
    }): Effect<GenerateTextResponse<ExtractTools<Options>, ExtractEncodedToolParameters<Options>>, ExtractError<Options>, ExtractServices<Options>>;
  };
  readonly streamText: {
    <Options extends NoExcessProperties<GenerateTextOptionsWithoutToolkit, Options>>(options: Options & Omit<GenerateTextOptions<{}>, "toolkit"> & {
      readonly toolkit?: undefined;
    }): Stream<StreamPart<{}, false>, ExtractError<Options>, ExtractServices<Options>>;
    <Tools extends Record<string, Any>, Options extends NoExcessProperties<GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }, Options>>(options: Options & GenerateTextOptions<Tools> & {
      readonly toolkit: ToolkitInput<Tools>;
    }): Stream<StreamPart<Tools, ExtractEncodedToolParameters<Options>>, ExtractError<Options>, ExtractServices<Options>>;
    <Options extends {
      readonly toolkit: WithHandler<any> | Effect<WithHandler<any>, never, any>;
    } & GenerateTextOptions<any> & Readonly<Record<Exclude<keyof Options, keyof GenerateTextOptions<any>>, never>>>(options: Options & GenerateTextOptions<ExtractTools<Options>> & {
      readonly toolkit: Options["toolkit"];
    }): Stream<StreamPart<ExtractTools<Options>, ExtractEncodedToolParameters<Options>>, ExtractError<Options>, ExtractServices<Options>>;
  };
}

ToolChoice type

Added in v4.0.0 Source

The tool choice mode for the language model.

auto (default): The model can decide whether or not to call tools, as well as which tools to call.
required: The model must call a tool but can decide which tool will be called.
none: The model must not call a tool.
{ tool: <tool_name> }: The model must call the specified tool.
{ mode?: "auto" (default) | "required", "oneOf": [<tool-names>] }: The model is restricted to the subset of tools specified by oneOf. When mode is "auto" or omitted, the model can decide whether or not a tool from the allowed subset of tools can be called. When mode is "required", the model must call one tool from the allowed subset of tools.

Signature

type ToolChoice<ToolName extends string> = "auto" | "none" | "required" | {
  readonly tool: ToolName;
} | {
  readonly mode?: "auto" | "required";
  readonly oneOf: ReadonlyArray<ToolName>;
}

Options

GenerateObjectOptions interface

Added in v4.0.0 Source

Configuration options for structured object generation.

Signature

interface GenerateObjectOptions<Tools extends Record<string, Tool.Any>, StructuredOutputSchema extends Schema.Top> extends GenerateTextOptions<Tools> {
  readonly objectName?: string;
  readonly schema: StructuredOutputSchema;
}

GenerateTextOptions interface

Added in v4.0.0 Source

Configuration options for text generation.

Signature

interface GenerateTextOptions<Tools extends Record<string, Tool.Any>> {
  readonly concurrency?: Concurrency;
  readonly disableToolCallResolution?: boolean;
  readonly prompt: RawInput;
  readonly toolChoice?: ToolChoice<{ [Name in string | number | symbol]: Tools[Name]["name"] }[keyof Tools]>;
  readonly toolkit?: ToolkitInput<Tools, never, any>;
}

ProviderOptions interface

Added in v4.0.0 Source

Configuration options passed along to language model provider implementations.

Details

This interface defines the normalized options that are passed to the underlying provider implementation, regardless of the specific provider being used.

Signature

interface ProviderOptions {
  readonly incrementalPrompt: Prompt | undefined;
  readonly previousResponseId: string | undefined;
  readonly prompt: Prompt;
  readonly responseFormat: {
    readonly type: "text";
  } | {
    readonly objectName: string;
    readonly schema: Top;
    readonly type: "json";
  };
  readonly span: Span;
  readonly toolChoice: ToolChoice<any>;
  readonly tools: readonly Array<Any>;
}

Services

defaultCodecTransformer

Added in v4.0.0 Source

The default codec transformer that passes schemas through without provider-specific rewrites.

When to use

Use as the codec transformer for provider implementations when the provider accepts the JSON Schema generated from an Effect Schema codec without provider-specific rewrites.

Details

The transformer returns the original codec, resolves a top-level $ref, and copies schema definitions into $defs.

See

CodecTransformer for the structured-output transformer contract
make for where this transformer is used as the default

Signature

declare const defaultCodecTransformer: CodecTransformer

LanguageModel

Added in v4.0.0 Source

Service tag for AI model services.

When to use

Use to access or provide text generation, streaming generation, structured output, and tool-calling capabilities through the Effect context.

Signature

declare class LanguageModel extends Shape<"effect/unstable/ai/LanguageModel", Service, this> {
  constructor(_: never);
}

Example

(Accessing the language model service)

import { Effect, Layer, Stream } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const FakeLanguageModel = Layer.effect(
  LanguageModel.LanguageModel,
  LanguageModel.make({
    generateText: () =>
      Effect.succeed([{
        type: "text",
        text: "Machine learning finds patterns in data."
      }]),
    streamText: () => Stream.empty
  })
)

const program = Effect.gen(function*() {
  const model = yield* LanguageModel.LanguageModel
  const response = yield* model.generateText({
    prompt: "What is machine learning?"
  })
  return response.text
})

await Effect.runPromise(program.pipe(Effect.provide(FakeLanguageModel))) // => "Machine learning finds patterns in data."

Text Generation

generateText

Added in v4.0.0 Source

Generates text using a language model.

Signature

declare const generateText: {
  <Options extends NoExcessProperties<GenerateTextOptionsWithoutToolkit, Options>>(options: Options & Omit<GenerateTextOptions<{}>, "toolkit"> & {
    readonly toolkit?: undefined;
  }): Effect<GenerateTextResponse<{}, false>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
  <Tools extends Record<string, Any>, Options extends NoExcessProperties<GenerateTextOptions<Tools> & {
    readonly toolkit: ToolkitInput<Tools>;
  }, Options>>(options: Options & GenerateTextOptions<Tools> & {
    readonly toolkit: ToolkitInput<Tools>;
  }): Effect<GenerateTextResponse<Tools, ExtractEncodedToolParameters<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
  <Options extends {
    readonly toolkit: ToolkitOption<any>;
  } & GenerateTextOptions<any> & Readonly<Record<Exclude<keyof Options, keyof GenerateTextOptions<any>>, never>>>(options: Options & GenerateTextOptions<ExtractTools<Options>> & {
    readonly toolkit: Options["toolkit"];
  }): Effect<GenerateTextResponse<ExtractTools<Options>, ExtractEncodedToolParameters<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
}

Example

(Generating text with options)

import { Effect, Layer, Stream } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const FakeLanguageModel = Layer.effect(
  LanguageModel.LanguageModel,
  LanguageModel.make({
    generateText: (options) =>
      Effect.succeed([
        {
          type: "text",
          text: options.toolChoice === "none"
            ? "Code flows through types / Errors become values / Programs stay composed"
            : "Unexpected tool choice"
        },
        {
          type: "finish",
          reason: "stop",
          usage: {
            inputTokens: { total: 6 },
            outputTokens: { total: 12 }
          }
        }
      ]),
    streamText: () => Stream.empty
  })
)

const program = LanguageModel.generateText({
  prompt: "Write a haiku about programming",
  toolChoice: "none"
}).pipe(
  Effect.map((response) => [response.text, response.usage.inputTokens.total]),
  Effect.provide(FakeLanguageModel)
)

await Effect.runPromise(program) // => ["Code flows through types / Errors become values / Programs stay composed", 6]

streamText

Added in v4.0.0 Source

Generates text using a language model with streaming output.

Details

Returns a stream of response parts that are emitted as soon as they are available from the model, enabling real-time text generation experiences.

Signature

declare const streamText: {
  <Options extends NoExcessProperties<GenerateTextOptionsWithoutToolkit, Options>>(options: Options & Omit<GenerateTextOptions<{}>, "toolkit"> & {
    readonly toolkit?: undefined;
  }): Stream<StreamPart<{}, false>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
  <Tools extends Record<string, Any>, Options extends NoExcessProperties<GenerateTextOptions<Tools> & {
    readonly toolkit: ToolkitInput<Tools>;
  }, Options>>(options: Options & GenerateTextOptions<Tools> & {
    readonly toolkit: ToolkitInput<Tools>;
  }): Stream<StreamPart<Tools, ExtractEncodedToolParameters<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
  <Options extends {
    readonly toolkit: ToolkitOption<any>;
  } & GenerateTextOptions<any> & Readonly<Record<Exclude<keyof Options, keyof GenerateTextOptions<any>>, never>>>(options: Options & GenerateTextOptions<ExtractTools<Options>> & {
    readonly toolkit: Options["toolkit"];
  }): Stream<StreamPart<ExtractTools<Options>, ExtractEncodedToolParameters<Options>>, ExtractError<Options>, LanguageModel | ExtractServices<Options>>;
}

Example

(Streaming text deltas)

import { Effect, Layer, Stream } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const FakeLanguageModel = Layer.effect(
  LanguageModel.LanguageModel,
  LanguageModel.make({
    generateText: () => Effect.succeed([]),
    streamText: () =>
      Stream.make(
        { type: "text-delta", id: "story", delta: "The explorer reached orbit." },
        { type: "text-delta", id: "story", delta: " Earth glowed below." }
      )
  })
)

const program = LanguageModel.streamText({
  prompt: "Write a story about a space explorer"
}).pipe(
  Stream.runFold(() => "", (text, part) => part.type === "text-delta" ? text + part.delta : text),
  Effect.provide(FakeLanguageModel)
)

await Effect.runPromise(program) // => "The explorer reached orbit. Earth glowed below."

Utility Types

CodecTransformer type

Added in v4.0.0 Source

A function that transforms a Schema.Codec into a provider-compatible form for structured output generation.

Details

Different language model providers have varying constraints on the JSON schemas they accept. A CodecTransformer rewrites a codec's encoded side to satisfy those constraints while preserving the decoded type. A provider schema may be less restrictive than the codec when the provider cannot express every constraint; the returned codec remains authoritative for validating model output.

Signature

type CodecTransformer = <T, E, RD, RE>(schema: Schema.ConstraintCodec<T, E, RD, RE>) => {
  readonly codec: Schema.ConstraintCodec<T, unknown, RD, RE>;
  readonly jsonSchema: JsonSchema.JsonSchema;
}

ExtractEncodedToolParameters type

Added in v4.0.0 Source

Utility type that determines whether language model responses contain encoded tool call parameters.

Signature

type ExtractEncodedToolParameters<Options> = Options extends {
  readonly disableToolCallResolution: true;
} ? true : false

ExtractError type

Added in v4.0.0 Source

Utility type that extracts the error type from LanguageModel options.

Details

Automatically infers the possible error types based on toolkit configuration and tool call resolution settings.

Signature

type ExtractError<Options> = Options extends {
  readonly disableToolCallResolution: true;
  readonly toolkit: infer ToolkitValue;
} ? ExtractErrorFromToolkitOption<Exclude<ToolkitValue, undefined>, true> : Options extends {
  readonly toolkit: infer ToolkitValue;
} ? ExtractErrorFromToolkitOption<Exclude<ToolkitValue, undefined>, false> : Options extends {
  readonly disableToolCallResolution: true;
} ? AiError.AiError : AiError.AiError

ExtractServices type

Added in v4.0.0 Source

Utility type that extracts the context requirements from LanguageModel options.

Details

Automatically infers the required services based on the toolkit configuration.

Signature

type ExtractServices<Options> = Options extends {
  readonly disableToolCallResolution: true;
} ? never : Options extends {
  readonly toolkit: infer Toolkit;
} ? ExtractServicesFromToolkitOption<Exclude<Toolkit, undefined>> : never

ExtractTools type

Added in v4.0.0 Source

Utility type that extracts the toolset from LanguageModel options.

Signature

type ExtractTools<Options> = Options extends {
  readonly toolkit: infer ToolkitValue;
} ? ExtractToolsFromToolkitOption<Exclude<ToolkitValue, undefined>> : {}

ToolkitInput type

Added in v4.0.0 Source

The supported toolkit input shapes for language model operation options.

Details

Unlike ToolkitOption, this type does not distribute over unions. It is intended for call-site assignability, while ToolkitOption remains the distributive helper used for extraction and inference.

Signature

type ToolkitInput<Tools extends Record<string, Tool.Any>, E = never, R = any> = ToolkitOption<Tools, E, R> | Toolkit.WithHandler<Tools> | Effect.Effect<Toolkit.WithHandler<Tools>, E, R>

ToolkitOption type

Added in v4.0.0 Source

The supported toolkit option shapes for language model operations.

Signature

type ToolkitOption<Tools extends Record<string, Tool.Any>, E = never, R = any> = Tools extends any ? Toolkit.WithHandler<Tools> | Effect.Effect<Toolkit.WithHandler<Tools>, E, R> : never