SchemaBinary unstable

A compact binary codec derived from the encoded side of a Schema.

The default wire format supports compatible schema evolution. An array of structs is written as a row run: rows declare their shape once and back-reference repeated strings, so both stay off the wire on later rows. Fingerprint mode uses positional layouts and rejects mismatches for smaller frames; its row shapes are presence masks instead of field id lists. Encoded results are arena-backed views; see toCodec for ownership details.

16 exports Since v4.0.0 Source

Annotations

fieldId

unstable

Since v4.0.0

Assigns an explicit wire field id to a struct property.

When to use

Use to preserve the wire id across a rename or resolve a hash collision.

Details

Valid ids are integers from 1 through 4294967295.

Signature

declare function fieldId(id: number): <S extends Top>(self: S) => S["Rebuild"]

Example Assigning a wire field id

import { Schema } from "effect"
import { SchemaBinary } from "effect/encoding"

const Person = Schema.Struct({
  id: Schema.String.pipe(SchemaBinary.fieldId(1))
})

Channels

decode

unstable

Since v4.0.0

Creates a channel that decodes chunks of binary frames into schema values.

Details

The channel keeps one frame parser for its lifetime: frames may be fragmented across chunks or concatenated within one, values completed before a failure remain observable, and a leftover incomplete frame fails when the upstream is done. Schemas with transformations run the schema pass per framed value, which supports async transformations and decoding services. Failures are Schema.SchemaError, matching toCodec.

Signature

declare function decode<S extends Constraint>(schema: S, options?: ParseOptions & Options & {
  readonly maxFrameSize?: number;
}): <IE = never, Done = unknown>() => Channel<readonly [S["Type"], S["Type"]], SchemaError | IE, Done, readonly [Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>], IE, Done, S["DecodingServices"]>

duplex

unstable

Since v4.0.0

Wraps a bidirectional byte channel with schema-driven binary encoding and decoding.

Details

Values sent to the wrapped channel are encoded with inputSchema as binary frames; bytes received from it are decoded with outputSchema.

Signature

declare const duplex: {
  <In extends Constraint, Out extends Constraint>(options: ParseOptions & Options & {
    readonly inputSchema: In;
    readonly maxFrameSize?: number;
    readonly outputSchema: Out;
  }): <OutErr, OutDone, InErr, InDone, R>(self: Channel<readonly [Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>], OutErr, OutDone, readonly [Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>], SchemaError | InErr, InDone, R>) => Channel<readonly [Out["Type"], Out["Type"]], SchemaError | OutErr, OutDone, readonly [In["Type"], In["Type"]], InErr, InDone, R | In["EncodingServices"] | Out["DecodingServices"]>;
  <Out extends Constraint, In extends Constraint, OutErr, OutDone, InErr, InDone, R>(self: Channel<readonly [Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>], OutErr, OutDone, readonly [Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>], SchemaError | InErr, InDone, R>, options: ParseOptions & Options & {
    readonly inputSchema: In;
    readonly maxFrameSize?: number;
    readonly outputSchema: Out;
  }): Channel<readonly [Out["Type"], Out["Type"]], SchemaError | OutErr, OutDone, readonly [In["Type"], In["Type"]], InErr, InDone, R | In["EncodingServices"] | Out["DecodingServices"]>;
}

encode

unstable

Since v4.0.0

Creates a channel that encodes chunks of schema values into binary frames.

Details

Each input chunk is emitted as one byte element holding concatenated frames written in one writer pass, so a batch costs no more allocations than a single frame. Schemas with transformations first run one schema pass per chunk to the binary-adjusted encoded side — supporting async transformations and encoding services — before that writer pass. Failures are Schema.SchemaError, matching toCodec. maxFrameSize only applies to decode and is ignored here.

Signature

declare function encode<S extends Constraint>(schema: S, options?: ParseOptions & Options & {
  readonly maxFrameSize?: number;
}): <IE = never, Done = unknown>() => Channel<readonly [Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>], SchemaError | IE, Done, readonly [S["Type"], S["Type"]], IE, Done, S["EncodingServices"]>

Constructors

encoder

unstable

Since v4.0.0

A writer for one connection. Every frame it produces shares the string dictionary that the matching parser rebuilds as it reads them, so the two have to be created from the same schema and the same options.

Signature

declare function encoder<S extends Constraint>(schema: S, options?: ParseOptions & Options & StreamOptions): Encoder

parser

unstable

Since v4.0.0

Creates a stateful parser for a stream of concatenated frames.

Details

Values completed before a failure remain observable. After a failure, the parser rejects further calls. Use maxFrameSize to limit buffered frames.

Signature

declare function parser<S extends Constraint>(schema: S, options?: ParseOptions & Options & StreamOptions & {
  readonly maxFrameSize?: number;
}): Parser<S["Type"]>

toCodec

unstable

Since v4.0.0

Derives a compact binary codec from a schema.

Details

The wire layout is compiled from the encoded side of the schema. Each encode/decode handles exactly one frame; use parser for streams. Derived codecs are memoized by schema identity and wire mode.

Encoded results are arena-backed views and may share a larger buffer. Use bytes.slice() when independent ownership is required.

Gotchas

Bytes carry the layout of the schema that wrote them. In the default mode, decode them with that schema or a compatible evolution of it; fingerprint mode requires an identical encoded layout. Most union codecs prefix values with a member selector, so they cannot read bytes written by a member's own codec.

Signature

declare function toCodec<S extends Constraint>(schema: S, options?: Options): toCodec<S>

Example Deriving a binary codec

import { Schema } from "effect"
import { SchemaBinary } from "effect/encoding"

const Person = Schema.Struct({ name: Schema.String, age: Schema.Number })
const codec = SchemaBinary.toCodec(Person)

const bytes = Schema.encodeUnknownSync(codec)({ name: "Ada", age: 36 })
const person = Schema.decodeUnknownSync(codec)(bytes)

Models

Encoder

interface unstable

Since v4.0.0

The writer returned by encoder.

Signature

interface Encoder {
  encode(value: unknown): Uint8Array<ArrayBuffer>;
  encodeMany(values: readonly Array<unknown>): Uint8Array<ArrayBuffer>;
}

Options

interface unstable

Since v4.0.0

Selects the wire mode.

Details

The default mode supports compatible schema evolution. fingerprint: true uses positional layouts and an 8-byte layout hash for smaller frames, but requires peers to use an identical encoded layout.

Signature

interface Options {
  readonly fingerprint?: boolean;
}

Parser

interface unstable

Since v4.0.0

A stateful frame parser for concatenated toCodec outputs.

Signature

interface Parser<T> {
  end: Effect<void, SchemaError>;
  endSync(): void;
  feed(chunk: Uint8Array): Effect<readonly Array<T>, SchemaError>;
  feedSync(chunk: Uint8Array): readonly Array<T>;
}

StreamOptions

interface unstable

Since v4.0.0

Options for the connection-scoped pair, encoder and parser.

Signature

interface StreamOptions {
  readonly dictionary?: boolean;
}

toCodec

interface unstable

Since v4.0.0

The codec type returned by toCodec.

Signature

interface toCodec<S extends Schema.Constraint> extends Codec<S["Type"], Uint8Array<ArrayBuffer>, S["DecodingServices"], S["EncodingServices"]> {
  constructor(_: never);
}

Schemas

int32

unstable

Since v4.0.0

A 32-bit signed integer written as four little-endian bytes.

When to use

Use when values regularly need more than four bytes as a varint, such as hashes or full-range ids. Small values are cheaper as Schema.Int.

Details

Rejects non-integers and values outside -2147483648 through 2147483647 instead of truncating them. -0 is written as 0.

Gotchas

Switching a field between this, uint32, Schema.Number or Schema.Int changes its encoding, so it is not a compatible evolution. In default mode the other reader can decode a different value without an error, for example after a signedness swap or inside an array of structs. Fingerprint mode rejects the layout mismatch.

Signature

declare const int32: Schema.Number

Example Declaring fixed-width integer fields

import { Schema } from "effect"
import { SchemaBinary } from "effect/encoding"

const Packet = Schema.Struct({
  version: SchemaBinary.uint32,
  offset: SchemaBinary.int32,
  sequence: SchemaBinary.uint64,
  timestamp: SchemaBinary.int64
})

int64

unstable

Since v4.0.0

A 64-bit signed integer, as a bigint, written as eight little-endian bytes.

Details

Rejects values outside -2^63 through 2^63 - 1 instead of wrapping them. As a struct field it costs one length byte more than the payload, because the eight-byte field kind belongs to floating-point numbers.

Gotchas

Switching a field between this, uint64 or Schema.BigInt is not a compatible evolution. All three are length-prefixed struct fields, and eight bytes can be valid for each, so a default-mode reader may decode the other form as a different value without an error. Fingerprint mode rejects the layout mismatch.

Signature

declare const int64: Schema.BigInt

uint32

unstable

Since v4.0.0

A 32-bit unsigned integer written as four little-endian bytes.

Details

Rejects non-integers and values outside 0 through 4294967295 instead of wrapping them. See int32 for when a fixed width pays off.

Signature

declare const uint32: Schema.Number

uint64

unstable

Since v4.0.0

A 64-bit unsigned integer, as a bigint, written as eight little-endian bytes.

Details

Rejects values outside 0 through 2^64 - 1 instead of wrapping them. See int64 for the struct field cost.

Gotchas

Switching a field between this, int64 or Schema.BigInt can decode as a different value without an error in default mode; see int64.

Signature

declare const uint64: Schema.BigInt