Skip to content

Introduction to Effect Schema

Welcome to the documentation for effect/Schema, a module for defining and using schemas to validate and transform data in TypeScript.

The effect/Schema module allows you to define a Schema<Type, Encoded, Requirements> that provides a blueprint for describing the structure and data types of your data. Once defined, you can leverage this schema to perform a range of operations, including:

OperationDescription
DecodingTransforming data from an input type Encoded to an output type Type.
EncodingConverting data from an output type Type back to an input type Encoded.
AssertingVerifying that a value adheres to the schema’s output type Type.
ArbitrariesGenerate arbitraries for fast-check testing.
JSON SchemasCreate JSON Schemas based on defined schemas.
EquivalenceCreate Equivalence based on defined schemas.
Pretty printingSupport pretty printing for data structures.
  • TypeScript 5.4 or newer.
  • The strict flag enabled in your tsconfig.json file.
  • (Optional) The exactOptionalPropertyTypes flag enabled in your tsconfig.json file.
{
"compilerOptions": {
"strict": true,
"exactOptionalPropertyTypes": true // optional
}
}

The effect/Schema module takes advantage of the exactOptionalPropertyTypes option of tsconfig.json. This option affects how optional properties are typed (to learn more about this option, you can refer to the official TypeScript documentation).

Example (With exactOptionalPropertyTypes Enabled)

import {
import Schema
Schema
} from "effect"
const
const Person: Schema.Struct<{
name: Schema.optionalWith<Schema.filter<Schema.Schema<string, string, never>>, {
exact: true;
}>;
}>
Person
=
import Schema
Schema
.
function Struct<{
name: Schema.optionalWith<Schema.filter<Schema.Schema<string, string, never>>, {
exact: true;
}>;
}>(fields: {
name: Schema.optionalWith<Schema.filter<Schema.Schema<string, string, never>>, {
exact: true;
}>;
}): Schema.Struct<...> (+1 overload)

@since3.10.0

Struct
({
name: Schema.optionalWith<Schema.filter<Schema.Schema<string, string, never>>, {
exact: true;
}>
name
:
import Schema
Schema
.
const optionalWith: <Schema.filter<Schema.Schema<string, string, never>>, {
exact: true;
}>(self: Schema.filter<Schema.Schema<string, string, never>>, options: {
exact: true;
}) => Schema.optionalWith<...> (+1 overload)

@since3.10.0

optionalWith
(
import Schema
Schema
.
class String
export String

@since3.10.0

String
.
Pipeable.pipe<typeof Schema.String, Schema.filter<Schema.Schema<string, string, never>>>(this: typeof Schema.String, ab: (_: typeof Schema.String) => Schema.filter<Schema.Schema<string, string, never>>): Schema.filter<...> (+21 overloads)
pipe
(
import Schema
Schema
.
const nonEmptyString: <string>(annotations?: Schema.Annotations.Filter<string, string> | undefined) => <I, R>(self: Schema.Schema<string, I, R>) => Schema.filter<Schema.Schema<string, I, R>>

@since3.10.0

nonEmptyString
()), {
exact: true
exact
: true
})
})
type
type Type = {
readonly name?: string;
}
Type
=
import Schema
Schema
.
namespace Schema

@since3.10.0

@since3.10.0

Schema
.
type Schema<in out A, in out I = A, out R = never>.Type<S> = S extends Schema.Schema.Variance<infer A, infer _I, infer _R> ? A : never

@since3.10.0

Type
<typeof
const Person: Schema.Struct<{
name: Schema.optionalWith<Schema.filter<Schema.Schema<string, string, never>>, {
exact: true;
}>;
}>
Person
>
/*
type Type = {
readonly name?: string;
}
*/
// @ts-expect-error
import Schema
Schema
.
decodeSync<{
readonly name?: string;
}, {
readonly name?: string;
}>(schema: Schema.Schema<{
readonly name?: string;
}, {
readonly name?: string;
}, never>, options?: ParseOptions): (i: {
readonly name?: string;
}, overrideOptions?: ParseOptions) => {
readonly name?: string;
}
export decodeSync

@since3.10.0

decodeSync
(
const Person: Schema.Struct<{
name: Schema.optionalWith<Schema.filter<Schema.Schema<string, string, never>>, {
exact: true;
}>;
}>
Person
)({
name?: string
name
:
var undefined
undefined
})
/*
Argument of type '{ name: undefined; }' is not assignable to parameter of type '{ readonly name?: string; }' with 'exactOptionalPropertyTypes: true'. Consider adding 'undefined' to the types of the target's properties.
Types of property 'name' are incompatible.
Type 'undefined' is not assignable to type 'string'.ts(2379)
*/

Here, notice that the type of name is “exact” (string), which means the type checker will catch any attempt to assign an invalid value (like undefined).

Example (With exactOptionalPropertyTypes Disabled)

If, for some reason, you can’t enable the exactOptionalPropertyTypes option (perhaps due to conflicts with other third-party libraries), you can still use effect/Schema. However, there will be a mismatch between the types and the runtime behavior:

import { Schema } from "effect"
const Person = Schema.Struct({
name: Schema.optionalWith(Schema.String.pipe(Schema.nonEmptyString()), {
exact: true
})
})
type Type = Schema.Schema.Type<typeof Person>
/*
type Type = {
readonly name?: string | undefined;
}
*/
// No type error, but a decoding failure occurs
Schema.decodeSync(Person)({ name: undefined })
/*
throws:
ParseError: { readonly name?: a non empty string }
└─ ["name"]
└─ a non empty string
└─ From side refinement failure
└─ Expected string, actual undefined
*/

In this case, the type of name is widened to string | undefined, which means the type checker won’t catch the invalid value (undefined). However, during decoding, you’ll encounter an error, indicating that undefined is not allowed.

The Schema type represents an immutable value that describes the structure of your data.

Here is the general form of a Schema:

┌─── Type of the decoded value
│ ┌─── Encoded type (input/output)
│ │ ┌─── Requirements (context)
▼ ▼ ▼
Schema<Type, Encoded, Requirements>

The Schema type has three type parameters with the following meanings:

ParameterDescription
TypeRepresents the type of value that a schema can succeed with during decoding.
EncodedRepresents the type of value that a schema can succeed with during encoding. By default, it’s equal to Type if not explicitly provided.
RequirementsSimilar to the Effect type, it represents the contextual data required by the schema to execute both decoding and encoding. If this type parameter is never (default if not explicitly provided), it means the schema has no requirements.

Examples

  • Schema<string> (defaulted to Schema<string, string, never>) represents a schema that decodes to string, encodes to string, and has no requirements.
  • Schema<number, string> (defaulted to Schema<number, string, never>) represents a schema that decodes to number from string, encodes a number to a string, and has no requirements.

Immutability. Schema values are immutable, and every function in the effect/Schema mpodule produces a new Schema value.

Modeling Data Structure. These values do not perform any actions themselves, they simply model or describe the structure of your data.

Interpretation by Compilers. A Schema can be interpreted by various “compilers” into specific operations, depending on the compiler type (decoding, encoding, pretty printing, arbitraries, etc…).

When working with data in TypeScript, you often need to handle data coming from or being sent to external systems. This data may not always match the format or types you expect, especially when dealing with user input, data from APIs, or data stored in different formats. To handle these discrepancies, we use decoding and encoding.

TermDescription
DecodingUsed for parsing data from external sources where you have no control over the data format.
EncodingUsed when sending data out to external sources, converting it to a format that is expected by those sources.

For instance, when working with forms in the frontend, you often receive untyped data in the form of strings. This data can be tampered with and does not natively support arrays or booleans. Decoding helps you validate and parse this data into more useful types like numbers, dates, and arrays. Encoding allows you to convert these types back into the string format expected by forms.

Below is a diagram that shows the relationship between encoding and decoding using a Schema<A, I, R>:

┌─────────┐ ┌───┐ ┌───┐ ┌─────────┐
| unknown | | A | | I | | unknown |
└─────────┘ └───┘ └───┘ └─────────┘
| | | |
| validate | | |
|─────────────►│ | |
| | | |
| is | | |
|─────────────►│ | |
| | | |
| asserts | | |
|─────────────►│ | |
| | | |
| encodeUnknown| | |
|─────────────────────────►| |
| | |
| encode | |
|──────────►│ |
| | |
| decode | |
| ◄─────────| |
| | |
| | decodeUnknown|
| ◄────────────────────────|

We’ll break down these concepts using an example with a Schema<Date, string, never>. This schema serves as a tool to transform a string into a Date and vice versa.

When we talk about “encoding,” we are referring to the process of changing a Date into a string. To put it simply, it’s the act of converting data from one format to another.

Conversely, “decoding” entails transforming a string back into a Date. It’s essentially the reverse operation of encoding, where data is returned to its original form.

Decoding from unknown involves two key steps:

  1. Checking: Initially, we verify that the input data (which is of the unknown type) matches the expected structure. In our specific case, this means ensuring that the input is indeed a string.

  2. Decoding: Following the successful check, we proceed to convert the string into a Date. This process completes the decoding operation, where the data is both validated and transformed.

Encoding from unknown involves two key steps:

  1. Checking: Initially, we verify that the input data (which is of the unknown type) matches the expected structure. In our specific case, this means ensuring that the input is indeed a Date.

  2. Encoding: Following the successful check, we proceed to convert the Date into a string. This process completes the encoding operation, where the data is both validated and transformed.

When working with schemas, there’s an important rule to keep in mind: your schemas should be crafted in a way that when you perform both encoding and decoding operations, you should end up with the original value.

In simpler terms, if you encode a value and then immediately decode it, the result should match the original value you started with. This rule ensures that your data remains consistent and reliable throughout the encoding and decoding process.