Introduction to Effect Schema

Welcome to the documentation for effect/Schema, a module for defining and using schemas to validate and transform data in TypeScript.

The effect/Schema module allows you to define a Schema<Type, Encoded, Requirements> that provides a blueprint for describing the structure and data types of your data. Once defined, you can leverage this schema to perform a range of operations, including:

Operation	Description
Decoding	Transforming data from an input type Encoded to an output type Type.
Encoding	Converting data from an output type Type back to an input type Encoded.
Asserting	Verifying that a value adheres to the schema’s output type Type.
Arbitraries	Generate arbitraries for fast-check testing.
JSON Schemas	Create JSON Schemas based on defined schemas.
Equivalence	Create Equivalence based on defined schemas.
Pretty printing	Support pretty printing for data structures.

Requirements

• TypeScript 5.4 or newer.
• The strict flag enabled in your tsconfig.json file.
• (Optional) The exactOptionalPropertyTypes flag enabled in your tsconfig.json file.

{
  "compilerOptions": {
    "strict": true,
    "exactOptionalPropertyTypes": true // optional
  }
}

The exactOptionalPropertyTypes Option

The effect/Schema module takes advantage of the exactOptionalPropertyTypes option of tsconfig.json. This option affects how optional properties are typed (to learn more about this option, you can refer to the official TypeScript documentation).

Example (With exactOptionalPropertyTypes Enabled)

import { Schema } from "effect"

const Person = Schema.Struct({
  name: Schema.optionalWith(Schema.String.pipe(Schema.nonEmptyString()), {
    exact: true
  })
})

type Type = Schema.Schema.Type<typeof Person>
/*
type Type = {
    readonly name?: string;
}
*/

// @ts-expect-error
Schema.decodeSync(Person)({ name: undefined })
/*
Argument of type '{ name: undefined; }' is not assignable to parameter of type '{ readonly name?: string; }' with 'exactOptionalPropertyTypes: true'. Consider adding 'undefined' to the types of the target's properties.
  Types of property 'name' are incompatible.
    Type 'undefined' is not assignable to type 'string'.ts(2379)
*/

Here, notice that the type of name is “exact” (string), which means the type checker will catch any attempt to assign an invalid value (like undefined).

Example (With exactOptionalPropertyTypes Disabled)

If, for some reason, you can’t enable the exactOptionalPropertyTypes option (perhaps due to conflicts with other third-party libraries), you can still use effect/Schema. However, there will be a mismatch between the types and the runtime behavior:

import { Schema } from "effect"

const Person = Schema.Struct({
  name: Schema.optionalWith(Schema.String.pipe(Schema.nonEmptyString()), {
    exact: true
  })
})

type Type = Schema.Schema.Type<typeof Person>
/*
type Type = {
    readonly name?: string | undefined;
}
*/

// No type error, but a decoding failure occurs
Schema.decodeSync(Person)({ name: undefined })
/*
throws:
ParseError: { readonly name?: a non empty string }
└─ ["name"]
   └─ a non empty string
      └─ From side refinement failure
         └─ Expected string, actual undefined
*/

In this case, the type of name is widened to string | undefined, which means the type checker won’t catch the invalid value (undefined). However, during decoding, you’ll encounter an error, indicating that undefined is not allowed.

The Schema Type

The Schema type represents an immutable value that describes the structure of your data.

Here is the general form of a Schema:

         ┌─── Type of the decoded value
         │        ┌─── Encoded type (input/output)
         │        │      ┌─── Requirements (context)
         ▼        ▼      ▼
Schema<Type, Encoded, Requirements>

The Schema type has three type parameters with the following meanings:

Parameter	Description
Type	Represents the type of value that a schema can succeed with during decoding.
Encoded	Represents the type of value that a schema can succeed with during encoding. By default, it’s equal to Type if not explicitly provided.
Requirements	Similar to the Effect type, it represents the contextual data required by the schema to execute both decoding and encoding. If this type parameter is never (default if not explicitly provided), it means the schema has no requirements.

Examples

• Schema<string> (defaulted to Schema<string, string, never>) represents a schema that decodes to string, encodes to string, and has no requirements.
• Schema<number, string> (defaulted to Schema<number, string, never>) represents a schema that decodes to number from string, encodes a number to a string, and has no requirements.

Type Parameter Abbreviations

In the Effect ecosystem, you may often encounter the type parameters of Schema abbreviated as A, I, and R respectively. This is just shorthand for the type value of type A, Input, and Requirements.

Understanding Schema Values

Immutability. Schema values are immutable, and every function in the effect/Schema module produces a new Schema value.

Modeling Data Structure. These values do not perform any actions themselves, they simply model or describe the structure of your data.

Interpretation by Compilers. A Schema can be interpreted by various “compilers” into specific operations, depending on the compiler type (decoding, encoding, pretty printing, arbitraries, etc…).

Understanding Decoding and Encoding

When working with data in TypeScript, you often need to handle data coming from or being sent to external systems. This data may not always match the format or types you expect, especially when dealing with user input, data from APIs, or data stored in different formats. To handle these discrepancies, we use decoding and encoding.

Term	Description
Decoding	Used for parsing data from external sources where you have no control over the data format.
Encoding	Used when sending data out to external sources, converting it to a format that is expected by those sources.

For instance, when working with forms in the frontend, you often receive untyped data in the form of strings. This data can be tampered with and does not natively support arrays or booleans. Decoding helps you validate and parse this data into more useful types like numbers, dates, and arrays. Encoding allows you to convert these types back into the string format expected by forms.

Below is a diagram that shows the relationship between encoding and decoding using a Schema<A, I, R>:

┌─────────┐       ┌───┐       ┌───┐       ┌─────────┐
| unknown |       | A |       | I |       | unknown |
└─────────┘       └───┘       └───┘       └─────────┘
     |              |           |              |
     | validate     |           |              |
     |─────────────►│           |              |
     |              |           |              |
     | is           |           |              |
     |─────────────►│           |              |
     |              |           |              |
     | asserts      |           |              |
     |─────────────►│           |              |
     |              |           |              |
     | encodeUnknown|           |              |
     |─────────────────────────►|              |
                    |           |              |
                    | encode    |              |
                    |──────────►│              |
                    |           |              |
                    |    decode |              |
                    | ◄─────────|              |
                    |           |              |
                    |           | decodeUnknown|
                    | ◄────────────────────────|

We’ll break down these concepts using an example with a Schema<Date, string, never>. This schema serves as a tool to transform a string into a Date and vice versa.

Encoding

When we talk about “encoding,” we are referring to the process of changing a Date into a string. To put it simply, it’s the act of converting data from one format to another.

Decoding

Conversely, “decoding” entails transforming a string back into a Date. It’s essentially the reverse operation of encoding, where data is returned to its original form.

Decoding From Unknown

Decoding from unknown involves two key steps:

1. Checking: Initially, we verify that the input data (which is of the unknown type) matches the expected structure. In our specific case, this means ensuring that the input is indeed a string.

2. Decoding: Following the successful check, we proceed to convert the string into a Date. This process completes the decoding operation, where the data is both validated and transformed.

Encoding From Unknown

Encoding from unknown involves two key steps:

1. Checking: Initially, we verify that the input data (which is of the unknown type) matches the expected structure. In our specific case, this means ensuring that the input is indeed a Date.

2. Encoding: Following the successful check, we proceed to convert the Date into a string. This process completes the encoding operation, where the data is both validated and transformed.

The Rule of Schemas

When working with schemas, there’s an important rule to keep in mind: your schemas should be crafted in a way that when you perform both encoding and decoding operations, you should end up with the original value.

In simpler terms, if you encode a value and then immediately decode it, the result should match the original value you started with. This rule ensures that your data remains consistent and reliable throughout the encoding and decoding process.

Ensure Consistency

As a general rule, schemas should be defined such that encode + decode return the original value.