Skip to content
// EFFECT DAYS 2026

EventLogServerEncrypted unstable

Serves encrypted event-log replication.

Encrypted EventLogRemote clients use this module when they need a remote synchronization endpoint that never sees plaintext events. The server stores encrypted entries and replication metadata keyed by client public key and store id, then streams encrypted changes back to clients for local decryption. This module defines the RPC handlers, server layer, storage contract, and in-memory storage layer for that encrypted server path.

18 exports Since v4.0.0 Source

Constructors

makeCompactionConfig

unstable
Since v4.0.0

Resolves compaction options against their defaults.

Details

Dies with an IllegalArgumentError unless grace is longer than both lag and stageTtl, stageTtl and sweepInterval are positive, and thresholdFloor is a non-negative integer.

Signature

declare function makeCompactionConfig(options?: CompactionOptions): Effect<CompactionConfig>

makeStorageMemory

unstable
Since v4.0.0

Creates an in-memory encrypted server Storage with default compaction tuning.

Details

Data, session authentication bindings, and streams are process-local and are released with the surrounding scope.

See

Signature

declare const makeStorageMemory: Effect.Effect<Storage["Service"], never, Scope.Scope>

makeStorageMemoryWith

unstable
Since v4.0.0

Creates an in-memory encrypted server Storage.

Details

Data, session authentication bindings, and streams are process-local and are released with the surrounding scope. The storage always provides compaction. Its sweep runs on Clock only while some store has a generation to prune or a stage to expire.

Dies when the compaction options are invalid (see makeCompactionConfig).

Signature

declare const makeStorageMemoryWith: (...args: [options: {
  readonly compaction?: CompactionOptions;
}]) => Effect<{
  readonly changes: (publicKey: string, storeId: StoreId, startSequence: number) => Stream<EncryptedRemoteBatch>;
  readonly compaction?: StorageCompaction;
  readonly entrySequences: (publicKey: string, storeId: StoreId, entryIds: readonly Array<EntryId>) => Effect<readonly Array<{
    readonly entryId: Uint8Array<ArrayBuffer> & Brand<"effect/eventlog/EventJournal/EntryId">;
    readonly sequence: number;
  }>>;
  readonly firstSequence?: number;
  readonly getId: Effect<RemoteId>;
  readonly getOrCreateSessionAuthBinding: (publicKey: string, signingPublicKey: Uint8Array<ArrayBuffer>) => Effect<Uint8Array<ArrayBuffer>>;
  readonly write: (publicKey: string, storeId: StoreId, batches: readonly Array<EncryptedBatch>) => Effect<readonly Array<EncryptedRemoteBatch>>;
}, never, Scope>

Errors

CompactionInvalidRequest

unstable
Since v4.0.0

Error raised when a compaction Part fails validation.

Signature

declare class CompactionInvalidRequest extends YieldableError<this> & {
  readonly _tag: "CompactionInvalidRequest";
} & Readonly<{
  readonly message: string;
  readonly stageId: string;
}> {
  constructor(args: {
    readonly message: string;
    readonly stageId: string;
  });
}

CompactionNotFound

unstable
Since v4.0.0

Error raised when a compaction stage has expired, was aborted or committed, or is unknown to the store.

Signature

declare class CompactionNotFound extends YieldableError<this> & {
  readonly _tag: "CompactionNotFound";
} & Readonly<{
  readonly stageId: string;
}> {
  constructor(args: {
    readonly stageId: string;
  });
}

CompactionRollbackError

unstable
Since v4.0.0

Error raised when a rollback is refused.

Details

NotActive means the expected activation is not the active one, NoPrevious that there is no earlier generation, and PruneStarted that the previous generation has started to be pruned.

Signature

declare class CompactionRollbackError extends YieldableError<this> & {
  readonly _tag: "CompactionRollbackError";
} & Readonly<{
  readonly reason: "NotActive" | "NoPrevious" | "PruneStarted";
}> {
  constructor(args: {
    readonly reason: "NotActive" | "NoPrevious" | "PruneStarted";
  });
}

Layers

layer

unstable
Since v4.0.0

Provides an encrypted event-log RPC server using EventLogRemoteRpcs and the encrypted server RPC handlers.

When to use

Use when you need an encrypted event-log RPC server for encrypted EventLogRemote replication over an existing RpcServer.Protocol.

Details

This layer installs EventLogRemoteRpcs on the provided RPC server protocol and wires those RPCs to layerRpcHandlers. Encrypted batches, session authentication bindings, remote ids, and change streams are delegated to Storage. It also installs EventLogCompactionRpcs, which clients call only when Storage provides compaction.

See

  • layerRpcHandlers for the encrypted handler layer without installing an RPC server protocol
  • Storage for the storage service required by this layer
  • layerStorageMemory for the process-local in-memory storage layer

Signature

declare const layer: Layer.Layer<never, never, RpcServer.Protocol | Storage>

layerRpcHandlers

unstable
Since v4.0.0

Provides RPC handlers for the encrypted event-log server.

Details

Decodes and persists writes through Storage, acknowledging their stored sequences. Change streams encode stored batches, excluding this connection's writes.

It also serves EventLogCompactionRpcs. If Storage provides compaction, it sets HelloResponse.compaction and applies the compaction policy; otherwise those handlers fail with NotFound.

Signature

declare const layerRpcHandlers: Layer.Layer<Rpc.ToHandler<RpcGroup.Rpcs<typeof EventLogRemoteRpcs>> | Rpc.ToHandler<RpcGroup.Rpcs<typeof EventLogCompactionRpcs>> | EventLogAuthentication, never, Storage>

layerStorageMemory

unstable
Since v4.0.0

Provides encrypted server Storage using the in-memory implementation with default compaction tuning.

See

Signature

declare const layerStorageMemory: Layer.Layer<Storage>

layerStorageMemoryWith

unstable
Since v4.0.0

Provides encrypted server Storage using the in-memory implementation.

Signature

declare function layerStorageMemoryWith(options: {
  readonly compaction?: CompactionOptions;
}): Layer<Storage>

Models

CompactionActivation

interface unstable
Since v4.0.0

One entry of a store's activation log.

Details

keptCount is undefined for the originals. restorable is false once the generation's prune has started.

Signature

interface CompactionActivation {
  readonly activationId: string;
  readonly active: boolean;
  readonly base: number;
  readonly keptCount: number | undefined;
  readonly restorable: boolean;
  readonly supersededAt: number | undefined;
}

CompactionBatchHeader

interface unstable
Since v4.0.0

The header of one compaction input batch.

Signature

interface CompactionBatchHeader {
  readonly entryIds: readonly Array<EntryId>;
  readonly sequence: number;
}

CompactionBeginResult

type unstable
Since v4.0.0

The result of opening a compaction stage.

Signature

type CompactionBeginResult = {
  readonly _tag: "Started";
  readonly stageId: string;
} | {
  readonly _tag: "Busy";
  readonly expiresAt: number;
}

CompactionConfig

interface unstable
Since v4.0.0

Resolved compaction tuning, shared by a storage and the server using it.

Signature

interface CompactionConfig {
  readonly grace: Duration;
  readonly lag: Duration;
  readonly stageTtl: Duration;
  readonly sweepInterval: Duration;
  readonly thresholdFloor: number;
}

CompactionOptions

interface unstable
Since v4.0.0

Tuning for encrypted remote compaction.

Details

grace is how long a superseded or withdrawn generation is kept before it is pruned (default 30 days). lag is how old data must be, by storage time, before it can be compacted (default 24 hours). stageTtl is how long a stage claim lasts after its latest Part (default 10 minutes). thresholdFloor is the smallest number of new entries that makes compaction wanted (default 1,000). sweepInterval is how often pruning runs (default 1 hour).

Signature

interface CompactionOptions {
  readonly grace?: Input;
  readonly lag?: Input;
  readonly stageTtl?: Input;
  readonly sweepInterval?: Input;
  readonly thresholdFloor?: number;
}

CompactionStoreStatus

interface unstable
Since v4.0.0

The compaction state of one store.

Details

base is firstSequence - 1 while the originals are active. lagHead is the highest last sequence in a storage-time hour that ended before now - lag.

Signature

interface CompactionStoreStatus {
  readonly activeKeptCount: number;
  readonly base: number;
  readonly head: number;
  readonly lagHead: number;
  readonly noOp: {
    readonly at: number;
    readonly count: number;
  } | undefined;
}

StorageCompaction

interface unstable
Since v4.0.0

Storage primitives for encrypted remote compaction.

Details

The server owns the policy: trigger threshold, lag, no-op backoff, commit checks and virtual sequences. Each primitive is atomic per store.

A store always has one active activation. Before the first commit it is the originals, with base firstSequence - 1. begin opens a stage pinned to the active activation and to through. The stage claim expires stageTtl after begin or the latest stored Part, and begin takes over an expired claim. Stage-scoped primitives fail with CompactionNotFound once the stage has expired, been aborted or committed, or when it belongs to another store, so a commit is answered only once.

stagePart stores one segment of encrypted snapshot batches, or answers AlreadyStored for a segment it already has. It doesn't check entry ids: the server's commit checks reject repeats, and readers dedupe by id. commit takes every staged batch's virtual start sequence in position order, and activates the staged set only if the pinned activation is still active.

rollback reactivates the previous generation under a new activation id and re-appends, at the head, the input of the stage that built the withdrawn one: the previous batch set, then the originals between the two bases. Clients that read the withdrawn snapshot get what it dropped through changes. A sweep on Clock prunes a generation, and the originals it replaced, grace after it was superseded or withdrawn.

Signature

interface StorageCompaction {
  readonly abort: (publicKey: string, storeId: StoreId, stageId: string) => Effect<void>;
  readonly begin: (publicKey: string, storeId: StoreId, options: {
    readonly through: number;
  }) => Effect<CompactionBeginResult>;
  readonly commit: (publicKey: string, storeId: StoreId, stageId: string, virtualSequences: readonly Array<number>) => Effect<"Committed" | "Superseded", CompactionNotFound>;
  readonly config: CompactionConfig;
  readonly fetch: (publicKey: string, storeId: StoreId, stageId: string, sequences: readonly Array<number>) => Stream<EncryptedRemoteBatch, CompactionNotFound>;
  readonly history: (publicKey: string, storeId: StoreId) => Effect<readonly Array<CompactionActivation>>;
  readonly inputHeaders: (publicKey: string, storeId: StoreId, stageId: string) => Stream<CompactionBatchHeader, CompactionNotFound>;
  readonly recordNoOp: (publicKey: string, storeId: StoreId, stageId: string) => Effect<void, CompactionNotFound>;
  readonly rollback: (publicKey: string, storeId: StoreId, expectedActivationId: string) => Effect<string, CompactionRollbackError>;
  readonly stagedHeaders: (publicKey: string, storeId: StoreId, stageId: string) => Stream<{
    readonly entryIds: readonly Array<EntryId>;
    readonly position: number;
  }, CompactionNotFound>;
  readonly stagePart: (publicKey: string, storeId: StoreId, stageId: string, segment: number, batches: readonly [EncryptedBatch, EncryptedBatch]) => Effect<"Stored" | "AlreadyStored", CompactionNotFound | CompactionInvalidRequest>;
  readonly status: (publicKey: string, storeId: StoreId) => Effect<CompactionStoreStatus>;
}

Services

Storage

unstable
Since v4.0.0

Defines the backing store service used by the encrypted event-log server.

When to use

Use to provide durable encrypted event-log persistence for an encrypted event-log server layer.

Details

It provides the server remote id, stores session authentication bindings, persists encrypted batches, and streams encrypted changes for a public key and store id.

A batch owns one remote sequence per entry, starting at its sequence. write stores a batch unless every one of its entry ids is already stored, which makes retried writes idempotent, and returns the batches it stored. entrySequences returns the sequence each entry was first stored at, so skipped batches can still be acknowledged. changes streams every batch whose last sequence is at or after startSequence, followed by newly stored batches.

Storages that support remote compaction provide compaction. With an active snapshot, changes serves snapshot batches by their virtual sequences for a start sequence at or below the snapshot base, then the originals above it.

Signature

declare class Storage extends Shape<"effect/eventlog/EventLogServer/Storage", {
  readonly changes: (publicKey: string, storeId: StoreId, startSequence: number) => Stream<EncryptedRemoteBatch>;
  readonly compaction?: StorageCompaction;
  readonly entrySequences: (publicKey: string, storeId: StoreId, entryIds: readonly Array<EntryId>) => Effect<readonly Array<{
    readonly entryId: Uint8Array<ArrayBuffer> & Brand<"effect/eventlog/EventJournal/EntryId">;
    readonly sequence: number;
  }>>;
  readonly firstSequence?: number;
  readonly getId: Effect<RemoteId>;
  readonly getOrCreateSessionAuthBinding: (publicKey: string, signingPublicKey: Uint8Array<ArrayBuffer>) => Effect<Uint8Array<ArrayBuffer>>;
  readonly write: (publicKey: string, storeId: StoreId, batches: readonly Array<EncryptedBatch>) => Effect<readonly Array<EncryptedRemoteBatch>>;
}, this> {
  constructor(_: never);
}