Tar unstable

Details

A trailing / is added to a non-empty path when missing, as tar tools expect for directories.

declare function directory(path: string, options?: EntryOptions): Entry

Details

String content is encoded as UTF-8, and size is the byte length of the content.

See

• fileStream for content that is streamed instead of held in memory
• encode for writing entries to an archive

declare function file(path: string, content: string | Uint8Array<ArrayBufferLike>, options?: EntryOptions): Entry

When to use

Use when the content is too large to hold in memory, or is read from a file or another stream, and its byte length is known up front.

Details

The defaults match file: mode 0o644, uid and gid 0, and an mtime at the Unix epoch. body is not run until the entry is encoded.

Gotchas

size must be the exact byte length of body: encode fails with a SizeMismatch TarError when the body produces fewer or more bytes.

See

• file for in-memory content
• encode for writing entries to an archive

declare function fileStream<E = never, R = never>(path: string, size: number, body: Stream<Uint8Array<ArrayBufferLike>, E, R>, options?: EntryOptions): Entry<E, R>

Details

target is the archive path of an entry written earlier in the same archive.

declare function hardlink(path: string, target: string, options?: EntryOptions): Entry

Details

Each output chunk holds one entry, and each entry body streams exactly header.size bytes straight from the input. Pulling the next entry skips whatever is left of the current body, like Go's tar.Reader. The decoder reads v7, USTAR, PAX extended headers (applying path, linkpath, size, uid, gid, uname, gname and mtime), GNU long names and links, and base-256 numbers. The records of a PAX global header apply to every entry that follows, except path, linkpath and size, which describe a single entry. Anything after the end-of-archive marker is ignored.

Gotchas

• Read each body before pulling the next entry: a body read afterwards fails with a BodyConsumed TarError. Operators that pull ahead, such as Stream.buffer, therefore can't sit between decode and the body reads.
• Read bodies while the entry stream runs, for example in Stream.mapEffect: once it has ended, as after Stream.runCollect, Stream.runHead or Stream.take, every body fails with a BodyConsumed TarError.
• A body reads the entry's bytes once: running it again continues where the previous run stopped.
• Directories, links, devices and FIFOs have no body, and their size is reported as 0.
• Paths and link targets are returned as written in the archive, so they can be absolute or contain .., and a symlink entry can redirect a later entry outside the target directory. Validate both before writing entries to disk.
• Names that are not valid UTF-8 are decoded with replacement characters.
• PAX extended headers and GNU long names of more than 1 MiB fail with an InvalidHeader TarError.
• The channel completes only when its input ends: input after the end-of-archive marker is read and discarded, so that the upstream done value can be passed on.
• GNU sparse entries fail with an Unsupported TarError.

See

• encode for the inverse operation

declare function decode<IE = never, Done = unknown>(): Channel<readonly [Entry<TarError | IE, never>, Entry<TarError | IE, never>], TarError | IE, Done, readonly [Uint8Array<ArrayBufferLike>, Uint8Array<ArrayBufferLike>], IE, Done>

Details

Each entry is written as a USTAR header, with the path split into the USTAR prefix and name fields when it is longer than 100 bytes. A PAX extended header is added only for what USTAR cannot hold: a long or non-ASCII path, link target, user or group name, a size of 8 GiB or more, a uid or gid above 2097151, an mtime outside the USTAR range, or custom pax records. The archive ends with two zero blocks.

Headers, padding and body chunks below 16 KiB are gathered into output chunks of up to 64 KiB, and larger body chunks pass through without copying, so bodies stream without being held in memory. Output is handed out before waiting for more entries, but up to 64 KiB of it can be held while a body is pulled.

Gotchas

• A body that produces any byte count other than header.size fails with a SizeMismatch TarError, so the size must be known before the body is written.
• header.pax may contain custom metadata such as SCHILY.xattr.*. Keys path, linkpath, size, uid, gid, uname, gname and mtime must be set through the corresponding header fields. These keys and GNU.sparse.* records fail with an InvalidHeader TarError.
• A File entry whose path ends with / fails with an InvalidHeader TarError, because other tools read it as a directory.
• Entries other than File and Other must have size 0, and devMajor and devMinor must not exceed 2097151; otherwise encoding fails with an InvalidHeader TarError.
• mtime is rounded down to whole seconds.
• typeflag is only written for Other entries; every other type writes its standard flag.

See

• decode for the inverse operation
• file for building entries from in-memory content
• fileStream for building entries from streamed content

declare function encode<IE = never, Done = unknown, In extends Entry<any, any> = Entry<never, never>>(): Channel<readonly [Uint8Array<ArrayBufferLike>, Uint8Array<ArrayBufferLike>], TarError | IE | In extends Entry<E, any> ? E : never, Done, readonly [In, In], IE, Done, In extends Entry<any, R> ? R : never>

Details

The kind field identifies the failure:

• InvalidHeader: a header has a bad checksum, a malformed field, or a value that cannot be written.
• Truncated: the input ended inside a header or an entry body.
• SizeMismatch: an entry body produced a byte count other than its declared size.
• Unsupported: the archive uses a feature this module does not read, such as GNU sparse files.
• BodyConsumed: an entry body was read after the decoder had moved past it.

declare class TarError extends YieldableError<this> & {
  readonly _tag: "TarError";
} & Readonly<{
  readonly kind: "InvalidHeader" | "Truncated" | "SizeMismatch" | "Unsupported" | "BodyConsumed";
  readonly message: string;
}> {
  constructor(args: {
    readonly kind: "InvalidHeader" | "Truncated" | "SizeMismatch" | "Unsupported" | "BodyConsumed";
    readonly message: string;
  });
  readonly "~effect/encoding/Tar/TarError": "~effect/encoding/Tar/TarError";
}

Details

Unset fields default to mode 0o644 (0o755 for directories, 0o777 for symlinks), uid and gid 0, empty user and group names, an mtime at the Unix epoch, and no extra PAX records. The defaults make archives reproducible.

interface EntryOptions {
  readonly gid?: number;
  readonly gname?: string;
  readonly mode?: number;
  readonly mtime?: Date;
  readonly pax?: Readonly<Record<string, string>>;
  readonly uid?: number;
  readonly uname?: string;
}

Details

Other covers every typeflag without a dedicated kind, such as GNU volume labels; the raw flag is kept in Header.typeflag.

type EntryType = "File" | "Directory" | "Symlink" | "Hardlink" | "CharDevice" | "BlockDevice" | "Fifo" | "Other"

Details

path and linkTarget are archive paths; hard link targets name another entry of the same archive. mode holds the permission bits, including setuid, setgid and sticky; file type bits, as in a stat mode, are dropped when decoding and encoding. pax holds the PAX extended header records, including those of global headers, that are not reflected in the other fields, such as SCHILY.xattr.*, atime or the comment of git archive. devMajor and devMinor are set for device entries only.

interface Header {
  readonly devMajor?: number;
  readonly devMinor?: number;
  readonly gid: number;
  readonly gname: string;
  readonly linkTarget?: string;
  readonly mode: number;
  readonly mtime: Date;
  readonly path: string;
  readonly pax: Readonly<Record<string, string>>;
  readonly size: number;
  readonly type: EntryType;
  readonly typeflag: string;
  readonly uid: number;
  readonly uname: string;
}