Skip to content

On-disk format

This page describes how immutable content-addressed storage (CAS) objects are serialized and named. The format lets AWC deduplicate, verify, and share objects across hosts.

1. Object identity

Every immutable object is named by an algorithm-qualified digest, shown as:

sha256:<64 lowercase hexadecimal characters>

The digest mixes a version tag and the object kind into the hash. This domain separation means the same byte payload can never be read as two object kinds. A blob, a file, a tree, and a snapshot that share the same bytes still get distinct names.

Two objects are identical if and only if they share the same object kind and the same canonical payload. The digest captures exactly that.

2. Canonical encoding

Structured objects use a deterministic canonical CBOR encoding as defined by RFC 8949:

  • Definite lengths only. Every byte string, text string, array, and map carries a definite length. Indefinite-length encodings are forbidden.
  • Shortest forms. Integers and lengths use the shortest available encoding.
  • Sorted map keys. Map entries are ordered by the bytewise comparison of the encoded key.
  • No duplicate keys within a single map.

Deterministic encoding keeps the serialized bytes stable, so the same logical object produces the same digest on every host.

3. Object kinds

AWC stores four object kinds:

  • BLOB — raw bytes. The canonical payload is the blob itself.
  • FILE — file metadata plus references to one or more content blobs. Small files reference a single blob. Larger files are split into chunks, each stored as its own blob. The logical file digest covers the complete file bytes. It is independent of how the file is chunked.
  • TREE — a directory listing of named child entries. Entries are sorted lexicographically by raw name bytes. Names must be non-empty and must not contain path separators or NUL bytes. Each entry names a child file, tree, or symlink by its digest.
  • SNAPSHOT — a pointer to a root tree, provenance and metadata, and an optional parent snapshot. Mutable control-plane state, such as human-readable names and retention policy, points at a snapshot record. It is not part of the snapshot's content identity.

4. Forward compatibility

Every serialized manifest carries a version field, and the digest domain carries a version tag. Future hash algorithms or structural changes can be introduced by bumping the tag. The tag keeps new objects distinct from existing ones.

See also

  • Data model — identifiers, object kinds, and the workspace, lease, ticket, and patch-bundle records that reference these objects.
  • Golden vectors — the versioned manifest test vectors.
  • Fuzz targets — the encode ∘ decode fixed-point fuzzers.