Skip to content

Content addressing

AWC names every immutable object by a digest computed from its bytes, not by where the bytes live. The address is the content. This single choice is why a fork costs nothing, why storage grows with distinct bytes rather than with workspace count, and why the cache can serve the same object to many readers.

Identity comes from the bytes

A content-addressed object is named by a digest derived from a canonical encoding of its own content. For the structured objects (file, tree, snapshot) that encoding is deterministic canonical CBOR; for a raw blob it is the bytes themselves. The digest domain folds in the object kind:

SHA256("awc:v1:" || object_kind || 0x00 || canonical_payload)

Mixing the kind into the domain means the same byte payload cannot be read as two different object types. The implementation computes this in one streaming pass, so an object is hashed without ever holding its full payload.

Immutability follows from the addressing scheme. If the address is the content, then changing the content changes the address. The spec therefore states that a published object MUST never change in place, and that a second upload under an existing digest MUST either be byte-identical or fail. The object store enforces this with conditional writes that refuse to overwrite an existing key.

Why forks cost nothing

A location-addressed store names bytes by where they sit: a volume, an inode, a path. Forking such a store means either copying the bytes, or maintaining a growing table of references from each workspace to each location. You pay per workspace, and the cost grows with the size of the base.

Content addressing inverts this. The address does not depend on location, so a workspace refers to a base snapshot by a single digest. Fork creates only a metadata record that points at that digest. The spec is explicit that fork work MUST stay independent of snapshot size, and that any work proportional to file count is import or checkpoint work, not fork work.

The dedup property is the same mechanism viewed from storage. Because the address is the bytes, two uploads of identical bytes in one tenant produce one key. A conditional create turns the second upload into a no-op. Durable storage therefore tracks distinct content, not workspace count. The acceptance scenario makes the shape concrete: 1000 forks of a 20 GiB base, each changing 50 MiB, must retain roughly 20 GiB plus 50 GiB plus overhead, not approach 20 TiB merely because 1000 branches exist.

The same shape applies to fetch. The in-flight request table is keyed by (tenant, digest), so many concurrent reads of one absent object share a single remote fetch. The fan-out gate restates this: 100 simultaneous reads of one object MUST produce one underlying fetch per host and tenant.

The tenant is part of the address

Dedup stops at the tenant boundary. The cache key MUST include the tenant, the hash algorithm, and the digest:

tenant_id + hash_algorithm + object_digest

The durable key carries the same tenant, in path form:

cas/v1/tenants/<tenant-id>/<algorithm>/<first-2>/<next-2>/<digest>

Identical bytes uploaded by two different tenants therefore produce two objects, two cache entries, and two fetches. The spec is deliberate about this: a digest collision across tenants does not imply a shared object or a shared cache key, and the system MUST NOT perform cross-tenant existence checks or deduplication.

The on-disk cache layout mirrors the same boundary, nesting objects under a tenant directory. In the implementation, the tenant scope is not an optional string; it is a mandatory argument to every CAS operation.

The reason is isolation. If a tenant could learn that bytes it never uploaded already exist in another tenant's namespace, that fact would be a side channel: an adversary who guesses content could probe for its presence, its size, or its cache warmth. Scoping every key, every existence check, and every fetch to a tenant closes that channel at the addressing layer rather than at a policy layer that might be bypassed.

A future deployment MAY deduplicate across an explicitly configured trust domain, but only with a separate security review. The MVP does not.

What is specified versus what is measured

The digest domain, the tenant-scoped namespace, the mandatory tenant scope, and the singleflight key are committed code, not just spec text. The dedup and fork-independence properties are structural consequences of that code path.

The cache-key composition has one wrinkle worth naming. The cache key lists three components: tenant, algorithm, digest. The singleflight key carries two fields, tenant and digest, because a digest compares equal only when both its algorithm and its raw bytes match. Two fields, three components, same identity.

The economic gates are a separate matter. The fan-out gate is defined for one host cache domain — 100 workspaces on one host — and the delta-storage gate is durable-storage accounting over many forks; neither is a multi-host test. The dedup ratio on a single warm host is a structural fact, and the measurements recorded in this repo so far are single-host. The cross-host storage and I/O case is not yet measured. See measuring AWC for what has and has not been demonstrated.