Data model reference¶
This page lists the identifiers, object kinds, manifest fields, and mutable control-plane records that make up AWC. Immutable object fields come from the object manifests. Mutable record columns come from the control-plane database rows and the API types.
For the byte-level canonical CBOR encoding, see On-disk format. For the SQL schema, see Database schema.
Identifiers¶
Human-facing resources use opaque, sortable identifiers of the form
<prefix>_<ulid>. The ULID is 26 characters, case-insensitive, and monotonic.
The prefix is fixed per resource kind.
| Resource | Prefix | Identifier type |
|---|---|---|
| Tenant | ten_ |
TenantId |
| Project | prj_ |
ProjectId |
| Snapshot | snap_ |
SnapshotId |
| Workspace | ws_ |
WorkspaceId |
| Writer lease | lease_ |
LeaseId |
| Host | host_ |
HostId |
| Import session | import_ |
ImportSessionId |
| Mount session | mount_ |
MountSessionId |
A second _ in the ULID portion is rejected. Each kind is a distinct type, so a
TenantId never aliases a ProjectId. The mount-session prefix (mount_) is
not in the standard prefix list; it is defined separately.
Content objects do not use these identifiers. They are named by digests.
Object digests¶
An immutable object is named by an algorithm-qualified digest.
| Property | Value |
|---|---|
| Canonical algorithm | sha256 |
| Raw length | 32 bytes |
| Wire form | <algorithm>:<hex>, lowercase hex only |
| Supported algorithms | Sha256 only |
The algorithm enum exists so keys and digests can migrate later without a format
break. The MVP writes and reads sha256 only. Uppercase hex is rejected on
parse.
Digest domain¶
A digest is computed over a kind-specific domain:
object_kindis the ASCII labelBLOB,FILE,TREE, orSNAPSHOT.0x00is one separator byte.- For
BLOB,canonical_payloadis the raw blob bytes. - For
FILE,TREE, andSNAPSHOT,canonical_payloadis the deterministic canonical CBOR encoding.
Object kind is part of the hashed domain. The same byte payload cannot be interpreted as two kinds.
Tenant-scoped content addressing¶
Content deduplication is scoped to a tenant. Identical bytes share one object within a tenant, never across tenants.
The canonical remote object key is:
For example:
<first-2> and <next-2> are the first two and next two hex characters of the
digest. A digest collision across tenants does not create a shared object or a
shared cache key.
The host-local cache uses the same tenant scoping:
| Layer | Key |
|---|---|
| Singleflight | (tenant_id, digest) |
| Cache index | primary key (tenant_id, digest) |
| On-disk path | objects/<tenant-id>/sha256/<first-2>/<next-2>/<hex> |
The cache index also records object_kind and validates it, although the digest
domain already includes the kind.
Object kinds¶
The CAS stores four immutable object kinds.
| Kind | Label | Holds |
|---|---|---|
BLOB |
BLOB |
Raw file or chunk bytes |
FILE |
FILE |
File metadata and references to one or more blobs |
TREE |
TREE |
Sorted directory entries |
SNAPSHOT |
SNAPSHOT |
Immutable root metadata and provenance |
A BLOB payload is unstructured bytes. Compression at rest is allowed only when
the object-store key still names the digest of the uncompressed bytes and the
stored envelope records the codec. The MVP default is no application compression
for source-code-sized blobs.
FILE manifest¶
A file is stored whole when it is smaller than the inline threshold, and split into fixed chunks otherwise.
| Field | Type | Notes |
|---|---|---|
version |
uint |
Always FORMAT_VERSION (1) |
size |
uint |
Logical file size in bytes |
mode |
uint |
POSIX mode bits |
mtime_ns |
int \| null |
Nanoseconds since the epoch, or null |
chunks |
array | One or more chunk entries |
content_digest |
text | Plain SHA-256 of the complete logical bytes |
Chunk fields:
| Field | Type | Notes |
|---|---|---|
offset |
uint |
Byte offset within the logical file |
length |
uint32 |
Chunk length in bytes |
digest |
text | Domain-separated blob digest of the chunk bytes |
Layout rules:
| Constant | Value | Behavior |
|---|---|---|
INLINE_THRESHOLD_BYTES |
8 MiB | Files with size < threshold use one blob reference |
CHUNK_SIZE_BYTES |
4 MiB | Files at or above the threshold split into fixed chunks |
- Chunks are contiguous, non-overlapping, start at offset zero, and sum to
size. - The final chunk may be smaller.
- An empty file is one zero-length chunk that references an empty blob.
- Each
chunks[].digestis a domain-separated blob digest. content_digestis the plain SHA-256 of the logical bytes. It is independent of chunking.- Content-defined chunking is deferred. The format permits it later.
TREE manifest¶
A tree is a sorted list of directory entries.
| Field | Type | Notes |
|---|---|---|
version |
uint |
Always FORMAT_VERSION (1) |
entries |
array | Sorted, de-duplicated entries |
Tree entry fields:
| Field | Type | Notes |
|---|---|---|
name |
bytes | Raw filename bytes; no UTF-8 assumption |
type |
text | FILE, TREE, or SYMLINK |
digest |
text | Digest of the referenced object |
mode |
uint |
POSIX mode bits |
symlink_target |
bytes | null | Raw target bytes; only for SYMLINK |
Rules:
- Entries are sorted lexicographically by raw name bytes.
- A name must be non-empty. A name must not contain
/(0x2f) or NUL (0x00). - Duplicate names are invalid.
- A
SYMLINKentry must carry a non-emptysymlink_target. The target is stored verbatim and is never followed during import. - Directory identity depends on entry names, types, metadata, and child digests.
A SYMLINK entry references a BLOB object that carries the link target.
SNAPSHOT manifest¶
A snapshot pins a root tree and records provenance and aggregate counters.
| Field | Type | Notes |
|---|---|---|
version |
uint |
Always FORMAT_VERSION (1) |
root_tree |
text | Digest of the root TREE manifest |
parent_snapshot |
text | null | Parent snapshot digest, or null |
logical_bytes |
uint |
Total logical bytes reachable from the root tree |
file_count |
uint |
Files reachable from the root tree |
tree_count |
uint |
Trees reachable from the root tree |
normalization_policy |
text | source-normalized or environment-preserving |
source |
map | Provenance; see below |
created_by |
text | Opaque principal reference |
created_at |
int |
Unix nanoseconds since the epoch |
labels |
map | Free-form text to text; sorted by key |
source sub-fields:
| Field | Type | Notes |
|---|---|---|
type |
text | directory, git, tar, or checkpoint |
locator |
text | null | Source path or URL |
immutable_revision |
text | null | Immutable revision, for example a Git commit |
The snapshot content digest excludes mutable control-plane aliases and retention state. Human-readable names point to a snapshot record. They are not part of content identity.
Snapshot closure¶
A snapshot may enter READY only when every object reachable from its root
manifest is present and valid in the tenant CAS. The graph validator walks the
snapshot, checks object kinds, and confirms that logical_bytes,
file_count, and tree_count match the traversed graph.
Format versioning¶
Every serialized manifest includes a version field.
| Constant | Value | Meaning |
|---|---|---|
FORMAT_VERSION |
1 |
Alias for the current write version |
CURRENT_WRITE_VERSION |
1 |
Version every new manifest is written at |
SUPPORTED_READ_VERSIONS |
[1] |
Versions a new daemon must read |
RETIRED_VERSIONS |
[] |
Versions no longer read |
A decoder accepts only a supported, non-retired version. Unknown optional
fields are ignored so newer formats stay decodable. The digest domain prefix
awc:v1: carries its own version tag, so the hash algorithm or structure can
migrate without ambiguity with existing objects.
Normalization policies¶
Snapshot creation must declare one normalization policy. The wire form is kebab-case.
| Policy | Wire value | Purpose |
|---|---|---|
SourceNormalized |
source-normalized |
Deterministic reuse for source repositories. Normalizes UID and GID to zero, normalizes mtime to zero, clears setuid and setgid bits, removes unsupported xattrs. |
EnvironmentPreserving |
environment-preserving |
Prepared dependency and build environments. Preserves mtime. Normalizes UID and GID to zero unless configured. Clears setuid and setgid bits by default. |
The selected policy is recorded in snapshot provenance.
Snapshot record (control plane)¶
The database row for a snapshot:
| Column | Type | Notes |
|---|---|---|
id |
String |
snap_<ulid> |
tenant_id |
String |
ten_<ulid> |
project_id |
String |
prj_<ulid> |
manifest_digest |
String |
Digest of the SNAPSHOT object |
root_tree_digest |
String |
Digest of the root TREE object |
state |
String |
Snapshot state wire value; see below |
logical_bytes |
i64 |
Denormalized from the manifest |
file_count |
i64 |
Denormalized from the manifest |
parent_snapshot_id |
String? |
Parent snapshot record id |
source_type |
String |
directory, git, tar, or checkpoint |
normalization_policy |
String |
source-normalized or environment-preserving |
created_by |
String |
Principal reference |
created_at |
DateTime |
|
labels |
jsonb |
Snapshot state values:
| Value | Meaning |
|---|---|
UPLOADING |
Object upload is in progress |
VALIDATING |
Objects uploaded; integrity checks are running |
READY |
Immutable and usable as a workspace base |
FAILED |
Validation or publication failed |
DELETING |
A delete is in progress |
DELETED |
The snapshot and its objects are removed |
DEGRADED |
Readable, but some backing objects are unavailable |
Only READY snapshots may be used as new workspace bases.
Workspace record¶
The database row for a workspace:
| Column | Type | Notes |
|---|---|---|
id |
String |
ws_<ulid> |
tenant_id |
String |
|
project_id |
String |
|
base_snapshot_id |
String |
|
head_snapshot_id |
String |
|
version |
i64 |
Optimistic concurrency |
state |
String |
Workspace state wire value |
writer_lease_generation |
i64 |
Current fencing generation |
ttl_expires_at |
DateTime |
TTL expiry time |
name |
String? |
Control-plane alias; not part of content identity |
created_by |
String |
|
created_at |
DateTime |
|
updated_at |
DateTime |
|
labels |
jsonb |
A fork creates a workspace whose base_snapshot_id and head_snapshot_id
initially reference the same snapshot. The name column is a control-plane
alias and is not part of content identity.
Workspace state values:
| Value | Meaning |
|---|---|
CREATING |
Being created from its base snapshot |
FAILED |
Creation failed |
IDLE |
Created and not mounted |
MOUNTED |
Mounted on at least one host |
CHECKPOINTING |
A checkpoint is being published |
EXPIRING |
TTL is expiring |
DELETED |
Removed from the control plane |
Writer lease record¶
The database row for a writer lease:
| Column | Type | Notes |
|---|---|---|
id |
String |
lease_<ulid> |
workspace_id |
String |
|
generation |
i64 |
Monotonic fencing token |
host_id |
String |
|
mount_session_id |
String? |
|
state |
String |
Lease state wire value |
granted_at |
DateTime |
Time the lease was granted |
expires_at |
DateTime |
|
released_at |
DateTime? |
Time the lease was released, if it was |
The generation is a monotonically increasing fencing token. A host with an
older generation may keep local bytes but must not publish a checkpoint.
Lease state values:
| Value | Live | Can publish | Meaning |
|---|---|---|---|
ISSUED |
yes | no | Issued, not yet claimed by a mount |
ACTIVE |
yes | yes | Held by a mounted writer |
RELEASING |
yes | yes | Holder is releasing and publishing final state |
RELEASED |
no | no | Released cleanly |
EXPIRED |
no | no | Reached lifetime without release |
REVOKED |
no | no | Forcibly revoked by the control plane |
Only an ACTIVE or RELEASING lease may advance the workspace head.
Mount session record¶
The database row for a mount session:
| Column | Type | Notes |
|---|---|---|
id |
String |
mount_<ulid> |
workspace_id |
String |
|
host_id |
String |
|
mode |
String |
READ_ONLY or READ_WRITE |
state |
String |
Mount session state wire value |
lease_id |
String? |
Set for a writable mount |
root_snapshot_digest |
String |
Digest of the sealed root |
target_path |
String |
Configured mount target path |
started_at |
DateTime |
|
ended_at |
DateTime? |
Mount session state values:
| Value | Meaning |
|---|---|
REQUESTED |
Received, not yet authorized |
AUTHORIZED |
Authorized, preparation not started |
PREPARING |
FUSE and overlays are being prepared |
MOUNTED |
Live and serving I/O |
UNMOUNTING |
Unmount in progress |
CLOSED |
Closed cleanly (terminal) |
FAILED |
Mount failed or was lost (terminal) |
Mount ticket¶
A mount ticket is a short-lived, signed capability. The typed API struct is
MountTicket. The signed payload is MountTicketClaims.
MountTicket fields:
| Field | Type | Notes |
|---|---|---|
ticket_id |
String |
Opaque ticket id |
tenant_id |
TenantId |
|
workspace_id |
WorkspaceId |
|
snapshot_id |
SnapshotId |
Snapshot the mount is sealed against |
root_digest |
Digest |
Content digest of the sealed root tree |
host_id |
HostId |
The single audience host |
mode |
MountMode |
READ_ONLY or READ_WRITE |
lease_generation |
u64? |
Fencing generation for READ_WRITE; null for read-only |
target_policy |
TargetPolicy |
Allowed bind roots |
issued_at |
Timestamp |
|
expires_at |
Timestamp |
|
nonce |
String |
Single-use replay defeat |
protocol_version |
u32 |
Internal protocol version |
signed_ticket |
String |
Opaque, server-signed token for MountWorkspace |
The signed claims (MountTicketClaims) carry the same identity fields and add
sealing for checkpoint safety:
| Claim | Type | Notes |
|---|---|---|
snapshot_manifest_digest |
Digest? |
Digest of the sealed SNAPSHOT manifest |
lease_id |
String? |
Control-plane lease identity |
mount_session_id |
String? |
|
workspace_version |
u64? |
Optimistic version sealed into the capability |
execution_cgroup |
String? |
Cgroup path for quiesce |
Tickets should use Ed25519 signatures and must be verifiable by awcd without a
synchronous control-plane call. The persisted ticket row stores signed_ticket
and does not store root_digest. The root digest lives on the typed struct, the
signed claims, and the mount session row.
Patch bundle¶
A patch bundle describes a delta against a declared base snapshot.
| Field | Type | Notes |
|---|---|---|
version |
u32 |
PATCH_BUNDLE_VERSION (1) |
base_snapshot_digest |
Digest? |
Base the delta is computed against |
result_snapshot_digest |
Digest? |
Result snapshot |
changes |
array | Ordered PatchChange entries |
embedded_blobs |
array | Complete binary payloads |
provenance |
ExportProvenance |
Free-form export provenance |
PatchChange variants:
| Variant | Carries |
|---|---|
AddFile |
path, mode, digest, content |
ModifyFile |
path, old_mode, new_mode, old_digest, new_digest, content |
ChangeMode |
path, old_mode, new_mode |
DeleteFile |
path, mode |
AddDir |
path, mode |
DeleteDir |
path |
AddSymlink |
path, target |
ModifySymlink |
path, old_target, new_target |
DeleteSymlink |
path |
FileContent representation:
| Variant | Carries |
|---|---|
Text |
unified_diff |
Binary |
blob_digest (reference into embedded_blobs) |
EmbeddedBlob fields:
| Field | Type | Notes |
|---|---|---|
digest |
Digest |
FILE manifest digest the payload belongs to |
size |
u64 |
Logical size in bytes |
data |
bytes | Raw payload |
ChangeMode, ModifySymlink, and DeleteSymlink are separate operations
rather than implicit.
Object-graph limits¶
The graph validator enforces structural ceilings on an untrusted graph.
| Limit | Value | Meaning |
|---|---|---|
MAX_OBJECT_BYTES |
64 MiB | Hard cap on a single object payload |
MAX_TREE_ENTRIES |
100,000 | Maximum entries in one tree |
MAX_TREE_DEPTH |
64 | Maximum tree nesting depth (root tree at depth 0) |
See also¶
- On-disk format — canonical CBOR encoding and digest domains.
- Database schema — SQL tables and columns.
- Limits — operational and per-object limits.
- Content addressing — why tenant scope is the deduplication boundary.
Current scope¶
This page reflects the current build. The following items are specified but not yet implemented:
- No second hash algorithm. Only
sha256is implemented. No other algorithm is wired in. - No content-defined chunking. Files split only at fixed 4 MiB boundaries. Content-defined chunking is deferred.
- No application-level blob compression envelope. The codec field is not implemented; blobs are stored raw.
- Hard links. The MVP may store hard links as independent file entries. It does not preserve hard-link identity.
root_digeston the ticket.root_digestis present on the ticket struct and the signed claims. The persisted ticket row storessigned_ticketinstead and carries the root digest on the mount session row.