The durability model¶

AWC makes durability an explicit act. A process calls checkpoint, or it does not. There is no middle state. Ordinary writes land on a host-local upper filesystem and stay there until a checkpoint publishes them. This is the opposite of a shared filesystem, where every write chases the network and fsync is the durability boundary.

Why durability is explicit, not fsync-based¶

The workload is agents that compile, install packages, and generate files at high rates. If every write had to reach shared storage, the data plane would pay round-trip and consistency costs on the hot path. AWC keeps the hot path local. The host overlays a private upper directory on an immutable snapshot, and tools see a normal local filesystem. Read-your-writes is the OverlayFS view, not a remote round-trip.

The cost of that speed is honesty about what is durable. fsync inside a writable mount touches the local upper filesystem only. It is not a remote guarantee.

fsync is local

fsync and fdatasync apply to the host-local upper filesystem. They do not imply a remote checkpoint.

So the product asks you to say when a state is worth keeping. That act is the checkpoint.

Two stores of durable truth¶

Durable state lives in two places, and they do different jobs.

The object store holds immutable content. Blobs, file manifests, tree manifests, and snapshot manifests are content-addressed and written once. Once an object is accepted, it does not change.

PostgreSQL holds the authoritative mutable pointer: the workspace head_snapshot_id and version. The object store can hold many candidate snapshots for one workspace. PostgreSQL decides which one is the head.

This split is why the system is safe. The object store is a set of immutable candidates. The database is the single arbiter of which candidate counts. See architecture for how the planes divide this work.

The atomic head advance¶

A checkpoint ends in one database transaction. That transaction either advances the head, or it changes nothing. There is no partial head.

The publisher uploads objects, confirms they are durable, and only then advances the head. A reader of an acknowledged head can always find every object the head references.

Why no head is ever reconstructed¶

If PostgreSQL is lost, you do not rebuild workspace heads by scanning the object store. The object store holds immutable candidate snapshots, many of them orphans from failed or superseded checkpoints. Without the database, there is no way to know which candidate a workspace last acknowledged. PostgreSQL determines the head.

PostgreSQL itself must run with automated failover and point-in-time recovery. That is a deployment duty, not something the daemon provides.

What is ephemeral¶

The upper directory is ephemeral by design. Writes a process makes to its writable mount live on one host. No other host sees them. No remote client sees them. They are not durable until a checkpoint commits.

This is where the distinction between acknowledged and unacknowledged matters. An acknowledged state is one whose publication transaction committed. The response came back, the head moved, and the version incremented. Everything else is unacknowledged. Unacknowledged writes survive a process crash and a daemon restart, because the upper directory and the kernel mount persist. They do not survive host loss.

Host loss is the sharp boundary

Host loss destroys the uncheckpointed upper delta. Only what the acknowledged head references survives. Everything that lived only in host-local files is gone.

Recovery¶

Each failure mode has a recovery procedure. The boundary to remember is the one above: host loss loses only un-checkpointed work.

The lease is the gate on publication¶

Durability is not just "the transaction committed." The transaction also fences the writer. Only the holder of the current lease generation may advance the head. A host that is paused, partitioned, or superseded holds a stale generation. Its publication fails even if it resumes later, because the generation it presents no longer equals the lease's.

This protects the head from split-brain. A stale host may keep writing to its own upper directory. But those bytes cannot become the canonical head through any normal path. They leave only through an explicit, operator-assisted recovery that never advances the head automatically. See fencing and split-brain and handle a stale writer.

Lost responses are safe¶

A checkpoint can fail after the transaction commits but before the response arrives. The client retries with the same idempotency key. The publication path treats the candidate snapshot as content-addressed. A retry that presents the same manifest either creates the snapshot row or reuses the existing one, then returns the already-committed result. The head advances exactly once. The audit event records whether the row was created or reused.

Objects uploaded before a failed or cancelled checkpoint are not wasted and not harmful. They are immutable orphans. Garbage collection reclaims them later.

Limits and what is not proven here¶

Be precise about what this model guarantees and what it does not.

PostgreSQL high availability is a deployment responsibility. The system expects automated failover and point-in-time recovery, but the daemon does not provide it. A single-node database is a single point of failure for publication.
Object-store durability is deployment-dependent. The system detects a missing object and marks the snapshot degraded rather than return wrong bytes. It cannot manufacture bytes the store has lost.
The properties on this page are design facts, not measured guarantees. They describe specified behavior confirmed against the code, not results from a chaos test.