Skip to content

Security model

AWC caches content that comes from repositories, generated code, and agent processes. None of that is trusted. The security model draws one line and then makes every other decision follow from it: the agent and everything it produces sit on the untrusted side; the daemon, the control plane, the signing keys, and the object-store credentials sit on the trusted side. Tenant isolation, mount tickets, mTLS, and the audit log are consequences of where that line falls, not independent features.

This page explains the why behind the design. For the exact field list of a mount ticket, see the reference; for the procedure to enroll a host, see the how-to.

One boundary, not many

The spec names the two sides explicitly. The untrusted side is the agent process, repository content, generated code, filenames and symlinks, and workspace upper contents. The trusted computing base is awcd, its cache and manifest verifier, the control-plane authorization and publication path, the PostgreSQL metadata transaction, the tenant content-addressed storage (CAS) credentials, and the signing keys and host identity.

The assumption that matters is this: agent code is capable of intentionally producing adversarial directory trees, symlink ladders, deep paths, large sparse files, inode churn, and resource-exhaustion workloads. A design that treats the agent as cooperative will be wrong in the cases where being right costs the most. So awcd treats every tree as hostile before it verifies it.

Tenant isolation by construction, not by query

AWC does not deduplicate content across tenants. Every object-store request is scoped to a tenant prefix, and the host cache key is the pair (tenant_id, digest), not the digest alone. A digest collision across tenants is two distinct rows. A snapshot that belongs to another tenant is reported as NotFound, never as a hit, so the store does not leak object existence, timing, size, or cache state across the boundary.

The rejected alternative is cross-tenant deduplication for the storage savings. The spec is explicit about why it is rejected: dedup creates a content-presence side channel, leaks cache state, and confuses trust domains. An attacker who can observe whether a digest already exists can infer what another tenant stored. The cost of saying no is duplicate bytes. The benefit is that no query path can accidentally cross tenants, because no such path exists.

A future deployment MAY deduplicate inside an explicitly configured organization trust domain, but only after a separate security review. The MVP does not.

Four principals, one decision

Every caller reduces to the same principal shape before authorization runs. There are four kinds:

  • User — a human authenticated through OIDC.
  • ServiceAccount — a machine authenticated by a scoped, revocable signed token.
  • Host — a registered host authenticated by mutual TLS.
  • MountTicket — a short-lived signed capability, not an account.

The reason to collapse these into one type is that the authorization engine then reasons about a single question: does this principal have this permission on this project? The engine is pure logic over project role memberships, and the control plane calls it on every external request. Project roles — Viewer, Developer, Operator, Admin, Billing viewer — are project-scoped, so a principal who is an Admin in one project is nothing in another.

Capability principals (hosts, mount tickets) deliberately do not go through project RBAC. They carry a narrow delegated capability instead. A signed mount ticket grants the right to mount one snapshot on one host. It does not replace project authorization for anything else.

Capabilities, not credentials in the sandbox

The model keeps long-lived secrets on the trusted side of the line. Long-lived object-store credentials MUST NOT enter agent sandboxes. What crosses the boundary is a short-lived, signed capability that the trusted side can verify on its own.

A mount ticket is the clearest example. The control plane signs it; awcd verifies it without a synchronous control-plane call. It is Ed25519-signed, audience-bound to one host, nonce-protected, and carries an expiry. It binds the tenant, the workspace, the sealed snapshot and root digest, the access mode, the fencing lease generation for a writable mount, and an allow-list of target roots. awcd refuses to mount anywhere outside those roots, and it MUST NOT accept an arbitrary host path from an untrusted sandbox.

Two implementation details matter because they remove whole classes of attack.

The signature covers the literal transmitted claim bytes, not a re-canonicalized form. Verification recomputes the signature over the exact bytes received. There is no JSON-canonicalization step to get wrong, so a sender cannot exploit parser disagreement between signer and verifier.

The signing key's debug representation is redacted. The same is true of the tenant-scoped object credential. A leaked log line must not hand over a working capability.

The object credential is the same idea applied to storage. It is a short-lived, Ed25519-signed capability scoped to one tenant prefix and a closed set of operations (get, put, delete, list), audience-bound to one host where practical. A deployment exchanges it for concrete short-lived STS or SigV4 material at the object-store gateway; that exchange is out of scope for the auth layer and lives with the CAS adapter.

mTLS is the identity root for hosts

A host proves who it is with a client certificate. Internal gRPC is mutually authenticated with TLS outside AWC_DEV_MODE=1. The daemon requires an https:// endpoint and a client identity, and the control plane mirrors the rule on the server side. Host credentials are short-lived where supported and restricted by tenant prefix and operation. Renewal provisions a new credential with a fresh fingerprint while the prior one stays valid until its own expiry, so rotation is overlap-safe. A credential can be revoked before its natural expiry.

AWC_DEV_MODE=1 is the single escape hatch. In that mode the daemon and control plane relax the requirements for the service-token key, the mount-ticket seed, the ticket public key, and TLS. The switch exists for local development and tests, not for deployment. Nothing else disables mTLS.

Why awcd is privileged but not a sandbox

AWC is not a complete sandbox. The agent platform keeps responsibility for process, network, syscall, and resource isolation. AWC's job is narrower: ensure a workspace mount exposes only the authorized snapshot and its private upper, that bind targets stay inside allow-listed roots, that symlinks cannot cause privileged host traversal by awcd, that agent code cannot write the shared base FUSE mount or the host CAS cache, and that one workspace cannot reach another workspace's upper directory.

awcd is privileged because it must. It performs OverlayFS and FUSE mounts and writes cgroup files to pause and resume execution for checkpointing. The systemd unit runs it as the unprivileged awc user and bounds it to a minimal capability set: CAP_SYS_ADMIN, CAP_DAC_READ_SEARCH, CAP_DAC_OVERRIDE, CAP_CHOWN, and CAP_FOWNER (the awcd.service unit). NoNewPrivileges, a @system-service plus @mount syscall filter, MemoryDenyWriteExecute, and a closed device policy that admits only /dev/fuse round out the hardening.

Be honest about what that hardening is not. awcd is not network-isolated. The unit sets RestrictAddressFamilies=AF_UNIX AF_INET, because the daemon must reach the control plane and the object store. If you need egress restrictions, enforce them at the network layer around the host, not in the unit.

The unit also deliberately omits the systemd directives that create a private mount namespace (ProtectSystem=, ProtectHome=, PrivateTmp= and related). awcd publishes its FUSE and OverlayFS mounts into the host namespace so that clients, the worker, and the acceptance checks can see them. A private namespace hid awcd's own mounts from the host during the pilot, so the directives were removed. The trade is real: awcd runs in the host mount namespace by design, and the capability set above is what bounds it there.

The threat: one compromised agent

Walk through what an attacker who compromises one agent process can and cannot do.

They hold that workspace's writer lease, if it is mounted READ_WRITE. They can write the private upper. They cannot read another tenant. Object-store requests are tenant-prefixed, the cache key includes the tenant, and a cross-tenant snapshot is NotFound. They cannot reach another workspace's upper directory.

They cannot forge a checkpoint for a workspace they do not hold the lease for. Checkpoint publication MUST include the current lease generation and the expected workspace version, and at most one unexpired lease generation may publish for a workspace. The mount ticket stamps the fencing generation into the signed claims, so a stale writer's generation is rejected at publication. See fencing and split-brain for how the generation defeats a split-brain writer.

They cannot redirect a mount at an arbitrary host path. The ticket's target policy allow-lists the roots, and awcd enforces the prefix match. A path that shares a prefix string but is not beneath a root — /srv/awc/runsX under root /srv/awc/runs — is rejected.

They cannot poison a trusted build namespace by uploading content with a known digest label. Content addressing alone is not sufficient. Every object is verified on ingest or first trusted use, object keys are immutable, conditional creation prevents overwriting a digest, and the object kind is part of the digest domain. Trusted and untrusted producer classes MAY use separate CAS trust namespaces, and production build artifacts MUST NOT trust content solely because an untrusted agent uploaded the same digest label.

Audit records what changed, never what was stored

The control plane records immutable audit events for privileged actions: snapshot publication and deletion, workspace creation, fork and deletion, lease acquisition, revocation and expiration, mount-ticket issuance, mount and unmount, checkpoint publication, export, retention-policy changes, host registration and revocation, recovery actions, and authorization failures.

The contract on contents is strict. An audit event records who acted, on what resource, from what state to what state, and whether it succeeded. It MUST NOT contain object contents, byte payloads, manifest bodies, file names from user data, credentials, tokens, or keys. Identifiers, certificate fingerprints, digests, and enum-valued verdicts are not secrets and are recorded for accountability. The action vocabulary uses a <resource>.<verb> shape — checkpoint.publish, host.revoke, signing_key.rotate, object_credential.issue — so the log sorts and greps by resource family.

Limits and what is not wired yet

The security objectives are specified in full. Not all of them are exercised in the current build to the same degree.

  • The audit vocabulary is live for checkpoint publication, retention and TTL, garbage collection, host lifecycle, signing-key and object-credential rotation, and import and snapshot lifecycle. The categories for lease, mount session, export, recovery, and authorization failures are specified but their owning service layers have not been wired yet. Of these, only lease, mount_session, and export carry reserved resource_type constants; recovery and authorization-failure auditing are named in the spec but have no reserved constant yet.
  • mTLS enforcement is implemented for the control channel. Customer-managed keys per tenant and encrypted host volumes are deployment choices, not build facts, and are not verified here.
  • The sandbox-isolation guarantees are design requirements on awcd. This page does not re-verify them with an adversarial test suite; the tracked security tests are the home for that work.
  • AWC_DEV_MODE=1 disables the credential and TLS requirements described above. Never run with it set outside a local development machine.