awcd daemon reference

awcd is the privileged per-host daemon. One instance runs on each compute host. It maintains the shared read-only snapshot filesystem, the host-local content cache, private upper directories, and OverlayFS and bind mounts. It serves its host API on a Unix-domain socket (spec §8.3).

Invocation

awcd <config.toml|-> [socket-path]

Argument	Required	Description
config.toml	yes	Path to a TOML config file. Pass - to load environment only.
socket-path	no	Path for the Unix-domain gRPC socket.

The socket path resolves in this order:

1. The second positional argument.
2. The AWCD_SOCKET environment variable.
3. /run/awcd/awcd.sock.

Configuration sources

awcd loads configuration in layers. Later layers override earlier ones.

file < environment (AWC_*) < flags

Source	How it is read
Config file	First positional argument. Parsed as TOML.
Environment	Variables prefixed with AWC. Nested keys use __.
Flags	Not exposed on the awcd command line in this build.

The packaged systemd unit reads /etc/awc/awcd.toml and imports /etc/awc/awcd.env through EnvironmentFile.

Config keys

Key	Environment variable	Default	Notes
host_id	AWC_HOST_ID	required	Must parse as a host_<ulid>.
control_plane_endpoint	AWC_CONTROL_PLANE_ENDPOINT	required	http or https. https is required outside dev mode.
basefs_mount	AWC_BASEFS_MOUNT	required	Absolute path. Must differ from the other roots.
cache_root	AWC_CACHE_ROOT	required	Absolute path. Must differ from the other roots.
workspace_root	AWC_WORKSPACE_ROOT	required	Absolute path. Must differ from the other roots.
allowed_bind_roots	AWC_ALLOWED_BIND_ROOTS	[]	Comma-separated absolute paths.
health_http_listen	AWC_HEALTH_HTTP_LISTEN	unset (off)	Loopback host:port, for example 127.0.0.1:9100.
cache.max_bytes	AWC_CACHE__MAX_BYTES	1000000000000 (1 TiB)	Total byte limit of the local cache.
cache.min_free_bytes	AWC_CACHE__MIN_FREE_BYTES	50000000000 (50 GiB)	Free-space reserve. Drives the pressure ladder.
cache.max_remote_fetches	AWC_CACHE__MAX_REMOTE_FETCHES	128	Global object-store read concurrency.
cache.max_remote_fetches_per_tenant	AWC_CACHE__MAX_REMOTE_FETCHES_PER_TENANT	32	Cannot exceed max_remote_fetches.
cache.max_prefetch_fetches	AWC_CACHE__MAX_PREFETCH_FETCHES	8	Cannot exceed max_remote_fetches.
checkpoint.max_concurrent	AWC_CHECKPOINT__MAX_CONCURRENT	4	Concurrent checkpoint uploads.
checkpoint.freeze_timeout_seconds	AWC_CHECKPOINT__FREEZE_TIMEOUT_SECONDS	10	Quiesce timeout for cutting a checkpoint.
checkpoint.preserve_local_grace_seconds	AWC_CHECKPOINT__PRESERVE_LOCAL_GRACE_SECONDS	3600	Grace period for preserved local changes.
leases.renew_interval_seconds	AWC_LEASES__RENEW_INTERVAL_SECONDS	30	Must be strictly less than expiry_seconds.
leases.expiry_seconds	AWC_LEASES__EXPIRY_SECONDS	90	Lease lifetime.
leases.disconnected_grace_seconds	AWC_LEASES__DISCONNECTED_GRACE_SECONDS	60	Grace period before revocation.

Defaults come from spec §22.9. Startup rejects contradictory or unsafe settings. The three roots must be distinct paths.

Other environment variables

These are read by awcd directly, not through the config layer.

Variable	Default	Purpose
AWC_CAS_URL	memory://	Object store for the verified cache. Accepts memory:// or s3://. The s3:// transport needs the s3 feature.
AWC_MOUNT_TICKET_PUBLIC_KEY	unset	Mount-ticket verifier public key, 64 hex characters. Required outside dev mode.
AWC_MOUNT_TICKET_PUBLIC_KEY_FILE	unset	Path to a file that holds the public key.
AWC_DEV_MODE	unset	Set to 1 to enable a built-in dev ticket key and to allow http:// control-plane endpoints.
AWC_CGROUP_ROOTS	unset	Colon-separated cgroup roots for the checkpoint runner. fuse feature only.
AWCD_SOCKET	unset	Fallback for the optional socket-path argument.

What awcd does

Responsibility	Status
Maintain the shared read-only base filesystem	Wired with the fuse feature. A NoopBaseMount runs without it.
Maintain the host-local content cache	Wired. The cache lives under cache_root.
Resolve snapshot manifests	Wired.
Fetch and verify content-addressed objects	Wired.
Coalesce concurrent fetches	Wired through the singleflight FetchCoordinator. fuse feature only.
Manage private upper directories	Wired.
Build and tear down OverlayFS mounts	Wired. fuse feature only.
Bind mounts into authorized namespaces	Wired.
Acquire and renew writer leases	Wired through the control-plane client.
Track mount and fencing state	Wired.
Pause and resume an execution cgroup for checkpointing	Wired through the checkpoint runner. fuse feature only.
Scan the writable delta and build a new snapshot	Wired through the checkpoint runner. fuse feature only.
Upload new objects to the tenant CAS	Wired. fuse feature only.
Publish checkpoints through the control plane	Wired through the control-plane client. fuse feature only.
Expose a local Unix-domain gRPC API	Wired.
Report metrics and mount status	Partial. See "Emitted metrics" and "Current scope".

awcd fails closed for new privileged mount operations when ticket verification or host authorization fails (spec §8.3).

Control-plane client

At startup, awcd builds a tonic client against control_plane_endpoint:

Client	gRPC service	Purpose
LeaseClient	LeaseService	Renew each leased session and cache the verdict.
CheckpointClient	CheckpointService	Publish checkpoints. fuse feature only.

A malformed endpoint is handled differently per client. The mount-lifecycle LeaseClient and the CheckpointClient abort startup: awcd exits with status 1. The lease-renewal LeaseClient is the exception. A malformed endpoint disables that client only, startup continues, and the lease cache stays cold, so restart reconciliation preserves sessions (spec §17.3).

Capabilities

awcd runs as the unprivileged awc user. It holds a minimal capability set so it can perform privileged mount operations and access /dev/fuse. The packaged systemd unit grants these capabilities:

Capability	Reason
CAP_SYS_ADMIN	Mount, cgroup, and setns operations.
CAP_DAC_READ_SEARCH	Checkpoint upper-directory bookkeeping.
CAP_DAC_OVERRIDE	Overlay-over-FUSE lowerdir resolution.
CAP_CHOWN	Checkpoint upper-directory bookkeeping.
CAP_FOWNER	Checkpoint upper-directory bookkeeping.

The unit allows /dev/fuse as a device. It does not set the systemd directives that create a private mount namespace, because awcd must publish its mounts into the host namespace (spec §15.4).

Health HTTP listener

The health and metrics surface is off by default. Set health_http_listen to a loopback host:port to enable it. The host API stays on the Unix socket.

When enabled, awcd serves three paths. All accept GET only.

Path	Success	Failure	Body
/healthz	200	503	Liveness report as JSON.
/readyz	200	503	Readiness report as JSON.
/metrics	200	none	Prometheus-style text exposition.

/healthz returns 200 when the process is up and the base filesystem is mounted. /readyz returns 200 only when the base filesystem, kernel compatibility, leases, free space, and control-plane and object-store connectivity are all verified. A host can stay ready for warm reads while /readyz returns 503 (spec §19.6).

Other HTTP methods return 405. Unknown paths return 404.

Disk pressure

awcd probes the filesystem that holds cache_root through statfs, and the I/O pressure stall through /proc/pressure/io. It classifies the host into one of six monotone states (spec §17.8). A background sampler refreshes the state every 5 seconds.

Rank	State	Wire label	Containment
0	Normal	NORMAL	None.
1	CacheEviction	CACHE_EVICTION	Evict disposable cache.
2	PrefetchDisabled	PREFETCH_DISABLED	Stop speculative prefetch.
3	NewMountsDisabled	NEW_MOUNTS_DISABLED	Reject new writable mounts.
4	UpperWritesAtRisk	UPPER_WRITES_AT_RISK	Existing upper writes may fail with ENOSPC. Warn the agent and the platform.
5	Emergency	EMERGENCY	Only warm reads continue.

Containment is strictly ordered. The cheap, safe lever fires first. Inode scarcity and PSI stalls can only escalate the state, never relax it.

The byte thresholds derive from cache.min_free_bytes (the reserve R):

Threshold	Multiple of R
eviction_free_bytes	3 * R
prefetch_disabled_free_bytes	2 * R
new_mounts_disabled_free_bytes	R
upper_risk_free_bytes	2/3 * R
emergency_free_bytes	1/3 * R
checkpoint_reserve_bytes	R

Default PSI thresholds: some stall 50 percent (5000 basis points), full stall 80 percent (8000 basis points). Default inode scarcity line: 1024 free inodes.

New writable mounts are refused while pressure blocks admission. The refusal is a visible disk-full error. awcd never deletes uncheckpointed upper data to relieve pressure (spec §17.8).

Emitted metrics

These metrics have a live emit site in awcd. The full metric catalog is in the metrics reference.

Metric	Type	Source	Notes
awc_host_disk_pressure_state	gauge	pressure sampler	Severity rank 0 through 5. Sampled every 5 seconds.
awc_singleflight_leader_fetches_total	counter	singleflight sampler	fuse feature only. Sampled every 5 seconds.
awc_singleflight_follower_requests_total	counter	singleflight sampler	fuse feature only. Sampled every 5 seconds.
awc_workspace_active_mounts	gauge	mount and unmount paths
awc_workspace_mounts_total	counter	mount path
awc_workspace_unmounts_total	counter	unmount path

The registry caps distinct label sets per instrument at 128. Recordings beyond the cap roll into a synthetic cardinality_overflow label (spec §19.2).

Signals and exit codes

Signal	Effect
SIGINT	Start a graceful drain.
SIGTERM	Start a graceful drain.

On a graceful drain, awcd stops the lease renewal task and the HTTP server, lets the gRPC server drain, reconciles live sessions against the authoritative lease, preserves uppers while the control plane is unreachable, and tears down mounts (spec §17.3).

Exit code	Cause
0	Clean shutdown after a drain signal.
1	Config, ticket-verifier, or control-plane client configuration error; health HTTP bind failure; base-mount build failure; daemon start failure; or a server exit with an error.

A missing config argument also prints usage and exits 1.

Current scope

This section tracks what is specified but not yet implemented or emitted in the current build. The build lags the spec in these areas.

Item	Status	Detail
Cache eviction	Specified-only	The pressure state machine computes the CacheEviction state and the evict_cache action, but no worker reclaims disposable cache objects. The module is pure.
Prefetch gate	Specified-only	The prefetch_allowed flag is computed from the state. No speculative prefetch path exists in awcd to read it.
awc_host_free_inodes gauge	Emitted	The pressure sampler sets it every 5 s from the newest probe sample.
awc_host_upper_usage_bytes	Emitted, total only	The pressure sampler sets the host total every 5 s. Per-workspace breakdown (spec §17.8) has no emit site.
Cache corruption probe	Not wired	The CACHE health component reports ok with the detail "operational; corruption probe not wired".
Control-plane connectivity probe	Not wired	The CONTROL_PLANE health component reports unknown.
Object-store connectivity probe	Not wired	The OBJECT_STORE health component reports unknown.
Remote-read admission limiter	Not attached	Spec §10.10. The FetchCoordinator coalesces fetches but applies no concurrency bound.
Provider circuit breakers	Not attached	Spec §10.11.
AWCD_HEALTH_PORT	Indirect	The systemd unit sets AWCD_HEALTH_PORT=9100 for its awc-health-check probe only. The awcd listen address comes from health_http_listen or AWC_HEALTH_HTTP_LISTEN.