Skip to content

systemd units

systemd units and the VM deployment procedure for AWC. This covers signed binaries, service units, a dedicated user and directories, capability boundaries, journald log integration, configuration, host enrollment, and drain, restart, and rollback behavior.

The artifacts are the systemd units and the packaging scripts:

  • the unit files: awcd.service, awc-worker.service, and awc-control.service;
  • the packaging scripts: build, sign, package, install, enroll, drain, and rollback. See Build and package.

Services

Unit Role Privilege
awcd.service per-host mount/cache daemon unprivileged awc user and minimal capabilities (CAP_SYS_ADMIN for mount/cgroup/setns; /dev/fuse for FUSE)
awc-worker.service background job-queue worker unprivileged awc user, no capabilities
awc-control.service control plane REST and internal mTLS API unprivileged awc user

awcd is the only privileged service. It runs as the awc user but holds CapabilityBoundingSet=CAP_SYS_ADMIN CAP_DAC_READ_SEARCH CAP_CHOWN CAP_FOWNER (as ambient capabilities) so it can perform the privileged mount, OverlayFS, FUSE, and cgroup duties in its responsibilities. It also deliberately omits the mount-namespace-creating sandboxing directives so its FUSE and OverlayFS mounts stay visible in the host mount namespace. See the hardening section below and the in-unit comments for exactly what is and is not applied.

Capability boundaries and hardening

The worker and control units take a maximal sandboxing posture: NoNewPrivileges, ProtectSystem=strict, ProtectHome, ProtectKernel*, ProtectControlGroups, ProtectClock, PrivateDevices, PrivateTmp, MemoryDenyWriteExecute, RestrictAddressFamilies=AF_INET AF_UNIX, and a SystemCallFilter=@system-service baseline.

awcd cannot take that posture. It is a privileged mount-publishing daemon whose FUSE base filesystem and per-workspace OverlayFS mounts must be visible in the host mount namespace — to clients, the worker, and the configure and acceptance checks. Each of ProtectSystem, ProtectHome, PrivateTmp, ProtectKernelModules, ProtectKernelLogs, ProtectClock, ProtectProc, and ReadWritePaths forces a private mount namespace whose mounts do not propagate back to the host (systemd.exec(5)). Using them hides awcd's own mounts from everything outside the unit. So awcd deliberately omits all eight — the same posture the Kubernetes DaemonSet takes to publish mounts (mountPropagation: Bidirectional).

What awcd keeps, and why each is safe:

  • ProtectHostname — UTS namespace only, not a mount namespace. It does not affect mount visibility.
  • StateDirectory, CacheDirectory, LogsDirectory, RuntimeDirectory, and ConfigurationDirectory — these bind-mount awcd's own directories but do not create a private mount namespace and do not disconnect mount propagation, so they do not hide awcd's mounts. RuntimeDirectory=awcd is also the sole creator of /run/awcd, the gRPC socket dir on the /run tmpfs, which the unprivileged awc user cannot create itself.
  • CapabilityBoundingSet and AmbientCapabilities=CAP_SYS_ADMIN CAP_DAC_READ_SEARCH CAP_CHOWN CAP_FOWNER for mount, cgroup, and FUSE duties.
  • NoNewPrivileges, MemoryDenyWriteExecute, RestrictAddressFamilies, LockPersonality, RestrictRealtime, RestrictSUIDSGID, RemoveIPC, DevicePolicy=closed with DeviceAllow=/dev/fuse rw, and SystemCallFilter=@system-service @mount.

In short, awcd matches the worker and control posture on every axis that does not require a private mount namespace. The namespace-creating directives are the intentional, documented exception. systemd-analyze security awcd.service therefore reports a weaker score than a sandboxed unit, by design.

Validate offline with:

systemd-analyze verify   deploy/systemd/*.service
systemd-analyze security deploy/systemd/awcd.service   # hardening score

Logging

All three units log to journald (StandardOutput=journal, SyslogIdentifier=). Tail with journalctl -u awcd -f. Structured events flow through the same stderr stream. See observability.

End-to-end VM procedure

# On a build host:
packaging/package.sh                     # -> dist/awc-<version>-<target>.tar.gz (signed)

# Copy the tarball to the VM, then as root:
tar xzf awc-<version>-<target>.tar.gz
cd awc-<target>
./install.sh                             # versioned /opt/awc layout, awc user, units, defaults
./enroll.sh --control-endpoint https://control.example.invalid
systemctl enable --now awcd awc-worker awc-control
journalctl -u awcd -f

For the full build and install steps, see Build and package and Install AWC.

Host enrollment

enroll.sh writes /etc/awc/awcd.toml and /etc/awc/awcd.env and creates the state directories. It is local enrollment: it mints a host id and writes a placeholder control endpoint. For a real deployment, replace the placeholder and provision mTLS identity. Host registration and the mTLS identity lifecycle are implemented, and awcd wires the control-plane client (lease lifecycle, checkpoint publisher, lease renewal) at startup. Outside AWC_DEV_MODE=1, awcd and awc-control MUST present TLS credentials or fail to start.

Drain, restart, rollback

  • Drain (drain.sh): sends SIGTERM to awcd, which reconciles live sessions against the authoritative lease, preserves uppers while the control plane is unreachable, and tears down mounts. systemctl stop bounds the drain via the unit's TimeoutStopSec. See Drain and restart a host.
  • Restart: Restart=on-failure with StartLimitBurst and StartLimitIntervalSec bounds flapping. Type=exec with an ExecStartPost liveness gate marks the unit active only once /healthz answers.
  • Rollback (rollback.sh): every install keeps its version under /opt/awc/versions/. current is a symlink. Rollback repoints the symlink, reloads systemd, and restarts the active services onto the older binaries. See Upgrade a deployment.

Current scope (honest)

  • awc-control.service serves the live control-plane API and requires its database, signing, service-token, and internal TLS configuration.
  • Featureless test builds can use a no-op base mount. Packaged releases enable the FUSE basefs. The unit grants /dev/fuse and the required mount capability.
  • Packaged workers include the S3 transport. Configure /etc/awc/worker.env with the deployment's durable CAS endpoint and credentials.