Wire a CI pipeline¶

This tutorial teaches you to drive AWC (Agent Workspace Cache) from a continuous integration job. You import the repository as a snapshot, fork a private workspace, mount it, run a build, then persist or triage the result. When you finish, your CI job runs against a cached fork and stores the output as an immutable snapshot.

AWC exposes five user-level primitives (spec §5.1):

snapshot → fork → mount → checkpoint → diff/export

You wire every one of them into one job.

Before you start¶

This tutorial assumes you already have an AWC deployment you can reach from the runner. You need:

A control plane (awc-control) you can reach over HTTPS.
A host daemon (awcd) on the runner, with its Unix socket available.
The awcctl binary, built with the s3 feature, on PATH.
Credentials and endpoints exposed as environment variables (below).
jq, to parse the JSON that awcctl --output json prints.
The Rust toolchain, because the example build runs make test.

If you do not have a deployment yet, start with run AWC locally and come back here.

Environment the CLI expects¶

awcctl reads its target services from environment variables. Set these on the runner before you call any command:

Variable	Meaning
`AWC_CONTROL_ENDPOINT`	Base URL of the control plane, for example `https://control.example`.
`AWC_TOKEN`	Bearer token that authenticates you to the control plane.
`AWC_TENANT`	Your tenant id. Required by snapshot and workspace commands.
`AWC_PROJECT`	Your project id. Required by most control-plane commands.
`AWC_HOST_SOCKET`	Path to the local `awcd` socket, for example `/run/awcd/awcd.sock`.
`AWC_CAS_URL`	Object store the snapshot upload writes to, for example `s3://awc-cas`.

Mutating commands also honor --idempotency-key (spec §13.1). In CI, set a unique key per attempt so a retry does not create a second resource.

The loop you will build¶

Each run of the job does this:

snapshot create  →  wait for READY  →  fork  →  mount  →  make test
                                                              │
                          ┌──────────────────────────────────┴───┐
                     checkpoint                              export
                     (persist the good state)            (triage a failure)
                                                              │
                                                          unmount

You will reproduce each box below.

1. Import the repository as a snapshot¶

Your CI provider checks the repository out onto the runner for you. Treat that working tree as the prepared directory, and import it.

BASE=$(awcctl --output json snapshot create \
        --from-dir . \
        --source-type git \
        --name "ci-${GITHUB_SHA::12}" \
        --label "ci.sha=${GITHUB_SHA}" \
    | jq -r .snapshot_id)

The --from-dir . flag points at the checked-out tree. The --source-type git flag records where the bytes came from. awcctl prints the new snapshot id as JSON.

--source-type records provenance only

Today the CLI walks the directory you give --from-dir for every source kind. It does not clone a Git repository and it does not extract a tar archive. --source-type git and --source-type tar only tag the snapshot so you can tell its origin apart later.

The spec (§5.2) intends a Git import at an immutable revision and a CI tar import as distinct paths. The current build treats both as provenance metadata over the same directory import. When you need dedicated Git or tar handling, see import from Git and import from a tar archive.

A successful snapshot returns a stable id, a root digest, and file and byte counts (spec §5.2). You capture the id in BASE.

2. Wait for the snapshot to become ready¶

A new snapshot starts in the VALIDATING state. The worker reads the object closure back from the store and checks it (spec §13.5, §20.2). A workspace can fork only from a READY snapshot, so poll until the state settles.

until [ "$(awcctl --output json snapshot inspect "$BASE" | jq -r .state)" = "READY" ]; do
  echo "waiting for $BASE to validate..."
  sleep 2
done

Why the wait exists

The closure check is what makes a READY snapshot trustworthy: it never references a missing object (spec §20.2). Until that check passes, the control plane refuses to fork from the snapshot.

3. Fork a workspace¶

Fork the snapshot into a private, mutable workspace. Forking copies metadata only. It does not copy content objects (spec §5.3), so it is cheap and fast.

WS=$(awcctl --output json workspace fork \
        --snapshot "$BASE" \
        --ttl 1h \
        --name "ci-run-${GITHUB_RUN_ID}" \
    | jq -r .id)

The --ttl 1h flag sets the workspace lifetime. The control plane returns a workspace id, which you store in WS. See fork a workspace for the full response.

4. Mount the workspace¶

Mount the workspace onto a target directory on the runner. The mount returns a mount_session_id, so capture it in one step. Checkpoint and unmount take the session, not the workspace.

SESSION=$(awcctl --output json workspace mount "$WS" --target "$PWD/ws" \
    | jq -r .mount_session_id)

The mount is read-write by default, which is what a build needs. Add --read-only for a mount that only inspects the tree. A read-write mount acquires an exclusive writer lease (spec §5.4). Only one writer can hold the workspace at a time.

The target must be allow-listed

awcd validates the target path against an allow-list before it binds (spec §13.9). It does not accept an arbitrary host path. Point --target at a directory under the root your awcd config permits. See configuration.

5. Run the build and tests¶

Work inside the mount as you would in any checkout. The merged view presents the snapshot contents, and writes land in a private upper directory.

cd ./ws
make test
cd ..

Here make test is the repository's own test target. It runs the full workspace unit and integration suite. Treat this step as your "first useful command" (spec §20.10). The time from fork to this command finishing is a primary product metric for AWC.

Capture the exit code so the job can branch on the result without aborting the cleanup steps.

rc=0
( cd ./ws && make test ) || rc=$?

6. Persist the result, or export the diff¶

Dirty state lives only in the mount until you checkpoint it. A checkpoint scans the upper directory, uploads new objects, and publishes a new immutable snapshot that becomes the workspace head (spec §5.5). Export then works on that snapshot.

On success: checkpoint¶

When the tests pass, checkpoint the clean state and keep the new snapshot id as your build artifact.

HEAD=$(awcctl --output json workspace checkpoint "$SESSION" \
        --mode cooperative \
        --message "ci green for ${GITHUB_SHA}" \
        --idempotency-key "ci-${GITHUB_RUN_ID}-ckpt" \
    | jq -r .snapshot_id)

On failure: checkpoint, then export the diff¶

awcctl snapshot export compares two snapshots. It cannot read a dirty mount directly, so checkpoint the failing state first. Then export that checkpoint against the original base to get a patch you can read during triage.

HEAD=$(awcctl --output json workspace checkpoint "$SESSION" \
        --mode cooperative \
        --message "ci triage for ${GITHUB_SHA}" \
    | jq -r .snapshot_id)

awcctl snapshot export "$HEAD" \
    --against "$BASE" \
    --format patch-bundle \
    --out ci.awcpatch

Upload ci.awcpatch as a build artifact. The patch-bundle format keeps unified diffs for text files and full payloads for binary files. That is enough to apply the changes against the declared base (spec §5.6). See diff and export.

Cooperative versus quiesced

The default checkpoint mode is quiesced, which pauses the writer while it cuts the snapshot (spec §5.5). Use --mode cooperative in CI when no external process writes to the mount during the checkpoint. Cooperative is best-effort and skips the freeze. That is the right trade for a build job whose commands have already stopped. See checkpoint a workspace.

7. Unmount and release the lease¶

Release the writer lease so the workspace can be forked or mounted again later. The default disposition is discard-local, which throws away any upper directory state the checkpoint did not capture. That is correct here, because you already persisted what you wanted in step 6.

awcctl workspace unmount "$SESSION"

Use --disposition checkpoint to have unmount cut a final checkpoint for you. Use --disposition preserve-local to keep the local upper directory on the host (spec §13.13). For a CI job, discard-local is the clean choice.

8. Put it together in a GitHub Actions job¶

The snippet below runs the whole loop. It assumes a self-hosted Linux runner with awcctl, awcd, jq, and the Rust toolchain on PATH. Build awcctl with the s3 feature. Store the control endpoint, token, and CAS URL as repository secrets.

name: ci

on:
  push:
    branches: [main]
  pull_request:

jobs:
  awc:
    runs-on: [self-hosted, linux, awc]
    steps:
      - uses: actions/checkout@v4

      - name: Run the build against a cached workspace
        env:
          AWC_CONTROL_ENDPOINT: ${{ secrets.AWC_CONTROL_ENDPOINT }}
          AWC_TOKEN: ${{ secrets.AWC_TOKEN }}
          AWC_TENANT: ${{ vars.AWC_TENANT }}
          AWC_PROJECT: ${{ vars.AWC_PROJECT }}
          AWC_HOST_SOCKET: /run/awcd/awcd.sock
          AWC_CAS_URL: ${{ secrets.AWC_CAS_URL }}
        run: |
          set -euo pipefail

          BASE=$(awcctl --output json snapshot create \
                  --from-dir . --source-type git --name "ci-${GITHUB_SHA::12}" \
                | jq -r .snapshot_id)

          until [ "$(awcctl --output json snapshot inspect "$BASE" | jq -r .state)" = "READY" ]; do
            sleep 2
          done

          WS=$(awcctl --output json workspace fork --snapshot "$BASE" --ttl 1h | jq -r .id)
          mkdir -p ws
          SESSION=$(awcctl --output json workspace mount "$WS" --target "$PWD/ws" | jq -r .mount_session_id)

          rc=0
          ( cd ./ws && make test ) || rc=$?

          HEAD=$(awcctl --output json workspace checkpoint "$SESSION" \
                  --mode cooperative --message "ci ${GITHUB_SHA}" \
                | jq -r .snapshot_id)
          awcctl snapshot export "$HEAD" --against "$BASE" \
                  --format patch-bundle --out ci.awcpatch

          awcctl workspace unmount "$SESSION"
          exit "$rc"

      - name: Upload the diff artifact
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: ci-diff
          path: ci.awcpatch

AWC does not ship a runner image

awcctl and awcd are not on a public package registry. You build and install them on the runner yourself. See build and package. The job above also assumes the control plane is already deployed and that the runner host is registered with it.

What you learned¶

You wired the five AWC primitives into one CI job:

You imported a checked-out tree with awcctl snapshot create.
You waited for the snapshot to reach READY before forking.
You forked metadata-only with awcctl workspace fork.
You mounted the workspace read-write and ran a real build inside it.
You checkpointed the result and exported the diff for triage.
You released the writer lease with awcctl workspace unmount.

Where to go next¶

Import from Git and import from a tar archive cover source-specific preparation in depth.
Fork a workspace explains batch fork and TTL.
Checkpoint a workspace compares the quiesced and cooperative modes.
Diff and export documents every export format.
Run AWC locally stands up the same services on one machine for rehearsal.