diff --git a/docs/RUNBOOK-PhaseC.md b/docs/RUNBOOK-PhaseC.md index 85feda3..f44dd23 100644 --- a/docs/RUNBOOK-PhaseC.md +++ b/docs/RUNBOOK-PhaseC.md @@ -35,50 +35,50 @@ ci validate host # smoke test the alias The script defines two helpers: -| Helper | Runs as | Use for | -|---------|---------------|---------| +| Helper | Runs as | Use for | +| ------- | ------------------ | --------------------------------------- | | `ci` | `ci-runner` (sudo) | everything (writes keyring, starts VMs) | -| `ci-me` | current user | read-only checks where sudo is noise | +| `ci-me` | current user | read-only checks where sudo is noise | ### 0.3 Live environment -| Thing | Value | -|-------|-------| -| Service user | `ci-runner` · Prod venv `/opt/ci/venv/bin/python` | -| Config | `/var/lib/ci/config.toml` | -| Linux template | `/var/lib/ci/templates/LinuxBuild2404/LinuxBuild2404.vmx` (snap `BaseClean-Linux`) | -| Windows templates | `/var/lib/ci/templates/WinBuild2025` · `WinBuild2022` (snap `BaseClean`) | -| Clones / Artifacts | `/var/lib/ci/build-vms` · `/var/lib/ci/artifacts` | -| Linux SSH key | `/var/lib/ci/keys/ci_linux` · cred target `BuildVMGuest` | +| Thing | Value | +| ------------------ | ---------------------------------------------------------------------------------- | +| Service user | `ci-runner` · Prod venv `/opt/ci/venv/bin/python` | +| Config | `/var/lib/ci/config.toml` | +| Linux template | `/var/lib/ci/templates/LinuxBuild2404/LinuxBuild2404.vmx` (snap `BaseClean-Linux`) | +| Windows templates | `/var/lib/ci/templates/WinBuild2025` · `WinBuild2022` (snap `BaseClean`) | +| Clones / Artifacts | `/var/lib/ci/build-vms` · `/var/lib/ci/artifacts` | +| Linux SSH key | `/var/lib/ci/keys/ci_linux` · cred target `BuildVMGuest` | --- ## 1. Quick reference — every command -| Command | Does | Section | -|---------|------|---------| -| `ci validate host` | host prerequisites check (no VM) | §2 | -| `ci validate guest --host IP` | probe a running guest's transport | §2 | -| `ci monitor disk` | free-space alert | §2 | -| `ci monitor runner` | watch + auto-restart act_runner | §2 | -| `ci report job` | job summaries from JSONL logs | §2 | -| `ci job ...` | **full pipeline** clone→build→collect→destroy | §3 | -| `ci vm new ...` | linked-clone a template | §4 | -| `ci wait-ready --vmx ...` | poll until SSH/WinRM ready | §4 | -| `ci build run ...` | run a build in a running guest | §4 | -| `ci artifacts collect ...` | copy artifacts out of a guest | §4 | -| `ci vm remove --vmx ...` | stop + delete one clone | §4 | -| `ci vm cleanup` | sweep orphaned clones + stale locks | §4 / §6 | -| `ci vmware --vmx ...` | open a VMX in the VMware Workstation GUI | §4 | -| `ci creds set --user ...` | store guest credentials in keyring | §5 | -| `ci template deploy-linux` | build the Linux template VM | §5 | -| `ci template prepare-linux` | provision Linux template over SSH | §5 | -| `ci template prepare-win` | provision Windows template over WinRM | §5 | -| `ci template backup` | timestamped copy of template dirs | §5 | -| `ci retention run` | purge old artifacts/logs | §6 | -| `ci bench measure ...` | phase-timing benchmark | §7 | -| `ci bench run ...` | concurrency burn-in | §7 | -| `ci smoke run ...` | one E2E job + assertions | §7 | +| Command | Does | Section | +| ----------------------------- | --------------------------------------------- | ------- | +| `ci validate host` | host prerequisites check (no VM) | §2 | +| `ci validate guest --host IP` | probe a running guest's transport | §2 | +| `ci monitor disk` | free-space alert | §2 | +| `ci monitor runner` | watch + auto-restart act_runner | §2 | +| `ci report job` | job summaries from JSONL logs | §2 | +| `ci job ...` | **full pipeline** clone→build→collect→destroy | §3 | +| `ci vm new ...` | linked-clone a template | §4 | +| `ci wait-ready --vmx ...` | poll until SSH/WinRM ready | §4 | +| `ci build run ...` | run a build in a running guest | §4 | +| `ci artifacts collect ...` | copy artifacts out of a guest | §4 | +| `ci vm remove --vmx ...` | stop + delete one clone | §4 | +| `ci vm cleanup` | sweep orphaned clones + stale locks | §4 / §6 | +| `ci vmware --vmx ...` | open a VMX in the VMware Workstation GUI | §4 | +| `ci creds set --user ...` | store guest credentials in keyring | §5 | +| `ci template deploy-linux` | build the Linux template VM | §5 | +| `ci template prepare-linux` | provision Linux template over SSH | §5 | +| `ci template prepare-win` | provision Windows template over WinRM | §5 | +| `ci template backup` | timestamped copy of template dirs | §5 | +| `ci retention run` | purge old artifacts/logs | §6 | +| `ci bench measure ...` | phase-timing benchmark | §7 | +| `ci bench run ...` | concurrency burn-in | §7 | +| `ci smoke run ...` | one E2E job + assertions | §7 | --- @@ -232,8 +232,21 @@ inherited `XAUTHORITY` (the desktop user's `~/.Xauthority`) is unreadable by ci-runner, so `vmware` drops it and relies on the xhost grant — which is normally already present (`xhost` lists `SI:localuser:ci-runner`). -**Only if you get `cannot open X display`**: the grant is missing. Add it from -the logged-in desktop user (not ci-runner): +The xhost grant is **per-session**: it is wiped on logout / X restart. To make +it persistent, `setup-host-linux.sh` (Step 8) installs an XDG autostart entry in +the desktop user's home that re-applies the grant at every graphical login. You +can (re)run it standalone: +```bash +# As root (or via sudo); --desktop-user defaults to $SUDO_USER +sudo CI_RUN_USER=ci-runner bash scripts/install-vmware-gui-access.sh --desktop-user simone +# Remove the grant + autostart entry: +sudo bash scripts/install-vmware-gui-access.sh --desktop-user simone --uninstall +``` +This applies the grant immediately (if a display is reachable) and writes +`~simone/.config/autostart/ci-runner-xhost.desktop` for future logins. + +**One-off, without the script**: add the grant from the logged-in desktop user +(not ci-runner): ```bash xhost +SI:localuser:ci-runner ``` @@ -341,13 +354,13 @@ Rollback: `sudo ./setup-host-linux.sh --with-pwsh` reinstalls pwsh; the original ## 9. Troubleshooting quick map -| Symptom | Cause | Fix | -|---------|-------|-----| -| `No such command 'bench'` | prod venv not refreshed | §0.1 `pip install .` | -| `validate host` keyring `[FAIL]` | credential missing/wrong user | §5 `creds set` | -| `validate guest` connect failure | VM not booted / wrong IP / firewall | confirm VM running + IP | -| `bench run` round FAIL: orphan | clone not destroyed | `ci vm cleanup`; check vmrun + `/var/lib/ci/build-vms` | -| WinRM jobs drop at concurrency 4 | known instability (TODO §3.6) | use `--concurrency 3` | -| IP acquire timeout (Linux) | vmnet8 DHCP wedged | `sudo systemctl restart vmware-networks` | +| Symptom | Cause | Fix | +| -------------------------------- | ------------------------------------- | ----------------------------------------------------------------------------------- | +| `No such command 'bench'` | prod venv not refreshed | §0.1 `pip install .` | +| `validate host` keyring `[FAIL]` | credential missing/wrong user | §5 `creds set` | +| `validate guest` connect failure | VM not booted / wrong IP / firewall | confirm VM running + IP | +| `bench run` round FAIL: orphan | clone not destroyed | `ci vm cleanup`; check vmrun + `/var/lib/ci/build-vms` | +| WinRM jobs drop at concurrency 4 | known instability (TODO §3.6) | use `--concurrency 3` | +| IP acquire timeout (Linux) | vmnet8 DHCP wedged | `sudo systemctl restart vmware-networks` | | `vmrun` "operation was canceled" | vmmon/vmnet unbuilt after kernel bump | `sudo vmware-modconfig --console --install-all` (or reboot; systemd guard rebuilds) | -| `ci: command not found` | alias not loaded | `source ~/.bashrc` or re-run §0.2 | +| `ci: command not found` | alias not loaded | `source ~/.bashrc` or re-run §0.2 |