From 9ecb767b187c2d12ef7225e8bcd106d2eebef84c Mon Sep 17 00:00:00 2001 From: Simone Date: Mon, 8 Jun 2026 21:37:24 +0200 Subject: [PATCH] =?UTF-8?q?docs(runbook):=20fix=20=C2=A73=20job=20example?= =?UTF-8?q?=20=E2=80=94=20required=20template=20flags?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `job` command enforces --template-path (job.py:447) and defaults --snapshot-name to BaseClean, which is wrong for Linux. The §3 example omitted both and failed with "--template-path is required". Add both flags (BaseClean-Linux) and note them as required. --- docs/RUNBOOK-PhaseC.md | 91 +++++++++++++++++++++++------------------- 1 file changed, 49 insertions(+), 42 deletions(-) diff --git a/docs/RUNBOOK-PhaseC.md b/docs/RUNBOOK-PhaseC.md index c1785cf..95544d1 100644 --- a/docs/RUNBOOK-PhaseC.md +++ b/docs/RUNBOOK-PhaseC.md @@ -47,43 +47,43 @@ sudo); `--uninstall` removes the block. The script defines two helpers: ### 0.3 The live environment -| Item | Value | -| ------------------ | ---------------------------------------------------------------------------------- | -| Service user | `ci-runner` · production venv `/opt/ci/venv/bin/python` | -| Configuration | `/var/lib/ci/config.toml` | +| Item | Value | +| ------------------ | -------------------------------------------------------------------------------------- | +| Service user | `ci-runner` · production venv `/opt/ci/venv/bin/python` | +| Configuration | `/var/lib/ci/config.toml` | | Linux template | `/var/lib/ci/templates/LinuxBuild2404/LinuxBuild2404.vmx` (snapshot `BaseClean-Linux`) | -| Windows templates | `/var/lib/ci/templates/WinBuild2025` · `WinBuild2022` (snapshot `BaseClean`) | -| Clones / artifacts | `/var/lib/ci/build-vms` · `/var/lib/ci/artifacts` | -| Linux SSH access | key `/var/lib/ci/keys/ci_linux`, user `ci_build`, credential target `BuildVMGuest` | +| Windows templates | `/var/lib/ci/templates/WinBuild2025` · `WinBuild2022` (snapshot `BaseClean`) | +| Clones / artifacts | `/var/lib/ci/build-vms` · `/var/lib/ci/artifacts` | +| Linux SSH access | key `/var/lib/ci/keys/ci_linux`, user `ci_build`, credential target `BuildVMGuest` | --- ## 1. Command index -| Command | Purpose | Section | -| ----------------------------- | --------------------------------------------- | ------- | -| `ci validate host` | Check host prerequisites (no VM) | [§2](#2-health-monitoring--reporting) | -| `ci validate guest --host IP` | Probe a running guest's transport | [§2](#2-health-monitoring--reporting) | -| `ci monitor disk` | Free-space alert | [§2](#2-health-monitoring--reporting) | -| `ci monitor runner` | Watch and auto-restart act_runner | [§2](#2-health-monitoring--reporting) | -| `ci report job` | Job summaries from the JSONL logs | [§2](#2-health-monitoring--reporting) | -| `ci job ...` | **Full pipeline**: clone → build → collect → destroy | [§3](#3-full-pipeline-job) | -| `ci vm new ...` | Linked-clone a template | [§4](#4-manual-vm-lifecycle) | -| `ci wait-ready --vmx ...` | Poll until SSH/WinRM is ready | [§4](#4-manual-vm-lifecycle) | -| `ci build run ...` | Run a build inside a running guest | [§4](#4-manual-vm-lifecycle) | -| `ci artifacts collect ...` | Copy artifacts out of a guest | [§4](#4-manual-vm-lifecycle) | -| `ci vm remove --vmx ...` | Stop and delete one clone | [§4](#4-manual-vm-lifecycle) | -| `ci vm cleanup` | Sweep orphaned clones and stale locks | [§4](#4-manual-vm-lifecycle) · [§6](#6-maintenance) | -| `ci vmware --vmx ...` | Open a VMX in the VMware Workstation GUI | [§4](#4-manual-vm-lifecycle) | -| `ci creds set --user ...` | Store guest credentials in the keyring | [§5](#5-credentials--template-provisioning) | -| `ci template deploy-linux` | Build the Linux template VM | [§5](#5-credentials--template-provisioning) | -| `ci template prepare-linux` | Provision the Linux template over SSH | [§5](#5-credentials--template-provisioning) | -| `ci template prepare-win` | Provision a Windows template over WinRM | [§5](#5-credentials--template-provisioning) | -| `ci template backup` | Timestamped copy of the template directories | [§5](#5-credentials--template-provisioning) | -| `ci retention run` | Purge old artifacts and logs | [§6](#6-maintenance) | -| `ci bench measure ...` | Phase-timing benchmark | [§7](#7-phase-c-testing) | -| `ci bench run ...` | Concurrency burn-in | [§7](#7-phase-c-testing) | -| `ci smoke run ...` | One end-to-end job with assertions | [§7](#7-phase-c-testing) | +| Command | Purpose | Section | +| ----------------------------- | ---------------------------------------------------- | --------------------------------------------------- | +| `ci validate host` | Check host prerequisites (no VM) | [§2](#2-health-monitoring--reporting) | +| `ci validate guest --host IP` | Probe a running guest's transport | [§2](#2-health-monitoring--reporting) | +| `ci monitor disk` | Free-space alert | [§2](#2-health-monitoring--reporting) | +| `ci monitor runner` | Watch and auto-restart act_runner | [§2](#2-health-monitoring--reporting) | +| `ci report job` | Job summaries from the JSONL logs | [§2](#2-health-monitoring--reporting) | +| `ci job ...` | **Full pipeline**: clone → build → collect → destroy | [§3](#3-full-pipeline-job) | +| `ci vm new ...` | Linked-clone a template | [§4](#4-manual-vm-lifecycle) | +| `ci wait-ready --vmx ...` | Poll until SSH/WinRM is ready | [§4](#4-manual-vm-lifecycle) | +| `ci build run ...` | Run a build inside a running guest | [§4](#4-manual-vm-lifecycle) | +| `ci artifacts collect ...` | Copy artifacts out of a guest | [§4](#4-manual-vm-lifecycle) | +| `ci vm remove --vmx ...` | Stop and delete one clone | [§4](#4-manual-vm-lifecycle) | +| `ci vm cleanup` | Sweep orphaned clones and stale locks | [§4](#4-manual-vm-lifecycle) · [§6](#6-maintenance) | +| `ci vmware --vmx ...` | Open a VMX in the VMware Workstation GUI | [§4](#4-manual-vm-lifecycle) | +| `ci creds set --user ...` | Store guest credentials in the keyring | [§5](#5-credentials--template-provisioning) | +| `ci template deploy-linux` | Build the Linux template VM | [§5](#5-credentials--template-provisioning) | +| `ci template prepare-linux` | Provision the Linux template over SSH | [§5](#5-credentials--template-provisioning) | +| `ci template prepare-win` | Provision a Windows template over WinRM | [§5](#5-credentials--template-provisioning) | +| `ci template backup` | Timestamped copy of the template directories | [§5](#5-credentials--template-provisioning) | +| `ci retention run` | Purge old artifacts and logs | [§6](#6-maintenance) | +| `ci bench measure ...` | Phase-timing benchmark | [§7](#7-phase-c-testing) | +| `ci bench run ...` | Concurrency burn-in | [§7](#7-phase-c-testing) | +| `ci smoke run ...` | One end-to-end job with assertions | [§7](#7-phase-c-testing) | --- @@ -137,6 +137,9 @@ IP=$(sudo -u ci-runner vmrun getGuestIPAddress /var/lib/ci/templates/LinuxBuild2 ci validate guest --host "$IP" --ssh --guest-os linux # Windows guest over WinRM +IP=$(sudo -u ci-runner vmrun getGuestIPAddress /var/lib/ci/templates/WinBuild2022/WinBuild2022.vmx) +ci validate guest --host "$IP" --winrm --guest-os windows + IP=$(sudo -u ci-runner vmrun getGuestIPAddress /var/lib/ci/templates/WinBuild2025/WinBuild2025.vmx) ci validate guest --host "$IP" --winrm --guest-os windows ``` @@ -179,6 +182,8 @@ by hand too. ```bash ci job \ --job-id manual-$(date +%s) \ + --template-path /var/lib/ci/templates/LinuxBuild2404/LinuxBuild2404.vmx \ + --snapshot-name BaseClean-Linux \ --repo-url https://gitea.emulab.it/Simone/myrepo.git \ --branch main \ --guest-os linux \ @@ -186,7 +191,9 @@ ci job \ --use-git-clone ``` -`--job-id`, `--repo-url`, and `--branch` are required. Other useful flags: +`--job-id`, `--repo-url`, `--branch`, and `--template-path` are required. The +default `--snapshot-name` is `BaseClean`, so Linux jobs must pass +`--snapshot-name BaseClean-Linux` explicitly. Other useful flags: - **Source**: `--commit`, `--submodules` / `--no-submodules`, `--use-git-clone` / `--host-clone`, `--use-shared-cache` @@ -443,13 +450,13 @@ original `.ps1` scripts remain in `scripts/archived/`. ## 9. Troubleshooting -| Symptom | Cause | Fix | -| ---------------------------------- | --------------------------------------- | ----------------------------------------------------------------------------------- | -| `No such command 'bench'` | Production venv not refreshed | [§0.1](#01-refresh-the-production-venv) — `pip install .` | -| `validate host` keyring `[FAIL]` | Credential missing or wrong user | [§5](#5-credentials--template-provisioning) — `creds set` | -| `validate guest` connect failure | VM not booted, wrong IP, or firewall | Confirm the VM is running and re-check its IP | -| `bench run` round FAIL: orphan | Clone was not destroyed | `ci vm cleanup`; check vmrun and `/var/lib/ci/build-vms` | -| WinRM jobs drop at concurrency 4 | Known instability (TODO §3.6) | Use `--concurrency 3` | -| IP-acquire timeout (Linux) | vmnet8 DHCP wedged | `sudo systemctl restart vmware-networks` | -| `vmrun` "operation was canceled" | vmmon/vmnet unbuilt after a kernel bump | `sudo vmware-modconfig --console --install-all` (or reboot; the systemd guard rebuilds) | -| `ci: command not found` | Alias not loaded | `source ~/.bashrc`, or re-run [§0.2](#02-install-the-ci-alias) | +| Symptom | Cause | Fix | +| -------------------------------- | --------------------------------------- | --------------------------------------------------------------------------------------- | +| `No such command 'bench'` | Production venv not refreshed | [§0.1](#01-refresh-the-production-venv) — `pip install .` | +| `validate host` keyring `[FAIL]` | Credential missing or wrong user | [§5](#5-credentials--template-provisioning) — `creds set` | +| `validate guest` connect failure | VM not booted, wrong IP, or firewall | Confirm the VM is running and re-check its IP | +| `bench run` round FAIL: orphan | Clone was not destroyed | `ci vm cleanup`; check vmrun and `/var/lib/ci/build-vms` | +| WinRM jobs drop at concurrency 4 | Known instability (TODO §3.6) | Use `--concurrency 3` | +| IP-acquire timeout (Linux) | vmnet8 DHCP wedged | `sudo systemctl restart vmware-networks` | +| `vmrun` "operation was canceled" | vmmon/vmnet unbuilt after a kernel bump | `sudo vmware-modconfig --console --install-all` (or reboot; the systemd guard rebuilds) | +| `ci: command not found` | Alias not loaded | `source ~/.bashrc`, or re-run [§0.2](#02-install-the-ci-alias) |