`vmgui` sent the child's stderr to /dev/null, so when the X display was
unreachable (the common case: sudo -u ci-runner loses the desktop user's
XAUTHORITY and xhost wasn't granted) the GUI died with "cannot open display"
and the command still printed success — "nothing opens".
Now it:
- pre-flights the display with xdpyinfo and aborts with the exact
`xhost +SI:localuser:ci-runner` hint instead of spawning a doomed process;
- drops an inherited XAUTHORITY that this user can't read (falls back to
host-based xhost auth);
- logs the GUI's stdio to /tmp/vmgui-*.log and grace-polls 1.5s, surfacing the
log if the GUI exits immediately;
- errors (non-zero) when no DISPLAY is resolvable.
Runbook §4.7 updated with the "if nothing opens" guidance. 13 tests; green ≥90%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add top-level `vmgui` to open the VMware Workstation GUI (renamed from the
earlier `vm open`). --vmx is optional: with it the GUI opens that VM, without
it the GUI starts bare. Power-on/fullscreen flags ignored (with a notice) when
no VMX is given. Needs an X display — sudo -u ci-runner strips $DISPLAY, so
--display / xhost; see runbook §4.7. 9 tests; suite green ≥90%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rewrite docs/RUNBOOK-PhaseC.md as a full manual-command reference: every
ci_orchestrator command with a working example as the ci-runner user, the `ci`
alias setup, quick-reference table, manual VM lifecycle (incl. vm open GUI
launch with the xhost/--display procedure), templating, maintenance, Phase C
testing, cutover checklist and a troubleshooting map.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
scripts/install-ci-alias.sh installs `ci` (runs the orchestrator as ci-runner
via the prod venv) and `ci-me` (current user). Idempotent, marker-bounded;
supports --system (/etc/profile.d), --uninstall. setup-host-linux.sh gains a
step [7/7] that installs it system-wide, passing CI_PROD_PYTHON/CI_RUN_USER.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Open a VMX in the interactive Workstation GUI (not the headless vmrun path) for
hands-on debugging of a clone. Spawns `vmware` detached (own session, stdio
detached) and returns the child PID.
Flags: --vmx (required), --power-on (-x), --fullscreen (-X), --new-window (-n),
--display (sudo -u ci-runner strips $DISPLAY, so allow forcing it),
--vmware-path. Warns when no X display is set. 7 tests; vm.py 96%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CLAUDE.md: Linux host no longer runs pwsh; PS 5.1 constraints scoped to
Windows guests + Windows runner.
- RUNBOOK: add "Phase C — Python equivalents" table; §6-§13 benchmark records
kept as historical provenance.
- TODO §3.7: close $IsWindows shim bug (resolved-by-replacement — bench run
bypasses the shim).
- plans: mark C1-C8 done + docs C9; flag operational cutover/uninstall pending.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Invert step 6 default: pwsh is no longer installed. Add --with-pwsh to opt in;
keep --skip-pwsh as a deprecated no-op alias. Update usage/help.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply analysis §4 option 1: pssa job runs on windows-build with
shell: powershell, gated to workflow_dispatch so push/PR don't queue it when
no Windows runner is registered. The python job stays on linux-build. No job
on a Linux runner uses pwsh anymore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove _Common.psm1, _Transport.psm1 and their Pester test, plus their sole
consumer Measure-CIBenchmark.ps1 — all replaced by `bench measure`. Leaving
the script would dangle an Import-Module on a deleted .psm1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add four native command groups so the Linux host no longer needs pwsh for
manual ops. __main__ registers all four at once, so they ship together.
- bench run (C2): N concurrent jobs in-process via concurrent.futures,
per-round orphan-clone + stale-lock assertions. Replaces
Test-CapacityBurnIn.ps1 + Start-BurnInTest*.ps1. No
Start-Job/pwsh/$IsWindows.
- bench measure (C3): clone/start/IP/transport/destroy timings appended to
benchmark.jsonl (legacy field names kept). Replaces
Measure-CIBenchmark.ps1.
- smoke run (C4): one E2E job; asserts exit 0 + artifact dir + job/success
event. ns7zip/nsinnounp presets. Replaces Test-Smoke.ps1
+ Test-*Build-Linux.ps1.
- validate host (C5): vmrun/snapshot/keyring/permissions/systemd checks.
- validate guest(C6): transport probe with explicit failure cause.
- creds set (C6): keyring writer matching credentials.py read scheme;
password via stdin/prompt only. Replaces
Set-CIGuestCredential.ps1.
All groups import backends.load_backend only (Phase D ESXi path stays open).
Suite green, coverage 95.10% (gate 90%); new modules at 100%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase B is closed (checklist Passi 1–9 [x], benchmark matrix §6–§14
recorded). Move the two finished working docs into the dated archive:
plans/PhaseB-user-checklist.md → plans/archived/2026-06-07/
plans/benchmark-windows-host.md → plans/archived/2026-06-07/
plans/ now holds only the forward-looking plans (idea-3 Phase C pwsh
removal, idea-4 Phase D ESXi, ideas-overview). Fixed the live links that
pointed at the moved/relocated files: idea-3 §4, idea-2 §8 (relative path),
and ideas-overview (idea-1/idea-2 now under archived/, Phase A+B marked ✅).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RUNBOOK §15: dual-boot operation — Linux⇄Windows switch procedure, per-OS
CI-stack table (roots/runner/vmrun), current GRUB behaviour (DEFAULT=0,
TIMEOUT=0 hidden → boots straight to Linux), post-boot verification commands
for both OSes, and an optional GRUB tweak (left unapplied — boot-to-Linux is
intentional so the box returns as the Linux CI host after any unattended
reboot).
Checklist: Passo 9 done; fixed the stale global tracking table (Passi 6–9
were still [ ] despite being complete). Phase B is now fully closed (1–9 [x]).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
idea-3- prefix was shared with idea-3-powershell-removal.md; rename the ESXi
idea to idea-4-esxi-support.md and update the ideas-overview link. Pure rename,
content unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
§11 Win-guest static (boot 29.75s, +24% vs Linux-host §8; static-IP floor
~22s confirmed host-independent), §12 Lin-guest DHCP (13.51s, +53%), §13
burn-in 4×10 (Win 36/40 — transient WinRM faults under 4× concurrency,
self-recovered; Lin 40/40). §14 adds the full host×guest×IP-mode matrix.
Plan checkboxes filled with results and the template-parity pre-req note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All sections done: SSH keys, BuildVMGuest password, Gitea PAT,
OpenSSH Server, template powered-off verification, CI backup.
Fixed errata: .vmsn always present (not error indicator), backup
switched from Git tar to 7z LZMA2, removed scheduled-task section
(no CI-* tasks exist on this host).
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
§10: single-job Measure-CIBenchmark on LinuxBuild2404 (DHCP, 10 iter) on the
Linux host — boot total 8.82s avg, near-deterministic even on DHCP (IP-acquire
7.45s, σ≈0.01). Guest-vs-guest table: Linux guest ~63% faster boot total than
the Windows static-IP guest (§8) on the same host.
plans/benchmark-windows-host.md: dual-boot plan to collect the symmetric
Windows-host numbers (§11 Win static, §12 Linux DHCP, §13 concurrent burn-in)
so the host×guest×IP-mode matrix is complete and Linux-vs-Windows is a
column-for-column comparison. Includes Windows paths, Credential Manager
prereqs, and the BaseClean-Linux snapshot caveat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Concurrent capacity burn-in run on the Linux host as ci-runner with the
static-IP pool: WinBuild2025 40/40 PASS (~78.6s/round) and LinuxBuild2404
40/40 PASS (~70.2s/round) — 80/80 jobs, 20/20 rounds, near-deterministic
(static IP eliminates the DHCP IP-acquire variance of the §6/§7 baselines).
- RUNBOOK §9: concurrent burn-in result + ip-pool reset/recovery note.
- PhaseB-user-checklist Passo 7/8 marked done; fixed two checklist bugs:
run as ci-runner (the simone path needs g+w on the /var/lib/ci root for
ip-pool.tmp, not just the sub-dirs), and the Linux command was missing
-SnapshotName BaseClean-Linux (default BaseClean is Windows-only).
- idea-2-linux-host §8 DoD: Phase B complete (B1–B7 + >2 weeks stability).
- TODO §2.8 [P2]: ip-pool is not reconciled against live clones; a killed
job leaks its lease permanently and exhausts the 4-IP pool. Proposed fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Postmortem of the Linux template deploy/prepare 'no IP' investigation:
separates the three distinct causes (apt index stale from --skip-update,
deploy --force snapshot-chain, wedged host vmnet-dhcpd) and documents the
recovery (restart vmware-networks). Changelog entry summarising the
toolchain, xvfb, --force cleanup and cloud-init seal changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an --xvfb switch to 'build run' and 'job' that wraps the Linux build
command in 'xvfb-run -a sh -c ...', giving GUI toolkits (GTK4/PyGObject)
a headless $DISPLAY. Exported env vars precede xvfb-run so they
propagate into the virtual-display session. No-op (with a notice) for
Windows guests.
The local-ci-build composite action exposes it as the 'xvfb' input
(default false), forwarded as INPUT_XVFB -> --xvfb, mirroring the
use-shared-cache/skip-artifact wiring.
Docs: WORKFLOW-AUTHORING.md parameter table + Linux example note;
LINUX-TEMPLATE-SETUP.md Step 4a (the template ships python3-gi
gir1.2-gtk-4.0 xvfb). Adds build-run unit tests (wrap, plain, Windows
no-op).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
deploy-linux --force reused the VM directory but left any prior snapshot
chain in place, so vmware-vdiskmanager refused the Step 4 resize with
'disk is part of a snapshot chain'. Add _clean_force_artifacts to remove
snapshot deltas (.vmdk), .vmsn, .vmsd and stale .lck before resize.
prepare-linux Step 7d seals the image: drop the stale cloud-init netplan
(leaving only the MAC-agnostic 99-ci-dhcp-allnics), regenerate networkd
config, and disable cloud-init so it never re-runs on linked clones.
deploy user-data sets package_update/upgrade: false so cloud-init does
not hold the dpkg lock at first boot and delay open-vm-tools past the
IP-detection poll; the step-7 IP timeout is raised 300s -> 600s as a
safety net for slow first boots.
Adds unit tests for _clean_force_artifacts (snapshot chain, lock dir,
clean-dir no-op) and updates prepare-linux stubs for the seal step.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Step 4a installs python3-gi gir1.2-gtk-4.0 xvfb for headless PyGObject
GUI tests, with import-gi and Xvfb assertions.
Step 1 now always runs apt-get update; --skip-update only skips the
(slow) full upgrade. Previously --skip-update skipped the index refresh
too, so a fresh package (gir1.2-gtk-4.0/xvfb) could not be located.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Linux clones could boot with the RTC interpreted as local time (guest
runs hours ahead of UTC). NTP then steps the clock backward at runtime,
landing after source mtimes were stamped, so make reported "Clock skew
detected" and future-dated files.
- build run: enable NTP and block until NTPSynchronized before stamping
the sources, so the backward correction happens pre-touch. Drop
`hwclock -s`, which read the local-time RTC and pushed the clock further
ahead.
- template prepare-linux: set-local-rtc 0 so clones interpret the RTC as
UTC and boot at the correct time; NTP then only nudges, never steps 2h.
Linux/SSH path only; WinRM branch untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Introduced a new test to verify behavior when SSH transport connects but is not ready, ensuring it returns False after the deadline.
- Added comprehensive tests for the `deploy-linux` template, covering various scenarios including missing public keys, VM directory existence, SHA256 mismatches, and failures during ISO creation and VM conversion.
- Implemented tests for SSH-related functions, including connection retries and streaming command execution, ensuring proper handling of exit statuses and progress indications.
- Enhanced existing tests to cover edge cases and ensure robustness in the deployment process.
- Implement tests for disk alert JSON output to include a message field when below threshold.
- Add tests for runner status when Gitea is online but no webhook URL is provided.
- Introduce tests for job report details to ensure no error line appears when there are no failure events.
- Add tests to skip empty JSONL files in job report listing.
- Install SIGTERM handler that raises KeyboardInterrupt so act_runner
workflow cancellations trigger the existing VM cleanup path
- Resolve guest_os before the ip_pool block and skip guestinfo IP
injection for Linux (template doesn't read VMX guestinfo to configure
network, causing SSH probe to fail against the wrong IP)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Static IP via ip_pool: 24 s boot-to-ready vs 102 s DHCP (4x faster),
IP acquire now deterministic (σ < 0.03 s vs σ ≈ 57 s with DHCP).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add -StaticIP/-Netmask/-Gateway params; injects guestinfo into cloned
VMX before start, mirroring job.py _inject_guestinfo_ip behaviour
- Add -GuestInfoOnly switch to Get-GuestIPAddress: skips vmrun
getGuestIPAddress fallback to prevent race where transient DHCP address
is returned before ci-static-ip.ps1 finishes reconfiguring the NIC
- Benchmark passes -GuestInfoOnly automatically when -StaticIP is set
- staticIP field added to benchmark.jsonl for DHCP vs static comparison
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- WinRM MaxProcessesPerShell: apply 100 if below threshold before asserting
- DesktopHeap SharedSection: apply 4096 KB registry value if below threshold before asserting
- Replace cleanmgr (hangs in WinRM Session 0, no window station) with direct
PowerShell cleanup for dumps, WER files and Recycle Bin; DISM already covers
component store and WU cache
- vcpkg git clone/checkout: wrap with EAP=Continue to suppress PS5.1
NativeCommandError on git stderr (gotcha #3 from AGENTS.md)
- template.py: filter CI_EXITCODE:N marker from displayed output
- ci-static-ip.ps1: switch from vmware-rpctool (removed in recent Tools) to
vmtoolsd --cmd; use temp-file redirect to capture output from native cmd
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cleanup now runs after the system reboots post-install (fully settled
state), not right after the WU loop. Sequence: WU loop → static IP →
validation → final reboot → WinRM ready → [cleanup] → autologin →
shutdown → snapshot.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move disk cleanup out of the Windows Update loop:
- PS scripts: extract Invoke-DiskCleanup function, add -CleanupOnly switch
for standalone post-update cleanup pass.
- Python command: always pass -SkipCleanup:$true during WU loop; after
loop exits 0, run a separate -CleanupOnly invocation (skipped if
--skip-cleanup). Fails fast if cleanup-only returns nonzero.
- Tests: 3 new tests for cleanup-runs-after-loop, skip-cleanup suppresses
it, and cleanup-failure propagates as ClickException.
- ruff --fix: remove unused shutil import, sort imports, fix UP037/UP035.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Simone
Simone