`--link` was still the brighter GitHub blue while everything else moved to
Gitea's #4183c4, so links read as a second, slightly different accent. One blue
now.
Contrast on this palette: 4.50:1 on --panel (#161718), 4.14:1 on --bg
(#1e1f20), 3.77:1 on --panel-2 (#22272e) — down from 4.79 / 4.41 / 4.01. Body
links sit on --panel and stay above the 4.5:1 AA threshold; the ones on the
darker surfaces (nav hover, table cells) land between AA-large and AA, as the
rest of this palette's dim text already does.
The dashboard could only show what was powered on this instant, so it could
not answer "did the last build pass?" — and no local execution history exists
to answer it from (`report job` reads a PowerShell-era JSONL path that nothing
writes on this host, and the recent artifact dirs are empty).
Recent runs now come from Gitea's Actions API (`operations/executions.py`):
`/api/v1/admin/actions/runs` for every repo on the instance, falling back to
`/api/v1/user/actions/runs` when the token is not admin-scoped. One list call
per refresh, cached 10s, so the card's 5s htmx poll costs Gitea at most one
call per window. Live VMs are grouped under the run that spawned them — the
job id is `{run_id}-{run_attempt}[-suffix]` and the clone name carries it — so
a matrix run collects several VMs on one row. Every failure mode (no token,
401, unreachable Gitea, malformed payload) degrades to a notice plus the plain
VM rows, each keeping its Remove button, so the card is never less useful than
the table it replaces. Orphans stay where they were: the sweep in Maintenance,
the count in the status-header tile.
Auth reuses the access token from the operator's own OIDC login, which
`auth.py` previously discarded. SECURITY: that token is kept server-side only
(`app.state.gitea_tokens`, dropped on logout) and never written into the
session — `SessionMiddleware` signs the session cookie but does not encrypt
it, and this service is deployed over plain HTTP on the LAN, so a bearer token
in there would be readable in the browser and on the wire. An expired token is
refreshed once; `[web] gitea_token_cred` names an optional keyring PAT for the
cases a login token cannot cover (service restart, non-admin operator).
`GET /vms` and `partials/vms.html` are gone; `POST /vms/{id}/remove` and
`POST /vms/cleanup` are untouched.
Design: docs/superpowers/specs/2026-07-25-executions-card-design.md
Adds operations/services.py: a fixed ManagedService allow-list (act-runner, act-runner-rust, vmware-host, ci-web) with find_service/service_states/control_service. control_service builds an argument list only (never shell=True), validates the action against ALLOWED_ACTIONS (ValueError->400), refuses stop on restart-only services (ci-web, allow_stop=False) without building a command, and defers ci-web's own restart via 'sudo -n systemd-run --on-active=2 systemctl restart ci-web.service' so the HTTP response returns first.
Route POST /services/{service_id}/{action} in web/routes.py mirrors vms_remove: require_user + require_csrf + two-step confirm gate + one audit entry per stage; unknown id / bad action / disallowed-stop short-circuit to 400 before any subprocess or audit; the vmware-host confirm step warns with the active-VM count. New /services read partial with state-aware Start/Stop/Restart buttons, polled every 15s.
deploy/sudoers.d/ci-web grants ci-runner exact-command NOPASSWD for exactly these nine systemctl lines plus the systemd-run deferral — no wildcards, no bare systemctl, no ci-web stop; setup-web-linux.sh installs it mode 0440 and validates with 'visudo -cf', rolling back on failure. Config gains [web].managed_services override; docs/WEB-UI.md documents the panel + security model.
Gates: 1044 tests pass (46 new security/unit tests), coverage 94.20%, ruff + mypy --strict clean. Not yet deployed: sudoers install + prod-venv reinstall + ci-web restart are a human privilege-boundary step.
Stage A: swap CSS tokens to the real Gitea dark palette (accent blue #4183c4); add Jinja formatting filters (humanbytes/duration/relative_time/localtime/shorten_path/thousands), a health status-header, and an inline SVG icon sprite; fix two UI bugs — unthemed action pages (htmx swap into #action-slot with a full-page HX-Request fallback) and polling wiping form input (hx-preserve).
Stage B: four read-only panels that degrade to notices (never 500) — host resources (/proc, two-sample CPU%), VMware module health, template inventory (.vmem warn + last backup), and a path-contained file browser; backup card becomes a validated template dropdown; expensive dir-size walk is TTL-cached. Adds operations/inventory.py plus host_resources()/vmware_module_health() in operations/host.py.
Stage C: parse_clone_name() adds a Job column (job_id + started) to the VM table; nav gains a Gitea link-out derived from the OIDC issuer origin (no fabricated run URLs); audit page gets action icons, relative time, and a client-side filter.
Gates: 989 tests pass, coverage 94.12%, ruff + mypy --strict clean. Stage D (service management) intentionally deferred.
CLAUDE.md and docs/ARCHITECTURE.md were still missing the operations/ and web/ packages added by the web-UI plan (W0-W7). Also records that ci-web.service is now live on this host (LAN + Gitea OIDC) in docs/WEB-UI.md and closes plans/web-ui-management-plan.md with a status header.
remove_vm() deletes vmx.parent, so containment must check the clone dir, not just the vmx: a crafted vmx_id of build_vms/x.vmx passed the old check and would rmtree the entire clone base. Now the parent must be strictly inside build_vms.
In open mode ([web.oidc] unset) the dashboard has no login. Binding a LAN/any address there would expose host-management unauthenticated, so ci web serve now aborts unless the bind host is loopback. Auth-configured deployments bind LAN as before.
backup_template used a hardcoded _BACKUP_BASE_DIR=/var/lib/ci/backups; now derives <config.paths.root>/backups (matches the CLI --backup-base-dir default) per the plan's no-hardcoded-path gotcha. Tests no longer need to monkeypatch the constant.
A job killed mid-flight never released its IP lease; with only 4 pool addresses a single leak exhausted the pool ("IP pool exhausted after 60s"). IpSlotPool.reconcile() releases leases whose owner clone-dir is gone; vm cleanup runs it only against the real build_vms base. Closes TODO §2.8.
Server-rendered management UI (FastAPI + HTMX + Jinja2), Gitea OIDC auth,
bare-systemd deploy, in-process operations/ seam reused by both CLI and web
(no subprocess, load_backend kept for the ESXi path). Includes the
implementer recipe (TDD, mypy --strict, coverage >=90, no CDN, secrets in
keyring only).
Added two implementer notes tying the refactor to open TODO items:
- W1: fold the TODO §2.8 IP-pool leak fix into operations/vms.cleanup_orphans
while moving it (same function the web "cleanup orphans" action calls).
- W2: TODO §2.9 (retention blind to backups/templates/ISO) is orthogonal —
operations/cache.py wraps only cargo-cache purge; do not widen W2 scope.
report.py's residual F:\CI\Logs default is already flagged under Rischi.
The shared composite action ran 3 steps under `shell: pwsh`, making every
CI job depend on PowerShell on the Linux host. Rewrote them in bash and
dropped the dead $IsLinux / F:\CI branches (Phase B left a single Linux
host serving both windows-build and linux-build runners).
Closes Phase C / idea-3: no CI job or systemd timer requires pwsh on the
Linux host anymore. Ticked all C1-C8 subtask checkboxes and set the plans
to "C chiusa". pwsh is deliberately left installed (operator choice, not a
technical blocker).
Measured through the real pipeline, not the unit tests: a Rust job goes from
186s to 84s (build phase 150s -> 47s), and 75s with the source modified, which
is the scenario CI actually runs. The entry is 645 MB and costs 8s to restore.
The test that decides whether this ships is row 4: a deliberate type error in
fury-core, built warm. The cache restored 645 MB of dependencies and the build
still went red with E0308, exit 1, publishing nothing. A cache that can turn a
red build green is the worst failure a CI system has because it is silent --
re-run that row whenever the prune rules change.
AGENTS.md #17 records something found while validating: `vmrun list` is
per-user. Run as the wrong user it answers "Total running VMs: 0" while a clone
is very much alive, which makes the "no VMs running" check before a
deploy --force (#14) unreliable. `ps | grep vmware-vmx` is user-independent and
was already the recommended diagnosis.
Measured first, as thread entry #12 requires: a cold furyrs pipeline is 134.7s,
target/ is 3.0 GiB raw and 2.5 GiB pruned, and the stored entry is 645 MiB. A
warm job pays 8.1s (upload + extract) to skip ~120s of dependency compilation.
Publishing costs 15.7s but only on a miss. Build it.
The assertion in the plan turned out to be worthless and is replaced. It
re-scanned for */.fingerprint/<member>-* after deleting exactly those paths, so
asked to catch a planted fury-core fingerprint it exited 0 -- the prune had
removed the evidence before the check looked. An assertion sharing its globs
with the deletion can only prove that rm ran; it cannot see the failure that
matters, which is a glob that never matched the artifact in the first place.
The replacement asks a different oracle. Cargo records in each .d depfile the
sources an artifact was built from: dependencies point into the registry,
anything built from workspace source points back into the workspace. After a
correct prune, zero surviving depfiles reference workspace source outside
target/. Verified negatively too -- an artifact named totallyunrelated-abc123.d
that no prune glob matches, declaring a dependency on fury-core/src/lib.rs,
makes the script exit 2 and refuse to publish.
zstd is already present on the template, so no rebuild is needed; the gzip
fallback stays for the plain LinuxBuild2404.
Documented with `sudo -u ci-runner`, copied from the neighbouring tools that
call the Gitea API. Wrong here: the guard fetches, fetching writes
.git/FETCH_HEAD, and that file belongs to the committing user -- so as
ci-runner it died with "impossibile aprire '.git/FETCH_HEAD': Permesso negato"
before comparing anything.
A guard that cannot run is worse than no guard, since the failure looks like
infrastructure noise rather than an answer. The question it asks is "does what
I have match what is published", so it belongs to whoever holds the working
tree.
Thread entry furyrs#12 measured where a 133s Rust job actually goes: ~120s is
compiling the ~490 external dependencies, which a committed Cargo.lock keeps
bit-identical between runs. The workspace crates cost 7s. #13 settled the
policy -- reuse dependency artifacts, never workspace ones, and keep v* tags
cold.
The plan departs from #12's cargo-chef shape for a concrete reason: the
orchestrator receives build-command as an opaque string (furyrs runs six cargo
invocations across three feature sets) and `cargo chef cook` must be handed the
same flags as the real build. Rather than guess them, invert the order -- build
normally, then prune the workspace members' artifacts out of target/ and
publish the remainder. No knowledge of the command, one entry holding every
feature graph, no new tool in the template.
That makes the safety property checkable rather than argued: after the prune,
an assertion looks for any surviving workspace fingerprint and the job refuses
to publish if it finds one. A missing entry costs 120s; a poisoned one costs a
wrong verdict, which is the worst failure a build system has because it is
silent.
Store is one immutable file per key with an atomic rename, so there is no
shared mutable directory and no advisory locking -- the objection furyrs raised
about VMware Shared Folders under concurrency. Retention of the store is in the
same plan: adding an unbounded directory without it would repeat TODO 2.9 at a
larger scale.
On a Linux host the UUP converter only produces the RTM install.wim (build
26100.1), so Windows Update had to install the entire cumulative anyway —
the UUP download/convert machinery added a slow ~5 GB step and several host
tools (aria2/wimtools/cabextract/chntpw) for no benefit.
Replace it: deploy-win now takes the install ISO directly via `--iso-url`
(urllib download + cache by file name under <CI_ROOT>/iso/, optional
--iso-sha256) or a local `--win-iso` (exactly one required). Patch + deploy
flow is unchanged. Supply an already-updated ISO (e.g. a massgrave link) to
skip the WU step entirely.
- Remove winimage/uup.py and its tests.
- Trim setup-host-linux.sh deps to p7zip-full + xorriso + genisoimage.
- Update RUNBOOK §5 and the Windows-host port plan. Coverage 93.6%.
- CLAUDE.md: Linux host no longer runs pwsh; PS 5.1 constraints scoped to
Windows guests + Windows runner.
- RUNBOOK: add "Phase C — Python equivalents" table; §6-§13 benchmark records
kept as historical provenance.
- TODO §3.7: close $IsWindows shim bug (resolved-by-replacement — bench run
bypasses the shim).
- plans: mark C1-C8 done + docs C9; flag operational cutover/uninstall pending.
Phase B is closed (checklist Passi 1–9 [x], benchmark matrix §6–§14
recorded). Move the two finished working docs into the dated archive:
plans/PhaseB-user-checklist.md → plans/archived/2026-06-07/
plans/benchmark-windows-host.md → plans/archived/2026-06-07/
plans/ now holds only the forward-looking plans (idea-3 Phase C pwsh
removal, idea-4 Phase D ESXi, ideas-overview). Fixed the live links that
pointed at the moved/relocated files: idea-3 §4, idea-2 §8 (relative path),
and ideas-overview (idea-1/idea-2 now under archived/, Phase A+B marked ✅).
RUNBOOK §15: dual-boot operation — Linux⇄Windows switch procedure, per-OS
CI-stack table (roots/runner/vmrun), current GRUB behaviour (DEFAULT=0,
TIMEOUT=0 hidden → boots straight to Linux), post-boot verification commands
for both OSes, and an optional GRUB tweak (left unapplied — boot-to-Linux is
intentional so the box returns as the Linux CI host after any unattended
reboot).
Checklist: Passo 9 done; fixed the stale global tracking table (Passi 6–9
were still [ ] despite being complete). Phase B is now fully closed (1–9 [x]).
idea-3- prefix was shared with idea-3-powershell-removal.md; rename the ESXi
idea to idea-4-esxi-support.md and update the ideas-overview link. Pure rename,
content unchanged.
All sections done: SSH keys, BuildVMGuest password, Gitea PAT,
OpenSSH Server, template powered-off verification, CI backup.
Fixed errata: .vmsn always present (not error indicator), backup
switched from Git tar to 7z LZMA2, removed scheduled-task section
(no CI-* tasks exist on this host).
§10: single-job Measure-CIBenchmark on LinuxBuild2404 (DHCP, 10 iter) on the
Linux host — boot total 8.82s avg, near-deterministic even on DHCP (IP-acquire
7.45s, σ≈0.01). Guest-vs-guest table: Linux guest ~63% faster boot total than
the Windows static-IP guest (§8) on the same host.
plans/benchmark-windows-host.md: dual-boot plan to collect the symmetric
Windows-host numbers (§11 Win static, §12 Linux DHCP, §13 concurrent burn-in)
so the host×guest×IP-mode matrix is complete and Linux-vs-Windows is a
column-for-column comparison. Includes Windows paths, Credential Manager
prereqs, and the BaseClean-Linux snapshot caveat.
Concurrent capacity burn-in run on the Linux host as ci-runner with the
static-IP pool: WinBuild2025 40/40 PASS (~78.6s/round) and LinuxBuild2404
40/40 PASS (~70.2s/round) — 80/80 jobs, 20/20 rounds, near-deterministic
(static IP eliminates the DHCP IP-acquire variance of the §6/§7 baselines).
- RUNBOOK §9: concurrent burn-in result + ip-pool reset/recovery note.
- PhaseB-user-checklist Passo 7/8 marked done; fixed two checklist bugs:
run as ci-runner (the simone path needs g+w on the /var/lib/ci root for
ip-pool.tmp, not just the sub-dirs), and the Linux command was missing
-SnapshotName BaseClean-Linux (default BaseClean is Windows-only).
- idea-2-linux-host §8 DoD: Phase B complete (B1–B7 + >2 weeks stability).
- TODO §2.8 [P2]: ip-pool is not reconciled against live clones; a killed
job leaks its lease permanently and exhausts the 4-IP pool. Proposed fix.
nsis-plugin-ns7zip is a real C++ project — wrong for lifecycle testing.
burnin-dummy simulates ~30s work and writes to dist/ with no external
deps. Documents one-time prerequisites (group, keyring, Gitea push).
Without an explicit build command, job.py runs dotnet build (which fails
on nsis-plugin-ns7zip in a fresh VM) and expects bin\CIOutput as artifact
source. With a trivial BuildCommand, artifact_source defaults to "dist"
and the lightweight command always succeeds.
Also documents one-time simone group/keyring prerequisites for running
the burn-in as simone with ci-runner group access.
Burn-in (4 jobs × 10 rounds), 7-day stability run, and RUNBOOK.md
Linux section are still open. Checklist stays active until all
acceptance criteria are met.
Moves all A/B closeouts, checklists, idea docs, and implementation plans
to plans/archived/2026-05-23/. Both phases are production-stable.
Active plans remaining in plans/:
- idea-3-powershell-removal.md (Phase C — in progress)
- idea-3-esxi-support.md (Phase D — future)
- ideas-overview.md (roadmap reference)
ns7zip matrix Win+Linux passes after the WinRM quotas + desktop-heap
fixes landed in the template scripts. Monitoring window completed
without regressions.
- Add plans/idea-3-powershell-removal.md: Phase C goal is to eliminate
the pwsh dependency from the Linux host by porting the remaining PS
scripts (bench run, validate host, smoke run) to Python sub-commands.
- Update idea-3-esxi-support.md header: Fase C → Fase D.
- Update ideas-overview.md: new four-phase table (A/B/C/D) with
prerequisiti and criteri for Phase C.
- Rename all "Hook futuri Fase C" (ESXi) → "Hook futuri Fase D" in
implementation-plan-A-B.md; add Phase C/D entries to summary and
references section.
- Update PhaseB-user-checklist.md end note and CLAUDE.md to point
Phase C → pwsh removal, Phase D → ESXi.
- Add `retention run` command (ports Invoke-RetentionPolicy.ps1): purges
old artifact/log dirs with aggressive mode when disk space is low.
- Add `template backup` command (ports Backup-CITemplate.ps1): 7z -mx=1
compressed archives in /var/lib/ci/backups/, prunes to keep-count=3,
stops/restarts act-runner.service around the copy.
- Update ci-retention-policy.service and ci-backup-template.service to use
Python; pwsh is no longer required on the Linux host.
- Fix ci-watch-runner-health.service: pass --service-name act-runner
(Linux service name, not Windows act_runner default).
- Fix _list_orphans in vm.py: wrap is_dir() inside the OSError try block
so a stat() failure on an entry is silently skipped rather than raised.
- Mark B5 complete in PhaseB-user-checklist.md.
User requirement (non-negotiable): the project must stay
first-class compatible with a Windows host permanently — Phase B
adds Linux *alongside* Windows, never replacing/degrading it.
- implementation-plan-A-B §2: categorical-constraint banner; B8
decommissioning cancelled (kept only as historical record);
master checklist B8 row updated.
- PhaseB-user-checklist: Passo 9 (B8) cancelled -> dual-host kept;
Passo 8 no longer a decommission gate; tracking table + closing
note updated.
Medium update: exec-summary status (A COMPLETATA, B ready), master
checklist A items [x], Phase A gating satisfied, §1 COMPLETATA banner
with real validation deviations (LocalSystem venv/keyring, github-free
artifacts, transport, shim common-params, deferred §8.1/§8.2), §7
A-side definition-of-done [x]. Fase B (§2/§3/§4) + risk matrix +
cronoprogramma left intact as the Phase B design reference alongside
PhaseB-user-checklist.md.
feature merged to main (753f47e), tag v2.0.0-phaseA pushed
(triggers build-ns7zip as final end-to-end validation of main).
All Passi 1-9 [x]. Residual non-blocking cleanup: delete the test
Gitea releases via UI (tags kept).
Start-BurnInTest.ps1 (lab wrapper: burnin-dummy + build.ps1) — 4
concurrent x 3 rounds all PASS, clean VM/lock teardown each round.
Note the gotcha: raw Test-CapacityBurnIn.ps1 without -BuildCommand
defaults to dotnet and fails on the non-.NET dummy repo.
Simone
Simone