diff --git a/docs/RUNBOOK.md b/docs/RUNBOOK.md index fc890eb..35c4a30 100644 --- a/docs/RUNBOOK.md +++ b/docs/RUNBOOK.md @@ -503,3 +503,39 @@ sudo -u ci-runner /opt/ci/venv/bin/python -c \ See `TODO.md` (IP-pool auto-reconciliation) for the proposed fix. +--- + +## 10. Linux guest single-job baseline — LinuxBuild2404 (B7 follow-up) + +Recorded 2026-06-07 — `Measure-CIBenchmark.ps1 -GuestOS Linux -Iterations 10`, +Linux Mint host + VMware Workstation Pro Linux, run as `ci-runner`, template +`LinuxBuild2404` / snapshot `BaseClean-Linux`, **DHCP** (no static IP — the +guest reads its DHCP lease and publishes it via `guestinfo.ci-ip`; the +Windows-only `ci-static-ip.ps1` task does not apply). Ready = SSH/22 probe. + +| Iter | Clone (s) | Start (s) | IP acquire (s) | Ready (s) | Destroy (s) | Boot total (s) | +| ------- | --------- | --------- | -------------- | --------- | ----------- | -------------- | +| 1–10 | 0.22 | 1.14–1.16 | 7.44–7.48 | 0.00–0.01 | 4.90–4.99 | 8.80–8.86 | +| **avg** | **0.22** | **1.14** | **7.45** | **0.00** | **4.94** | **8.82** | + +**Key finding**: the Linux guest is near-deterministic *even on DHCP* +(σ ≈ 0.01 s on IP-acquire) — no static-IP injection needed. IP-acquire is +7.45 s vs Windows-guest 58 s (DHCP, §6) / 21.8 s (static, §8). Boot total +8.82 s — ~2.7× faster than the Windows static-IP guest. + +**Guest-vs-guest comparison (Linux host, single-job):** + +| Metric | Win guest static (§8) | Lin guest DHCP (§10) | Lin advantage | +| ---------- | --------------------- | -------------------- | ------------- | +| Clone | 0.40 s | 0.22 s | −45% | +| Start | 1.88 s | 1.14 s | −39% | +| IP acquire | 21.77 s | 7.45 s | **−66%** | +| Ready | 0.00 s | 0.00 s | ≈ same | +| Destroy | 10.40 s | 4.94 s | −53% | +| Boot total | 24.05 s | 8.82 s | **−63%** | + +Caveat: different IP modes (Win static-IP task vs Lin DHCP+guestinfo) — the +comparison reflects the *as-deployed* path for each guest, not an +IP-mode-controlled A/B. The Win guest's static-IP floor (~21.8 s) is dominated +by Windows boot + Task Scheduler + NIC reconfig; the Linux guest needs no such +in-guest agent. diff --git a/plans/benchmark-windows-host.md b/plans/benchmark-windows-host.md new file mode 100644 index 0000000..e2a91c9 --- /dev/null +++ b/plans/benchmark-windows-host.md @@ -0,0 +1,144 @@ +# Plan — Windows-host benchmarks (comparable to Linux-host §6–§10) + +> **Scopo**: la macchina è dual-boot (Windows ⇄ Linux, stesso hardware). +> Le baseline RUNBOOK §6–§10 sono quasi tutte sul **Linux host**. Questo +> plan raccoglie i numeri mancanti sul **Windows host** per chiudere la +> matrice host × guest × IP-mode e rendere il confronto Linux-vs-Windows +> completo e simmetrico. +> +> **Prerequisito**: riavviare la macchina nel boot entry **Windows**. +> Tutto gira su Windows PowerShell 5.1 o pwsh 7; usare la stessa +> `Measure-CIBenchmark.ps1` / `Test-CapacityBurnIn.ps1` del repo (sono +> cross-platform, i default path passano a `F:\CI\...` quando `$IsWindows`). + +--- + +## 1. Cosa esiste già e cosa manca + +| RUNBOOK § | Host | Guest | IP mode | Stato | +| --------- | ----- | -------------- | ------- | ---------- | +| §6 | Win | WinBuild2025 | DHCP | ✓ 2026-05-17 | +| §7 | Linux | WinBuild2025 | DHCP | ✓ | +| §8 | Linux | WinBuild2025 | static | ✓ | +| §9 | Linux | Win + Linux | burn-in (static pool) | ✓ | +| §10 | Linux | LinuxBuild2404 | DHCP | ✓ | + +**Mancanti sul Windows host** (target di questo plan): + +| Nuovo § | Host | Guest | IP mode | Pair-with | +| ------- | ---- | -------------- | ------- | --------- | +| §11 | Win | WinBuild2025 | static | §8 | +| §12 | Win | LinuxBuild2404 | DHCP | §10 | +| §13 | Win | Win + Linux | burn-in 4×10 | §9 | + +§6 copre già Win×Win×DHCP. Opzionale: ri-eseguirlo nella stessa sessione per +coerenza (stesso disco/fragmentazione del momento). + +--- + +## 2. Prerequisiti sul Windows host + +- [ ] Boot in Windows; verificare VMware Workstation + `vmrun.exe` in + `C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe`. +- [ ] Template fully powered-off con snapshot integri: + ```powershell + & 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' -T ws listSnapshots 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' + & 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' -T ws listSnapshots 'F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx' + ``` + Attesi: `BaseClean` e `BaseClean-Linux`. +- [ ] Credenziali in **Windows Credential Manager** (non keyring file): + target `BuildVMGuest` (guest WinRM) e `GiteaPAT` (clone in-VM). + Verifica: `cmdkey /list | findstr /i "BuildVMGuest GiteaPAT"`. +- [ ] Chiave SSH guest Linux presente: `F:\CI\keys\ci_linux` (per §12/§13 + lato LinuxBuild2404). +- [ ] Gitea raggiungibile (`http://10.10.20.11:3100`) e repo + `Simone/burnin-dummy` con `build.ps1` + `build.sh` (per §13). +- [ ] `F:\CI\BuildVMs\` privo di cloni orfani e `F:\CI\ip-pool.json` + (se esiste) pulito — stesso caveat lease stale del Linux host + (RUNBOOK §9 / TODO §2.8). Reset se serve: + ```powershell + Set-Content -Path 'F:\CI\ip-pool.json' -Value '{}' -NoNewline + ``` + +--- + +## 3. §11 — WinBuild2025 static IP (pair §8) + +```powershell +.\scripts\Measure-CIBenchmark.ps1 ` + -TemplatePath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' ` + -SnapshotName BaseClean ` + -StaticIP 192.168.79.200 -Netmask 255.255.255.0 ` + -Iterations 10 +``` +- [ ] 10 iter completate, tabella per-fase stampata. +- [ ] Confronto vs §8 (Linux host static): atteso clone/start simili, + IP-acquire dominato da `ci-static-ip.ps1` (~21.8 s floor, indipendente + dall'host). Delta host atteso piccolo sulle fasi non-IP. + +## 4. §12 — LinuxBuild2404 DHCP (pair §10) + +```powershell +.\scripts\Measure-CIBenchmark.ps1 ` + -TemplatePath 'F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx' ` + -SnapshotName BaseClean-Linux ` + -GuestOS Linux ` + -Iterations 10 +``` +- [ ] 10 iter completate. +- [ ] Confronto vs §10 (Linux host, Lin guest DHCP, boot total 8.82 s): + isola l'**effetto host** sul guest Linux. Se Win host ≫ 8.82 s, + l'overhead è host-side (NTFS / NAT vmnet8 Windows), non guest. + +## 5. §13 — Concurrent burn-in 4×10 (pair §9) + +```powershell +# WinBuild2025 +.\scripts\Test-CapacityBurnIn.ps1 ` + -Parallelism 4 -Rounds 10 ` + -TemplatePath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' ` + -RepoUrl 'http://10.10.20.11:3100/Simone/burnin-dummy.git' ` + -CloneBaseDir 'F:\CI\BuildVMs\' ` + -ArtifactBaseDir 'F:\CI\Artifacts\' ` + -VmrunPath 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' ` + -BuildCommand 'powershell -ExecutionPolicy Bypass -File .\build.ps1' + +# LinuxBuild2404 (snapshot OBBLIGATORIO, default è BaseClean=Windows) +.\scripts\Test-CapacityBurnIn.ps1 ` + -Parallelism 4 -Rounds 10 ` + -TemplatePath 'F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx' ` + -SnapshotName BaseClean-Linux ` + -RepoUrl 'http://10.10.20.11:3100/Simone/burnin-dummy.git' ` + -CloneBaseDir 'F:\CI\BuildVMs\' ` + -ArtifactBaseDir 'F:\CI\Artifacts\' ` + -VmrunPath 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' ` + -GuestOS Linux ` + -BuildCommand 'bash build.sh' +``` +- [ ] Entrambi 40/40 PASS, 10/10 round. +- [ ] Round avg vs §9 (Linux host: Win ~78.6 s, Lin ~70.2 s): delta = effetto + host sotto concorrenza 4×. + +--- + +## 6. Output e documentazione + +- Tutti i run appendono a `F:\CI\Logs\benchmark.jsonl` (single-job) e + stampano summary (burn-in). +- [ ] Copiare/annotare i risultati e aggiungere a `docs/RUNBOOK.md` le + sezioni **§11–§13** con le stesse tabelle di §8/§9/§10 e una riga + finale di **confronto host Linux-vs-Windows** per ciascuna coppia. +- [ ] Pulizia post-run: `F:\CI\BuildVMs\` vuoto, artifact `burnin-*` + rimossi, `F:\CI\ip-pool.json` reset. + +--- + +## 7. Matrice finale attesa (dopo questo plan) + +| Host \ (Guest, mode) | Win/DHCP | Win/static | Lin/DHCP | burn-in 4×10 | +| -------------------- | -------- | ---------- | -------- | ------------ | +| **Linux** | §7 | §8 | §10 | §9 | +| **Windows** | §6 | §11 | §12 | §13 | + +Confronto colonna-per-colonna = **effetto host** a guest+mode costanti. +Confronto riga-per-riga = **effetto guest/IP-mode** a host costante.