docs: Linux-guest benchmark (RUNBOOK §10) + Windows-host benchmark plan
§10: single-job Measure-CIBenchmark on LinuxBuild2404 (DHCP, 10 iter) on the Linux host — boot total 8.82s avg, near-deterministic even on DHCP (IP-acquire 7.45s, σ≈0.01). Guest-vs-guest table: Linux guest ~63% faster boot total than the Windows static-IP guest (§8) on the same host. plans/benchmark-windows-host.md: dual-boot plan to collect the symmetric Windows-host numbers (§11 Win static, §12 Linux DHCP, §13 concurrent burn-in) so the host×guest×IP-mode matrix is complete and Linux-vs-Windows is a column-for-column comparison. Includes Windows paths, Credential Manager prereqs, and the BaseClean-Linux snapshot caveat. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -503,3 +503,39 @@ sudo -u ci-runner /opt/ci/venv/bin/python -c \
|
|||||||
|
|
||||||
See `TODO.md` (IP-pool auto-reconciliation) for the proposed fix.
|
See `TODO.md` (IP-pool auto-reconciliation) for the proposed fix.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Linux guest single-job baseline — LinuxBuild2404 (B7 follow-up)
|
||||||
|
|
||||||
|
Recorded 2026-06-07 — `Measure-CIBenchmark.ps1 -GuestOS Linux -Iterations 10`,
|
||||||
|
Linux Mint host + VMware Workstation Pro Linux, run as `ci-runner`, template
|
||||||
|
`LinuxBuild2404` / snapshot `BaseClean-Linux`, **DHCP** (no static IP — the
|
||||||
|
guest reads its DHCP lease and publishes it via `guestinfo.ci-ip`; the
|
||||||
|
Windows-only `ci-static-ip.ps1` task does not apply). Ready = SSH/22 probe.
|
||||||
|
|
||||||
|
| Iter | Clone (s) | Start (s) | IP acquire (s) | Ready (s) | Destroy (s) | Boot total (s) |
|
||||||
|
| ------- | --------- | --------- | -------------- | --------- | ----------- | -------------- |
|
||||||
|
| 1–10 | 0.22 | 1.14–1.16 | 7.44–7.48 | 0.00–0.01 | 4.90–4.99 | 8.80–8.86 |
|
||||||
|
| **avg** | **0.22** | **1.14** | **7.45** | **0.00** | **4.94** | **8.82** |
|
||||||
|
|
||||||
|
**Key finding**: the Linux guest is near-deterministic *even on DHCP*
|
||||||
|
(σ ≈ 0.01 s on IP-acquire) — no static-IP injection needed. IP-acquire is
|
||||||
|
7.45 s vs Windows-guest 58 s (DHCP, §6) / 21.8 s (static, §8). Boot total
|
||||||
|
8.82 s — ~2.7× faster than the Windows static-IP guest.
|
||||||
|
|
||||||
|
**Guest-vs-guest comparison (Linux host, single-job):**
|
||||||
|
|
||||||
|
| Metric | Win guest static (§8) | Lin guest DHCP (§10) | Lin advantage |
|
||||||
|
| ---------- | --------------------- | -------------------- | ------------- |
|
||||||
|
| Clone | 0.40 s | 0.22 s | −45% |
|
||||||
|
| Start | 1.88 s | 1.14 s | −39% |
|
||||||
|
| IP acquire | 21.77 s | 7.45 s | **−66%** |
|
||||||
|
| Ready | 0.00 s | 0.00 s | ≈ same |
|
||||||
|
| Destroy | 10.40 s | 4.94 s | −53% |
|
||||||
|
| Boot total | 24.05 s | 8.82 s | **−63%** |
|
||||||
|
|
||||||
|
Caveat: different IP modes (Win static-IP task vs Lin DHCP+guestinfo) — the
|
||||||
|
comparison reflects the *as-deployed* path for each guest, not an
|
||||||
|
IP-mode-controlled A/B. The Win guest's static-IP floor (~21.8 s) is dominated
|
||||||
|
by Windows boot + Task Scheduler + NIC reconfig; the Linux guest needs no such
|
||||||
|
in-guest agent.
|
||||||
|
|||||||
@@ -0,0 +1,144 @@
|
|||||||
|
# Plan — Windows-host benchmarks (comparable to Linux-host §6–§10)
|
||||||
|
|
||||||
|
> **Scopo**: la macchina è dual-boot (Windows ⇄ Linux, stesso hardware).
|
||||||
|
> Le baseline RUNBOOK §6–§10 sono quasi tutte sul **Linux host**. Questo
|
||||||
|
> plan raccoglie i numeri mancanti sul **Windows host** per chiudere la
|
||||||
|
> matrice host × guest × IP-mode e rendere il confronto Linux-vs-Windows
|
||||||
|
> completo e simmetrico.
|
||||||
|
>
|
||||||
|
> **Prerequisito**: riavviare la macchina nel boot entry **Windows**.
|
||||||
|
> Tutto gira su Windows PowerShell 5.1 o pwsh 7; usare la stessa
|
||||||
|
> `Measure-CIBenchmark.ps1` / `Test-CapacityBurnIn.ps1` del repo (sono
|
||||||
|
> cross-platform, i default path passano a `F:\CI\...` quando `$IsWindows`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Cosa esiste già e cosa manca
|
||||||
|
|
||||||
|
| RUNBOOK § | Host | Guest | IP mode | Stato |
|
||||||
|
| --------- | ----- | -------------- | ------- | ---------- |
|
||||||
|
| §6 | Win | WinBuild2025 | DHCP | ✓ 2026-05-17 |
|
||||||
|
| §7 | Linux | WinBuild2025 | DHCP | ✓ |
|
||||||
|
| §8 | Linux | WinBuild2025 | static | ✓ |
|
||||||
|
| §9 | Linux | Win + Linux | burn-in (static pool) | ✓ |
|
||||||
|
| §10 | Linux | LinuxBuild2404 | DHCP | ✓ |
|
||||||
|
|
||||||
|
**Mancanti sul Windows host** (target di questo plan):
|
||||||
|
|
||||||
|
| Nuovo § | Host | Guest | IP mode | Pair-with |
|
||||||
|
| ------- | ---- | -------------- | ------- | --------- |
|
||||||
|
| §11 | Win | WinBuild2025 | static | §8 |
|
||||||
|
| §12 | Win | LinuxBuild2404 | DHCP | §10 |
|
||||||
|
| §13 | Win | Win + Linux | burn-in 4×10 | §9 |
|
||||||
|
|
||||||
|
§6 copre già Win×Win×DHCP. Opzionale: ri-eseguirlo nella stessa sessione per
|
||||||
|
coerenza (stesso disco/fragmentazione del momento).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Prerequisiti sul Windows host
|
||||||
|
|
||||||
|
- [ ] Boot in Windows; verificare VMware Workstation + `vmrun.exe` in
|
||||||
|
`C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe`.
|
||||||
|
- [ ] Template fully powered-off con snapshot integri:
|
||||||
|
```powershell
|
||||||
|
& 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' -T ws listSnapshots 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx'
|
||||||
|
& 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' -T ws listSnapshots 'F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx'
|
||||||
|
```
|
||||||
|
Attesi: `BaseClean` e `BaseClean-Linux`.
|
||||||
|
- [ ] Credenziali in **Windows Credential Manager** (non keyring file):
|
||||||
|
target `BuildVMGuest` (guest WinRM) e `GiteaPAT` (clone in-VM).
|
||||||
|
Verifica: `cmdkey /list | findstr /i "BuildVMGuest GiteaPAT"`.
|
||||||
|
- [ ] Chiave SSH guest Linux presente: `F:\CI\keys\ci_linux` (per §12/§13
|
||||||
|
lato LinuxBuild2404).
|
||||||
|
- [ ] Gitea raggiungibile (`http://10.10.20.11:3100`) e repo
|
||||||
|
`Simone/burnin-dummy` con `build.ps1` + `build.sh` (per §13).
|
||||||
|
- [ ] `F:\CI\BuildVMs\` privo di cloni orfani e `F:\CI\ip-pool.json`
|
||||||
|
(se esiste) pulito — stesso caveat lease stale del Linux host
|
||||||
|
(RUNBOOK §9 / TODO §2.8). Reset se serve:
|
||||||
|
```powershell
|
||||||
|
Set-Content -Path 'F:\CI\ip-pool.json' -Value '{}' -NoNewline
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. §11 — WinBuild2025 static IP (pair §8)
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\scripts\Measure-CIBenchmark.ps1 `
|
||||||
|
-TemplatePath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
|
||||||
|
-SnapshotName BaseClean `
|
||||||
|
-StaticIP 192.168.79.200 -Netmask 255.255.255.0 `
|
||||||
|
-Iterations 10
|
||||||
|
```
|
||||||
|
- [ ] 10 iter completate, tabella per-fase stampata.
|
||||||
|
- [ ] Confronto vs §8 (Linux host static): atteso clone/start simili,
|
||||||
|
IP-acquire dominato da `ci-static-ip.ps1` (~21.8 s floor, indipendente
|
||||||
|
dall'host). Delta host atteso piccolo sulle fasi non-IP.
|
||||||
|
|
||||||
|
## 4. §12 — LinuxBuild2404 DHCP (pair §10)
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\scripts\Measure-CIBenchmark.ps1 `
|
||||||
|
-TemplatePath 'F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx' `
|
||||||
|
-SnapshotName BaseClean-Linux `
|
||||||
|
-GuestOS Linux `
|
||||||
|
-Iterations 10
|
||||||
|
```
|
||||||
|
- [ ] 10 iter completate.
|
||||||
|
- [ ] Confronto vs §10 (Linux host, Lin guest DHCP, boot total 8.82 s):
|
||||||
|
isola l'**effetto host** sul guest Linux. Se Win host ≫ 8.82 s,
|
||||||
|
l'overhead è host-side (NTFS / NAT vmnet8 Windows), non guest.
|
||||||
|
|
||||||
|
## 5. §13 — Concurrent burn-in 4×10 (pair §9)
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# WinBuild2025
|
||||||
|
.\scripts\Test-CapacityBurnIn.ps1 `
|
||||||
|
-Parallelism 4 -Rounds 10 `
|
||||||
|
-TemplatePath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
|
||||||
|
-RepoUrl 'http://10.10.20.11:3100/Simone/burnin-dummy.git' `
|
||||||
|
-CloneBaseDir 'F:\CI\BuildVMs\' `
|
||||||
|
-ArtifactBaseDir 'F:\CI\Artifacts\' `
|
||||||
|
-VmrunPath 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' `
|
||||||
|
-BuildCommand 'powershell -ExecutionPolicy Bypass -File .\build.ps1'
|
||||||
|
|
||||||
|
# LinuxBuild2404 (snapshot OBBLIGATORIO, default è BaseClean=Windows)
|
||||||
|
.\scripts\Test-CapacityBurnIn.ps1 `
|
||||||
|
-Parallelism 4 -Rounds 10 `
|
||||||
|
-TemplatePath 'F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx' `
|
||||||
|
-SnapshotName BaseClean-Linux `
|
||||||
|
-RepoUrl 'http://10.10.20.11:3100/Simone/burnin-dummy.git' `
|
||||||
|
-CloneBaseDir 'F:\CI\BuildVMs\' `
|
||||||
|
-ArtifactBaseDir 'F:\CI\Artifacts\' `
|
||||||
|
-VmrunPath 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' `
|
||||||
|
-GuestOS Linux `
|
||||||
|
-BuildCommand 'bash build.sh'
|
||||||
|
```
|
||||||
|
- [ ] Entrambi 40/40 PASS, 10/10 round.
|
||||||
|
- [ ] Round avg vs §9 (Linux host: Win ~78.6 s, Lin ~70.2 s): delta = effetto
|
||||||
|
host sotto concorrenza 4×.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Output e documentazione
|
||||||
|
|
||||||
|
- Tutti i run appendono a `F:\CI\Logs\benchmark.jsonl` (single-job) e
|
||||||
|
stampano summary (burn-in).
|
||||||
|
- [ ] Copiare/annotare i risultati e aggiungere a `docs/RUNBOOK.md` le
|
||||||
|
sezioni **§11–§13** con le stesse tabelle di §8/§9/§10 e una riga
|
||||||
|
finale di **confronto host Linux-vs-Windows** per ciascuna coppia.
|
||||||
|
- [ ] Pulizia post-run: `F:\CI\BuildVMs\` vuoto, artifact `burnin-*`
|
||||||
|
rimossi, `F:\CI\ip-pool.json` reset.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Matrice finale attesa (dopo questo plan)
|
||||||
|
|
||||||
|
| Host \ (Guest, mode) | Win/DHCP | Win/static | Lin/DHCP | burn-in 4×10 |
|
||||||
|
| -------------------- | -------- | ---------- | -------- | ------------ |
|
||||||
|
| **Linux** | §7 | §8 | §10 | §9 |
|
||||||
|
| **Windows** | §6 | §11 | §12 | §13 |
|
||||||
|
|
||||||
|
Confronto colonna-per-colonna = **effetto host** a guest+mode costanti.
|
||||||
|
Confronto riga-per-riga = **effetto guest/IP-mode** a host costante.
|
||||||
Reference in New Issue
Block a user