Simone/local-ci-cd-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'feat/phase-c-powershell-removal' (Phase C: pwsh-free Linux host)
Lint / pssa (push) Has been skipped
Details
Lint / python (push) Successful in 19s
Details

Browse Source
This commit is contained in:
Simone
2026-06-07 20:16:41 +02:00
parent 8f1b3b7eb1 962a69d287
commit decdf73202
20 changed files with 3339 additions and 1235 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitea/workflows/lint.yml
+14 -6
View File
@@ -1,8 +1,13 @@
# PSScriptAnalyzer + Python lint — runs on every push/PR that touches code files.
# Requires PSScriptAnalyzer installed on the runner host:
# Install-Module PSScriptAnalyzer -Scope AllUsers -Force
# Python lint runs on every push/PR (linux-build runner) and blocks the PR.
#
# Failures block the PR. Fix warnings with:
# PSScriptAnalyzer (pssa) runs on the Windows runner (windows-build) on-demand
# only — triggered via workflow_dispatch — because the .ps1/.psm1 targets are
# Windows-native (PS 5.1). It is intentionally NOT queued on push/PR so it does
# not block the pipeline when no Windows runner is registered (dual-boot Passo 9
# may be down). This keeps the Linux host/runner free of pwsh (Phase C goal).
# Requires PSScriptAnalyzer installed on the Windows runner host:
# Install-Module PSScriptAnalyzer -Scope AllUsers -Force
# Fix warnings with:
# Invoke-ScriptAnalyzer -Path scripts\ -Recurse -Fix
@@ -27,7 +32,10 @@ on:
jobs:
pssa:
runs-on: linux-build
# On-demand only: runs on the Windows runner (PS 5.1), never on push/PR,
# so it cannot block the pipeline when no Windows runner is registered.
if: ${{ github.event_name == 'workflow_dispatch' }}
runs-on: windows-build
timeout-minutes: 10
steps:
@@ -37,7 +45,7 @@ jobs:
fetch-depth: 1
- name: Run PSScriptAnalyzer
shell: pwsh
shell: powershell
run: |
$ErrorActionPreference = 'Stop'
CLAUDE.md
+4
View File
@@ -10,6 +10,8 @@ Self-hosted CI/CD system running on a Windows 11 host with VMware Workstation. E
**Current state**: The Python orchestrator (`src/ci_orchestrator/`) is the production entry point. The PowerShell scripts in `scripts/` are 3-line shims that forward to the Python CLI. All new logic goes in Python.
**Phase C (PowerShell-free Linux host)**: the manual burn-in / benchmark / smoke / host-validation / credential-setup tooling that used to run on the Linux host via `pwsh` is now native Python: `bench run`, `bench measure`, `smoke run`, `validate host`, `validate guest`, `creds set`. The Linux host no longer needs `pwsh` for any operation — it may be uninstalled (`setup-host-linux.sh` no longer installs it by default; opt in with `--with-pwsh`). PSScriptAnalyzer lint runs on a Windows runner on-demand, not on the Linux runner.
---
## Development Commands
@@ -115,6 +117,8 @@ The config file is auto-discovered at `$CI_ROOT/config.toml` (Linux default: `/v
## PowerShell 5.1 Constraints
These constraints apply to the `.ps1` / `.psm1` scripts that still run **inside Windows guests** and on the optional Windows runner. The **Linux host itself no longer runs any `.ps1`** (Phase C) — do not add PowerShell to the Linux host orchestration path.
All `.ps1` / `.psm1` scripts run on **Windows PowerShell 5.1** (not Core/7). Every script must start with:
```powershell
TODO.md
+8 -7
View File
@@ -849,17 +849,18 @@ liberi su 63.7). **Mitigazioni**: ridurre vCPU/VM (lega a §3.5), `-Parallelism
alzare connect-timeout pypsrp + retry a livello job in `_windows_build`.
**Rifiori quando**: si vuole concorrenza 4× Windows-guest affidabile su host Windows.
### 3.7 [P3] [ ] Shim Invoke-CIJob.ps1 — `$IsWindows` non guardato (PS 5.1 + StrictMode)
### 3.7 [P3] [x] Shim Invoke-CIJob.ps1 — `$IsWindows` non guardato (PS 5.1 + StrictMode) — RISOLTO 2026-06-07 (Fase C)
File: [scripts/Invoke-CIJob.ps1:37](scripts/Invoke-CIJob.ps1).
`if ($null -ne $IsWindows -and $IsWindows -eq $false)` dereferenzia `$IsWindows`,
inesistente su Windows PowerShell 5.1 → con `Set-StrictMode -Version Latest`
lancia `VariableIsUndefined` e il job fallisce subito. Si manifesta solo quando lo
shim gira sotto PS 5.1 **senza** `CI_VENV_PYTHON` settato (il ramo auto-detect).
Produzione setta `CI_VENV_PYTHON` via runner, quindi non si vede; colpisce invocazioni
manuali/burn-in. **Fix**: guardare con `if ((Test-Path variable:IsWindows) -and -not $IsWindows)`
o `Get-Variable IsWindows -EA SilentlyContinue`.
**Workaround attuale**: esportare `CI_VENV_PYTHON` prima di lanciare lo shim.
lancia `VariableIsUndefined` e il job fallisce subito. Si manifestava solo quando lo
shim girava sotto PS 5.1 **senza** `CI_VENV_PYTHON` settato (il ramo auto-detect),
cioè nelle invocazioni manuali/burn-in.
**Risoluzione**: il percorso burn-in manuale non passa più dallo shim — `bench run`
(Fase C) lancia i job concorrenti in-process via `concurrent.futures`, senza
`Start-Job`/`pwsh`/`$IsWindows`. Lo shim resta solo per il runner Windows, dove
`CI_VENV_PYTHON` è sempre settato. Il bug non è più raggiungibile sull'host Linux.
### 8.1 [P3] [ ] Mirror GitHub Actions su Gitea locale (rimuovi dipendenza github.com)
File: [.gitea/workflows/lint.yml](.gitea/workflows/lint.yml), Gitea (org `actions`).
docs/RUNBOOK.md
+22 -2
View File
@@ -738,8 +738,28 @@ running §11–§13; required for any Windows-host orchestration:
`Invoke-CIJob.ps1` interactively. The shim's auto-detect branch dereferences
`$IsWindows`, which is undefined under Windows PowerShell 5.1 + `StrictMode`
and throws `VariableIsUndefined`; setting `CI_VENV_PYTHON` skips that branch.
Production sets it via the runner (`CI_PYTHON_LAUNCHER`). A code fix to guard
the `$IsWindows` reference is tracked in `TODO.md`.
Production sets it via the runner (`CI_PYTHON_LAUNCHER`). *Note (Phase C): on
the Linux host this is moot — burn-in now runs via `bench run` (below), which
never touches the shim.*
### Phase C — Python equivalents (run these on the Linux host; no `pwsh`)
The manual `.ps1` procedures above are superseded by native CLI commands. Invoke
as the service user (`sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator ...`):
| Old PowerShell (Linux host) | Python command |
| --------------------------------------------------- | ------------------------------------------------ |
| `Test-CapacityBurnIn.ps1 -Parallelism 4 -Rounds 10` | `bench run --concurrency 4 --rounds 10 --guest-os linux` |
| `Measure-CIBenchmark.ps1 -Iterations N` | `bench measure --iterations N --guest-os linux` |
| `Test-Smoke.ps1 -GuestOS Linux` | `smoke run --guest-os linux` |
| `Validate-HostState.ps1` (host checks) | `validate host` |
| `Test-CIGuestWinRM.ps1` (transport diag) | `validate guest --host <IP> [--winrm\|--ssh]` |
| `Set-CIGuestCredential.ps1` | `creds set --target BuildVMGuest --user <u> --password-stdin` |
`bench run` writes a JSON report to `CI_ARTIFACTS/bench/`; `bench measure` appends
to `benchmark.jsonl` (same fields as before, so the §6–§13 trend lines stay
comparable). The benchmark records in §6–§13 above are kept as historical
provenance — they were captured with the `.ps1` tools before the cutover.
---
plans/idea-3-implementation-plan.md
+185
View File
@@ -0,0 +1,185 @@
# Fase C — Piano di applicazione (eliminazione `pwsh` dall'host Linux)
> Piano operativo per [idea-3-powershell-removal.md](idea-3-powershell-removal.md).
> Generato il 2026-06-07. Eseguire i passi **in ordine**: ogni passo è
> autonomo, testabile e con rollback banale (i `.ps1` non vengono rimossi
> finché C9 non chiude il burn-in di validazione).
---
## Summary — checklist master
Stato passi (spunta man mano):
- [x] **C1** — Scaffolding gruppi CLI `bench` / `smoke` / `validate` / `creds` + test skeleton
- [x] **C2** — `bench run`: porting burn-in concorrenza (`Test-CapacityBurnIn` + `Start-BurnInTest*`)
- [x] **C3** — `bench measure`: porting timing fasi (`Measure-CIBenchmark`) + retire `_Common.psm1`/`_Transport.psm1`/`_Common.Tests.ps1`
- [x] **C4** — `smoke run`: porting smoke + build E2E (`Test-Smoke`, `Test-*Build-Linux`)
- [x] **C5** — `validate host`: equivalente Linux di `Validate-HostState`
- [x] **C6** — `creds set` (rimpiazza `Set-CIGuestCredential`) + `validate guest` (diag transport)
- [x] **C7** — Lint CI: risolvere dipendenza `pwsh` di `lint.yml` (decisione §4 dell'analisi)
- [x] **C8** — `setup-host-linux.sh`: `pwsh` non più installato di default (`--with-pwsh` opt-in)
- [~] **C9** — Docs ✅ + cutover/burn-in ≥1 settimana ⏳ + disinstallazione `pwsh` ⏳ (operativi, vedi sotto)
**Definition of done globale**: tutti i criteri del §7 dell'analisi
verificati; `pwsh` rimosso dall'host Linux; pytest verde ≥90%.
---
## Convenzioni trasversali (valgono per ogni passo)
- Codice nuovo solo in `src/ci_orchestrator/`. `click.echo`, no `print()`,
no `subprocess(..., shell=True)`, type hints completi, `mypy --strict`.
- Importare **solo** `backends.load_backend` + transport esistenti
(`transport/ssh.py`, `transport/winrm.py`). Mai `WorkstationVmrunBackend`.
- Ogni nuovo comando ha test in `tests/python/test_commands_<gruppo>.py`
con backend e transport mockati (no VM reali in unit test).
- Gate coverage 90% per ogni PR. `ruff` + `mypy --strict` verdi.
- Un passo = una PR. Non toccare il branch WinRM e SSH nello stesso commit
(gotcha CLAUDE.md): se un comando ha entrambi i rami, separarli o
isolare dietro `if guest_os == "linux"`.
---
## C1 — Scaffolding
**Obiettivo**: registrare i gruppi vuoti e i file, senza logica reale,
per fissare l'interfaccia CLI e i test.
- [ ] Creare `commands/bench.py` con `@click.group("bench")` e i comandi
stub `run` / `measure` (firma completa, corpo `raise NotImplementedError`
dietro `--dry-run` per i test).
- [ ] Creare `commands/smoke.py` (`@click.group("smoke")` → `run`).
- [ ] Creare `commands/validate.py` (`@click.group("validate")` → `host`, `guest`).
- [ ] Creare `commands/creds.py` (`@click.group("creds")` → `set`).
- [ ] Registrarli in `__main__.py` (`cli.add_command(...)`).
- [ ] `tests/python/test_commands_bench.py` ecc.: verificare `--help` e
registrazione comandi (smoke test CLI).
- **Done**: `python -m ci_orchestrator bench --help` ecc. funzionano;
pytest verde.
## C2 — `bench run` (burn-in concorrenza)
**Sostituisce**: `Test-CapacityBurnIn.ps1`, `Start-BurnInTest.ps1`,
`Start-BurnInTest-Linux.ps1`.
- [ ] Implementare il lancio di N job concorrenti **in-process**
(`concurrent.futures` su `commands/job.py`) o come N subprocess del
CLI Python — **niente `Start-Job`/`pwsh`/`$IsWindows`** (chiude il bug
shim f22b142).
- [ ] Per ogni round, asserire: tutti i job exit 0; nessuna clone-dir
orfana in `CI_BUILD_VMS`; nessun `vm-start.lock` stale (>5 min).
- [ ] Report JSON in `CI_ARTIFACTS/bench/` + tabella riepilogo a video.
- [ ] Default lab: `--concurrency 4 --rounds 10`, build-command no-op
(`ping`/`sleep ~30s`) come negli wrapper.
- [ ] Test: round con job mockati (PASS/FAIL), rilevamento orfani e lock.
- **Done**: output equivalente a `Test-CapacityBurnIn.ps1` su Win e Linux.
## C3 — `bench measure` (timing fasi) + retire moduli PS
**Sostituisce**: `Measure-CIBenchmark.ps1` (+ `_Common.psm1`,
`_Transport.psm1`, `tests/_Common.Tests.ps1`).
- [ ] Misurare fasi clone / start / IP acquire / transport ready / destroy
via `load_backend` + transport, su 1+ VM effimere.
- [ ] Append a `benchmark.jsonl` (stesso formato, per continuità trend).
- [ ] Test con backend/transport mockati che simulano i tempi di fase.
- [ ] **Rimuovere** `scripts/_Common.psm1`, `scripts/_Transport.psm1`,
`tests/_Common.Tests.ps1` (nessun altro consumatore — verificato).
- [ ] Aggiornare `lint.yml` se elencava esplicitamente quei path.
- **Done**: `bench measure` riproduce le metriche; moduli PS rimossi;
pytest verde.
## C4 — `smoke run`
**Sostituisce**: `Test-Smoke.ps1`, `Test-Ns7zipBuild-Linux.ps1`,
`Test-NsinnounpBuild-Linux.ps1` (e i corrispettivi Win restano per fallback).
- [ ] `smoke run` esegue un job no-op (marker file) end-to-end e verifica:
exit 0, dir artifact creata, evento `job-success` nel JSONL.
- [ ] `--guest-os windows|linux`; preset build-command per le build reali
ns7zip / nsinnounp (opzionali, dietro flag o `--build-command`).
- [ ] Test: pipeline mockata, asserzioni su artifact dir + evento.
- **Done**: `smoke run --guest-os linux` e `--guest-os windows` passano.
## C5 — `validate host`
**Nuovo** equivalente Linux (non porting 1:1 di `Validate-HostState.ps1`).
- [ ] Verificare: `vmrun` presente/eseguibile; snapshot template presenti
(`BaseClean` / `BaseClean-Linux`); keyring file-based leggibile
(`keyring.get_credential`); permessi `/var/lib/ci/...`; unit
`act-runner` / `ci-*` attive (`systemctl is-active`).
- [ ] Exit 0 = tutto ok; non-zero + report per check fallito.
- [ ] Test: ogni check mockato (pass/fail) → exit code corretto.
- **Done**: `validate host` esce 0 su host configurato.
## C6 — `creds set` + `validate guest`
**Sostituisce**: `Set-CIGuestCredential.ps1` (host Linux) e
`Test-CIGuestWinRM.ps1` (parte diagnostica host-side).
- [ ] `creds set --target ... --user ... --password-stdin`: scrive nel
backend `keyring` letto da `ci_orchestrator`
(`keyrings.alt.file.PlaintextKeyring`). Password mai su command line.
- [ ] `validate guest --host IP [--winrm|--ssh]`: tenta connessione +
comando banale, riporta causa esplicita su fallimento (la sonda
`is_ready` ingoia gli errori — vedi gotcha).
- [ ] Test: scrittura/lettura credenziale (keyring mockato); diag con
transport mockato (connect ok / auth fail / connect fail).
- **Done**: credenziali settabili e transport diagnosticabile senza `pwsh`.
## C7 — Lint CI (decisione `pwsh`)
**Risolve dipendenza #5**. Applicare l'opzione scelta nel §4 dell'analisi.
- [ ] Decidere: (1) spostare job `pssa` su runner Windows / on-demand
[consigliata]; (2) isolare `pwsh` nel solo runner di lint; (3)
rimuovere PSSA da CI.
- [ ] Aggiornare `.gitea/workflows/lint.yml` di conseguenza.
- [ ] Verificare che nessun job che gira su `runs-on: linux-build` invochi
più `shell: pwsh`.
- **Done**: pipeline lint verde senza `pwsh` su runner Linux.
## C8 — `setup-host-linux.sh`
- [ ] Invertire lo step 6: non installare `pwsh` di default; aggiungere
flag `--with-pwsh` per opt-in (mantenere `--skip-pwsh` come alias
no-op deprecato o rimuoverlo).
- [ ] Aggiornare l'help e gli esempi d'uso nello script.
- **Done**: `setup-host-linux.sh` su host pulito non installa `pwsh`.
## C9 — Docs, cutover, burn-in, disinstallazione
Parte **docs** completata in codice (2026-06-07). Parte **operativa** (cutover
+ burn-in di validazione + disinstallazione) resta da eseguire sull'host reale.
- [x] `docs/RUNBOOK.md`: aggiunta tabella «Phase C — Python equivalents» + nota
che i record §6–§13 restano provenance storica; op-note §13 punto 3 aggiornato.
- [x] `CLAUDE.md`: nota "host Linux non usa più `pwsh`" (constraints PS 5.1
restano per script guest Windows + runner Windows).
- [x] `TODO.md`: chiusa voce bug shim `$IsWindows` (§3.7, risolto-per-rimpiazzo).
- [ ] **Cutover** (operativo): usare esclusivamente i comandi Python per ≥1
settimana (burn-in `bench run` periodico + smoke), `pwsh` ancora installato.
- [ ] **Burn-in verde ≥1 settimana** → `sudo apt-get remove powershell` sull'host
Linux.
- [ ] Riconfermare i criteri "C done" del §7 dell'analisi sull'host reale.
- **Done (codice/docs)**: porting C1–C8 mergeabile, suite verde ≥90%.
- **Done (operativo, pendente)**: `pwsh` assente dall'host dopo burn-in verde.
---
## Ordine consigliato e parallelizzabilità
`C1` prima di tutto. `C2`–`C6` indipendenti tra loro (parallelizzabili in
PR separate, ognuna sopra C1). `C3` deve precedere la rimozione dei moduli
`.psm1`. `C7` e `C8` indipendenti dai porting. `C9` ultimo, dopo che
C2–C8 sono merge-ati e il burn-in di validazione è verde.
## Rischi
- **WinRM 4× concorrenza instabile** (TODO noto): può far fallire `bench run`
su Windows con concorrenza alta — non è regressione del porting. Tarare
`--concurrency` e annotare.
- **Equivalenza output**: validare A/B (`.ps1` vs Python) finché `pwsh` è
ancora installato (prerequisito §6) prima della disinstallazione in C9.
plans/idea-3-powershell-removal.md
+153 -58
View File
@@ -1,103 +1,198 @@
# Fase C — Eliminazione dipendenza PowerShell dall'host Linux
> **Stato**: **pianificata**, da avviare dopo che Fase B è stabile in
> produzione (≥1 settimana di burn-in, Passo 8 completato).
> Vedi [overview](ideas-overview.md).
> **Stato**: **codice completato** (C1–C8 implementati + docs C9, 2026-06-07);
> resta solo la parte **operativa** di C9 (cutover ≥1 settimana + `apt-get
> remove powershell` sull'host reale).
> Vedi [overview](ideas-overview.md) e il
> [piano di applicazione](idea-3-implementation-plan.md) (checklist operativa).
>
> **Analisi rifatta il 2026-06-07** sullo stato reale del repo (la versione
> precedente proponeva gruppi CLI inesistenti e ometteva metà della
> superficie `pwsh`).
---
## 1. Obiettivo
Rendere l'host Linux completamente indipendente da PowerShell Core (`pwsh`).
Dopo la Fase B l'orchestrazione ordinaria (job, timer, monitoring, backup)
è già Python puro. Rimangono script PS usati per test manuali, burn-in e
validazione host che richiedono `pwsh` installato.
Dopo la Fase B l'orchestrazione ordinaria (job, timer, monitoring, backup,
retention, deploy template) è già Python puro. Restano script `.ps1`
eseguiti **manualmente** sull'host Linux con `pwsh` per: burn-in di
capacità, benchmark di timing, smoke/E2E test, validazione stato host e
setup credenziali. Inoltre il job di lint CI usa `shell: pwsh`.
**C done**: nessuno script in `scripts/*.ps1` viene più eseguito
direttamente sull'host Linux; `pwsh` può essere disinstallato senza
impatti operativi. I `.ps1` rimangono nel repo come riferimento storico
e per il runner Windows (fallback dual-boot).
direttamente sull'host Linux, e nessun job CI che gira su un runner Linux
richiede `pwsh`; `pwsh` può essere disinstallato dall'host Linux senza
impatti operativi. I `.ps1` restano nel repo come riferimento storico e
per l'eventuale runner Windows (fallback dual-boot Passo 9).
---
## 2. Script da portare
## 2. Superficie `pwsh` reale sull'host Linux
| Script PS | Uso attuale (Linux host) | Target Python |
| ---------------------------- | -------------------------------- | -------------------------------------------------------- |
| `Test-CapacityBurnIn.ps1` | burn-in B7: 4 job × 10 round | `python -m ci_orchestrator bench run` |
| `Start-BurnInTest.ps1` | burn-in Windows single-shot | `python -m ci_orchestrator bench run --guest-os windows` |
| `Start-BurnInTest-Linux.ps1` | burn-in Linux single-shot | `python -m ci_orchestrator bench run --guest-os linux` |
| `Validate-HostState.ps1` | verifica stato host pre-cutover | `python -m ci_orchestrator validate host` |
| `Measure-CIBenchmark.ps1` | raccolta metriche timing | integrato in `bench run --collect-metrics` |
| `Test-Smoke.ps1` | smoke end-to-end manuale | `python -m ci_orchestrator smoke run` |
| `Test-CapacityBurnIn.ps1` | già referenziato in B7 checklist | vedi sopra |
Lo stato CLI attuale (`src/ci_orchestrator/__main__.py`) espone i gruppi:
`wait-ready`, `vm`, `build`, `artifacts`, `monitor`, `report`,
`retention`, `template`, `job`. **Non esistono** gruppi `bench`,
`smoke`, `validate`, `creds` — vanno creati.
Script **non da portare** (girano solo su Windows o sono già shim):
- `Register-CIScheduledTasks.ps1` — Windows Scheduler, non serve su Linux
- `Invoke-RetentionPolicy.ps1` / `Backup-CITemplate.ps1` — già portati in B5
- Tutti gli script shim in `scripts/` — 3 righe che chiamano Python, restano
- Script guest (`Install-CIToolchain-*.ps1`, `Prepare-*.ps1`, ecc.) — girano
dentro le VM Windows, non sull'host
### 2.1 Cosa blocca davvero la disinstallazione di `pwsh`
| # | Dipendenza | Dove | Tipo |
| - | ---------- | ---- | ---- |
| 1 | Burn-in di capacità (4 job × N round) | `Test-CapacityBurnIn.ps1` + wrapper `Start-BurnInTest-Linux.ps1` (lanciato via `sudo -u ci-runner -H pwsh …`, vedi PhaseB-user-checklist §B7) | host Linux, manuale |
| 2 | Benchmark timing fasi (clone/start/IP/transport/destroy) | `Measure-CIBenchmark.ps1` — **unico consumatore** di `scripts/_Common.psm1` e `scripts/_Transport.psm1` | host Linux, manuale |
| 3 | Smoke end-to-end | `Test-Smoke.ps1 -GuestOS Linux` | host Linux, manuale |
| 4 | Build E2E reali (validazione pipeline) | `Test-Ns7zipBuild-Linux.ps1`, `Test-NsinnounpBuild-Linux.ps1` | host Linux, manuale |
| 5 | Lint CI (PSScriptAnalyzer) | `.gitea/workflows/lint.yml` → `shell: pwsh`, `runs-on: linux-build` | runner Linux, automatico |
| 6 | Installazione `pwsh` di default | `setup-host-linux.sh` step 6/6 (opt-out `--skip-pwsh`) | provisioning host |
| 7 | Pester (test dei moduli PS) | `tests/_Common.Tests.ps1` | host/CI, manuale |
| 8 | Setup credenziali guest | `Set-CIGuestCredential.ps1` (equivalente Linux mancante in CLI) | host Linux, manuale |
### 2.2 Script Windows-only (NON bloccano `pwsh` su Linux)
Hardcodano `F:\`, richiedono `#Requires -RunAsAdministrator` o usano il
Windows Credential Manager: **non girano sull'host Linux**, quindi non
vanno portati per l'obiettivo di Fase C. Restano per il runner Windows.
- `Validate-HostState.ps1` — logica Credential Manager/ACL NTFS, path `F:\`.
Serve però un **equivalente Linux** (`validate host`), non un porting 1:1.
- `Set-CIGuestCredential.ps1` — scrive nel vault SYSTEM via scheduled task.
Su Linux le credenziali stanno in `keyrings.alt.file.PlaintextKeyring`
(B3, `secret-tool` scartato). Serve un `creds set` Linux-nativo.
- `Test-CIGuestWinRM.ps1` — diagnostica WinRM, `RunAsAdministrator`.
- `Start-BurnInTest.ps1`, `Test-NsinnounpBuild.ps1`, `Test-E2E-Section3.3.ps1`
— wrapper Windows (path `F:\`, template WinBuild).
### 2.3 Già fuori scope (confermato)
- `Register-CIScheduledTasks.ps1` — Windows Scheduler.
- `Invoke-RetentionPolicy.ps1` / `Backup-CITemplate.ps1` — portati in B5
(`retention run` / `template backup`).
- Shim in `scripts/` (3 righe → Python) — restano.
- Script guest (`Install-CIToolchain-*`, `Prepare-*`, `Deploy-*`,
`template/guest-setup/windows/*`) — girano dentro le VM, non sull'host.
---
## 3. Architettura prevista
Nuovi sub-comandi CLI sotto due nuovi gruppi:
## 3. Architettura prevista (nuovi gruppi CLI)
```
python -m ci_orchestrator bench run --guest-os windows|linux
--template <path>
--concurrency N
--rounds N
--collect-metrics
--concurrency N # default 4
--rounds N # default 10
--build-command "..."
--json-out PATH
python -m ci_orchestrator bench measure --guest-os windows|linux
--iterations N
--json-out PATH # append a benchmark.jsonl
python -m ci_orchestrator validate host # verifica stato host
python -m ci_orchestrator smoke run # end-to-end smoke test
python -m ci_orchestrator smoke run --guest-os windows|linux
[--build-command "..."] # default: marker no-op
python -m ci_orchestrator validate host # vmrun, snapshot template, keyring,
# permessi dir, act-runner systemd attivo
python -m ci_orchestrator validate guest --host IP [--winrm|--ssh] # diag transport
python -m ci_orchestrator creds set --target BuildVMGuest --user ... [--password-stdin]
```
Il gruppo `bench` sostituisce i tre script `*BurnIn*.ps1` con logica
unificata: avvia N clone in parallelo, misura tempo per job, confronta
con baseline, emette un report JSON in `CI_ARTIFACTS/bench/`.
- **`bench run`** unifica `Test-CapacityBurnIn.ps1` + `Start-BurnInTest*.ps1`:
lancia N job concorrenti (riusando `commands/job.py` in-process o come
subprocess del CLI Python — niente `Start-Job`/`pwsh`), per R round,
e asserisce dopo ogni round: tutti exit 0, nessuna clone-dir orfana in
`CI_BUILD_VMS`, nessun `vm-start.lock` stale. Emette report JSON.
- **`bench measure`** porta `Measure-CIBenchmark.ps1` (fasi
clone/start/IP/transport/destroy) → consente di **retirare**
`_Common.psm1` e `_Transport.psm1` (nessun altro consumatore).
- **`smoke run`** porta `Test-Smoke.ps1` + i `Test-*Build-Linux.ps1`
(build E2E reali come preset opzionali del build-command).
- **`validate host`** è un nuovo equivalente Linux (non un porting di
`Validate-HostState.ps1`): verifica `vmrun`, presenza snapshot template,
leggibilità keyring file-based, permessi `/var/lib/ci/...`, unit
`act-runner`/`ci-*` attive.
- **`creds set`** sostituisce `Set-CIGuestCredential.ps1` su Linux usando
direttamente il backend `keyring` letto da `ci_orchestrator`.
Il gruppo `validate` porta `Validate-HostState.ps1`: verifica vmrun,
template snapshot, credenziali, permessi directory, act-runner attivo.
**Vincolo di design**: i nuovi comandi devono importare
`backends.load_backend` e i transport esistenti, **mai**
`WorkstationVmrunBackend` diretto (rotta ESXi Fase D aperta).
---
## 4. Aggiornamenti collaterali
## 4. Nodo aperto — lint PSScriptAnalyzer (dipendenza #5)
Gli script `.ps1` restano nel repo (guest + fallback Windows), quindi il
lint PSSA ha ancora valore. Ma `lint.yml` lo esegue con `shell: pwsh` su
`runs-on: linux-build`, il che reintroduce `pwsh` sul runner Linux.
**Decisione applicata (C7, 2026-06-07): opzione 1.** Il job `pssa` gira ora su
`runs-on: windows-build`, `shell: powershell` (PS 5.1), con guardia
`if: github.event_name == 'workflow_dispatch'` così non si accoda su push/PR se
il runner Windows non è registrato. Il job `python` resta su `linux-build`.
Nessun job su runner Linux usa più `pwsh`.
Tre opzioni considerate:
1. **(Consigliata)** Spostare il job `pssa` su un runner Windows
(`runs-on: windows-build`, `shell: powershell` PS 5.1) — coerente col
fatto che i `.ps1` target sono Windows-native. Richiede un runner
Windows registrato (dual-boot Passo 9) o eseguirlo solo on-demand.
2. Mantenere `pwsh` **solo nel container/runner di lint**, non sull'host
orchestratore: l'obiettivo "host Linux senza pwsh" resta soddisfatto se
il job gira in un runner isolato. Pragmatica ma ambigua sul "C done".
3. Rimuovere il job PSSA da CI e affidare il lint PS a un hook locale
manuale. Riduce copertura.
---
## 5. Aggiornamenti collaterali
- `setup-host-linux.sh`: invertire il default dello step 6 (no install
`pwsh`; flag `--with-pwsh` per opt-in) o rimuovere lo step.
- `.gitea/workflows/lint.yml`: applicare la decisione del §4.
- `tests/_Common.Tests.ps1`: rimuovere insieme ai moduli `.psm1` quando
`bench measure` li retira (Passo C3); la copertura passa a pytest.
- `CLAUDE.md`: la sezione "PowerShell 5.1 Constraints" resta valida per gli
script guest Windows; aggiungere nota che l'host Linux non usa più `pwsh`.
- `docs/RUNBOOK.md`: aggiornare procedure burn-in / smoke / validazione ai
nuovi comandi Python.
- `plans/archived/2026-06-07/PhaseB-user-checklist.md` §B7 (archiviato):
sostituire il comando `pwsh Test-CapacityBurnIn.ps1` con
`python -m ci_orchestrator bench run`.
- `deploy/systemd/README.md`: nessun riferimento a `pwsh`.
- `CLAUDE.md`: rimuovere nota "PowerShell 5.1 Constraints" se non più
rilevante per l'host (rimane valida per gli script guest Windows).
- `docs/RUNBOOK.md`: aggiornare procedure di burn-in e smoke test.
nota che il comando `pwsh Test-CapacityBurnIn.ps1` è superato da
`bench run`.
- `TODO.md`: chiudere la voce shim `$IsWindows` bug (f22b142) — sparisce
con la rimozione di `Start-Job`/`pwsh` nel burn-in.
---
## 5. Prerequisiti
## 6. Prerequisiti
- Fase B completata e stabile (≥1 settimana, Passo 8 ✅).
- `pwsh` ancora installato durante il porting (per confronto output A/B).
- Nessuna dipendenza da `pwsh` nei timer systemd (già soddisfatta da B5).
---
## 6. Criteri di "C done"
## 7. Criteri di "C done"
- `python -m ci_orchestrator bench run --concurrency 4 --rounds 10` esegue
burn-in completo (Win + Linux) con output equivalente a `Test-CapacityBurnIn.ps1`.
- `python -m ci_orchestrator validate host` esce con 0 su host configurato
correttamente.
- `pwsh --version` può essere assente sull'host Linux senza che nessun
test, timer o script CI fallisca.
- `bench run --concurrency 4 --rounds 10` (Win + Linux) produce esito
equivalente a `Test-CapacityBurnIn.ps1` (stesse asserzioni di cleanup).
- `bench measure` riproduce le fasi di `Measure-CIBenchmark.ps1`; i moduli
`_Common.psm1`/`_Transport.psm1` e `tests/_Common.Tests.ps1` rimossi.
- `smoke run --guest-os linux` e `--guest-os windows` passano.
- `validate host` esce 0 su host configurato correttamente.
- `creds set` sostituisce `Set-CIGuestCredential.ps1` sull'host Linux.
- Decisione §4 applicata: nessun job CI su runner Linux richiede `pwsh`.
- `setup-host-linux.sh` non installa più `pwsh` di default.
- `pwsh --version` può essere assente sull'host Linux senza che alcun
test, timer, job CI o procedura di RUNBOOK fallisca.
- Suite pytest ≥90% coverage, verde.
---
## 7. Rollback
## 8. Rollback
I `.ps1` originali rimangono nel repo e funzionano su host Windows o su
Linux con `pwsh` installato. Il rollback è semplicemente reinstallare
`pwsh` e usare gli script esistenti.
I `.ps1` originali restano nel repo e funzionano su host Windows o su
Linux con `pwsh` reinstallato (`setup-host-linux.sh --with-pwsh`). Il
rollback è reinstallare `pwsh` e usare gli script esistenti. Nessuna
distruzione di dati: il porting aggiunge comandi, non rimuove i `.ps1`
finché il burn-in di validazione non è verde.
scripts/Measure-CIBenchmark.ps1
-335
View File
@@ -1,335 +0,0 @@
#Requires -Version 5.1
<#
.SYNOPSIS
Measures CI infrastructure phase timings: clone, start, IP acquire, transport ready, destroy.
.DESCRIPTION
Creates one or more ephemeral VMs from the template, times each phase, destroys
them, then prints a summary table. Results are appended to benchmark.jsonl
for long-term trend tracking.
Phases measured:
clone — vmrun clone (linked clone from snapshot)
start — vmrun start (until command returns)
ip — vmrun getGuestIPAddress (polling until IP appears)
ready — TCP port reachable: 5986 (WinRM, Windows) or 22 (SSH, Linux)
destroy — stop + deleteVM + dir removal
No actual build is run.
.PARAMETER TemplatePath
Full path to the template VM's .vmx file.
.PARAMETER SnapshotName
Snapshot name to clone from. Default: BaseClean
.PARAMETER CloneBaseDir
Directory for ephemeral clone folders.
.PARAMETER VmrunPath
Path to vmrun executable.
.PARAMETER GuestOS
'Windows', 'Linux', or 'Auto' (default). Auto reads guestOS from the VMX:
ubuntu-* → Linux (probe SSH/22); anything else → Windows (probe WinRM/5986).
.PARAMETER Iterations
Number of clone→ready→destroy cycles to run. Default: 1
.PARAMETER TimeoutSeconds
Maximum seconds to wait for IP and transport per iteration. Default: 300
.PARAMETER OutputDir
Directory where benchmark.jsonl is appended.
.EXAMPLE
# Single baseline run (Linux host, Windows template)
.\Measure-CIBenchmark.ps1
# 4 iterations, Linux guest template
.\Measure-CIBenchmark.ps1 -TemplatePath /var/lib/ci/templates/LinuxBuild2404/LinuxBuild2404.vmx `
-SnapshotName BaseClean-Linux -Iterations 4
# Windows host
.\Measure-CIBenchmark.ps1 -TemplatePath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
-Iterations 3
#>
[CmdletBinding()]
param(
[string] $TemplatePath = $(if ($null -ne $IsWindows -and $IsWindows -eq $false) {
'/var/lib/ci/templates/WinBuild2025/WinBuild2025.vmx'
} else {
'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx'
}),
[string] $SnapshotName = 'BaseClean',
[string] $CloneBaseDir = $(if ($null -ne $IsWindows -and $IsWindows -eq $false) {
'/var/lib/ci/build-vms/'
} else {
'F:\CI\BuildVMs'
}),
[string] $VmrunPath = $(if ($null -ne $IsWindows -and $IsWindows -eq $false) {
'/usr/bin/vmrun'
} else {
'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe'
}),
[ValidateSet('Auto', 'Windows', 'Linux')]
[string] $GuestOS = 'Auto',
[ValidateRange(1, 10)]
[int] $Iterations = 1,
[ValidateRange(60, 600)]
[int] $TimeoutSeconds = 300,
[string] $OutputDir = $(if ($null -ne $IsWindows -and $IsWindows -eq $false) {
'/var/lib/ci/logs/'
} else {
'F:\CI\Logs'
}),
# Optional static IP injection (mirrors ip_pool in job.py).
# If supplied, guestinfo.ip-assignment/netmask/gateway are written into the
# cloned VMX before start so ci-static-ip.ps1 applies the IP at boot.
[string] $StaticIP = '',
[string] $Netmask = '255.255.255.0',
[string] $Gateway = ''
)
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
$ProgressPreference = 'SilentlyContinue'
Import-Module (Join-Path $PSScriptRoot '_Common.psm1') -Force
Resolve-VmrunPath -VmrunPath $VmrunPath | Out-Null
# Cross-platform TCP port probe (replaces Test-NetConnection which is Windows-only).
function Test-TcpPort {
param([string]$ComputerName, [int]$Port, [int]$TimeoutMs = 3000)
try {
$tcp = [System.Net.Sockets.TcpClient]::new()
$ar = $tcp.BeginConnect($ComputerName, $Port, $null, $null)
$ok = $ar.AsyncWaitHandle.WaitOne($TimeoutMs, $false)
$connected = $ok -and $tcp.Connected
$tcp.Close()
return $connected
} catch {
return $false
}
}
# Resolve effective GuestOS from VMX when Auto.
$resolvedGuestOS = $GuestOS
if ($resolvedGuestOS -eq 'Auto') {
$vmxRaw = Get-Content -LiteralPath $TemplatePath -Raw -ErrorAction SilentlyContinue
if ($vmxRaw -match 'guestOS\s*=\s*"([^"]+)"' -and $Matches[1] -like 'ubuntu*') {
$resolvedGuestOS = 'Linux'
} else {
$resolvedGuestOS = 'Windows'
}
Write-Host "[bench] GuestOS auto-detected: $resolvedGuestOS"
}
$transportPort = if ($resolvedGuestOS -eq 'Linux') { 22 } else { 5986 }
$transportLabel = if ($resolvedGuestOS -eq 'Linux') { 'SSH/22' } else { 'WinRM/5986' }
if (-not (Test-Path $TemplatePath -PathType Leaf)) {
throw "Template VMX not found: $TemplatePath"
}
if (-not (Test-Path $CloneBaseDir)) {
New-Item -ItemType Directory -Path $CloneBaseDir -Force | Out-Null
}
if (-not (Test-Path $OutputDir)) {
New-Item -ItemType Directory -Path $OutputDir -Force | Out-Null
}
function Set-GuestInfoIP {
[CmdletBinding(SupportsShouldProcess)]
param([string]$VmxPath, [string]$IP, [string]$Mask, [string]$Gw)
$keys = @('guestinfo.ip-assignment', 'guestinfo.netmask', 'guestinfo.gateway')
$lines = (Get-Content -LiteralPath $VmxPath -Encoding UTF8) |
Where-Object { ($_ -split '=')[0].Trim().ToLower() -notin $keys }
$lines += "guestinfo.ip-assignment = `"$IP`""
if ($Mask) { $lines += "guestinfo.netmask = `"$Mask`"" }
if ($Gw) { $lines += "guestinfo.gateway = `"$Gw`"" }
Set-Content -LiteralPath $VmxPath -Value $lines -Encoding UTF8
}
$benchmarkLog = Join-Path $OutputDir 'benchmark.jsonl'
$results = [System.Collections.Generic.List[pscustomobject]]::new()
$useStaticIP = $StaticIP -match '^\d{1,3}(\.\d{1,3}){3}$'
if ($useStaticIP) {
Write-Host "[bench] Static IP mode: $StaticIP / $Netmask gw=$Gateway"
} else {
Write-Host "[bench] DHCP mode (no -StaticIP supplied)"
}
Write-Host ""
Write-Host "=== CI Benchmark template: $(Split-Path $TemplatePath -Leaf) snapshot: $SnapshotName iterations: $Iterations ==="
Write-Host ""
for ($i = 1; $i -le $Iterations; $i++) {
Write-Host "--- Iteration $i / $Iterations ---"
$runId = "bench-$(Get-Date -Format 'yyyyMMdd-HHmmss')-$i"
$cloneVmx = $null
$cloneDir = $null
$t = [ordered]@{
clone = $null
start = $null
ip = $null
ready = $null
destroy = $null
deltaKB = $null
vmIP = $null
error = $null
}
try {
# ── Phase: clone ──────────────────────────────────────────────────────
$cloneName = "Clone_${runId}"
$cloneDir = Join-Path $CloneBaseDir $cloneName
$cloneVmx = Join-Path $cloneDir "$cloneName.vmx"
Write-Host "[bench] Cloning..."
$sw = [System.Diagnostics.Stopwatch]::StartNew()
$null = Invoke-Vmrun -VmrunPath $VmrunPath -Operation clone `
-Arguments @($TemplatePath, $cloneVmx, 'linked', '-snapshot', $SnapshotName) `
-ThrowOnError
$t.clone = [math]::Round($sw.Elapsed.TotalSeconds, 2)
Write-Host "[bench] clone: $($t.clone)s"
# Measure linked-clone disk footprint
try {
$cloneSizeBytes = (Get-ChildItem -Path $cloneDir -Recurse -File -ErrorAction Stop |
Measure-Object -Property Length -Sum).Sum
$t.deltaKB = [math]::Round($cloneSizeBytes / 1KB, 0)
Write-Host "[bench] clone size: $($t.deltaKB) KB"
} catch {
Write-Warning "[bench] Could not measure clone size: $_"
}
# ── Static IP injection (before start) ───────────────────────────────
if ($useStaticIP) {
Set-GuestInfoIP -VmxPath $cloneVmx -IP $StaticIP -Mask $Netmask -Gw $Gateway
Write-Host "[bench] guestinfo injected: $StaticIP"
}
# ── Phase: start ──────────────────────────────────────────────────────
Write-Host "[bench] Starting VM..."
$sw.Restart()
Invoke-Vmrun -VmrunPath $VmrunPath -Operation start `
-Arguments @($cloneVmx, 'nogui') -ThrowOnError | Out-Null
$t.start = [math]::Round($sw.Elapsed.TotalSeconds, 2)
Write-Host "[bench] start: $($t.start)s"
# ── Phase: IP acquire (primary: guestinfo.ci-ip, fallback: getGuestIPAddress) ──
Write-Host "[bench] Waiting for guest IP..."
$sw.Restart()
$vmIP = Get-GuestIPAddress -VmrunPath $VmrunPath -VmxPath $cloneVmx `
-TimeoutSeconds $TimeoutSeconds -GuestInfoOnly:$useStaticIP
if (-not $vmIP) { throw "Timed out waiting for guest IP after $TimeoutSeconds s" }
$t.ip = [math]::Round($sw.Elapsed.TotalSeconds, 2)
$t.vmIP = $vmIP
Write-Host "[bench] ip ($vmIP): $($t.ip)s"
# ── Phase: transport ready (SSH/22 for Linux, WinRM/5986 for Windows) ──
Write-Host "[bench] Waiting for $transportLabel..."
$sw.Restart()
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
$transportUp = $false
while ((Get-Date) -lt $deadline) {
$transportUp = Test-TcpPort -ComputerName $vmIP -Port $transportPort
if ($transportUp) { break }
Start-Sleep -Seconds 3
}
if (-not $transportUp) {
throw "Timed out waiting for $transportLabel on $vmIP after $TimeoutSeconds s"
}
$t.ready = [math]::Round($sw.Elapsed.TotalSeconds, 2)
Write-Host "[bench] ready ($transportLabel): $($t.ready)s"
}
catch {
$t.error = "$_"
Write-Warning "[bench] Iteration $i failed: $_"
}
finally {
# ── Phase: destroy ────────────────────────────────────────────────────
if ($cloneVmx -and (Test-Path $cloneVmx -PathType Leaf)) {
Write-Host "[bench] Destroying clone..."
$sw = [System.Diagnostics.Stopwatch]::StartNew()
& (Join-Path $PSScriptRoot 'Remove-BuildVM.ps1') `
-VMPath $cloneVmx -VmrunPath $VmrunPath -GracefulTimeoutSeconds 10 `
-ErrorAction SilentlyContinue
$t.destroy = [math]::Round($sw.Elapsed.TotalSeconds, 2)
Write-Host "[bench] destroy: $($t.destroy)s"
} elseif ($cloneDir -and (Test-Path $cloneDir)) {
Remove-Item $cloneDir -Recurse -Force -ErrorAction SilentlyContinue
}
}
$totalBootSec = if ($null -ne $t.ip -and $null -ne $t.ready) {
[math]::Round($t.clone + $t.start + $t.ip + $t.ready, 2)
} else { $null }
$result = [pscustomobject]@{
ts = (Get-Date -Format 'o')
runId = $runId
iteration = $i
template = Split-Path $TemplatePath -Leaf
snapshot = $SnapshotName
guestOS = $resolvedGuestOS
staticIP = if ($useStaticIP) { $StaticIP } else { $null }
cloneSec = $t.clone
startSec = $t.start
ipSec = $t.ip
readySec = $t.ready
destroySec = $t.destroy
totalBootSec = $totalBootSec
deltaKB = $t.deltaKB
vmIP = $t.vmIP
error = $t.error
}
$results.Add($result)
# Append to JSONL log
try {
$result | ConvertTo-Json -Compress -Depth 3 |
Add-Content -Path $benchmarkLog -Encoding UTF8 -ErrorAction Stop
} catch {
Write-Warning "[bench] Could not write benchmark.jsonl: $_"
}
Write-Host ""
}
# ── Summary table ─────────────────────────────────────────────────────────────
Write-Host "=== Results ==="
$results | Format-Table @(
@{ L = 'Iter'; E = { $_.iteration } }
@{ L = 'Clone(s)'; E = { $_.cloneSec } }
@{ L = 'Start(s)'; E = { $_.startSec } }
@{ L = 'IP(s)'; E = { $_.ipSec } }
@{ L = 'Ready(s)'; E = { $_.readySec } }
@{ L = 'Destroy(s)'; E = { $_.destroySec } }
@{ L = 'Boot total'; E = { $_.totalBootSec } }
@{ L = 'Delta(KB)'; E = { $_.deltaKB } }
@{ L = 'IP'; E = { $_.vmIP } }
@{ L = 'Error'; E = { $_.error } }
) -AutoSize
if ($Iterations -gt 1) {
$ok = @($results | Where-Object { -not $_.error })
if ($ok.Count -gt 0) {
$avgBoot = [math]::Round(($ok | ForEach-Object { $_.totalBootSec } |
Measure-Object -Average).Average, 2)
Write-Host "Average boot-to-ready ($($ok.Count) successful runs): $avgBoot s"
}
}
Write-Host ""
Write-Host "Results appended to: $benchmarkLog"
scripts/_Common.psm1
-435
View File
@@ -1,435 +0,0 @@
#Requires -Version 5.1
<#
.SYNOPSIS
Shared helper functions for the Local CI/CD System scripts.
.DESCRIPTION
Import this module at the top of any CI script that needs WinRM session
options or vmrun wrappers:
Import-Module (Join-Path $PSScriptRoot '_Common.psm1') -Force
Exported functions:
New-CISessionOption — PSSessionOption for WinRM over self-signed TLS
Resolve-VmrunPath — Validates vmrun.exe path, throws if missing
Invoke-Vmrun — Uniform vmrun -T ws wrapper; returns ExitCode + Output
#>
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
function New-CISessionOption {
<#
.SYNOPSIS
Returns a PSSessionOption for WinRM HTTPS with self-signed certificate.
.DESCRIPTION
All CI build VMs use a self-signed TLS cert on WinRM port 5986.
This option set skips CA, CN, and revocation checks — appropriate for
an isolated lab network where PKI is not present.
.OUTPUTS
[System.Management.Automation.Remoting.PSSessionOption]
#>
[OutputType([System.Management.Automation.Remoting.PSSessionOption])]
[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseShouldProcessForStateChangingFunctions', '',
Justification = 'New-CISessionOption creates an in-memory PSSessionOption object; no system state is changed.')]
[CmdletBinding()]
param()
New-PSSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck
}
function Resolve-VmrunPath {
<#
.SYNOPSIS
Validates that vmrun.exe exists at the specified path and returns it.
.DESCRIPTION
Throws a descriptive error if the file is not found, rather than letting
a later & call fail with a less useful message.
.PARAMETER VmrunPath
Full path to vmrun.exe.
Default: C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe
.OUTPUTS
[string] The validated VmrunPath.
#>
[OutputType([string])]
[CmdletBinding()]
param(
[string] $VmrunPath = 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe'
)
if (-not (Test-Path $VmrunPath -PathType Leaf)) {
throw "vmrun.exe not found at: $VmrunPath`nInstall VMware Workstation Pro and update -VmrunPath."
}
return $VmrunPath
}
function Invoke-Vmrun {
<#
.SYNOPSIS
Runs a vmrun -T ws <Operation> command and returns output and exit code.
.DESCRIPTION
Uniform wrapper so all vmrun calls share the same -T ws flag and output
capture pattern. Use -ThrowOnError for operations that must succeed
(clone, start); omit it for best-effort operations (stop, getState).
.PARAMETER VmrunPath
Full path to vmrun.exe.
.PARAMETER Operation
vmrun sub-command: clone, start, stop, deleteVM, list, getState,
listSnapshots, getGuestIPAddress, etc.
.PARAMETER Arguments
Additional positional arguments after the operation name.
.PARAMETER ThrowOnError
When set, throws if vmrun exits non-zero.
.OUTPUTS
[pscustomobject] with:
ExitCode [int] — vmrun exit code
Output [string[]] — combined stdout/stderr lines
#>
[OutputType([pscustomobject])]
[CmdletBinding()]
param(
[Parameter(Mandatory)] [string] $VmrunPath,
[Parameter(Mandatory)] [string] $Operation,
[string[]] $Arguments = @(),
[switch] $ThrowOnError
)
$cmdArgs = @('-T', 'ws', $Operation) + $Arguments
$output = & $VmrunPath @cmdArgs 2>&1
$exit = $LASTEXITCODE
if ($ThrowOnError -and $exit -ne 0) {
throw "vmrun $Operation failed (exit $exit): $($output -join '; ')"
}
return [pscustomobject]@{
ExitCode = $exit
Output = $output
}
}
function Invoke-VmrunBounded {
<#
.SYNOPSIS
Runs a vmrun -T ws <Operation> command with a hard wall-clock timeout.
.DESCRIPTION
Uses [System.Diagnostics.Process] directly (not Start-Process -PassThru) because
Start-Process in PowerShell 5.1 does not reliably populate Process.ExitCode after
WaitForExit(ms) when output is redirected. ReadToEndAsync() is used for stdout and
stderr to prevent deadlock if output buffers fill up before the process exits.
Use for lifecycle operations: start (180s), stop (60s), deleteVM (90s).
Keep Invoke-Vmrun for fast queries: list, readVariable, getGuestIPAddress, getState.
.PARAMETER VmrunPath
Full path to vmrun.exe.
.PARAMETER Operation
vmrun sub-command (start, stop, deleteVM, clone, …).
.PARAMETER Arguments
Additional positional arguments after the operation name.
.PARAMETER TimeoutSeconds
Hard timeout in seconds. Process is killed if not finished within this time.
Default: 120.
.PARAMETER ThrowOnTimeout
When set, throws if the timeout is reached before vmrun exits.
.PARAMETER ThrowOnError
When set, throws if vmrun exits with a non-zero code.
.OUTPUTS
[pscustomobject] with:
ExitCode [int] — vmrun exit code (-1 if killed due to timeout)
Output [string] — combined stdout/stderr text
TimedOut [bool] — true if the process was killed for exceeding TimeoutSeconds
ElapsedSeconds [double] — wall-clock seconds from start to exit/kill
#>
[OutputType([pscustomobject])]
[CmdletBinding()]
param(
[Parameter(Mandatory)] [string] $VmrunPath,
[Parameter(Mandatory)] [string] $Operation,
[string[]] $Arguments = @(),
[int] $TimeoutSeconds = 120,
[switch] $ThrowOnTimeout,
[switch] $ThrowOnError
)
$cmdArgs = @('-T', 'ws', $Operation) + $Arguments
# Quote tokens that contain spaces so the argument string is correct when
# passed to ProcessStartInfo.Arguments as a single string.
$quotedArgs = $cmdArgs | ForEach-Object {
if ($_ -match '\s') { '"' + ($_ -replace '"', '""') + '"' } else { $_ }
}
$argStr = $quotedArgs -join ' '
$psi = New-Object System.Diagnostics.ProcessStartInfo
$psi.FileName = $VmrunPath
$psi.Arguments = $argStr
$psi.UseShellExecute = $false
$psi.CreateNoWindow = $true
$psi.RedirectStandardOutput = $true
$psi.RedirectStandardError = $true
$proc = New-Object System.Diagnostics.Process
$proc.StartInfo = $psi
$sw = [System.Diagnostics.Stopwatch]::StartNew()
$timedOut = $false
[void]$proc.Start()
# Begin async reads before WaitForExit to avoid deadlock when output buffers fill.
$stdoutTask = $proc.StandardOutput.ReadToEndAsync()
$stderrTask = $proc.StandardError.ReadToEndAsync()
$exited = $proc.WaitForExit($TimeoutSeconds * 1000)
if (-not $exited) {
$timedOut = $true
# Kill the entire process tree (taskkill /T) to release any inherited pipe
# handles held by child processes (e.g. cmd.exe spawning ping.exe in tests,
# or vmrun spawning helper subprocesses). Plain proc.Kill() only kills the
# direct process; child processes keep the write end of stdout open.
if ($null -ne $IsWindows -and $IsWindows -eq $false) {
# .NET 5+ Kill(entireProcessTree) — works on Linux/macOS.
$proc.Kill($true)
} else {
& taskkill /F /T /PID $proc.Id 2>&1 | Out-Null
}
}
$sw.Stop()
$outputText = ''
if (-not $timedOut) {
# Process exited normally: drain stdout/stderr (up to 5s safety net).
[void][System.Threading.Tasks.Task]::WhenAll($stdoutTask, $stderrTask).Wait(5000)
$stdoutText = ''
$stderrText = ''
if ($stdoutTask.Status -eq [System.Threading.Tasks.TaskStatus]::RanToCompletion) {
$stdoutText = $stdoutTask.Result
}
if ($stderrTask.Status -eq [System.Threading.Tasks.TaskStatus]::RanToCompletion) {
$stderrText = $stderrTask.Result
}
$parts = @($stdoutText, $stderrText) | Where-Object { -not [string]::IsNullOrWhiteSpace($_) }
$outputText = ($parts | ForEach-Object { $_.TrimEnd() }) -join "`n"
}
# When timed-out: process was killed; output is irrelevant and potentially
# incomplete. Skip drain entirely — no point blocking for a killed process.
$exitCode = if ($timedOut) { -1 } else { $proc.ExitCode }
$result = [pscustomobject]@{
ExitCode = $exitCode
Output = $outputText
TimedOut = $timedOut
ElapsedSeconds = [math]::Round($sw.Elapsed.TotalSeconds, 1)
}
if ($timedOut -and $ThrowOnTimeout) {
throw "vmrun $Operation timed out after ${TimeoutSeconds}s"
}
if (-not $timedOut -and $exitCode -ne 0 -and $ThrowOnError) {
throw "vmrun $Operation failed (exit $exitCode): $outputText"
}
return $result
}
function Get-GuestIPAddress {
<#
.SYNOPSIS
Polls a running VMware guest VM until its IP address is available, then returns it.
.DESCRIPTION
Primary method: reads the 'ci-ip' guestVar written by ci-report-ip.service (Linux)
or the equivalent guest-side script via vmware-rpctool. This uses the VMware VMCI
channel and does not require TCP/IP connectivity from the host.
Fallback: vmrun getGuestIPAddress (reliable on Windows guests; may be slow on Linux).
Returns the detected IP as a plain string, or an empty string if the timeout expires.
.PARAMETER VmrunPath
Full path to vmrun.exe.
.PARAMETER VmxPath
Full path to the clone VM's .vmx file.
.PARAMETER TimeoutSeconds
Maximum seconds to poll before giving up. Default: 120.
.OUTPUTS
[string] Detected IPv4 address, or empty string on timeout.
#>
[OutputType([string])]
[CmdletBinding()]
param(
[Parameter(Mandatory)] [string] $VmrunPath,
[Parameter(Mandatory)] [string] $VmxPath,
[int] $TimeoutSeconds = 120,
# When set, skip vmrun getGuestIPAddress fallback and poll only
# guestinfo.ci-ip. Use when guestinfo.ip-assignment was injected into
# the VMX — avoids returning a transient DHCP address before
# ci-static-ip.ps1 has finished reconfiguring the NIC.
[switch] $GuestInfoOnly
)
$modeLabel = if ($GuestInfoOnly) { 'guestinfo only' } else { 'guestinfo + getGuestIPAddress' }
Write-Host "[Get-GuestIPAddress] Polling for VM IP (max ${TimeoutSeconds}s, mode: $modeLabel)..."
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
$attempt = 0
while ((Get-Date) -lt $deadline) {
$attempt++
# Primary: guestinfo.ci-ip written by ci-report-ip.service via vmware-rpctool.
# NOTE: vmrun readVariable guestVar reads WITHOUT the 'guestinfo.' prefix.
$savedEap = $ErrorActionPreference; $ErrorActionPreference = 'Continue'
$ipOut = (& $VmrunPath -T ws readVariable $VmxPath guestVar 'ci-ip' 2>&1).Trim()
$ipRc = $LASTEXITCODE
$ErrorActionPreference = $savedEap
if ($attempt -le 3 -or ($attempt % 10 -eq 0)) {
Write-Host "[Get-GuestIPAddress] [attempt $attempt] guestVar ci-ip: rc=$ipRc out='$ipOut'"
}
if ($ipRc -eq 0 -and $ipOut -match '^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$') {
Write-Host "[Get-GuestIPAddress] IP via guestinfo: $ipOut"
return $ipOut
}
# Fallback: vmrun getGuestIPAddress — skipped when GuestInfoOnly is set
# to avoid returning a transient DHCP address during static-IP reconfiguration.
if (-not $GuestInfoOnly) {
$savedEap = $ErrorActionPreference; $ErrorActionPreference = 'Continue'
$ipOut2 = (& $VmrunPath -T ws getGuestIPAddress $VmxPath 2>&1).Trim()
$ipRc2 = $LASTEXITCODE
$ErrorActionPreference = $savedEap
if ($ipRc2 -eq 0 -and $ipOut2 -match '^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$') {
Write-Host "[Get-GuestIPAddress] IP via getGuestIPAddress: $ipOut2"
return $ipOut2
}
}
Start-Sleep -Seconds 2
}
return ''
}
function Get-GuestDiagnostics {
<#
.SYNOPSIS
Best-effort collection of guest diagnostics after a build failure.
Never throws — always safe to call from a catch/finally block.
.PARAMETER IPAddress
IP address of the guest VM.
.PARAMETER GuestOS
'Windows' or 'Linux'.
.PARAMETER Credential
PSCredential for WinRM (Windows guests). Ignored for Linux.
.PARAMETER SshKeyPath
Path to SSH private key (Linux guests). Ignored for Windows.
.PARAMETER SshUser
SSH username (Linux guests). Default: ci_build.
.PARAMETER DestinationDir
Host directory where diagnostics files are written.
#>
[CmdletBinding()]
param(
[Parameter(Mandatory)] [string] $IPAddress,
[Parameter(Mandatory)] [ValidateSet('Windows','Linux')]
[string] $GuestOS,
[pscredential] $Credential,
[string] $SshKeyPath,
[string] $SshUser = 'ci_build',
[Parameter(Mandatory)] [string] $DestinationDir
)
Write-Host "[Diagnostics] Collecting guest diagnostics from $IPAddress ($GuestOS) -> $DestinationDir"
try {
if (-not (Test-Path $DestinationDir)) {
New-Item -ItemType Directory -Path $DestinationDir -Force | Out-Null
}
} catch {
Write-Warning "[Diagnostics] Could not create destination dir: $_"
return
}
if ($GuestOS -eq 'Windows') {
if (-not $Credential) {
Write-Warning "[Diagnostics] No Credential supplied for Windows guest — skipping."
return
}
try {
$sessionOption = New-CISessionOption
$session = New-PSSession -ComputerName $IPAddress `
-Credential $Credential `
-SessionOption $sessionOption `
-ErrorAction Stop
try {
# Collect: event log errors (last 50), running processes, disk usage
$diagScript = {
$out = [System.Text.StringBuilder]::new()
[void]$out.AppendLine("=== System Event Log (last 50 errors) ===")
Get-EventLog -LogName System -EntryType Error -Newest 50 -ErrorAction SilentlyContinue |
ForEach-Object { [void]$out.AppendLine("$($_.TimeGenerated) $($_.Source): $($_.Message)") }
[void]$out.AppendLine("`n=== Running Processes ===")
Get-Process -ErrorAction SilentlyContinue |
Sort-Object CPU -Descending |
Select-Object -First 30 |
ForEach-Object { [void]$out.AppendLine("$($_.Name) PID=$($_.Id) CPU=$($_.CPU)") }
[void]$out.AppendLine("`n=== Disk Free Space ===")
Get-PSDrive -PSProvider FileSystem -ErrorAction SilentlyContinue |
ForEach-Object { [void]$out.AppendLine("$($_.Name): Free=$([math]::Round($_.Free/1GB,1))GB Used=$([math]::Round($_.Used/1GB,1))GB") }
$out.ToString()
}
$diagText = Invoke-Command -Session $session -ScriptBlock $diagScript -ErrorAction Stop
$diagText | Set-Content -Path (Join-Path $DestinationDir 'guest-diag.txt') -Encoding UTF8
Write-Host "[Diagnostics] Windows diagnostics written."
# Best-effort: copy CI build log
$buildLog = 'C:\CI\build.log'
try {
Copy-Item -FromSession $session -Path $buildLog `
-Destination (Join-Path $DestinationDir 'build.log') `
-ErrorAction Stop
Write-Host "[Diagnostics] Build log copied."
} catch {
Write-Host "[Diagnostics] Build log not found in guest (skipping)."
}
} finally {
Remove-PSSession $session -ErrorAction SilentlyContinue
}
} catch {
Write-Warning "[Diagnostics] Windows guest diagnostics failed (non-fatal): $_"
}
}
else {
# Linux
if (-not $SshKeyPath) {
Write-Warning "[Diagnostics] No SshKeyPath supplied for Linux guest — skipping."
return
}
$sshOpts = @('-i', $SshKeyPath, '-o', 'StrictHostKeyChecking=no',
'-o', 'UserKnownHostsFile=NUL',
'-o', 'ConnectTimeout=10', '-o', 'BatchMode=yes')
try {
# Collect: journalctl errors, ps, df
$diagCmd = "journalctl -p err -n 50 --no-pager 2>/dev/null; echo '=== ps ==='; ps aux --sort=-%cpu | head -30; echo '=== df ==='; df -h"
$savedEap = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
$diagOutput = & ssh.exe @sshOpts "${SshUser}@${IPAddress}" $diagCmd 2>&1
$ErrorActionPreference = $savedEap
$diagOutput | Set-Content -Path (Join-Path $DestinationDir 'guest-diag.txt') -Encoding UTF8
Write-Host "[Diagnostics] Linux diagnostics written."
# Best-effort: copy CI build log
$scpSrc = "${SshUser}@${IPAddress}:/opt/ci/build/ci-build.log"
$scpDst = Join-Path $DestinationDir 'build.log'
$ErrorActionPreference = 'Continue'
& scp.exe @sshOpts $scpSrc $scpDst 2>&1 | Out-Null
$ErrorActionPreference = $savedEap
if (Test-Path $scpDst) { Write-Host "[Diagnostics] Build log copied." }
else { Write-Host "[Diagnostics] Build log not found in guest (skipping)." }
} catch {
Write-Warning "[Diagnostics] Linux guest diagnostics failed (non-fatal): $_"
}
}
}
Export-ModuleMember -Function New-CISessionOption, Resolve-VmrunPath, Invoke-Vmrun, Invoke-VmrunBounded, Get-GuestIPAddress, Get-GuestDiagnostics
scripts/_Transport.psm1
-201
View File
@@ -1,201 +0,0 @@
#Requires -Version 5.1
<#
.SYNOPSIS
SSH/SCP transport helpers for Linux build VMs.
.DESCRIPTION
SSH-only transport layer for the Linux branch of the CI/CD system.
The existing WinRM transport (used by Windows VMs) is NOT modified.
Exported functions:
Invoke-SshCommand — run a command in a Linux guest via ssh.exe
Copy-SshItem — copy files to/from a Linux guest via scp.exe
Test-SshReady — poll until SSH is ready (port 22 open + login succeeds)
Requirements:
- ssh.exe and scp.exe in PATH (default on Windows 11 / Server 2022)
- SSH private key at the path provided
- Guest user must be set up for key-based auth (no passphrase)
#>
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
$ProgressPreference = 'SilentlyContinue' # suppress Write-Progress (Test-NetConnection, IWR, etc.)
function Invoke-SshCommand {
<#
.SYNOPSIS
Run a command in a Linux guest via ssh.exe
#>
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[string] $IP,
[string] $User = 'ci_build',
[string] $KeyPath = 'F:\CI\keys\ci_linux',
[Parameter(Mandatory)]
[string] $Command,
[int] $ConnectTimeout = 10,
# Persistent known_hosts file for CI jobs.
# When non-empty: StrictHostKeyChecking=accept-new + this file.
# When empty (default): permissive mode (StrictHostKeyChecking=no, UserKnownHostsFile=NUL)
# — used during template provisioning when host key is not yet known.
[string] $KnownHostsFile = '',
# Capture + return stdout/stderr instead of printing to console.
[switch] $PassThru,
# Don't throw on non-zero exit — just return the output.
[switch] $AllowFail
)
if ($KnownHostsFile -ne '') {
$sshHostKeyOpts = @('-o', 'StrictHostKeyChecking=accept-new', '-o', "UserKnownHostsFile=$KnownHostsFile")
} else {
$sshHostKeyOpts = @('-o', 'StrictHostKeyChecking=no', '-o', 'UserKnownHostsFile=NUL')
}
$sshArgs = @(
'-i', $KeyPath
) + $sshHostKeyOpts + @(
'-o', "ConnectTimeout=$ConnectTimeout",
'-o', 'BatchMode=yes',
"$User@$IP",
$Command
)
if ($PassThru) {
$output = & ssh @sshArgs 2>&1
$exit = $LASTEXITCODE
}
else {
& ssh @sshArgs
$exit = $LASTEXITCODE
$output = @()
}
if ($exit -ne 0 -and -not $AllowFail) {
throw "[SSH] Command failed (exit $exit): $Command"
}
return [pscustomobject]@{
ExitCode = [int]$exit
Output = [string[]]$output
}
}
function Copy-SshItem {
<#
.SYNOPSIS
Copy files to/from Linux guest via scp.exe
#>
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[string] $Source,
[Parameter(Mandatory)]
[string] $Destination,
# Guest IP — used to build user@host prefix when Direction is set.
[string] $IP = '',
[string] $User = 'ci_build',
[Parameter(Mandatory)]
[string] $KeyPath,
[ValidateSet('ToGuest', 'FromGuest', 'Direct')]
[string] $Direction = 'Direct',
# Pass -r to scp (recursive copy).
[switch] $Recurse,
[int] $ConnectTimeout = 10,
# Persistent known_hosts file for CI jobs.
# When non-empty: StrictHostKeyChecking=accept-new + this file.
# When empty (default): permissive mode — used during template provisioning.
[string] $KnownHostsFile = ''
)
if ($Direction -eq 'ToGuest') {
$Destination = "$User@${IP}:$Destination"
}
elseif ($Direction -eq 'FromGuest') {
$Source = "$User@${IP}:$Source"
}
# Direction = 'Direct': use Source + Destination as-is
if ($KnownHostsFile -ne '') {
$scpHostKeyOpts = @('-o', 'StrictHostKeyChecking=accept-new', '-o', "UserKnownHostsFile=$KnownHostsFile")
} else {
$scpHostKeyOpts = @('-o', 'StrictHostKeyChecking=no', '-o', 'UserKnownHostsFile=NUL')
}
$scpArgs = @(
'-i', $KeyPath
) + $scpHostKeyOpts + @(
'-o', "ConnectTimeout=$ConnectTimeout"
)
if ($Recurse) { $scpArgs += '-r' }
$scpArgs += @($Source, $Destination)
& scp @scpArgs
if ($LASTEXITCODE -ne 0) {
throw "[SCP] Copy failed (exit $LASTEXITCODE): $Source -> $Destination"
}
}
function Test-SshReady {
<#
.SYNOPSIS
Poll until SSH port 22 is open and a login command succeeds.
#>
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[string] $IP,
[string] $User = 'ci_build',
[string] $KeyPath = 'F:\CI\keys\ci_linux',
[int] $TimeoutSec = 300,
[int] $PollSec = 10
)
$deadline = (Get-Date).AddSeconds($TimeoutSec)
# Phase 1: wait for port 22 to be open
$portOpen = $false
while (-not $portOpen -and (Get-Date) -lt $deadline) {
$portOpen = (Test-NetConnection -ComputerName $IP -Port 22 `
-InformationLevel Quiet -WarningAction SilentlyContinue `
-ErrorAction SilentlyContinue)
if (-not $portOpen) {
Start-Sleep -Seconds $PollSec
}
}
if (-not $portOpen) {
throw "[Test-SshReady] Timed out after ${TimeoutSec}s waiting for SSH at $IP"
}
# Phase 2: try ssh echo — success when output contains 'ready'
while ((Get-Date) -lt $deadline) {
$result = Invoke-SshCommand -IP $IP -User $User -KeyPath $KeyPath `
-Command 'echo ready' -AllowFail -PassThru
if ($result.ExitCode -eq 0 -and ($result.Output -join '') -like '*ready*') {
return $true
}
Start-Sleep -Seconds $PollSec
}
throw "[Test-SshReady] Timed out after ${TimeoutSec}s waiting for SSH at $IP"
}
Export-ModuleMember -Function Invoke-SshCommand, Copy-SshItem, Test-SshReady
setup-host-linux.sh
+18 -9
View File
@@ -11,8 +11,10 @@
# portando i contenuti alla root della partizione (nessun dato perso)
# 3. Crea il layout directory e permessi
# 4. Installa Python 3.11+ e crea venv produzione in /opt/ci/venv
# 5. (Opzionale) Installa PowerShell Core
# 6. (Opzionale) Clona il repo e installa il package nel venv
# 5. (Opzionale) Clona il repo e installa il package nel venv
# 6. (Opt-in, --with-pwsh) Installa PowerShell Core
# L'host Linux NON ha più bisogno di pwsh per il normale funzionamento
# (orchestratore Python). Installalo solo se ti serve per script legacy.
set -euo pipefail
@@ -24,7 +26,7 @@ OPT_CI="/opt/ci"
REPO_URL="https://gitea.emulab.it/Simone/local-ci-cd-system.git"
SKIP_PYTHON=false
SKIP_PWSH=false
WITH_PWSH=false
SKIP_CLONE=false
# ── Colori ───────────────────────────────────────────────────────────────────
@@ -45,12 +47,16 @@ Opzionali:
-r <path> CI root (default: /var/lib/ci)
-u <utente> Utente di servizio (default: ci-runner)
--skip-python Salta installazione Python/venv
--skip-pwsh Salta installazione PowerShell Core
--skip-clone Salta clone repo + pip install
--with-pwsh Installa PowerShell Core (opt-in; NON installato di default).
L'host Linux non ha più bisogno di pwsh per il normale
funzionamento — usa questo flag solo per script legacy.
--skip-pwsh DEPRECATO: no-op. pwsh non è più installato di default.
Esempi:
sudo bash setup-host-linux.sh -d /dev/sdf1
sudo bash setup-host-linux.sh -d /dev/sdf1 --skip-pwsh --skip-clone
sudo bash setup-host-linux.sh -d /dev/sdf1 --with-pwsh
sudo bash setup-host-linux.sh -d /dev/sdf1 --skip-clone
EOF
exit 1
}
@@ -63,7 +69,10 @@ while [[ $# -gt 0 ]]; do
-r) CI_ROOT="$2"; shift 2 ;;
-u) CI_USER="$2"; shift 2 ;;
--skip-python) SKIP_PYTHON=true; shift ;;
--skip-pwsh) SKIP_PWSH=true; shift ;;
--with-pwsh) WITH_PWSH=true; shift ;;
--skip-pwsh)
echo "WARN: --skip-pwsh è deprecato e non ha più effetto: pwsh non è installato di default (usa --with-pwsh per installarlo)." >&2
shift ;;
--skip-clone) SKIP_CLONE=true; shift ;;
-h|--help) usage ;;
*) die "Parametro sconosciuto: $1"; ;;
@@ -230,8 +239,8 @@ else
info "[5/6] Clone repo — SKIPPED"
fi
# ── Step 6: PowerShell Core ───────────────────────────────────────────────────
if [[ "$SKIP_PWSH" == false ]]; then
# ── Step 6: PowerShell Core (opt-in via --with-pwsh) ──────────────────────────
if [[ "$WITH_PWSH" == true ]]; then
info "[6/6] PowerShell Core"
if command -v pwsh &>/dev/null; then
warn "pwsh già installato: $(pwsh --version) — skip"
@@ -254,7 +263,7 @@ if [[ "$SKIP_PWSH" == false ]]; then
echo " Installato: $(pwsh --version)"
fi
else
info "[6/6] PowerShell Core — SKIPPED"
info "[6/6] PowerShell Core — SKIPPED (opt-in con --with-pwsh)"
fi
# ── Riepilogo ─────────────────────────────────────────────────────────────────
src/ci_orchestrator/__main__.py
+8
View File
@@ -13,12 +13,16 @@ import click
from ci_orchestrator import __version__
from ci_orchestrator.commands.artifacts import artifacts
from ci_orchestrator.commands.bench import bench
from ci_orchestrator.commands.build import build
from ci_orchestrator.commands.creds import creds
from ci_orchestrator.commands.job import job
from ci_orchestrator.commands.monitor import monitor
from ci_orchestrator.commands.report import report
from ci_orchestrator.commands.retention import retention
from ci_orchestrator.commands.smoke import smoke
from ci_orchestrator.commands.template import template
from ci_orchestrator.commands.validate import validate
from ci_orchestrator.commands.vm import vm
# Re-export so legacy A1 tests that patched ``cli_module.<name>`` keep working.
@@ -46,6 +50,10 @@ cli.add_command(report)
cli.add_command(retention)
cli.add_command(template)
cli.add_command(job)
cli.add_command(bench)
cli.add_command(smoke)
cli.add_command(validate)
cli.add_command(creds)
if __name__ == "__main__": # pragma: no cover
src/ci_orchestrator/commands/bench.py
+704
View File
@@ -0,0 +1,704 @@
"""``bench`` sub-commands.
Phase C ports the manual PowerShell benchmark/burn-in tooling to Python so
the Linux host no longer needs ``pwsh``:
* ``bench run`` — concurrency burn-in (replaces ``Test-CapacityBurnIn.ps1``
+ ``Start-BurnInTest*.ps1``).
* ``bench measure`` — phase timing (replaces ``Measure-CIBenchmark.ps1``).
Design rule: import ``backends.load_backend`` and the existing transports
only — never ``WorkstationVmrunBackend`` directly (keeps the Phase D ESXi
path open). No ``Start-Job``/``pwsh``/``$IsWindows``.
"""
from __future__ import annotations
import contextlib
import json
import shutil
import socket
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from dataclasses import asdict, dataclass, field
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING
import click
from ci_orchestrator.backends import load_backend
from ci_orchestrator.backends.errors import BackendError, BackendNotAvailable
from ci_orchestrator.backends.protocol import VmHandle
from ci_orchestrator.config import Config, load_config
if TYPE_CHECKING: # pragma: no cover
from ci_orchestrator.backends.protocol import VmBackend
# Clone directories left behind by a job are named ``Clone_<job_id>_...``
# (see commands/job.py:_make_clone_name and commands/vm.py:vm_new).
_CLONE_PREFIX = "Clone_"
# A vm-start.lock older than this (seconds) is considered stale leftover.
_STALE_LOCK_SECONDS = 5 * 60
# Linux guests probe SSH/22; everything else probes WinRM/5986.
_SSH_PORT = 22
_WINRM_PORT = 5986
def _now() -> float:
return time.monotonic()
def _timestamp() -> str:
return datetime.now(tz=UTC).strftime("%Y%m%d-%H%M%S")
# ─────────────────────────────────────────────────────────────────── group
@click.group("bench")
def bench() -> None:
"""Capacity burn-in and phase-timing benchmarks."""
# ═══════════════════════════════════════════════════════════════ bench run
@dataclass
class JobResult:
"""Outcome of a single burn-in job slot."""
job_id: str
slot: int
exit_code: int
status: str
elapsed_sec: float
error: str = ""
@dataclass
class RoundResult:
"""Outcome of one burn-in round (all slots + cleanup assertions)."""
round: int
passed: int
failed: int
status: str
elapsed_sec: float
orphans: list[str] = field(default_factory=list)
stale_lock: bool = False
jobs: list[JobResult] = field(default_factory=list)
def _run_one_job(
*,
job_id: str,
guest_os: str,
build_command: str,
clone_base_dir: Path,
) -> tuple[int, str]:
"""Run a single job end to end, returning ``(exit_code, error_message)``.
Invokes the in-process ``commands.job.job`` command (the same code path
production uses) rather than re-spawning ``python -m ci_orchestrator`` or
a PowerShell child — closes the ``Start-Job``/``$IsWindows`` shim bug.
Isolated in its own module-level function so the burn-in harness can be
unit tested by monkeypatching it (no real VMs needed).
"""
from click.testing import CliRunner # local import: drives the job command
from ci_orchestrator.commands.job import job
args = [
"--job-id",
job_id,
"--repo-url",
"",
"--branch",
"main",
"--guest-os",
guest_os,
"--clone-base-dir",
str(clone_base_dir),
"--skip-artifact",
]
if build_command:
args += ["--build-command", build_command]
result = CliRunner().invoke(job, args, catch_exceptions=True)
if result.exit_code == 0:
return 0, ""
tail = result.output.strip().splitlines()
return result.exit_code, tail[-1] if tail else ""
def _find_orphans(clone_base: Path, job_ids: list[str]) -> list[Path]:
"""Return clone dirs in ``clone_base`` belonging to the given jobs."""
if not clone_base.is_dir():
return []
prefixes = tuple(f"{_CLONE_PREFIX}{jid}_" for jid in job_ids)
return [
entry
for entry in clone_base.iterdir()
if entry.is_dir() and entry.name.startswith(prefixes)
]
def _stale_lock(lock_file: Path, max_age_seconds: int = _STALE_LOCK_SECONDS) -> bool:
"""True if ``lock_file`` exists and is older than ``max_age_seconds``."""
if not lock_file.is_file():
return False
try:
mtime = lock_file.stat().st_mtime
except OSError:
return False
return (time.time() - mtime) > max_age_seconds
def _run_round(
*,
round_no: int,
concurrency: int,
guest_os: str,
build_command: str,
clone_base_dir: Path,
lock_file: Path,
) -> RoundResult:
"""Launch ``concurrency`` concurrent jobs and assert cleanup invariants."""
click.echo(f"\n[bench run] Round {round_no}: launching {concurrency} job(s)...")
started = _now()
ts = _timestamp()
slots = {
slot: f"burnin-r{round_no}-j{slot}-{ts}"
for slot in range(1, concurrency + 1)
}
jobs: list[JobResult] = []
with ThreadPoolExecutor(max_workers=concurrency) as pool:
futures = {
pool.submit(
_run_one_job,
job_id=job_id,
guest_os=guest_os,
build_command=build_command,
clone_base_dir=clone_base_dir,
): (slot, job_id)
for slot, job_id in slots.items()
}
for fut in as_completed(futures):
slot, job_id = futures[fut]
try:
exit_code, error = fut.result()
except Exception as exc: # pragma: no cover - defensive
exit_code, error = 1, str(exc)
status = "PASS" if exit_code == 0 else "FAIL"
jobs.append(
JobResult(
job_id=job_id,
slot=slot,
exit_code=exit_code,
status=status,
elapsed_sec=round(_now() - started, 2),
error=error,
)
)
jobs.sort(key=lambda j: j.slot)
passed = sum(1 for j in jobs if j.status == "PASS")
failed = len(jobs) - passed
# ── Assertion: no orphaned clone directories ─────────────────────────
orphans = _find_orphans(clone_base_dir, list(slots.values()))
if orphans:
failed += 1
# ── Assertion: no stale vm-start.lock ────────────────────────────────
stale = _stale_lock(lock_file)
if stale:
failed += 1
status = "PASS" if failed == 0 else "FAIL"
result = RoundResult(
round=round_no,
passed=passed,
failed=failed,
status=status,
elapsed_sec=round(_now() - started, 2),
orphans=[str(p) for p in orphans],
stale_lock=stale,
jobs=jobs,
)
for j in jobs:
line = f" slot {j.slot} {j.job_id:<32} {j.status:<5} exit={j.exit_code}"
if j.error:
line += f" ({j.error})"
click.echo(line)
click.echo(
f" clone cleanup : {'OK' if not orphans else f'FAIL ({len(orphans)} orphan)'}"
)
for p in result.orphans:
click.echo(f" orphan: {p}")
click.echo(f" lock cleanup : {'OK' if not stale else 'FAIL (stale lock)'}")
click.echo(
f" [Round {round_no}] {status} "
f"({passed} PASS, {failed} FAIL, {result.elapsed_sec}s)"
)
return result
def _write_run_report(report: dict[str, object], json_out: Path) -> None:
json_out.parent.mkdir(parents=True, exist_ok=True)
json_out.write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
@bench.command("run")
@click.option(
"--guest-os",
type=click.Choice(["windows", "linux"], case_sensitive=False),
default="linux",
show_default=True,
help="Guest OS family of the build VMs under test.",
)
@click.option(
"--concurrency",
type=click.IntRange(1, 32),
default=4,
show_default=True,
help="Number of concurrent jobs per round.",
)
@click.option(
"--rounds",
type=click.IntRange(1, 1000),
default=10,
show_default=True,
help="Number of sequential rounds to run.",
)
@click.option(
"--build-command",
"build_command",
default="",
help="Build command each job runs (default: a no-op sleep/ping).",
)
@click.option(
"--clone-base-dir",
"clone_base_dir",
default=None,
help="Override clone base dir (defaults to config.paths.build_vms).",
)
@click.option(
"--json-out",
"json_out",
default=None,
help="Path for the JSON report (default: CI_ARTIFACTS/bench/<ts>.json).",
)
def bench_run(
guest_os: str,
concurrency: int,
rounds: int,
build_command: str,
clone_base_dir: str | None,
json_out: str | None,
) -> None:
"""Run a concurrency burn-in and assert cleanup invariants per round.
Replaces ``Test-CapacityBurnIn.ps1`` + ``Start-BurnInTest*.ps1``. For
each of ``--rounds`` rounds it launches ``--concurrency`` concurrent
jobs (in-process, via :func:`concurrent.futures`), then asserts: every
job exited 0, no orphaned clone dir remains in the clone base dir, and
no ``vm-start.lock`` older than 5 minutes is left behind.
"""
guest_os = guest_os.lower()
config = load_config()
base = Path(clone_base_dir) if clone_base_dir else config.paths.build_vms
lock_file = base / "vm-start.lock"
started_at = datetime.now(tz=UTC)
out_path = (
Path(json_out)
if json_out
else config.paths.artifacts / "bench" / f"{_timestamp()}.json"
)
click.echo("=" * 60)
click.echo("[bench run] capacity burn-in")
click.echo(f" guest-os : {guest_os}")
click.echo(f" concurrency : {concurrency}")
click.echo(f" rounds : {rounds}")
click.echo(f" clone base : {base}")
click.echo(f" build cmd : {build_command or '(default no-op)'}")
click.echo("=" * 60)
round_results: list[RoundResult] = []
for r in range(1, rounds + 1):
round_results.append(
_run_round(
round_no=r,
concurrency=concurrency,
guest_os=guest_os,
build_command=build_command,
clone_base_dir=base,
lock_file=lock_file,
)
)
total_pass = sum(rr.passed for rr in round_results)
total_fail = sum(rr.failed for rr in round_results)
rounds_failed = sum(1 for rr in round_results if rr.status == "FAIL")
overall = "PASS" if total_fail == 0 else "FAIL"
# ── Summary table ────────────────────────────────────────────────────
click.echo("\n" + "=" * 60)
click.echo("[bench run] SUMMARY")
click.echo(f" {'Round':<7}{'Pass':<6}{'Fail':<6}{'Elapsed':<10}Status")
click.echo(f" {'-' * 5:<7}{'-' * 4:<6}{'-' * 4:<6}{'-' * 8:<10}{'-' * 6}")
for rr in round_results:
click.echo(
f" {rr.round:<7}{rr.passed:<6}{rr.failed:<6}"
f"{str(rr.elapsed_sec) + 's':<10}{rr.status}"
)
click.echo("")
click.echo(f" Total jobs : {rounds * concurrency}")
click.echo(f" Passed : {total_pass}")
click.echo(f" Failed : {total_fail}")
click.echo(f" Rounds OK : {rounds - rounds_failed} / {rounds}")
click.echo(f"\n OVERALL: {overall}")
click.echo("=" * 60)
report: dict[str, object] = {
"ts": started_at.isoformat(),
"guestOS": guest_os,
"concurrency": concurrency,
"rounds": rounds,
"buildCommand": build_command,
"totalJobs": rounds * concurrency,
"totalPass": total_pass,
"totalFail": total_fail,
"roundsFailed": rounds_failed,
"overall": overall,
"results": [asdict(rr) for rr in round_results],
}
try:
_write_run_report(report, out_path)
click.echo(f"[bench run] report written: {out_path}")
except OSError as exc:
click.echo(f"[bench run] could not write report: {exc}", err=True)
if total_fail > 0:
raise SystemExit(1)
# ═══════════════════════════════════════════════════════════ bench measure
@dataclass
class PhaseTiming:
"""Per-iteration phase timings, mirroring ``Measure-CIBenchmark.ps1``.
Field names of the serialised record (see :meth:`to_record`) are kept
identical to the PowerShell version for trend-log continuity.
"""
iteration: int
run_id: str
template: str
snapshot: str
guest_os: str
clone_sec: float | None = None
start_sec: float | None = None
ip_sec: float | None = None
ready_sec: float | None = None
destroy_sec: float | None = None
delta_kb: int | None = None
vm_ip: str | None = None
error: str | None = None
@property
def total_boot_sec(self) -> float | None:
if self.ip_sec is None or self.ready_sec is None:
return None
return round(
(self.clone_sec or 0.0)
+ (self.start_sec or 0.0)
+ self.ip_sec
+ self.ready_sec,
2,
)
def to_record(self, ts: str) -> dict[str, object]:
"""Build a JSON record with the legacy ``Measure-CIBenchmark`` keys."""
return {
"ts": ts,
"runId": self.run_id,
"iteration": self.iteration,
"template": self.template,
"snapshot": self.snapshot,
"guestOS": self.guest_os,
"cloneSec": self.clone_sec,
"startSec": self.start_sec,
"ipSec": self.ip_sec,
"readySec": self.ready_sec,
"destroySec": self.destroy_sec,
"totalBootSec": self.total_boot_sec,
"deltaKB": self.delta_kb,
"vmIP": self.vm_ip,
"error": self.error,
}
def _dir_size_kb(path: Path) -> int | None:
"""Total size of files under ``path`` in KiB, or None on error."""
try:
total = sum(p.stat().st_size for p in path.rglob("*") if p.is_file())
except OSError:
return None
return round(total / 1024)
def _tcp_open(host: str, port: int, timeout: float = 3.0) -> bool:
"""Return True if a TCP connection to ``host:port`` succeeds."""
try:
with socket.create_connection((host, port), timeout=timeout):
return True
except OSError:
return False
def _wait_port(host: str, port: int, deadline: float, poll: float = 3.0) -> bool:
while _now() < deadline:
if _tcp_open(host, port):
return True
time.sleep(poll)
return False
def _measure_iteration(
*,
backend: VmBackend,
iteration: int,
template: Path,
snapshot: str,
guest_os: str,
clone_base_dir: Path,
transport_port: int,
timeout_seconds: float,
) -> PhaseTiming:
"""Clone → start → IP → transport-ready → destroy, timing each phase."""
run_id = f"bench-{_timestamp()}-{iteration}"
clone_name = f"{_CLONE_PREFIX}{run_id}"
clone_dir = clone_base_dir / clone_name
clone_vmx = clone_dir / f"{clone_name}.vmx"
timing = PhaseTiming(
iteration=iteration,
run_id=run_id,
template=template.name,
snapshot=snapshot,
guest_os=guest_os,
)
handle: VmHandle | None = None
try:
# ── clone ────────────────────────────────────────────────────────
click.echo(f"[bench measure] iteration {iteration}: cloning...")
t0 = _now()
handle = backend.clone_linked(
template=str(template),
snapshot=snapshot,
name=clone_name,
destination=str(clone_vmx),
)
timing.clone_sec = round(_now() - t0, 2)
timing.delta_kb = _dir_size_kb(clone_dir)
# ── start ────────────────────────────────────────────────────────
click.echo("[bench measure] starting...")
t0 = _now()
backend.start(handle, headless=True)
timing.start_sec = round(_now() - t0, 2)
# ── IP acquire ────────────────────────────────────────────────────
click.echo("[bench measure] waiting for guest IP...")
t0 = _now()
deadline = t0 + timeout_seconds
vm_ip: str | None = None
while _now() < deadline:
vm_ip = backend.get_ip(handle, timeout=min(5.0, deadline - _now()))
if vm_ip:
break
if not vm_ip:
raise TimeoutError(f"no guest IP after {timeout_seconds}s")
timing.ip_sec = round(_now() - t0, 2)
timing.vm_ip = vm_ip
# ── transport ready ───────────────────────────────────────────────
click.echo(f"[bench measure] waiting for port {transport_port}...")
t0 = _now()
if not _wait_port(vm_ip, transport_port, _now() + timeout_seconds):
raise TimeoutError(
f"port {transport_port} on {vm_ip} not open after {timeout_seconds}s"
)
timing.ready_sec = round(_now() - t0, 2)
except (BackendError, TimeoutError, OSError) as exc:
timing.error = str(exc)
click.echo(f"[bench measure] iteration {iteration} failed: {exc}", err=True)
finally:
# ── destroy ───────────────────────────────────────────────────────
t0 = _now()
if handle is not None:
with contextlib.suppress(BackendError):
backend.stop(handle, hard=True)
with contextlib.suppress(BackendError):
backend.delete(handle)
if clone_dir.exists():
shutil.rmtree(clone_dir, ignore_errors=True)
timing.destroy_sec = round(_now() - t0, 2)
return timing
def _resolve_template(config: Config, guest_os: str) -> tuple[Path, str]:
"""Pick a default template VMX + snapshot for ``guest_os``."""
if guest_os == "linux":
return (
config.paths.templates / "LinuxBuild2404" / "LinuxBuild2404.vmx",
"BaseClean-Linux",
)
return (
config.paths.templates / "WinBuild2025" / "WinBuild2025.vmx",
"BaseClean",
)
@bench.command("measure")
@click.option(
"--guest-os",
type=click.Choice(["windows", "linux"], case_sensitive=False),
default="linux",
show_default=True,
help="Guest OS family of the VM to benchmark.",
)
@click.option(
"--iterations",
type=click.IntRange(1, 100),
default=1,
show_default=True,
help="Number of ephemeral VMs to time.",
)
@click.option(
"--template-path",
"template_path",
default=None,
help="Template VMX path (defaults to the per-guest-os template).",
)
@click.option(
"--snapshot-name",
"snapshot_name",
default=None,
help="Snapshot to clone from (defaults to BaseClean / BaseClean-Linux).",
)
@click.option(
"--clone-base-dir",
"clone_base_dir",
default=None,
help="Override clone base dir (defaults to config.paths.build_vms).",
)
@click.option(
"--timeout-seconds",
"timeout_seconds",
type=click.FloatRange(30.0, 1200.0),
default=300.0,
show_default=True,
help="Per-iteration timeout for IP and transport readiness.",
)
@click.option(
"--json-out",
"json_out",
default=None,
help="Path to append JSONL records (default: CI_ARTIFACTS/benchmark.jsonl).",
)
def bench_measure(
guest_os: str,
iterations: int,
template_path: str | None,
snapshot_name: str | None,
clone_base_dir: str | None,
timeout_seconds: float,
json_out: str | None,
) -> None:
"""Measure clone/start/IP/transport/destroy phase timings.
Replaces ``Measure-CIBenchmark.ps1``. Records are appended to
``benchmark.jsonl`` with the same field names so historical trend data
stays comparable.
"""
guest_os = guest_os.lower()
config = load_config()
default_tpl, default_snap = _resolve_template(config, guest_os)
template = Path(template_path) if template_path else default_tpl
snapshot = snapshot_name or default_snap
base = Path(clone_base_dir) if clone_base_dir else config.paths.build_vms
transport_port = _SSH_PORT if guest_os == "linux" else _WINRM_PORT
out_path = Path(json_out) if json_out else config.paths.artifacts / "benchmark.jsonl"
if not template.is_file():
raise click.ClickException(f"Template VMX not found: {template}")
try:
backend = load_backend(config)
except BackendNotAvailable as exc:
raise click.ClickException(f"backend unavailable: {exc}") from exc
base.mkdir(parents=True, exist_ok=True)
out_path.parent.mkdir(parents=True, exist_ok=True)
click.echo("=" * 60)
click.echo("[bench measure] phase timing")
click.echo(f" template : {template.name}")
click.echo(f" snapshot : {snapshot}")
click.echo(f" guest-os : {guest_os} (port {transport_port})")
click.echo(f" iterations : {iterations}")
click.echo("=" * 60)
timings: list[PhaseTiming] = []
for i in range(1, iterations + 1):
timing = _measure_iteration(
backend=backend,
iteration=i,
template=template,
snapshot=snapshot,
guest_os=guest_os,
clone_base_dir=base,
transport_port=transport_port,
timeout_seconds=timeout_seconds,
)
timings.append(timing)
record = timing.to_record(datetime.now(tz=UTC).isoformat())
try:
with out_path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(record) + "\n")
except OSError as exc:
click.echo(f"[bench measure] could not append jsonl: {exc}", err=True)
# ── Summary table ────────────────────────────────────────────────────
click.echo("\n[bench measure] Results")
header = f" {'Iter':<5}{'Clone':<8}{'Start':<8}{'IP':<8}{'Ready':<8}{'Destroy':<9}{'Boot':<8}IP"
click.echo(header)
for t in timings:
click.echo(
f" {t.iteration:<5}"
f"{t.clone_sec!s:<8}{t.start_sec!s:<8}{t.ip_sec!s:<8}"
f"{t.ready_sec!s:<8}{t.destroy_sec!s:<9}"
f"{t.total_boot_sec!s:<8}{t.vm_ip or '-'}"
)
if t.error:
click.echo(f" error: {t.error}")
successful = [t for t in timings if t.error is None and t.total_boot_sec is not None]
if len(successful) > 1:
boots = [t.total_boot_sec for t in successful if t.total_boot_sec is not None]
avg = round(sum(boots) / len(boots), 2)
click.echo(f"\n Average boot-to-ready ({len(successful)} ok): {avg}s")
click.echo(f"\n[bench measure] records appended to: {out_path}")
__all__ = ["bench"]
src/ci_orchestrator/commands/creds.py
+87
View File
@@ -0,0 +1,87 @@
"""``creds`` sub-commands.
Phase C replaces ``Set-CIGuestCredential.ps1`` on the Linux host with a
keyring-native writer (``keyrings.alt.file.PlaintextKeyring``), matching the
read path in :mod:`ci_orchestrator.credentials`.
* ``creds set`` — store a guest username/password under a target. Passwords
are never passed on the command line.
Write scheme (must mirror :meth:`KeyringCredentialStore.get`):
* ``keyring.set_password("{target}:meta", "username", <user>)`` — so the
reader can recover the username for file backends where
``get_credential(target, None)`` is a no-op.
* ``keyring.set_password(target, <user>, <password>)`` — the password,
keyed by the username under the target service.
"""
from __future__ import annotations
import sys
import click
def _store_credential(target: str, user: str, password: str) -> None:
"""Write the two keyring entries the credential reader expects.
Raises :class:`RuntimeError` if ``keyring`` is not installed.
"""
try:
import keyring
except ImportError as exc: # pragma: no cover - dep declared in pyproject
raise RuntimeError(
"keyring is not installed; run `pip install keyring`."
) from exc
# 1. Username under the "{target}:meta" service, key "username".
keyring.set_password(f"{target}:meta", "username", user)
# 2. Password under the target service, keyed by the username.
keyring.set_password(target, user, password)
@click.group("creds")
def creds() -> None:
"""Manage guest credentials in the keyring."""
@creds.command("set")
@click.option(
"--target",
default="BuildVMGuest",
show_default=True,
help="Credential target name (matches config.guest_cred_target).",
)
@click.option("--user", "user", required=True, help="Guest username to store.")
@click.option(
"--password-stdin",
"password_stdin",
is_flag=True,
default=False,
help="Read the password from stdin instead of prompting.",
)
def creds_set(target: str, user: str, password_stdin: bool) -> None:
"""Store guest credentials so ci_orchestrator can read them back."""
if password_stdin:
# Read the whole of stdin and strip a single trailing newline only,
# so passwords containing internal whitespace survive intact.
password = sys.stdin.readline().rstrip("\n").rstrip("\r")
else:
password = click.prompt(
"Guest password",
hide_input=True,
confirmation_prompt=True,
)
if not password:
raise click.UsageError("password must not be empty.")
try:
_store_credential(target, user, password)
except RuntimeError as exc:
raise click.ClickException(str(exc)) from exc
click.echo(f"Stored credential for target {target!r} (user={user}).")
__all__ = ["creds"]
src/ci_orchestrator/commands/smoke.py
+351
View File
@@ -0,0 +1,351 @@
"""``smoke`` sub-commands.
Phase C4 ports the manual smoke / E2E build tooling to Python so the Linux
host no longer needs ``pwsh``:
* ``smoke run`` — end-to-end no-op (or real build) job, replacing
``Test-Smoke.ps1`` and ``Test-*Build-Linux.ps1``.
The command runs **one** full end-to-end job through the in-process
``job`` orchestrator (reusing ``commands/job.py`` — no re-spawn of
``python -m ci_orchestrator``) and then asserts the three smoke
invariants of ``Test-Smoke.ps1``:
1. the pipeline exited 0,
2. the per-job artifact directory was created, and
3. a ``job``/``success`` event is present in the ``invoke-ci.jsonl``
event log (the same JSONL format ``report job`` consumes).
Because the in-process ``job`` command does not itself emit the JSONL
event log (that was the job of the retired ``Invoke-CIJob.ps1``), this
command writes the ``job``/``start`` and ``job``/``success`` events
around the orchestration, exactly as the PowerShell wrapper did.
Design rule: import ``backends.load_backend`` and the existing transports
only — never ``WorkstationVmrunBackend`` directly (keeps the Phase D ESXi
path open).
"""
from __future__ import annotations
import json
import time
from datetime import UTC, datetime
from pathlib import Path
import click
from ci_orchestrator.commands.job import job as job_command
from ci_orchestrator.config import Config, load_config
# ─────────────────────────────────────────────────────── OS presets
# Per-guest defaults mirroring ``Test-Smoke.ps1``: a trivial build that
# writes a marker file into the standard output directory, plus the
# matching template / snapshot / artifact-source for the OS family.
_LINUX_TEMPLATE = "/var/lib/ci/templates/LinuxBuild2404/LinuxBuild2404.vmx"
_WINDOWS_TEMPLATE = r"F:\CI\Templates\WinBuild2025\WinBuild2025.vmx"
_NOOP_BUILD_LINUX = "mkdir -p /opt/ci/output && echo smoke-ok > /opt/ci/output/smoke.txt"
_NOOP_BUILD_WINDOWS = (
"New-Item -ItemType Directory -Path C:\\CI\\output -Force | Out-Null; "
'Set-Content C:\\CI\\output\\smoke.txt "smoke-ok"'
)
# Real Linux build E2E presets (ports of ``Test-*Build-Linux.ps1``).
# Exposed via ``--preset`` so the no-op default is never overridden by a
# hard-coded ``F:\`` Windows path. Each preset supplies a build command
# and the in-guest artifact source directory.
_PRESETS: dict[str, tuple[str, str]] = {
"ns7zip": (
"python3 build_plugin.py --host linux --7zip-version 26.01 "
"--configs x64-unicode --verbose",
"plugins",
),
"nsinnounp": (
"python3 build_plugin.py --final --dist-dir dist",
"dist",
),
}
def _default_template(guest_os: str) -> str:
return _LINUX_TEMPLATE if guest_os == "linux" else _WINDOWS_TEMPLATE
def _default_snapshot(guest_os: str) -> str:
return "BaseClean-Linux" if guest_os == "linux" else "BaseClean"
def _noop_build_command(guest_os: str) -> tuple[str, str]:
"""Return ``(build_command, guest_artifact_source)`` for the marker job."""
if guest_os == "linux":
return _NOOP_BUILD_LINUX, "/opt/ci/output"
return _NOOP_BUILD_WINDOWS, "C:\\CI\\output"
def _utcnow_iso() -> str:
return datetime.now(tz=UTC).isoformat()
def _write_event(
jsonl_path: Path,
*,
job_id: str,
phase: str,
status: str,
elapsed_sec: float | None = None,
error: str | None = None,
) -> None:
"""Append one event to ``invoke-ci.jsonl`` in ``report job`` format."""
data: dict[str, object] = {}
if elapsed_sec is not None:
data["elapsedSec"] = int(elapsed_sec)
if error is not None:
data["error"] = error
record: dict[str, object] = {
"jobId": job_id,
"phase": phase,
"status": status,
"ts": _utcnow_iso(),
"data": data,
}
jsonl_path.parent.mkdir(parents=True, exist_ok=True)
with jsonl_path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(record) + "\n")
def _has_job_success(jsonl_path: Path) -> bool:
"""True iff the JSONL log contains a ``job``/``success`` event."""
try:
text = jsonl_path.read_text(encoding="utf-8")
except OSError:
return False
for line in text.splitlines():
line = line.strip()
if not line:
continue
try:
obj = json.loads(line)
except json.JSONDecodeError:
continue
if not isinstance(obj, dict):
continue
if obj.get("phase") == "job" and obj.get("status") == "success":
return True
return False
# ────────────────────────────────────────────────────────── command
@click.group("smoke")
def smoke() -> None:
"""End-to-end smoke and build validation."""
@smoke.command("run")
@click.option(
"--guest-os",
type=click.Choice(["windows", "linux"], case_sensitive=False),
default="linux",
show_default=True,
help="Guest OS family to smoke-test.",
)
@click.option(
"--build-command",
"build_command",
default="",
help="Build command to run (default: a marker no-op job).",
)
@click.option(
"--preset",
type=click.Choice(sorted(_PRESETS), case_sensitive=False),
default=None,
help="Real Linux build E2E preset (ns7zip / nsinnounp). Linux only.",
)
@click.option(
"--repo-url",
"repo_url",
default="https://example.invalid/smoke-test.git",
show_default=True,
help="Repository to build (default: a placeholder; the no-op marker "
"job never depends on the clone).",
)
@click.option("--branch", default="main", show_default=True)
@click.option(
"--job-id",
"job_id",
default=None,
help="Job id (default: smoke-<timestamp>).",
)
@click.option(
"--template-path",
"template_path",
default=None,
help="Template VMX path (default: per-guest-os standard template).",
)
@click.option(
"--snapshot-name",
"snapshot_name",
default=None,
help="Snapshot to clone from (default: per-guest-os BaseClean[-Linux]).",
)
@click.option(
"--ssh-key-path",
"ssh_key_path",
default=None,
help="SSH private key for the guest (Linux).",
)
@click.option(
"--log-dir",
"log_dir",
default=None,
help="Base directory for invoke-ci.jsonl logs (default: CI_ARTIFACTS).",
)
def smoke_run(
guest_os: str,
build_command: str,
preset: str | None,
repo_url: str,
branch: str,
job_id: str | None,
template_path: str | None,
snapshot_name: str | None,
ssh_key_path: str | None,
log_dir: str | None,
) -> None:
"""Run one end-to-end job and assert success + artifact + event."""
guest_os = guest_os.lower()
if preset is not None:
preset = preset.lower()
if guest_os != "linux":
raise click.UsageError("--preset is only supported with --guest-os linux.")
if build_command:
raise click.UsageError("--preset and --build-command are mutually exclusive.")
preset_cmd, artifact_source = _PRESETS[preset]
effective_build_command = preset_cmd
elif build_command:
effective_build_command = build_command
# A custom command writes to the standard output dir for the OS.
_, artifact_source = _noop_build_command(guest_os)
else:
effective_build_command, artifact_source = _noop_build_command(guest_os)
config: Config = load_config()
resolved_job_id = job_id or f"smoke-{datetime.now(tz=UTC).strftime('%Y%m%d-%H%M%S')}"
resolved_template = template_path or _default_template(guest_os)
resolved_snapshot = snapshot_name or _default_snapshot(guest_os)
artifact_base: Path = config.paths.artifacts
# Logs live alongside (not inside) the artifacts tree so that writing the
# event log never accidentally creates the per-job artifact directory and
# masks a build that produced no artifacts (mirrors F:\CI\Logs vs
# F:\CI\Artifacts in Test-Smoke.ps1).
log_base = Path(log_dir) if log_dir else config.paths.root / "logs"
job_artifact_dir = artifact_base / resolved_job_id
jsonl_path = log_base / resolved_job_id / "invoke-ci.jsonl"
click.echo("=" * 60)
click.echo(f"[smoke] Guest OS : {guest_os}")
click.echo(f"[smoke] Template : {resolved_template}")
click.echo(f"[smoke] Snapshot : {resolved_snapshot}")
click.echo(f"[smoke] Job id : {resolved_job_id}")
if preset is not None:
click.echo(f"[smoke] Preset : {preset}")
click.echo(f"[smoke] Artifacts : {job_artifact_dir}")
click.echo("=" * 60)
_write_event(
jsonl_path, job_id=resolved_job_id, phase="job", status="start"
)
start = time.monotonic()
failed = False
try:
job_command.callback( # type: ignore[misc]
job_id=resolved_job_id,
repo_url=repo_url,
branch=branch,
commit="",
configuration="Release",
build_command=effective_build_command,
guest_artifact_source=artifact_source,
submodules=True,
use_git_clone=True,
use_shared_cache=False,
skip_artifact=False,
use_xvfb=False,
gitea_credential_target="GiteaPAT",
template_path=resolved_template,
snapshot_name=resolved_snapshot,
clone_base_dir=None,
artifact_base_dir=str(artifact_base),
guest_os=guest_os,
guest_credential_target=None,
ssh_key_path=ssh_key_path,
ssh_user="ci_build",
ssh_known_hosts_file=None,
extra_env_json="",
guest_cpu=0,
guest_memory_mb=0,
ready_timeout=600.0,
poll_interval=5.0,
vmrun_path=None,
)
except click.ClickException as exc:
failed = True
_write_event(
jsonl_path,
job_id=resolved_job_id,
phase="job",
status="failure",
elapsed_sec=time.monotonic() - start,
error=exc.format_message(),
)
click.echo(f"[smoke] FAIL: pipeline raised: {exc.format_message()}", err=True)
if not failed:
_write_event(
jsonl_path,
job_id=resolved_job_id,
phase="job",
status="success",
elapsed_sec=time.monotonic() - start,
)
# ── Assert the three smoke invariants ───────────────────────────────
click.echo("\n[smoke] Verifying results...")
checks_ok = True
if failed:
checks_ok = False
else:
click.echo("[smoke] OK: pipeline exit 0")
if job_artifact_dir.is_dir():
click.echo(f"[smoke] OK: artifact dir exists: {job_artifact_dir}")
else:
click.echo(
f"[smoke] FAIL: artifact dir not found: {job_artifact_dir}", err=True
)
checks_ok = False
if _has_job_success(jsonl_path):
click.echo("[smoke] OK: JSONL contains job/success event")
else:
click.echo(
f"[smoke] FAIL: JSONL missing job/success event in: {jsonl_path}",
err=True,
)
checks_ok = False
click.echo("")
if checks_ok:
click.echo(f"[smoke] PASSED — {guest_os} smoke test completed successfully.")
return
raise click.ClickException("smoke test FAILED — one or more checks did not pass.")
__all__ = ["smoke"]
src/ci_orchestrator/commands/validate.py
+285
View File
@@ -0,0 +1,285 @@
"""``validate`` sub-commands.
Phase C adds Linux-native host/guest validation, replacing host-side use of
``Validate-HostState.ps1`` and the diagnostic part of ``Test-CIGuestWinRM.ps1``:
* ``validate host`` — vmrun present, template snapshots present, keyring
readable, ``/var/lib/ci`` permissions, systemd units active.
* ``validate guest`` — connect to a guest over WinRM/SSH and run a trivial
command, reporting the explicit failure cause.
The host checks are a Linux-native equivalent of ``Validate-HostState.ps1``
(not a 1:1 port): no Windows drive paths, no Windows Credential Manager, no
NTFS ACLs. The guest check is the diagnostic core of ``Test-CIGuestWinRM.ps1``:
unlike the ``is_ready`` probe (which swallows errors and returns a bool),
this reports the explicit failure cause.
"""
from __future__ import annotations
import os
import shutil
import subprocess
from dataclasses import dataclass
import click
from ci_orchestrator.backends import load_backend
from ci_orchestrator.backends.errors import BackendError, BackendNotAvailable
from ci_orchestrator.backends.protocol import VmHandle
from ci_orchestrator.config import Config, load_config
from ci_orchestrator.credentials import KeyringCredentialStore
from ci_orchestrator.transport.errors import TransportCommandError, TransportConnectError
from ci_orchestrator.transport.ssh import SshTransport
from ci_orchestrator.transport.winrm import WinRmTransport
# Each template directory and the snapshot a clone is taken from. Mirrors the
# "VM templates and snapshots" table in CLAUDE.md.
_TEMPLATE_SNAPSHOTS: tuple[tuple[str, str], ...] = (
("WinBuild2025", "BaseClean"),
("WinBuild2022", "BaseClean"),
("LinuxBuild2404", "BaseClean-Linux"),
)
# systemd units expected to be active on a configured Linux host.
_EXPECTED_UNITS: tuple[str, ...] = ("act-runner.service",)
@dataclass(frozen=True)
class CheckResult:
"""Outcome of a single host check."""
name: str
ok: bool
detail: str
@click.group("validate")
def validate() -> None:
"""Host and guest readiness checks."""
# ──────────────────────────────────────────────────────── validate host
def _check_vmrun(config: Config) -> CheckResult:
"""vmrun is present and executable (via the backend factory)."""
try:
backend = load_backend(config)
except BackendNotAvailable as exc:
return CheckResult("vmrun", False, str(exc))
except BackendError as exc: # pragma: no cover - defensive
return CheckResult("vmrun", False, str(exc))
vmrun_path = getattr(backend, "vmrun_path", None)
if vmrun_path:
return CheckResult("vmrun", True, f"found at {vmrun_path}")
return CheckResult("vmrun", True, "backend loaded")
def _check_snapshots(config: Config) -> CheckResult:
"""Each known template VMX exists and exposes its expected snapshot."""
try:
backend = load_backend(config)
except BackendError as exc:
return CheckResult("snapshots", False, f"backend unavailable: {exc}")
templates_dir = config.paths.templates
problems: list[str] = []
found: list[str] = []
for template_name, snapshot in _TEMPLATE_SNAPSHOTS:
vmx = templates_dir / template_name / f"{template_name}.vmx"
if not vmx.is_file():
# A host may run only a subset of templates; skip absent ones
# rather than failing on them.
continue
try:
snapshots = backend.list_snapshots(VmHandle(identifier=str(vmx)))
except BackendError as exc:
problems.append(f"{template_name}: listSnapshots failed ({exc})")
continue
if snapshot in snapshots:
found.append(f"{template_name}->{snapshot}")
else:
problems.append(
f"{template_name}: snapshot {snapshot!r} absent "
f"(have: {', '.join(snapshots) or 'none'})"
)
if problems:
return CheckResult("snapshots", False, "; ".join(problems))
if not found:
return CheckResult(
"snapshots", False, f"no known template VMX found under {templates_dir}"
)
return CheckResult("snapshots", True, ", ".join(found))
def _check_keyring(config: Config) -> CheckResult:
"""The file-based keyring vault is readable for the guest cred target."""
target = config.guest_cred_target
try:
cred = KeyringCredentialStore().get(target)
except KeyError:
return CheckResult(
"keyring", False, f"credential {target!r} not found in keyring"
)
except RuntimeError as exc:
return CheckResult("keyring", False, str(exc))
return CheckResult("keyring", True, f"{target!r} readable (user={cred.username})")
def _check_permissions(config: Config) -> CheckResult:
"""``$CI_ROOT`` and key sub-dirs exist and are readable+writable."""
root = config.paths.root
to_check = [root, config.paths.build_vms, config.paths.artifacts]
problems: list[str] = []
for path in to_check:
if not path.exists():
problems.append(f"{path}: missing")
continue
if not os.access(path, os.R_OK | os.W_OK):
problems.append(f"{path}: not read/write for current user")
if problems:
return CheckResult("permissions", False, "; ".join(problems))
return CheckResult("permissions", True, f"{root} read/write")
def _systemctl_is_active(unit: str) -> str | None:
"""Return ``systemctl is-active`` state, or ``None`` if unavailable."""
systemctl = shutil.which("systemctl")
if systemctl is None:
return None
try:
result = subprocess.run(
[systemctl, "is-active", unit],
capture_output=True,
text=True,
check=False,
timeout=10.0,
)
except (subprocess.TimeoutExpired, OSError):
return None
return result.stdout.strip().lower() or None
def _check_units() -> CheckResult:
"""Expected systemd units report ``active``."""
systemctl = shutil.which("systemctl")
if systemctl is None:
return CheckResult(
"systemd", False, "systemctl not found (not a systemd host?)"
)
inactive: list[str] = []
active: list[str] = []
for unit in _EXPECTED_UNITS:
state = _systemctl_is_active(unit)
if state == "active":
active.append(unit)
else:
inactive.append(f"{unit}={state or 'unknown'}")
if inactive:
return CheckResult("systemd", False, "; ".join(inactive))
return CheckResult("systemd", True, ", ".join(active))
def run_host_checks(config: Config) -> list[CheckResult]:
"""Run every host check and return the per-check results."""
return [
_check_vmrun(config),
_check_snapshots(config),
_check_keyring(config),
_check_permissions(config),
_check_units(),
]
@validate.command("host")
def validate_host() -> None:
"""Check host prerequisites; exit non-zero on any failure."""
config = load_config()
results = run_host_checks(config)
failed = [r for r in results if not r.ok]
for result in results:
marker = "OK " if result.ok else "FAIL"
click.echo(f"[{marker}] {result.name}: {result.detail}")
if failed:
click.echo(f"\n{len(failed)} check(s) failed.")
raise SystemExit(1)
click.echo("\nAll host checks passed.")
# ──────────────────────────────────────────────────────── validate guest
def _resolve_use_winrm(use_winrm: bool | None, guest_os: str | None) -> bool:
"""Decide the transport: explicit flag wins, else infer from guest OS."""
if use_winrm is not None:
return use_winrm
if guest_os is not None:
return guest_os.lower() != "linux"
raise click.UsageError(
"specify the transport: pass --winrm/--ssh or --guest-os."
)
def _probe_winrm(host: str, config: Config) -> CheckResult:
"""Connect over WinRM and run a trivial command, reporting the cause."""
try:
cred = KeyringCredentialStore().get(config.guest_cred_target)
except (KeyError, RuntimeError) as exc:
return CheckResult("winrm", False, f"credential lookup failed: {exc}")
transport = WinRmTransport(host, cred.username, cred.password)
try:
result = transport.run("$true | Out-Null", check=True)
except TransportConnectError as exc:
return CheckResult("winrm", False, f"connect/auth failed: {exc}")
except TransportCommandError as exc:
return CheckResult("winrm", False, f"command failed: {exc}")
finally:
transport.close()
return CheckResult("winrm", True, f"connected; exit={result.returncode}")
def _probe_ssh(host: str, config: Config) -> CheckResult:
"""Connect over SSH and run a trivial command, reporting the cause."""
key_path = str(config.ssh_key_path) if config.ssh_key_path else None
transport = SshTransport(host, key_path=key_path)
try:
result = transport.run("true", check=True, timeout=10.0)
except TransportConnectError as exc:
return CheckResult("ssh", False, f"connect/auth failed: {exc}")
except TransportCommandError as exc:
return CheckResult("ssh", False, f"command failed: {exc}")
finally:
transport.close()
return CheckResult("ssh", True, f"connected; exit={result.returncode}")
@validate.command("guest")
@click.option("--host", "host", required=True, help="Guest IP or hostname.")
@click.option(
"--winrm/--ssh",
"use_winrm",
default=None,
help="Force transport (default: inferred from --guest-os).",
)
@click.option(
"--guest-os",
type=click.Choice(["windows", "linux"], case_sensitive=False),
default=None,
help="Guest OS family (used to pick transport when --winrm/--ssh omitted).",
)
def validate_guest(
host: str,
use_winrm: bool | None,
guest_os: str | None,
) -> None:
"""Probe a guest's transport and report the explicit failure cause."""
config = load_config()
winrm = _resolve_use_winrm(use_winrm, guest_os)
result = _probe_winrm(host, config) if winrm else _probe_ssh(host, config)
marker = "OK " if result.ok else "FAIL"
click.echo(f"[{marker}] {result.name} {host}: {result.detail}")
if not result.ok:
raise SystemExit(1)
__all__ = ["validate"]
tests/_Common.Tests.ps1
-181
View File
@@ -1,181 +0,0 @@
#Requires -Version 5.1
<#
.SYNOPSIS
Pester v5 tests for scripts/_Common.psm1
#>
BeforeAll {
Import-Module (Join-Path $PSScriptRoot '..\scripts\_Common.psm1') -Force
# Fake vmrun: accepts any args, exits with %FAKE_VMRUN_EXIT% (default 0)
$script:FakeVmrun = Join-Path $env:TEMP "fake-vmrun-$PID.cmd"
Set-Content $script:FakeVmrun -Value @'
@echo off
echo fake-vmrun-output
if "%FAKE_VMRUN_EXIT%"=="" ( exit /b 0 ) else ( exit /b %FAKE_VMRUN_EXIT% )
'@
}
AfterAll {
Remove-Item $script:FakeVmrun -Force -ErrorAction SilentlyContinue
}
# ─────────────────────────────────────────────────────────────────────────────
Describe 'New-CISessionOption' {
It 'returns a PSSessionOption' {
$opt = New-CISessionOption
$opt | Should -BeOfType [System.Management.Automation.Remoting.PSSessionOption]
}
It 'has SkipCACheck = true' {
(New-CISessionOption).SkipCACheck | Should -Be $true
}
It 'has SkipCNCheck = true' {
(New-CISessionOption).SkipCNCheck | Should -Be $true
}
It 'has SkipRevocationCheck = true' {
(New-CISessionOption).SkipRevocationCheck | Should -Be $true
}
}
# ─────────────────────────────────────────────────────────────────────────────
Describe 'Resolve-VmrunPath' {
It 'returns the path when the file exists' {
$tmp = [System.IO.Path]::GetTempFileName()
try {
Resolve-VmrunPath -VmrunPath $tmp | Should -Be $tmp
} finally {
Remove-Item $tmp -Force -ErrorAction SilentlyContinue
}
}
It 'throws with a descriptive message when file is missing' {
{ Resolve-VmrunPath -VmrunPath 'C:\DoesNotExist\vmrun.exe' } |
Should -Throw -ExpectedMessage '*vmrun.exe not found*'
}
}
# ─────────────────────────────────────────────────────────────────────────────
Describe 'Invoke-Vmrun' {
It 'returns ExitCode 0 and Output when command succeeds' {
$env:FAKE_VMRUN_EXIT = '0'
$r = Invoke-Vmrun -VmrunPath $script:FakeVmrun -Operation 'list'
$r.ExitCode | Should -Be 0
"$($r.Output)" | Should -Match 'fake-vmrun-output'
}
It 'returns non-zero ExitCode without throwing by default' {
$env:FAKE_VMRUN_EXIT = '1'
$r = Invoke-Vmrun -VmrunPath $script:FakeVmrun -Operation 'clone'
$r.ExitCode | Should -Be 1
}
It 'throws when -ThrowOnError and ExitCode is non-zero' {
$env:FAKE_VMRUN_EXIT = '1'
{ Invoke-Vmrun -VmrunPath $script:FakeVmrun -Operation 'clone' -ThrowOnError } |
Should -Throw -ExpectedMessage '*clone failed*'
}
It 'does not throw when -ThrowOnError and ExitCode is 0' {
$env:FAKE_VMRUN_EXIT = '0'
{ Invoke-Vmrun -VmrunPath $script:FakeVmrun -Operation 'start' -ThrowOnError } |
Should -Not -Throw
}
AfterEach {
$env:FAKE_VMRUN_EXIT = $null
}
}
# ─────────────────────────────────────────────────────────────────────────────
Describe 'Invoke-VmrunBounded' {
BeforeAll {
# Fake vmrun that sleeps for $FAKE_VMRUN_SLEEP seconds then exits with
# $FAKE_VMRUN_EXIT. Both default to 0 if unset.
$script:SlowFakeVmrun = Join-Path $env:TEMP "fake-vmrun-slow-$PID.cmd"
Set-Content $script:SlowFakeVmrun -Value @'
@echo off
if "%FAKE_VMRUN_SLEEP%"=="" goto :nosleep
ping -n %FAKE_VMRUN_SLEEP% 127.0.0.1 >nul
:nosleep
echo fake-vmrun-bounded-output
if "%FAKE_VMRUN_EXIT%"=="" ( exit /b 0 ) else ( exit /b %FAKE_VMRUN_EXIT% )
'@
}
AfterAll {
Remove-Item $script:SlowFakeVmrun -Force -ErrorAction SilentlyContinue
}
AfterEach {
$env:FAKE_VMRUN_EXIT = $null
$env:FAKE_VMRUN_SLEEP = $null
}
It 'returns ExitCode 0, TimedOut false and Output when command succeeds quickly' {
$env:FAKE_VMRUN_EXIT = '0'
$env:FAKE_VMRUN_SLEEP = '0'
$r = Invoke-VmrunBounded -VmrunPath $script:SlowFakeVmrun -Operation 'start' -Arguments @('some.vmx', 'nogui') -TimeoutSeconds 30
$r.ExitCode | Should -Be 0
$r.TimedOut | Should -Be $false
"$($r.Output)" | Should -Match 'fake-vmrun-bounded-output'
$r.ElapsedSeconds | Should -BeGreaterOrEqual 0
}
It 'returns non-zero ExitCode without throwing by default' {
$env:FAKE_VMRUN_EXIT = '3'
$env:FAKE_VMRUN_SLEEP = '0'
$r = Invoke-VmrunBounded -VmrunPath $script:SlowFakeVmrun -Operation 'deleteVM' -TimeoutSeconds 30
$r.ExitCode | Should -Be 3
$r.TimedOut | Should -Be $false
}
It 'throws when -ThrowOnError and ExitCode is non-zero' {
$env:FAKE_VMRUN_EXIT = '2'
$env:FAKE_VMRUN_SLEEP = '0'
{ Invoke-VmrunBounded -VmrunPath $script:SlowFakeVmrun -Operation 'stop' -TimeoutSeconds 30 -ThrowOnError } |
Should -Throw -ExpectedMessage '*stop failed*'
}
It 'does not throw when -ThrowOnError and ExitCode is 0' {
$env:FAKE_VMRUN_EXIT = '0'
$env:FAKE_VMRUN_SLEEP = '0'
{ Invoke-VmrunBounded -VmrunPath $script:SlowFakeVmrun -Operation 'start' -TimeoutSeconds 30 -ThrowOnError } |
Should -Not -Throw
}
It 'times out and returns TimedOut = true when process exceeds TimeoutSeconds' {
# FAKE_VMRUN_SLEEP uses ping -n N which waits roughly (N-1) seconds.
# Set sleep >> timeout so the process is guaranteed to still be running.
$env:FAKE_VMRUN_SLEEP = '60'
$env:FAKE_VMRUN_EXIT = '0'
$r = Invoke-VmrunBounded -VmrunPath $script:SlowFakeVmrun -Operation 'start' -TimeoutSeconds 2
$r.TimedOut | Should -Be $true
$r.ExitCode | Should -Be -1
$r.ElapsedSeconds | Should -BeLessOrEqual 10 # killed well before 60s
}
It 'throws when -ThrowOnTimeout and process times out' {
$env:FAKE_VMRUN_SLEEP = '60'
$env:FAKE_VMRUN_EXIT = '0'
{ Invoke-VmrunBounded -VmrunPath $script:SlowFakeVmrun -Operation 'deleteVM' -TimeoutSeconds 2 -ThrowOnTimeout } |
Should -Throw -ExpectedMessage '*timed out*'
}
It 'does not throw on timeout when neither flag is set' {
$env:FAKE_VMRUN_SLEEP = '60'
$env:FAKE_VMRUN_EXIT = '0'
{ Invoke-VmrunBounded -VmrunPath $script:SlowFakeVmrun -Operation 'start' -TimeoutSeconds 2 } |
Should -Not -Throw
}
It 'ElapsedSeconds is populated and plausible' {
$env:FAKE_VMRUN_EXIT = '0'
$env:FAKE_VMRUN_SLEEP = '0'
$r = Invoke-VmrunBounded -VmrunPath $script:SlowFakeVmrun -Operation 'list' -TimeoutSeconds 30
$r.ElapsedSeconds | Should -BeGreaterOrEqual 0
$r.ElapsedSeconds | Should -BeLessThan 30
}
}
tests/python/test_commands_bench.py
+487
View File
@@ -0,0 +1,487 @@
"""Tests for the ``bench`` sub-commands (Phase C2 burn-in + C3 measure).
The C1 ``--help`` smoke tests are kept; behavioural tests mock the backend
and the per-job runner so no real VMs are spun up.
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
import pytest
from click.testing import CliRunner
import ci_orchestrator.commands.bench as bench_module
from ci_orchestrator.__main__ import cli
from ci_orchestrator.backends.errors import BackendNotAvailable, BackendOperationFailed
from ci_orchestrator.backends.protocol import VmHandle
from ci_orchestrator.commands.bench import (
JobResult,
PhaseTiming,
_find_orphans,
_stale_lock,
bench,
)
# ── C1 smoke tests (kept) ───────────────────────────────────────────────────
def test_bench_help_lists_subcommands() -> None:
result = CliRunner().invoke(cli, ["bench", "--help"])
assert result.exit_code == 0, result.output
assert "run" in result.output
assert "measure" in result.output
def test_bench_run_help() -> None:
result = CliRunner().invoke(cli, ["bench", "run", "--help"])
assert result.exit_code == 0, result.output
assert "--concurrency" in result.output
assert "--rounds" in result.output
def test_bench_measure_help() -> None:
result = CliRunner().invoke(cli, ["bench", "measure", "--help"])
assert result.exit_code == 0, result.output
assert "--iterations" in result.output
# ── helpers ─────────────────────────────────────────────────────────────────
def test_find_orphans_matches_job_prefix(tmp_path: Path) -> None:
(tmp_path / "Clone_burnin-r1-j1-x_20260101_aaa").mkdir()
(tmp_path / "Clone_other-job_20260101_bbb").mkdir()
(tmp_path / "not-a-clone").mkdir()
orphans = _find_orphans(tmp_path, ["burnin-r1-j1-x"])
assert len(orphans) == 1
assert orphans[0].name.startswith("Clone_burnin-r1-j1-x_")
def test_find_orphans_missing_base_dir(tmp_path: Path) -> None:
assert _find_orphans(tmp_path / "ghost", ["j1"]) == []
def test_stale_lock_detects_old_file(tmp_path: Path) -> None:
import os
lock = tmp_path / "vm-start.lock"
lock.write_text("x")
old = lock.stat().st_mtime - (10 * 60)
os.utime(lock, (old, old))
assert _stale_lock(lock) is True
def test_stale_lock_fresh_file_is_ok(tmp_path: Path) -> None:
lock = tmp_path / "vm-start.lock"
lock.write_text("x")
assert _stale_lock(lock) is False
def test_stale_lock_missing_file(tmp_path: Path) -> None:
assert _stale_lock(tmp_path / "absent.lock") is False
# ── bench run ───────────────────────────────────────────────────────────────
@pytest.fixture
def _ci_paths(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> Path:
"""Point load_config at temp build-vms/artifacts dirs."""
env = {
"CI_BUILD_VMS": str(tmp_path / "build-vms"),
"CI_ARTIFACTS": str(tmp_path / "artifacts"),
}
(tmp_path / "build-vms").mkdir()
(tmp_path / "artifacts").mkdir()
real = bench_module.load_config
monkeypatch.setattr(bench_module, "load_config", lambda: real(env=env))
return tmp_path
def test_bench_run_all_pass(monkeypatch: pytest.MonkeyPatch, _ci_paths: Path) -> None:
calls: list[str] = []
def _fake_job(*, job_id: str, **_kw: Any) -> tuple[int, str]:
calls.append(job_id)
return 0, ""
monkeypatch.setattr(bench_module, "_run_one_job", _fake_job)
result = CliRunner().invoke(
cli, ["bench", "run", "--concurrency", "3", "--rounds", "2"]
)
assert result.exit_code == 0, result.output
assert "OVERALL: PASS" in result.output
assert len(calls) == 6 # 3 jobs * 2 rounds
# Report file written under artifacts/bench.
reports = list((_ci_paths / "artifacts" / "bench").glob("*.json"))
assert len(reports) == 1
data = json.loads(reports[0].read_text())
assert data["overall"] == "PASS"
assert data["totalJobs"] == 6
assert len(data["results"]) == 2
def test_bench_run_failing_job_marks_round_fail(
monkeypatch: pytest.MonkeyPatch, _ci_paths: Path
) -> None:
def _fake_job(*, job_id: str, slot: int = 0, **_kw: Any) -> tuple[int, str]:
return 1, "clone failed: boom"
monkeypatch.setattr(bench_module, "_run_one_job", _fake_job)
result = CliRunner().invoke(
cli, ["bench", "run", "--concurrency", "1", "--rounds", "1"]
)
assert result.exit_code == 1
assert "OVERALL: FAIL" in result.output
assert "clone failed: boom" in result.output
def test_bench_run_orphan_dir_fails_round(
monkeypatch: pytest.MonkeyPatch, _ci_paths: Path
) -> None:
base = _ci_paths / "build-vms"
def _fake_job(*, job_id: str, **_kw: Any) -> tuple[int, str]:
# Leave an orphan clone dir behind for this job.
(base / f"Clone_{job_id}_20260101_orphan").mkdir()
return 0, ""
monkeypatch.setattr(bench_module, "_run_one_job", _fake_job)
result = CliRunner().invoke(
cli, ["bench", "run", "--concurrency", "1", "--rounds", "1"]
)
assert result.exit_code == 1
assert "orphan" in result.output
assert "OVERALL: FAIL" in result.output
def test_bench_run_stale_lock_fails_round(
monkeypatch: pytest.MonkeyPatch, _ci_paths: Path
) -> None:
base = _ci_paths / "build-vms"
monkeypatch.setattr(bench_module, "_run_one_job", lambda **_kw: (0, ""))
# _stale_lock reports a stale lock regardless of file age.
monkeypatch.setattr(bench_module, "_stale_lock", lambda _p: True)
result = CliRunner().invoke(
cli,
["bench", "run", "--concurrency", "1", "--rounds", "1", "--clone-base-dir", str(base)],
)
assert result.exit_code == 1
assert "stale lock" in result.output
def test_bench_run_custom_json_out(
monkeypatch: pytest.MonkeyPatch, _ci_paths: Path
) -> None:
out = _ci_paths / "custom" / "report.json"
monkeypatch.setattr(bench_module, "_run_one_job", lambda **_kw: (0, ""))
result = CliRunner().invoke(
cli,
["bench", "run", "--concurrency", "1", "--rounds", "1", "--json-out", str(out)],
)
assert result.exit_code == 0, result.output
assert out.is_file()
def test_run_one_job_invokes_job_command(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""_run_one_job drives the real job command and maps exit codes."""
import ci_orchestrator.commands.job as job_module
captured: dict[str, Any] = {}
class _FakeResult:
exit_code = 0
output = "ok\n"
class _FakeRunner:
def invoke(self, _cmd: Any, args: list[str], **_kw: Any) -> _FakeResult:
captured["args"] = args
return _FakeResult()
monkeypatch.setattr("click.testing.CliRunner", lambda *a, **k: _FakeRunner())
del job_module # imported for symmetry; not otherwise needed
code, err = bench_module._run_one_job(
job_id="j1",
guest_os="linux",
build_command="echo hi",
clone_base_dir=tmp_path,
)
assert code == 0
assert err == ""
assert "--job-id" in captured["args"]
assert "--build-command" in captured["args"]
def test_run_one_job_failure_returns_tail(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
class _FakeResult:
exit_code = 1
output = "line one\nError: clone failed\n"
class _FakeRunner:
def invoke(self, *_a: Any, **_k: Any) -> _FakeResult:
return _FakeResult()
monkeypatch.setattr("click.testing.CliRunner", lambda *a, **k: _FakeRunner())
code, err = bench_module._run_one_job(
job_id="j1", guest_os="linux", build_command="", clone_base_dir=tmp_path
)
assert code == 1
assert err == "Error: clone failed"
# ── bench measure ───────────────────────────────────────────────────────────
class _FakeBackend:
def __init__(self, *, ip: str | None = "10.0.0.5", fail: str = "") -> None:
self._ip = ip
self._fail = fail
self.calls: list[str] = []
def clone_linked(
self, template: str, snapshot: str, name: str, destination: str | None = None
) -> VmHandle:
self.calls.append("clone")
if destination:
d = Path(destination)
d.parent.mkdir(parents=True, exist_ok=True)
d.write_text('guestOS = "ubuntu-64"', encoding="utf-8")
if self._fail == "clone":
raise BackendOperationFailed("clone", 1, "no")
return VmHandle(identifier=destination or name, name=name)
def start(self, handle: VmHandle, headless: bool = True) -> None:
self.calls.append("start")
def get_ip(self, handle: VmHandle, timeout: float = 0.0) -> str | None:
self.calls.append("get_ip")
return self._ip
def stop(self, handle: VmHandle, hard: bool = False) -> None:
self.calls.append("stop")
def delete(self, handle: VmHandle) -> None:
self.calls.append("delete")
def _make_template(tmp_path: Path) -> Path:
tpl = tmp_path / "tpl" / "LinuxBuild2404.vmx"
tpl.parent.mkdir(parents=True)
tpl.write_text('guestOS = "ubuntu-64"', encoding="utf-8")
return tpl
@pytest.fixture
def _measure_paths(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> Path:
env = {
"CI_BUILD_VMS": str(tmp_path / "build-vms"),
"CI_ARTIFACTS": str(tmp_path / "artifacts"),
}
real = bench_module.load_config
monkeypatch.setattr(bench_module, "load_config", lambda: real(env=env))
return tmp_path
def test_bench_measure_happy_path(
monkeypatch: pytest.MonkeyPatch, _measure_paths: Path
) -> None:
tpl = _make_template(_measure_paths)
backend = _FakeBackend(ip="10.0.0.5")
monkeypatch.setattr(bench_module, "load_backend", lambda _c: backend)
# transport ready immediately
monkeypatch.setattr(bench_module, "_tcp_open", lambda *a, **k: True)
result = CliRunner().invoke(
cli,
[
"bench",
"measure",
"--guest-os",
"linux",
"--iterations",
"2",
"--template-path",
str(tpl),
"--timeout-seconds",
"30",
],
)
assert result.exit_code == 0, result.output
assert "clone" in backend.calls
assert "delete" in backend.calls
jsonl = _measure_paths / "artifacts" / "benchmark.jsonl"
assert jsonl.is_file()
records = [json.loads(line) for line in jsonl.read_text().splitlines()]
assert len(records) == 2
# Legacy field names preserved.
assert set(records[0]) >= {
"ts", "runId", "iteration", "template", "snapshot", "guestOS",
"cloneSec", "startSec", "ipSec", "readySec", "destroySec",
"totalBootSec", "deltaKB", "vmIP", "error",
}
assert records[0]["vmIP"] == "10.0.0.5"
assert records[0]["error"] is None
assert "Average boot-to-ready" in result.output
def test_bench_measure_ip_timeout_records_error(
monkeypatch: pytest.MonkeyPatch, _measure_paths: Path
) -> None:
tpl = _make_template(_measure_paths)
backend = _FakeBackend(ip=None) # never returns an IP
monkeypatch.setattr(bench_module, "load_backend", lambda _c: backend)
# Make the IP-wait loop terminate fast: a monotonic clock that jumps
# 100s per call, so any deadline is exceeded within a couple of polls.
counter = {"t": 0.0}
def _fast_now() -> float:
counter["t"] += 100.0
return counter["t"]
monkeypatch.setattr(bench_module, "_now", _fast_now)
result = CliRunner().invoke(
cli,
["bench", "measure", "--template-path", str(tpl), "--timeout-seconds", "30"],
)
assert result.exit_code == 0, result.output
jsonl = _measure_paths / "artifacts" / "benchmark.jsonl"
rec = json.loads(jsonl.read_text().splitlines()[0])
assert rec["error"] is not None
assert rec["readySec"] is None
# destroy still ran (finally block).
assert "delete" in backend.calls
def test_bench_measure_template_missing(
monkeypatch: pytest.MonkeyPatch, _measure_paths: Path
) -> None:
result = CliRunner().invoke(
cli, ["bench", "measure", "--template-path", str(_measure_paths / "nope.vmx")]
)
assert result.exit_code != 0
assert "Template VMX not found" in result.output
def test_bench_measure_backend_unavailable(
monkeypatch: pytest.MonkeyPatch, _measure_paths: Path
) -> None:
tpl = _make_template(_measure_paths)
def _raise(_c: Any) -> Any:
raise BackendNotAvailable("no vmrun")
monkeypatch.setattr(bench_module, "load_backend", _raise)
result = CliRunner().invoke(
cli, ["bench", "measure", "--template-path", str(tpl)]
)
assert result.exit_code != 0
assert "backend unavailable" in result.output
def test_bench_measure_clone_failure_records_error(
monkeypatch: pytest.MonkeyPatch, _measure_paths: Path
) -> None:
tpl = _make_template(_measure_paths)
backend = _FakeBackend(fail="clone")
monkeypatch.setattr(bench_module, "load_backend", lambda _c: backend)
result = CliRunner().invoke(
cli, ["bench", "measure", "--template-path", str(tpl), "--timeout-seconds", "30"]
)
assert result.exit_code == 0, result.output
jsonl = _measure_paths / "artifacts" / "benchmark.jsonl"
rec = json.loads(jsonl.read_text().splitlines()[0])
assert rec["error"] is not None
assert rec["cloneSec"] is None
def test_bench_measure_transport_port_timeout(
monkeypatch: pytest.MonkeyPatch, _measure_paths: Path
) -> None:
tpl = _make_template(_measure_paths)
backend = _FakeBackend(ip="10.0.0.9")
monkeypatch.setattr(bench_module, "load_backend", lambda _c: backend)
# IP is acquired immediately, but the transport port never opens.
monkeypatch.setattr(bench_module, "_wait_port", lambda *a, **k: False)
result = CliRunner().invoke(
cli, ["bench", "measure", "--template-path", str(tpl), "--timeout-seconds", "30"]
)
assert result.exit_code == 0, result.output
jsonl = _measure_paths / "artifacts" / "benchmark.jsonl"
rec = json.loads(jsonl.read_text().splitlines()[0])
assert rec["readySec"] is None
assert "port" in (rec["error"] or "")
def test_bench_measure_destroy_suppresses_backend_errors(
monkeypatch: pytest.MonkeyPatch, _measure_paths: Path
) -> None:
tpl = _make_template(_measure_paths)
class _FlakyBackend(_FakeBackend):
def stop(self, handle: VmHandle, hard: bool = False) -> None:
self.calls.append("stop")
raise BackendOperationFailed("stop", 1, "no")
def delete(self, handle: VmHandle) -> None:
self.calls.append("delete")
raise BackendOperationFailed("delete", 1, "no")
backend = _FlakyBackend(ip="10.0.0.5")
monkeypatch.setattr(bench_module, "load_backend", lambda _c: backend)
monkeypatch.setattr(bench_module, "_tcp_open", lambda *a, **k: True)
result = CliRunner().invoke(
cli, ["bench", "measure", "--template-path", str(tpl), "--timeout-seconds", "30"]
)
# stop/delete raising must not crash the command.
assert result.exit_code == 0, result.output
assert "stop" in backend.calls
assert "delete" in backend.calls
def test_bench_run_report_write_failure_is_tolerated(
monkeypatch: pytest.MonkeyPatch, _ci_paths: Path
) -> None:
monkeypatch.setattr(bench_module, "_run_one_job", lambda **_kw: (0, ""))
def _boom(_report: Any, _path: Any) -> None:
raise OSError("disk full")
monkeypatch.setattr(bench_module, "_write_run_report", _boom)
result = CliRunner().invoke(
cli, ["bench", "run", "--concurrency", "1", "--rounds", "1"]
)
assert result.exit_code == 0, result.output
assert "could not write report" in result.output
def test_bench_measure_default_template_windows(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""--guest-os windows resolves the WinBuild default template + snapshot."""
from ci_orchestrator.config import load_config as real_load
cfg = real_load(env={"CI_ROOT": str(tmp_path)})
tpl, snap = bench_module._resolve_template(cfg, "windows")
assert tpl.name == "WinBuild2025.vmx"
assert snap == "BaseClean"
tpl_l, snap_l = bench_module._resolve_template(cfg, "linux")
assert tpl_l.name == "LinuxBuild2404.vmx"
assert snap_l == "BaseClean-Linux"
# ── dataclass behaviour ─────────────────────────────────────────────────────
def test_phase_timing_total_boot_none_when_incomplete() -> None:
t = PhaseTiming(iteration=1, run_id="r", template="t", snapshot="s", guest_os="linux")
assert t.total_boot_sec is None
t.clone_sec, t.start_sec, t.ip_sec, t.ready_sec = 1.0, 2.0, 3.0, 4.0
assert t.total_boot_sec == 10.0
def test_job_result_dataclass_defaults() -> None:
jr = JobResult(job_id="j", slot=1, exit_code=0, status="PASS", elapsed_sec=1.0)
assert jr.error == ""
assert bench is not None
tests/python/test_commands_creds.py
+184
View File
@@ -0,0 +1,184 @@
"""Tests for ``creds set`` (C6).
CLI registration smoke tests (``--help``) are kept; behavioural tests mock
the ``keyring`` module and include a round-trip via ``KeyringCredentialStore``.
"""
from __future__ import annotations
import sys
import types
from typing import Any
import pytest
from click.testing import CliRunner
from ci_orchestrator.__main__ import cli
from ci_orchestrator.credentials import KeyringCredentialStore
# ── --help smoke tests (kept) ───────────────────────────────────────────────
def test_creds_help_lists_set() -> None:
result = CliRunner().invoke(cli, ["creds", "--help"])
assert result.exit_code == 0, result.output
assert "set" in result.output
def test_creds_set_help() -> None:
result = CliRunner().invoke(cli, ["creds", "set", "--help"])
assert result.exit_code == 0, result.output
assert "--target" in result.output
assert "--user" in result.output
# ── fake keyring ────────────────────────────────────────────────────────────
def _install_fake_keyring(monkeypatch: pytest.MonkeyPatch) -> dict[tuple[str, str], str]:
"""Install a fake ``keyring`` module with a two-key password store.
Returns the backing dict so tests can inspect what was written. The fake
implements both ``set_password`` (writer) and ``get_password`` /
``get_credential`` (reader), so round-trips work end to end.
"""
store: dict[tuple[str, str], str] = {}
fake = types.ModuleType("keyring")
def set_password(service: str, key: str, password: str) -> None:
store[(service, key)] = password
def get_password(service: str, key: str) -> str | None:
return store.get((service, key))
def get_credential(service: str, _username: str | None) -> Any:
# File backends are a no-op here (mirrors PlaintextKeyring); the reader
# falls through to the two-entry scheme.
return None
fake.set_password = set_password # type: ignore[attr-defined]
fake.get_password = get_password # type: ignore[attr-defined]
fake.get_credential = get_credential # type: ignore[attr-defined]
monkeypatch.setitem(sys.modules, "keyring", fake)
return store
# ── write scheme ────────────────────────────────────────────────────────────
def test_creds_set_writes_two_entries(monkeypatch: pytest.MonkeyPatch) -> None:
store = _install_fake_keyring(monkeypatch)
result = CliRunner().invoke(
cli,
["creds", "set", "--target", "BuildVMGuest", "--user", "ci_build",
"--password-stdin"],
input="s3cret\n",
)
assert result.exit_code == 0, result.output
# username under "{target}:meta"
assert store[("BuildVMGuest:meta", "username")] == "ci_build"
# password under target keyed by username
assert store[("BuildVMGuest", "ci_build")] == "s3cret"
def test_creds_set_round_trips_via_store(monkeypatch: pytest.MonkeyPatch) -> None:
"""Round-trip: write with `creds set`, read back with the reader."""
_install_fake_keyring(monkeypatch)
result = CliRunner().invoke(
cli,
["creds", "set", "--target", "BuildVMGuest", "--user", "WINBUILD\\ci",
"--password-stdin"],
input="p@ss word!\n",
)
assert result.exit_code == 0, result.output
cred = KeyringCredentialStore().get("BuildVMGuest")
assert cred.username == "WINBUILD\\ci"
assert cred.password == "p@ss word!"
def test_creds_set_default_target(monkeypatch: pytest.MonkeyPatch) -> None:
store = _install_fake_keyring(monkeypatch)
result = CliRunner().invoke(
cli,
["creds", "set", "--user", "ci_build", "--password-stdin"],
input="abc\n",
)
assert result.exit_code == 0, result.output
assert store[("BuildVMGuest:meta", "username")] == "ci_build"
# ── password sourcing ───────────────────────────────────────────────────────
def test_creds_set_prompts_when_no_stdin_flag(
monkeypatch: pytest.MonkeyPatch,
) -> None:
store = _install_fake_keyring(monkeypatch)
# click.prompt with confirmation_prompt reads the value twice.
result = CliRunner().invoke(
cli,
["creds", "set", "--target", "T", "--user", "u"],
input="hunter2\nhunter2\n",
)
assert result.exit_code == 0, result.output
assert store[("T", "u")] == "hunter2"
def test_creds_set_empty_password_rejected(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_install_fake_keyring(monkeypatch)
result = CliRunner().invoke(
cli,
["creds", "set", "--target", "T", "--user", "u", "--password-stdin"],
input="\n",
)
assert result.exit_code != 0
assert "empty" in result.output.lower()
def test_creds_set_strips_trailing_newline(
monkeypatch: pytest.MonkeyPatch,
) -> None:
store = _install_fake_keyring(monkeypatch)
result = CliRunner().invoke(
cli,
["creds", "set", "--target", "T", "--user", "u", "--password-stdin"],
input="secret\r\n",
)
assert result.exit_code == 0, result.output
assert store[("T", "u")] == "secret"
def test_creds_set_never_accepts_password_option() -> None:
"""The CLI must NOT expose a --password option."""
result = CliRunner().invoke(cli, ["creds", "set", "--help"])
assert "--password " not in result.output
assert "--password=" not in result.output
def test_creds_set_keyring_missing(monkeypatch: pytest.MonkeyPatch) -> None:
"""When keyring import fails, surface a clean ClickException."""
import ci_orchestrator.commands.creds as creds_module
def _raise(_t: str, _u: str, _p: str) -> None:
raise RuntimeError("keyring is not installed; run `pip install keyring`.")
monkeypatch.setattr(creds_module, "_store_credential", _raise)
result = CliRunner().invoke(
cli,
["creds", "set", "--target", "T", "--user", "u", "--password-stdin"],
input="x\n",
)
assert result.exit_code != 0
assert "keyring is not installed" in result.output
def test_store_credential_invokes_keyring(monkeypatch: pytest.MonkeyPatch) -> None:
"""Unit-level: _store_credential writes both entries to the module."""
import ci_orchestrator.commands.creds as creds_module
store = _install_fake_keyring(monkeypatch)
creds_module._store_credential("Tgt", "user1", "pw1")
assert store[("Tgt:meta", "username")] == "user1"
assert store[("Tgt", "user1")] == "pw1"
tests/python/test_commands_smoke.py
+286
View File
@@ -0,0 +1,286 @@
"""Tests for ``ci_orchestrator smoke`` (Phase C1 scaffolding + C4 behaviour)."""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
import pytest
from click.testing import CliRunner
import ci_orchestrator.commands.smoke as smoke_module
from ci_orchestrator.__main__ import cli
from ci_orchestrator.config import BackendConfig, Config, Paths
# ──────────────────────────────────────────────────────── C1 --help smoke
def test_smoke_help_lists_run() -> None:
result = CliRunner().invoke(cli, ["smoke", "--help"])
assert result.exit_code == 0, result.output
assert "run" in result.output
def test_smoke_run_help() -> None:
result = CliRunner().invoke(cli, ["smoke", "run", "--help"])
assert result.exit_code == 0, result.output
assert "--guest-os" in result.output
# ──────────────────────────────────────────────────────────── helpers
def _patch_config(monkeypatch: pytest.MonkeyPatch, artifacts: Path) -> None:
paths = Paths(
root=artifacts.parent,
templates=artifacts.parent / "templates",
build_vms=artifacts.parent / "build-vms",
artifacts=artifacts,
keys=artifacts.parent / "keys",
)
cfg = Config(paths=paths, backend=BackendConfig(), ip_pool=None)
monkeypatch.setattr(smoke_module, "load_config", lambda: cfg)
def _patch_job(
monkeypatch: pytest.MonkeyPatch,
*,
artifacts: Path,
make_artifact: bool = True,
raises: bool = False,
) -> dict[str, list[Any]]:
"""Replace the in-process ``job`` command with a recording fake."""
calls: dict[str, list[Any]] = {"job": []}
def _fake_callback(**kwargs: Any) -> None:
calls["job"].append(kwargs)
if make_artifact:
job_dir = artifacts / kwargs["job_id"]
job_dir.mkdir(parents=True, exist_ok=True)
(job_dir / "smoke.txt").write_text("smoke-ok", encoding="utf-8")
if raises:
import click
raise click.ClickException("clone failed: boom")
# ``job_command`` is a click.Command; patch its callback.
monkeypatch.setattr(smoke_module.job_command, "callback", _fake_callback)
return calls
def _read_events(jsonl_path: Path) -> list[dict[str, Any]]:
out: list[dict[str, Any]] = []
for line in jsonl_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if line:
out.append(json.loads(line))
return out
# ──────────────────────────────────────────────────────── C4 behaviour
def test_smoke_run_linux_happy_path(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
_patch_config(monkeypatch, artifacts)
calls = _patch_job(monkeypatch, artifacts=artifacts)
result = CliRunner().invoke(
cli, ["smoke", "run", "--guest-os", "linux", "--job-id", "smoke-1"]
)
assert result.exit_code == 0, result.output
assert "PASSED" in result.output
# The job ran once with the Linux no-op marker preset.
assert len(calls["job"]) == 1
kw = calls["job"][0]
assert kw["guest_os"] == "linux"
assert kw["job_id"] == "smoke-1"
assert "smoke.txt" in kw["build_command"]
assert kw["guest_artifact_source"] == "/opt/ci/output"
# Artifact dir was created and the success event is present.
assert (artifacts / "smoke-1").is_dir()
events = _read_events(tmp_path / "logs" / "smoke-1" / "invoke-ci.jsonl")
assert any(e["phase"] == "job" and e["status"] == "start" for e in events)
assert any(e["phase"] == "job" and e["status"] == "success" for e in events)
def test_smoke_run_windows_uses_windows_preset(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
_patch_config(monkeypatch, artifacts)
calls = _patch_job(monkeypatch, artifacts=artifacts)
result = CliRunner().invoke(
cli, ["smoke", "run", "--guest-os", "windows", "--job-id", "smoke-win"]
)
assert result.exit_code == 0, result.output
kw = calls["job"][0]
assert kw["guest_os"] == "windows"
assert kw["guest_artifact_source"] == "C:\\CI\\output"
assert "C:\\CI\\output" in kw["build_command"]
def test_smoke_run_failure_when_job_raises(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
_patch_config(monkeypatch, artifacts)
_patch_job(monkeypatch, artifacts=artifacts, make_artifact=False, raises=True)
result = CliRunner().invoke(
cli, ["smoke", "run", "--guest-os", "linux", "--job-id", "smoke-fail"]
)
assert result.exit_code != 0
assert "FAILED" in result.output
# A failure event was logged and there is no success event.
events = _read_events(tmp_path / "logs" / "smoke-fail" / "invoke-ci.jsonl")
assert any(e["status"] == "failure" for e in events)
assert not any(e["status"] == "success" for e in events)
def test_smoke_run_fails_when_artifact_missing(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
_patch_config(monkeypatch, artifacts)
# Job "succeeds" but produces no artifact dir → smoke must still fail.
_patch_job(monkeypatch, artifacts=artifacts, make_artifact=False)
result = CliRunner().invoke(
cli, ["smoke", "run", "--guest-os", "linux", "--job-id", "smoke-noart"]
)
assert result.exit_code != 0
assert "artifact dir not found" in result.output
def test_smoke_run_preset_ns7zip(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
_patch_config(monkeypatch, artifacts)
calls = _patch_job(monkeypatch, artifacts=artifacts)
result = CliRunner().invoke(
cli,
["smoke", "run", "--guest-os", "linux", "--preset", "ns7zip", "--job-id", "p1"],
)
assert result.exit_code == 0, result.output
kw = calls["job"][0]
assert "build_plugin.py" in kw["build_command"]
assert "7zip-version" in kw["build_command"]
assert kw["guest_artifact_source"] == "plugins"
def test_smoke_run_preset_nsinnounp(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
_patch_config(monkeypatch, artifacts)
calls = _patch_job(monkeypatch, artifacts=artifacts)
result = CliRunner().invoke(
cli,
["smoke", "run", "--preset", "nsinnounp", "--job-id", "p2"],
)
assert result.exit_code == 0, result.output
kw = calls["job"][0]
assert kw["build_command"] == "python3 build_plugin.py --final --dist-dir dist"
assert kw["guest_artifact_source"] == "dist"
def test_smoke_run_preset_rejected_on_windows(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
_patch_config(monkeypatch, artifacts)
_patch_job(monkeypatch, artifacts=artifacts)
result = CliRunner().invoke(
cli, ["smoke", "run", "--guest-os", "windows", "--preset", "ns7zip"]
)
assert result.exit_code != 0
assert "only supported with --guest-os linux" in result.output
def test_smoke_run_preset_and_build_command_mutually_exclusive(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
_patch_config(monkeypatch, artifacts)
_patch_job(monkeypatch, artifacts=artifacts)
result = CliRunner().invoke(
cli,
[
"smoke", "run", "--preset", "ns7zip",
"--build-command", "echo hi",
],
)
assert result.exit_code != 0
assert "mutually exclusive" in result.output
def test_smoke_run_custom_build_command(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
_patch_config(monkeypatch, artifacts)
calls = _patch_job(monkeypatch, artifacts=artifacts)
result = CliRunner().invoke(
cli,
[
"smoke", "run", "--guest-os", "linux",
"--build-command", "make all",
"--job-id", "custom-1",
],
)
assert result.exit_code == 0, result.output
kw = calls["job"][0]
assert kw["build_command"] == "make all"
# Custom command keeps the OS-default artifact source.
assert kw["guest_artifact_source"] == "/opt/ci/output"
def test_has_job_success_tolerates_malformed_jsonl(tmp_path: Path) -> None:
"""Garbage/missing log lines must not raise and must report no success."""
missing = tmp_path / "nope.jsonl"
assert smoke_module._has_job_success(missing) is False
jsonl = tmp_path / "log.jsonl"
jsonl.write_text(
"\n" # blank line
"not json\n" # JSONDecodeError
"[1, 2, 3]\n" # valid JSON but not a dict
'{"phase": "job", "status": "start"}\n', # dict, no success
encoding="utf-8",
)
assert smoke_module._has_job_success(jsonl) is False
jsonl.write_text('{"phase": "job", "status": "success"}\n', encoding="utf-8")
assert smoke_module._has_job_success(jsonl) is True
def test_smoke_run_default_job_id_and_separate_log_dir(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
artifacts = tmp_path / "artifacts"
logs = tmp_path / "logs"
_patch_config(monkeypatch, artifacts)
calls = _patch_job(monkeypatch, artifacts=artifacts)
result = CliRunner().invoke(
cli, ["smoke", "run", "--log-dir", str(logs)]
)
assert result.exit_code == 0, result.output
generated_id = calls["job"][0]["job_id"]
assert generated_id.startswith("smoke-")
# JSONL written under the override log dir, artifacts under config dir.
assert (logs / generated_id / "invoke-ci.jsonl").is_file()
assert (artifacts / generated_id).is_dir()
tests/python/test_commands_validate.py
+542
View File
@@ -0,0 +1,542 @@
"""Tests for ``validate host`` (C5) and ``validate guest`` (C6).
CLI registration smoke tests (``--help``) are kept; behavioural tests mock
the backend, keyring, systemctl, and transports.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
import pytest
from click.testing import CliRunner
import ci_orchestrator.commands.validate as validate_module
from ci_orchestrator.__main__ import cli
from ci_orchestrator.backends.errors import (
BackendNotAvailable,
BackendOperationFailed,
)
from ci_orchestrator.config import BackendConfig, Config, Paths
from ci_orchestrator.credentials import Credential
from ci_orchestrator.transport.errors import (
TransportCommandError,
TransportConnectError,
)
# ── --help smoke tests (kept) ───────────────────────────────────────────────
def test_validate_help_lists_subcommands() -> None:
result = CliRunner().invoke(cli, ["validate", "--help"])
assert result.exit_code == 0, result.output
assert "host" in result.output
assert "guest" in result.output
def test_validate_guest_help() -> None:
result = CliRunner().invoke(cli, ["validate", "guest", "--help"])
assert result.exit_code == 0, result.output
assert "--host" in result.output
# ── helpers ─────────────────────────────────────────────────────────────────
def _make_config(tmp_path: Path) -> Config:
root = tmp_path / "ci"
templates = root / "templates"
build_vms = root / "build-vms"
artifacts = root / "artifacts"
keys = root / "keys"
for p in (root, templates, build_vms, artifacts, keys):
p.mkdir(parents=True, exist_ok=True)
return Config(
paths=Paths(
root=root,
templates=templates,
build_vms=build_vms,
artifacts=artifacts,
keys=keys,
),
backend=BackendConfig(type="workstation"),
guest_cred_target="BuildVMGuest",
)
class _FakeBackend:
def __init__(self, snapshots: dict[str, list[str]] | None = None) -> None:
self.vmrun_path = "/usr/bin/vmrun"
self._snapshots = snapshots or {}
def list_snapshots(self, handle: Any) -> list[str]:
return self._snapshots.get(handle.identifier, [])
def _make_template_vmx(config: Config, name: str) -> str:
vmx_dir = config.paths.templates / name
vmx_dir.mkdir(parents=True, exist_ok=True)
vmx = vmx_dir / f"{name}.vmx"
vmx.write_text('config.version = "8"', encoding="utf-8")
return str(vmx)
# ── validate host: individual checks ────────────────────────────────────────
def test_check_vmrun_ok(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
config = _make_config(tmp_path)
monkeypatch.setattr(validate_module, "load_backend", lambda _c: _FakeBackend())
result = validate_module._check_vmrun(config)
assert result.ok
assert "vmrun" in result.detail
def test_check_vmrun_no_path_attr(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
class _NoPath:
pass
monkeypatch.setattr(validate_module, "load_backend", lambda _c: _NoPath())
result = validate_module._check_vmrun(config)
assert result.ok
assert "backend loaded" in result.detail
def test_check_vmrun_unavailable(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
def _raise(_c: Any) -> Any:
raise BackendNotAvailable("vmrun missing")
monkeypatch.setattr(validate_module, "load_backend", _raise)
result = validate_module._check_vmrun(config)
assert not result.ok
assert "missing" in result.detail
def test_check_snapshots_backend_unavailable(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
def _raise(_c: Any) -> Any:
raise BackendNotAvailable("no vmrun")
monkeypatch.setattr(validate_module, "load_backend", _raise)
result = validate_module._check_snapshots(config)
assert not result.ok
assert "backend unavailable" in result.detail
def test_check_snapshots_list_raises(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
_make_template_vmx(config, "LinuxBuild2404")
class _Backend:
vmrun_path = "/usr/bin/vmrun"
def list_snapshots(self, _handle: Any) -> list[str]:
raise BackendOperationFailed("listSnapshots", 1, "boom")
monkeypatch.setattr(validate_module, "load_backend", lambda _c: _Backend())
result = validate_module._check_snapshots(config)
assert not result.ok
assert "listSnapshots failed" in result.detail
def test_check_keyring_runtime_error(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
class _Store:
def get(self, target: str) -> Credential:
raise RuntimeError("keyring is not installed")
monkeypatch.setattr(validate_module, "KeyringCredentialStore", _Store)
result = validate_module._check_keyring(config)
assert not result.ok
assert "keyring is not installed" in result.detail
def test_check_permissions_not_writable(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
def _no_access(_path: Any, _mode: int) -> bool:
return False
monkeypatch.setattr(validate_module.os, "access", _no_access)
result = validate_module._check_permissions(config)
assert not result.ok
assert "not read/write" in result.detail
def test_check_snapshots_ok(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
win_vmx = _make_template_vmx(config, "WinBuild2025")
backend = _FakeBackend({win_vmx: ["BaseClean"]})
monkeypatch.setattr(validate_module, "load_backend", lambda _c: backend)
result = validate_module._check_snapshots(config)
assert result.ok, result.detail
assert "WinBuild2025->BaseClean" in result.detail
def test_check_snapshots_missing_snapshot(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
lin_vmx = _make_template_vmx(config, "LinuxBuild2404")
backend = _FakeBackend({lin_vmx: ["SomethingElse"]})
monkeypatch.setattr(validate_module, "load_backend", lambda _c: backend)
result = validate_module._check_snapshots(config)
assert not result.ok
assert "BaseClean-Linux" in result.detail
def test_check_snapshots_no_templates(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
backend = _FakeBackend({})
monkeypatch.setattr(validate_module, "load_backend", lambda _c: backend)
result = validate_module._check_snapshots(config)
assert not result.ok
assert "no known template VMX found" in result.detail
def test_check_keyring_ok(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
class _Store:
def get(self, target: str) -> Credential:
return Credential(username="ci_build", password="x")
monkeypatch.setattr(validate_module, "KeyringCredentialStore", _Store)
result = validate_module._check_keyring(config)
assert result.ok
assert "ci_build" in result.detail
def test_check_keyring_missing(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
class _Store:
def get(self, target: str) -> Credential:
raise KeyError(target)
monkeypatch.setattr(validate_module, "KeyringCredentialStore", _Store)
result = validate_module._check_keyring(config)
assert not result.ok
def test_check_permissions_ok(tmp_path: Path) -> None:
config = _make_config(tmp_path)
result = validate_module._check_permissions(config)
assert result.ok, result.detail
def test_check_permissions_missing_dir(tmp_path: Path) -> None:
config = _make_config(tmp_path)
# Remove a required dir.
config.paths.build_vms.rmdir()
result = validate_module._check_permissions(config)
assert not result.ok
assert "missing" in result.detail
def test_check_units_no_systemctl(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(validate_module.shutil, "which", lambda _x: None)
result = validate_module._check_units()
assert not result.ok
assert "systemctl" in result.detail
def test_check_units_active(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(validate_module.shutil, "which", lambda _x: "/bin/systemctl")
monkeypatch.setattr(validate_module, "_systemctl_is_active", lambda _u: "active")
result = validate_module._check_units()
assert result.ok
def test_check_units_inactive(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(validate_module.shutil, "which", lambda _x: "/bin/systemctl")
monkeypatch.setattr(validate_module, "_systemctl_is_active", lambda _u: "inactive")
result = validate_module._check_units()
assert not result.ok
assert "inactive" in result.detail
def test_systemctl_is_active_invokes_subprocess(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(validate_module.shutil, "which", lambda _x: "/bin/systemctl")
class _CP:
stdout = "active\n"
monkeypatch.setattr(
validate_module.subprocess, "run", lambda *a, **k: _CP()
)
assert validate_module._systemctl_is_active("act-runner.service") == "active"
def test_systemctl_is_active_no_binary(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(validate_module.shutil, "which", lambda _x: None)
assert validate_module._systemctl_is_active("x") is None
def test_systemctl_is_active_timeout(monkeypatch: pytest.MonkeyPatch) -> None:
import subprocess as _sp
monkeypatch.setattr(validate_module.shutil, "which", lambda _x: "/bin/systemctl")
def _boom(*_a: Any, **_k: Any) -> Any:
raise _sp.TimeoutExpired(cmd="systemctl", timeout=10.0)
monkeypatch.setattr(validate_module.subprocess, "run", _boom)
assert validate_module._systemctl_is_active("x") is None
# ── validate host: end-to-end command ───────────────────────────────────────
def test_validate_host_all_pass(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
win_vmx = _make_template_vmx(config, "WinBuild2025")
backend = _FakeBackend({win_vmx: ["BaseClean"]})
class _Store:
def get(self, target: str) -> Credential:
return Credential("ci_build", "x")
monkeypatch.setattr(validate_module, "load_config", lambda: config)
monkeypatch.setattr(validate_module, "load_backend", lambda _c: backend)
monkeypatch.setattr(validate_module, "KeyringCredentialStore", _Store)
monkeypatch.setattr(validate_module.shutil, "which", lambda _x: "/bin/systemctl")
monkeypatch.setattr(validate_module, "_systemctl_is_active", lambda _u: "active")
result = CliRunner().invoke(cli, ["validate", "host"])
assert result.exit_code == 0, result.output
assert "All host checks passed" in result.output
def test_validate_host_reports_failures_and_exits_nonzero(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
def _raise(_c: Any) -> Any:
raise BackendNotAvailable("vmrun missing")
class _Store:
def get(self, target: str) -> Credential:
raise KeyError(target)
monkeypatch.setattr(validate_module, "load_config", lambda: config)
monkeypatch.setattr(validate_module, "load_backend", _raise)
monkeypatch.setattr(validate_module, "KeyringCredentialStore", _Store)
monkeypatch.setattr(validate_module.shutil, "which", lambda _x: None)
result = CliRunner().invoke(cli, ["validate", "host"])
assert result.exit_code == 1, result.output
assert "FAIL" in result.output
assert "check(s) failed" in result.output
# ── validate guest ──────────────────────────────────────────────────────────
def test_resolve_use_winrm_explicit() -> None:
assert validate_module._resolve_use_winrm(True, None) is True
assert validate_module._resolve_use_winrm(False, "windows") is False
def test_resolve_use_winrm_inferred() -> None:
assert validate_module._resolve_use_winrm(None, "linux") is False
assert validate_module._resolve_use_winrm(None, "windows") is True
def test_resolve_use_winrm_requires_hint() -> None:
import click as _click
with pytest.raises(_click.UsageError):
validate_module._resolve_use_winrm(None, None)
class _FakeTransport:
"""Records connect/run/close and raises a configured error on run."""
def __init__(self, *args: Any, raise_on_run: Exception | None = None, **kw: Any) -> None:
self._raise = raise_on_run
self.closed = False
def run(self, *_a: Any, **_k: Any) -> Any:
if self._raise is not None:
raise self._raise
class _R:
returncode = 0
return _R()
def close(self) -> None:
self.closed = True
def _install_keyring_store(monkeypatch: pytest.MonkeyPatch) -> None:
class _Store:
def get(self, target: str) -> Credential:
return Credential("Administrator", "pw")
monkeypatch.setattr(validate_module, "KeyringCredentialStore", _Store)
def test_probe_winrm_ok(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
config = _make_config(tmp_path)
_install_keyring_store(monkeypatch)
transport = _FakeTransport()
monkeypatch.setattr(validate_module, "WinRmTransport", lambda *a, **k: transport)
result = validate_module._probe_winrm("10.0.0.5", config)
assert result.ok
assert transport.closed
def test_probe_winrm_auth_fail(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
_install_keyring_store(monkeypatch)
transport = _FakeTransport(raise_on_run=TransportConnectError("auth denied"))
monkeypatch.setattr(validate_module, "WinRmTransport", lambda *a, **k: transport)
result = validate_module._probe_winrm("10.0.0.5", config)
assert not result.ok
assert "auth denied" in result.detail
def test_probe_winrm_command_fail(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
_install_keyring_store(monkeypatch)
transport = _FakeTransport(raise_on_run=TransportCommandError(1, "", "boom"))
monkeypatch.setattr(validate_module, "WinRmTransport", lambda *a, **k: transport)
result = validate_module._probe_winrm("10.0.0.5", config)
assert not result.ok
assert "command failed" in result.detail
def test_probe_winrm_credential_missing(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
class _Store:
def get(self, target: str) -> Credential:
raise KeyError(target)
monkeypatch.setattr(validate_module, "KeyringCredentialStore", _Store)
result = validate_module._probe_winrm("10.0.0.5", config)
assert not result.ok
assert "credential lookup failed" in result.detail
def test_probe_ssh_ok(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
config = _make_config(tmp_path)
transport = _FakeTransport()
monkeypatch.setattr(validate_module, "SshTransport", lambda *a, **k: transport)
result = validate_module._probe_ssh("10.0.0.6", config)
assert result.ok
assert transport.closed
def test_probe_ssh_connect_fail(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
transport = _FakeTransport(raise_on_run=TransportConnectError("no route"))
monkeypatch.setattr(validate_module, "SshTransport", lambda *a, **k: transport)
result = validate_module._probe_ssh("10.0.0.6", config)
assert not result.ok
assert "no route" in result.detail
def test_probe_ssh_command_fail(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
transport = _FakeTransport(raise_on_run=TransportCommandError(2, "", "nope"))
monkeypatch.setattr(validate_module, "SshTransport", lambda *a, **k: transport)
result = validate_module._probe_ssh("10.0.0.6", config)
assert not result.ok
assert "command failed" in result.detail
def test_validate_guest_ssh_ok(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
transport = _FakeTransport()
monkeypatch.setattr(validate_module, "load_config", lambda: config)
monkeypatch.setattr(validate_module, "SshTransport", lambda *a, **k: transport)
result = CliRunner().invoke(
cli, ["validate", "guest", "--host", "10.0.0.6", "--ssh"]
)
assert result.exit_code == 0, result.output
assert "OK" in result.output
def test_validate_guest_winrm_fail_exits_nonzero(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
_install_keyring_store(monkeypatch)
transport = _FakeTransport(raise_on_run=TransportConnectError("bad creds"))
monkeypatch.setattr(validate_module, "load_config", lambda: config)
monkeypatch.setattr(validate_module, "WinRmTransport", lambda *a, **k: transport)
result = CliRunner().invoke(
cli, ["validate", "guest", "--host", "10.0.0.5", "--winrm"]
)
assert result.exit_code == 1, result.output
assert "FAIL" in result.output
def test_validate_guest_requires_transport_hint(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
monkeypatch.setattr(validate_module, "load_config", lambda: config)
result = CliRunner().invoke(cli, ["validate", "guest", "--host", "10.0.0.5"])
assert result.exit_code != 0
assert "transport" in result.output.lower()
def test_validate_guest_infers_from_guest_os(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
config = _make_config(tmp_path)
transport = _FakeTransport()
monkeypatch.setattr(validate_module, "load_config", lambda: config)
monkeypatch.setattr(validate_module, "SshTransport", lambda *a, **k: transport)
result = CliRunner().invoke(
cli, ["validate", "guest", "--host", "10.0.0.6", "--guest-os", "linux"]
)
assert result.exit_code == 0, result.output