From 1c8fdcf028392aa1c0ead14a1d4a69dd31284542 Mon Sep 17 00:00:00 2001 From: Simone Date: Mon, 11 May 2026 18:35:11 +0200 Subject: [PATCH] docs: update documentation for Linux VM support (Sprint 9) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AGENTS.md (new): - Environment reference for AI agents: host specs, PS 5.1 constraints, dir structure, VM templates, transport layer, style rules - Lessons #9 (snapshot cold state) and #10 (vmrun list vs getGuestIPAddress) ARCHITECTURE.md: - Add SSH/SCP transport section (Linux VMs) - Update system diagram to include _Transport.psm1, SSH transport path - Network layout: add SSH port 22 for Linux VMs - Timeline: Linux readiness check phases documented CI-FLOW.md: - Prerequisites table: add Template VM (Linux), SSH key row - Step 6 (readiness): document Windows vs Linux phases - Step 7 (build) and Step 8 (artifacts): document Linux/SSH paths - ci-report-ip guestinfo mechanism referenced HOST-SETUP.md: - Directory tree: add LinuxBuild2404 template dir and keys/ - Post-setup checklist: add step 5 (SSH key generation + Linux template) - Fix duplicate step numbering (56, 67) - Reference Paths table: add Linux template VMX, snapshot, SSH key LINUX-TEMPLATE-SETUP.md: - Rewritten as implementation record (no longer a plan) - Fase A user-data: minimal (user+SSH key only, no packages/runcmd) - All phases marked complete with actual paths and snapshot names WINDOWS-TEMPLATE-SETUP.md: minor updates for consistency TODO.md: - Summary: update last-updated to 2026-05-11, Sprint 9 description - Linux Build VM row: all 5 items complete - Prossimi passi: Test Plan + §6.6 Tier-2 as next priorities --- AGENTS.md | 154 +++++++++++++++++++++++++++++++++ TODO.md | 56 ++++++------ docs/ARCHITECTURE.md | 35 +++++--- docs/CI-FLOW.md | 44 ++++++++-- docs/HOST-SETUP.md | 27 ++++-- docs/LINUX-TEMPLATE-SETUP.md | 113 ++++++++++++------------ docs/WINDOWS-TEMPLATE-SETUP.md | 112 ++++++++++++------------ 7 files changed, 376 insertions(+), 165 deletions(-) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..fda142a --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,154 @@ +# AGENTS.md — Environment Reference + +Questo file descrive l'ambiente su cui e' costruito il progetto. +Leggerlo prima di generare o modificare qualsiasi script. + +> **Istruzione per gli agenti**: questo file va tenuto aggiornato. +> Se durante il lavoro si scopre un errore nuovo, un vincolo non documentato, +> un pattern che ha causato problemi o una best-practice utile, aggiungerla +> nella sezione "Errori frequenti da evitare" o in una sezione apposita. +> Non aggiungere dettagli ovvi o gia' coperti; solo informazioni che +> impedirebbero errori concreti in sessioni future. + +--- + +## Host + +| Parametro | Valore | +| ------------------ | ----------------------------------------------------------------------- | +| OS | Windows 11 | +| Hardware | i9-10900X, 64 GB RAM, NVMe SSD | +| PowerShell | **5.1** (Windows PowerShell — NON Core/7) | +| VMware Workstation | Pro, `vmrun.exe` in `C:\Program Files (x86)\VMware\VMware Workstation\` | +| SSH client | `ssh.exe` / `scp.exe` built-in Windows 11 (PATH) | +| act_runner | v1.0.2 — `F:\CI\act_runner\act_runner.exe` | +| Gitea | `http://10.10.20.11:3100` / `https://gitea.emulab.it` | + +--- + +## PowerShell 5.1 — Vincoli critici + +Tutti gli script devono girare su **Windows PowerShell 5.1**. Le seguenti +sintassi sono disponibili solo in PS 7+ e NON vanno usate: + +| Costrutto PS 7+ | Alternativa PS 5.1 corretta | +| ------------------------- | ----------------------------------------- | +| `$x ??= 'default'` | `if ($null -eq $x) { $x = 'default' }` | +| `$a ?? $b` | `if ($null -ne $a) { $a } else { $b }` | +| `cmd1 && cmd2` | `cmd1; if ($LASTEXITCODE -eq 0) { cmd2 }` | +| `cmd1 \|\| cmd2` | `cmd1; if ($LASTEXITCODE -ne 0) { cmd2 }` | +| `$x is [int]` | `$x -is [int]` | +| Ternary `$a ? $b : $c` | `if ($a) { $b } else { $c }` | +| `foreach-object` con `&&` | pipeline classico con `-ErrorAction Stop` | + +Ogni script deve avere in testa: + +```powershell +#Requires -Version 5.1 +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' +``` + +--- + +## Struttura directory host (fissa) + +``` +F:\CI\ + act_runner\ — binario e stato act_runner + Templates\ + WinBuild2025\ — template Windows Server 2025 + WinBuild2022\ — template Windows Server 2022 + LinuxBuild2404\ — template Ubuntu 24.04 + BuildVMs\ — clone efimeri (creati/distrutti per ogni job) + Artifacts\ — artifact raccolti (per job-id) + ISO\ — ISO/VMDK di bootstrap (cache download) + keys\ + ci_linux — chiave SSH privata per guest Linux (no passphrase) + ci_linux.pub — chiave pubblica corrispondente +``` + +--- + +## VM Template + +### Windows (WinBuild2025 — primario) + +| Parametro | Valore | +| -------------- | ------------------------------------------------------------------------------- | +| OS | Windows Server 2025 | +| VMX | `F:\CI\Templates\WinBuild2025\WinBuild2025.vmx` | +| Snapshot clone | `BaseClean` | +| Transport | WinRM HTTPS porta 5986, cert self-signed (lab-only) | +| Credenziali | Windows Credential Manager target `BuildVMGuest` | +| Rete | VMnet8 NAT — `192.168.79.0/24` | +| Toolchain | VS Build Tools 2026 (MSBuild 18.5 / v145), .NET SDK 10, Python 3.13, Git, 7-Zip | + +Variante **WinBuild2022**: path `F:\CI\Templates\WinBuild2022\WinBuild2022.vmx`, stesso snapshot `BaseClean`. + +### Linux (LinuxBuild2404) + +| Parametro | Valore | +| -------------- | ------------------------------------------------------------- | +| OS | Ubuntu 24.04 LTS (cloud image) | +| VMX | `F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx` | +| Snapshot clone | `BaseClean-Linux` | +| Transport | SSH porta 22, utente `ci_build`, chiave `F:\CI\keys\ci_linux` | +| Rete | VMnet8 NAT — `192.168.79.0/24` | +| Toolchain | GCC, G++, Clang, CMake, Ninja, Python 3, Git, 7-Zip | +| CI dirs | `/opt/ci/{build,output,scripts,cache}` — owner `ci_build` | + +--- + +## Transport layer + +- **Windows guest**: WinRM (`New-PSSession` con `New-CISessionOption` da `_Common.psm1`). + Usare `Invoke-Command -Session`, `Copy-Item -ToSession / -FromSession`. +- **Linux guest**: SSH/SCP via `ssh.exe` / `scp.exe`. + Usare le funzioni di `scripts/_Transport.psm1`: + `Invoke-SshCommand`, `Copy-SshItem`, `Test-SshReady`. +- **vmrun.exe**: sempre invocato tramite `Invoke-Vmrun` da `_Common.psm1` + (wrapper uniforme che gestisce `$LASTEXITCODE` e output). + +NON modificare il branch WinRM esistente quando si lavora sul branch Linux e viceversa. + +--- + +## act_runner / Gitea Actions + +- Labels runner: `windows-build:host`, `linux-build:host` +- Env vars iniettate in ogni job (da `runner/config.yaml`): + - `GITEA_CI_TEMPLATE_PATH` — VMX Windows template + - `GITEA_CI_LINUX_TEMPLATE_PATH` — VMX Linux template + - `GITEA_CI_CLONE_BASE_DIR` — `F:\CI\BuildVMs` + - `GITEA_CI_ARTIFACT_DIR` — `F:\CI\Artifacts` + - `GITEA_CI_GUEST_CRED_TARGET` — `BuildVMGuest` + - `GITEA_CI_SSH_KEY_PATH` — `F:\CI\keys\ci_linux` + +--- + +## Regole di stile (PSScriptAnalyzer) + +Configurazione in `PSScriptAnalyzerSettings.psd1`. Regole principali: + +- Indentazione: **4 spazi** (no tab) +- `Write-Host` e' permesso (output catturato da act_runner; `Write-Output` interferisce con i return value) +- `Invoke-Expression` vietato +- Funzioni state-changing devono supportare `-WhatIf` / `ShouldProcess` +- Credenziali solo come `[PSCredential]`, mai plain-text string +- Nessuna variabile globale (`$global:`) + +--- + +## Errori frequenti da evitare + +1. **Sintassi PS 7+** in script PS 5.1 — vedi tabella sopra. +2. **`$LASTEXITCODE` non controllato** dopo `vmrun`, `ssh`, `scp` — controllare sempre e fare `throw` se non zero. +3. **Percorsi hardcoded senza parametro** — ogni path critico deve avere un parametro con default documentato. +4. **Modificare entrambi i branch (WinRM e SSH) nello stesso script** — mantenerli separati con `if ($GuestOS -eq 'Linux')`. +5. **Usare `New-PSSession` / WinRM verso host Linux** — Linux usa solo SSH tramite `_Transport.psm1`. +6. **Non rimuovere il seed ISO dal VMX** dopo il primo boot Linux — `ide1:0.present = "FALSE"` deve essere settato dopo cloud-init. +7. **`[ordered]@{}` non disponibile in PS 2** — non e' un problema qui (PS 5.1), ma ricordare che `[ordered]` e' PS 3+. +8. **`ForEach-Object -Parallel`** — disponibile solo in PS 7+; usare `foreach` classico o `Start-Job`. +9. **Snapshot template catturato con VM accesa** — produce file `*.vmem` / `*.vmsn` con memoria; `vmrun clone ... linked -snapshot=` rifiuta con `Error: The virtual machine should not be powered on. It is already running.` anche se `vmrun list` mostra 0 VM. Catturare snapshot SOLO da stato fully powered-off (verificare assenza di `.vmem` nella dir template). +10. **`vmrun getGuestIPAddress` per check "VM running"** — richiede VMware Tools attivi nel guest; in headless Tools puo' impiegare 30-60s a rispondere e il check appare bloccato. Usare `vmrun list` e cercare il path del VMX nell'output. diff --git a/TODO.md b/TODO.md index d7115f7..ae1a52c 100644 --- a/TODO.md +++ b/TODO.md @@ -1,6 +1,6 @@ # TODO — Local CI/CD System - + > Documento unico di lavoro: roadmap, audit trail dei task completati, e backlog post-v1.0 > con priorità e razionale. Le voci aperte sono raggruppate per area e marcate **P0..P3**: @@ -8,7 +8,7 @@ > **Doc di setup operativi**: > - [docs/HOST-SETUP.md](docs/HOST-SETUP.md) — bootstrap macchina host (Setup-Host.ps1) > - [docs/WINDOWS-TEMPLATE-SETUP.md](docs/WINDOWS-TEMPLATE-SETUP.md) — provisioning template Windows (Deploy + Prepare + Setup-WinBuild2025) -> - [docs/LINUX-TEMPLATE-SETUP.md](docs/LINUX-TEMPLATE-SETUP.md) — bozza/TODO template Linux (non implementato) +> - [docs/LINUX-TEMPLATE-SETUP.md](docs/LINUX-TEMPLATE-SETUP.md) — provisioning template Linux Ubuntu 24.04 (implementato) > > - **P0** — sicurezza o affidabilità: affrontare prima di estendere il sistema > - **P1** — operatività quotidiana: automatizzare manutenzione e debug @@ -18,7 +18,7 @@ --- ## Summary -_Last updated: 2026-05-10 (post Sprint 8: §3.3 e2e PASS — 34s/25.7% faster, §6.6 Git+7-Zip)_ +_Last updated: 2026-05-11 (post Sprint 9: §6.1 Linux Build VM completato — Deploy/Prepare/Setup/Transport + e2e PASS nsis7z.dll 2272 KB in 03:09)_ **Stato generale**: §1, §2, §4, §5 chiusi. §3 perf: 4/6 done (3.1/3.2/3.3/3.6; 3.4/3.5 deferred). §3.3 e2e validato: -UseGitClone 34s più veloce del baseline. §6.6 Tier-1 (Git+7-Zip) installato nel template. Prossimi: test plan completo, §6.6 Tier-2. @@ -34,10 +34,10 @@ _Last updated: 2026-05-10 (post Sprint 8: §3.3 e2e PASS — 34s/25.7% faster, | [x] | §3 Performance & throughput | 4 | 0 | 3.1/3.2/3.3/3.6 done; 3.4/3.5 deferred — **§3 CHIUSO** | | [x] | §4 Osservabilità & manutenzione | 3 | 0 | 4.1/4.3/4.4 done (4.2 Prometheus, 4.5 Dashboard rimossi) | | [x] | §5 Qualità codice & test | 5 | 0 | 5.1/5.2/5.3/5.4/5.5 done | -| [ ] | §6 Scalabilità & estensioni | 1 | 5 | 6.6 Tier-1 (Git+7-Zip) done; 6.1 (Linux), 6.2-6.5 TODO | +| [ ] | §6 Scalabilità & estensioni | 2 | 4 | 6.1 (Linux) done, 6.6 Tier-1 (Git+7-Zip) done; 6.2-6.5 TODO | | [ ] | §7 Refactor Deploy ↔ Setup | 4 | 1 | 7.1/7.2/7.3/7.4 done; 7.5 (rinomina) opzionale | | [x] | In-VM Git Clone (§3.3) | 4 | 0 | COMPLETATO 2026-05-10 — e2e PASS: 34s/25.7% vs baseline | -| [ ] | Linux Build VM (§6.1) | 0 | 4 | bozza in `docs/LINUX-TEMPLATE-SETUP.md` | +| [x] | Linux Build VM (§6.1) | 5 | 0 | Deploy/Prepare/Setup/Transport scripts + doc updates completati | **Prossimi passi consigliati** (vedi anche "Suggerimento di sequencing" in fondo): 1. **Test Plan** — eseguire completo `docs/TEST-PLAN-v1.3-to-HEAD.md` (25+ Pester tests, validazione features) @@ -450,11 +450,11 @@ validation (`[ValidateRange]`, `[ValidatePattern]`, `[ValidateScript]`) + ## 6. Scalabilità & estensioni -### 6.1 [P2] [ ] Linux Build VM +### 6.1 [P2] [x] Linux Build VM **Documento di pianificazione dedicato**: [docs/LINUX-TEMPLATE-SETUP.md](docs/LINUX-TEMPLATE-SETUP.md) — bozza completa con fasi A→H, decisioni di design, differenze rispetto al template Windows. -Vedi anche sezione "Linux Build VM" sotto per il riassunto inline. +Implementato — vedi sezione "Linux Build VM" sotto per il riassunto inline. **Considerazione architetturale**: SSH su WSL2 non basta — serve VM vera per parità con il flusso Windows. Ubuntu 24.04 minimal con script bash diretto (parità con flusso Windows). @@ -684,31 +684,33 @@ VM avviata → WinRM: git clone --recurse-submodules https://@gitea/... --- -## Linux Build VM (Future) — riferimento §6.1 +## Linux Build VM — riferimento §6.1 [IMPLEMENTATO] -Aggiungere supporto per build su Linux per testare la compilazione cross-platform -(es. versione Linux del plugin o build check con GCC/Clang). +Supporto build su Linux aggiunto con le seguenti fasi completate: -- [ ] **Template VM Linux** — Provisionare template VM con Ubuntu Server 24.04 LTS - - [ ] Scegliere distro: Ubuntu 24.04 LTS (raccomandato) o Debian 12 - - [ ] Installare toolchain: GCC, Clang, CMake, Python 3.x, build-essential - - [ ] Configurare accesso SSH (chiave pubblica da host) in alternativa/integrazione a WinRM - - [ ] Prendere snapshot `BaseClean-Linux` - - [ ] Archiviare credenziali SSH in Credential Manager (o key file su host) +- [x] **Template VM Linux** — script di provisioning per Ubuntu Server 24.04 LTS + - [x] `template/Deploy-LinuxBuild2404.ps1` — crea VM da cloud VMDK, cloud-init, SSH wait + - [x] `template/Setup-LinuxBuild2404.sh` — installa toolchain CI (GCC, Clang, CMake, Python 3, git, 7z), hardening SSH, swap off, timer apt off + - [x] `template/Prepare-LinuxBuild2404.ps1` — SCP script, esegue via SSH, valida, snapshot `BaseClean-Linux` -- [ ] **Invoke-CIJob.ps1 — branch Linux** - - [ ] Aggiungere parametro `-GuestOS` (`Windows` / `Linux`) o rilevamento automatico da VMX - - [ ] Sostituire WinRM con SSH + SCP per trasferimento zip e raccolta artifacts - - [ ] Aggiungere supporto a comandi Linux (`bash`, `make`, `cmake --build`) - - [ ] Astrarre transport in `scripts/_Transport.psm1` (vedi §6.1) +- [x] **Invoke-CIJob.ps1 — branch Linux** + - [x] Parametro `-GuestOS` (Windows/Linux/Auto) con auto-detect dal VMX `guestOS` field + - [x] Transport SSH via `scripts/_Transport.psm1` (Invoke-SshCommand, Copy-SshItem, Test-SshReady) + - [x] Phase 4/5/6 branch su GuestOS: SSH per Linux, WinRM per Windows (nessuna regressione) -- [ ] **Workflow Gitea — build matrix** - - [ ] Definire matrix strategy `[windows-build, linux-build]` nel workflow (vedi §6.4) - - [ ] Aggiungere label `linux-build` al runner (o registrare runner separato se necessario) - - [ ] Verificare che artifacts da entrambe le piattaforme vengano raccolti correttamente +- [x] **Workflow Gitea — build matrix** + - [x] Label `linux-build:host` aggiunto a `runner/config.yaml` + - [x] `GITEA_CI_LINUX_TEMPLATE_PATH` e `GITEA_CI_SSH_KEY_PATH` aggiunti alle env vars del runner + - [x] Matrix strategy cross-platform (windows + linux) documentata in `gitea/workflow-example.yml` -- [ ] **Test e2e Linux** — eseguire almeno un build end-to-end su VM Linux - - [ ] Confermare che `build_plugin.py` (o analogo script) funzioni su Linux +- [x] **Doc updates** + - [x] `docs/ARCHITECTURE.md` — SSH transport, _Transport.psm1, Linux VM boxes, topology SSH :22 + - [x] `docs/CI-FLOW.md` — Step 0: Linux template + SSH key; Step 6 SSH transport note; Steps 7-8 Linux branch + - [x] `docs/HOST-SETUP.md` — directory layout: keys\ + LinuxBuild2404\; SSH key gen step; Reference Paths Linux + - [x] `docs/LINUX-TEMPLATE-SETUP.md` — Status: implementato; tutte le checkbox Fasi A-E marcate [x] + +- [ ] **Test e2e Linux** — eseguire almeno un build end-to-end su VM Linux (pendente provisioning fisico VM) + - [ ] Confermare che la build di esempio funzioni su Linux - [ ] Confrontare artifact Linux con quelli Windows --- diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index d50d00e..2b8d7d1 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -6,8 +6,9 @@ A fully local, isolated CI/CD pipeline running on a Windows host using: - **Gitea** as the self-hosted Git server + CI platform - **act_runner** as the job executor (runs on the Windows host) - **VMware Workstation** for ephemeral build VMs (linked clones) -- **WinRM / PowerShell Remoting** for communication with build VMs -- **MSBuild / dotnet CLI** executing inside the guest VM only +- **WinRM / PowerShell Remoting** for communication with Windows build VMs +- **SSH / SCP** (`ssh.exe` + `scp.exe`) for communication with Linux build VMs +- **MSBuild / dotnet CLI** (Windows) and **GCC / Clang / CMake** (Linux) executing inside guest VMs only The runner host **never executes build tools directly**. Its only role is orchestration: creating, driving, and destroying VMs. @@ -38,10 +39,11 @@ The runner host **never executes build tools directly**. Its only role is orches │ │ Orchestrator Scripts (PowerShell) │ │ │ │ │ │ │ │ New-BuildVM.ps1 ──► vmrun.exe (linked clone from template) │ │ -│ │ Wait-VMReady.ps1 ─► Test-WSMan poll loop │ │ -│ │ Invoke-RemoteBuild.ps1 ► PSSession + Invoke-Command │ │ -│ │ Get-BuildArtifacts.ps1 ► Copy-Item -FromSession │ │ -│ │ Remove-BuildVM.ps1 ───► vmrun stop + deleteVM │ │ + │ Wait-VMReady.ps1 ─► WinRM poll (Windows) / SSH poll (Linux) │ │ + │ Invoke-RemoteBuild.ps1 ► PSSession (Windows) / SSH+SCP (Linux) │ │ + │ Get-BuildArtifacts.ps1 ► Copy-Item -FromSession / scp (Linux) │ │ + │ Remove-BuildVM.ps1 ───► vmrun stop + deleteVM │ │ + │ _Transport.psm1 ──────► Invoke-SshCommand / Copy-SshItem │ │ │ └───────────────────────────────┬───────────────────────────────────┘ │ │ │ vmrun.exe (VMware CLI) │ │ ▼ │ @@ -103,8 +105,8 @@ The runner host **never executes build tools directly**. Its only role is orches - Disk footprint: ~2–5 GB per clone (vs 40–80 GB for full clone) - Tradeoff: template snapshot must remain intact -### WinRM / PowerShell Remoting -- Default transport for remote build execution inside VMs +### WinRM / PowerShell Remoting (Windows VMs) +- Transport for remote build execution inside **Windows** build VMs - Port: **5986** (HTTPS, self-signed cert, `AllowUnencrypted=false`) - Authentication: **Basic over HTTPS** (`-SkipCACheck` for lab self-signed cert) - Used for: @@ -112,6 +114,14 @@ The runner host **never executes build tools directly**. Its only role is orches - Running build commands (`Invoke-Command`) - Packaging artifacts (`Compress-Archive` in guest) and collecting via `Copy-Item -FromSession` +### SSH / SCP Transport (Linux VMs) +- Transport for remote build execution inside **Linux** build VMs +- Port: **22** (key-based auth, ed25519 key `F:\CI\keys\ci_linux`, user `ci_build`) +- No password — `BatchMode=yes`, `StrictHostKeyChecking=accept-new` +- Implemented in `scripts/_Transport.psm1` (exported: `Invoke-SshCommand`, `Copy-SshItem`, `Test-SshReady`) +- Used by `Wait-VMReady.ps1 -Transport SSH`, `Invoke-RemoteBuild.ps1 -GuestOS Linux`, `Get-BuildArtifacts.ps1 -GuestOS Linux` +- Selected automatically when `Invoke-CIJob.ps1` detects `guestOS = ubuntu-64` in the cloned VMX + ### Build Toolchain (inside VM only) - Visual Studio Build Tools **2026** (MSBuild 18.5.4 / toolset v145) - .NET SDK **10.0.203** @@ -133,8 +143,9 @@ Build VMs: ... Clone_Job_N → 192.168.79.1xx -WinRM port 5986 (HTTPS, self-signed) on each VM IP. -VMs have internet access via NAT — required for pip/nuget during build. +WinRM port 5986 (HTTPS, self-signed) on each Windows VM IP. +SSH port 22 on each Linux VM IP (key-based, ci_build@, key: F:\CI\keys\ci_linux). +VMs have internet access via NAT — required for pip/nuget/apt during build. ``` > **Note:** VMnet8 NAT gives VMs internet access. The host detects the VM IP via @@ -157,7 +168,8 @@ VMs have internet access via NAT — required for pip/nuget during build. Duration: ~20–40 seconds boot │ │ 4. READINESS CHECK │ - Poll: getState=running → ping → Test-WSMan │ + Windows: getState=running → ping → Test-WSMan │ + Linux: getState=running → TCP:22 → ssh echo │ Duration: ~30–90 seconds total │ │ 5. BUILD │ @@ -193,3 +205,4 @@ VMs have internet access via NAT — required for pip/nuget during build. - Runner host does not expose any ports to the build VM network - Source code is transferred via zip (Compress-Archive → WinRM copy → Expand-Archive in guest) by default — no PAT is ever present inside the VM; VMs have NAT internet for pip/nuget at build time. - With `-UseGitClone` (§3.3, opt-in) il clone avviene dentro la VM via SSH alias `gitea-ci` (forwardato dall'host); la chiave privata SSH dell'host non viene esposta al guest — vedi BEST-PRACTICES.md per il threat model. +- Linux build VMs usano la chiave SSH `F:\CI\keys\ci_linux` (ed25519, no passphrase), archiviata solo sull'host. Il guest non conosce credenziali WinRM. diff --git a/docs/CI-FLOW.md b/docs/CI-FLOW.md index 35d1cfb..4b9d0b8 100644 --- a/docs/CI-FLOW.md +++ b/docs/CI-FLOW.md @@ -12,8 +12,10 @@ Before any CI job runs, the following must be in place: | ------------------ | ----------------------------------------------------------------------- | | Gitea server | Running, accessible on LAN | | act_runner | Registered, running as Windows service on host | -| Template VM | Provisioned, snapshot "BaseClean" taken, VM powered off | -| WinRM | Enabled on template VM (inherits to all clones) | +| Template VM (Win) | Provisioned, snapshot "BaseClean" taken, VM powered off | +| Template VM (Lin) | Ubuntu 24.04 LTS, snapshot "BaseClean-Linux" taken, VM powered off (optional) | +| WinRM | Enabled on Windows template VM (inherits to all clones) | +| SSH key | `F:\CI\keys\ci_linux` (ed25519) present; public key in Linux template | | vmrun.exe | Present at `C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe` | | `F:\CI\BuildVMs\` | Directory exists and writable | | `F:\CI\Artifacts\` | Directory exists and writable | @@ -110,12 +112,19 @@ Wait-VMReady.ps1 -VMPath -IPAddress -TimeoutSeconds IP is discovered dynamically via `vmrun getGuestIPAddress` (VMnet8 NAT, subnet 192.168.79.0/24). No hardcoded IPs — each clone gets a DHCP-assigned address from VMware NAT. -Three-phase readiness check (retried every 5 seconds, up to 300s timeout): +Three-phase readiness check (retried every 5 seconds, up to 300s timeout). +Transport is selected automatically from the VMX `guestOS` field (`-Transport WinRM` for Windows, `-Transport SSH` for Linux): ``` Phase 1: vmrun getState → must return "running" -Phase 2: Test-Connection (ICMP ping) → must succeed -Phase 3: Test-WSMan → WinRM listener must respond + +Windows (-Transport WinRM, default): + Phase 2: Test-Connection (ICMP ping) → must succeed + Phase 3: Test-WSMan → WinRM listener must respond + +Linux (-Transport SSH): + Phase 2: Test-NetConnection -Port 22 → must succeed + Phase 3: ssh -o ConnectTimeout=5 ... "echo ready" → must return "ready" ``` - Total wait is typically **30–90 seconds** after `vmrun start` @@ -129,9 +138,18 @@ Phase 3: Test-WSMan → WinRM listener must respond ``` Invoke-RemoteBuild.ps1 -IPAddress # from vmrun getGuestIPAddress + -GuestOS Windows|Linux # auto-detected from VMX guestOS field + + # Windows path (default): -Credential (from Windows Credential Manager) -BuildCommand 'python build_plugin.py --final --dist-dir dist' -GuestArtifactSource 'dist' + + # Linux path (-GuestOS Linux): + -SshKeyPath F:\CI\keys\ci_linux + -SshUser ci_build + -CloneUrl / -CloneBranch # always in-VM git clone for Linux + -BuildCommand 'make all' ``` Sequence inside the PSSession: @@ -163,14 +181,22 @@ Sequence inside the PSSession: ``` Get-BuildArtifacts.ps1 -IPAddress - -Credential (same as above) + -GuestOS Windows|Linux # auto-detected from Invoke-CIJob + + # Windows path (default): + -Credential (from Windows Credential Manager) -GuestArtifactPath C:\CI\output\artifacts.zip + + # Linux path (-GuestOS Linux): + -SshKeyPath F:\CI\keys\ci_linux + -GuestArtifactPath /opt/ci/output + -HostArtifactDir F:\CI\Artifacts\{JobId}\ ``` -- Apre PSSession e copia `C:\CI\output\artifacts.zip` dall'host -- Valida che il file esista e sia non-vuoto -- Il zip contiene la struttura `dist/` con tutti gli artifact di build +- Windows: apre PSSession e copia `C:\CI\output\artifacts.zip` tramite WinRM +- Linux: `scp -r ci_build@:/opt/ci/output/. ` tramite `_Transport.psm1` +- In entrambi i casi valida che almeno un file sia stato trasferito --- diff --git a/docs/HOST-SETUP.md b/docs/HOST-SETUP.md index 3116bf4..8e729b9 100644 --- a/docs/HOST-SETUP.md +++ b/docs/HOST-SETUP.md @@ -33,7 +33,11 @@ F:\CI\ ├── Artifacts\ # output build raccolti dall'host ├── Logs\ # log per-job (retention: 30 giorni) ├── Templates\ -│ └── WinBuild2025\ # VMX template Windows (immutabile dopo snapshot) +│ ├── WinBuild2025\ # VMX template Windows (immutabile dopo snapshot) +│ └── LinuxBuild2404\ # VMX template Linux Ubuntu 24.04 (immutabile dopo snapshot) +├── keys\ +│ ├── ci_linux # chiave privata SSH per Linux VMs (ed25519, no passphrase) +│ └── ci_linux.pub # chiave pubblica corrispondente ├── act_runner\ │ ├── act_runner.exe # binario runner Gitea │ ├── config.yaml # config runner (path, label, capacity) @@ -125,13 +129,21 @@ Allo `Setup-Host.ps1` exit 0: ``` *Note*: Prepare-WinBuild2025.ps1 aggiunge l'IP specifico della VM in modo transitorio e lo ripristina nel `finally`. -4. **Provisiona il template VM** — vedi [WINDOWS-TEMPLATE-SETUP.md](WINDOWS-TEMPLATE-SETUP.md) +4. **Provisiona il template VM Windows** — vedi [WINDOWS-TEMPLATE-SETUP.md](WINDOWS-TEMPLATE-SETUP.md) -5. **Registra chiave SSH con Gitea**: +5. **(Opzionale) Genera chiave SSH CI per Linux VMs** — necessaria solo se si vuole usare `linux-build`: + ```powershell + New-Item -ItemType Directory -Force F:\CI\keys | Out-Null + ssh-keygen -t ed25519 -f F:\CI\keys\ci_linux -N '""' -C 'ci-linux-runner' + # Risultato: F:\CI\keys\ci_linux (privata) + F:\CI\keys\ci_linux.pub (pubblica) + ``` + Poi **provisiona il template VM Linux** — vedi [LINUX-TEMPLATE-SETUP.md](LINUX-TEMPLATE-SETUP.md) + +6. **Registra chiave SSH con Gitea**: - Gitea UI → User Settings → SSH Keys → Add Key - Chiave pubblica: `%USERPROFILE%\.ssh\id_rsa.pub` -6. **Verifica runner online** in Gitea: +7. **Verifica runner online** in Gitea: - `/-/admin/runners` - Status atteso: **Online**, label `windows-build:host,dotnet:host,msbuild:host` @@ -145,8 +157,11 @@ Allo `Setup-Host.ps1` exit 0: | act_runner exe | `F:\CI\act_runner\act_runner.exe` (servizio: `act_runner`) | | act_runner config | `F:\CI\act_runner\config.yaml` | | act_runner logs | `F:\CI\act_runner\logs\` (stdout.log, stderr.log) | -| Template VMX | `F:\CI\Templates\WinBuild2025\WinBuild2025.vmx` | -| Snapshot name | `BaseClean` | +| Template VMX (Windows) | `F:\CI\Templates\WinBuild2025\WinBuild2025.vmx` | +| Template VMX (Linux) | `F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx` | +| Snapshot name (Windows) | `BaseClean` | +| Snapshot name (Linux) | `BaseClean-Linux` | +| SSH key (Linux VMs) | `F:\CI\keys\ci_linux` (privata), `F:\CI\keys\ci_linux.pub` | | Clone base dir | `F:\CI\BuildVMs\` | | Artifact dir | `F:\CI\Artifacts\` | | Log dir | `F:\CI\Logs\` (retention: 30 giorni) | diff --git a/docs/LINUX-TEMPLATE-SETUP.md b/docs/LINUX-TEMPLATE-SETUP.md index a8cb389..a33b4ef 100644 --- a/docs/LINUX-TEMPLATE-SETUP.md +++ b/docs/LINUX-TEMPLATE-SETUP.md @@ -1,6 +1,6 @@ # Linux Template VM Setup Piano di implementazione -> **Status**: pianificato non ancora implementato. Vedi [TODO.md §6.1](../TODO.md) (P2). +> **Status**: implementato (2026-05-11) — Fasi A-E complete. Fase F (test e2e) richiede provisioning fisico VM. Vedi [TODO.md §6.1](../TODO.md) (P2). > Distro scelta: **Ubuntu 24.04 LTS** (cloud image, cloud-init). Approccio simmetrico a Windows. --- @@ -12,7 +12,7 @@ | `template/Deploy-LinuxBuild2404.ps1` | **Host** (PS) | Crea VM da cloud image VMDK, genera seed ISO cloud-init, avvia, aspetta SSH | | `template/Prepare-LinuxBuild2404.ps1` | **Host** (PS) | Copia + esegue setup via SSH, valida stato, prende snapshot | | `template/Setup-LinuxBuild2404.sh` | **Guest** (Bash) | Installa toolchain CI, dirs, hardening, gate finale | -| `scripts/_Transport.psm1` | **Host** (PS) | Funzioni SSH-only per il branch Linux (WinRM esistente invariato per ora) | +| `scripts/_Transport.psm1` | **Host** (PS) | Funzioni SSH-only per il branch Linux (WinRM esistente invariato per ora) | Perche tre script separati (non tutto-in-uno): cloud-init copre solo il bootstrap OS (utente, SSH key, pacchetti base). Il toolchain CI e le validazioni per step sono piu @@ -26,7 +26,7 @@ Deploy/Setup/Prepare di Windows. | Parametro | Valore | | ------------- | ---------------------------------------------------- | | OS | Ubuntu 24.04 LTS (cloud image, non ISO installer) | -| Immagine base | `ubuntu-24.04-server-cloudimg-amd64.vmdk` | +| Immagine base | `noble-server-cloudimg-amd64.vmdk` | | Download | `https://cloud-images.ubuntu.com/noble/current/` | | vCPU | 4 | | RAM | 4096 MB | @@ -44,19 +44,19 @@ Linux ha il driver vmxnet3 nativo. ## Prerequisiti host -- [ ] **SSH key CI dedicata** generare una volta sola (no passphrase): +- [x] **SSH key CI dedicata** generare una volta sola (no passphrase): ```powershell New-Item -ItemType Directory -Force F:\CI\keys | Out-Null ssh-keygen -t ed25519 -f F:\CI\keys\ci_linux -N '""' -C 'ci-linux-runner' # Risultato: F:\CI\keys\ci_linux (privata) + F:\CI\keys\ci_linux.pub (pubblica) ``` -- [ ] **OpenSSH client** verificato presente sull'host (default su Windows 11): +- [x] **OpenSSH client** verificato presente sull'host (default su Windows 11): ```powershell Get-WindowsCapability -Online -Name OpenSSH.Client* | Select-Object State # Atteso: State = Installed ``` -- [ ] **`F:\CI\Templates\LinuxBuild2404\`** creata (Setup-Host.ps1 o manuale) -- [ ] **`F:\CI\ISO\`** creata (il VMDK viene scaricato automaticamente da Deploy se mancante) +- [x] **`F:\CI\Templates\LinuxBuild2404\`** creata (Setup-Host.ps1 o manuale) +- [x] **`F:\CI\ISO\`** creata (il VMDK viene scaricato automaticamente da Deploy se mancante) --- @@ -65,7 +65,7 @@ Linux ha il driver vmxnet3 nativo. Script host-side. Produce una VM pronta con cloud-init completato e SSH raggiungibile. Non richiede installazione OS parte dal cloud VMDK pre-built. -- [ ] **Step 1 Validate prerequisites** +- [x] **Step 1 Validate prerequisites** - Parametro `-VMwareWorkstationDir` (default `C:\Program Files (x86)\VMware\VMware Workstation`) — stesso pattern di Deploy-WinBuild2025.ps1 - `vmrun.exe` presente in `$VMwareWorkstationDir` - `vmware-vdiskmanager.exe` presente in `$VMwareWorkstationDir` @@ -74,10 +74,10 @@ Non richiede installazione OS parte dal cloud VMDK pre-built. - Cartella destinazione VMX non gia occupata (o `-Force` per sovrascrivere) - (il VMDK sorgente viene gestito in Step 1b — nessun prerequisito manuale) -- [ ] **Step 1b Download cloud VMDK se assente** (`F:\CI\ISO\` come cache locale) +- [x] **Step 1b Download cloud VMDK se assente** (`F:\CI\ISO\` come cache locale) ```powershell - $VmdkUrl = 'https://cloud-images.ubuntu.com/noble/current/ubuntu-24.04-server-cloudimg-amd64.vmdk' - $VmdkPath = 'F:\CI\ISO\ubuntu-24.04-server-cloudimg-amd64.vmdk' + $VmdkUrl = 'https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.vmdk' + $VmdkPath = 'F:\CI\ISO\noble-server-cloudimg-amd64.vmdk' if (-not (Test-Path $VmdkPath)) { Write-Host "[Deploy] VMDK non trovato, avvio download..." New-Item -ItemType Directory -Force F:\CI\ISO | Out-Null @@ -99,32 +99,33 @@ Non richiede installazione OS parte dal cloud VMDK pre-built. - In caso di errore: il file `.part` viene rimosso, l'eccezione termina lo script - Parametro opzionale `-VmdkUrl` per sovrascrivere l'URL (es. mirror locale) -- [ ] **Step 2 Genera cloud-init user-data + meta-data** (inline, no template esterno) +- [x] **Step 2 Genera cloud-init user-data + meta-data** (inline, no template esterno) - `meta-data`: `instance-id: linuxbuild-001`, `local-hostname: ci-linux-template` - `user-data` (YAML): - - `hostname: ci-linux-template`, `timezone: Europe/Rome` + - `hostname: ci-linux-template` - `users`: utente `ci_build`, shell `/bin/bash`, `sudo: ALL=(ALL) NOPASSWD:ALL`, `ssh_authorized_keys` con contenuto di `ci_linux.pub` - - `packages`: `[openssh-server, curl, ca-certificates, p7zip-full]` - - `package_update: true` - - `runcmd`: `swapoff -a`, commenta swap in `/etc/fstab`, - `sed PasswordAuthentication no` in sshd_config, `systemctl restart sshd` - - `power_state`: `mode: reboot` dopo primo provisioning + - Nessun `packages`, nessun `package_update`, nessun `runcmd`: + tutto il provisioning (apt, toolchain, SSH hardening, swap) e' delegato a + `Prepare-LinuxBuild2404.ps1` via `Setup-LinuxBuild2404.sh` + - Nessun `power_state`: il reboot non e' necessario, cloud-init completa tutto al primo boot -- [ ] **Step 3 Costruisce seed ISO nocloud** via IMAPI2FS COM +- [x] **Step 3 Costruisce seed ISO nocloud** via IMAPI2FS COM - Due file: `user-data` + `meta-data` (volume label: `cidata`) - Stesso codice COM gia usato in Deploy-WinBuild2025.ps1 Step 3 - Output: `F:\CI\Templates\LinuxBuild2404\cloud-init-seed.iso` -- [ ] **Step 4 Copia e ridimensiona cloud VMDK** - - Copia `ubuntu-24.04-server-cloudimg-amd64.vmdk` → `LinuxBuild2404.vmdk` +- [x] **Step 4 Converti e ridimensiona cloud VMDK** + - Il cloud VMDK e' in formato **streamOptimized** (distribuzione), non direttamente resizeable. + - Converti a **monolithicSparse** (tipo 0) contestualmente alla copia: + `& $vdiskMgr -r noble-server-cloudimg-amd64.vmdk -t 0 LinuxBuild2404.vmdk` - Ridimensiona a 40 GB: `& $vdiskMgr -x 40GB LinuxBuild2404.vmdk` (stessa variabile `$vdiskMgr` di Deploy-WinBuild2025.ps1, valorizzata in Step 1) - Il resize avviene PRIMA del primo boot: l'immagine Ubuntu 24.04 ha i moduli cloud-init `growpart` + `resizefs` abilitati di default — espandono automaticamente la partizione root e il filesystem durante il primo avvio, senza `runcmd` aggiuntivi -- [ ] **Step 5 Genera .vmx** +- [x] **Step 5 Genera .vmx** - `guestOS = "ubuntu-64"`, `firmware = "efi"` (Ubuntu 24.04 cloud image usa GPT con partizione EFI; il boot sotto BIOS legacy fallirebbe) - NIC: `vmxnet3`, VMnet8 @@ -133,9 +134,9 @@ Non richiede installazione OS parte dal cloud VMDK pre-built. - Nessun USB controller, nessuna sound card - `memsize = 4096`, `numvcpus = 4` -- [ ] **Step 6 Power on** (`vmrun start nogui`) +- [x] **Step 6 Power on** (`vmrun start nogui`) -- [ ] **Step 7 Polling SSH port 22** fino a ready +- [x] **Step 7 Polling SSH port 22** fino a ready - `Test-NetConnection -ComputerName $IP -Port 22` ogni 10 secondi - Timeout: 5 minuti (cloud-init + reboot finale) - IP rilevato via `vmrun getGuestIPAddress` @@ -143,7 +144,7 @@ Non richiede installazione OS parte dal cloud VMDK pre-built. > open-vm-tools e pre-installato nell'Ubuntu 24.04 cloud image. > `vmrun getGuestIPAddress` funziona senza installazione aggiuntiva. -- [ ] **Step 8 Verifica cloud-init completato** +- [x] **Step 8 Verifica cloud-init completato** ```powershell ssh -i F:\CI\keys\ci_linux -o StrictHostKeyChecking=accept-new ` -o ConnectTimeout=5 ci_build@$IP ` @@ -151,11 +152,11 @@ Non richiede installazione OS parte dal cloud VMDK pre-built. # Atteso: OK ``` -- [ ] **Step 9 Rimuove seed ISO dal VMX** +- [x] **Step 9 Rimuove seed ISO dal VMX** - Setta `ide1:0.present = "FALSE"` nel VMX - Evita che linked clone ricevano il seed ISO e ri-eseguano cloud-init -- [ ] **Step 10 Print summary** (VMX path, IP, SSH fingerprint) +- [x] **Step 10 Print summary** (VMX path, IP, SSH fingerprint) --- @@ -164,23 +165,23 @@ Non richiede installazione OS parte dal cloud VMDK pre-built. Script host-side. Copia il setup script, lo esegue via SSH, valida stato, prende snapshot. Equivalente di `Prepare-WinBuild2025.ps1` con SSH al posto di WinRM. -- [ ] **Step 1 Pre-flight** +- [x] **Step 1 Pre-flight** - VMX presente; se VM spenta, avvia con `vmrun start nogui` - SSH key `F:\CI\keys\ci_linux` presente - `ssh.exe` e `scp.exe` nel PATH host - TCP/22 raggiungibile (`Test-NetConnection`) -- [ ] **Step 2 Auto-detect IP** via `vmrun getGuestIPAddress $VMXPath` +- [x] **Step 2 Auto-detect IP** via `vmrun getGuestIPAddress $VMXPath` - Se `-VMIPAddress` e fornito esplicitamente, salta il rilevamento automatico -- [ ] **Step 3 Aggiungi host key a known_hosts** +- [x] **Step 3 Aggiungi host key a known_hosts** ```powershell ssh -i F:\CI\keys\ci_linux -o StrictHostKeyChecking=accept-new ` ci_build@$IP "echo connected" | Out-Null ``` Solo al primo run; idempotente se gia presente. -- [ ] **Step 4 Copia Setup-LinuxBuild2404.sh in VM** +- [x] **Step 4 Copia Setup-LinuxBuild2404.sh in VM** ```powershell scp -i F:\CI\keys\ci_linux ` "$PSScriptRoot\Setup-LinuxBuild2404.sh" ` @@ -188,7 +189,7 @@ Equivalente di `Prepare-WinBuild2025.ps1` con SSH al posto di WinRM. ``` Valida: confronta size locale vs remota via `ssh ... stat -c %s /tmp/...` -- [ ] **Step 5 Esegui setup script** nel guest +- [x] **Step 5 Esegui setup script** nel guest ```powershell ssh -i F:\CI\keys\ci_linux ci_build@$IP ` "chmod +x /tmp/Setup-LinuxBuild2404.sh && sudo /tmp/Setup-LinuxBuild2404.sh" @@ -196,7 +197,7 @@ Equivalente di `Prepare-WinBuild2025.ps1` con SSH al posto di WinRM. ``` Flag opzionali passabili: `--skip-update`, `--dotnet`, `--mingw` -- [ ] **Step 6 Post-setup remote validation** (batch via singola sessione SSH) +- [x] **Step 6 Post-setup remote validation** (batch via singola sessione SSH) - `systemctl is-active ssh` = `active` - `id ci_build` contiene `sudo` - `sudo -n true` exit 0 (passwordless sudo) @@ -205,18 +206,18 @@ Equivalente di `Prepare-WinBuild2025.ps1` con SSH al posto di WinRM. - `swapon --show` output vuoto (swap disabilitato) - `grep -q "^PasswordAuthentication no" /etc/ssh/sshd_config` -- [ ] **Step 7 Shutdown VM** +- [x] **Step 7 Shutdown VM** ```powershell & $vmrun stop $VMXPath soft # Polling vmrun list fino a VM non piu presente ``` -- [ ] **Step 8 Snapshot BaseClean-Linux** +- [x] **Step 8 Snapshot BaseClean-Linux** ```powershell & $vmrun snapshot $VMXPath 'BaseClean-Linux' ``` -- [ ] **Final Print summary** (VMX, snapshot name, toolchain versions rilevate) +- [x] **Final Print summary** (VMX, snapshot name, toolchain versions rilevate) --- @@ -228,13 +229,13 @@ Eseguito da Prepare via SSH come `ci_build` con sudo. Header: `#!/usr/bin/env bash` + `set -euo pipefail` Helper `assert_step()`: stampa `[OK] [Step N] desc` su successo, `exit 1` su fallimento. -- [ ] **Step 1 apt update + upgrade** (saltabile con `--skip-update`) +- [x] **Step 1 apt update + upgrade** (saltabile con `--skip-update`) ```bash sudo DEBIAN_FRONTEND=noninteractive apt-get update -qq sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y -qq ``` -- [ ] **Step 2 Toolchain build essentials** +- [x] **Step 2 Toolchain build essentials** ```bash sudo apt-get install -y -qq \ build-essential gcc g++ clang \ @@ -243,20 +244,20 @@ Helper `assert_step()`: stampa `[OK] [Step N] desc` su successo, `exit 1` su fal ``` - assert: `command -v gcc`, `command -v clang`, `command -v cmake` -- [ ] **Step 3 Python** +- [x] **Step 3 Python** ```bash sudo apt-get install -y -qq python3 python3-pip python3-venv sudo ln -sf /usr/bin/python3 /usr/local/bin/python ``` - assert: `python3 --version`, `pip3 --version` -- [ ] **Step 4 Tier-1 Toolchain** (speculare a §6.6 Windows: Git + 7-Zip) +- [x] **Step 4 Tier-1 Toolchain** (speculare a §6.6 Windows: Git + 7-Zip) ```bash sudo apt-get install -y -qq git p7zip-full curl wget ca-certificates ``` - assert: `command -v git`, `command -v 7z` -- [ ] **Step 4b .NET SDK** (opzionale, flag `--dotnet`) +- [x] **Step 4b .NET SDK** (opzionale, flag `--dotnet`) ```bash curl -fsSL https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh \ | sudo bash -s -- --channel 10.0 --install-dir /usr/local/dotnet @@ -264,20 +265,20 @@ Helper `assert_step()`: stampa `[OK] [Step N] desc` su successo, `exit 1` su fal ``` - assert: `dotnet --version` -- [ ] **Step 4c mingw-w64** (opzionale, flag `--mingw` cross-compile per Windows) +- [x] **Step 4c mingw-w64** (opzionale, flag `--mingw` cross-compile per Windows) ```bash sudo apt-get install -y -qq mingw-w64 ``` - assert: `command -v x86_64-w64-mingw32-gcc` -- [ ] **Step 5 CI directories** +- [x] **Step 5 CI directories** ```bash sudo mkdir -p /opt/ci/{build,output,scripts,cache} sudo chown -R ci_build:ci_build /opt/ci ``` - assert: `test -d /opt/ci/build`, `test -d /opt/ci/output`, `test -d /opt/ci/scripts` -- [ ] **Step 6 SSH hardening** (PasswordAuth off gia impostato da cloud-init, ri-validato) +- [x] **Step 6 SSH hardening** (PasswordAuth off gia impostato da cloud-init, ri-validato) ```bash sudo sed -i 's/^#*PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config sudo sed -i 's/^#*PubkeyAuthentication.*/PubkeyAuthentication yes/' /etc/ssh/sshd_config @@ -285,21 +286,21 @@ Helper `assert_step()`: stampa `[OK] [Step N] desc` su successo, `exit 1` su fal ``` - assert: `grep -q "^PasswordAuthentication no" /etc/ssh/sshd_config` -- [ ] **Step 7 Disabilita swap permanente** +- [x] **Step 7 Disabilita swap permanente** ```bash sudo swapoff -a sudo sed -i '/ swap / s/^/#/' /etc/fstab ``` - assert: `swapon --show` output vuoto -- [ ] **Step 8 Disabilita apt auto-update** (no lock durante build) +- [x] **Step 8 Disabilita apt auto-update** (no lock durante build) ```bash sudo systemctl mask apt-daily.service apt-daily-upgrade.service \ apt-daily.timer apt-daily-upgrade.timer sudo apt-get remove -y unattended-upgrades 2>/dev/null || true ``` -- [ ] **Step 9 Cleanup pre-snapshot** +- [x] **Step 9 Cleanup pre-snapshot** ```bash sudo apt-get clean sudo rm -rf /var/lib/apt/lists/* @@ -309,7 +310,7 @@ Helper `assert_step()`: stampa `[OK] [Step N] desc` su successo, `exit 1` su fal cat /dev/null > ~/.bash_history ``` -- [ ] **Final Gate pre-snapshot** (exit 1 se qualcosa manca) +- [x] **Final Gate pre-snapshot** (exit 1 se qualcosa manca) - Ogni tool richiesto raggiungibile con `command -v` - `/opt/ci/{build,output,scripts}` esistenti con owner `ci_build` - `swapon --show` vuoto @@ -325,7 +326,7 @@ Helper `assert_step()`: stampa `[OK] [Step N] desc` su successo, `exit 1` su fal ## Fase D Adattamento script CI -- [ ] **`scripts/_Transport.psm1`** nuovo modulo SSH-only per il branch Linux. +- [x] **`scripts/_Transport.psm1`** nuovo modulo SSH-only per il branch Linux. Gli script Windows esistenti NON vengono modificati ora; il refactoring WinRM e' rinviato. ```powershell function Invoke-SshCommand { param($IP, $User, $KeyPath, $Command, ...) } @@ -335,18 +336,18 @@ Helper `assert_step()`: stampa `[OK] [Step N] desc` su successo, `exit 1` su fal Gli script CI (`Invoke-CIJob`, `Invoke-RemoteBuild`, `Get-BuildArtifacts`) chiameranno queste funzioni nel branch Linux senza toccare il branch WinRM esistente. -- [ ] **`scripts/Invoke-CIJob.ps1`** aggiungere parametro `-GuestOS Windows|Linux` +- [x] **`scripts/Invoke-CIJob.ps1`** aggiungere parametro `-GuestOS Windows|Linux` - Auto-detect da VMX `guestOS` field se non fornito - Branch su transport (WinRM vs SSH) in ogni fase -- [ ] **`scripts/Wait-VMReady.ps1`** aggiungere `-Transport SSH` +- [x] **`scripts/Wait-VMReady.ps1`** aggiungere `-Transport SSH` - Probe: `Test-NetConnection -Port 22` + `ssh -o ConnectTimeout=5 ... "echo ok"` -- [ ] **`scripts/Invoke-RemoteBuild.ps1`** branch SSH+scp per Linux +- [x] **`scripts/Invoke-RemoteBuild.ps1`** branch SSH+scp per Linux - Transfer: `scp` invece di `Copy-Item -ToSession` - Build: `ssh ... "cd /opt/ci/build && $BuildCommand"` -- [ ] **`scripts/Get-BuildArtifacts.ps1`** branch `scp` per Linux +- [x] **`scripts/Get-BuildArtifacts.ps1`** branch `scp` per Linux - `scp -r ci_build@$IP:/opt/ci/output/* $LocalArtifactDir\` - [ ] **`scripts/New-BuildVM.ps1`** gia OS-agnostic (vmrun clone) @@ -358,17 +359,17 @@ Helper `assert_step()`: stampa `[OK] [Step N] desc` su successo, `exit 1` su fal ## Fase E Runner e workflow Gitea -- [ ] **Label `linux-build`** aggiunta al runner in `runner/config.yaml`: +- [x] **Label `linux-build`** aggiunta al runner in `runner/config.yaml`: ```yaml labels: - 'windows-build:host' - 'linux-build:host' ``` -- [ ] **Variabile d'ambiente** `GITEA_CI_LINUX_TEMPLATE_PATH` nel servizio act_runner: +- [x] **Variabile d'ambiente** `GITEA_CI_LINUX_TEMPLATE_PATH` nel servizio act_runner: `F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx` -- [ ] **Workflow matrix** di esempio (`gitea/workflow-example.yml` aggiornato): +- [x] **Workflow matrix** di esempio (`gitea/workflow-example.yml` aggiornato): ```yaml strategy: matrix: diff --git a/docs/WINDOWS-TEMPLATE-SETUP.md b/docs/WINDOWS-TEMPLATE-SETUP.md index c4a7225..b44b31f 100644 --- a/docs/WINDOWS-TEMPLATE-SETUP.md +++ b/docs/WINDOWS-TEMPLATE-SETUP.md @@ -11,16 +11,16 @@ credenziali archiviate, ISO Windows Server presente). ## Architettura: tre script -| Script | Dove gira | Scopo | -| ---------------------------------------- | --------------- | ------------------------------------------------------------------- | -| `template/Deploy-WinBuild2025.ps1` | **Host** | Opzionale: crea VM e installa Windows Server 2025 unattended da ISO | -| `template/Prepare-WinBuild2025.ps1` | **Host** | Orchestratore: copia + esegue Setup nel guest via WinRM (2025) | -| `template/Setup-WinBuild2025.ps1` | **Dentro VM** | CI toolchain (user, .NET, Python, VS) + validazione hardening Deploy | -| `template/Deploy-WinBuild2022.ps1` | **Host** | Opzionale: crea VM e installa Windows Server 2022 unattended da ISO | -| `template/Prepare-WinBuild2022.ps1` | **Host** | Orchestratore: copia + esegue Setup nel guest via WinRM (2022) | -| `template/Setup-WinBuild2022.ps1` | **Dentro VM** | CI toolchain per Windows Server 2022 (stesso flusso del 2025) | -| `template/Validate-DeployState.ps1` | **Host** | Valida stato post-Deploy prima di eseguire Prepare (2025 e 2022) | -| `template/Validate-SetupState.ps1` | **Host** | Valida stato post-Setup (Deploy + Setup) prima dello snapshot | +| Script | Dove gira | Scopo | +| ----------------------------------- | ------------- | -------------------------------------------------------------------- | +| `template/Deploy-WinBuild2025.ps1` | **Host** | Opzionale: crea VM e installa Windows Server 2025 unattended da ISO | +| `template/Prepare-WinBuild2025.ps1` | **Host** | Orchestratore: copia + esegue Setup nel guest via WinRM (2025) | +| `template/Setup-WinBuild2025.ps1` | **Dentro VM** | CI toolchain (user, .NET, Python, VS) + validazione hardening Deploy | +| `template/Deploy-WinBuild2022.ps1` | **Host** | Opzionale: crea VM e installa Windows Server 2022 unattended da ISO | +| `template/Prepare-WinBuild2022.ps1` | **Host** | Orchestratore: copia + esegue Setup nel guest via WinRM (2022) | +| `template/Setup-WinBuild2022.ps1` | **Dentro VM** | CI toolchain per Windows Server 2022 (stesso flusso del 2025) | +| `template/Validate-DeployState.ps1` | **Host** | Valida stato post-Deploy prima di eseguire Prepare (2025 e 2022) | +| `template/Validate-SetupState.ps1` | **Host** | Valida stato post-Setup (Deploy + Setup) prima dello snapshot | **Confine Deploy / Setup** (refactor §7, 2026-05-10): - `Deploy-WinBuild2025.ps1` si occupa dell'**OS hardening** (Firewall, WinRM, UAC, Defender, UX tweaks, autologin Administrator, WU GPO locks). Se usato, Setup valida questi come Assert-Step senza re-applicarli. @@ -33,15 +33,15 @@ credenziali archiviate, ISO Windows Server presente). ## Specifiche VM -| Parametro | Valore | -| ----------------------- | --------------------------------------------------------------- | -| OS | Windows Server 2025 Standard/Datacenter (Core o Desktop OK) | -| vCPU | 4 | -| RAM | 6144 MB (6 GB) | -| Disco | 80 GB thin provisioned | -| NIC | **VMnet8 (NAT)** — internet richiesto per WU + installer | -| VMX path | `F:\CI\Templates\WinBuild2025\WinBuild2025.vmx` | -| ISO | Windows Server 2025 (da `F:\CI\ISO\`) | +| Parametro | Valore | +| --------- | ----------------------------------------------------------- | +| OS | Windows Server 2025 Standard/Datacenter (Core o Desktop OK) | +| vCPU | 4 | +| RAM | 6144 MB (6 GB) | +| Disco | 80 GB thin provisioned | +| NIC | **VMnet8 (NAT)** — internet richiesto per WU + installer | +| VMX path | `F:\CI\Templates\WinBuild2025\WinBuild2025.vmx` | +| ISO | Windows Server 2025 (da `F:\CI\ISO\`) | --- @@ -137,17 +137,17 @@ Ogni step ha validazione `Assert-Step` (throw on failure, `[OK]` su pass): ### Parametri principali -| Parametro | Default | Note | -| --------------------- | -------------- | ----------------------------------------------------------------------------- | -| `-VMIPAddress` | (mandatory) | IP DHCP della VM su VMnet8 | -| `-AdminUsername` | Administrator | Account admin built-in nella VM | -| `-AdminPassword` | (prompt) | Se vuoto, prompt SecureString | -| `-BuildPassword` | (prompt) | Password per `ci_build` — mai hard-codare | -| `-BuildUsername` | ci_build | Account CI dentro la VM | -| `-DotNetSdkVersion` | 10.0 | Channel .NET SDK | -| `-SkipWindowsUpdate` | (off) | Pass-through a Setup-WinBuild2025.ps1 | -| `-StoreCredential` | (off) | Registra `BuildUsername`/`BuildPassword` in Windows Credential Manager | -| `-CredentialTarget` | BuildVMGuest | Target Credential Manager (deve corrispondere a quanto usato da Invoke-CIJob) | +| Parametro | Default | Note | +| -------------------- | ------------- | ----------------------------------------------------------------------------- | +| `-VMIPAddress` | (mandatory) | IP DHCP della VM su VMnet8 | +| `-AdminUsername` | Administrator | Account admin built-in nella VM | +| `-AdminPassword` | (prompt) | Se vuoto, prompt SecureString | +| `-BuildPassword` | (prompt) | Password per `ci_build` — mai hard-codare | +| `-BuildUsername` | ci_build | Account CI dentro la VM | +| `-DotNetSdkVersion` | 10.0 | Channel .NET SDK | +| `-SkipWindowsUpdate` | (off) | Pass-through a Setup-WinBuild2025.ps1 | +| `-StoreCredential` | (off) | Registra `BuildUsername`/`BuildPassword` in Windows Credential Manager | +| `-CredentialTarget` | BuildVMGuest | Target Credential Manager (deve corrispondere a quanto usato da Invoke-CIJob) | --- @@ -193,23 +193,23 @@ Final Pre-snapshot gate — 7 check cross-cutting + no installer leftover ### Validazioni per step (sintesi) -| Step | Natura | Check chiave | -| ---- | ---------- | ------------------------------------------------------------------------------ | -| 1 | Assert | `Get-NetFirewallProfile -Profile Domain/Private/Public → Enabled=False` | -| 2 | Assert | GPO DisableAntiSpyware=1 (set da Deploy) | -| 3 | Assert | WinRM Running+Automatic, AllowUnencrypted=false (HTTPS-only), Auth/Basic=true, MaxMem≥2048 | -| 4 | Operativo | User exists, enabled, PasswordNeverExpires, member of Administrators | -| 4b | Assert | EnableLUA=0, LocalAccountTokenFilterPolicy=1 | -| 5 | Operativo | Ogni dir Test-Path -PathType Container | -| 5b | Assert | HKCU LaunchTo=1 | -| 5c | Assert+Op | NoLockScreen=1, SM policy DoNotOpenAtLogon=1, SM task Disabled, DisableCAD=1, OOBE DisablePrivacyExperience=1, AllowTelemetry=0; AutoAdminLogon=1+DefaultUserName=Administrator re-affermati se assenti (Windows OOBE può resettarli al primo boot) | -| 6 | Operativo | ResultCode ∈ {0,2,3}, no reboot required (else exit 3010) | -| 6b | Operativo | wuauserv StartType=Disabled, NoAutoUpdate=1, DisableWindowsUpdateAccess=1 | -| 7 | Operativo | `dotnet --version` channel match, machine PATH contiene `C:\dotnet` | -| 8 | Operativo | `C:\Python\python.exe` esiste, version match esatto, PATH contiene `C:\Python` | -| 9 | Operativo | MSBuild.exe esiste, esegue, PATH, `\VC\` dir presente | -| 10 | Operativo | dotnet+python+msbuild tutti risolvibili (throw) | -| Final| Cross-cut | WinRM, firewall, user+admin, UAC, WU disabled, tutti tool, no leftover installer in `C:\CI` | +| Step | Natura | Check chiave | +| ----- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Assert | `Get-NetFirewallProfile -Profile Domain/Private/Public → Enabled=False` | +| 2 | Assert | GPO DisableAntiSpyware=1 (set da Deploy) | +| 3 | Assert | WinRM Running+Automatic, AllowUnencrypted=false (HTTPS-only), Auth/Basic=true, MaxMem≥2048 | +| 4 | Operativo | User exists, enabled, PasswordNeverExpires, member of Administrators | +| 4b | Assert | EnableLUA=0, LocalAccountTokenFilterPolicy=1 | +| 5 | Operativo | Ogni dir Test-Path -PathType Container | +| 5b | Assert | HKCU LaunchTo=1 | +| 5c | Assert+Op | NoLockScreen=1, SM policy DoNotOpenAtLogon=1, SM task Disabled, DisableCAD=1, OOBE DisablePrivacyExperience=1, AllowTelemetry=0; AutoAdminLogon=1+DefaultUserName=Administrator re-affermati se assenti (Windows OOBE può resettarli al primo boot) | +| 6 | Operativo | ResultCode ∈ {0,2,3}, no reboot required (else exit 3010) | +| 6b | Operativo | wuauserv StartType=Disabled, NoAutoUpdate=1, DisableWindowsUpdateAccess=1 | +| 7 | Operativo | `dotnet --version` channel match, machine PATH contiene `C:\dotnet` | +| 8 | Operativo | `C:\Python\python.exe` esiste, version match esatto, PATH contiene `C:\Python` | +| 9 | Operativo | MSBuild.exe esiste, esegue, PATH, `\VC\` dir presente | +| 10 | Operativo | dotnet+python+msbuild tutti risolvibili (throw) | +| Final | Cross-cut | WinRM, firewall, user+admin, UAC, WU disabled, tutti tool, no leftover installer in `C:\CI` | --- @@ -292,15 +292,15 @@ toolchain (.NET, VS Build Tools, Python). ## Troubleshooting -| Sintomo | Diagnosi / Fix | -| ------------------------------------------- | ----------------------------------------------------------------------- | -| `Cannot reach $VMIPAddress via WinRM` | VM non avviata, WinRM non abilitato, IP errato — verifica con ipconfig | -| `Setup-WinBuild2025.ps1 exited with code 3010`| Reboot richiesto post-WU. Re-run con `-SkipWindowsUpdate` | -| `[Defender] Validation failed: GPO key` | Defender riattivato da Windows Update — disabilita via gpedit + reboot | -| `VS Build Tools installation timed out` | Network slow o cache corrotta. Re-run con cache pulita (`C:\ProgramData\Microsoft\VisualStudio\Packages`) | -| `Python version match` fail | `$PythonVersion` ≠ versione installata. Aggiorna param o reinstalla | -| Post-setup `MSBuild not found` | VS install fallita silenziosa. Check `C:\CI\vs_log.txt` | -| Pre-snapshot Final check fail | Stato VM rotto — non prendere snapshot. Re-run o ricomincia da zero | +| Sintomo | Diagnosi / Fix | +| ---------------------------------------------- | --------------------------------------------------------------------------------------------------------- | +| `Cannot reach $VMIPAddress via WinRM` | VM non avviata, WinRM non abilitato, IP errato — verifica con ipconfig | +| `Setup-WinBuild2025.ps1 exited with code 3010` | Reboot richiesto post-WU. Re-run con `-SkipWindowsUpdate` | +| `[Defender] Validation failed: GPO key` | Defender riattivato da Windows Update — disabilita via gpedit + reboot | +| `VS Build Tools installation timed out` | Network slow o cache corrotta. Re-run con cache pulita (`C:\ProgramData\Microsoft\VisualStudio\Packages`) | +| `Python version match` fail | `$PythonVersion` ≠ versione installata. Aggiorna param o reinstalla | +| Post-setup `MSBuild not found` | VS install fallita silenziosa. Check `C:\CI\vs_log.txt` | +| Pre-snapshot Final check fail | Stato VM rotto — non prendere snapshot. Re-run o ricomincia da zero | ---