Simone
6104811e7e
Linux clones could boot with the RTC interpreted as local time (guest runs hours ahead of UTC). NTP then steps the clock backward at runtime, landing after source mtimes were stamped, so make reported "Clock skew detected" and future-dated files. - build run: enable NTP and block until NTPSynchronized before stamping the sources, so the backward correction happens pre-touch. Drop `hwclock -s`, which read the local-time RTC and pushed the clock further ahead. - template prepare-linux: set-local-rtc 0 so clones interpret the RTC as UTC and boot at the correct time; NTP then only nudges, never steps 2h. Linux/SSH path only; WinRM branch untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Local CI/CD System
Sistema CI/CD self-hosted su Windows per build isolati in VM efimere VMware.
Stato: production-ready — build Windows e Linux verificati e2e; burn-in concorrente a 4 job PASS; §3.3 in-VM git clone attivabile via -UseGitClone
Panoramica
Il runner act_runner riceve i job da Gitea Actions.
Per ogni job, lo script Invoke-CIJob.ps1 orchestra l'intero ciclo:
Gitea push/tag
│
▼
act_runner (label: windows-build | linux-build)
│
▼
Invoke-CIJob.ps1
│
├─ 1. git clone repo su host (con submoduli)
├─ 2. Linked clone da snapshot BaseClean / BaseClean-Linux
├─ 3. Avvio VM headless
├─ 4. Attesa readiness (WinRM/5986 per Windows, SSH/22 per Linux)
├─ 5. Source transfer + build (WinRM per Windows, SSH/SCP per Linux)
├─ 6. Copia artifacts in F:\CI\Artifacts\<jobid>\
└─ finally: stop + rimozione VM clone
Ogni build gira in una VM usa e getta ripristinata dallo snapshot: Windows Server 2025/2022 (WinRM) o Ubuntu 24.04 (SSH).
Requisiti Host
| Componente | Versione / Dettaglio |
|---|---|
| OS | Windows 11 |
| CPU/RAM | i9-10900X, 64 GB RAM |
| VMware Workstation Pro | installato, vmrun.exe disponibile |
| PowerShell | 5.1 |
| act_runner | v1.0.2 — F:\CI\act_runner\act_runner.exe |
| Gitea | self-hosted http://10.10.20.11:3100 / https://gitea.emulab.it |
Template VM
Windows (snapshot BaseClean)
| Componente | Versione |
|---|---|
| OS | Windows Server 2025 |
| VMX | F:\CI\Templates\WinBuild2025\WinBuild2025.vmx (env GITEA_CI_TEMPLATE_PATH) |
| VS Build Tools | 2026 (MSBuild 18.5.4 / toolset v145) |
| .NET SDK | 10.0.203 |
| Python | 3.13.3 |
| Rete | VMnet8 NAT — 192.168.79.0/24 |
| WinRM | HTTPS/5986, Basic auth, self-signed cert (SkipCACheck — lab-only) |
Variante Windows Server 2022: *WinBuild2022.ps1 + VMX F:\CI\Templates\WinBuild2022\WinBuild2022.vmx.
Linux (snapshot BaseClean-Linux)
| Componente | Versione |
|---|---|
| OS | Ubuntu 24.04 LTS (cloud image) |
| VMX | F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx (env GITEA_CI_LINUX_TEMPLATE_PATH) |
| Toolchain | GCC, G++, Clang, CMake, Ninja, Python 3, Git, 7-Zip |
| Rete | VMnet8 NAT — 192.168.79.0/24 |
| SSH | porta 22, utente ci_build, chiave F:\CI\keys\ci_linux |
| Runner label | linux-build:host |
Struttura repository
├── Setup-Host.ps1 # Bootstrap host: dirs F:\CI\, credenziali, act_runner service
├── PSScriptAnalyzerSettings.psd1 # Regole linting condivise (§5.4)
│
├── scripts/
│ ├── _Common.psm1 # Shared module (vmrun wrapper, diagnostics)
│ ├── _Transport.psm1 # SSH/SCP helpers per guest Linux (Invoke-SshCommand, Copy-SshItem)
│ ├── Invoke-CIJob.ps1 # Orchestratore principale (Windows WinRM + Linux SSH, -UseGitClone)
│ ├── New-BuildVM.ps1 # Linked clone da template
│ ├── Wait-VMReady.ps1 # Polling readiness (WinRM/5986 o SSH/22)
│ ├── Invoke-RemoteBuild.ps1 # Build remoto (WinRM per Windows, SSH/SCP per Linux)
│ ├── Get-BuildArtifacts.ps1 # Raccolta artifacts (WinRM o SCP)
│ ├── Remove-BuildVM.ps1 # Stop + rimozione VM clone (-WhatIf)
│ ├── Cleanup-OrphanedBuildVMs.ps1 # Cleanup VM orfane (§2.2)
│ ├── Invoke-RetentionPolicy.ps1 # Retention artifact + log (§2.3)
│ ├── Watch-DiskSpace.ps1 # Alert spazio libero F: (§4.3)
│ ├── Watch-RunnerHealth.ps1 # Health check runner (§2.7)
│ ├── Backup-CITemplate.ps1 # Backup VMDK template (§2.6)
│ ├── Set-TemplateSharedFolders.ps1 # Cache NuGet/pip via shared folders (§3.1)
│ ├── Measure-CIBenchmark.ps1 # Benchmark fasi pipeline (§3.6)
│ ├── Register-CIScheduledTasks.ps1 # Task Scheduler bootstrap
│ ├── Test-CapacityBurnIn.ps1 # Burn-in concorrente N job × R round
│ ├── Start-BurnInTest-Linux.ps1 # Wrapper burn-in specifico per LinuxBuild2404
│ └── Get-CIJobSummary.ps1 # Report sintetico ultima esecuzione job
│
├── template/
│ ├── autounattend.template.xml # Template XML per installazione Windows unattended
│ ├── Deploy-WinBuild2025.ps1 # Deploy VM + install Windows 2025 unattended (HOST)
│ ├── Prepare-WinBuild2025.ps1 # Provisioning toolchain 2025 via WinRM (HOST-side)
│ ├── Install-CIToolchain-WinBuild2025.ps1 # Toolchain CI 2025 nella VM (GUEST-side)
│ ├── Deploy-WinBuild2022.ps1 # Deploy VM + install Windows 2022 unattended (HOST)
│ ├── Prepare-WinBuild2022.ps1 # Provisioning toolchain 2022 via WinRM (HOST-side)
│ ├── Install-CIToolchain-WinBuild2022.ps1 # Toolchain CI 2022 nella VM (GUEST-side)
│ ├── Deploy-LinuxBuild2404.ps1 # Deploy cloud image Ubuntu 24.04 su VMware (HOST)
│ ├── Prepare-LinuxBuild2404.ps1# Provisioning toolchain Linux via SSH + snapshot (HOST-side)
│ ├── Install-CIToolchain-Linux2404.sh # Toolchain CI Ubuntu 24.04 nel guest (GUEST-side)
│ ├── Validate-DeployState.ps1 # Validazione stato post-Deploy (2025, 2022, Linux)
│ └── Validate-SetupState.ps1 # Validazione stato post-Setup (pre-snapshot)
│
├── tests/ # Pester v5 unit tests (§5.1)
│
├── gitea/
│ ├── actions/
│ │ └── local-ci-build/action.yml # Composite action riusabile (wrap Invoke-CIJob.ps1)
│ ├── workflows/
│ │ ├── build-ns7zip.yml # Workflow matrix Windows+Linux per nsis-plugin-ns7zip
│ │ └── lint.yml # PSScriptAnalyzer su push/PR (§5.4)
│ └── workflow-example.yml # Template generico adattabile
│
├── runner/
│ ├── config.yaml # Configurazione act_runner (no secrets)
│ └── Install-Runner.ps1
│
├── docs/
│ ├── ARCHITECTURE.md # Diagramma sistema e componenti
│ ├── CI-FLOW.md # Flusso dettagliato passo-passo
│ ├── HOST-SETUP.md # Bootstrap macchina host
│ ├── WINDOWS-TEMPLATE-SETUP.md # Provisioning template Windows
│ ├── LINUX-TEMPLATE-SETUP.md # Bozza template Linux (§6.1, non implementato)
│ ├── OPTIMIZATION.md # Layout disco, snapshot, cache, tuning
│ ├── BEST-PRACTICES.md # Sicurezza, WinRM, credenziali, threat model
│ ├── RUNBOOK.md # Triage incident comuni (§4.4)
│ ├── TEST-PLAN-v1.3-to-HEAD.md # Test plan completo features post-v1.3
│ ├── Setup-GiteaSSH.md # Configurazione SSH alias per Gitea
│ └── archived/ # Doc storici / sign-off
│
└── TODO.md # Roadmap, audit trail task completati, backlog
Python orchestrator (Phase A — in progress)
A new Python 3.11+ rewrite of the orchestrator lives under src/ci_orchestrator/.
Phase A (see implementation-plan-A-B) is on
branch feature/python-rewrite-and-linux-migration. As of A5 the Python CLI is the
production entry point invoked by gitea/actions/local-ci-build/action.yml;
the PowerShell scripts in scripts/ are 3-line shims that forward to the
Python sub-commands listed below.
Setup
# Production venv on the runner host (one-time + on every code deploy).
# NON-editable: act_runner runs as LocalSystem and cannot resolve an
# editable .pth into the dev tree or a transient RunnerWork dir
# (-> "No module named ci_orchestrator"). Re-run `pip install .` after
# each code change. No CI workflow may install into this venv.
python -m venv F:\CI\python\venv
F:\CI\python\venv\Scripts\python.exe -m pip install --upgrade pip
F:\CI\python\venv\Scripts\python.exe -m pip install .
# Local dev venv (NOT committed; see .gitignore) — editable is fine here
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -e ".[dev]"
Configuration: copy config.example.toml to
F:\CI\config.toml (or set $env:CI_CONFIG). Environment variables
(CI_ROOT, CI_TEMPLATES, CI_BUILD_VMS, CI_ARTIFACTS, CI_KEYS)
override the file. Dependency manifest and tool config (ruff, mypy,
pytest, coverage) live in pyproject.toml.
CLI sub-commands
# Discover commands
python -m ci_orchestrator --help
# Wait for a VM to become reachable (WinRM 5986 or SSH 22 — auto-detected)
python -m ci_orchestrator wait-ready `
--vmx F:\CI\BuildVMs\smoke\smoke.vmx --guest-os windows --timeout 300
# Linked-clone lifecycle
python -m ci_orchestrator vm new `
--template-path F:\CI\Templates\WinBuild2025\WinBuild2025.vmx `
--snapshot-name BaseClean `
--clone-base-dir F:\CI\BuildVMs `
--job-id smoke-001
python -m ci_orchestrator vm remove --vmx F:\CI\BuildVMs\Clone_smoke-001_*\Clone_smoke-001_*.vmx
python -m ci_orchestrator vm cleanup --max-age-hours 12
# Build + artifact collection
python -m ci_orchestrator build run `
--vmx F:\CI\BuildVMs\Clone_smoke-001_*\*.vmx `
--script .\build.ps1 --workdir C:\build --guest-os windows
python -m ci_orchestrator artifacts collect `
--vmx F:\CI\BuildVMs\Clone_smoke-001_*\*.vmx `
--remote-path C:\build\dist --local-dir F:\CI\Artifacts\smoke-001
# Monitoring (driven by Task Scheduler / systemd timers in B5)
python -m ci_orchestrator monitor disk --min-free-gb 50 --drive-letter F
python -m ci_orchestrator monitor runner --state-dir F:\CI\act_runner
# Reporting
python -m ci_orchestrator report job --job-id smoke-001
# End-to-end orchestrator (replaces Invoke-CIJob.ps1)
python -m ci_orchestrator job `
--job-id smoke-001 `
--repo-url ssh://gitea-ci/Simone/nsis-plugin-ns7zip.git `
--branch main `
--template-path F:\CI\Templates\WinBuild2025\WinBuild2025.vmx `
--build-command "python build_plugin.py --final --dist-dir dist" `
--guest-artifact-source dist
The PowerShell wrappers (scripts\Invoke-CIJob.ps1,
scripts\New-BuildVM.ps1, scripts\Wait-VMReady.ps1,
scripts\Remove-BuildVM.ps1, scripts\Cleanup-OrphanedBuildVMs.ps1,
scripts\Invoke-RemoteBuild.ps1, scripts\Get-BuildArtifacts.ps1,
scripts\Watch-DiskSpace.ps1, scripts\Watch-RunnerHealth.ps1,
scripts\Get-CIJobSummary.ps1) are kept as 3-line shims for Task
Scheduler and external callers — they translate PascalCase parameters
to the Python kebab-case CLI and propagate the exit code.
Local validation gates
.\.venv\Scripts\python.exe -m ruff check src tests\python
.\.venv\Scripts\python.exe -m mypy --strict src
.\.venv\Scripts\python.exe -m pytest tests\python -q --cov=ci_orchestrator --cov-fail-under=90
Coverage gate is 90%. Regression tests for the
AGENTS.md "Errori frequenti da evitare" #9–#12 live in
tests/python/test_agents_errors.py
and must stay green.
Setup rapido
1a. Template VM Windows
Opzione A — Deploy automatico (crea VM + installa Windows unattended da ISO):
cd N:\Code\Workspace\Local-CI-CD-System\template
# Windows Server 2025 (corrente):
.\Deploy-WinBuild2025.ps1 -ISOPath F:\CI\ISO\WinSrv2025.iso `
-VMPath F:\CI\Templates\WinBuild2025\WinBuild2025.vmx
# Windows Server 2022 (alternativa):
.\Deploy-WinBuild2022.ps1 -ISOPath F:\CI\ISO\WinSrv2022.iso `
-VMPath F:\CI\Templates\WinBuild2022\WinBuild2022.vmx
Opzione B — installazione manuale: installa Windows nella VM, abilita WinRM
(winrm quickconfig -q + Enable-PSRemoting -Force), annota l'IP VMnet8.
Provisioning toolchain (dopo A o B, dalla directory template/):
# Windows Server 2025:
.\Prepare-WinBuild2025.ps1 -VMIPAddress 192.168.79.x -StoreCredential
# Windows Server 2022:
.\Prepare-WinBuild2022.ps1 -VMIPAddress 192.168.79.x -StoreCredential
Poi spegni la VM e prendi lo snapshot con nome esatto BaseClean.
1b. Template VM Linux
cd N:\Code\Workspace\Local-CI-CD-System\template
# Deploy cloud image Ubuntu 24.04 (scarica ISO/cloud img, crea VM):
.\Deploy-LinuxBuild2404.ps1 -VMPath F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx
Provisioning toolchain (SSH — la VM deve essere avviata e raggiungibile):
.\Prepare-LinuxBuild2404.ps1 -VMIPAddress 192.168.79.x `
-VMXPath F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx `
-TakeSnapshot
-TakeSnapshot esegue automaticamente: reset machine-id → shutdown → snapshot BaseClean-Linux.
Dettagli: docs/LINUX-TEMPLATE-SETUP.md.
2. Credenziali guest
# PowerShell elevato — una volta sola sull'host.
# Scrive nel vault SYSTEM (act_runner gira come LocalSystem; un
# New-StoredCredential dalla sessione utente NON sarebbe leggibile dal
# runner). Username host-qualificato col computer name del guest
# (bare 'ci_build' -> NTLM SEC_E_UNKNOWN_CREDENTIALS).
.\scripts\Set-CIGuestCredential.ps1 -UserName 'WINBUILD-2025\ci_build'
3. act_runner
# Registra il runner con Gitea (interattivo, usa il registration token da Gitea Admin)
F:\CI\act_runner\act_runner.exe register
# Installa come servizio Windows
nssm install act_runner "F:\CI\act_runner\act_runner.exe" daemon
nssm start act_runner
Imposta queste variabili d'ambiente nel servizio (runner/config.yaml):
| Variabile | Valore |
|---|---|
GITEA_CI_TEMPLATE_PATH |
F:\CI\Templates\WinBuild2025\WinBuild2025.vmx |
GITEA_CI_LINUX_TEMPLATE_PATH |
F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx |
GITEA_CI_CLONE_BASE_DIR |
F:\CI\BuildVMs |
GITEA_CI_ARTIFACT_DIR |
F:\CI\Artifacts |
GITEA_CI_GUEST_CRED_TARGET |
BuildVMGuest |
GITEA_CI_SSH_KEY_PATH |
F:\CI\keys\ci_linux |
Labels runner: windows-build:host (job Windows) e linux-build:host (job Linux).
Dettagli completi: docs/HOST-SETUP.md.
4. SSH per Gitea
Vedi docs/Setup-GiteaSSH.md per configurare l'alias gitea-ci
(SSH port 2222, usato dai workflow live in git clone ssh://gitea-ci/...).
Uso — build manuale (test)
Windows guest:
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Invoke-CIJob.ps1' `
-JobId 'test-win-001' `
-RepoUrl 'ssh://gitea-ci/Simone/nsis-plugin-ns7zip.git' `
-Branch 'main' `
-TemplatePath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
-Submodules `
-BuildCommand 'python build_plugin.py --final --dist-dir dist' `
-GuestArtifactSource 'dist' `
-GuestCredentialTarget 'BuildVMGuest'
Linux guest:
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Invoke-CIJob.ps1' `
-JobId 'test-linux-001' `
-RepoUrl 'ssh://gitea-ci/Simone/nsis-plugin-ns7zip.git' `
-Branch 'main' `
-TemplatePath 'F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx' `
-SnapshotName 'BaseClean-Linux' `
-GuestOS 'Linux' `
-BuildCommand 'bash build.sh' `
-GuestArtifactSource '/opt/ci/output'
Artifacts raccolti in: F:\CI\Artifacts\<jobid>\.
Aggiungi -UseGitClone per clonare direttamente in VM (skip host-zip-transfer; richiede Git in template — già incluso nel toolchain).
Aggiungi -SkipArtifact per job che non producono output (es. lint, test puri).
Workflow Gitea
Il file gitea/workflows/build-ns7zip.yml è il workflow
attivo per Simone/nsis-plugin-ns7zip. Si attiva su push di tag v* o manualmente.
Usa una strategy matrix [windows, linux] che esegue in parallelo su entrambi i runner.
La composite action gitea/actions/local-ci-build/action.yml asconde tutta la logica Invoke-CIJob.ps1 e accetta questi input principali:
| Input | Descrizione |
|---|---|
build-command |
Comando di build da eseguire nel guest |
guest-os |
Windows (default) o Linux |
artifact-source |
Path artifacts nel guest |
skip-artifact |
true per job senza output |
repo-url |
URL SSH del repository |
job-id-suffix |
Suffisso per disambiguare artifact dir in matrix |
Per adattare ad altri repository, copia gitea/workflow-example.yml
e aggiorna build-command, artifact-source, repo-url.
Documentazione
| File | Contenuto |
|---|---|
| docs/ARCHITECTURE.md | Diagramma componenti e flusso dati |
| docs/CI-FLOW.md | Descrizione dettagliata di ogni fase del pipeline |
| docs/HOST-SETUP.md | Bootstrap macchina host (Setup-Host.ps1) |
| docs/WINDOWS-TEMPLATE-SETUP.md | Provisioning template Windows (Deploy + Prepare + Setup) |
| docs/LINUX-TEMPLATE-SETUP.md | Template Ubuntu 24.04: Deploy-LinuxBuild2404, Prepare-LinuxBuild2404, snapshot BaseClean-Linux |
| docs/OPTIMIZATION.md | Layout NVMe, snapshot strategy, cache, tuning |
| docs/BEST-PRACTICES.md | Sicurezza WinRM, threat model, credenziali |
| docs/RUNBOOK.md | Triage incident comuni (runner offline, clone fail, ecc.) |
| docs/TEST-PLAN-v1.3-to-HEAD.md | Test plan completo features post-v1.3 |
| docs/Setup-GiteaSSH.md | Configurazione SSH alias gitea-ci |
| TODO.md | Roadmap, audit trail task completati, backlog |
Note di sicurezza
WinRM usa HTTPS/5986 con certificato self-signed e Basic auth.
-SkipCACheck/-SkipCNCheck sono accettabili in lab isolato (il cert non è di una CA trusted).
Per ambienti condivisi o di produzione, usare un certificato CA valida e rimuovere -SkipCACheck.
Vedi docs/BEST-PRACTICES.md.