feat: enhance documentation and scripts for in-VM Git cloning and template setup
- Updated README.md to reflect the new in-VM git clone feature with `-UseGitClone`. - Modified Setup-Host.ps1 to correct VMX path references. - Revised various documentation files (ARCHITECTURE.md, CI-FLOW.md, HOST-SETUP.md, WINDOWS-TEMPLATE-SETUP.md) to include updated paths and features. - Added Setup-GiteaSSH.md for SSH configuration instructions. - Archived outdated documentation and updated README.md for clarity on current resources.
This commit is contained in:
Simone
@@ -117,7 +117,7 @@ The runner host **never executes build tools directly**. Its only role is orches
|
||||
- .NET SDK **10.0.203**
|
||||
- Python **3.13.3** (per build script personalizzati come `build_plugin.py`)
|
||||
- NuGet CLI (optional, dotnet restore covers most cases)
|
||||
- Git: NOT installed by default — source arrives via WinRM zip transfer. See TODO.md `-UseGitClone` for the in-VM git clone opt-in path.
|
||||
- **Git for Windows 2.54** + **7-Zip 26.01** (Tier-1 toolchain, §6.6 — DONE 2026-05-10): consente l'opt-in `-UseGitClone` di `Invoke-CIJob.ps1` per clonare il repository direttamente in VM (skip host zip transfer, e2e PASS, -25.7% durata pipeline).
|
||||
|
||||
---
|
||||
|
||||
@@ -191,4 +191,5 @@ VMs have internet access via NAT — required for pip/nuget during build.
|
||||
- WinRM credentials are stored in Windows Credential Manager on host (not hardcoded)
|
||||
- Each VM is fully destroyed after the job — no state carries between builds
|
||||
- Runner host does not expose any ports to the build VM network
|
||||
- Source code is transferred via zip (Compress-Archive → WinRM copy → Expand-Archive in guest) — no PAT is ever present inside the VM; VMs have NAT internet for pip/nuget at build time
|
||||
- Source code is transferred via zip (Compress-Archive → WinRM copy → Expand-Archive in guest) by default — no PAT is ever present inside the VM; VMs have NAT internet for pip/nuget at build time.
|
||||
- With `-UseGitClone` (§3.3, opt-in) il clone avviene dentro la VM via SSH alias `gitea-ci` (forwardato dall'host); la chiave privata SSH dell'host non viene esposta al guest — vedi BEST-PRACTICES.md per il threat model.
|
||||
|
||||
+5
-2
@@ -138,8 +138,11 @@ Sequence inside the PSSession:
|
||||
|
||||
```
|
||||
1. New-PSSession → open WinRM session to VM
|
||||
2. Compress-Archive (host) → copy zip to C:\CI\build.zip (guest) → Expand-Archive to C:\CI\build\
|
||||
(source code è già clonato sull'host in Phase 1 — non si fa git clone nella VM)
|
||||
2a. (default) Compress-Archive (host) → copy zip to C:\CI\build.zip (guest) → Expand-Archive to C:\CI\build\
|
||||
(source code è già clonato sull'host in Phase 1 — nessun git clone nella VM)
|
||||
2b. (opt-in, `-UseGitClone`, §3.3 DONE 2026-05-10)
|
||||
git clone direttamente in VM tramite `ssh://gitea-ci/...` (richiede Git+7-Zip in template, §6.6 Tier-1)
|
||||
Vantaggio: skip host-zip-transfer, -25.7% durata pipeline (e2e PASS). Verifica SHA commit post-clone.
|
||||
3. Invoke-Expression "$BuildCommand" in C:\CI\build\
|
||||
Esempio: python build_plugin.py --final --dist-dir dist
|
||||
a. Build 4 configurazioni MSBuild in parallelo (x86-unicode, x64-unicode, cli-x86, cli-x64)
|
||||
|
||||
+8
-8
@@ -33,13 +33,13 @@ F:\CI\
|
||||
├── Artifacts\ # output build raccolti dall'host
|
||||
├── Logs\ # log per-job (retention: 30 giorni)
|
||||
├── Templates\
|
||||
│ └── WinBuild\ # VMX template Windows (immutabile dopo snapshot)
|
||||
│ └── WinBuild2025\ # VMX template Windows (immutabile dopo snapshot)
|
||||
├── act_runner\
|
||||
│ ├── act_runner.exe # binario runner Gitea
|
||||
│ ├── config.yaml # config runner (path, label, capacity)
|
||||
│ └── logs\ # stdout/stderr del servizio
|
||||
├── Cache\
|
||||
│ └── NuGet\ # cache NuGet condivisa via shared folder (TODO §3.1)
|
||||
│ └── NuGet\ # cache NuGet condivisa via shared folder (§3.1 — DONE 2026-05-10, vedi scripts/Set-TemplateSharedFolders.ps1)
|
||||
├── RunnerWork\ # working dir di act_runner (job staging)
|
||||
└── ISO\ # ISO di installazione (Windows Server 2025, ecc.)
|
||||
```
|
||||
@@ -161,13 +161,13 @@ Allo `Setup-Host.ps1` exit 0:
|
||||
|
||||
## Maintenance & operational tasks
|
||||
|
||||
Voci di backlog (vedi [TODO.md](../TODO.md)):
|
||||
Task schedulati (vedi `scripts/Register-CIScheduledTasks.ps1` per il bootstrap):
|
||||
|
||||
- **§2.2** Cleanup VM orfane schedulato (`scripts/Cleanup-OrphanedBuildVMs.ps1` ogni 6h, `-MaxAgeHours 4`)
|
||||
- **§2.3** Retention artifact + log (Artifacts > 30 gg, Logs > 30 gg, guard libero < 50 GB)
|
||||
- **§2.7** Health check runner (query API Gitea ogni 15 min, restart su offline)
|
||||
- **§4.3** Disk space alert (`F:` < 50 GB → eventcreate/webhook)
|
||||
- **§1.2** Restringere `TrustedHosts` da `*` a `192.168.79.*`
|
||||
- **§2.2** Cleanup VM orfane (`scripts/Cleanup-OrphanedBuildVMs.ps1`, ogni 6h, `-MaxAgeHours 4`)
|
||||
- **§2.3** Retention artifact + log (`scripts/Invoke-RetentionPolicy.ps1`, daily)
|
||||
- **§2.6** Backup template (`scripts/Backup-CITemplate.ps1`, weekly)
|
||||
- **§2.7** Health check runner (`scripts/Watch-RunnerHealth.ps1`, ogni 15 min)
|
||||
- **§4.3** Disk space alert (`scripts/Watch-DiskSpace.ps1`, ogni ora)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -104,7 +104,7 @@ F:\CI\Cache\NuGet\ ← shared folder on host
|
||||
|
||||
### VMware Shared Folder Configuration (in template VMX)
|
||||
|
||||
Add to `WinBuild.vmx` before taking the `BaseClean` snapshot:
|
||||
Add to `WinBuild2025.vmx` before taking the `BaseClean` snapshot:
|
||||
```ini
|
||||
sharedFolder0.present = "TRUE"
|
||||
sharedFolder0.enabled = "TRUE"
|
||||
|
||||
@@ -7,14 +7,14 @@ Organized by sprint area with pre-requisites, test steps, and pass/fail criteria
|
||||
|
||||
## Overview
|
||||
|
||||
| Area | Items | Status | Baseline |
|
||||
| ----------------------------- | ----------------------- | -------- | -------------------------------------------------------------------------------------- |
|
||||
| **Sprint 2** (Reliability) | 2.1/2.2/2.3/2.5 | Partial | §2.2/§2.3 validated locally 2026-05-10; §2.1/§2.5 need real VMs |
|
||||
| **Sprint 3** (Observability) | 4.1/4.3/4.4 | Partial | §4.3/§4.4 validated locally 2026-05-10; §4.1 (JSONL) needs real build job |
|
||||
| **Sprint 4-6** (Quality/Perf) | 1.4/3.1/3.6/5.2/5.3/5.4 | Partial | §1.4/§5.2/§5.3/§5.4 validated 2026-05-10; §3.1/§3.6 need real VMs |
|
||||
| **Sprint 7** (Testing) | 5.1 | [x] Done | 22/22 Pester tests passing in 12.9s (2026-05-10) |
|
||||
| **§1.6** (Docs) | Threat model | [x] Done | BEST-PRACTICES.md verified 2026-05-10 |
|
||||
| **§3.2** (Perf) | 7-Zip compression | Partial | Fallback path validated; in-VM 7-Zip speed test needs real VM |
|
||||
| Area | Items | Status | Baseline |
|
||||
| ----------------------------- | ----------------------- | -------- | ------------------------------------------------------------------------- |
|
||||
| **Sprint 2** (Reliability) | 2.1/2.2/2.3/2.5 | Partial | §2.2/§2.3 validated locally 2026-05-10; §2.1/§2.5 need real VMs |
|
||||
| **Sprint 3** (Observability) | 4.1/4.3/4.4 | Partial | §4.3/§4.4 validated locally 2026-05-10; §4.1 (JSONL) needs real build job |
|
||||
| **Sprint 4-6** (Quality/Perf) | 1.4/3.1/3.6/5.2/5.3/5.4 | Partial | §1.4/§5.2/§5.3/§5.4 validated 2026-05-10; §3.1/§3.6 need real VMs |
|
||||
| **Sprint 7** (Testing) | 5.1 | [x] Done | 22/22 Pester tests passing in 12.9s (2026-05-10) |
|
||||
| **§1.6** (Docs) | Threat model | [x] Done | BEST-PRACTICES.md verified 2026-05-10 |
|
||||
| **§3.2** (Perf) | 7-Zip compression | Partial | Fallback path validated; in-VM 7-Zip speed test needs real VM |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -37,7 +37,7 @@ credenziali archiviate, ISO Windows Server presente).
|
||||
| RAM | 6144 MB (6 GB) |
|
||||
| Disco | 80 GB thin provisioned |
|
||||
| NIC | **VMnet8 (NAT)** — internet richiesto per WU + installer |
|
||||
| VMX path | `F:\CI\Templates\WinBuild\WinBuild2025.vmx` |
|
||||
| VMX path | `F:\CI\Templates\WinBuild2025\WinBuild2025.vmx` |
|
||||
| ISO | Windows Server 2025 (da `F:\CI\ISO\`) |
|
||||
|
||||
---
|
||||
@@ -48,7 +48,7 @@ credenziali archiviate, ISO Windows Server presente).
|
||||
- File → New Virtual Machine → Custom
|
||||
- Hardware: 4 vCPU, 6144 MB, 80 GB thin
|
||||
- Network: **VMnet8 (NAT)**
|
||||
- VMX: `F:\CI\Templates\WinBuild\WinBuild2025.vmx`
|
||||
- VMX: `F:\CI\Templates\WinBuild2025\WinBuild2025.vmx`
|
||||
- CD/DVD: monta ISO Windows Server 2025
|
||||
|
||||
### A.2 Installa Windows Server 2025
|
||||
@@ -219,7 +219,7 @@ Final Pre-snapshot gate — 7 check cross-cutting + no installer leftover
|
||||
- VM → Snapshot → Take Snapshot
|
||||
- **Nome esatto: `BaseClean`** (case-sensitive, usato da `New-BuildVM.ps1`)
|
||||
3. **Lascia la VM spenta** — non modificarla mai più dopo lo snapshot
|
||||
4. **Verifica `runner/config.yaml`**: `GITEA_CI_TEMPLATE_PATH=F:\CI\Templates\WinBuild\WinBuild2025.vmx`
|
||||
4. **Verifica `runner/config.yaml`**: `GITEA_CI_TEMPLATE_PATH=F:\CI\Templates\WinBuild2025\WinBuild2025.vmx`
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# Local CI/CD System — Design & Status
|
||||
|
||||
## Sistema implementato e operativo (2026-05-08)
|
||||
|
||||
### Hardware
|
||||
- Windows 11 host — CPU i9-10900X, 64 GB RAM, NVMe SSD
|
||||
|
||||
### Hypervisor
|
||||
- VMware Workstation Pro
|
||||
- Template VM: Windows Server 2025
|
||||
- VMX: `F:\CI\Templates\WinBuild\CI-WinBuild.vmx`
|
||||
- Snapshot: `BaseClean`
|
||||
- Toolchain: VS Build Tools 2026 (MSBuild 18.5.4 / v145), .NET SDK 10.0.203, Python 3.13.3
|
||||
|
||||
### CI System
|
||||
- Gitea self-hosted — `http://10.10.20.11:3100`
|
||||
- act_runner v1.0.2 — Windows service su host, label `windows-build`
|
||||
- Build VMs: subnet VMnet8 NAT — `192.168.79.0/24`
|
||||
- WinRM HTTPS/5986 (Basic auth over HTTPS, self-signed cert, SkipCACheck lab-only)
|
||||
|
||||
### Architettura implementata
|
||||
1. Gitea Actions enqueue job → act_runner lo prende
|
||||
2. `Invoke-CIJob.ps1` orchestra 6 fasi:
|
||||
- **Phase 1**: `git clone` repo (con submoduli) su host in `C:\Temp\ci-src-<jobid>`
|
||||
- **Phase 2**: `New-BuildVM.ps1` — linked clone da template snapshot `BaseClean`
|
||||
- **Phase 3**: `vmrun start` VM headless
|
||||
- **Phase 4**: `Wait-VMReady.ps1` — polling WinRM readiness (~5s)
|
||||
- **Phase 5**: `Invoke-RemoteBuild.ps1` — trasferisce sorgente (zip), esegue build via WinRM, raccoglie `artifacts.zip`
|
||||
- **Phase 6**: `Get-BuildArtifacts.ps1` — copia zip in `F:\CI\Artifacts\<jobid>\`
|
||||
3. `finally`: `Remove-BuildVM.ps1` — stop VM + rimozione directory clone
|
||||
|
||||
### Caso d'uso implementato e testato
|
||||
- Repo: `Simone/nsis-plugin-nsinnounp` (NSIS plugin C++ — MSBuild)
|
||||
- Build command: `python build_plugin.py --final --dist-dir dist`
|
||||
- 4 configurazioni parallele: x86-unicode, x64-unicode, cli-x86, cli-x64
|
||||
- Artifact: `dist/x86-unicode/nsInnoUnp.dll`, `dist/amd64-unicode/nsInnoUnp.dll`, `dist/nsInnoUnpCLI.exe`, `dist/nsInnoUnpCLI64.exe`
|
||||
- e2e-009: SUCCESS in 02:09, cleanup automatico confermato
|
||||
|
||||
### Fix critici applicati durante sviluppo
|
||||
- **TRK0002 Windows quota exhaustion**: `build_plugin.py` divide thread count per numero di build parallele
|
||||
- **MSBuild detection per VS 2026**: fix in `build_plugin.py` per vswhere con VS 2026 BuildTools
|
||||
- **Output filter nascondeva errori**: filtro cambiato da substring a prefix match
|
||||
- **--dist-dir**: nuovo flag per raccogliere artifact in cartella piatta pre-clean
|
||||
- **--no-clean / --no-copy**: flag CI-friendly per skip cleanup e skip copia verso plugins dir
|
||||
@@ -4,9 +4,15 @@ Date: 2026-05-10
|
||||
|
||||
## Archived Files
|
||||
|
||||
### Setup-GiteaSSH.md
|
||||
**Reason**: Obsolete — current workflow uses HTTP URLs + credential manager for Gitea access, not SSH key-based authentication.
|
||||
**When to revisit**: If SSH-based Git clones become necessary (e.g., for server-to-server scenarios).
|
||||
### PLAN.md (archived 2026-05-10)
|
||||
**Reason**: Pianificazione iniziale (ultimo update 2026-05-08) superata dallo stato post Sprint 8.
|
||||
Stato corrente del sistema, decisioni tecniche e roadmap sono ora distribuiti tra:
|
||||
- `TODO.md` (root) per audit trail, summary status e backlog
|
||||
- `docs/ARCHITECTURE.md` per il design del sistema
|
||||
- `docs/CI-FLOW.md` per il flusso operativo
|
||||
- `docs/RUNBOOK.md` per il triage incident
|
||||
|
||||
Resta come riferimento storico.
|
||||
|
||||
### TEST-7.4-e2e.md
|
||||
**Reason**: Sprint 7.4 refactor validation document. Specific to §7.4 Deploy↔Setup refactor completion testing.
|
||||
@@ -16,6 +22,13 @@ Date: 2026-05-10
|
||||
- Post-refactor sign-off checklist
|
||||
**Useful for**: §6.6 (Toolchain Tier-1) template updates, Linux template setup (§6.1)
|
||||
|
||||
## De-archived
|
||||
|
||||
### Setup-GiteaSSH.md (restored 2026-05-10)
|
||||
Originariamente archiviato come "obsolete (HTTP+credential manager)" ma di fatto il workflow
|
||||
live (`gitea/workflows/build-nsis.yml`) e `Setup-Host.ps1` continuano a usare l'alias SSH
|
||||
`gitea-ci` (porta 2222) per i clone Git. Ripristinato in `docs/Setup-GiteaSSH.md`.
|
||||
|
||||
## Active Documentation
|
||||
|
||||
All current, operational docs remain in parent directory:
|
||||
@@ -24,7 +37,8 @@ All current, operational docs remain in parent directory:
|
||||
- CI-FLOW.md
|
||||
- HOST-SETUP.md
|
||||
- WINDOWS-TEMPLATE-SETUP.md
|
||||
- LINUX-TEMPLATE-SETUP.md (draft for §6.1)
|
||||
- OPTIMIZATION.md
|
||||
- RUNBOOK.md
|
||||
- TEST-PLAN-v1.3-to-HEAD.md
|
||||
- LINUX-TEMPLATE-SETUP.md (draft for §6.1)
|
||||
- Setup-GiteaSSH.md
|
||||
|
||||
Reference in New Issue
Block a user