Simone
1c8fdcf028
AGENTS.md (new): - Environment reference for AI agents: host specs, PS 5.1 constraints, dir structure, VM templates, transport layer, style rules - Lessons #9 (snapshot cold state) and #10 (vmrun list vs getGuestIPAddress) ARCHITECTURE.md: - Add SSH/SCP transport section (Linux VMs) - Update system diagram to include _Transport.psm1, SSH transport path - Network layout: add SSH port 22 for Linux VMs - Timeline: Linux readiness check phases documented CI-FLOW.md: - Prerequisites table: add Template VM (Linux), SSH key row - Step 6 (readiness): document Windows vs Linux phases - Step 7 (build) and Step 8 (artifacts): document Linux/SSH paths - ci-report-ip guestinfo mechanism referenced HOST-SETUP.md: - Directory tree: add LinuxBuild2404 template dir and keys/ - Post-setup checklist: add step 5 (SSH key generation + Linux template) - Fix duplicate step numbering (56, 67) - Reference Paths table: add Linux template VMX, snapshot, SSH key LINUX-TEMPLATE-SETUP.md: - Rewritten as implementation record (no longer a plan) - Fase A user-data: minimal (user+SSH key only, no packages/runcmd) - All phases marked complete with actual paths and snapshot names WINDOWS-TEMPLATE-SETUP.md: minor updates for consistency TODO.md: - Summary: update last-updated to 2026-05-11, Sprint 9 description - Linux Build VM row: all 5 items complete - Prossimi passi: Test Plan + §6.6 Tier-2 as next priorities
209 lines
13 KiB
Markdown
209 lines
13 KiB
Markdown
# Architecture — Local CI/CD System
|
||
|
||
## Overview
|
||
|
||
A fully local, isolated CI/CD pipeline running on a Windows host using:
|
||
- **Gitea** as the self-hosted Git server + CI platform
|
||
- **act_runner** as the job executor (runs on the Windows host)
|
||
- **VMware Workstation** for ephemeral build VMs (linked clones)
|
||
- **WinRM / PowerShell Remoting** for communication with Windows build VMs
|
||
- **SSH / SCP** (`ssh.exe` + `scp.exe`) for communication with Linux build VMs
|
||
- **MSBuild / dotnet CLI** (Windows) and **GCC / Clang / CMake** (Linux) executing inside guest VMs only
|
||
|
||
The runner host **never executes build tools directly**. Its only role is orchestration: creating, driving, and destroying VMs.
|
||
|
||
---
|
||
|
||
## System Diagram
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────────────┐
|
||
│ DEVELOPER WORKSTATION │
|
||
│ │
|
||
│ git push → Gitea (self-hosted, LAN) │
|
||
└───────────────────────────┬─────────────────────────────────────────────┘
|
||
│ HTTP/HTTPS (Gitea Actions API)
|
||
▼
|
||
┌─────────────────────────────────────────────────────────────────────────┐
|
||
│ WINDOWS HOST (i9-10900X · 64 GB RAM · NVMe SSD) │
|
||
│ │
|
||
│ ┌───────────────────┐ polls job queue ┌──────────────────┐ │
|
||
│ │ Gitea Server │◄────────────────────────►│ act_runner │ │
|
||
│ │ (Actions API) │ reports status/logs │ (Windows svc) │ │
|
||
│ └───────────────────┘ └────────┬─────────┘ │
|
||
│ │ │
|
||
│ calls Invoke-CIJob.ps1 │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ ┌───────────────────────────────────────────────────────────────────┐ │
|
||
│ │ Orchestrator Scripts (PowerShell) │ │
|
||
│ │ │ │
|
||
│ │ New-BuildVM.ps1 ──► vmrun.exe (linked clone from template) │ │
|
||
│ Wait-VMReady.ps1 ─► WinRM poll (Windows) / SSH poll (Linux) │ │
|
||
│ Invoke-RemoteBuild.ps1 ► PSSession (Windows) / SSH+SCP (Linux) │ │
|
||
│ Get-BuildArtifacts.ps1 ► Copy-Item -FromSession / scp (Linux) │ │
|
||
│ Remove-BuildVM.ps1 ───► vmrun stop + deleteVM │ │
|
||
│ _Transport.psm1 ──────► Invoke-SshCommand / Copy-SshItem │ │
|
||
│ └───────────────────────────────┬───────────────────────────────────┘ │
|
||
│ │ vmrun.exe (VMware CLI) │
|
||
│ ▼ │
|
||
│ ┌───────────────────────────────────────────────────────────────────┐ │
|
||
│ │ VMware Workstation │ │
|
||
│ │ │ │
|
||
│ │ Template VM ──snapshot "BaseClean"──┐ │ │
|
||
│ │ ├── Clone_Job_001.vmx │ │
|
||
│ │ ├── Clone_Job_002.vmx │ │
|
||
│ │ └── Clone_Job_003.vmx │ │
|
||
│ └───────────────────────────────────────────────────────────────────┘ │
|
||
│ │ WinRM :5986 HTTPS (192.168.79.0/24 NAT) │
|
||
└──────────────────────────────────┼──────────────────────────────────────┘
|
||
│
|
||
┌────────────────────────┼──────────────────────────┐
|
||
│ │ │
|
||
▼ ▼ ▼
|
||
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
|
||
│ Build VM #1 │ │ Build VM #2 │ │ Build VM #N │
|
||
│ (ephemeral) │ │ (ephemeral) │ │ (ephemeral) │
|
||
│ │ │ │ │ │
|
||
│ VS Build Tools │ │ VS Build Tools │ │ VS Build Tools │
|
||
│ 2026 (v145) │ │ 2026 (v145) │ │ 2026 (v145) │
|
||
│ .NET SDK 10 │ │ .NET SDK 10 │ │ .NET SDK 10 │
|
||
│ Python 3.13.3 │ │ Python 3.13.3 │ │ Python 3.13.3 │
|
||
│ WinRM enabled │ │ WinRM enabled │ │ WinRM enabled │
|
||
│ │ │ │ │ │
|
||
│ > unzip src │ │ > unzip src │ │ > unzip src │
|
||
│ > python build │ │ > python build │ │ > python build │
|
||
│ > dist/ │ │ > dist/ │ │ > dist/ │
|
||
│ │ │ │ │ │
|
||
│ [DESTROYED] │ │ [DESTROYED] │ │ [DESTROYED] │
|
||
└──────────────────┘ └──────────────────┘ └──────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## Component Descriptions
|
||
|
||
### Gitea (Self-hosted Git + CI)
|
||
- Hosts Git repositories
|
||
- Provides Gitea Actions (GitHub Actions-compatible YAML workflow engine)
|
||
- Manages job queue and reports build status to commits/PRs
|
||
- Stores build artifacts uploaded by act_runner
|
||
|
||
### act_runner (Job Executor)
|
||
- Runs as a Windows service on the host
|
||
- Polls Gitea Actions API for pending jobs
|
||
- Executes `.gitea/workflows/*.yml` steps
|
||
- **Configured with `container.enabled: false`** — does NOT use Docker
|
||
- Each step runs as a PowerShell command on the host (orchestration only)
|
||
- Supports `capacity: N` for concurrent job execution (N parallel VMs)
|
||
|
||
### VMware Workstation + vmrun CLI
|
||
- Manages the template VM and all ephemeral clones
|
||
- `vmrun.exe` is the command-line interface for all VM lifecycle operations
|
||
- **Linked clones** are used: each build VM is a delta clone from a frozen template snapshot
|
||
- Clone creation: ~5–15 seconds (vs 60–120 seconds for full clone)
|
||
- Disk footprint: ~2–5 GB per clone (vs 40–80 GB for full clone)
|
||
- Tradeoff: template snapshot must remain intact
|
||
|
||
### WinRM / PowerShell Remoting (Windows VMs)
|
||
- Transport for remote build execution inside **Windows** build VMs
|
||
- Port: **5986** (HTTPS, self-signed cert, `AllowUnencrypted=false`)
|
||
- Authentication: **Basic over HTTPS** (`-SkipCACheck` for lab self-signed cert)
|
||
- Used for:
|
||
- Transferring source: `Compress-Archive` on host → WinRM copy → `Expand-Archive` in guest
|
||
- Running build commands (`Invoke-Command`)
|
||
- Packaging artifacts (`Compress-Archive` in guest) and collecting via `Copy-Item -FromSession`
|
||
|
||
### SSH / SCP Transport (Linux VMs)
|
||
- Transport for remote build execution inside **Linux** build VMs
|
||
- Port: **22** (key-based auth, ed25519 key `F:\CI\keys\ci_linux`, user `ci_build`)
|
||
- No password — `BatchMode=yes`, `StrictHostKeyChecking=accept-new`
|
||
- Implemented in `scripts/_Transport.psm1` (exported: `Invoke-SshCommand`, `Copy-SshItem`, `Test-SshReady`)
|
||
- Used by `Wait-VMReady.ps1 -Transport SSH`, `Invoke-RemoteBuild.ps1 -GuestOS Linux`, `Get-BuildArtifacts.ps1 -GuestOS Linux`
|
||
- Selected automatically when `Invoke-CIJob.ps1` detects `guestOS = ubuntu-64` in the cloned VMX
|
||
|
||
### Build Toolchain (inside VM only)
|
||
- Visual Studio Build Tools **2026** (MSBuild 18.5.4 / toolset v145)
|
||
- .NET SDK **10.0.203**
|
||
- Python **3.13.3** (per build script personalizzati come `build_plugin.py`)
|
||
- NuGet CLI (optional, dotnet restore covers most cases)
|
||
- **Git for Windows 2.54** + **7-Zip 26.01** (Tier-1 toolchain, §6.6 — DONE 2026-05-10): consente l'opt-in `-UseGitClone` di `Invoke-CIJob.ps1` per clonare il repository direttamente in VM (skip host zip transfer, e2e PASS, -25.7% durata pipeline).
|
||
|
||
---
|
||
|
||
## Network Topology
|
||
|
||
```
|
||
Host NIC (LAN) → 192.168.1.x → Gitea server (10.10.20.11:3100), developer machines
|
||
VMware VMnet8 (NAT) → 192.168.79.0/24 → Build VMs (DHCP, internet access via NAT)
|
||
|
||
Build VMs:
|
||
Clone_Job_001 → 192.168.79.131
|
||
Clone_Job_002 → 192.168.79.132
|
||
...
|
||
Clone_Job_N → 192.168.79.1xx
|
||
|
||
WinRM port 5986 (HTTPS, self-signed) on each Windows VM IP.
|
||
SSH port 22 on each Linux VM IP (key-based, ci_build@<ip>, key: F:\CI\keys\ci_linux).
|
||
VMs have internet access via NAT — required for pip/nuget/apt during build.
|
||
```
|
||
|
||
> **Note:** VMnet8 NAT gives VMs internet access. The host detects the VM IP via
|
||
> `vmrun getGuestIPAddress` after boot — no static IP assignment needed.
|
||
|
||
---
|
||
|
||
## VM Lifecycle
|
||
|
||
```
|
||
1. TEMPLATE (static, locked)
|
||
└─ Snapshot: "BaseClean" ───────────────────────┐
|
||
│
|
||
2. CLONE CREATION │
|
||
vmrun clone <template.vmx> <clone.vmx> linked ──►│
|
||
Duration: ~5–15 seconds │
|
||
│
|
||
3. START │
|
||
vmrun start <clone.vmx> nogui │
|
||
Duration: ~20–40 seconds boot │
|
||
│
|
||
4. READINESS CHECK │
|
||
Windows: getState=running → ping → Test-WSMan │
|
||
Linux: getState=running → TCP:22 → ssh echo │
|
||
Duration: ~30–90 seconds total │
|
||
│
|
||
5. BUILD │
|
||
Copy script → Invoke-Command → collect artifacts │
|
||
Duration: varies (seconds to minutes) │
|
||
│
|
||
6. DESTROY │
|
||
vmrun stop hard -> vmrun deleteVM → Remove-Item │
|
||
Duration: ~5–10 seconds │
|
||
Result: zero disk footprint remaining ▼
|
||
```
|
||
|
||
---
|
||
|
||
## Resource Limits & Parallel Capacity
|
||
|
||
| Resource | Host Total | Per VM (est.) | Max VMs |
|
||
| --------------- | ---------------------- | --------------- | ---------------- |
|
||
| RAM | 64 GB | ~6–8 GB | 8–10 |
|
||
| CPU | 20 threads (i9-10900X) | ~4 vCPU | 5 |
|
||
| NVMe IOPS | High | Clone delta I/O | ~6–8 |
|
||
| **Recommended** | — | — | **4 concurrent** |
|
||
|
||
`act_runner` `capacity: 4` is the conservative default. Raise to 6 after benchmarking.
|
||
|
||
---
|
||
|
||
## Security Boundaries
|
||
|
||
- Build VMs are on VMnet8 NAT (192.168.79.0/24) — host reaches VMs, VMs have internet access
|
||
- WinRM credentials are stored in Windows Credential Manager on host (not hardcoded)
|
||
- Each VM is fully destroyed after the job — no state carries between builds
|
||
- Runner host does not expose any ports to the build VM network
|
||
- Source code is transferred via zip (Compress-Archive → WinRM copy → Expand-Archive in guest) by default — no PAT is ever present inside the VM; VMs have NAT internet for pip/nuget at build time.
|
||
- With `-UseGitClone` (§3.3, opt-in) il clone avviene dentro la VM via SSH alias `gitea-ci` (forwardato dall'host); la chiave privata SSH dell'host non viene esposta al guest — vedi BEST-PRACTICES.md per il threat model.
|
||
- Linux build VMs usano la chiave SSH `F:\CI\keys\ci_linux` (ed25519, no passphrase), archiviata solo sull'host. Il guest non conosce credenziali WinRM.
|