Simone
f373c0c24b
P0 bugs: - Invoke-CIJob.ps1: skip --depth 1 when $Commit specified (shallow clone + checkout specific commit was silently failing) - Get-BuildArtifacts.ps1: check .Length -eq 0 not rounded sizeKB (false fail on artifacts smaller than 50 bytes) - build-nsis.yml: upgrade upload-artifact v3 -> v4 (v3 deprecated) P0 security: - Remove hardcoded default password CIBuild!ChangeMe2026 from all scripts, README, TODO; prompt Read-Host -AsSecureString at runtime instead - Setup-TemplateVM.ps1: replace `net user $u $p /add` (password in argv / audit log 4688) with New-LocalUser + local ConvertTo-SecureString - Prepare-TemplateSetup.ps1: save and restore WSMan AllowUnencrypted + TrustedHosts in finally block (was permanently mutating host WinRM config) - Setup-TemplateVM.ps1: remove duplicate MaxMemoryPerShellMB (winrm set + Set-Item both setting same value) P1 doc (stale Host-Only / VMnet11 era): - Setup-TemplateVM.ps1: rewrite NETWORK REQUIREMENT block + final steps (system uses VMnet8 NAT permanently, not VMnet11 Host-Only) - Invoke-CIJob.ps1 Phase 1 comment: correct network model - ARCHITECTURE.md: remove Git from toolchain list (not installed); clarify zip-transfer rationale (no PAT in VM, not network isolation) - BEST-PRACTICES.md: rewrite Step 8 as NAT topology verification P1 minor: - Invoke-RemoteBuild.ps1: remove wasted first New-Item (created then immediately removed and recreated) - Wait-VMReady.ps1: remove unused $elapsed; simplify try/catch around native command (exit codes don't throw) - Install-Runner.ps1: add deprecation notice pointing to Setup-Host.ps1 P2 operational: - Add scripts/Cleanup-OrphanedBuildVMs.ps1 with -WhatIf support - Get-BuildArtifacts.ps1 -IncludeLogs: add -Recurse (msbuild logs in subdir) - Add gitea/workflows/lint.yml (PSScriptAnalyzer on .ps1 push/PR) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
195 lines
12 KiB
Markdown
195 lines
12 KiB
Markdown
# Architecture — Local CI/CD System
|
||
|
||
## Overview
|
||
|
||
A fully local, isolated CI/CD pipeline running on a Windows host using:
|
||
- **Gitea** as the self-hosted Git server + CI platform
|
||
- **act_runner** as the job executor (runs on the Windows host)
|
||
- **VMware Workstation** for ephemeral build VMs (linked clones)
|
||
- **WinRM / PowerShell Remoting** for communication with build VMs
|
||
- **MSBuild / dotnet CLI** executing inside the guest VM only
|
||
|
||
The runner host **never executes build tools directly**. Its only role is orchestration: creating, driving, and destroying VMs.
|
||
|
||
---
|
||
|
||
## System Diagram
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────────────┐
|
||
│ DEVELOPER WORKSTATION │
|
||
│ │
|
||
│ git push → Gitea (self-hosted, LAN) │
|
||
└───────────────────────────┬─────────────────────────────────────────────┘
|
||
│ HTTP/HTTPS (Gitea Actions API)
|
||
▼
|
||
┌─────────────────────────────────────────────────────────────────────────┐
|
||
│ WINDOWS HOST (i9-10900X · 64 GB RAM · NVMe SSD) │
|
||
│ │
|
||
│ ┌───────────────────┐ polls job queue ┌──────────────────┐ │
|
||
│ │ Gitea Server │◄────────────────────────►│ act_runner │ │
|
||
│ │ (Actions API) │ reports status/logs │ (Windows svc) │ │
|
||
│ └───────────────────┘ └────────┬─────────┘ │
|
||
│ │ │
|
||
│ calls Invoke-CIJob.ps1 │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ ┌───────────────────────────────────────────────────────────────────┐ │
|
||
│ │ Orchestrator Scripts (PowerShell) │ │
|
||
│ │ │ │
|
||
│ │ New-BuildVM.ps1 ──► vmrun.exe (linked clone from template) │ │
|
||
│ │ Wait-VMReady.ps1 ─► Test-WSMan poll loop │ │
|
||
│ │ Invoke-RemoteBuild.ps1 ► PSSession + Invoke-Command │ │
|
||
│ │ Get-BuildArtifacts.ps1 ► Copy-Item -FromSession │ │
|
||
│ │ Remove-BuildVM.ps1 ───► vmrun stop + deleteVM │ │
|
||
│ └───────────────────────────────┬───────────────────────────────────┘ │
|
||
│ │ vmrun.exe (VMware CLI) │
|
||
│ ▼ │
|
||
│ ┌───────────────────────────────────────────────────────────────────┐ │
|
||
│ │ VMware Workstation │ │
|
||
│ │ │ │
|
||
│ │ Template VM ──snapshot "BaseClean"──┐ │ │
|
||
│ │ ├── Clone_Job_001.vmx │ │
|
||
│ │ ├── Clone_Job_002.vmx │ │
|
||
│ │ └── Clone_Job_003.vmx │ │
|
||
│ └───────────────────────────────────────────────────────────────────┘ │
|
||
│ │ WinRM :5985 (192.168.79.0/24 NAT) │
|
||
└──────────────────────────────────┼──────────────────────────────────────┘
|
||
│
|
||
┌────────────────────────┼──────────────────────────┐
|
||
│ │ │
|
||
▼ ▼ ▼
|
||
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
|
||
│ Build VM #1 │ │ Build VM #2 │ │ Build VM #N │
|
||
│ (ephemeral) │ │ (ephemeral) │ │ (ephemeral) │
|
||
│ │ │ │ │ │
|
||
│ VS Build Tools │ │ VS Build Tools │ │ VS Build Tools │
|
||
│ 2026 (v145) │ │ 2026 (v145) │ │ 2026 (v145) │
|
||
│ .NET SDK 10 │ │ .NET SDK 10 │ │ .NET SDK 10 │
|
||
│ Python 3.13.3 │ │ Python 3.13.3 │ │ Python 3.13.3 │
|
||
│ WinRM enabled │ │ WinRM enabled │ │ WinRM enabled │
|
||
│ │ │ │ │ │
|
||
│ > unzip src │ │ > unzip src │ │ > unzip src │
|
||
│ > python build │ │ > python build │ │ > python build │
|
||
│ > dist/ │ │ > dist/ │ │ > dist/ │
|
||
│ │ │ │ │ │
|
||
│ [DESTROYED] │ │ [DESTROYED] │ │ [DESTROYED] │
|
||
└──────────────────┘ └──────────────────┘ └──────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## Component Descriptions
|
||
|
||
### Gitea (Self-hosted Git + CI)
|
||
- Hosts Git repositories
|
||
- Provides Gitea Actions (GitHub Actions-compatible YAML workflow engine)
|
||
- Manages job queue and reports build status to commits/PRs
|
||
- Stores build artifacts uploaded by act_runner
|
||
|
||
### act_runner (Job Executor)
|
||
- Runs as a Windows service on the host
|
||
- Polls Gitea Actions API for pending jobs
|
||
- Executes `.gitea/workflows/*.yml` steps
|
||
- **Configured with `container.enabled: false`** — does NOT use Docker
|
||
- Each step runs as a PowerShell command on the host (orchestration only)
|
||
- Supports `capacity: N` for concurrent job execution (N parallel VMs)
|
||
|
||
### VMware Workstation + vmrun CLI
|
||
- Manages the template VM and all ephemeral clones
|
||
- `vmrun.exe` is the command-line interface for all VM lifecycle operations
|
||
- **Linked clones** are used: each build VM is a delta clone from a frozen template snapshot
|
||
- Clone creation: ~5–15 seconds (vs 60–120 seconds for full clone)
|
||
- Disk footprint: ~2–5 GB per clone (vs 40–80 GB for full clone)
|
||
- Tradeoff: template snapshot must remain intact
|
||
|
||
### WinRM / PowerShell Remoting
|
||
- Default transport for remote build execution inside VMs
|
||
- Port: **5985** (HTTP, in-lab use); migrate to 5986/HTTPS for hardened setups
|
||
- Authentication: **Basic** (unencrypted — acceptable for isolated lab)
|
||
- Used for:
|
||
- Transferring source: `Compress-Archive` on host → WinRM copy → `Expand-Archive` in guest
|
||
- Running build commands (`Invoke-Command`)
|
||
- Packaging artifacts (`Compress-Archive` in guest) and collecting via `Copy-Item -FromSession`
|
||
|
||
### Build Toolchain (inside VM only)
|
||
- Visual Studio Build Tools **2026** (MSBuild 18.5.4 / toolset v145)
|
||
- .NET SDK **10.0.203**
|
||
- Python **3.13.3** (per build script personalizzati come `build_plugin.py`)
|
||
- NuGet CLI (optional, dotnet restore covers most cases)
|
||
- Git: NOT installed by default — source arrives via WinRM zip transfer. See TODO.md `-UseGitClone` for the in-VM git clone opt-in path.
|
||
|
||
---
|
||
|
||
## Network Topology
|
||
|
||
```
|
||
Host NIC (LAN) → 192.168.1.x → Gitea server (10.10.20.11:3100), developer machines
|
||
VMware VMnet8 (NAT) → 192.168.79.0/24 → Build VMs (DHCP, internet access via NAT)
|
||
|
||
Build VMs:
|
||
Clone_Job_001 → 192.168.79.131
|
||
Clone_Job_002 → 192.168.79.132
|
||
...
|
||
Clone_Job_N → 192.168.79.1xx
|
||
|
||
WinRM port 5985 (HTTP) on each VM IP.
|
||
VMs have internet access via NAT — required for pip/nuget during build.
|
||
```
|
||
|
||
> **Note:** VMnet8 NAT gives VMs internet access. The host detects the VM IP via
|
||
> `vmrun getGuestIPAddress` after boot — no static IP assignment needed.
|
||
|
||
---
|
||
|
||
## VM Lifecycle
|
||
|
||
```
|
||
1. TEMPLATE (static, locked)
|
||
└─ Snapshot: "BaseClean" ───────────────────────┐
|
||
│
|
||
2. CLONE CREATION │
|
||
vmrun clone <template.vmx> <clone.vmx> linked ──►│
|
||
Duration: ~5–15 seconds │
|
||
│
|
||
3. START │
|
||
vmrun start <clone.vmx> nogui │
|
||
Duration: ~20–40 seconds boot │
|
||
│
|
||
4. READINESS CHECK │
|
||
Poll: getState=running → ping → Test-WSMan │
|
||
Duration: ~30–90 seconds total │
|
||
│
|
||
5. BUILD │
|
||
Copy script → Invoke-Command → collect artifacts │
|
||
Duration: varies (seconds to minutes) │
|
||
│
|
||
6. DESTROY │
|
||
vmrun stop hard -> vmrun deleteVM → Remove-Item │
|
||
Duration: ~5–10 seconds │
|
||
Result: zero disk footprint remaining ▼
|
||
```
|
||
|
||
---
|
||
|
||
## Resource Limits & Parallel Capacity
|
||
|
||
| Resource | Host Total | Per VM (est.) | Max VMs |
|
||
| --------------- | ---------------------- | --------------- | ---------------- |
|
||
| RAM | 64 GB | ~6–8 GB | 8–10 |
|
||
| CPU | 20 threads (i9-10900X) | ~4 vCPU | 5 |
|
||
| NVMe IOPS | High | Clone delta I/O | ~6–8 |
|
||
| **Recommended** | — | — | **4 concurrent** |
|
||
|
||
`act_runner` `capacity: 4` is the conservative default. Raise to 6 after benchmarking.
|
||
|
||
---
|
||
|
||
## Security Boundaries
|
||
|
||
- Build VMs are on VMnet8 NAT (192.168.79.0/24) — host reaches VMs, VMs have internet access
|
||
- WinRM credentials are stored in Windows Credential Manager on host (not hardcoded)
|
||
- Each VM is fully destroyed after the job — no state carries between builds
|
||
- Runner host does not expose any ports to the build VM network
|
||
- Source code is transferred via zip (Compress-Archive → WinRM copy → Expand-Archive in guest) — no PAT is ever present inside the VM; VMs have NAT internet for pip/nuget at build time
|