Simone/local-ci-cd-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
0d2218cd4d5a8da3751692be23dea0a82eb8089d
local-ci-cd-system/docs/ARCHITECTURE.md
T
Simone f373c0c24b fix: apply all FIX-TODO items (P0/P1/P2) 
P0 bugs:
- Invoke-CIJob.ps1: skip --depth 1 when $Commit specified (shallow clone
  + checkout specific commit was silently failing)
- Get-BuildArtifacts.ps1: check .Length -eq 0 not rounded sizeKB (false
  fail on artifacts smaller than 50 bytes)
- build-nsis.yml: upgrade upload-artifact v3 -> v4 (v3 deprecated)

P0 security:
- Remove hardcoded default password CIBuild!ChangeMe2026 from all scripts,
  README, TODO; prompt Read-Host -AsSecureString at runtime instead
- Setup-TemplateVM.ps1: replace `net user $u $p /add` (password in argv /
  audit log 4688) with New-LocalUser + local ConvertTo-SecureString
- Prepare-TemplateSetup.ps1: save and restore WSMan AllowUnencrypted +
  TrustedHosts in finally block (was permanently mutating host WinRM config)
- Setup-TemplateVM.ps1: remove duplicate MaxMemoryPerShellMB (winrm set
  + Set-Item both setting same value)

P1 doc (stale Host-Only / VMnet11 era):
- Setup-TemplateVM.ps1: rewrite NETWORK REQUIREMENT block + final steps
  (system uses VMnet8 NAT permanently, not VMnet11 Host-Only)
- Invoke-CIJob.ps1 Phase 1 comment: correct network model
- ARCHITECTURE.md: remove Git from toolchain list (not installed); clarify
  zip-transfer rationale (no PAT in VM, not network isolation)
- BEST-PRACTICES.md: rewrite Step 8 as NAT topology verification

P1 minor:
- Invoke-RemoteBuild.ps1: remove wasted first New-Item (created then
  immediately removed and recreated)
- Wait-VMReady.ps1: remove unused $elapsed; simplify try/catch around
  native command (exit codes don't throw)
- Install-Runner.ps1: add deprecation notice pointing to Setup-Host.ps1

P2 operational:
- Add scripts/Cleanup-OrphanedBuildVMs.ps1 with -WhatIf support
- Get-BuildArtifacts.ps1 -IncludeLogs: add -Recurse (msbuild logs in subdir)
- Add gitea/workflows/lint.yml (PSScriptAnalyzer on .ps1 push/PR)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 00:39:52 +02:00

12 KiB
Raw Blame History

Architecture — Local CI/CD System

Overview

A fully local, isolated CI/CD pipeline running on a Windows host using:

  • Gitea as the self-hosted Git server + CI platform
  • act_runner as the job executor (runs on the Windows host)
  • VMware Workstation for ephemeral build VMs (linked clones)
  • WinRM / PowerShell Remoting for communication with build VMs
  • MSBuild / dotnet CLI executing inside the guest VM only

The runner host never executes build tools directly. Its only role is orchestration: creating, driving, and destroying VMs.

System Diagram

┌─────────────────────────────────────────────────────────────────────────┐
│  DEVELOPER WORKSTATION                                                  │
│                                                                         │
│  git push → Gitea (self-hosted, LAN)                                    │
└───────────────────────────┬─────────────────────────────────────────────┘
                            │  HTTP/HTTPS (Gitea Actions API)
                            ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  WINDOWS HOST  (i9-10900X · 64 GB RAM · NVMe SSD)                       │
│                                                                         │
│  ┌───────────────────┐     polls job queue      ┌──────────────────┐    │
│  │   Gitea Server    │◄────────────────────────►│   act_runner     │    │
│  │  (Actions API)    │   reports status/logs    │  (Windows svc)   │    │
│  └───────────────────┘                          └────────┬─────────┘    │
│                                                          │              │
│                                           calls Invoke-CIJob.ps1        │
│                                                          │              │
│                                                          ▼              │
│  ┌───────────────────────────────────────────────────────────────────┐  │
│  │  Orchestrator Scripts (PowerShell)                                │  │
│  │                                                                   │  │
│  │  New-BuildVM.ps1 ──► vmrun.exe (linked clone from template)       │  │
│  │  Wait-VMReady.ps1 ─► Test-WSMan poll loop                         │  │
│  │  Invoke-RemoteBuild.ps1 ► PSSession + Invoke-Command              │  │
│  │  Get-BuildArtifacts.ps1 ► Copy-Item -FromSession                  │  │
│  │  Remove-BuildVM.ps1 ───► vmrun stop + deleteVM                    │  │
│  └───────────────────────────────┬───────────────────────────────────┘  │
│                                  │  vmrun.exe (VMware CLI)              │
│                                  ▼                                      │
│  ┌───────────────────────────────────────────────────────────────────┐  │
│  │  VMware Workstation                                               │  │
│  │                                                                   │  │
│  │  Template VM ──snapshot "BaseClean"──┐                            │  │
│  │                                      ├── Clone_Job_001.vmx        │  │
│  │                                      ├── Clone_Job_002.vmx        │  │
│  │                                      └── Clone_Job_003.vmx        │  │
│  └───────────────────────────────────────────────────────────────────┘  │
│                                  │  WinRM :5985 (192.168.79.0/24 NAT)   │
└──────────────────────────────────┼──────────────────────────────────────┘
                                   │
          ┌────────────────────────┼──────────────────────────┐
          │                        │                          │
          ▼                        ▼                          ▼
┌──────────────────┐    ┌──────────────────┐    ┌──────────────────┐
│  Build VM #1     │    │  Build VM #2     │    │  Build VM #N     │
│  (ephemeral)     │    │  (ephemeral)     │    │  (ephemeral)     │
│                  │    │                  │    │                  │
│  VS Build Tools  │    │  VS Build Tools  │    │  VS Build Tools  │
│  2026 (v145)     │    │  2026 (v145)     │    │  2026 (v145)     │
│  .NET SDK 10     │    │  .NET SDK 10     │    │  .NET SDK 10     │
│  Python 3.13.3   │    │  Python 3.13.3   │    │  Python 3.13.3   │
│  WinRM enabled   │    │  WinRM enabled   │    │  WinRM enabled   │
│                  │    │                  │    │                  │
│  > unzip src     │    │  > unzip src     │    │  > unzip src     │
│  > python build  │    │  > python build  │    │  > python build  │
│  > dist/         │    │  > dist/         │    │  > dist/         │
│                  │    │                  │    │                  │
│  [DESTROYED]     │    │  [DESTROYED]     │    │  [DESTROYED]     │
└──────────────────┘    └──────────────────┘    └──────────────────┘

Component Descriptions

Gitea (Self-hosted Git + CI)

  • Hosts Git repositories
  • Provides Gitea Actions (GitHub Actions-compatible YAML workflow engine)
  • Manages job queue and reports build status to commits/PRs
  • Stores build artifacts uploaded by act_runner

act_runner (Job Executor)

  • Runs as a Windows service on the host
  • Polls Gitea Actions API for pending jobs
  • Executes .gitea/workflows/*.yml steps
  • Configured with container.enabled: false — does NOT use Docker
  • Each step runs as a PowerShell command on the host (orchestration only)
  • Supports capacity: N for concurrent job execution (N parallel VMs)

VMware Workstation + vmrun CLI

  • Manages the template VM and all ephemeral clones
  • vmrun.exe is the command-line interface for all VM lifecycle operations
  • Linked clones are used: each build VM is a delta clone from a frozen template snapshot
    • Clone creation: ~5–15 seconds (vs 60–120 seconds for full clone)
    • Disk footprint: ~2–5 GB per clone (vs 40–80 GB for full clone)
    • Tradeoff: template snapshot must remain intact

WinRM / PowerShell Remoting

  • Default transport for remote build execution inside VMs
  • Port: 5985 (HTTP, in-lab use); migrate to 5986/HTTPS for hardened setups
  • Authentication: Basic (unencrypted — acceptable for isolated lab)
  • Used for:
    • Transferring source: Compress-Archive on host → WinRM copy → Expand-Archive in guest
    • Running build commands (Invoke-Command)
    • Packaging artifacts (Compress-Archive in guest) and collecting via Copy-Item -FromSession

Build Toolchain (inside VM only)

  • Visual Studio Build Tools 2026 (MSBuild 18.5.4 / toolset v145)
  • .NET SDK 10.0.203
  • Python 3.13.3 (per build script personalizzati come build_plugin.py)
  • NuGet CLI (optional, dotnet restore covers most cases)
  • Git: NOT installed by default — source arrives via WinRM zip transfer. See TODO.md -UseGitClone for the in-VM git clone opt-in path.

Network Topology

Host NIC (LAN)       → 192.168.1.x     → Gitea server (10.10.20.11:3100), developer machines
VMware VMnet8 (NAT)  → 192.168.79.0/24 → Build VMs (DHCP, internet access via NAT)

Build VMs:
  Clone_Job_001 → 192.168.79.131
  Clone_Job_002 → 192.168.79.132
  ...
  Clone_Job_N   → 192.168.79.1xx

WinRM port 5985 (HTTP) on each VM IP.
VMs have internet access via NAT — required for pip/nuget during build.

Note: VMnet8 NAT gives VMs internet access. The host detects the VM IP via vmrun getGuestIPAddress after boot — no static IP assignment needed.

VM Lifecycle

1. TEMPLATE (static, locked)
   └─ Snapshot: "BaseClean"  ───────────────────────┐
                                                    │
2. CLONE CREATION                                   │
   vmrun clone <template.vmx> <clone.vmx> linked ──►│
   Duration: ~5–15 seconds                          │
                                                    │
3. START                                            │
   vmrun start <clone.vmx> nogui                    │
   Duration: ~20–40 seconds boot                    │
                                                    │
4. READINESS CHECK                                  │
   Poll: getState=running → ping → Test-WSMan       │
   Duration: ~30–90 seconds total                   │
                                                    │
5. BUILD                                            │
   Copy script → Invoke-Command → collect artifacts │
   Duration: varies (seconds to minutes)            │
                                                    │
6. DESTROY                                          │
   vmrun stop hard -> vmrun deleteVM → Remove-Item  │
   Duration: ~5–10 seconds                          │
   Result: zero disk footprint remaining            ▼

Resource Limits & Parallel Capacity

Resource Host Total Per VM (est.) Max VMs
RAM 64 GB ~6–8 GB 8–10
CPU 20 threads (i9-10900X) ~4 vCPU 5
NVMe IOPS High Clone delta I/O ~6–8
Recommended — — 4 concurrent

act_runner capacity: 4 is the conservative default. Raise to 6 after benchmarking.

Security Boundaries

  • Build VMs are on VMnet8 NAT (192.168.79.0/24) — host reaches VMs, VMs have internet access
  • WinRM credentials are stored in Windows Credential Manager on host (not hardcoded)
  • Each VM is fully destroyed after the job — no state carries between builds
  • Runner host does not expose any ports to the build VM network
  • Source code is transferred via zip (Compress-Archive → WinRM copy → Expand-Archive in guest) — no PAT is ever present inside the VM; VMs have NAT internet for pip/nuget at build time
Reference in New Issue View Git Blame Copy Permalink