Simone
6ba8cca70a
- ARCHITECTURE.md: fix WinRM subnet (192.168.1179), VM box shows VS 2026/Python 3.13/unzip+python build instead of .NET SDK 8/git clone/dotnet build - BEST-PRACTICES.md: fix VMnet11/192.168.11 VMnet8 NAT/192.168.79 - CI-FLOW.md: Step 3 workflow YAML updated to actual Invoke-CIJob.ps1 params (Submodules, BuildCommand, GuestArtifactSource); note that -VMIPAddress is auto-detected - OPTIMIZATION.md: BaseClean tier updated to VS BuildTools 2026, .NET SDK 10.0.203, Python 3.13.3 - gitea/workflow-example.yml: remove BUILD_VM_IP (auto-detected), replace -VMIPAddress/-Configuration with -Submodules/-BuildCommand/-GuestArtifactSource
198 lines
6.5 KiB
Markdown
198 lines
6.5 KiB
Markdown
# Optimization Strategies
|
||
|
||
## 1. Linked Clone Disk Layout
|
||
|
||
**Goal:** Minimize clone creation time and I/O contention between concurrent VMs.
|
||
|
||
### Recommended NVMe Partition Layout
|
||
|
||
```
|
||
NVMe SSD (e.g. 2TB)
|
||
├── C:\ — Windows host OS, VMware Workstation installation
|
||
├── F:\CI\
|
||
│ ├── Templates\ — Template VM VMX + base VMDK (parent snapshot)
|
||
│ ├── BuildVMs\ — Ephemeral linked clone VMs (delta VMDKs)
|
||
│ ├── Artifacts\ — Collected build artifacts (per job)
|
||
│ ├── Cache\ — NuGet / npm cache (see §4)
|
||
│ └── RunnerWork\ — act_runner workspace (checkout, step scripts)
|
||
```
|
||
|
||
**Why separate directories matter:**
|
||
- Template VMDK reads (CoW base) and clone delta writes happen simultaneously
|
||
- Keeping them on the same fast NVMe avoids I/O stalls; a separate spinning disk for
|
||
the clone directory would bottleneck clone creation
|
||
- Artifacts and cache dirs have sequential I/O patterns; they can share space
|
||
|
||
---
|
||
|
||
## 2. Snapshot Strategy
|
||
|
||
### Tiered Snapshot Model
|
||
|
||
```
|
||
Tier 0: Base OS Install (Windows only, no tools)
|
||
└── Snapshot: "OSBase" — rarely updated (yearly or on major Windows updates)
|
||
|
||
Tier 1: Build Toolchain
|
||
└── Snapshot: "BaseClean" ← CLONE SOURCE (weekly refresh)
|
||
Includes: VS Build Tools 2026, .NET SDK 10.0.203, Python 3.13.3, Git, WinRM config
|
||
Requirement: ALL linked clones must reference this exact snapshot
|
||
```
|
||
|
||
### Refresh Schedule
|
||
|
||
| Snapshot | Refresh Frequency | Trigger |
|
||
| ----------- | ----------------- | ------------------------------------------ |
|
||
| `OSBase` | Quarterly | Windows cumulative update |
|
||
| `BaseClean` | Weekly/Monthly | .NET SDK patch, security update, VS update |
|
||
|
||
> **Note:** Windows Server 2025 KMS lease = 180 giorni. Prima della scadenza:
|
||
> boot template su VMnet8 (NAT) → `slmgr /ato` → spegni → nuovo snapshot `BaseClean`.
|
||
|
||
---
|
||
|
||
## 3. Parallel Build Capacity
|
||
|
||
### RAM Budget (i9-10900X · 64 GB)
|
||
|
||
| Component | RAM Usage |
|
||
| ------------------------------------- | --------- |
|
||
| Windows host OS | ~4 GB |
|
||
| VMware Workstation | ~0.5 GB |
|
||
| Gitea server | ~0.5–1 GB |
|
||
| act_runner service | ~100 MB |
|
||
| Each build VM (idle) | ~2–3 GB |
|
||
| Each build VM (active MSBuild/Python) | ~6–8 GB |
|
||
| **Headroom target (20%)** | ~13 GB |
|
||
|
||
```
|
||
Available for VMs: 64 - 4 - 0.5 - 1 - 0.1 - 13 = ~45 GB
|
||
Max VMs at peak: 45 / 8 = ~5.6 → safe limit = 4
|
||
```
|
||
|
||
**Recommendation:** `capacity: 4` in `config.yaml`.
|
||
|
||
### CPU Budget (i9-10900X: 10 cores / 20 threads)
|
||
|
||
- Template VM: 4 vCPU configurati
|
||
- Con 4 VM parallele: 4 × 4 = 16 thread, lascia 4 per host OS / Gitea / runner
|
||
- `build_plugin.py` usa `get_optimal_thread_count()` che rileva i core della VM e divide per il numero di build parallele (fix TRK0002)
|
||
|
||
### VM Configuration (template VMX)
|
||
|
||
```ini
|
||
numvcpus = "4"
|
||
cpuid.coresPerSocket = "2"
|
||
memsize = "6144" # 6 GB RAM per VM
|
||
scsi0.virtualDev = "pvscsi"
|
||
ethernet0.virtualDev = "vmxnet3"
|
||
```
|
||
|
||
---
|
||
|
||
## 4. NuGet / Package Cache on Host
|
||
|
||
**Problem:** Each build VM does a fresh `dotnet restore`, re-downloading NuGet packages every time.
|
||
|
||
**Solution:** Map a host-side NuGet cache directory as a shared folder into each VM.
|
||
|
||
### Setup (host side)
|
||
|
||
```
|
||
F:\CI\Cache\NuGet\ ← shared folder on host
|
||
```
|
||
|
||
### VMware Shared Folder Configuration (in template VMX)
|
||
|
||
Add to `WinBuild.vmx` before taking the `BaseClean` snapshot:
|
||
```ini
|
||
sharedFolder0.present = "TRUE"
|
||
sharedFolder0.enabled = "TRUE"
|
||
sharedFolder0.readAccess = "TRUE"
|
||
sharedFolder0.writeAccess = "TRUE"
|
||
sharedFolder0.hostPath = "F:\\CI\\Cache\\NuGet"
|
||
sharedFolder0.guestName = "nuget-cache"
|
||
sharedFolder.maxNum = "1"
|
||
```
|
||
|
||
Inside the VM, the shared folder appears as `\\vmware-host\Shared Folders\nuget-cache`.
|
||
|
||
### NuGet Cache Redirect (in `Invoke-RemoteBuild.ps1`)
|
||
|
||
Add to the remote build ScriptBlock:
|
||
```powershell
|
||
$env:NUGET_PACKAGES = '\\vmware-host\Shared Folders\nuget-cache'
|
||
dotnet restore --packages $env:NUGET_PACKAGES
|
||
```
|
||
|
||
**Result:** First build per package downloads once; subsequent builds read from the host cache.
|
||
Cache is shared across all concurrent VMs (NuGet packages are safe for concurrent reads).
|
||
|
||
---
|
||
|
||
## 5. Clone Pre-Warming
|
||
|
||
For latency-sensitive pipelines, keep N pre-warmed clones ready in "booted and idle" state.
|
||
|
||
**Tradeoff:** Saves ~45–90 seconds of startup per job, but clones remain running (consuming RAM) until used.
|
||
|
||
**Implementation sketch:**
|
||
|
||
```powershell
|
||
# Pre-warmer job (separate scheduled task on host)
|
||
# Runs between CI jobs to maintain a pool of warm VMs
|
||
$poolSize = 2 # Keep 2 VMs warm at all times
|
||
|
||
# Check current running clones
|
||
$runningClones = Get-ChildItem F:\CI\WarmPool\ -Filter *.vmx
|
||
if ($runningClones.Count -lt $poolSize) {
|
||
# Create and start new clone in warm pool
|
||
$vmx = .\New-BuildVM.ps1 -TemplatePath $template -CloneBaseDir F:\CI\WarmPool -JobId "warm-$(Get-Random)"
|
||
vmrun -T ws start $vmx nogui
|
||
}
|
||
```
|
||
|
||
> **Note:** Pre-warming is an advanced optimization. Start without it and add
|
||
> only if CI overhead (startup time) is a demonstrated bottleneck.
|
||
|
||
---
|
||
|
||
## 6. Artifact Storage Management
|
||
|
||
Build artifacts accumulate quickly. Automate cleanup:
|
||
|
||
```powershell
|
||
# Scheduled task: run daily
|
||
# Remove artifact dirs older than 30 days
|
||
Get-ChildItem 'F:\CI\Artifacts' -Directory |
|
||
Where-Object { $_.LastWriteTime -lt (Get-Date).AddDays(-30) } |
|
||
Remove-Item -Recurse -Force
|
||
|
||
# Remove orphaned clone directories (VMs that were not properly deleted)
|
||
Get-ChildItem 'F:\CI\BuildVMs' -Directory |
|
||
Where-Object { $_.LastWriteTime -lt (Get-Date).AddHours(-4) } |
|
||
ForEach-Object {
|
||
$vmx = Get-ChildItem $_.FullName -Filter *.vmx | Select-Object -First 1
|
||
if ($vmx) {
|
||
vmrun -T ws stop $vmx.FullName hard 2>$null
|
||
}
|
||
Remove-Item $_.FullName -Recurse -Force
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 7. NUMA & CPU Affinity (Advanced)
|
||
|
||
The i9-10900X is a single NUMA node CPU (no NUMA effects). CPU pinning is not needed.
|
||
|
||
However, you can set CPU affinity per VM group to reduce vCPU migration overhead:
|
||
|
||
```powershell
|
||
# Optional: pin act_runner process to cores 0-3 (leave build VMs on 4-19)
|
||
$runnerPid = (Get-Process act_runner).Id
|
||
Start-Process -FilePath 'cmd' -ArgumentList "/c start /affinity 0xF /b /wait powershell" -NoNewWindow
|
||
```
|
||
|
||
For most workloads, Windows scheduler handles this well — skip affinity unless profiling shows issues.
|