chore: initial commit local CI/CD system (e2e verified, production-ready)
Sistema CI locale basato su VMware Workstation + Gitea Actions + act_runner. Testato e2e con nsis-plugin-nsinnounp (MSBuild + Python, 4 configurazioni parallele). - scripts/: Invoke-CIJob, Invoke-RemoteBuild, New/Remove-BuildVM, Wait-VMReady, Get-BuildArtifacts - runner/: act_runner config (windows-build label, capacity 4) - gitea/workflows/: build-nsis.yml (template per progetti MSBuild/Python) - template/: script di provisioning template VM (VS BuildTools 2026, .NET SDK 10, Python 3.13) - docs/: ARCHITECTURE, CI-FLOW, OPTIMIZATION, BEST-PRACTICES, Setup-GiteaSSH Verificato: e2e-009 SUCCESS in 02:09, cleanup automatico VM confermato.
This commit is contained in:
Simone
+254
@@ -0,0 +1,254 @@
|
||||
# CI Pipeline Flow — Step by Step
|
||||
|
||||
## Trigger to Artifact: Complete Pipeline Walk-Through
|
||||
|
||||
---
|
||||
|
||||
### Step 0 — Prerequisites (one-time setup)
|
||||
|
||||
Before any CI job runs, the following must be in place:
|
||||
|
||||
| Component | State |
|
||||
| ------------------ | ----------------------------------------------------------------------- |
|
||||
| Gitea server | Running, accessible on LAN |
|
||||
| act_runner | Registered, running as Windows service on host |
|
||||
| Template VM | Provisioned, snapshot "BaseClean" taken, VM powered off |
|
||||
| WinRM | Enabled on template VM (inherits to all clones) |
|
||||
| vmrun.exe | Present at `C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe` |
|
||||
| `F:\CI\BuildVMs\` | Directory exists and writable |
|
||||
| `F:\CI\Artifacts\` | Directory exists and writable |
|
||||
|
||||
---
|
||||
|
||||
### Step 1 — Developer Pushes Code
|
||||
|
||||
```
|
||||
Developer → git push origin main
|
||||
```
|
||||
|
||||
- Git push lands on Gitea server
|
||||
- Gitea evaluates `.gitea/workflows/build.yml` in the repository
|
||||
- If the workflow trigger matches (e.g. `on: push` to `main`), Gitea enqueues a new **Actions job**
|
||||
- Job is tagged with the label defined in the workflow (`runs-on: windows-build`)
|
||||
|
||||
---
|
||||
|
||||
### Step 2 — act_runner Picks Up the Job
|
||||
|
||||
```
|
||||
act_runner (host) ─polling─► Gitea Actions API
|
||||
◄─job data─
|
||||
```
|
||||
|
||||
- act_runner polls the Gitea API approximately every 5–10 seconds
|
||||
- Detects a pending job matching its labels (`windows-build`)
|
||||
- Downloads the workflow YAML and job parameters
|
||||
- If `capacity` slots are available, starts job execution immediately
|
||||
- If all slots are busy, job waits in Gitea queue until a slot frees
|
||||
|
||||
---
|
||||
|
||||
### Step 3 — act_runner Executes Workflow Steps
|
||||
|
||||
The workflow YAML defines steps that run as PowerShell commands **on the host**.
|
||||
The first substantive step calls `Invoke-CIJob.ps1`:
|
||||
|
||||
```yaml
|
||||
- name: Build in ephemeral VM
|
||||
shell: pwsh
|
||||
run: |
|
||||
.\scripts\Invoke-CIJob.ps1 `
|
||||
-JobId "${{ github.run_id }}" `
|
||||
-RepoUrl "${{ env.GITEA_REPO_URL }}" `
|
||||
-Branch "${{ github.ref_name }}" `
|
||||
-Commit "${{ github.sha }}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 4 — Invoke-CIJob.ps1: Clone VM
|
||||
|
||||
```
|
||||
New-BuildVM.ps1
|
||||
vmrun.exe -T ws clone <template.vmx> <clone.vmx> linked -snapshot "BaseClean"
|
||||
```
|
||||
|
||||
- Generates unique clone name: `Clone_{JobId}_{yyyyMMdd_HHmmss}`
|
||||
- Creates clone directory under `F:\CI\BuildVMs\`
|
||||
- **Linked clone** creation takes ~5–15 seconds
|
||||
- The template snapshot "BaseClean" is **not modified**
|
||||
- Returns the full path to the new `.vmx` file
|
||||
|
||||
---
|
||||
|
||||
### Step 5 — Start VM
|
||||
|
||||
```
|
||||
vmrun.exe -T ws start <clone.vmx> nogui
|
||||
```
|
||||
|
||||
- Starts the cloned VM in headless (no GUI) mode
|
||||
- VM boots from the "BaseClean" state: clean OS, build tools installed, WinRM enabled
|
||||
- Boot takes ~20–40 seconds
|
||||
|
||||
---
|
||||
|
||||
### Step 6 — Wait for VM Readiness
|
||||
|
||||
```
|
||||
Wait-VMReady.ps1 -VMPath <clone.vmx> -IPAddress <dynamic-NAT-ip> -TimeoutSeconds 300
|
||||
```
|
||||
|
||||
IP is discovered dynamically via `vmrun getGuestIPAddress` (VMnet8 NAT, subnet 192.168.79.0/24).
|
||||
No hardcoded IPs — each clone gets a DHCP-assigned address from VMware NAT.
|
||||
|
||||
Three-phase readiness check (retried every 5 seconds, up to 300s timeout):
|
||||
|
||||
```
|
||||
Phase 1: vmrun getState → must return "running"
|
||||
Phase 2: Test-Connection (ICMP ping) → must succeed
|
||||
Phase 3: Test-WSMan → WinRM listener must respond
|
||||
```
|
||||
|
||||
- Total wait is typically **30–90 seconds** after `vmrun start`
|
||||
- If timeout exceeded, VM is destroyed and job fails with a clear error message
|
||||
- On success, IP and credentials are ready for PSSession
|
||||
|
||||
---
|
||||
|
||||
### Step 7 — Remote Build Execution
|
||||
|
||||
```
|
||||
Invoke-RemoteBuild.ps1
|
||||
-IPAddress <dynamic-NAT-ip> # from vmrun getGuestIPAddress
|
||||
-Credential (from Windows Credential Manager)
|
||||
-BuildCommand 'python build_plugin.py --final --dist-dir dist'
|
||||
-GuestArtifactSource 'dist'
|
||||
```
|
||||
|
||||
Sequence inside the PSSession:
|
||||
|
||||
```
|
||||
1. New-PSSession → open WinRM session to VM
|
||||
2. Compress-Archive (host) → copy zip to C:\CI\build.zip (guest) → Expand-Archive to C:\CI\build\
|
||||
(source code è già clonato sull'host in Phase 1 — non si fa git clone nella VM)
|
||||
3. Invoke-Expression "$BuildCommand" in C:\CI\build\
|
||||
Esempio: python build_plugin.py --final --dist-dir dist
|
||||
a. Build 4 configurazioni MSBuild in parallelo (x86-unicode, x64-unicode, cli-x86, cli-x64)
|
||||
b. Raccoglie artifact in C:\CI\build\dist\
|
||||
c. Cleanup build dir (--final)
|
||||
4. Compress-Archive C:\CI\build\$GuestArtifactSource C:\CI\output\artifacts.zip
|
||||
5. Cattura exit code; se non-zero il job fallisce con errore chiaro
|
||||
6. Remove-PSSession
|
||||
```
|
||||
|
||||
- Output standard e stderr dalla build sono streamati all'host e catturati da act_runner per i log Gitea Actions
|
||||
- Se la build fallisce (exit code ≠ 0), il controllo passa al blocco `finally`
|
||||
|
||||
---
|
||||
|
||||
### Step 8 — Collect Artifacts
|
||||
|
||||
```
|
||||
Get-BuildArtifacts.ps1
|
||||
-IPAddress <dynamic-NAT-ip>
|
||||
-Credential (same as above)
|
||||
-GuestArtifactPath C:\CI\output\artifacts.zip
|
||||
-HostArtifactDir F:\CI\Artifacts\{JobId}\
|
||||
```
|
||||
|
||||
- Apre PSSession e copia `C:\CI\output\artifacts.zip` dall'host
|
||||
- Valida che il file esista e sia non-vuoto
|
||||
- Il zip contiene la struttura `dist/` con tutti gli artifact di build
|
||||
|
||||
---
|
||||
|
||||
### Step 9 — Destroy VM
|
||||
|
||||
```
|
||||
Remove-BuildVM.ps1 -VMPath <clone.vmx>
|
||||
```
|
||||
|
||||
Sequence:
|
||||
|
||||
```
|
||||
1. vmrun stop <clone.vmx> soft (graceful shutdown attempt)
|
||||
2. Wait 5 seconds
|
||||
3. vmrun getState → if still "running":
|
||||
vmrun stop <clone.vmx> hard (force off)
|
||||
4. vmrun deleteVM <clone.vmx> (removes VMX + VMDK delta files)
|
||||
5. Remove-Item <clone dir> -Recurse (removes any leftover files)
|
||||
```
|
||||
|
||||
- VM is gone from disk within ~5–10 seconds
|
||||
- No state, no temp files, no credentials remain in the clone
|
||||
- Template VM and "BaseClean" snapshot are unaffected
|
||||
|
||||
> **Note:** This step runs in a `finally` block inside `Invoke-CIJob.ps1`, so it
|
||||
> executes even if Steps 6–8 failed. A build VM is **never** left running.
|
||||
|
||||
---
|
||||
|
||||
### Step 10 — Report Status & Upload Artifacts
|
||||
|
||||
```
|
||||
act_runner ─► Gitea Actions API (job status: success / failure)
|
||||
act_runner ─► Gitea artifact store (uploads F:\CI\Artifacts\{JobId}\*)
|
||||
```
|
||||
|
||||
- act_runner reads the exit code from `Invoke-CIJob.ps1`
|
||||
- Reports `success` or `failure` to Gitea
|
||||
- Uploads artifacts using `actions/upload-artifact` step in the workflow YAML
|
||||
- Build log (stdout from all steps) is stored in Gitea Actions UI
|
||||
- Commit/PR status badge is updated
|
||||
|
||||
---
|
||||
|
||||
## Parallel Build Flow
|
||||
|
||||
When multiple pushes happen concurrently (or multiple PRs are open):
|
||||
|
||||
```
|
||||
act_runner (capacity: 4)
|
||||
├── Job #1 ──► Clone_Job_001 (192.168.79.x) → build → destroy
|
||||
├── Job #2 ──► Clone_Job_002 (192.168.79.x) → build → destroy
|
||||
├── Job #3 ──► Clone_Job_003 (192.168.79.x) → build → destroy
|
||||
└── Job #4 ──► Clone_Job_004 (192.168.79.x) → build → destroy
|
||||
(Job #5 waits in Gitea queue)
|
||||
```
|
||||
|
||||
Note: each clone gets a dynamic IP from VMware NAT DHCP (192.168.79.0/24). IP is discovered
|
||||
automatically via `vmrun getGuestIPAddress` — no static assignment required.
|
||||
|
||||
- Each job creates its own independently named clone
|
||||
- Clones share the parent "BaseClean" snapshot (read-only CoW)
|
||||
- No filesystem or process overlap between concurrent builds
|
||||
- act_runner enforces the `capacity` limit
|
||||
|
||||
---
|
||||
|
||||
## Failure Scenarios
|
||||
|
||||
| Failure Point | Behavior |
|
||||
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
|
||||
| VM clone fails | `Invoke-CIJob.ps1` throws; no VM to destroy; job fails immediately |
|
||||
| VM never reaches ready state (timeout) | `Wait-VMReady.ps1` throws; `finally` destroys the (running) VM |
|
||||
| Build fails inside VM | `Invoke-RemoteBuild.ps1` throws; `finally` collects partial artifacts (if any), destroys VM |
|
||||
| Artifact collection fails | Job marked failed; VM is still destroyed in `finally` |
|
||||
| Host machine loses power mid-job | `act_runner` resumes on reboot, marks in-flight job as failed; orphan VMs must be cleaned up manually (see TODO.md) |
|
||||
|
||||
---
|
||||
|
||||
## Timing Summary (typical build)
|
||||
|
||||
| Phase | Typical Duration |
|
||||
| ---------------------------- | -------------------- |
|
||||
| Gitea → act_runner poll | 5–10 s |
|
||||
| Linked clone creation | 5–15 s |
|
||||
| VM boot | 20–40 s |
|
||||
| WinRM readiness | 10–30 s (after boot) |
|
||||
| git clone (from local Gitea) | 2–10 s |
|
||||
| dotnet restore + build | project-dependent |
|
||||
| Artifact collection | 2–10 s |
|
||||
| VM destruction | 5–10 s |
|
||||
| **Overhead total** | **~60–120 s** |
|
||||
Reference in New Issue
Block a user