chore: initial commit local CI/CD system (e2e verified, production-ready)
Sistema CI locale basato su VMware Workstation + Gitea Actions + act_runner. Testato e2e con nsis-plugin-nsinnounp (MSBuild + Python, 4 configurazioni parallele). - scripts/: Invoke-CIJob, Invoke-RemoteBuild, New/Remove-BuildVM, Wait-VMReady, Get-BuildArtifacts - runner/: act_runner config (windows-build label, capacity 4) - gitea/workflows/: build-nsis.yml (template per progetti MSBuild/Python) - template/: script di provisioning template VM (VS BuildTools 2026, .NET SDK 10, Python 3.13) - docs/: ARCHITECTURE, CI-FLOW, OPTIMIZATION, BEST-PRACTICES, Setup-GiteaSSH Verificato: e2e-009 SUCCESS in 02:09, cleanup automatico VM confermato.
This commit is contained in:
@@ -0,0 +1,281 @@
|
||||
# Best Practices — Stability, Security & Operations
|
||||
|
||||
## 1. Credential Management
|
||||
|
||||
### Do NOT store credentials in scripts or config files
|
||||
|
||||
The scripts use `-GuestCredentialTarget` (a Windows Credential Manager target name)
|
||||
rather than plaintext username/password parameters. Store credentials once:
|
||||
|
||||
```powershell
|
||||
# Run on host (once, before first CI job)
|
||||
cmdkey /generic:BuildVMGuest /user:MACHINENAME\ci_build /pass:YourStrongPassword
|
||||
```
|
||||
|
||||
Retrieve in scripts via the `CredentialManager` PowerShell module:
|
||||
|
||||
```powershell
|
||||
Install-Module CredentialManager -Scope CurrentUser
|
||||
$cred = Get-StoredCredential -Target 'BuildVMGuest'
|
||||
```
|
||||
|
||||
### Rotate credentials quarterly
|
||||
|
||||
1. Update password in the template VM (requires rebuilding `BaseClean` snapshot)
|
||||
2. Update Windows Credential Manager on the host:
|
||||
```
|
||||
cmdkey /generic:BuildVMGuest /user:MACHINENAME\ci_build /pass:NewPassword
|
||||
```
|
||||
3. No script changes required — they reference the target name, not the password.
|
||||
|
||||
---
|
||||
|
||||
## 2. WinRM Security
|
||||
|
||||
### Current setup (HTTP / port 5985)
|
||||
|
||||
Acceptable for an **isolated lab network** where:
|
||||
- Build VMs are on a dedicated host-only VMware network (192.168.11.0/24 — VMnet11 dedicated CI)
|
||||
- No external traffic reaches port 5985
|
||||
- Credentials are managed via Windows Credential Manager
|
||||
|
||||
### Recommended upgrade: WinRM over HTTPS (port 5986)
|
||||
|
||||
```powershell
|
||||
# Inside the template VM (before taking BaseClean snapshot):
|
||||
|
||||
# Create self-signed certificate
|
||||
$cert = New-SelfSignedCertificate `
|
||||
-Subject "CN=$(hostname)" `
|
||||
-CertStoreLocation Cert:\LocalMachine\My `
|
||||
-KeyUsage DigitalSignature, KeyEncipherment `
|
||||
-KeyAlgorithm RSA `
|
||||
-KeyLength 2048
|
||||
|
||||
# Create HTTPS WinRM listener
|
||||
New-WSManInstance WinRM/Config/Listener `
|
||||
-SelectorSet @{ Transport = 'HTTPS'; Address = '*' } `
|
||||
-ValueSet @{ Hostname = $(hostname); CertificateThumbprint = $cert.Thumbprint }
|
||||
|
||||
# Allow port 5986
|
||||
New-NetFirewallRule -Name 'WinRM-HTTPS-CI' -DisplayName 'WinRM HTTPS CI' `
|
||||
-Direction Inbound -Protocol TCP -LocalPort 5986 -Action Allow
|
||||
```
|
||||
|
||||
```powershell
|
||||
# On the HOST (in scripts), use HTTPS session options:
|
||||
$sessionOptions = New-PSSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck
|
||||
$session = New-PSSession -ComputerName $ip -Port 5986 -UseSSL `
|
||||
-Credential $cred -SessionOption $sessionOptions
|
||||
```
|
||||
|
||||
> `-SkipCACheck` is acceptable for a self-signed cert in an isolated lab. Do NOT
|
||||
> use this against externally accessible machines.
|
||||
|
||||
---
|
||||
|
||||
## 3. act_runner Service Stability
|
||||
|
||||
### Windows Service Recovery Policy
|
||||
|
||||
The `Install-Runner.ps1` script configures automatic service restart on failure:
|
||||
- Restart after 1st failure: 5 seconds
|
||||
- Restart after 2nd failure: 10 seconds
|
||||
- Restart after subsequent: 30 seconds
|
||||
|
||||
Verify in Services → act_runner → Properties → Recovery tab.
|
||||
|
||||
### Monitor the service
|
||||
|
||||
```powershell
|
||||
# Check service status
|
||||
Get-Service act_runner | Select-Object Status, StartType
|
||||
|
||||
# View last 50 log lines
|
||||
Get-EventLog -LogName Application -Source act_runner -Newest 50 | Format-List
|
||||
|
||||
# Restart if needed
|
||||
Restart-Service act_runner
|
||||
```
|
||||
|
||||
### Scheduled health check (optional)
|
||||
|
||||
Create a scheduled task that verifies the runner appears "Online" in Gitea via API:
|
||||
|
||||
```powershell
|
||||
# Check runner status via Gitea API every 15 minutes
|
||||
$response = Invoke-RestMethod `
|
||||
-Uri "http://gitea.local/api/v1/admin/runners" `
|
||||
-Headers @{ Authorization = "token $env:GITEA_API_TOKEN" }
|
||||
|
||||
$runnerOnline = $response | Where-Object { $_.name -eq 'local-windows-runner' -and $_.status -eq 'online' }
|
||||
if (-not $runnerOnline) {
|
||||
# Send alert (email, webhook, etc.) or restart service
|
||||
Restart-Service act_runner
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Template VM Integrity
|
||||
|
||||
The "BaseClean" snapshot is the foundation of every build. If it is corrupted,
|
||||
**all builds fail immediately**.
|
||||
|
||||
### Protection measures
|
||||
|
||||
1. **Never power on the template VM for reasons other than planned maintenance.**
|
||||
Configure VMware Workstation to prevent accidental starts: right-click → Settings →
|
||||
Options → Advanced → disable "Allow background snapshots".
|
||||
|
||||
2. **Backup the parent VMDK before any template changes:**
|
||||
```powershell
|
||||
# Before any template maintenance
|
||||
$templateDir = 'F:\CI\Templates\WinBuild'
|
||||
$backupDir = "F:\CI\Backups\Template_$(Get-Date -Format yyyyMMdd)"
|
||||
Copy-Item $templateDir $backupDir -Recurse
|
||||
```
|
||||
|
||||
3. **Keep a list of all current linked clones** before refreshing the snapshot.
|
||||
If any clone exists when you modify the parent, it may break.
|
||||
Check: `vmrun list` — should return no build VMs during maintenance window.
|
||||
|
||||
4. **Version the snapshot name** to make rollback easy:
|
||||
Instead of reusing "BaseClean", name snapshots `BaseClean_20260101`.
|
||||
Update `config.yaml` `envs.GITEA_CI_SNAPSHOT_NAME` when rotating.
|
||||
|
||||
---
|
||||
|
||||
## 5. Orphaned VM Cleanup
|
||||
|
||||
If the host loses power mid-job or act_runner crashes, ephemeral VMs may not be
|
||||
destroyed. Run this cleanup script on host startup or as a daily scheduled task:
|
||||
|
||||
```powershell
|
||||
# Cleanup-OrphanedBuildVMs.ps1
|
||||
$vmrun = 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe'
|
||||
$cloneBase = 'F:\CI\BuildVMs'
|
||||
$maxAgeHours = 4 # No job should run longer than 4 hours
|
||||
|
||||
Get-ChildItem $cloneBase -Directory |
|
||||
Where-Object { $_.LastWriteTime -lt (Get-Date).AddHours(-$maxAgeHours) } |
|
||||
ForEach-Object {
|
||||
$vmx = Get-ChildItem $_.FullName -Filter '*.vmx' | Select-Object -First 1
|
||||
if ($vmx) {
|
||||
Write-Host "Cleaning orphan: $($vmx.FullName)"
|
||||
& $vmrun -T ws stop $vmx.FullName hard 2>$null
|
||||
& $vmrun -T ws deleteVM $vmx.FullName 2>$null
|
||||
}
|
||||
Remove-Item $_.FullName -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Gitea Repository Configuration
|
||||
|
||||
### Required repository settings for workflows to run
|
||||
|
||||
1. Enable Actions for the repository: Settings → Repository → Actions → Enable
|
||||
2. Add secrets if needed: Settings → Secrets and Variables → Actions
|
||||
3. Protect main branch: Settings → Branches → Branch protection rules
|
||||
|
||||
### Workflow file location
|
||||
|
||||
Workflows **must** be at `.gitea/workflows/*.yml` (not `.github/workflows/`).
|
||||
|
||||
```
|
||||
your-repo/
|
||||
└── .gitea/
|
||||
└── workflows/
|
||||
└── build.yml ← copy from gitea/workflow-example.yml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Logging & Observability
|
||||
|
||||
### act_runner logs
|
||||
|
||||
The runner daemon writes to stdout (captured by the Windows service manager).
|
||||
Increase verbosity for debugging:
|
||||
|
||||
```yaml
|
||||
# runner/config.yaml
|
||||
log:
|
||||
level: debug # change from "info" to "debug"
|
||||
format: text
|
||||
```
|
||||
|
||||
### Per-job build logs
|
||||
|
||||
`Invoke-CIJob.ps1` outputs timestamped phase banners. act_runner captures all
|
||||
stdout/stderr and uploads it to Gitea Actions → job log viewer.
|
||||
|
||||
For persistent local logs:
|
||||
|
||||
```powershell
|
||||
# In your workflow YAML, redirect output to a log file:
|
||||
- name: Build in ephemeral VM
|
||||
shell: pwsh
|
||||
run: |
|
||||
.\scripts\Invoke-CIJob.ps1 ... *>&1 | Tee-Object -FilePath "F:\CI\Logs\job-${{ github.run_id }}.log"
|
||||
```
|
||||
|
||||
### Windows Event Log
|
||||
|
||||
act_runner (when installed as a service) writes events to Windows Event Log →
|
||||
Application source "act_runner". Check with:
|
||||
|
||||
```powershell
|
||||
Get-EventLog -LogName Application -Source '*runner*' -Newest 20
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Network Isolation Verification
|
||||
|
||||
After setting up the host-only VMware network, verify that build VMs cannot
|
||||
reach the internet (important for supply-chain security):
|
||||
|
||||
```powershell
|
||||
# From inside a build VM via WinRM:
|
||||
Invoke-Command -Session $session -ScriptBlock {
|
||||
$result = Test-Connection 8.8.8.8 -Count 1 -Quiet
|
||||
if ($result) {
|
||||
Write-Warning "VM has internet access — check VMware network adapter type"
|
||||
} else {
|
||||
Write-Host "VM correctly isolated (no internet)"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Build VMs should only reach:
|
||||
- The host (for WinRM connection)
|
||||
- Gitea server (for git clone, if reachable via host-only network)
|
||||
- NuGet cache share (host-side shared folder)
|
||||
|
||||
---
|
||||
|
||||
## 9. Updating the Build Toolchain
|
||||
|
||||
When a new .NET SDK or VS Build Tools version is released:
|
||||
|
||||
1. **During a maintenance window** (no CI jobs running):
|
||||
```
|
||||
vmrun list ← must be empty
|
||||
```
|
||||
2. Boot the template VM
|
||||
3. Run updates:
|
||||
```powershell
|
||||
# Update .NET SDK
|
||||
& "C:\Users\ci_build\AppData\Local\Microsoft\dotnet\dotnet-install.ps1" -Channel 8.0
|
||||
|
||||
# Update VS Build Tools via Visual Studio Installer
|
||||
"C:\Program Files (x86)\Microsoft Visual Studio\Installer\vs_installer.exe" update --quiet --norestart
|
||||
```
|
||||
4. Verify tools work (run a test build manually)
|
||||
5. Shut down VM
|
||||
6. Take new snapshot: `BaseClean_$(Get-Date -Format yyyyMMdd)`
|
||||
7. Update `SnapshotName` in `runner/config.yaml`
|
||||
8. Delete the old snapshot after confirming new one works for 1 week
|
||||
Reference in New Issue
Block a user