From 7d12dedddda9e0c4ebf83beb6d8d0a4a760cfd9d Mon Sep 17 00:00:00 2001 From: Simone Date: Mon, 11 May 2026 22:35:36 +0200 Subject: [PATCH] =?UTF-8?q?Sprint=2013-14:=20=C2=A76.2=20composite=20actio?= =?UTF-8?q?n=20+=20=C2=A76.4=20build=20matrix=20+=20=C2=A76.5=20secret=20i?= =?UTF-8?q?njection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §6.2 - New composite action gitea/actions/local-ci-build/action.yml Wraps Invoke-CIJob.ps1 for reuse across repos. Inputs: build-command, artifact-source, submodules, guest-os, use-git-clone, configuration, template-path, snapshot-name, artifact-name, artifact-retention-days. Anti-injection: all params via INPUT_* env vars, never interpolated in shell. §6.4 - Build matrix windows+linux in build-nsis.yml gitea/workflows/build-nsis.yml rewritten with strategy.matrix target: [windows, linux], fail-fast: false, routes to {target}-build runner. Added action inputs: job-id-suffix (matrix disambig for artifact dirs), repo-url (SSH URL override for gitea-ci alias). §6.5 - ExtraGuestEnv secret injection chain New param -ExtraGuestEnv [hashtable] in Invoke-CIJob.ps1 and Invoke-RemoteBuild.ps1. Windows: SetEnvironmentVariable in WinRM session ScriptBlock before build command. Linux: export KEY='val'; prefix before cd && buildcmd (POSIX single-quote escaping). Composite action input extra-guest-env-json (JSON object -> ConvertFrom-Json -> hashtable), never logged. PSScriptAnalyzer fixes: _Common.psm1, Invoke-RetentionPolicy.ps1, Watch-RunnerHealth.ps1 (empty catch, ShouldProcess suppression, etc.) Docs/test: TEST-PLAN-v1.3-to-HEAD.md updated §3.3/§6.1/§5.4 status. TODO.md: §6 fully closed (6.1-6.2 done, 6.3 deferred, 6.4-6.5 done, 6.6 done). --- TODO.md | 131 ++++++++----- docs/TEST-PLAN-v1.3-to-HEAD.md | 6 +- gitea/actions/local-ci-build/action.yml | 245 ++++++++++++++++++++++++ gitea/workflow-example.yml | 49 +++++ gitea/workflows/build-nsis.yml | 48 ++--- scripts/Invoke-CIJob.ps1 | 16 +- scripts/Invoke-RemoteBuild.ps1 | 24 ++- scripts/Invoke-RetentionPolicy.ps1 | 1 + scripts/Watch-RunnerHealth.ps1 | 2 +- scripts/_Common.psm1 | 2 + 10 files changed, 431 insertions(+), 93 deletions(-) create mode 100644 gitea/actions/local-ci-build/action.yml diff --git a/TODO.md b/TODO.md index f64816a..433bf49 100644 --- a/TODO.md +++ b/TODO.md @@ -1,6 +1,9 @@ # TODO — Local CI/CD System - + + + + > Documento unico di lavoro: roadmap, audit trail dei task completati, e backlog post-v1.0 > con priorità e razionale. Le voci aperte sono raggruppate per area e marcate **P0..P3**: @@ -18,32 +21,32 @@ --- ## Summary -_Last updated: 2026-05-11 (post Sprint 11: §6.6 Tier-2 Toolchain DONE — PowerShell 7.6.1, NSIS 3.12, CMake 3.31.6, Node.js 22.14.0, WiX 4.0.5, gh CLI 2.92.0, Sysinternals, vcpkg — Step 12 in Setup-WinBuild2025.ps1)_ +_Last updated: 2026-05-11 (post Sprint 14: §6.4+§6.5 DONE — build-nsis.yml matrix windows+linux, ExtraGuestEnv chain in Invoke-CIJob/Invoke-RemoteBuild, job-id-suffix/repo-url/extra-guest-env-json inputs in action.yml; post Sprint 13: §6.2 DONE)_ -**Stato generale**: §1, §2, §4, §5 chiusi. §3 completo. §6.1 Linux Build VM completato con e2e PASS (nsis7z.dll 2272 KB, parità Windows). §6.6 Tier-1+Tier-2 completati nel template (Step 11+12). Prossimi P2: Test Plan + §6.2-6.5. +**Stato generale**: §1, §2, §4, §5 chiusi. §3 completo. §6 quasi chiuso: 6.1-6.2 DONE, 6.3 deferred, 6.4+6.5 DONE, 6.6 DONE. Prossimi: Test Plan validation con VM reali + §7.5 (optional). -| Stato | Area | Done | Open | Note | -| ----- | ------------------------------- | ---: | ---: | ------------------------------------------------------------------- | -| [x] | Setup & Infrastructure | 8 | 0 | Gitea + runner + dirs + rete tutti operativi | -| [x] | Template VM (Fasi A→D) | 4 | 0 | Snapshot `BaseClean` preso, credenziali in Credential Manager | -| [x] | Scripts Verification | 3 | 0 | e2e-008/009 SUCCESS, cleanup orfani + retry deleteVM ok | -| [x] | Runner Configuration | 2 | 0 | `config.yaml`, capacity 4, `BuildVMGuest` | -| [ ] | Gitea Workflow Integration | 4 | 1 | manca solo §P3 generalizzazione workflow | -| [x] | §1 Sicurezza & hardening | 3 | 0 | 1.1, 1.4, 1.6 done; 1.2/1.3/1.5/1.7/1.8 deferred (home lab) | -| [x] | §2 Affidabilità & resilienza | 7 | 0 | tutto done (2.1-2.7) | -| [x] | §3 Performance & throughput | 4 | 0 | 3.1/3.2/3.3/3.6 done; 3.4/3.5 deferred — **§3 CHIUSO** | -| [x] | §4 Osservabilità & manutenzione | 3 | 0 | 4.1/4.3/4.4 done (4.2 Prometheus, 4.5 Dashboard rimossi) | -| [x] | §5 Qualità codice & test | 5 | 0 | 5.1/5.2/5.3/5.4/5.5 done | -| [ ] | §6 Scalabilità & estensioni | 2 | 4 | 6.1 (Linux) done, 6.6 Tier-1+Tier-2 done (Step 11+12); 6.2-6.5 TODO | -| [ ] | §7 Refactor Deploy ↔ Setup | 4 | 1 | 7.1/7.2/7.3/7.4 done; 7.5 (rinomina) opzionale | -| [x] | In-VM Git Clone (§3.3) | 4 | 0 | COMPLETATO 2026-05-10 — e2e PASS: 34s/25.7% vs baseline | -| [x] | Linux Build VM (§6.1) | 5 | 0 | Deploy/Prepare/Setup/Transport + doc + e2e PASS nsis7z.dll 2272 KB | +| Stato | Area | Done | Open | Note | +| ----- | ------------------------------- | ---: | ---: | ------------------------------------------------------------------ | +| [x] | Setup & Infrastructure | 8 | 0 | Gitea + runner + dirs + rete tutti operativi | +| [x] | Template VM (Fasi A→D) | 4 | 0 | Snapshot `BaseClean` preso, credenziali in Credential Manager | +| [x] | Scripts Verification | 3 | 0 | e2e-008/009 SUCCESS, cleanup orfani + retry deleteVM ok | +| [x] | Runner Configuration | 2 | 0 | `config.yaml`, capacity 4, `BuildVMGuest` | +| [ ] | Gitea Workflow Integration | 4 | 1 | manca solo §P3 generalizzazione workflow | +| [x] | §1 Sicurezza & hardening | 3 | 0 | 1.1, 1.4, 1.6 done; 1.2/1.3/1.5/1.7/1.8 deferred (home lab) | +| [x] | §2 Affidabilità & resilienza | 7 | 0 | tutto done (2.1-2.7) | +| [x] | §3 Performance & throughput | 4 | 0 | 3.1/3.2/3.3/3.6 done; 3.4/3.5 deferred — **§3 CHIUSO** | +| [x] | §4 Osservabilità & manutenzione | 3 | 0 | 4.1/4.3/4.4 done (4.2 Prometheus, 4.5 Dashboard rimossi) | +| [x] | §5 Qualità codice & test | 5 | 0 | 5.1/5.2/5.3/5.4/5.5 done | +| [x] | §6 Scalabilità & estensioni | 5 | 0 | 6.1-6.2 done, 6.3 deferred (home lab), 6.4-6.5 done, 6.6 done | +| [ ] | §7 Refactor Deploy ↔ Setup | 4 | 1 | 7.1/7.2/7.3/7.4 done; 7.5 (rinomina) opzionale | +| [x] | In-VM Git Clone (§3.3) | 4 | 0 | COMPLETATO 2026-05-10 — e2e PASS: 34s/25.7% vs baseline | +| [x] | Linux Build VM (§6.1) | 5 | 0 | Deploy/Prepare/Setup/Transport + doc + e2e PASS nsis7z.dll 2272 KB | **Prossimi passi consigliati** (vedi anche "Suggerimento di sequencing" in fondo): -1. **Test Plan** — eseguire completo `docs/TEST-PLAN-v1.3-to-HEAD.md` (25+ Pester tests, validazione features) -2. **Security** — Rifiori se condizioni cambiano; vedi sezione "Deferred (Home Lab)" -3. **§3.5** — vCPU/RAM tuning per workload (P2, deferred — home lab optimization) -4. **§3.4** — Pre-warm pool di cloni (P3, deferred — profiling-dependent optimization) +1. **Test Plan** — eseguire `Invoke-Pester tests\ -Output Detailed` + validare §2.1/§3.1/§3.6 con VM reali +2. **Test Plan** — validazione con VM reali (§2.1 IP race, §3.1 shared cache, §3.6 benchmark, §4.1 JSONL) — prossima priorità +3. **Security** — Rifiori se condizioni cambiano; vedi sezione "Deferred (Home Lab)" +4. **§3.5** — vCPU/RAM tuning per workload (P2, deferred — home lab optimization) --- ## Setup & Infrastructure @@ -463,21 +466,31 @@ Sostituire WinRM con SSH key-based; `Invoke-Command` non funziona — usare `ssh Aggiungere `scripts/_Transport.psm1` con due implementazioni (`WinRM`, `SSH`) dietro la stessa interfaccia. `Invoke-CIJob.ps1` sceglie dal parametro `-GuestOS`. -### 6.2 [P3] [ ] Workflow generico riutilizzabile (composite action) -File: [gitea/workflow-example.yml](gitea/workflow-example.yml). +### 6.2 [P3] [x] Workflow generico riutilizzabile (composite action) +File: [gitea/actions/local-ci-build/action.yml](gitea/actions/local-ci-build/action.yml). -Trasformare in *composite action* Gitea (`.gitea/actions/local-ci-build/action.yml`) -richiamabile da qualsiasi repo: +**Status**: COMPLETATO 2026-05-11 — Composite action implementata. + +Inputs: `build-command` (required), `artifact-source`, `submodules`, `guest-os`, `use-git-clone`, +`configuration`, `template-path`, `snapshot-name`, `artifact-name`, `artifact-retention-days`. +Outputs: `artifact-path`, `artifact-name`. +Inputs passati via variabili d'ambiente (no injection). Upload artifact incluso nell'action. + +Usage da qualsiasi repo: ```yaml -- uses: ./Simone/local-ci-build@v1 - with: - build-command: 'python build_plugin.py --final' - artifact-source: 'dist' - submodules: true +jobs: + build: + runs-on: windows-build # or linux-build + steps: + - uses: actions/checkout@v4 + - uses: Simone/local-ci-system/.gitea/actions/local-ci-build@main + with: + build-command: 'python build_plugin.py --final --dist-dir dist' + artifact-source: 'dist' + submodules: 'true' ``` -Riduce la duplicazione su nuovi repo e centralizza l'evoluzione del wrapper. -### 6.3 [P3] [ ] Multi-host runner federation +### 6.3 [DEFERRED — home lab, bassissima priorità] Multi-host runner federation Quando 4 VM in parallelo non bastano, l'opzione naturale è un secondo host Windows con stesso `Setup-Host.ps1` registrato come secondo runner Gitea (label `windows-build:host-2`). Già supportato lato Gitea, basta replicare il setup. @@ -485,20 +498,34 @@ Già supportato lato Gitea, basta replicare il setup. **Premessa**: prima di scalare orizzontale, profilare se il collo di bottiglia è CPU (i9-10900X 20T è abbondante) o I/O (NVMe). Se è I/O, una seconda NVMe sullo stesso host basta. -### 6.4 [P3] [ ] Build matrix nel workflow +**Deferred**: in un home lab single-host questa funzionalità non serve. Non pianificare. + +### 6.4 [P3] [x] Build matrix nel workflow File: [gitea/workflows/build-nsis.yml](gitea/workflows/build-nsis.yml). -Attualmente single job. Quando arriverà la VM Linux: -```yaml -strategy: - matrix: - target: [windows, linux] -runs-on: ${{ matrix.target }}-build -``` +**Status**: COMPLETATO 2026-05-11 — `build-nsis.yml` riscritto con matrix strategy. -### 6.5 [P3] [ ] Secret injection workflow-level -Per chiavi di firma (Authenticode, GPG), usare i Gitea secrets nel workflow e passarli a -`Invoke-CIJob.ps1` come param + env, mai loggati. Stesso modello di §1.5 per il PAT. +Matrix `target: [windows, linux]`, `runs-on: ${{ matrix.target }}-build`, +`fail-fast: false`. Usa la composite action con `job-id-suffix: ${{ matrix.target }}` +per artifact dir separate e `repo-url: ssh://gitea-ci/Simone/nsis-plugin-nsinnounp.git`. +Artifact separati: `nsis-plugin-nsinnounp-windows-` e `nsis-plugin-nsinnounp-linux-`. + +### 6.5 [P3] [x] Secret injection workflow-level + +**Status**: COMPLETATO 2026-05-11 — `ExtraGuestEnv [hashtable]` aggiunto a: +- `Invoke-CIJob.ps1` (param `-ExtraGuestEnv`, forwarded a Invoke-RemoteBuild) +- `Invoke-RemoteBuild.ps1` (param `-ExtraGuestEnv`, iniettato nei guest): + - **Windows**: `[System.Environment]::SetEnvironmentVariable` nel session ScriptBlock prima del build command + - **Linux**: `export KEY='val'; ` prefisso prima della `cd && buildcmd` +- `action.yml`: input `extra-guest-env-json` (JSON object), parsato via `ConvertFrom-Json`, mai loggato + +Usage dal workflow: +```yaml +- uses: Simone/local-ci-system/.gitea/actions/local-ci-build@main + with: + build-command: 'python sign_and_build.py' + extra-guest-env-json: '{"SIGN_PASS":"${{ secrets.SIGN_PASS }}","CERT_THUMBPRINT":"${{ secrets.CERT_THUMBPRINT }}"}' +``` ### 6.6 [P2] [x] Toolchain Tier-1 + Tier-2 in Setup-WinBuild2025 File: [template/Setup-WinBuild2025.ps1](template/Setup-WinBuild2025.ps1), @@ -522,13 +549,13 @@ build più ampi senza toccare il template. ### Tier-2 — Step 12 (implementato) -| Tool | Versione | Razionale | -| ---------------- | ---------------------- | -------------------------------------------------- | -| **CMake** | 3.31.6 | Build system C/C++ cross-platform | -| **Node.js LTS** | 22.14.0 | Frontend/Electron pipelines, npm toolchain | -| **WiX Toolset** | 4.0.5 (dotnet tool) | MSI building da .NET projects | -| **Sysinternals** | latest (CDN Microsoft) | Diagnostica process/handle quando build flaky | -| **vcpkg** | HEAD (shallow clone) | Package manager C++ per dipendenze native | +| Tool | Versione | Razionale | +| ---------------- | ---------------------- | --------------------------------------------- | +| **CMake** | 3.31.6 | Build system C/C++ cross-platform | +| **Node.js LTS** | 22.14.0 | Frontend/Electron pipelines, npm toolchain | +| **WiX Toolset** | 4.0.5 (dotnet tool) | MSI building da .NET projects | +| **Sysinternals** | latest (CDN Microsoft) | Diagnostica process/handle quando build flaky | +| **vcpkg** | HEAD (shallow clone) | Package manager C++ per dipendenze native | **Approccio**: - Pinning hash SHA256 per ogni installer (vedi §1.3) diff --git a/docs/TEST-PLAN-v1.3-to-HEAD.md b/docs/TEST-PLAN-v1.3-to-HEAD.md index cb3b172..bdcc546 100644 --- a/docs/TEST-PLAN-v1.3-to-HEAD.md +++ b/docs/TEST-PLAN-v1.3-to-HEAD.md @@ -12,9 +12,11 @@ Organized by sprint area with pre-requisites, test steps, and pass/fail criteria | **Sprint 2** (Reliability) | 2.1/2.2/2.3/2.5 | Partial | §2.2/§2.3 validated locally 2026-05-10; §2.1/§2.5 need real VMs | | **Sprint 3** (Observability) | 4.1/4.3/4.4 | Partial | §4.3/§4.4 validated locally 2026-05-10; §4.1 (JSONL) needs real build job | | **Sprint 4-6** (Quality/Perf) | 1.4/3.1/3.6/5.2/5.3/5.4 | Partial | §1.4/§5.2/§5.3/§5.4 validated 2026-05-10; §3.1/§3.6 need real VMs | -| **Sprint 7** (Testing) | 5.1 | [x] Done | 22/22 Pester tests passing in 12.9s (2026-05-10) | +| **Sprint 7** (Testing) | 5.1 | [x] Done | 22/22 Pester tests passing in 12.9s (re-validated 2026-05-11) | | **§1.6** (Docs) | Threat model | [x] Done | BEST-PRACTICES.md verified 2026-05-10 | | **§3.2** (Perf) | 7-Zip compression | Partial | Fallback path validated; in-VM 7-Zip speed test needs real VM | +| **§5.4** (PSScriptAnalyzer) | 0 errors, 298 warnings | [x] Done | All 6 actionable warnings fixed 2026-05-11; 0 Error severity violations | +| **§6.1** (Linux Build VM) | Deploy/Prepare/e2e | [x] Done | Test-Ns7zipBuild-Linux.ps1 4xPASS 2026-05-11 — nsis7z.dll 2272 KB | --- @@ -754,7 +756,7 @@ Test-Path 'C:\Temp\test-extract\test-7z.txt' - [x] 7-Zip command available post-setup (C:\Program Files\7-Zip\7z.exe) - [x] Final pre-snapshot gate includes git + 7z validation - [x] Both tools survive template snapshot + linked clone cycle -- [ ] In-VM git clone tested end-to-end with -UseGitClone flag (§3.3 implementation pending) +- [x] In-VM git clone tested end-to-end with -UseGitClone flag (§3.3 e2e-010 PASS 2026-05-10 — 34s / 25.7% more speed vs baseline) --- diff --git a/gitea/actions/local-ci-build/action.yml b/gitea/actions/local-ci-build/action.yml new file mode 100644 index 0000000..7b65c88 --- /dev/null +++ b/gitea/actions/local-ci-build/action.yml @@ -0,0 +1,245 @@ +# .gitea/actions/local-ci-build/action.yml +# +# Reusable composite action — Local CI/CD System +# +# Wraps Invoke-CIJob.ps1 (ephemeral VMware VM pipeline) into a single callable +# step, usable from any Gitea repository without duplicating the orchestration +# logic. +# +# Stored in: Simone/local-ci-system / .gitea/actions/local-ci-build/action.yml +# +# Usage from a calling repo's workflow: +# +# jobs: +# build: +# runs-on: windows-build # or linux-build +# steps: +# - uses: actions/checkout@v4 +# - uses: Simone/local-ci-system/.gitea/actions/local-ci-build@main +# with: +# build-command: 'python build_plugin.py --final --dist-dir dist' +# artifact-source: 'dist' +# submodules: 'true' +# +# Runner label determines the guest OS template: +# windows-build -> WinBuild2025 (WinRM/HTTPS transport) +# linux-build -> LinuxBuild2404 (SSH transport) + +name: Local CI Build +description: > + Builds a project using an ephemeral VMware VM managed by Invoke-CIJob.ps1. + Handles the full VM lifecycle: clone template -> start VM -> run build -> + collect artifacts -> destroy VM. + Compatible with windows-build (WinRM) and linux-build (SSH) runners. + +inputs: + build-command: + description: > + Shell command to execute inside the build VM. + Example: 'python build_plugin.py --final --dist-dir dist' + required: true + + artifact-source: + description: > + Path inside VM where build outputs are located, relative to the CI work + directory. Collected to F:\CI\Artifacts\{JobId}\ on the host. + required: false + default: 'dist' + + submodules: + description: > + Set to "true" to clone the repository with --recurse-submodules. + required: false + default: 'false' + + guest-os: + description: > + Guest OS type: Windows, Linux, or Auto. + Auto (default) detects the OS from the VMX guestOS field after cloning. + required: false + default: 'Auto' + + use-git-clone: + description: > + Set to "true" to clone the repository inside the VM rather than + transferring a zip from the host. Requires Git in the template (see + Setup-WinBuild2025.ps1 Step 12 / Setup-LinuxBuild2404.sh). + Preferred for repositories larger than ~200 MB. + required: false + default: 'false' + + configuration: + description: > + Build configuration string forwarded to the build command (Release/Debug). + required: false + default: 'Release' + + template-path: + description: > + Override the VMX template path. Leave empty (default) to use the value + from the GITEA_CI_TEMPLATE_PATH runner environment variable set in + runner/config.yaml. + required: false + default: '' + + snapshot-name: + description: > + Override the snapshot name to clone from. Leave empty (default) to use + GITEA_CI_SNAPSHOT_NAME runner env var, or "BaseClean" if that is also + unset. + required: false + default: '' + + artifact-name: + description: > + Name for the artifact uploaded to Gitea. Leave empty to use the default: + build-{run_id}-{sha}. + required: false + default: '' + + artifact-retention-days: + description: 'Days to retain uploaded artifacts in Gitea.' + required: false + default: '14' + + job-id-suffix: + description: > + Optional suffix appended to the job ID: run_id-run_attempt-SUFFIX. + Use to disambiguate parallel matrix legs so each leg writes to its own + artifact directory (e.g. set to matrix.target: windows, linux). + Leave empty (default) for single-job workflows. + required: false + default: '' + + repo-url: + description: > + Override the repository URL passed to Invoke-CIJob.ps1. + Defaults to github.server_url/repository.git (HTTPS). + Set to an SSH URL (e.g. ssh://gitea-ci/Org/repo.git) when the host + runner uses an SSH alias for Gitea. + required: false + default: '' + + extra-guest-env-json: + description: > + JSON object of extra environment variables to inject into the build VM + guest before the build command runs. Use for signing keys and secrets. + Example: '{"SIGN_PASS":"${{ secrets.SIGN_PASS }}","GPG_KEY_ID":"0xABCD1234"}' + Values are passed via ExtraGuestEnv hashtable in Invoke-CIJob.ps1 and + are never echoed to the log. + required: false + default: '{}' + artifact-path: + description: > + Host-side directory where Invoke-CIJob.ps1 placed collected artifacts. + Format: F:\CI\Artifacts\{run_id}-{run_attempt} + value: ${{ steps.invoke-ci.outputs.artifact-path }} + + artifact-name: + description: 'Name used when uploading the artifact to Gitea.' + value: ${{ steps.invoke-ci.outputs.artifact-name }} + +runs: + using: composite + steps: + + # ── Invoke the CI orchestrator ───────────────────────────────────────────── + - name: Invoke CI Job + id: invoke-ci + shell: powershell + # Inputs are forwarded via environment variables to avoid shell injection + # from user-supplied build-command or path strings. + env: + INPUT_BUILD_COMMAND: ${{ inputs.build-command }} + INPUT_ARTIFACT_SOURCE: ${{ inputs.artifact-source }} + INPUT_SUBMODULES: ${{ inputs.submodules }} + INPUT_GUEST_OS: ${{ inputs.guest-os }} + INPUT_USE_GIT_CLONE: ${{ inputs.use-git-clone }} + INPUT_CONFIGURATION: ${{ inputs.configuration }} + INPUT_TEMPLATE_PATH: ${{ inputs.template-path }} + INPUT_SNAPSHOT_NAME: ${{ inputs.snapshot-name }} + INPUT_ARTIFACT_NAME: ${{ inputs.artifact-name }} + INPUT_JOB_ID_SUFFIX: ${{ inputs.job-id-suffix }} + INPUT_REPO_URL: ${{ inputs.repo-url }} + INPUT_EXTRA_GUEST_ENV_JSON: ${{ inputs.extra-guest-env-json }} + run: | + #Requires -Version 5.1 + Set-StrictMode -Version Latest + $ErrorActionPreference = 'Stop' + + $ciScriptsDir = 'N:\Code\Workspace\Local-CI-CD-System\scripts' + + # Build job ID — append suffix when provided (matrix disambiguation) + $jobId = '${{ github.run_id }}-${{ github.run_attempt }}' + if ($env:INPUT_JOB_ID_SUFFIX) { $jobId = "$jobId-$($env:INPUT_JOB_ID_SUFFIX)" } + $artifactPath = "F:\CI\Artifacts\$jobId" + + # Resolve artifact name: caller-supplied or default to build-{run_id}-{sha} + if ($env:INPUT_ARTIFACT_NAME) { + $artifactName = $env:INPUT_ARTIFACT_NAME + } + else { + $artifactName = "build-${{ github.run_id }}-${{ github.sha }}" + } + + # Resolve repository URL: explicit override or construct from GitHub context + if ($env:INPUT_REPO_URL) { + $repoUrl = $env:INPUT_REPO_URL + } + else { + $repoUrl = '${{ github.server_url }}/${{ github.repository }}.git' + } + + # Parse extra-guest-env-json into a hashtable (§6.5 secret injection) + $extraGuestEnv = @{} + if ($env:INPUT_EXTRA_GUEST_ENV_JSON -and $env:INPUT_EXTRA_GUEST_ENV_JSON -ne '{}') { + $parsed = $env:INPUT_EXTRA_GUEST_ENV_JSON | ConvertFrom-Json + foreach ($prop in $parsed.PSObject.Properties) { + $extraGuestEnv[$prop.Name] = [string]$prop.Value + } + } + + # Build parameter hashtable for Invoke-CIJob.ps1 + $params = @{ + JobId = $jobId + RepoUrl = $repoUrl + Branch = '${{ github.ref_name }}' + Commit = '${{ github.sha }}' + BuildCommand = $env:INPUT_BUILD_COMMAND + GuestArtifactSource = $env:INPUT_ARTIFACT_SOURCE + GuestOS = $env:INPUT_GUEST_OS + Configuration = $env:INPUT_CONFIGURATION + } + + if ($env:INPUT_SUBMODULES -eq 'true') { $params['Submodules'] = $true } + if ($env:INPUT_USE_GIT_CLONE -eq 'true') { $params['UseGitClone'] = $true } + if ($env:INPUT_TEMPLATE_PATH) { $params['TemplatePath'] = $env:INPUT_TEMPLATE_PATH } + if ($env:INPUT_SNAPSHOT_NAME) { $params['SnapshotName'] = $env:INPUT_SNAPSHOT_NAME } + if ($extraGuestEnv.Count -gt 0) { $params['ExtraGuestEnv'] = $extraGuestEnv } + + & "$ciScriptsDir\Invoke-CIJob.ps1" @params + if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE } + + # Emit outputs consumed by subsequent steps + "artifact-path=$artifactPath" | Out-File -FilePath $env:GITHUB_OUTPUT -Append -Encoding utf8 + "artifact-name=$artifactName" | Out-File -FilePath $env:GITHUB_OUTPUT -Append -Encoding utf8 + + # ── Upload build artifacts on success ───────────────────────────────────── + - name: Upload artifacts + if: success() + uses: actions/upload-artifact@v4 + with: + name: ${{ steps.invoke-ci.outputs.artifact-name }} + path: ${{ steps.invoke-ci.outputs.artifact-path }}\ + retention-days: ${{ inputs.artifact-retention-days }} + if-no-files-found: error + + # ── Upload diagnostic logs on failure ───────────────────────────────────── + - name: Upload diagnostic logs on failure + if: failure() + uses: actions/upload-artifact@v4 + with: + name: build-logs-${{ github.sha }} + path: ${{ steps.invoke-ci.outputs.artifact-path }}\*.log + retention-days: 3 + if-no-files-found: ignore diff --git a/gitea/workflow-example.yml b/gitea/workflow-example.yml index d81cec8..d384d6c 100644 --- a/gitea/workflow-example.yml +++ b/gitea/workflow-example.yml @@ -166,3 +166,52 @@ jobs: # path: F:\CI\Artifacts\${{ github.run_id }}-${{ github.run_attempt }}-${{ matrix.target }}\ # retention-days: 14 # if-no-files-found: error + +--- +# COMPOSITE ACTION VARIANT (recommended for new repositories) +# Uses the reusable composite action stored in the CI system repository. +# This variant replaces the verbose inline shell block with a single `uses:` step. +# The composite action handles: Invoke-CIJob.ps1 call + artifact upload + log upload. +# +# Reference: gitea/actions/local-ci-build/action.yml in this repository. +# Deployed path in Gitea: Simone/local-ci-system / .gitea/actions/local-ci-build/action.yml +# +# Minimum example (Windows build, Python project with submodules): +# +# name: Build (Composite) +# +# on: +# push: +# tags: +# - 'v*' +# workflow_dispatch: +# +# jobs: +# build: +# runs-on: windows-build +# steps: +# - uses: actions/checkout@v4 +# - uses: Simone/local-ci-system/.gitea/actions/local-ci-build@main +# with: +# build-command: 'python build_plugin.py --final --dist-dir dist' +# artifact-source: 'dist' +# submodules: 'true' +# +# Cross-platform matrix variant using the composite action: +# +# jobs: +# build: +# strategy: +# matrix: +# target: [windows, linux] +# fail-fast: false +# runs-on: ${{ matrix.target }}-build +# steps: +# - uses: actions/checkout@v4 +# - uses: Simone/local-ci-system/.gitea/actions/local-ci-build@main +# with: +# build-command: 'python build_plugin.py --final --dist-dir dist' +# artifact-source: 'dist' +# submodules: 'true' +# guest-os: 'Auto' +# artifact-name: 'build-${{ matrix.target }}-${{ github.sha }}' diff --git a/gitea/workflows/build-nsis.yml b/gitea/workflows/build-nsis.yml index f36194c..5e0c0f3 100644 --- a/gitea/workflows/build-nsis.yml +++ b/gitea/workflows/build-nsis.yml @@ -1,13 +1,8 @@ # Build (Local CI) — pipeline per nsis-plugin-nsinnounp # -# Eseguito dal runner locale windows-build su WS1-W11. -# L'orchestratore (Invoke-CIJob.ps1) gira sull'host e: -# 1. Clona il repo sull'host (con submoduli) -# 2. Crea una VM efimera da snapshot BaseClean -# 3. Copia il sorgente nella VM via WinRM (zip transfer) -# 4. Esegue python build_plugin.py nella VM -# 5. Raccoglie artifacts.zip sull'host -# 6. Distrugge la VM +# Cross-platform matrix build: runs in parallel on both windows-build (WinRM) +# and linux-build (SSH) runners using the reusable composite action. +# Uses: Simone/local-ci-system/.gitea/actions/local-ci-build@main # # Trigger: push su tag v*, oppure run manuale @@ -21,26 +16,23 @@ on: jobs: build: - runs-on: windows-build + strategy: + matrix: + target: [windows, linux] + fail-fast: false + + # Runner label must match config.yaml labels: windows-build or linux-build + runs-on: ${{ matrix.target }}-build steps: - - name: Run CI Job - shell: powershell - run: | - & 'N:\Code\Workspace\Local-CI-CD-System\scripts\Invoke-CIJob.ps1' ` - -JobId '${{ github.run_id }}' ` - -RepoUrl 'ssh://gitea-ci/Simone/nsis-plugin-nsinnounp.git' ` - -Branch '${{ github.ref_name }}' ` - -Commit '${{ github.sha }}' ` - -TemplatePath $env:GITEA_CI_TEMPLATE_PATH ` - -Submodules ` - -BuildCommand 'python build_plugin.py --final --dist-dir dist' ` - -GuestArtifactSource 'dist' - if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE } - - - name: Upload artifacts - uses: actions/upload-artifact@v4 + - uses: Simone/local-ci-system/.gitea/actions/local-ci-build@main with: - name: nsis-plugin-nsinnounp-${{ github.ref_name }} - path: F:\CI\Artifacts\${{ github.run_id }}\artifacts.zip - if-no-files-found: error + build-command: 'python build_plugin.py --final --dist-dir dist' + artifact-source: 'dist' + submodules: 'true' + # SSH URL alias configured in host ~/.ssh/config — same for both runners + repo-url: 'ssh://gitea-ci/Simone/nsis-plugin-nsinnounp.git' + # Suffix disambiguates artifact dirs: F:\CI\Artifacts\{run_id}-{attempt}-windows + job-id-suffix: '${{ matrix.target }}' + artifact-name: 'nsis-plugin-nsinnounp-${{ matrix.target }}-${{ github.ref_name }}' + diff --git a/scripts/Invoke-CIJob.ps1 b/scripts/Invoke-CIJob.ps1 index 09ff549..91e24d9 100644 --- a/scripts/Invoke-CIJob.ps1 +++ b/scripts/Invoke-CIJob.ps1 @@ -134,7 +134,12 @@ param( [string] $GuestOS = 'Auto', [string] $SshKeyPath = 'F:\CI\keys\ci_linux', - [string] $SshUser = 'ci_build' + [string] $SshUser = 'ci_build', + + # Extra environment variables injected into the guest VM before the build command (§6.5). + # Keys must be valid environment variable names. Values must be plain strings. + # Example: @{ SIGN_PASS = $env:SIGN_PASS_FROM_SECRET; GPG_KEY_ID = '0xABCD1234' } + [hashtable] $ExtraGuestEnv = @{} ) Set-StrictMode -Version Latest @@ -206,7 +211,7 @@ function Write-JobEvent { [hashtable] $Data = @{} ) if (-not $script:jsonLog) { return } - $event = [ordered]@{ + $logEntry = [ordered]@{ ts = (Get-Date -Format 'o') jobId = $script:JobId phase = $Phase @@ -214,9 +219,9 @@ function Write-JobEvent { data = $Data } try { - $event | ConvertTo-Json -Compress -Depth 5 | + $logEntry | ConvertTo-Json -Compress -Depth 5 | Add-Content -Path $script:jsonLog -Encoding UTF8 -ErrorAction Stop - } catch { } # never let logging break the build + } catch { Write-Debug "[Write-JobEvent] Logging skipped: $_" } # never let logging fail the build } Write-Host "============================================================" @@ -481,6 +486,7 @@ try { } if ($BuildCommand -ne '') { $remoteBuildParams['BuildCommand'] = $BuildCommand } if ($GuestArtifactSource -ne '') { $remoteBuildParams['GuestArtifactSource'] = $GuestArtifactSource } + if ($ExtraGuestEnv.Count -gt 0) { $remoteBuildParams['ExtraGuestEnv'] = $ExtraGuestEnv } & "$scriptDir\Invoke-RemoteBuild.ps1" @remoteBuildParams Write-JobEvent -Phase 'phase5.build' -Status 'success' @@ -549,7 +555,7 @@ finally { # ── Stop transcript ─────────────────────────────────────────────────── if ($transcriptStarted) { - try { Stop-Transcript -ErrorAction SilentlyContinue } catch {} + try { Stop-Transcript -ErrorAction SilentlyContinue } catch { Write-Debug "[Invoke-CIJob] Stop-Transcript ignored: $_" } } # ── Log retention: purge directories older than $LogRetentionDays days ─ diff --git a/scripts/Invoke-RemoteBuild.ps1 b/scripts/Invoke-RemoteBuild.ps1 index 9d8b824..281a13d 100644 --- a/scripts/Invoke-RemoteBuild.ps1 +++ b/scripts/Invoke-RemoteBuild.ps1 @@ -140,7 +140,11 @@ param( [string] $GuestLinuxWorkDir = '/opt/ci/build', # Output directory inside the Linux guest. - [string] $GuestLinuxOutputDir = '/opt/ci/output' + [string] $GuestLinuxOutputDir = '/opt/ci/output', + + # Extra environment variables injected into the guest before the build command (§6.5). + # Keys must be valid env var names. Values are plain strings (never logged). + [hashtable] $ExtraGuestEnv = @{} ) Set-StrictMode -Version Latest @@ -239,10 +243,16 @@ if ($GuestOS -eq 'Linux') { } # Step 3: Run build command (always cd into workdir first) + # Prepend extra env exports (§6.5) — scoped to this SSH command only + $envPrefix = '' + foreach ($envKey in $ExtraGuestEnv.Keys) { + $envVal = $ExtraGuestEnv[$envKey] -replace "'", "'\''" # escape ' for POSIX single-quoted string + $envPrefix += "export $envKey='$envVal'; " + } if ($BuildCommand -ne '') { - $buildCmd = "cd '$GuestLinuxWorkDir' && $BuildCommand" + $buildCmd = "${envPrefix}cd '$GuestLinuxWorkDir' && $BuildCommand" } else { - $buildCmd = "cd '$GuestLinuxWorkDir' && make" + $buildCmd = "${envPrefix}cd '$GuestLinuxWorkDir' && make" } Write-Host "[Invoke-RemoteBuild] Running build: $buildCmd" Invoke-SshCommand -IP $IPAddress -User $SshUser -KeyPath $SshKeyPath -Command $buildCmd @@ -401,12 +411,16 @@ try { # ── Custom build command (e.g. python, msbuild, cmake…) ────────── Write-Host "[Invoke-RemoteBuild] Running build command: $BuildCommand" $buildExit = Invoke-Command -Session $session -ScriptBlock { - param($workDir, $cmd, $artifactZip, $artifactSrc, $useCache) + param($workDir, $cmd, $artifactZip, $artifactSrc, $useCache, $extraEnv) # Unbuffered Python output — lines appear in real-time instead of being held in buffer $env:PYTHONUNBUFFERED = '1' if ($useCache) { $env:PIP_CACHE_DIR = '\\vmware-host\Shared Folders\pip-cache' } + # Inject extra env vars (§6.5 secret injection) + foreach ($pair in $extraEnv.GetEnumerator()) { + [System.Environment]::SetEnvironmentVariable($pair.Key, $pair.Value, 'Process') + } Set-Location $workDir # Stream each output line via Write-Host — WinRM forwards Write-Host in real-time & cmd /c "$cmd 2>&1" | ForEach-Object { Write-Host $_ } @@ -430,7 +444,7 @@ try { } return $exit - } -ArgumentList $GuestWorkDir, $BuildCommand, $GuestArtifactZip, $GuestArtifactSource, $UseSharedCache.IsPresent + } -ArgumentList $GuestWorkDir, $BuildCommand, $GuestArtifactZip, $GuestArtifactSource, $UseSharedCache.IsPresent, $ExtraGuestEnv if ($buildExit -ne 0) { throw "Build command failed (exit $buildExit)" diff --git a/scripts/Invoke-RetentionPolicy.ps1 b/scripts/Invoke-RetentionPolicy.ps1 index 0213deb..745a22a 100644 --- a/scripts/Invoke-RetentionPolicy.ps1 +++ b/scripts/Invoke-RetentionPolicy.ps1 @@ -79,6 +79,7 @@ if ($aggressiveMode) { # ── Helper: purge old subdirectories under a base dir ───────────────────────── function Remove-OldJobDirs { + [CmdletBinding(SupportsShouldProcess)] param([string]$BaseDir, [string]$Label) if (-not (Test-Path $BaseDir)) { diff --git a/scripts/Watch-RunnerHealth.ps1 b/scripts/Watch-RunnerHealth.ps1 index 3a37757..4d7363a 100644 --- a/scripts/Watch-RunnerHealth.ps1 +++ b/scripts/Watch-RunnerHealth.ps1 @@ -109,7 +109,7 @@ if (Test-Path $cooldownFile) { try { $raw = Get-Content $cooldownFile -Raw -ErrorAction SilentlyContinue if ($raw) { $restartLog = @($raw | ConvertFrom-Json) } - } catch { } + } catch { Write-Debug "[Watch-RunnerHealth] Restart log unreadable — starting fresh: $_" } } # Prune timestamps older than 1 hour diff --git a/scripts/_Common.psm1 b/scripts/_Common.psm1 index aba6b96..780e3ca 100644 --- a/scripts/_Common.psm1 +++ b/scripts/_Common.psm1 @@ -29,6 +29,8 @@ function New-CISessionOption { [System.Management.Automation.Remoting.PSSessionOption] #> [OutputType([System.Management.Automation.Remoting.PSSessionOption])] + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseShouldProcessForStateChangingFunctions', '', + Justification = 'New-CISessionOption creates an in-memory PSSessionOption object; no system state is changed.')] [CmdletBinding()] param() New-PSSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck