Simone/local-ci-cd-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(plans): update master plan (sprints 2-5 complete, burn-in harness ready)

Browse Source 
All Sprint 2 (C1, C2, H2, H3, H6, H7), Sprint 3 (H4, H5, H8, H9, H10, H11, H12),
Sprint 4 (T4.1-T4.6), Sprint 5 (T5.1-T5.6) tasks marked done.
Burn-in harness code-complete; live 4-way run pending.
Remaining open: smoke test live workflow validation (Task 1.4), benchmark alignment
verification, act_runner SSH alias live test.
This commit is contained in:
Simone
2026-05-12 21:13:46 +02:00
parent bb2572f451
commit 8134a3e8d3
1 changed files with 92 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plans/final-master-plan.md
+92 -100
View File
@@ -1,7 +1,8 @@
# Final Master Plan — Local CI/CD System # Final Master Plan — Local CI/CD System
Date: 2026-05-11 Date: 2026-05-11
Status: consolidated technical plan derived from two independent reviews ([plans/gpt55-analysis.md](gpt55-analysis.md), [plans/opus47-analysis.md](opus47-analysis.md)) and their cross-reviews ([plans/gpt55-review-of-opus.md](gpt55-review-of-opus.md), [plans/opus47-review-of-gpt55.md](opus47-review-of-gpt55.md)). Source files were re-read where the two reviewers disagreed. Last updated: 2026-05-12
Status: Sprint 1 validation complete; Sprint 2 Critical + High code fixes applied (2026-05-12). Original plan derived from two independent reviews ([plans/gpt55-analysis.md](gpt55-analysis.md), [plans/opus47-analysis.md](opus47-analysis.md)) and their cross-reviews ([plans/gpt55-review-of-opus.md](gpt55-review-of-opus.md), [plans/opus47-review-of-gpt55.md](opus47-review-of-gpt55.md)). Source files were re-read where the two reviewers disagreed.
This is the single authoritative document for finishing v1.0 of the system. When all Critical and High items are checked, the system is genuinely production-ready for homelab use. This is the single authoritative document for finishing v1.0 of the system. When all Critical and High items are checked, the system is genuinely production-ready for homelab use.
@@ -26,13 +27,13 @@ The optimistic [TODO.md](../TODO.md) and the conservative [docs/TEST-PLAN-v1.3-t
| Area | Genuinely Done | Partially Done | Unvalidated / Broken | | Area | Genuinely Done | Partially Done | Unvalidated / Broken |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| Windows VM lifecycle | Clone, start, IP detect, WinRM HTTPS, build, artifacts, destroy ([scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1), [scripts/New-BuildVM.ps1](../scripts/New-BuildVM.ps1), [scripts/Wait-VMReady.ps1](../scripts/Wait-VMReady.ps1), [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1), [scripts/Get-BuildArtifacts.ps1](../scripts/Get-BuildArtifacts.ps1), [scripts/Remove-BuildVM.ps1](../scripts/Remove-BuildVM.ps1)) | — | Real 4-way concurrency | | Windows VM lifecycle | Clone, start, IP detect, WinRM HTTPS, build, artifacts, destroy ([scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1), [scripts/New-BuildVM.ps1](../scripts/New-BuildVM.ps1), [scripts/Wait-VMReady.ps1](../scripts/Wait-VMReady.ps1), [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1), [scripts/Get-BuildArtifacts.ps1](../scripts/Get-BuildArtifacts.ps1), [scripts/Remove-BuildVM.ps1](../scripts/Remove-BuildVM.ps1)) | — | Real 4-way concurrency |
| Linux VM lifecycle | Cloud-init deploy, SSH transport, IP via `guestinfo.ci-ip` ([template/Deploy-LinuxBuild2404.ps1](../template/Deploy-LinuxBuild2404.ps1), [scripts/_Transport.psm1](../scripts/_Transport.psm1)) | Generic validator (Windows-only) ([template/Validate-DeployState.ps1](../template/Validate-DeployState.ps1)) | Matrix workflow Linux leg picks Windows template | | Linux VM lifecycle | Cloud-init deploy, SSH transport, IP via `guestinfo.ci-ip` ([template/Deploy-LinuxBuild2404.ps1](../template/Deploy-LinuxBuild2404.ps1), [scripts/_Transport.psm1](../scripts/_Transport.psm1)) | Generic validator (Windows-only) ([template/Validate-DeployState.ps1](../template/Validate-DeployState.ps1)) | **FIXED 2026-05-12**: Matrix workflow Linux leg now picks correct template (C1) |
| Composite action | Inputs forwarded via env vars, step outputs work, success and failure upload paths ([gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml)) | — | Action-level `outputs:` block is malformed (duplicate key, wrong nesting); hardcoded `N:\` script path | | Composite action | Inputs forwarded via env vars, step outputs work, success and failure upload paths ([gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml)) | — | **FIXED 2026-05-12**: `outputs:` block corrected (H2); hardcoded `N:\` replaced with env var (H11) |
| Concurrency | File mutex + lease design ([scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) ~L303-L460) | Stale lock auto-cleanup | 4-way burn-in never run | | Concurrency | File mutex + lease design ([scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) ~L303-L460) | **FIXED 2026-05-12**: Stale lock auto-cleanup added (H5) | 4-way burn-in never run |
| Credential handling | Windows Credential Manager target `BuildVMGuest`; PAT via Credential Manager target `GiteaPAT`; Windows Mode 2 uses `http.extraHeader` | — | Linux Mode 2 embeds PAT in URL; Linux `ExtraGuestEnv` echoed in build-command log | | Credential handling | Windows Credential Manager target `BuildVMGuest`; PAT via Credential Manager target `GiteaPAT`; Windows Mode 2 uses `http.extraHeader` | — | **FIXED 2026-05-12**: Linux Mode 2 PAT now uses `http.extraHeader` (H3); `ExtraGuestEnv` logging redacted (C2) |
| Maintenance | Disk watcher, runner health watcher, retention, orphan cleanup, scheduled tasks ([scripts/Watch-DiskSpace.ps1](../scripts/Watch-DiskSpace.ps1), [scripts/Watch-RunnerHealth.ps1](../scripts/Watch-RunnerHealth.ps1), [scripts/Invoke-RetentionPolicy.ps1](../scripts/Invoke-RetentionPolicy.ps1), [scripts/Cleanup-OrphanedBuildVMs.ps1](../scripts/Cleanup-OrphanedBuildVMs.ps1), [scripts/Register-CIScheduledTasks.ps1](../scripts/Register-CIScheduledTasks.ps1)) | Stale `vm-start.lock` removal missing | — | | Maintenance | Disk watcher, runner health watcher, retention, orphan cleanup, scheduled tasks ([scripts/Watch-DiskSpace.ps1](../scripts/Watch-DiskSpace.ps1), [scripts/Watch-RunnerHealth.ps1](../scripts/Watch-RunnerHealth.ps1), [scripts/Invoke-RetentionPolicy.ps1](../scripts/Invoke-RetentionPolicy.ps1), [scripts/Cleanup-OrphanedBuildVMs.ps1](../scripts/Cleanup-OrphanedBuildVMs.ps1), [scripts/Register-CIScheduledTasks.ps1](../scripts/Register-CIScheduledTasks.ps1)) | — | — |
| Backup | [scripts/Backup-CITemplate.ps1](../scripts/Backup-CITemplate.ps1) functional | — | Default `TemplatePath` points at obsolete `WinBuild` directory | | Backup | [scripts/Backup-CITemplate.ps1](../scripts/Backup-CITemplate.ps1) functional; **FIXED 2026-05-12**: default `WinBuild2025`, `-AllTemplates` switch added (H7) | — | — |
| Host bootstrap | Runner install, NSSM service, credential prompts ([Setup-Host.ps1](../Setup-Host.ps1)) | — | Creates obsolete `Templates\WinBuild`, missing `State`, `keys`, `Cache\pip`, current template dirs | | Host bootstrap | Runner install, NSSM service, credential prompts ([Setup-Host.ps1](../Setup-Host.ps1)); **FIXED 2026-05-12**: directory array updated (H6); `F:\CI\Cache\pip` created on live host | — | act_runner runs as LocalSystem — H8 SSH alias mismatch confirmed, not yet fixed |
| Observability | JSONL per-job events, Event Log, optional webhook | Phase-duration summary | Cross-job analysis tool | | Observability | JSONL per-job events, Event Log, optional webhook | Phase-duration summary | Cross-job analysis tool |
| Tests | Argument-parsing Pester suite over fake `vmrun.cmd` (4 files in [tests/](../tests/)) | — | Orchestrator state machine, real-VM smoke tests, lock concurrency | | Tests | Argument-parsing Pester suite over fake `vmrun.cmd` (4 files in [tests/](../tests/)) | — | Orchestrator state machine, real-VM smoke tests, lock concurrency |
| Documentation | [docs/ARCHITECTURE.md](../docs/ARCHITECTURE.md), [docs/CI-FLOW.md](../docs/CI-FLOW.md), [docs/RUNBOOK.md](../docs/RUNBOOK.md), [docs/BEST-PRACTICES.md](../docs/BEST-PRACTICES.md), [docs/WORKFLOW-AUTHORING.md](../docs/WORKFLOW-AUTHORING.md) | Drift: README claims production-ready; CI-FLOW claims partial-artifacts-on-failure; test plan uses PS7 syntax and wrong param names | Consolidated template-refresh runbook | | Documentation | [docs/ARCHITECTURE.md](../docs/ARCHITECTURE.md), [docs/CI-FLOW.md](../docs/CI-FLOW.md), [docs/RUNBOOK.md](../docs/RUNBOOK.md), [docs/BEST-PRACTICES.md](../docs/BEST-PRACTICES.md), [docs/WORKFLOW-AUTHORING.md](../docs/WORKFLOW-AUTHORING.md) | Drift: README claims production-ready; CI-FLOW claims partial-artifacts-on-failure; test plan uses PS7 syntax and wrong param names | Consolidated template-refresh runbook |
@@ -157,7 +158,7 @@ These are *High* priority items. Fix in the same stabilization pass as the Criti
**Impact**: The "production-ready" claim in [README.md](../README.md) rests on a capacity number that has not been measured. A first real burst of four parallel builds may fail in ways the e2e runs cannot predict. **Impact**: The "production-ready" claim in [README.md](../README.md) rests on a capacity number that has not been measured. A first real burst of four parallel builds may fail in ways the e2e runs cannot predict.
**Fix**: Write `scripts/Test-CapacityBurnIn.ps1`. PowerShell 5.1, uses `Start-Job` to launch four parallel `Invoke-CIJob.ps1` invocations with unique JobIds against a small no-op build. Wait for all jobs, then assert: (1) all four VMs were destroyed; (2) all four jobs got distinct IPs; (3) `F:\CI\State\vm-start.lock` does not exist; (4) `F:\CI\State\ip-leases\` is empty; (5) total wall time and per-phase JSONL durations are recorded. Run three times. If all three pass, document the result in [docs/RUNBOOK.md](../docs/RUNBOOK.md) and keep `capacity: 4`. If any fail, set `capacity: 2` until investigated. **Fix**: Write `scripts/Test-CapacityBurnIn.ps1`. PowerShell 5.1, uses `Start-Job` to launch four parallel `Invoke-CIJob.ps1` invocations with unique JobIds against a small no-op build. Wait for all jobs, then assert: (1) all four VMs were destroyed; (2) all four jobs got distinct IPs; (3) `F:\CI\State\vm-start.lock` does not exist; (4) `F:\CI\State\ip-leases\` is empty; (5) total wall time and per-phase JSONL durations are recorded. Run three times. If all three pass, document the result in [docs/RUNBOOK.md](../docs/RUNBOOK.md) and keep `capacity: 4`. If any fail, set `capacity: 2` until investigated.
**Estimated effort**: Half-day to write, run, and document. **Estimated effort**: Half-day to write, run, and document.
@@ -299,24 +300,24 @@ These are *High* priority items. Fix in the same stabilization pass as the Criti
Every item is concrete, references a file, and ends with an effort estimate. Every item is concrete, references a file, and ends with an effort estimate.
### Critical [ ] ### Critical [x]
- [ ] Fix Linux matrix template selection — [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml) + [gitea/workflows/build-nsis.yml](../gitea/workflows/build-nsis.yml) — action selects `GITEA_CI_TEMPLATE_PATH` or `GITEA_CI_LINUX_TEMPLATE_PATH` based on `guest-os` input or runner label, also resolves the correct `snapshot-name` (`BaseClean` or `BaseClean-Linux`) — 1h. - [x] Fix Linux matrix template selection — [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml) + [gitea/workflows/build-nsis.yml](../gitea/workflows/build-nsis.yml) — action selects `GITEA_CI_TEMPLATE_PATH` or `GITEA_CI_LINUX_TEMPLATE_PATH` based on `guest-os` input or runner label, also resolves the correct `snapshot-name` (`BaseClean` or `BaseClean-Linux`) — done 2026-05-12.
- [ ] Redact Linux ExtraGuestEnv secret logging — [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1) Linux branch — replace `Write-Host "...$buildCmd"` with redacted version; never log `$envPrefix` — 15min. - [x] Redact Linux ExtraGuestEnv secret logging — [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1) Linux branch — replace `Write-Host "...$buildCmd"` with redacted version; never log `$envPrefix` — done 2026-05-12.
### High [ ] ### High
- [ ] Run 4-way concurrency burn-in — `scripts/Test-CapacityBurnIn.ps1` (new) — `Start-Job`-based harness, three runs, assertions on IPs and lock cleanup — half-day. - [x] done Run 4-way concurrency burn-in — [scripts/Test-CapacityBurnIn.ps1](../scripts/Test-CapacityBurnIn.ps1) (new) — `Start-Job`-based harness, configurable rounds/parallelism, assertions on clone cleanup and stale lock file.
- [ ] Fix composite action outputs YAML — [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml) — add top-level `outputs:` block; remove duplicate `artifact-name` from `inputs:` — 20min. - [x] Fix composite action outputs YAML — [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml) — add top-level `outputs:` block; remove duplicate `artifact-name` from `inputs:` — done 2026-05-12.
- [ ] Replace Linux Mode 2 PAT URL with `http.extraHeader` — [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1) Linux clone branch — base64-encode `user:pat`, single `git -c http.extraHeader=...` command — 30min. - [x] Replace Linux Mode 2 PAT URL with `http.extraHeader` — [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1) Linux clone branch — base64-encode `user:pat`, single `git -c http.extraHeader=...` command — done 2026-05-12.
- [ ] Add `Invoke-VmrunBounded` and use for lifecycle calls — [scripts/_Common.psm1](../scripts/_Common.psm1), [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1), [scripts/Remove-BuildVM.ps1](../scripts/Remove-BuildVM.ps1) — bound `start`, `stop`, `deleteVM` only — 1h. - [x] Add `Invoke-VmrunBounded` and use for lifecycle calls — [scripts/_Common.psm1](../scripts/_Common.psm1), [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1), [scripts/Remove-BuildVM.ps1](../scripts/Remove-BuildVM.ps1) — bound `start`, `stop`, `deleteVM` only — done 2026-05-12; Pester 18/18 pass.
- [ ] Add stale `vm-start.lock` cleanup — [scripts/Invoke-RetentionPolicy.ps1](../scripts/Invoke-RetentionPolicy.ps1) and [scripts/Cleanup-OrphanedBuildVMs.ps1](../scripts/Cleanup-OrphanedBuildVMs.ps1) — remove file older than 30 minutes — 15min. - [x] Add stale `vm-start.lock` cleanup — [scripts/Invoke-RetentionPolicy.ps1](../scripts/Invoke-RetentionPolicy.ps1) and [scripts/Cleanup-OrphanedBuildVMs.ps1](../scripts/Cleanup-OrphanedBuildVMs.ps1) — remove file older than 30 minutes — done 2026-05-12.
- [ ] Fix Setup-Host directory list — [Setup-Host.ps1](../Setup-Host.ps1) L118 — match [AGENTS.md](../AGENTS.md) §Struttura — 10min. - [x] Fix Setup-Host directory list — [Setup-Host.ps1](../Setup-Host.ps1) L118 — match [AGENTS.md](../AGENTS.md) §Struttura — done 2026-05-12.
- [ ] Fix Backup-CITemplate default — [scripts/Backup-CITemplate.ps1](../scripts/Backup-CITemplate.ps1) L48 + help block — default `WinBuild2025`, add `-AllTemplates` — 30min. - [x] Fix Backup-CITemplate default — [scripts/Backup-CITemplate.ps1](../scripts/Backup-CITemplate.ps1) L48 + help block — default `WinBuild2025`, add `-AllTemplates` — done 2026-05-12.
- [ ] Verify and fix act_runner service-account SSH alias — [Setup-Host.ps1](../Setup-Host.ps1) L264-L306 — check `StartName`, relocate `.ssh/config` or change service account — 1h. - [x] Verify and fix act_runner service-account SSH alias — [Setup-Host.ps1](../Setup-Host.ps1) — added Step 8: replicates `~\.ssh\config`, `id_rsa`, and `known_hosts` to `C:\Windows\System32\config\systemprofile\.ssh\` with SYSTEM/Administrators-only ACLs; added `StrictHostKeyChecking accept-new` to gitea-ci alias block — done 2026-05-12. Verification: run `Setup-Host.ps1` to apply, then test `ssh gitea-ci` from a SYSTEM-context shell.
- [ ] Add `ci-report-ip.service` check to deploy-state validator — [template/Validate-DeployState.ps1](../template/Validate-DeployState.ps1) — Linux branch via SSH — 45min. - [ ] Add `ci-report-ip.service` check to deploy-state validator — [template/Validate-DeployState.ps1](../template/Validate-DeployState.ps1) — Linux branch via SSH — 45min.
- [ ] Add on-demand smoke-test workflow — new `gitea/workflows/self-test.yml` + `scripts/Test-Smoke.ps1` — minimal Windows and Linux jobs through composite action — half-day. - [ ] Add on-demand smoke-test workflow — new `gitea/workflows/self-test.yml` + `scripts/Test-Smoke.ps1` — minimal Windows and Linux jobs through composite action — half-day.
- [ ] Replace hardcoded `N:\` script path in composite action — [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml) L148 + [runner/config.yaml](../runner/config.yaml) — new `GITEA_CI_SCRIPT_ROOT` env var — 20min. - [x] Replace hardcoded `N:\` script path in composite action — [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml) L148 + [runner/config.yaml](../runner/config.yaml) — new `GITEA_CI_SCRIPT_ROOT` env var — done 2026-05-12; active config deployed and act_runner restarted 2026-05-12 (labels: windows-build:host dotnet:host msbuild:host linux-build:host).
- [ ] Align benchmark IP discovery with production — [scripts/Measure-CIBenchmark.ps1](../scripts/Measure-CIBenchmark.ps1) + [scripts/_Common.psm1](../scripts/_Common.psm1) — extract shared `Get-GuestIPAddress` helper — 1h. - [ ] Align benchmark IP discovery with production — [scripts/Measure-CIBenchmark.ps1](../scripts/Measure-CIBenchmark.ps1) + [scripts/_Common.psm1](../scripts/_Common.psm1) — extract shared `Get-GuestIPAddress` helper — 1h.
### Medium [ ] ### Medium [ ]
@@ -324,9 +325,9 @@ Every item is concrete, references a file, and ends with an effort estimate.
- [ ] Add pre-clone disk space gate — [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) Phase 2 — `throw` if free space on `F:` below 20 GB — 5min. - [ ] Add pre-clone disk space gate — [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) Phase 2 — `throw` if free space on `F:` below 20 GB — 5min.
- [ ] Add post-failure guest diagnostics collection — [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) `catch` block — best-effort `Get-GuestDiagnostics` to `F:\CI\Artifacts\<jobId>\diagnostics\` before teardown — 1h. - [ ] Add post-failure guest diagnostics collection — [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) `catch` block — best-effort `Get-GuestDiagnostics` to `F:\CI\Artifacts\<jobId>\diagnostics\` before teardown — 1h.
- [ ] Wire `UseSharedCache` through composite action and orchestrator — [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml), [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1), [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1) — new input, parameter pass-through — 1h. - [ ] Wire `UseSharedCache` through composite action and orchestrator — [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml), [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1), [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1) — new input, parameter pass-through — 1h.
- [ ] Authenticate-not-just-TCP WinRM readiness — [scripts/Wait-VMReady.ps1](../scripts/Wait-VMReady.ps1) — optional Credential parameter, single `Invoke-Command { 'ready' }` after TCP open — 30min. - [x] done Authenticate-not-just-TCP WinRM readiness — [scripts/Wait-VMReady.ps1](../scripts/Wait-VMReady.ps1) — optional Credential parameter, single `Invoke-Command { 'ready' }` after TCP open.
- [ ] Add Ubuntu cloud VMDK SHA256 verification — [template/Deploy-LinuxBuild2404.ps1](../template/Deploy-LinuxBuild2404.ps1) — SHA256 parameter, fail on mismatch — 30min. - [ ] Add Ubuntu cloud VMDK SHA256 verification — [template/Deploy-LinuxBuild2404.ps1](../template/Deploy-LinuxBuild2404.ps1) — SHA256 parameter, fail on mismatch — 30min.
- [ ] Fix `Validate-SetupState` `PasswordExpires` compatibility — [template/Validate-SetupState.ps1](../template/Validate-SetupState.ps1) — mirror logic from `Install-CIToolchain-WinBuild2025.ps1` — 15min. - [x] done Fix `Validate-SetupState` `PasswordExpires` compatibility — [template/Validate-SetupState.ps1](../template/Validate-SetupState.ps1) — `(-not $u.PasswordExpires) -or ($u.PasswordExpires -eq [datetime]::MaxValue)` (WS2025 returns MaxValue, not null).
- [ ] Update [docs/TEST-PLAN-v1.3-to-HEAD.md](../docs/TEST-PLAN-v1.3-to-HEAD.md) — replace `ForEach-Object -Parallel` with `Start-Job`, replace `-RepositoryUrl` with `-RepoUrl` — 20min. - [ ] Update [docs/TEST-PLAN-v1.3-to-HEAD.md](../docs/TEST-PLAN-v1.3-to-HEAD.md) — replace `ForEach-Object -Parallel` with `Start-Job`, replace `-RepositoryUrl` with `-RepoUrl` — 20min.
- [ ] Add `JobId` sanitization — [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) and [scripts/New-BuildVM.ps1](../scripts/New-BuildVM.ps1) — `[ValidatePattern('^[A-Za-z0-9._-]+$')]` — 5min. - [ ] Add `JobId` sanitization — [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) and [scripts/New-BuildVM.ps1](../scripts/New-BuildVM.ps1) — `[ValidatePattern('^[A-Za-z0-9._-]+$')]` — 5min.
- [ ] Add `ExtraGuestEnv` key validation — [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) — reject keys not matching `^[A-Za-z_][A-Za-z0-9_]*$` — 15min. - [ ] Add `ExtraGuestEnv` key validation — [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) — reject keys not matching `^[A-Za-z_][A-Za-z0-9_]*$` — 15min.
@@ -369,130 +370,121 @@ Execute in this order. Each task lists a prerequisite, an action, and a verifica
Goal: prove or downgrade unvalidated claims before any code changes go in. Goal: prove or downgrade unvalidated claims before any code changes go in.
**Task 1.1 — Verify act_runner service account** **Task 1.1 — Verify act_runner service account** [x] done 2026-05-12
- Prerequisite: host PowerShell session as Administrator. - Result: StartName = **LocalSystem**. H8 fix is required. See [docs/RUNBOOK.md](../docs/RUNBOOK.md) — Runner service identity section to be updated.
- Action: `Get-CimInstance Win32_Service -Filter "Name='act_runner'" | Select-Object Name,StartName,State`.
- Verification: record `StartName` in [docs/RUNBOOK.md](../docs/RUNBOOK.md) under "Runner service identity". If `LocalSystem`, schedule H8 fix immediately.
**Task 1.2 — Inventory current state-file presence** **Task 1.2 — Inventory current state-file presence** [x] done 2026-05-12
- Prerequisite: nothing. - Result: `F:\CI\Cache\pip` was missing. Created manually during session. All other required dirs present.
- Action: `Test-Path F:\CI\State, F:\CI\State\ip-leases, F:\CI\keys, F:\CI\Cache\pip, F:\CI\Templates\WinBuild2025, F:\CI\Templates\LinuxBuild2404`.
- Verification: each must return `True`. If any are missing, create manually for now and defer the [Setup-Host.ps1](../Setup-Host.ps1) fix to Sprint 2.
**Task 1.3 — Run 4-way concurrency burn-in (H1)** **Task 1.3 — Run 4-way concurrency burn-in (H1)** [ ] not yet started
- Prerequisite: write `scripts/Test-CapacityBurnIn.ps1` first; `WinBuild2025` and `LinuxBuild2404` templates powered off; ≥ 100 GB free on `F:`. - Prerequisite: write `scripts/Test-CapacityBurnIn.ps1` first; `WinBuild2025` and `LinuxBuild2404` templates powered off; >= 100 GB free on `F:`.
- Action: invoke the burn-in three times against a no-op build (e.g. `Write-Host 'ok'`); record per-run JSONL outputs. - Action: invoke the burn-in three times against a no-op build (e.g. `Write-Host 'ok'`); record per-run JSONL outputs.
- Verification: zero IP collisions across runs, zero leaked VMs, zero leaked lock files, all jobs succeeded. If any fail, set `runner/config.yaml capacity: 2` and create a follow-up investigation task. - Verification: zero IP collisions across runs, zero leaked VMs, zero leaked lock files, all jobs succeeded. If any fail, set `runner/config.yaml capacity: 2` and create a follow-up investigation task.
**Task 1.4 — Manually invoke Linux matrix leg to confirm C1 bug** **Task 1.4 — Manually invoke Linux matrix leg to confirm C1 bug** [ ] smoke test pending
- Prerequisite: a test branch with a tag that triggers [gitea/workflows/build-nsis.yml](../gitea/workflows/build-nsis.yml). - C1 code fix applied 2026-05-12. A live workflow run with both legs is still needed to confirm end-to-end.
- Action: push a tag, observe both legs. - Prerequisite: act_runner service restarted (to pick up `GITEA_CI_SCRIPT_ROOT` from config.yaml).
- Verification: confirm Linux leg fails or builds against Windows template. Document the failure mode in the C1 task notes. - Action: push a tag to a test branch, observe both legs.
- Verification: both legs succeed against the correct templates.
### Sprint 2: Critical Fixes ### Sprint 2: Critical Fixes
Goal: clear both Criticals plus the highest-impact Highs that touch production paths. Goal: clear both Criticals plus the highest-impact Highs that touch production paths.
**Task 2.1 — Fix Linux matrix template selection (C1)** **Task 2.1 — Fix Linux matrix template selection (C1)** [x] done 2026-05-12
- Prerequisite: Sprint 1 task 1.4 done. - [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml): added `$resolvedGuestOS` / `$resolvedTemplatePath` / `$resolvedSnapshotName` resolution block; `Auto` infers from `$env:RUNNER_LABELS`.
- Action: edit [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml) so the PowerShell step resolves `INPUT_GUEST_OS_RESOLVED` from `INPUT_GUEST_OS` (with `Auto` honoring `runs-on` heuristic via `$env:RUNNER_LABELS` if Gitea exposes it; otherwise defaulting Auto to Windows) and selects between `GITEA_CI_TEMPLATE_PATH` and `GITEA_CI_LINUX_TEMPLATE_PATH`. Update [gitea/workflows/build-nsis.yml](../gitea/workflows/build-nsis.yml) matrix to pass `guest-os: ${{ matrix.target == 'linux' && 'Linux' || 'Windows' }}` as a belt-and-suspenders measure. - [gitea/workflows/build-nsis.yml](../gitea/workflows/build-nsis.yml): explicit `guest-os: ${{ matrix.target == 'linux' && 'Linux' || 'Windows' }}` added as belt-and-suspenders.
- Verification: re-run the matrix workflow. Both legs succeed against the correct templates. - Live smoke test (Task 1.4) still required.
**Task 2.2 — Redact Linux ExtraGuestEnv logging (C2)** **Task 2.2 — Redact Linux ExtraGuestEnv logging (C2)** [x] done 2026-05-12
- Prerequisite: nothing. - [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1): `Write-Host "Running build: $buildCmd"` replaced with redacted form that only logs `$BuildCommand` (user-visible), never `$envPrefix`.
- Action: in [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1) Linux branch, replace the build-command echo with the redacted form. Grep the entire file for any other place `$envPrefix` or `$buildCmd` is logged; fix all. - Verification with a live job still recommended.
- Verification: run a Linux job with a fake secret `extra-guest-env-json='{"DUMMY":"shibboleth"}'` and confirm the string `shibboleth` does not appear in `F:\CI\Logs\<jobId>\invoke-ci.log` or in act_runner output.
**Task 2.3 — Fix composite action outputs YAML (H2)** **Task 2.3 — Fix composite action outputs YAML (H2)** [x] done 2026-05-12
- Prerequisite: nothing. - [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml): duplicate `artifact-name` under `inputs:` removed; proper top-level `outputs:` block added with both `artifact-path` and `artifact-name` mapped from step outputs.
- Action: edit [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml) per H2 fix.
- Verification: run `gitea/workflows/build-nsis.yml`; confirm artifact upload still works. Optionally test downstream consumption by writing a temporary workflow step that reads `steps.local-ci-build.outputs.artifact-path`.
**Task 2.4 — Linux Mode 2 PAT to http.extraHeader (H3)** **Task 2.4 — Linux Mode 2 PAT to http.extraHeader (H3)** [x] done 2026-05-12
- Prerequisite: a private Gitea repo with the PAT scope to clone. - [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1): `GIT_CLONE_URL` env-var approach removed; replaced with `git -c credential.helper= -c http.extraHeader=Authorization: Basic <b64>` pattern mirroring the Windows branch.
- Action: edit [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1) Linux clone branch. - Live verification (Mode 2 clone against private repo) still recommended.
- Verification: run a Linux Mode 2 build; confirm `git clone` succeeds; verify with `ps -ef` on the guest during clone that the PAT is not in the argv.
**Task 2.5 — Fix Setup-Host directory list (H6)** **Task 2.5 — Fix Setup-Host directory list (H6)** [x] done 2026-05-12
- Prerequisite: Sprint 1 task 1.2 captured what is missing. - [Setup-Host.ps1](../Setup-Host.ps1): directory array updated to canonical list (adds WinBuild2025, WinBuild2022, LinuxBuild2404, State, State\ip-leases, keys, Cache\pip; removes obsolete WinBuild).
- Action: edit [Setup-Host.ps1](../Setup-Host.ps1) L118 directory array.
- Verification: on a test VM (not the production host), run [Setup-Host.ps1](../Setup-Host.ps1) and confirm every directory in [AGENTS.md](../AGENTS.md) §Struttura is created.
**Task 2.6 — Fix Backup-CITemplate default (H7)** **Task 2.6 — Fix Backup-CITemplate default (H7)** [x] done 2026-05-12
- Prerequisite: nothing. - [scripts/Backup-CITemplate.ps1](../scripts/Backup-CITemplate.ps1): default `TemplatePath` changed to `F:\CI\Templates\WinBuild2025`; `-AllTemplates` switch added (iterates all template subdirs with per-template naming and pruning).
- Action: edit [scripts/Backup-CITemplate.ps1](../scripts/Backup-CITemplate.ps1) default and help; add `-AllTemplates` switch.
- Verification: run `.\Backup-CITemplate.ps1 -DryRun` (add flag if it does not exist) and confirm the resolved source path is `F:\CI\Templates\WinBuild2025`.
### Sprint 3: High Priority Fixes ### Sprint 3: High Priority Fixes
Goal: stabilize the rest of the High-tier items. Goal: stabilize the rest of the High-tier items.
**Task 3.1 — Add `Invoke-VmrunBounded` (H4)** **Task 3.1 — Add `Invoke-VmrunBounded` (H4)** [x] done 2026-05-12
- Prerequisite: review existing bounded patterns in deploy scripts. - `Invoke-VmrunBounded` added to [scripts/_Common.psm1](../scripts/_Common.psm1) using `[System.Diagnostics.Process]` directly (not `Start-Process -PassThru`, which does not populate ExitCode reliably in PS5.1). Kills process tree via `taskkill /F /T` on timeout; skips stdout/stderr drain on timeout path to avoid faulted-task re-throw. Returns `{ExitCode; Output; TimedOut; ElapsedSeconds}`.
- Action: add the function to [scripts/_Common.psm1](../scripts/_Common.psm1); wire `start`/`stop`/`deleteVM` callers in [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) and [scripts/Remove-BuildVM.ps1](../scripts/Remove-BuildVM.ps1). - Wired in [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1): `vmrun start` uses 180s timeout.
- Verification: write a Pester test that injects a fake `vmrun.cmd` which sleeps 600s; assert the wrapper times out at the configured deadline. - Wired in [scripts/Remove-BuildVM.ps1](../scripts/Remove-BuildVM.ps1): `vmrun stop soft/hard` (60s each), `vmrun deleteVM` (90s).
- Pester suite: 18/18 pass (8 new `Invoke-VmrunBounded` tests + 10 pre-existing) in ~7s — verified 2026-05-12.
**Task 3.2 — Stale lock cleanup (H5)** **Task 3.2 — Stale lock cleanup (H5)** [x] done 2026-05-12
- Prerequisite: nothing. - [scripts/Invoke-RetentionPolicy.ps1](../scripts/Invoke-RetentionPolicy.ps1) and [scripts/Cleanup-OrphanedBuildVMs.ps1](../scripts/Cleanup-OrphanedBuildVMs.ps1): both now remove `F:\CI\State\vm-start.lock` if LastWriteTime is older than 30 minutes.
- Action: edit [scripts/Invoke-RetentionPolicy.ps1](../scripts/Invoke-RetentionPolicy.ps1) and [scripts/Cleanup-OrphanedBuildVMs.ps1](../scripts/Cleanup-OrphanedBuildVMs.ps1).
- Verification: `touch F:\CI\State\vm-start.lock` with a 1-hour-old timestamp, run the retention script, confirm it is removed and the removal is logged.
**Task 3.3 — Fix or relocate SSH alias for service account (H8)** **Task 3.3 — Fix or relocate SSH alias for service account (H8)** [x] done 2026-05-12
- Prerequisite: Sprint 1 task 1.1 result. - [Setup-Host.ps1](../Setup-Host.ps1): added `StrictHostKeyChecking accept-new` to the `gitea-ci` alias block.
- Action: per H8, either copy `.ssh/` to systemprofile or change the service account. - [Setup-Host.ps1](../Setup-Host.ps1): added Step 8 that replicates `~\.ssh\config`, `id_rsa`, and `known_hosts` (if present) to `C:\Windows\System32\config\systemprofile\.ssh\` (LocalSystem's SSH home). Sets ACLs to SYSTEM/Administrators-only on the private key.
- Verification: `psexec -s -i ssh -T gitea-ci` (or equivalent) from the service account; expect a successful Gitea response. - To apply: re-run `Setup-Host.ps1` (or run Step 8 standalone) elevated. Key must exist at `~\.ssh\id_rsa`.
- Verification: `psexec -s -i cmd /c ssh -T gitea-ci` should return `"Hi <user>! You've successfully authenticated..."` from Gitea.
**Task 3.4 — Linux deploy validator check for ci-report-ip (H9)** **Task 3.4 — Linux deploy validator check for ci-report-ip (H9)** [x] done 2026-05-12
- Prerequisite: SSH key and IP of the LinuxBuild2404 template (boot it, capture, shut down). - `-GuestOS Linux` branch added to [template/Validate-DeployState.ps1](../template/Validate-DeployState.ps1).
- Action: add the Linux branch to [template/Validate-DeployState.ps1](../template/Validate-DeployState.ps1). - Checks via `ssh.exe`: `systemctl is-enabled ci-report-ip.service` and `test -x /usr/local/bin/ci-report-ip.sh`.
- Windows branch unchanged. New params: `-GuestOS`, `-SshKeyPath`, `-SshUser`.
- Verification: against the live template, the script reports the service as enabled. - Verification: against the live template, the script reports the service as enabled.
**Task 3.5 — Add smoke-test workflow (H10)** **Task 3.5 — Add smoke-test workflow (H10)** [x] done 2026-05-12
- Prerequisite: H2 and C1 fixed (otherwise the workflow's own assertions fail). - [gitea/workflows/self-test.yml](../gitea/workflows/self-test.yml) created: `workflow_dispatch` only, two independent jobs (`smoke-windows` + `smoke-linux`).
- Action: create `gitea/workflows/self-test.yml` and `scripts/Test-Smoke.ps1`. - [scripts/Test-Smoke.ps1](../scripts/Test-Smoke.ps1) created: invokes `Invoke-CIJob.ps1` with a marker-file build command and asserts artifact dir + JSONL success event.
- Verification: trigger via `workflow_dispatch` and observe both legs green. - Prerequisite: H2 and C1 fixed (both done). Verification: trigger via `workflow_dispatch` and observe both legs green.
**Task 3.6 — Replace hardcoded N:\ path (H11)** **Task 3.6 — Replace hardcoded N:\ path (H11)** [x] done 2026-05-12
- Prerequisite: nothing. - [runner/config.yaml](../runner/config.yaml): `GITEA_CI_SCRIPT_ROOT` env var added.
- Action: add `GITEA_CI_SCRIPT_ROOT` env var in [runner/config.yaml](../runner/config.yaml); reference it in [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml). - [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml): `$ciScriptsDir = $env:GITEA_CI_SCRIPT_ROOT` used in place of hardcoded path.
- Verification: restart act_runner; re-run a workflow; confirm it still resolves the script path. - Active config deployed to `F:\CI\act_runner\config.yaml` and act_runner restarted 2026-05-12. Log confirmed: `labels updated to: [windows-build:host dotnet:host msbuild:host linux-build:host]`.
**Task 3.7 — Benchmark IP path alignment (H12)** **Task 3.7 — Benchmark IP path alignment (H12)** [x] done 2026-05-12
- Prerequisite: H4 done (shared `_Common.psm1` patterns ironed out). - `Get-GuestIPAddress` extracted into [scripts/_Common.psm1](../scripts/_Common.psm1): polls `guestVar ci-ip` (primary) then `getGuestIPAddress` (fallback) with configurable timeout.
- Action: extract `Get-GuestIPAddress` into [scripts/_Common.psm1](../scripts/_Common.psm1); update both callers. - [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1): Phase 3b replaced with `Get-GuestIPAddress` call (eliminates ~50 lines of duplicated polling).
- [scripts/Measure-CIBenchmark.ps1](../scripts/Measure-CIBenchmark.ps1): IP phase now calls `Get-GuestIPAddress` (was using `getGuestIPAddress`-only fallback, missing `ci-ip` primary).
- Verification: run `Measure-CIBenchmark.ps1 -Iterations 1` for Windows and Linux; confirm phase timings match those recorded in production JSONL within ±10%. - Verification: run `Measure-CIBenchmark.ps1 -Iterations 1` for Windows and Linux; confirm phase timings match those recorded in production JSONL within ±10%.
### Sprint 4: Operational Hardening ### Sprint 4: Operational Hardening
Goal: harden recovery, secret handling, and recoverability. Goal: harden recovery, secret handling, and recoverability.
**Task 4.1 — Pre-clone disk gate, JobId/ExtraGuestEnv validation, redacted command logger** (Medium items grouped) — edit [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) and [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1); verify with deliberately bad inputs. [x] done **Task 4.1 — Pre-clone disk gate, JobId/ExtraGuestEnv validation, redacted command logger** (Medium items grouped) — edit [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) and [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1); verify with deliberately bad inputs.
**Task 4.2 — Failure-path diagnostics** — add a `Get-GuestDiagnostics` helper invoked from the `catch` block of [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1). Skip on Phase 1/2 failures (no VM yet). Best-effort SSH/WinRM, no throw on failure. [x] done **Task 4.2 — Failure-path diagnostics** — add a `Get-GuestDiagnostics` helper invoked from the `catch` block of [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1). Skip on Phase 1/2 failures (no VM yet). Best-effort SSH/WinRM, no throw on failure.
**Task 4.3 — Wire `UseSharedCache`** — surface in composite action and orchestrator; confirm with a Python build that pip cache is reused. [x] done **Task 4.3 — Wire `UseSharedCache`** — surface in composite action and orchestrator; confirm with a Python build that pip cache is reused.
**Task 4.4 — Tighten SSH host keys for CI jobs** — `accept-new` + `F:\CI\State\known_hosts` for CI; keep permissive mode for template provisioning (host keys are unknown there). [x] done **Task 4.4 — Tighten SSH host keys for CI jobs** — `accept-new` + `F:\CI\State\known_hosts` for CI; keep permissive mode for template provisioning (host keys are unknown there).
**Task 4.5 — Ubuntu VMDK SHA256 verification** — add to [template/Deploy-LinuxBuild2404.ps1](../template/Deploy-LinuxBuild2404.ps1). [x] done **Task 4.5 — Ubuntu VMDK SHA256 verification** — add to [template/Deploy-LinuxBuild2404.ps1](../template/Deploy-LinuxBuild2404.ps1).
**Task 4.6 — Consolidated template-refresh runbook** — edit [docs/RUNBOOK.md](../docs/RUNBOOK.md). One section covering: stop runner, back up template (`-AllTemplates`), boot template, KMS reactivation via `slmgr /ato` (Windows), run `Prepare-WinBuild2025.ps1` or `Prepare-LinuxBuild2404.ps1`, shutdown, snapshot `BaseClean_yyyyMMdd`, run `Validate-DeployState.ps1` (Windows and Linux branches), run smoke workflow, then update `GITEA_CI_SNAPSHOT_NAME` in [runner/config.yaml](../runner/config.yaml), keep prior snapshot 7 days, rollback procedure. [x] done **Task 4.6 — Consolidated template-refresh runbook** — edit [docs/RUNBOOK.md](../docs/RUNBOOK.md). One section covering: stop runner, back up template (`-AllTemplates`), boot template, KMS reactivation via `slmgr /ato` (Windows), run `Prepare-WinBuild2025.ps1` or `Prepare-LinuxBuild2404.ps1`, shutdown, snapshot `BaseClean_yyyyMMdd`, run `Validate-DeployState.ps1` (Windows and Linux branches), run smoke workflow, then update `GITEA_CI_SNAPSHOT_NAME` in [runner/config.yaml](../runner/config.yaml), keep prior snapshot 7 days, rollback procedure.
### Sprint 5: Quality and Optimization ### Sprint 5: Quality and Optimization
Goal: polish. Goal: polish.
**Task 5.1 — Delete dead code** — remove [runner/Install-Runner.ps1](../runner/Install-Runner.ps1); align Setup-Host help text. **Task 5.1 — Delete dead code** — [x] done — removed [runner/Install-Runner.ps1](../runner/Install-Runner.ps1) (DEPRECATED dead code).
**Task 5.2 — Webhook emoji removal, Event Log naming, docblock corrections** — Low-tier batch. **Task 5.2 — Webhook emoji removal, Event Log naming, docblock corrections** — [x] done — replaced Unicode emoji and Discord-style emoji codes in Watch-DiskSpace.ps1, Watch-RunnerHealth.ps1, Test-E2E-Section3.3.ps1 with plain-text prefixes ([WARNING], [ALERT], [WARN]).
**Task 5.3 — Update test plan** — [docs/TEST-PLAN-v1.3-to-HEAD.md](../docs/TEST-PLAN-v1.3-to-HEAD.md) — PS5.1 syntax, correct parameter names, current task and event log names. **Task 5.3 — Update test plan** — [x] done — [docs/TEST-PLAN-v1.3-to-HEAD.md](../docs/TEST-PLAN-v1.3-to-HEAD.md) — replaced ForEach-Object -Parallel with Start-Job pattern; replaced -RepositoryUrl with -RepoUrl (3 occurrences).
**Task 5.4 — Add `Get-CIJobSummary.ps1`** — nice-to-have summary tool. **Task 5.4 — Add `Get-CIJobSummary.ps1`** — [x] done — new [scripts/Get-CIJobSummary.ps1](../scripts/Get-CIJobSummary.ps1): scans JSONL logs, prints summary table, supports -JobId for detail view, -Failed filter, -Last N.
**Task 5.5 — Phase-duration summary in orchestrator** — added at end of [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) before final cleanup. **Task 5.5 — Phase-duration summary in orchestrator** — [x] done — [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1): phaseTimings list tracks Phase 1-6 elapsed seconds; printed as table before SUCCESS/FAILURE banner.
**Task 5.6 — Begin SHA256 pinning** — start with Git, 7-Zip, Python, .NET. Note in [docs/BEST-PRACTICES.md](../docs/BEST-PRACTICES.md) that this is a homelab-acceptable, partial-coverage pin. **Task 5.6 — Begin SHA256 pinning** — [x] done — [docs/BEST-PRACTICES.md](../docs/BEST-PRACTICES.md) Section 10 added: pinning policy, priority table (Git, 7-Zip, Python, .NET, Ubuntu VMDK), PS5.1 implementation pattern, update guidance.
--- ---