Simone/local-ci-cd-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(plans): add multi-agent technical review and final master plan

Browse Source 
- plans/gpt55-analysis.md: independent analysis by GPT-5.5
- plans/opus47-analysis.md: independent analysis by Opus 4.7
- plans/gpt55-review-of-opus.md: GPT-5.5 cross-review of Opus analysis
- plans/opus47-review-of-gpt55.md: Opus 4.7 cross-review of GPT-5.5 analysis
- plans/final-master-plan.md: consolidated master plan (8129 words)

Severity: 2 Critical, 12 High, 16 Medium, 10 Low, 6 Nice-to-have.
Top priorities: C2 secret logging in Invoke-RemoteBuild.ps1, C1 Linux
matrix template selection, H1 capacity=4 burn-in.
This commit is contained in:
Simone
2026-05-11 23:47:06 +02:00
parent 4758fbc519
commit ec13461862
5 changed files with 2642 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plans/gpt55-review-of-opus.md
+231
View File
@@ -0,0 +1,231 @@
# Cross-Review of Opus 4.7 Analysis
Date: 2026-05-11
Reviewer: GPT-5.5, senior DevOps cross-review role
Scope: cross-review of [plans/opus47-analysis.md](opus47-analysis.md) against [plans/gpt55-analysis.md](gpt55-analysis.md) and verified source files in the Local-CI-CD-System workspace.
## 1. Overall Assessment
The Opus 4.7 analysis is broadly useful and technically literate. It understands the system's intended shape: a Windows 11 host running Gitea Actions through act_runner, creating ephemeral VMware Workstation linked clones, running builds over WinRM or SSH, collecting artifacts, and destroying the clone in a `finally` path. Its strongest quality is that it does not try to replace the architecture with an enterprise platform. It recognizes that the single-host, file-state, PowerShell 5.1 design is the right design for this homelab, then focuses most of its critique on validation, concurrency, and operational hygiene.
The strongest insight in Opus is the focus on concurrency as an unproven capability. The review correctly calls out that [runner/config.yaml](../runner/config.yaml#L14-L17) advertises `capacity: 4`, while the implementation depends on a DHCP-sensitive clone/start/IP critical section in [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1#L303-L409). The fact that the lock is well designed does not prove it works under four simultaneous real jobs. Opus is right that this is the most important claim to validate before treating the system as unattended-daily-use ready.
The biggest weakness is severity calibration. Opus often identifies the right issue but labels it too high or too low depending on whether it matches the review's narrative. It calls the malformed composite-action outputs a LOW issue even though artifact upload can break the live workflow. It labels the SYSTEM-vs-user SSH alias mismatch HIGH while explicitly admitting it was unverified. It calls the template-refresh runbook absent even though several docs contain partial procedures. It also misses a likely live Linux-matrix bug: the composite action does not map `linux-build` to `GITEA_CI_LINUX_TEMPLATE_PATH`, so the Linux matrix leg can still default to the Windows template unless the caller overrides `template-path` and `guest-os`.
## 2. Technical Accuracy Verification
### CRITICAL: `capacity: 4` is unvalidated
Opus claim: "`capacity: 4` is configured in `runner/config.yaml`, but the IP-allocation mutex in `scripts/Invoke-CIJob.ps1` has only ever been exercised with a single job at a time" ([plans/opus47-analysis.md](opus47-analysis.md#L15)). It later says, "Either run the burn-in or set `capacity: 1` until it is done" ([plans/opus47-analysis.md](opus47-analysis.md#L732)).
Verdict: verified correct, with one priority nuance.
The configured capacity is real: [runner/config.yaml](../runner/config.yaml#L14-L17) sets `capacity: 4`. The implementation serializes clone, start, IP discovery, and lease creation using `F:\CI\State\vm-start.lock` and a `FileShare.None` handle in [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1#L303-L325). Inside that lock, the script creates the linked clone, starts it, polls `guestVar ci-ip`, falls back to `getGuestIPAddress`, and writes an IP lease file before releasing the lock in [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1#L326-L415). The design is sane, but the design is not the same as evidence.
The repo itself supports Opus's concern. TODO says `capacity: 4` is configured but the race was not reproduced in the e2e runs because they were one job at a time ([TODO.md](../TODO.md#L226)). It also notes why `getGuestIPAddress` alone is insufficient for collision detection ([TODO.md](../TODO.md#L231)). So the claim is not speculative; it is aligned with the project notes.
Where I would adjust Opus is the label. I agree it is a top priority, but I would call it HIGH unless the README or operational process currently depends on four-way parallelism for correctness. The system can be operated safely at lower capacity; the failure mode is unvalidated throughput and possible race behavior, not guaranteed data loss. In a final master plan, the task should be: run a PS5.1-compatible four-job burn-in, document results, then either keep `capacity: 4` or temporarily set it to 1 or 2. That is more pragmatic than treating the current setting as an immediate emergency.
### HIGH: act_runner SYSTEM-vs-user SSH alias mismatch
Opus claim: "The alias lives in the user's `~/.ssh/config` - which user? `act_runner` runs as `SYSTEM`... So `Setup-Host.ps1` writes the SSH config to the user's `~/.ssh/config`, not `SYSTEM`'s. HIGH" ([plans/opus47-analysis.md](opus47-analysis.md#L414)). Opus then adds: "Treat as HIGH unverified" ([plans/opus47-analysis.md](opus47-analysis.md#L416)).
Verdict: partially correct and over-prioritized as written.
The code confirms the alias is written under the interactive user's profile: [Setup-Host.ps1](../Setup-Host.ps1#L294-L306) uses `$env:USERPROFILE\.ssh\config`. The live workflow does use the alias: [gitea/workflows/build-nsis.yml](../gitea/workflows/build-nsis.yml#L31-L35) sets `repo-url: 'ssh://gitea-ci/Simone/nsis-plugin-nsinnounp.git'`. If act_runner runs as LocalSystem, OpenSSH will not read the interactive user's SSH config or keys. That would break host-side clone mode.
The assumption that act_runner definitely runs as SYSTEM is weaker. [scripts/Register-CIScheduledTasks.ps1](../scripts/Register-CIScheduledTasks.ps1#L38-L43) says scheduled maintenance tasks run as SYSTEM; it does not install act_runner. The act_runner NSSM installation in [Setup-Host.ps1](../Setup-Host.ps1#L264-L273) does not set an explicit service account. On Windows, a service installed without an explicit account commonly runs as LocalSystem, so Opus's concern is plausible. But Opus should not have derived act_runner's identity from the scheduled-task script. The correct verification is `Get-CimInstance Win32_Service -Filter "Name='act_runner'" | Select StartName` on the host, plus a test clone under that identity.
I would keep this in the master plan, but classify it as HIGH only until the service identity is confirmed. The fix should be concrete and low ceremony: either configure SSH alias and key under `C:\Windows\System32\config\systemprofile\.ssh`, run the service under the intended user account, or stop relying on the alias and use an explicit SSH URL plus `GIT_SSH_COMMAND`/known key path.
### HIGH: `Validate-DeployState.ps1` missing `ci-report-ip.service` check
Opus claim: "there is no validation in `Validate-DeployState.ps1` that the guest service is enabled in a fresh template" and missing it means Linux jobs fall back to unreliable `getGuestIPAddress` ([plans/opus47-analysis.md](opus47-analysis.md#L135)).
Verdict: partially correct, with important nuance.
The core technical claim is right for `Validate-DeployState.ps1`. That script is Windows-oriented: it connects over WinRM HTTPS/5986 and checks Firewall, Defender, WinRM, UAC, Explorer, Server Manager, AutoAdminLogon, and Windows Update state ([template/Validate-DeployState.ps1](../template/Validate-DeployState.ps1#L42-L151)). There is no Linux branch and no `systemctl is-enabled ci-report-ip.service` check. If someone treats `Validate-DeployState.ps1` as the generic template validation entry point, it will not catch a missing Linux IP-reporting service.
But Opus overstates the absence of validation across the whole repo. The Linux installer creates the service in [template/Install-CIToolchain-Linux2404.sh](../template/Install-CIToolchain-Linux2404.sh#L203-L220) and asserts it is enabled in [template/Install-CIToolchain-Linux2404.sh](../template/Install-CIToolchain-Linux2404.sh#L221). The Linux prepare script also validates both the enabled service and executable script in [template/Prepare-LinuxBuild2404.ps1](../template/Prepare-LinuxBuild2404.ps1#L361-L362). So the service is validated in the Linux provisioning path, just not in the standalone deploy validator that Opus focused on.
The recommendation is still good: add a Linux-aware validation script or extend validation naming so operators cannot confuse a Windows deploy validator with a cross-template final gate. I would call this MEDIUM-HIGH rather than HIGH. It is a real recovery and refresh-risk issue, but not an entirely missing guardrail.
### HIGH: no codified template-refresh runbook
Opus claim: "No documented template-refresh runbook" ([plans/opus47-analysis.md](opus47-analysis.md#L377)) and "KMS re-activation, snapshot rebuild, validation steps live in operator memory" ([plans/opus47-analysis.md](opus47-analysis.md#L516)).
Verdict: partially correct and exaggerated.
There is no single authoritative runbook named "Template Refresh" that covers Windows and Linux with exact commands, preflight checks, backup, validation, smoke build, promotion, and rollback. In that narrow sense, Opus is correct. A final master plan should include one.
However, the process is not only in operator memory. [docs/WINDOWS-TEMPLATE-SETUP.md](../docs/WINDOWS-TEMPLATE-SETUP.md#L280-L289) contains a semestral refresh procedure: boot template, reactivate with `slmgr /ato`, rerun Prepare, shut down, take a versioned snapshot, update `GITEA_CI_SNAPSHOT_NAME`, and keep recent snapshots. [docs/BEST-PRACTICES.md](../docs/BEST-PRACTICES.md#L196-L219) explains BaseClean integrity, backup-before-change, clone checks, and snapshot versioning. [docs/BEST-PRACTICES.md](../docs/BEST-PRACTICES.md#L347-L359) also lists toolchain update steps. [docs/RUNBOOK.md](../docs/RUNBOOK.md#L88-L92) covers missing-snapshot recovery.
The better criticism is fragmentation and drift, not absence. Some docs still refer to old `WinBuild` backup paths ([docs/BEST-PRACTICES.md](../docs/BEST-PRACTICES.md#L207-L211)), while current host docs and runner config use `WinBuild2025` and `LinuxBuild2404` ([docs/HOST-SETUP.md](../docs/HOST-SETUP.md#L160-L163), [runner/config.yaml](../runner/config.yaml#L20-L30)). The master plan should consolidate the procedure into [docs/RUNBOOK.md](../docs/RUNBOOK.md) or a dedicated template-refresh runbook, not invent an enterprise image-management workflow.
### MEDIUM: Linux Mode 2 PAT injection
Opus claim: Linux Mode 2 builds an authenticated URL and stores it in `GIT_CLONE_URL`; Windows uses `http.extraHeader`, which is safer ([plans/opus47-analysis.md](opus47-analysis.md#L123)).
Verdict: verified correct, and slightly understated.
The Linux path rewrites the clone URL with `[uri]::EscapeDataString($pat.UserName)` and the PAT, then exports it as `GIT_CLONE_URL` in the SSH command string ([scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1#L212-L230)). It then runs `git clone "$GIT_CLONE_URL"` and unsets the variable in [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1#L226-L230). That keeps the non-authenticated URL out of the normal log line, but the credential-bearing URL is still part of the command sent through SSH and visible to process/environment inspection on the guest while the command runs.
The Windows path uses `git -c credential.helper= -c http.extraHeader=Authorization: Basic <b64>` inside the WinRM session ([scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1#L323-L334)). That is not perfect because the header can still be visible in process arguments inside the guest, but it avoids embedding credentials in a URL and avoids Git storing the credential URL.
I would keep this as MEDIUM if all builds are trusted and the PAT is scoped narrowly. I would raise it to HIGH if private repositories, signing material, or broader Gitea scopes are used. My own analysis caught an adjacent issue that Opus missed: Linux `ExtraGuestEnv` values are directly included in `$buildCmd`, and the script logs `Write-Host "[Invoke-RemoteBuild] Running build: $buildCmd"` in [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1#L239-L248). That is a direct cleartext secret-log path, stronger than the PAT URL concern.
### MEDIUM: `Measure-CIBenchmark.ps1` uses `getGuestIPAddress`
Opus claim: `Measure-CIBenchmark.ps1` uses `getGuestIPAddress` directly, contradicting the lesson encoded in readiness handling ([plans/opus47-analysis.md](opus47-analysis.md#L122)).
Verdict: verified correct.
The benchmark describes its `ip` phase as `vmrun getGuestIPAddress` in [scripts/Measure-CIBenchmark.ps1](../scripts/Measure-CIBenchmark.ps1#L16-L23). The implementation polls `Invoke-Vmrun -Operation getGuestIPAddress` until a dotted IPv4 appears in [scripts/Measure-CIBenchmark.ps1](../scripts/Measure-CIBenchmark.ps1#L129-L146). Production IP detection primarily reads `guestVar 'ci-ip'` in [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1#L348-L377) and only then falls back to `getGuestIPAddress` in [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1#L379-L389).
This is exactly the kind of benchmark drift that creates false confidence. It is not a production outage by itself, but it means benchmark data cannot be treated as representative of Linux production behavior. MEDIUM is the right severity. The pragmatic fix is to extract IP acquisition into a shared helper and call the same helper from both production and benchmark scripts.
### MEDIUM: hardcoded `N:\` paths in composite action
Opus claim: the composite action hardcodes `$ciScriptsDir = 'N:\Code\Workspace\Local-CI-CD-System\scripts'`, and this couples the action to one host path ([plans/opus47-analysis.md](opus47-analysis.md#L404)).
Verdict: verified correct, and the same pattern appears elsewhere by design.
The composite action hardcodes the script directory in [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml#L148). That makes the action non-portable even though the action is presented as reusable. Opus's suggested `${{ github.action_path }}` approach is directionally right, but the exact relative path needs care because this action lives under [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml), not necessarily `.gitea/actions` in the consuming repository.
There is a second hardcoded `N:\` default in scheduled task registration: [scripts/Register-CIScheduledTasks.ps1](../scripts/Register-CIScheduledTasks.ps1#L39-L43). That one is less concerning because scheduled tasks are host-local maintenance. The composite action is more important because it affects workflow execution and reuse.
MEDIUM is fair for the action. For the scheduled tasks, LOW is enough in this fixed homelab unless the host path is expected to change.
### Composite action outputs
Opus claim: the action emits `artifact-path` and `artifact-name` as step outputs ([plans/opus47-analysis.md](opus47-analysis.md#L399)), but its metadata outputs appear inside `inputs` and should be restructured ([plans/opus47-analysis.md](opus47-analysis.md#L405)).
Verdict: Opus identified the bug but under-rated it.
The action metadata places `artifact-path` and a second `artifact-name` under the `inputs:` map in [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml#L123-L141). There is no top-level `outputs:` block before `runs:`. The step does write outputs to `$env:GITHUB_OUTPUT` in [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml#L224-L225), and upload steps consume those outputs in [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml#L229-L244). Depending on Gitea's composite-action implementation, step outputs may still work inside the same composite, but callers will not have a clean top-level action output contract, and metadata validation is wrong.
My analysis classified this as CRITICAL because artifact upload is core workflow behavior. On reflection, HIGH is a better final-plan severity: it can break artifact publication even when VM build succeeds, but it is a small YAML fix and not a VM lifecycle failure. Opus's LOW is too soft.
## 3. Omissions
Opus missed the direct Linux `ExtraGuestEnv` secret logging path. It discussed the action putting secret JSON into a runner environment variable ([plans/opus47-analysis.md](opus47-analysis.md#L279)), but the stronger issue is in [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1#L239-L248): the Linux path builds `export KEY='value'; ...` and logs the entire command. If `extra-guest-env-json` contains `SIGN_PASS`, the secret can be printed to local transcripts and Gitea job logs. This should outrank the action-env note.
Opus missed setup and backup path drift as an operational risk. [Setup-Host.ps1](../Setup-Host.ps1#L104-L113) still creates `F:\CI\Templates\WinBuild` rather than the current `WinBuild2025`, `WinBuild2022`, and `LinuxBuild2404` directories described in [AGENTS.md](../AGENTS.md#L42-L51) and [runner/config.yaml](../runner/config.yaml#L20-L30). Opus mentions the password documentation mismatch and deprecated runner installer, but not that host rebuild from scratch can miss current directories. My analysis flagged this as HIGH because it affects disaster recovery and new-host setup.
Opus missed stale `vm-start.lock` cleanup. The lock handle is released correctly during normal script flow, as Opus verified. But after a host crash, a stale lock file can still cause confusing 10-minute waits and failure until manual cleanup. [docs/RUNBOOK.md](../docs/RUNBOOK.md#L72-L105) documents manual removal, but neither [scripts/Invoke-RetentionPolicy.ps1](../scripts/Invoke-RetentionPolicy.ps1) nor [scripts/Cleanup-OrphanedBuildVMs.ps1](../scripts/Cleanup-OrphanedBuildVMs.ps1) removes it automatically. This is a very homelab-practical fix.
Opus missed broken or stale test-plan commands that matter because this review is about trust. [docs/TEST-PLAN-v1.3-to-HEAD.md](../docs/TEST-PLAN-v1.3-to-HEAD.md#L56-L58) uses PS7-only `ForEach-Object -Parallel` and the wrong `-RepositoryUrl` parameter, while [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1#L80) defines `RepoUrl`. That makes the validation plan non-copy-pasteable on the mandated PS5.1 host.
Both analyses missed a likely Linux matrix wiring bug. [gitea/workflows/build-nsis.yml](../gitea/workflows/build-nsis.yml#L20-L35) runs a matrix over `windows` and `linux`, but it does not pass `guest-os`, `template-path`, or `snapshot-name`. The action defaults `guest-os` to `Auto` in [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml#L52-L57) and leaves `template-path` empty so `Invoke-CIJob.ps1` uses `GITEA_CI_TEMPLATE_PATH` in [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1#L114). The runner config has a separate `GITEA_CI_LINUX_TEMPLATE_PATH` in [runner/config.yaml](../runner/config.yaml#L27-L30), but the action does not consume it. Unless act_runner has per-label environment behavior not shown here, the Linux leg can clone the Windows template. This is more urgent than several Opus HIGH items because it affects the advertised Windows/Linux matrix directly.
Both analyses also under-emphasized that Mode 1 host-side cloning with `ssh://gitea-ci/...` depends on host SSH identity, while Mode 2 in-guest clone for Linux should probably use HTTPS reachable from the guest plus PAT or deploy key. The test helper [scripts/Test-E2E-Section3.3.ps1](../scripts/Test-E2E-Section3.3.ps1#L113-L115) distinguishes guest HTTPS and host SSH URLs, but the composite workflow presents one `repo-url` for both matrix legs.
## 4. Over-Engineered or Excessive Suggestions
Opus's static IP pool recommendation is technically coherent but premature. It proposes `ip-pool.json`, static assignment, Windows `netsh` or Linux cloud-init injection, and per-slot locking ([plans/opus47-analysis.md](opus47-analysis.md#L638-L654)). For capacity 4 on a single VMware Workstation host, this adds a lot of moving parts. The current lock-plus-lease model is easy to understand and already avoids simultaneous DHCP start races. First run the burn-in. If it fails because DHCP collisions actually occur, then static IPs are justified. Until then, static IP assignment is a phase-3 redesign, not a stabilization task.
The generic transport interface recommendation also smells like abstraction ahead of need. Opus suggests `Invoke-GuestCommand`, `Copy-GuestItem`, and `Wait-GuestTransportReady` wrappers behind SSH and WinRM ([plans/opus47-analysis.md](opus47-analysis.md#L656-L667)). That is reasonable in a multi-OS product, but this repo has two transports with very different operational failure modes. Explicit branches in [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1) are verbose, but they are debuggable. A small shared helper for quoting, secret redaction, and bounded native execution would pay off more than a unified transport abstraction.
Moving JSONL logs into SQLite is useful, but not a prerequisite for reliability. Opus suggests importing job history into `F:\CI\State\history.db` ([plans/opus47-analysis.md](opus47-analysis.md#L674-L677)). A 40-line `Get-CIJobSummary.ps1` over existing JSONL files is enough for a homelab. SQLite can wait until there is enough history and query pain to justify a schema.
The suggestion to switch the project from PS5.1 to PowerShell 7 conflicts with the explicit project constraint. Opus admits this is outside the documented constraints ([plans/opus47-analysis.md](opus47-analysis.md#L688-L700)). For this system, PS5.1 is not accidental. [AGENTS.md](../AGENTS.md#L21-L36) lists PS7-only constructs to avoid, and scripts consistently carry `#Requires -Version 5.1`. A PS7 migration may be pleasant for the author, but it is not a cross-review recommendation that belongs in the master stabilization plan.
The idea to validate templates automatically at the start of each CI job is also heavy. Opus proposes calling validation scripts from the orchestrator with a cached stamp ([plans/opus47-analysis.md](opus47-analysis.md#L704-L716)). That risks turning every build into a template-health workflow and adds failure modes around validation connectivity. A better homelab pattern is: validate on template refresh, then run a smoke build, then promote the snapshot. Do not make every normal build re-prove the template unless a recent incident shows drift.
## 5. Architectural Conflicts
The PowerShell 7 recommendation is the clearest conflict. This project is intentionally PS5.1 because the host is Windows 11 and the scripts are expected to run with built-in Windows PowerShell. Suggesting PS7-only features like `ForEach-Object -Parallel`, `??`, or `Test-Json` undermines the constraint documented in [AGENTS.md](../AGENTS.md#L21-L36) and repeated in the user's request. The right improvement is PS5.1-compatible `Start-Job` for concurrency tests and explicit JSON parsing/validation.
Static IP injection conflicts with the current VMware linked-clone simplicity. The system currently relies on the template snapshot plus runtime DHCP, with guestinfo reporting as a bridge. Static assignment would require modifying guest network state per clone or injecting boot-time config. That is possible, but it creates OS-specific customization paths and more persistent state. It is not invalid, but it should be gated by failed burn-in evidence.
A centralized `F:\CI\State\config.json` for all paths is a mixed recommendation. It could reduce drift, but it also creates another source of truth alongside runner config, script defaults, and docs. The homelab already has [AGENTS.md](../AGENTS.md#L42-L60) as the environment contract and [runner/config.yaml](../runner/config.yaml#L18-L30) as act_runner's environment injection. The master plan should first align existing defaults, not introduce a fourth config surface.
## 6. Incorrect Assumptions
Opus assumes `ci-report-ip.service` is "referenced but not in any file I read" ([plans/opus47-analysis.md](opus47-analysis.md#L133)). That is factually incomplete. The service is created in [template/Install-CIToolchain-Linux2404.sh](../template/Install-CIToolchain-Linux2404.sh#L203-L220), enabled and asserted in [template/Install-CIToolchain-Linux2404.sh](../template/Install-CIToolchain-Linux2404.sh#L221), and checked by Prepare in [template/Prepare-LinuxBuild2404.ps1](../template/Prepare-LinuxBuild2404.ps1#L361-L362). The critique should be about missing generic validation, not missing implementation.
Opus assumes scheduled-task SYSTEM identity proves act_runner SYSTEM identity. [scripts/Register-CIScheduledTasks.ps1](../scripts/Register-CIScheduledTasks.ps1#L38-L43) clearly applies to maintenance tasks, not the NSSM service. The NSSM install in [Setup-Host.ps1](../Setup-Host.ps1#L264-L273) likely defaults to LocalSystem, but that requires service inspection to prove.
Opus states that the action emits `artifact-path` and `artifact-name` as step outputs as a positive finding ([plans/opus47-analysis.md](opus47-analysis.md#L399)), then separately notes the metadata is malformed. The implementation writes to `$env:GITHUB_OUTPUT`, yes, but the action-level output contract is malformed in [gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml#L132-L141). Treating the composite action as "complete" is too generous.
Opus says the Linux `GIT_CLONE_URL` method keeps the PAT "never in args" in its surrounding discussion. The credential-bearing URL is still embedded in the remote command string passed to SSH. It may not be in Git's argv as the literal URL if shell expansion happens first, but it is absolutely in the shell environment and in command text sent by the host. That distinction matters.
## 7. Priority Validation
I agree with Opus that concurrency validation belongs near the top. I would set it as HIGH, or CRITICAL only if the project continues to claim production-ready status while advertising `capacity: 4` as proven. The correct action is a burn-in, not a redesign.
I disagree with HIGH for the SSH alias mismatch until service identity is confirmed. It is a plausible live bug and should be verified immediately, but the analysis itself says unverified. Use HIGH-PENDING-VERIFICATION or MEDIUM until `StartName` is known.
I would downgrade the `Validate-DeployState.ps1` Linux service check from HIGH to MEDIUM-HIGH because the Linux prepare/install path already validates the service. The missing piece is a consolidated final gate and clearer naming.
I would upgrade the composite action output bug from LOW to HIGH. It affects artifact publication, which is central to a CI pipeline. The fix is small, but severity is about impact, not effort.
I would upgrade Linux `ExtraGuestEnv` secret logging to HIGH when secrets are used. Opus did not list this specific issue. A secret printed to logs is more concrete than a PAT visible to same-user process inspection.
I would add a new HIGH item: fix Linux matrix template selection. The current action/workflow combination appears not to pass the Linux template path or Linux snapshot name, despite [runner/config.yaml](../runner/config.yaml#L27-L30) defining them. This can make the live matrix fail before reaching the deeper Linux transport concerns.
## 8. Pragmatic Improvements
Replace Opus's "static IP pool" recommendation with "run burn-in first, then decide." The immediate task should be a PS5.1 `Start-Job` harness that launches four real jobs with unique job IDs, records lock wait time, verifies distinct IP leases, and checks all VMs are destroyed. If it passes three times, keep `capacity: 4`. If it fails, then choose between capacity reduction and static IP design.
Replace "template validation at every job start" with "template promotion gate." A homelab-appropriate refresh flow is: stop runner, back up template, update template, run Validate-Setup or Linux validation, shut down cleanly, take a versioned snapshot, run one Windows and one Linux smoke build, update runner config, keep prior snapshot for one week.
Replace the generic transport abstraction with three focused helpers: `Invoke-NativeBounded` for `vmrun`/`ssh`/`scp`, `ConvertTo-SafeShellSingleQuotedString` for Linux command construction, and `Write-CIRedactedCommand` for logs. These address real failure modes without turning the scripts into a framework.
For the SSH alias issue, do not build a broad identity-management system. Add a host check that prints the act_runner service account, tests `ssh -T gitea-ci` as that account if possible, and documents exactly where the SSH config/key must live. If the service runs as LocalSystem, either install the alias under systemprofile or run act_runner under a dedicated `ci_runner` Windows account.
For the action hardcoded path, use a single environment variable such as `GITEA_CI_SCRIPT_ROOT` in [runner/config.yaml](../runner/config.yaml), or compute from `github.action_path` only if the action repository layout makes that reliable. Since the scripts are not inside a downstream repository checkout in every usage model, a runner env var may be simpler and more honest for this homelab.
## 9. Coherence with Homelab Scope
Opus mostly stays within homelab scope in its findings. It does not demand Kubernetes, centralized secret vaults, Prometheus, or ESXi. Its main recommendations are burn-in, smoke testing, small validation checks, cleanup of dead code, and better docs. That is the right posture.
Where it drifts is in the "What Would I Change" section. Static IP pools, generic transport interfaces, SQLite history, JSON schema for action inputs, template validation cached in every job, and a PS7 migration are not crazy ideas, but they are not the next moves for a single-operator VMware Workstation system. They should be explicitly marked as optional future design notes, not allowed to compete with immediate fixes like action outputs, Linux secret logging, template path selection, and service-account verification.
The review's tone is sometimes too commercial-production oriented. It repeatedly contrasts homelab with paid environments, which is fair context, but the final master plan should not optimize for commercial standards. It should optimize for repeatable local recovery, understandable scripts, and low operational surprise.
## 10. Consolidated Recommendation List
Carry forward these Opus recommendations largely unchanged:
1. Run and document a real four-way concurrency burn-in before trusting `capacity: 4`.
2. Add an on-demand end-to-end smoke test, preferably a manual Gitea workflow that runs one minimal Windows and one minimal Linux job.
3. Fix Linux Mode 2 PAT handling so credentials are not embedded in clone URLs.
4. Update `Measure-CIBenchmark.ps1` to use the same IP acquisition path as production.
5. Add or consolidate validation for `ci-report-ip.service` in the Linux template final gate.
6. Remove or hard-stop the deprecated runner installer so operators do not follow stale setup paths.
7. Align `Setup-Host.ps1` password documentation with actual behavior.
8. Fix hardcoded action script-root handling.
Carry forward these Opus recommendations, but modify them:
1. Treat the SYSTEM-vs-user SSH alias as "verify immediately" rather than proven HIGH. After verifying service identity, fix the SSH config/key location or service account.
2. Treat template-refresh docs as fragmented and drifted, not absent. Consolidate them into one runbook with backup, validation, smoke test, promotion, and rollback.
3. Treat static IP allocation as a fallback if burn-in fails, not a default redesign.
4. Keep transport branches explicit. Add small helpers for bounded native calls, quoting, and redaction instead of a full transport abstraction.
5. Add job summaries from JSONL before considering SQLite.
6. Keep PS5.1. Do not migrate to PS7 as part of stabilization.
Drop or defer these Opus recommendations from the master plan:
1. PS7 migration.
2. Per-job automatic template validation.
3. SQLite history database.
4. Full generic transport facade.
5. Static IP pool unless the measured DHCP/guestinfo design fails.
6. Schema-heavy composite action validation beyond basic input checks.
Add these recommendations that Opus missed:
1. Fix the composite action output metadata as HIGH priority: move `artifact-path` and action-level `artifact-name` under a top-level `outputs:` block and remove the duplicate input/output confusion.
2. Fix Linux `ExtraGuestEnv` logging immediately: never print expanded commands containing secret exports.
3. Fix Linux matrix template selection: when `runs-on: linux-build`, pass `guest-os: Linux`, use `GITEA_CI_LINUX_TEMPLATE_PATH`, and use `BaseClean-Linux` unless the Linux template has been aligned to `BaseClean`.
4. Fix [Setup-Host.ps1](../Setup-Host.ps1#L104-L113) directory creation to match current [AGENTS.md](../AGENTS.md#L42-L60) and [runner/config.yaml](../runner/config.yaml#L20-L30).
5. Add stale `vm-start.lock` cleanup to startup/retention maintenance, with conservative age checks and clear logging.
6. Correct [docs/TEST-PLAN-v1.3-to-HEAD.md](../docs/TEST-PLAN-v1.3-to-HEAD.md#L56-L58) to use PS5.1-compatible concurrency and the actual `-RepoUrl` parameter.
7. Add `JobId` and `ExtraGuestEnv` key validation in [scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1).
8. Wire `UseSharedCache` through the top-level workflow path only after the correctness fixes above.
Final verdict: Opus is a strong review, and most of its technical instincts are right. It is best used as a source of findings, not as-is as the master plan. The final plan should keep Opus's concurrency and validation focus, correct its overstatements, raise the issues it underplayed, and stay firmly homelab-pragmatic: fix the live workflow contract, prove capacity, prevent secret leaks, align setup paths, and consolidate refresh operations before adding new architecture.