Refactor documentation and analysis in final master plan, opus47 analysis, and opus47 review of GPT-5.5
- Updated the final master plan to improve clarity and consistency in the status of various capabilities, including fixes and enhancements made as of 2026-05-12. - Revised the opus47 analysis to provide a more structured overview of the system's production readiness, including detailed coverage of OWASP Top 10 risks and operational gaps. - Enhanced the opus47 review of GPT-5.5 by clarifying severity ratings and rationales for various items, while adding new high-severity items that were previously overlooked.
This commit is contained in:
Simone
+12
-12
@@ -24,19 +24,19 @@ Top 3 risks today, in order: (1) the Linux matrix leg builds against the Windows
|
||||
|
||||
The optimistic [TODO.md](../TODO.md) and the conservative [docs/TEST-PLAN-v1.3-to-HEAD.md](../docs/TEST-PLAN-v1.3-to-HEAD.md) disagree. The honest state, reconciled against the source files, is below.
|
||||
|
||||
| Area | Genuinely Done | Partially Done | Unvalidated / Broken |
|
||||
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
|
||||
| Windows VM lifecycle | Clone, start, IP detect, WinRM HTTPS, build, artifacts, destroy ([scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1), [scripts/New-BuildVM.ps1](../scripts/New-BuildVM.ps1), [scripts/Wait-VMReady.ps1](../scripts/Wait-VMReady.ps1), [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1), [scripts/Get-BuildArtifacts.ps1](../scripts/Get-BuildArtifacts.ps1), [scripts/Remove-BuildVM.ps1](../scripts/Remove-BuildVM.ps1)) | — | Real 4-way concurrency |
|
||||
| Linux VM lifecycle | Cloud-init deploy, SSH transport, IP via `guestinfo.ci-ip` ([template/Deploy-LinuxBuild2404.ps1](../template/Deploy-LinuxBuild2404.ps1), [scripts/_Transport.psm1](../scripts/_Transport.psm1)) | Generic validator (Windows-only) ([template/Validate-DeployState.ps1](../template/Validate-DeployState.ps1)) | **FIXED 2026-05-12**: Matrix workflow Linux leg now picks correct template (C1) |
|
||||
| Composite action | Inputs forwarded via env vars, step outputs work, success and failure upload paths ([gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml)) | — | **FIXED 2026-05-12**: `outputs:` block corrected (H2); hardcoded `N:\` replaced with env var (H11) |
|
||||
| Concurrency | File mutex + lease design ([scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) ~L303-L460) | **FIXED 2026-05-12**: Stale lock auto-cleanup added (H5) | 4-way burn-in never run |
|
||||
| Area | Genuinely Done | Partially Done | Unvalidated / Broken |
|
||||
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
|
||||
| Windows VM lifecycle | Clone, start, IP detect, WinRM HTTPS, build, artifacts, destroy ([scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1), [scripts/New-BuildVM.ps1](../scripts/New-BuildVM.ps1), [scripts/Wait-VMReady.ps1](../scripts/Wait-VMReady.ps1), [scripts/Invoke-RemoteBuild.ps1](../scripts/Invoke-RemoteBuild.ps1), [scripts/Get-BuildArtifacts.ps1](../scripts/Get-BuildArtifacts.ps1), [scripts/Remove-BuildVM.ps1](../scripts/Remove-BuildVM.ps1)) | — | Real 4-way concurrency |
|
||||
| Linux VM lifecycle | Cloud-init deploy, SSH transport, IP via `guestinfo.ci-ip` ([template/Deploy-LinuxBuild2404.ps1](../template/Deploy-LinuxBuild2404.ps1), [scripts/_Transport.psm1](../scripts/_Transport.psm1)) | Generic validator (Windows-only) ([template/Validate-DeployState.ps1](../template/Validate-DeployState.ps1)) | **FIXED 2026-05-12**: Matrix workflow Linux leg now picks correct template (C1) |
|
||||
| Composite action | Inputs forwarded via env vars, step outputs work, success and failure upload paths ([gitea/actions/local-ci-build/action.yml](../gitea/actions/local-ci-build/action.yml)) | — | **FIXED 2026-05-12**: `outputs:` block corrected (H2); hardcoded `N:\` replaced with env var (H11) |
|
||||
| Concurrency | File mutex + lease design ([scripts/Invoke-CIJob.ps1](../scripts/Invoke-CIJob.ps1) ~L303-L460) | **FIXED 2026-05-12**: Stale lock auto-cleanup added (H5) | 4-way burn-in never run |
|
||||
| Credential handling | Windows Credential Manager target `BuildVMGuest`; PAT via Credential Manager target `GiteaPAT`; Windows Mode 2 uses `http.extraHeader` | — | **FIXED 2026-05-12**: Linux Mode 2 PAT now uses `http.extraHeader` (H3); `ExtraGuestEnv` logging redacted (C2) |
|
||||
| Maintenance | Disk watcher, runner health watcher, retention, orphan cleanup, scheduled tasks ([scripts/Watch-DiskSpace.ps1](../scripts/Watch-DiskSpace.ps1), [scripts/Watch-RunnerHealth.ps1](../scripts/Watch-RunnerHealth.ps1), [scripts/Invoke-RetentionPolicy.ps1](../scripts/Invoke-RetentionPolicy.ps1), [scripts/Cleanup-OrphanedBuildVMs.ps1](../scripts/Cleanup-OrphanedBuildVMs.ps1), [scripts/Register-CIScheduledTasks.ps1](../scripts/Register-CIScheduledTasks.ps1)) | — | — |
|
||||
| Backup | [scripts/Backup-CITemplate.ps1](../scripts/Backup-CITemplate.ps1) functional; **FIXED 2026-05-12**: default `WinBuild2025`, `-AllTemplates` switch added (H7) | — | — |
|
||||
| Host bootstrap | Runner install, NSSM service, credential prompts ([Setup-Host.ps1](../Setup-Host.ps1)); **FIXED 2026-05-12**: directory array updated (H6); `F:\CI\Cache\pip` created on live host | — | act_runner runs as LocalSystem — H8 SSH alias mismatch confirmed, not yet fixed |
|
||||
| Observability | JSONL per-job events, Event Log, optional webhook | Phase-duration summary | Cross-job analysis tool |
|
||||
| Tests | Argument-parsing Pester suite over fake `vmrun.cmd` (4 files in [tests/](../tests/)) | — | Orchestrator state machine, real-VM smoke tests, lock concurrency |
|
||||
| Documentation | [docs/ARCHITECTURE.md](../docs/ARCHITECTURE.md), [docs/CI-FLOW.md](../docs/CI-FLOW.md), [docs/RUNBOOK.md](../docs/RUNBOOK.md), [docs/BEST-PRACTICES.md](../docs/BEST-PRACTICES.md), [docs/WORKFLOW-AUTHORING.md](../docs/WORKFLOW-AUTHORING.md) | Drift: README claims production-ready; CI-FLOW claims partial-artifacts-on-failure; test plan uses PS7 syntax and wrong param names | Consolidated template-refresh runbook |
|
||||
| Maintenance | Disk watcher, runner health watcher, retention, orphan cleanup, scheduled tasks ([scripts/Watch-DiskSpace.ps1](../scripts/Watch-DiskSpace.ps1), [scripts/Watch-RunnerHealth.ps1](../scripts/Watch-RunnerHealth.ps1), [scripts/Invoke-RetentionPolicy.ps1](../scripts/Invoke-RetentionPolicy.ps1), [scripts/Cleanup-OrphanedBuildVMs.ps1](../scripts/Cleanup-OrphanedBuildVMs.ps1), [scripts/Register-CIScheduledTasks.ps1](../scripts/Register-CIScheduledTasks.ps1)) | — | — |
|
||||
| Backup | [scripts/Backup-CITemplate.ps1](../scripts/Backup-CITemplate.ps1) functional; **FIXED 2026-05-12**: default `WinBuild2025`, `-AllTemplates` switch added (H7) | — | — |
|
||||
| Host bootstrap | Runner install, NSSM service, credential prompts ([Setup-Host.ps1](../Setup-Host.ps1)); **FIXED 2026-05-12**: directory array updated (H6); `F:\CI\Cache\pip` created on live host | — | act_runner runs as LocalSystem — H8 SSH alias mismatch confirmed, not yet fixed |
|
||||
| Observability | JSONL per-job events, Event Log, optional webhook | Phase-duration summary | Cross-job analysis tool |
|
||||
| Tests | Argument-parsing Pester suite over fake `vmrun.cmd` (4 files in [tests/](../tests/)) | — | Orchestrator state machine, real-VM smoke tests, lock concurrency |
|
||||
| Documentation | [docs/ARCHITECTURE.md](../docs/ARCHITECTURE.md), [docs/CI-FLOW.md](../docs/CI-FLOW.md), [docs/RUNBOOK.md](../docs/RUNBOOK.md), [docs/BEST-PRACTICES.md](../docs/BEST-PRACTICES.md), [docs/WORKFLOW-AUTHORING.md](../docs/WORKFLOW-AUTHORING.md) | Drift: README claims production-ready; CI-FLOW claims partial-artifacts-on-failure; test plan uses PS7 syntax and wrong param names | Consolidated template-refresh runbook |
|
||||
|
||||
The README is genuinely ahead of the evidence. The accurate label is *feature-complete homelab CI awaiting stabilization pass*. Everything in this plan exists to close that gap.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user