/var/lib/ci/ is owned by ci-runner; running as simone causes
PermissionError on artifact dir creation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All shims defaulted to F:\CI\python\venv\Scripts\python.exe when
CI_VENV_PYTHON was not set. On Linux host this caused instant failure.
Now uses $IsWindows (PS 6+ automatic var; absent on PS 5.1 = Windows)
to select /opt/ci/venv/bin/python on Linux. Windows paths unchanged.
Affects: Invoke-CIJob, Get-BuildArtifacts, Cleanup-OrphanedBuildVMs,
New-BuildVM, Remove-BuildVM, Wait-VMReady, Get-CIJobSummary,
Invoke-RemoteBuild, Watch-DiskSpace, Watch-RunnerHealth,
Set-CIGuestCredential, Test-CIGuestWinRM.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Passing $psExe as a second -ArgumentList item caused double-wrapping of
the string array, collapsing all args into one string when splatted.
Computing $IsWindows inside the worker avoids the extra param entirely.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
powershell.exe does not exist on Linux — child process launch failed
instantly (ExitCode null, Elapsed 00:00). Now selects powershell.exe
on Windows and pwsh on Linux via $IsWindows automatic variable.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Data from Measure-CIBenchmark.ps1 × 4 (2026-05-17, commit 36913ab):
avg boot 60.6s, IP-acquire dominant (20–85s, σ≈26s). Adds ±20% ranges
for B7 comparison and guidance on sample size given IP-phase variance.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Burn-in (4 jobs × 10 rounds), 7-day stability run, and RUNBOOK.md
Linux section are still open. Checklist stays active until all
acceptance criteria are met.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Moves all A/B closeouts, checklists, idea docs, and implementation plans
to plans/archived/2026-05-23/. Both phases are production-stable.
Active plans remaining in plans/:
- idea-3-powershell-removal.md (Phase C — in progress)
- idea-3-esxi-support.md (Phase D — future)
- ideas-overview.md (roadmap reference)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Relocates the graphify BFS/path query script from scripts/ (a PowerShell
folder) into a new tools/ directory for Python developer utilities.
The script is not part of the CI pipeline — it's a local query tool for
navigating graphify-out/graph.json during codebase investigation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
graphify-out/ contains the knowledge graph, AST/semantic cache, and
HTML report — all regenerated by /graphify. No value in tracking them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ns7zip matrix Win+Linux passes after the WinRM quotas + desktop-heap
fixes landed in the template scripts. Monitoring window completed
without regressions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Parallel MSBuild fan-out (e.g. ns7zip x86/x64/x86-ansi) over pypsrp
Linux→Win fails with MSBuild MSB6003 / Win32 1816 'Not enough quota'
because two default guest limits are too tight for concurrent native
processes via WinRM:
- Plugin Microsoft.PowerShell MaxMemoryPerShellMB = 150 MB
(pypsrp uses the plugin endpoint, not the Shell endpoint that
Deploy was previously bumping)
- Non-interactive desktop heap SharedSection 3rd field = 768 KB
(Session 0 csrss; CreateProcess fails when 3+ cl.exe spawn)
Pre-migration Win→Win went through Invoke-Command on PSSession with
quotas inherited from the local process, so the limits never bit.
Changes:
- Deploy-WinBuild2025.ps1 / Deploy-WinBuild2022.ps1: also set the
plugin-level MaxMemoryPerShellMB=2048, bump MaxProcessesPerShell
25→100, patch SharedSection 3rd field 768→4096 (reboot needed for
csrss to pick up, applied by post-install shutdown).
- Install-CIToolchain-WinBuild2025/2022.ps1: add Assert-Step for
plugin quota, MaxProcessesPerShell≥100, SharedSection≥4096; fail
hard if any are under threshold.
- docs/WINDOWS-TEMPLATE-SETUP.md: new "WinRM quotas e desktop heap"
section explaining the diagnosis + one-shot patch for legacy
templates; updated Step 3 validation row + troubleshooting entry.
- AGENTS.md: error #13 with symptom, root cause, and pointer to docs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If ci-backup-template.service is killed mid-run (SIGTERM), the Python
finally block that restarts act-runner.service is skipped. A subsequent
backup then finds the runner already inactive, _stop_runner returns
False, and the finally restart is gated off — leaving the runner down
indefinitely (incident: 21 mag 2026, runner stayed off ~80 min).
Add ExecStopPost=-/bin/systemctl start act-runner.service so systemd
guarantees the runner is restarted regardless of how the unit exits
(success, failure, signal). The `-` prefix preserves the unit's own
exit status.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add plans/idea-3-powershell-removal.md: Phase C goal is to eliminate
the pwsh dependency from the Linux host by porting the remaining PS
scripts (bench run, validate host, smoke run) to Python sub-commands.
- Update idea-3-esxi-support.md header: Fase C → Fase D.
- Update ideas-overview.md: new four-phase table (A/B/C/D) with
prerequisiti and criteri for Phase C.
- Rename all "Hook futuri Fase C" (ESXi) → "Hook futuri Fase D" in
implementation-plan-A-B.md; add Phase C/D entries to summary and
references section.
- Update PhaseB-user-checklist.md end note and CLAUDE.md to point
Phase C → pwsh removal, Phase D → ESXi.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Add `retention run` command (ports Invoke-RetentionPolicy.ps1): purges
old artifact/log dirs with aggressive mode when disk space is low.
- Add `template backup` command (ports Backup-CITemplate.ps1): 7z -mx=1
compressed archives in /var/lib/ci/backups/, prunes to keep-count=3,
stops/restarts act-runner.service around the copy.
- Update ci-retention-policy.service and ci-backup-template.service to use
Python; pwsh is no longer required on the Linux host.
- Fix ci-watch-runner-health.service: pass --service-name act-runner
(Linux service name, not Windows act_runner default).
- Fix _list_orphans in vm.py: wrap is_dir() inside the OSError try block
so a stat() failure on an entry is silently skipped rather than raised.
- Mark B5 complete in PhaseB-user-checklist.md.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
User requirement (non-negotiable): the project must stay
first-class compatible with a Windows host permanently — Phase B
adds Linux *alongside* Windows, never replacing/degrading it.
- implementation-plan-A-B §2: categorical-constraint banner; B8
decommissioning cancelled (kept only as historical record);
master checklist B8 row updated.
- PhaseB-user-checklist: Passo 9 (B8) cancelled -> dual-host kept;
Passo 8 no longer a decommission gate; tracking table + closing
note updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
build-ns7zip builds an EXTERNAL repo (nsis-plugin-ns7zip). The
`push: tags v*` trigger (inherited from when this workflow lived in
the plugin repo) made every CI-system tag — including the Phase A
marker v2.0.0-phaseA — auto-build and publish an ns7zip release on
the CI-system repo. Remove the tag trigger; manual dispatch only.
The release job's tag-gate `if` now never fires (skipped) — release
flow already validated with a throwaway tag.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Medium update: exec-summary status (A COMPLETATA, B ready), master
checklist A items [x], Phase A gating satisfied, §1 COMPLETATA banner
with real validation deviations (LocalSystem venv/keyring, github-free
artifacts, transport, shim common-params, deferred §8.1/§8.2), §7
A-side definition-of-done [x]. Fase B (§2/§3/§4) + risk matrix +
cronoprogramma left intact as the Phase B design reference alongside
PhaseB-user-checklist.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
feature merged to main (2aa12bb), tag v2.0.0-phaseA pushed
(triggers build-ns7zip as final end-to-end validation of main).
All Passi 1-9 [x]. Residual non-blocking cleanup: delete the test
Gitea releases via UI (tags kept).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
feature merged to main (2aa12bb, tree == feature). Passi 6/7/8 [x],
Passo 9 in progress (release tag pending).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The blind -X -> --x shims forwarded PowerShell common parameters
(-ErrorAction, -Verbose, ...) to the Python CLI, e.g.
Measure-CIBenchmark.ps1 -> Remove-BuildVM.ps1 -ErrorAction
SilentlyContinue produced `ci_orchestrator vm remove --error-action`
-> "No such option" -> destroy failed (orphan VM left). Skip
common switches and consume value-taking ones. Applied to all 7
blind-loop shims; param-style shims (New-BuildVM, Invoke-RemoteBuild,
Get-BuildArtifacts) are not affected. PSSA clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Start-BurnInTest.ps1 (lab wrapper: burnin-dummy + build.ps1) — 4
concurrent x 3 rounds all PASS, clean VM/lock teardown each round.
Note the gotcha: raw Test-CapacityBurnIn.ps1 without -BuildCommand
defaults to dotnet and fails on the non-.NET dummy repo.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
All workflows passed; the "if one fails" item never triggered. Replace
the dangling [ ] with a historical note so Passo 6 has no false-open
task.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
lint, self-test, build-ns7zip (Win+Linux) and the release job
(Gitea release + asset upload via REST API, auto token) all green.
Phase A Gitea end-to-end validated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
build-nsInnoUnp.yml shipped without the closing brace of the
asset-upload foreach (and no EOF newline) since its origin; the
release job had never run (tag-only) so the PS parse error
"MissingEndCurlyBrace" was latent until the test tag triggered it.
Close the loop, add per-asset + final Write-Host. PS parse verified.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Remove-Securely: suppress PSUseShouldProcessForStateChangingFunctions
(internal best-effort shred; ShouldProcess prompts are undesirable)
+ add [CmdletBinding()].
- Replace the two empty catch blocks (Set-CIGuestCredential.ps1,
Test-CIGuestWinRM.ps1) with Write-Verbose so PSAvoidUsingEmptyCatchBlock
is satisfied while keeping best-effort behavior.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
WSL on the host builds the 3 configs in parallel in ~103s (= one
config) — the parallel design is correct. The 4-vCPU CI VM ran 3x
make -j on 4 cores -> oversubscription thrash -> ~3x slower. Not
structural / not RAM. Bump guest-cpu to 8 (job.py patches the clone
VMX numvcpus); host is 10c/20t, this matrix is 2 VMs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
job.py already patches the clone VMX (numvcpus/memsize) via
--guest-cpu/--guest-memory-mb; expose them as action inputs
(default '0' = keep template). build-ns7zip.yml sets
guest-memory-mb: 8192 — the parallel MinGW cross-compile thrashes at
the 4 GB Linux template default (Windows template is 6 GB).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
After a memory-heavy parallel build the guest can momentarily fail to
launch wsmprovhost (WSManFault 1018), which broke artifact collect
(now does Compress-Archive + fetch over WinRM). run() retries up to 4x
with exponential backoff on transient WSMan markers, dropping the
cached shell so a fresh one is opened once the guest recovers.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A bare `if: github.ref_type == 'tag'` on a needs-dependent job can
leave `release` stuck in "blocked" instead of skipped for
workflow_dispatch runs on Gitea. Use the explicit
${{ github.event_name == 'push' && github.ref_type == 'tag' }} form.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- SshTransport.run_streaming: incremental channel recv, forwards
stdout/stderr live (no more one-block-at-end). _linux_build uses it
with PYTHONUNBUFFERED so the guest python flushes promptly.
- _tail_lines split out for failure messages (no double print).
- Windows live streaming deferred (TODO 8.2) — pypsrp execute_ps
returns only at completion.
- build-ns7zip.yml: use-git-clone:'true' (in-guest), repo-url ->
https://gitea.emulab.it/... (guest-reachable; host SSH alias was
host-only). Requires GiteaPAT in the LocalSystem keyring or a public
nsis-plugin-ns7zip.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ns7zip now supports --dist (commit 57133b7 on nsis-plugin-ns7zip):
DLLs land in <repo>/dist/<config>. Build with `python build_plugin.py
--dist` and collect artifact-source 'dist'.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Linux template provides `python`, so the OS-aware split is
unnecessary. Use `python build_plugin.py` for both matrix legs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
build_plugin.py is a dispatcher: it auto-detects host/toolset and
forwards extra args to the per-version script, which rejected
--final/--dist-dir. Both the Windows (legacy 2601_vs2026) and Linux
build scripts copy nsis7z.dll into <repo>/plugins/{x86-ansi,
x86-unicode,amd64-unicode}, not dist.
- build-command: just `python build_plugin.py` (python3 on Linux —
Ubuntu 24.04 has no `python`).
- artifact-source: 'plugins' (fixes the "dist missing" failure too).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Workflow now targets Simone/nsis-plugin-ns7zip. Renamed the file and
the workflow `name:`, and updated all references in active docs
(plans/archived and docs/archived left as frozen history).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Switch the cross-repo build target to Simone/nsis-plugin-ns7zip
(repo-url, artifact-name, release asset name).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
On a non-zero build the guest stdout/stderr was written unflushed
then a ClickException raised, so the cause was lost (only
"build command failed (exit 1)" surfaced). _report_build_output now
emits the output with markers, flushes both streams, and returns the
last lines which are appended to the failure message — diagnosable
even if act_runner buffering drops the body. Applied to Linux + Windows.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Simone
Simone