- Introduced a new test to verify behavior when SSH transport connects but is not ready, ensuring it returns False after the deadline.
- Added comprehensive tests for the `deploy-linux` template, covering various scenarios including missing public keys, VM directory existence, SHA256 mismatches, and failures during ISO creation and VM conversion.
- Implemented tests for SSH-related functions, including connection retries and streaming command execution, ensuring proper handling of exit statuses and progress indications.
- Enhanced existing tests to cover edge cases and ensure robustness in the deployment process.
- Implement tests for disk alert JSON output to include a message field when below threshold.
- Add tests for runner status when Gitea is online but no webhook URL is provided.
- Introduce tests for job report details to ensure no error line appears when there are no failure events.
- Add tests to skip empty JSONL files in job report listing.
Move disk cleanup out of the Windows Update loop:
- PS scripts: extract Invoke-DiskCleanup function, add -CleanupOnly switch
for standalone post-update cleanup pass.
- Python command: always pass -SkipCleanup:$true during WU loop; after
loop exits 0, run a separate -CleanupOnly invocation (skipped if
--skip-cleanup). Fails fast if cleanup-only returns nonzero.
- Tests: 3 new tests for cleanup-runs-after-loop, skip-cleanup suppresses
it, and cleanup-failure propagates as ClickException.
- ruff --fix: remove unused shutil import, sort imports, fix UP037/UP035.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
input() with a prompt string on some terminals (zsh + Linux Mint) echoes
^M and the prompt doesn't flush before reading, causing the prompt to
appear stuck or not accept input.
Replace with explicit sys.stdout.write()+flush() + sys.stdin.readline()
which guarantees the prompt is flushed before blocking on stdin, and
readline() + .strip() handles both \n and \r\n line endings correctly.
Tests updated to use CliRunner(input=) which populates sys.stdin inside
the runner context (monkeypatching sys.stdin.readline directly is ignored
by CliRunner which replaces sys.stdin entirely).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
click.confirm() doesn't strip \r — "y\r" doesn't match "y" and the prompt
loops or fails. Replace with raw input().strip().rstrip('\r').lower() which
handles both \n and \r\n line endings.
Also add --recreate-snapshot flag to skip the interactive prompt entirely
(useful for scripted re-provisioning).
Update snapshot_exists_skip/recreate tests to monkeypatch builtins.input
instead of using CliRunner input=.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- --store-credential: after snapshot, writes build-user credentials into
the system keyring using the two-entry scheme (service:meta + service)
that KeyringCredentialStore.get() reads. Works on Windows (Credential
Manager) and Linux (SecretService / file backend).
- --credential-target: keyring target name, default BuildVMGuest (matches
config.guest_cred_target).
- Rewrote module docstring: full prose documentation of both `template
backup` and `template prepare-win` including all 12 provisioning steps.
- 2 new tests covering --store-credential with custom and default targets.
- mypy --strict clean; coverage 90.03%.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the abandoned PSWSMan approach (New-PSSession hangs on Linux)
with a Python-native implementation using the proven pypsrp WinRmTransport.
- New `template prepare-win` command in commands/template.py: starts VM,
waits for WinRM, uploads and runs Install-CIToolchain-*.ps1, handles
Windows Update 3010 reboot loop (max 10 iterations), installs CI-StaticIp
scheduled task, runs post-setup validation, graceful shutdown, snapshot.
- Helpers: _ps_escape, _wait_tcp, _parse_exit_marker, _wait_winrm.
- 40 new tests in test_commands_template.py covering all major branches.
- Fix test_commands_job.py: patch load_config in _patch_common so tests
don't pick up live /var/lib/ci/config.toml ip_pool and fail on lock.
- Coverage gate: 90.06% (was 88.95% before fix).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eliminates VMware-Tools IP-detect polling (observed: 39–177 s variance on
Linux host B7 benchmark). When [ip_pool] is configured the orchestrator:
1. Allocates a slot IP from a JSON-backed pool (fcntl-locked on Linux)
2. Injects guestinfo.ip-assignment / netmask / gateway into the cloned VMX
before vmrun start
3. Skips _wait_for_ip entirely — uses the pre-assigned IP immediately
4. Releases the slot in _destroy_clone (all paths: success/failure/interrupt)
New modules / changes:
src/ci_orchestrator/ip_pool.py IpSlotPool + _file_lock context manager
src/ci_orchestrator/config.py IpPoolConfig dataclass; [ip_pool] TOML
src/ci_orchestrator/commands/job.py _inject_guestinfo_ip helper; pool wiring
tests/python/test_ip_pool.py 12 unit tests (acquire/release/persist)
tests/python/test_config.py ip_pool config parsing tests
tests/python/test_commands_job.py pool path + _inject_guestinfo_ip tests
config.example.toml [ip_pool] section (commented example)
Coverage 90.02%; mypy --strict clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add `retention run` command (ports Invoke-RetentionPolicy.ps1): purges
old artifact/log dirs with aggressive mode when disk space is low.
- Add `template backup` command (ports Backup-CITemplate.ps1): 7z -mx=1
compressed archives in /var/lib/ci/backups/, prunes to keep-count=3,
stops/restarts act-runner.service around the copy.
- Update ci-retention-policy.service and ci-backup-template.service to use
Python; pwsh is no longer required on the Linux host.
- Fix ci-watch-runner-health.service: pass --service-name act-runner
(Linux service name, not Windows act_runner default).
- Fix _list_orphans in vm.py: wrap is_dir() inside the OSError try block
so a stat() failure on an entry is silently skipped rather than raised.
- Mark B5 complete in PhaseB-user-checklist.md.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- SshTransport.run_streaming: incremental channel recv, forwards
stdout/stderr live (no more one-block-at-end). _linux_build uses it
with PYTHONUNBUFFERED so the guest python flushes promptly.
- _tail_lines split out for failure messages (no double print).
- Windows live streaming deferred (TODO 8.2) — pypsrp execute_ps
returns only at completion.
- build-ns7zip.yml: use-git-clone:'true' (in-guest), repo-url ->
https://gitea.emulab.it/... (guest-reachable; host SSH alias was
host-only). Requires GiteaPAT in the LocalSystem keyring or a public
nsis-plugin-ns7zip.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The job command previously discarded --use-git-clone and --submodules
(`del`) and always in-guest cloned with submodules off — making both
action inputs dead no-ops (submodule repos silently broken).
- job.py: --use-git-clone/--host-clone and --submodules/--no-submodules
are real booleans defaulting to ON. Default = in-guest clone with
submodules. --host-clone clones on the host into a temp dir
(_host_clone) and transfers a zip; temp dir cleaned in finally.
clone_submodules now honoured (was hardcoded False).
- action.yml: use-git-clone / submodules default 'true'; forward the
explicit positive/negative flag so the Python default is overridden
deterministically.
- self-test.yml: add smoke-{windows,linux}-hostclone jobs covering the
host-side transport (default jobs already cover in-guest).
- tests: default-transport, host-clone, and --no-submodules coverage.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Windows produced an intermediate C:\CI\output\artifacts.zip that
upload-artifact then re-zipped (zip-in-zip), while Linux deposited raw
files. Make both paths consistent: the build stages raw outputs into a
collect dir; collect zips it only as a transport archive, fetches, and
extracts into the host artifact dir; actions/upload-artifact produces
the single final archive.
- _windows_build: param artifact_zip -> output_dir; stage raw files
(in-place when source IS the collect dir, else wipe+copy), no
Compress-Archive; absolute artifact-source resolved.
- _windows_collect: zip guest dir as transport, fetch, extract (mirror
_linux_collect); guest_path is now the collect directory.
- job.py / build_run CLI: pass output_dir; rename --guest-artifact-zip
-> --guest-output-dir.
- Tests updated for the raw-file contract.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
12 dedicated tests covering: (#9) clone_linked surfaces 'should not be powered on' as typed BackendError; (#10) is_running uses 'vmrun list' only and never falls back to getGuestIPAddress (incl. on vmrun list failure); (#11) two consecutive 'vm new' invocations with identical inputs MUST produce distinct clone names/destinations (timestamp suffix), guarding the orchestrator-side safety net against the machine-id duplicate-IP failure mode; (#12) SshTransport defaults to paramiko AutoAddPolicy with no known_hosts, never shells out to ssh-keygen/ssh.exe/scp.exe, wraps paramiko import/exec/SFTP errors as TransportConnectError. Coverage: 80.10% (gate 80%).
Phase A4 of plans/implementation-plan-A-B.md. Implements the full job orchestrator (clone -> start -> wait -> probe -> build -> collect -> guaranteed cleanup) as a new commands/job.py click command, registered under python -m ci_orchestrator job. Backend selection goes through backends.load_backend(config) so Phase C can swap in remote drivers without touching the command. The legacy scripts/Invoke-CIJob.ps1 is replaced by a thin PS 5.1 shim that delegates to the Python CLI; tests/python/test_commands_job.py adds 13 cases covering Linux/Windows happy paths, override application, skip-artifact, and cleanup on every failure mode.
Four test files under tests/ using fake vmrun.cmd (exit code via
%FAKE_VMRUN_EXIT% env var) — no VMware or network dependency:
_Common.Tests.ps1 — New-CISessionOption TLS flags, Resolve-VmrunPath
throw/return, Invoke-Vmrun ExitCode/Output/ThrowOnError
New-BuildVM.Tests.ps1 — throw on missing template, throw+cleanup on clone
failure, Clone_{JobId}_{timestamp} name format
Remove-BuildVM.Tests.ps1 — no-op on missing VMX, partial dir cleanup,
-WhatIf preservation, full destroy removes dir
Wait-VMReady.Tests.ps1 — throw on missing vmrun, timeout throw,
IP ValidatePattern accept/reject
Run: Invoke-Pester tests\ -Output Detailed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Simone
Simone