Commit Graph

14 Commits

Author SHA1 Message Date
Simone 4f22d11a84 feat(build): live-stream Linux build output; ns7zip via in-guest clone
Lint / pssa (push) Failing after 26s
Lint / python (push) Successful in 52s
- SshTransport.run_streaming: incremental channel recv, forwards
  stdout/stderr live (no more one-block-at-end). _linux_build uses it
  with PYTHONUNBUFFERED so the guest python flushes promptly.
- _tail_lines split out for failure messages (no double print).
- Windows live streaming deferred (TODO 8.2) — pypsrp execute_ps
  returns only at completion.
- build-ns7zip.yml: use-git-clone:'true' (in-guest), repo-url ->
  https://gitea.emulab.it/... (guest-reachable; host SSH alias was
  host-only). Requires GiteaPAT in the LocalSystem keyring or a public
  nsis-plugin-ns7zip.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 20:51:48 +02:00
Simone 29dfbf1cae feat(job): wire use-git-clone + submodules; default in-guest clone
Lint / pssa (push) Failing after 26s
Lint / python (push) Successful in 54s
The job command previously discarded --use-git-clone and --submodules
(`del`) and always in-guest cloned with submodules off — making both
action inputs dead no-ops (submodule repos silently broken).

- job.py: --use-git-clone/--host-clone and --submodules/--no-submodules
  are real booleans defaulting to ON. Default = in-guest clone with
  submodules. --host-clone clones on the host into a temp dir
  (_host_clone) and transfers a zip; temp dir cleaned in finally.
  clone_submodules now honoured (was hardcoded False).
- action.yml: use-git-clone / submodules default 'true'; forward the
  explicit positive/negative flag so the Python default is overridden
  deterministically.
- self-test.yml: add smoke-{windows,linux}-hostclone jobs covering the
  host-side transport (default jobs already cover in-guest).
- tests: default-transport, host-clone, and --no-submodules coverage.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 17:00:46 +02:00
Simone 456db0a3e2 refactor(artifacts): harmonize Windows to Linux raw-file collection
Lint / pssa (push) Failing after 23s
Lint / python (push) Successful in 50s
Windows produced an intermediate C:\CI\output\artifacts.zip that
upload-artifact then re-zipped (zip-in-zip), while Linux deposited raw
files. Make both paths consistent: the build stages raw outputs into a
collect dir; collect zips it only as a transport archive, fetches, and
extracts into the host artifact dir; actions/upload-artifact produces
the single final archive.

- _windows_build: param artifact_zip -> output_dir; stage raw files
  (in-place when source IS the collect dir, else wipe+copy), no
  Compress-Archive; absolute artifact-source resolved.
- _windows_collect: zip guest dir as transport, fetch, extract (mirror
  _linux_collect); guest_path is now the collect directory.
- job.py / build_run CLI: pass output_dir; rename --guest-artifact-zip
  -> --guest-output-dir.
- Tests updated for the raw-file contract.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 16:30:34 +02:00
Simone e810747557 docs,config: aggiorna AGENTS.md, config.example, HOST-SETUP.md, checklist faseA e fix vari orchestrator (workstation, build, job, wait, winrm, test) 2026-05-17 00:14:04 +02:00
Simone dc8449a0d7 test: cover easy missing branches in report/config/credentials/ssh/wait/job
Lint / pssa (push) Successful in 34s
Lint / python (push) Successful in 41s
2026-05-14 23:23:55 +02:00
Simone b2b31f4b6e test: add coverage for monitor/vm/winrm to reach 90%% gate
Lint / pssa (push) Successful in 35s
Lint / python (push) Successful in 42s
Bring monitor.py 49->99%%, vm.py 79->98%%, winrm.py 76->100%%. Total project coverage 80.10%% -> 90.69%%, satisfying the 90%% gate set in commit 451a61c.
2026-05-14 23:14:08 +02:00
Simone 5459672b42 test(a5): add pytest regression tests for AGENTS.md errors #9-#12
12 dedicated tests covering: (#9) clone_linked surfaces 'should not be powered on' as typed BackendError; (#10) is_running uses 'vmrun list' only and never falls back to getGuestIPAddress (incl. on vmrun list failure); (#11) two consecutive 'vm new' invocations with identical inputs MUST produce distinct clone names/destinations (timestamp suffix), guarding the orchestrator-side safety net against the machine-id duplicate-IP failure mode; (#12) SshTransport defaults to paramiko AutoAddPolicy with no known_hosts, never shells out to ssh-keygen/ssh.exe/scp.exe, wraps paramiko import/exec/SFTP errors as TransportConnectError. Coverage: 80.10% (gate 80%).
2026-05-14 18:34:04 +02:00
Simone f5091d0903 feat(a4): port Invoke-CIJob.ps1 to ci_orchestrator job
Phase A4 of plans/implementation-plan-A-B.md. Implements the full job orchestrator (clone -> start -> wait -> probe -> build -> collect -> guaranteed cleanup) as a new commands/job.py click command, registered under python -m ci_orchestrator job. Backend selection goes through backends.load_backend(config) so Phase C can swap in remote drivers without touching the command. The legacy scripts/Invoke-CIJob.ps1 is replaced by a thin PS 5.1 shim that delegates to the Python CLI; tests/python/test_commands_job.py adds 13 cases covering Linux/Windows happy paths, override application, skip-artifact, and cleanup on every failure mode.
2026-05-14 18:34:03 +02:00
Simone 816a15503e feat(a3): port build pipeline (vm new, build run, artifacts collect)
Implements Phase A3 of plans/implementation-plan-A-B.md:

- commands/vm.py: vm new (replaces New-BuildVM.ps1)

- commands/build.py: build run (replaces Invoke-RemoteBuild.ps1) with WinRM/SSH dispatch and stdout streaming for act_runner

- commands/artifacts.py: artifacts collect (replaces Get-BuildArtifacts.ps1) using transport.fetch()

- 3 PS scripts reduced to shims preserving \0

- Pester tests/{New-BuildVM,Wait-VMReady,Remove-BuildVM}.Tests.ps1 removed; equivalent cases covered in pytest

- Phase C hook: build/artifacts accept opaque VmHandle/vmx; no Windows path assumptions

Tests: 91 pytest, ruff clean, mypy --strict clean, coverage 78.27% (>=70 gate).

Master checklist + step A3 attivita + 'definizione di fatto' updated; A3-closeout.md added.
2026-05-14 18:34:03 +02:00
Simone 80f6661ad5 feat(a2): port leaf PS scripts to ci_orchestrator CLI
Implements Phase A2 of plans/implementation-plan-A-B.md:

- commands/wait.py    -> wait-ready (replaces Wait-VMReady.ps1)

- commands/vm.py      -> vm remove + vm cleanup (replaces Remove-BuildVM.ps1, Cleanup-OrphanedBuildVMs.ps1); cleanup accepts injected VmBackend (Phase C ESXi hook preserved)

- commands/monitor.py -> monitor disk + monitor runner (replaces Watch-DiskSpace.ps1, Watch-RunnerHealth.ps1)

- commands/report.py  -> report job (replaces Get-CIJobSummary.ps1)

Each PS script reduced to a 3-line shim that delegates to the Python CLI and preserves \0.

Tests: 69 pytest cases across new test_commands_*.py modules; ruff clean, mypy --strict clean, coverage 74.5% (>=70 gate).
2026-05-14 18:34:03 +02:00
Simone bd31a3f2f3 Phase A1: bootstrap Python ci_orchestrator package
- pyproject.toml (hatchling) with ruff/mypy/pytest dev deps
- src/ci_orchestrator/: config, credentials, backends (Protocol + WorkstationVmrunBackend), transport (WinRM via pypsrp, SSH via paramiko)
- CLI entry point with PoC 'wait-ready' subcommand (Windows + Linux guests)
- tests/python/: 35 unit tests, 83% coverage, all backends/transport mocked
- gitea/workflows/lint.yml: new 'python' job (ruff + mypy --strict + pytest --cov-fail-under=70)
- config.example.toml + README setup section + .gitignore Python entries

Neutral VmBackend Protocol prepared for Phase C ESXi extension.
See plans/implementation-plan-A-B.md step A1.
2026-05-14 18:34:02 +02:00
Simone 509d1fc284 feat(sprint3): Invoke-VmrunBounded, Get-GuestIPAddress, transport and concurrency hardening
H4: Invoke-VmrunBounded added to _Common.psm1 (Start-Process + Wait-Process + taskkill on
    timeout, returns {ExitCode;Output;TimedOut;ElapsedSeconds}). Wired in Invoke-CIJob.ps1
    for vmrun start (180s) and Remove-BuildVM.ps1 for stop/deleteVM (60s/90s).
H5: Cleanup-OrphanedBuildVMs.ps1 removes stale vm-start.lock (>30 min). ValidateRange
    lowered to 0 to allow emergency -MaxAgeHours 0. Invoke-RetentionPolicy.ps1 same.
H8: _Transport.psm1 SSH hardening: StrictHostKeyChecking=accept-new, F:\CI\State\known_hosts.
H9: Validate-DeployState.ps1 Linux branch added: checks ci-report-ip.service enabled,
    ci-report-ip.sh executable via SSH. New params: -GuestOS, -SshKeyPath, -SshUser.
H12: Get-GuestIPAddress extracted into _Common.psm1 (guestinfo.ci-ip primary, getGuestIPAddress
     fallback). Invoke-CIJob.ps1 Phase 3b replaced with single call. Measure-CIBenchmark.ps1
     updated to match.
Also: _Common.psm1 adds Write-CIRedactedCommand and ConvertTo-SafeShellSingleQuotedString.
     Pester suite updated: 30/30 pass.
2026-05-12 21:12:33 +02:00
Simone fa2acc4b46 fix(tests): lower ValidateRange minimums and use .ps1 fake vmrun for Pester 2026-05-10 22:50:25 +02:00
Simone f46e4a8ca3 feat(sprint7): §5.1 — Pester smoke tests (no real VM required)
Four test files under tests/ using fake vmrun.cmd (exit code via
%FAKE_VMRUN_EXIT% env var) — no VMware or network dependency:

  _Common.Tests.ps1     — New-CISessionOption TLS flags, Resolve-VmrunPath
                          throw/return, Invoke-Vmrun ExitCode/Output/ThrowOnError
  New-BuildVM.Tests.ps1 — throw on missing template, throw+cleanup on clone
                          failure, Clone_{JobId}_{timestamp} name format
  Remove-BuildVM.Tests.ps1 — no-op on missing VMX, partial dir cleanup,
                             -WhatIf preservation, full destroy removes dir
  Wait-VMReady.Tests.ps1 — throw on missing vmrun, timeout throw,
                           IP ValidatePattern accept/reject

Run: Invoke-Pester tests\ -Output Detailed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 20:12:19 +02:00