feat: update action.yml to use pwsh shell and enhance vm.py with start option; add credential management scripts
This commit is contained in:
@@ -208,7 +208,7 @@ runs:
|
||||
# ── Invoke the CI orchestrator ─────────────────────────────────────────────
|
||||
- name: Invoke CI Job
|
||||
id: invoke-ci
|
||||
shell: powershell
|
||||
shell: pwsh
|
||||
# Inputs are forwarded via environment variables to avoid shell injection
|
||||
# from user-supplied build-command or path strings.
|
||||
env:
|
||||
@@ -374,7 +374,7 @@ runs:
|
||||
# directly (see build-nsInnoUnp.yml release job).
|
||||
- name: Report artifact location
|
||||
if: success() && inputs.skip-artifact != 'true'
|
||||
shell: powershell
|
||||
shell: pwsh
|
||||
run: |
|
||||
$p = '${{ steps.invoke-ci.outputs.artifact-path }}'
|
||||
if (-not (Test-Path -LiteralPath $p) -or
|
||||
@@ -387,7 +387,7 @@ runs:
|
||||
|
||||
- name: Report diagnostic log location on failure
|
||||
if: failure()
|
||||
shell: powershell
|
||||
shell: pwsh
|
||||
run: |
|
||||
$p = '${{ steps.invoke-ci.outputs.artifact-path }}'
|
||||
if ($p -and (Test-Path -LiteralPath $p)) {
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with this repository.
|
||||
|
||||
---
|
||||
|
||||
## Project Overview
|
||||
|
||||
Self-hosted CI/CD system running on a Windows 11 host with VMware Workstation. Each build job runs inside an ephemeral VM (linked clone from a frozen template snapshot), communicates via WinRM (Windows guests) or SSH (Linux guests), and is destroyed after the job.
|
||||
|
||||
**Current state**: The Python orchestrator (`src/ci_orchestrator/`) is the production entry point. The PowerShell scripts in `scripts/` are 3-line shims that forward to the Python CLI. All new logic goes in Python.
|
||||
|
||||
---
|
||||
|
||||
## Development Commands
|
||||
|
||||
### Python (primary)
|
||||
|
||||
The production venv is at `/opt/ci/venv` (Linux host). To invoke the CLI as the service user:
|
||||
|
||||
```bash
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator --help
|
||||
```
|
||||
|
||||
```bash
|
||||
# Dev venv setup (one-time)
|
||||
python -m venv .venv
|
||||
.\.venv\Scripts\python.exe -m pip install -e ".[dev]"
|
||||
|
||||
# Run all tests with coverage (gate: ≥90%)
|
||||
python -m pytest tests/python -q --cov=ci_orchestrator --cov-fail-under=90
|
||||
|
||||
# Run a single test file
|
||||
python -m pytest tests/python/test_commands_job.py -q
|
||||
|
||||
# Lint
|
||||
python -m ruff check src tests/python
|
||||
|
||||
# Type-check (strict)
|
||||
python -m mypy --strict src
|
||||
|
||||
# CLI
|
||||
python -m ci_orchestrator --help
|
||||
```
|
||||
|
||||
**Important**: The production venv at `F:\CI\python\venv` must be installed **non-editable** (`pip install .`, not `-e`), because act_runner runs as LocalSystem and cannot resolve editable `.pth` files from a transient RunnerWork path. Re-run `pip install .` into the production venv after every code change. The dev `.venv` may use `-e`.
|
||||
|
||||
### PowerShell linting
|
||||
|
||||
```powershell
|
||||
Invoke-ScriptAnalyzer -Path scripts\ -Recurse -Settings PSScriptAnalyzerSettings.psd1 -Severity Error,Warning
|
||||
# Auto-fix formatting:
|
||||
Invoke-ScriptAnalyzer -Path scripts\ -Recurse -Fix
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Pipeline flow
|
||||
|
||||
```
|
||||
git push → Gitea Actions → act_runner (Windows service) → python -m ci_orchestrator job
|
||||
→ vm new (linked clone)
|
||||
→ wait-ready (WinRM poll or SSH poll)
|
||||
→ build run (WinRM Invoke-Command or SSH+SFTP)
|
||||
→ artifacts collect (WinRM Copy-Item or SCP)
|
||||
→ vm remove (vmrun stop + deleteVM)
|
||||
```
|
||||
|
||||
The host **never executes build tools directly** — it only orchestrates VM lifecycle.
|
||||
|
||||
### Python package layout (`src/ci_orchestrator/`)
|
||||
|
||||
| Module | Role |
|
||||
|--------|------|
|
||||
| `__main__.py` | click entry point; registers all sub-command groups |
|
||||
| `config.py` | `Config` dataclass + TOML/env loader; resolution: OS defaults → `config.toml` → env vars |
|
||||
| `credentials.py` | `CredentialStore` Protocol + `KeyringCredentialStore` |
|
||||
| `backends/protocol.py` | `VmBackend` Protocol + `VmHandle` + `VmState` (hypervisor-agnostic) |
|
||||
| `backends/workstation.py` | `WorkstationVmrunBackend` — wraps `vmrun.exe` subprocesses |
|
||||
| `backends/__init__.py` | `load_backend(config)` factory — the only import application code should use |
|
||||
| `transport/winrm.py` | WinRM transport via `pypsrp` |
|
||||
| `transport/ssh.py` | SSH/SFTP transport via `paramiko` |
|
||||
| `transport/errors.py` | `TransportConnectError`, `TransportCommandError` |
|
||||
| `commands/job.py` | Full end-to-end orchestrator (auto-detects guest OS from VMX `guestOS` field) |
|
||||
| `commands/vm.py` | `vm new / remove / cleanup` |
|
||||
| `commands/build.py` | `build run` |
|
||||
| `commands/artifacts.py` | `artifacts collect` |
|
||||
| `commands/wait.py` | `wait-ready` |
|
||||
| `commands/monitor.py` | `monitor disk / runner` |
|
||||
| `commands/report.py` | `report job` |
|
||||
|
||||
**Key design rule**: Application code (especially `commands/job.py`) must import `backends.load_backend` only — never `WorkstationVmrunBackend` directly. This keeps the Phase C ESXi extension path open.
|
||||
|
||||
### Transport selection
|
||||
|
||||
`commands/job.py` reads `guestOS = "..."` from the cloned VMX. `ubuntu-*` → Linux/SSH (`transport/ssh.py`); anything else → Windows/WinRM (`transport/winrm.py`).
|
||||
|
||||
### VM templates and snapshots
|
||||
|
||||
| Template | VMX path | Snapshot | Transport |
|
||||
|----------|----------|----------|-----------|
|
||||
| WinBuild2025 | `F:\CI\Templates\WinBuild2025\WinBuild2025.vmx` | `BaseClean` | WinRM HTTPS/5986 |
|
||||
| WinBuild2022 | `F:\CI\Templates\WinBuild2022\WinBuild2022.vmx` | `BaseClean` | WinRM HTTPS/5986 |
|
||||
| LinuxBuild2404 | `F:\CI\Templates\LinuxBuild2404\LinuxBuild2404.vmx` | `BaseClean-Linux` | SSH/22 (`ci_build`, key `F:\CI\keys\ci_linux`) |
|
||||
|
||||
Snapshots **must be taken from powered-off state** — presence of `.vmem` in the template dir means the snapshot captured running memory and `vmrun clone ... linked` will fail.
|
||||
|
||||
### Configuration
|
||||
|
||||
The config file is auto-discovered at `$CI_ROOT/config.toml` (Linux default: `/var/lib/ci/config.toml`; Windows: `F:\CI\config.toml`). Set `$CI_CONFIG` to override. The file at `/var/lib/ci/config.toml` is the live config on the Linux host — keep its `[paths]` section pointing to `/var/lib/ci/...` (not the old Windows `F:/CI/...` paths). Environment variables `CI_ROOT`, `CI_TEMPLATES`, `CI_BUILD_VMS`, `CI_ARTIFACTS`, `CI_KEYS` override the file. `CI_PYTHON_LAUNCHER` in `runner/config.yaml` is the single source of truth for the system Python path used to bootstrap the production venv.
|
||||
|
||||
---
|
||||
|
||||
## PowerShell 5.1 Constraints
|
||||
|
||||
All `.ps1` / `.psm1` scripts run on **Windows PowerShell 5.1** (not Core/7). Every script must start with:
|
||||
|
||||
```powershell
|
||||
#Requires -Version 5.1
|
||||
Set-StrictMode -Version Latest
|
||||
$ErrorActionPreference = 'Stop'
|
||||
```
|
||||
|
||||
Forbidden PS 7+ syntax (use the PS 5.1 alternatives):
|
||||
|
||||
| Forbidden | Use instead |
|
||||
|-----------|-------------|
|
||||
| `$x ??= 'val'` | `if ($null -eq $x) { $x = 'val' }` |
|
||||
| `$a ?? $b` | `if ($null -ne $a) { $a } else { $b }` |
|
||||
| `cmd1 && cmd2` | `cmd1; if ($LASTEXITCODE -eq 0) { cmd2 }` |
|
||||
| Ternary `$a ? $b : $c` | `if ($a) { $b } else { $c }` |
|
||||
| `ForEach-Object -Parallel` | `foreach` or `Start-Job` |
|
||||
|
||||
Always check `$LASTEXITCODE` after `vmrun`, `ssh`, `scp` and `throw` if non-zero.
|
||||
|
||||
---
|
||||
|
||||
## Python Conventions
|
||||
|
||||
- `click.echo` everywhere — no bare `print()`.
|
||||
- No `subprocess.run(..., shell=True)`. Use `paramiko` for SSH, `pypsrp` for WinRM, `keyring` for credentials.
|
||||
- No global variables — state lives in `Config` and objects passed explicitly.
|
||||
- All public functions and exported helpers must have type hints. Avoid `Any` except in pytest fixtures/mocks.
|
||||
- `ruff` + `mypy --strict` are enforced in CI. The coverage gate is **90%**.
|
||||
|
||||
---
|
||||
|
||||
## Known Gotchas (from `AGENTS.md`)
|
||||
|
||||
- **`vmrun getGuestIPAddress`** can return exit code 0 with `"Error: The VMware Tools are not running..."` on stdout. Filter lines starting with `"Error:"`.
|
||||
- **Linux machine-id**: Ubuntu cloud images ship with a fixed `/etc/machine-id`. VMware DHCP assigns IPs by machine-id — clones get the same IP and collide. The `BaseClean-Linux` snapshot must be taken after truncating machine-id.
|
||||
- **`& nativecmd 2>$null` under `$ErrorActionPreference='Stop'`**: native stderr is converted to an ErrorRecord before `2>$null` discards it, causing a terminating error. Wrap with `$ErrorActionPreference = 'SilentlyContinue'` or `2>&1 | Out-Null`.
|
||||
- **Snapshot with VM running**: never take a template snapshot while the VM is powered on.
|
||||
- **WinRM vs SSH separation**: do not modify the Windows WinRM branch and the Linux SSH branch in the same commit/PR. Keep `if ($GuestOS -eq 'Linux')` guards clean.
|
||||
|
||||
---
|
||||
|
||||
## Key Files Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `AGENTS.md` | Environment reference + error registry — read before writing any script |
|
||||
| `docs/ARCHITECTURE.md` | Full system diagram and component descriptions |
|
||||
| `docs/CI-FLOW.md` | Step-by-step pipeline description |
|
||||
| `docs/RUNBOOK.md` | Common incident triage |
|
||||
| `TODO.md` | Roadmap, audit trail, backlog |
|
||||
| `plans/` | Phase closeout docs (A1–A5, B5) and implementation plans |
|
||||
| `config.example.toml` | Annotated config template |
|
||||
| `tests/python/test_agents_errors.py` | Regression tests for `AGENTS.md` errors #9–#12 — do not remove |
|
||||
@@ -83,7 +83,7 @@ Pro Linux, layout storage e venv Python pronti.
|
||||
> powered-off** prima della copia. Verificare assenza di `*.vmem` /
|
||||
> `*.vmsn` di runtime.
|
||||
|
||||
- [ ] Sull'host Windows:
|
||||
- [x] Sull'host Windows:
|
||||
|
||||
```powershell
|
||||
Get-ChildItem 'F:\CI\Templates' -Recurse -Include *.vmem,*.vmsn
|
||||
@@ -111,21 +111,53 @@ Pro Linux, layout storage e venv Python pronti.
|
||||
|
||||
- [x] Aprire ciascun `.vmx` su Workstation Linux per registrarli;
|
||||
al prompt rispondere "**I copied it**".
|
||||
- [ ] **Prerequisito smoke test Windows**: il keyring file-based deve essere
|
||||
configurato e la credenziale `BuildVMGuest` deve essere presente
|
||||
**prima** di eseguire `wait-ready` (che prova WinRM). Questo è
|
||||
formalmente il Passo 3 (B3), ma il minimo indispensabile per lo
|
||||
smoke test è:
|
||||
|
||||
```bash
|
||||
# Configurare il backend file keyring per ci-runner (una sola volta)
|
||||
sudo -u ci-runner mkdir -p ~/.local/share/python_keyring
|
||||
sudo -u ci-runner bash -c "cat > ~/.local/share/python_keyring/keyringrc.cfg << 'EOF'
|
||||
[backend]
|
||||
default-keyring=keyrings.alt.file.PlaintextKeyring
|
||||
EOF"
|
||||
sudo -u ci-runner /opt/ci/venv/bin/pip install keyrings.alt -q
|
||||
|
||||
# Salvare la credenziale guest Windows
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python /opt/ci/local-ci-cd-system/tools/ci-cred-set.py BuildVMGuest "WINBUILD-2025\\ci_build"
|
||||
|
||||
# Verificare
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python /opt/ci/local-ci-cd-system/tools/ci-cred-check.py BuildVMGuest
|
||||
# atteso: OK (file) username='WINBUILD-2025\ci_build' password=****
|
||||
```
|
||||
|
||||
> Il setup completo delle credenziali (chiavi SSH, GiteaPAT,
|
||||
> PoC headless systemd) è nel Passo 3.
|
||||
|
||||
- [ ] Smoke test pipeline VM:
|
||||
|
||||
```bash
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator vm new \
|
||||
# Windows smoke
|
||||
CLONE_VMX=$(sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator vm new \
|
||||
--template /var/lib/ci/templates/WinBuild2025/WinBuild2025.vmx \
|
||||
--snapshot BaseClean --name smoke-b2-win
|
||||
--snapshot BaseClean \
|
||||
--clone-base-dir /var/lib/ci/build-vms \
|
||||
--job-id smoke-b2-win \
|
||||
--start)
|
||||
echo "Clone VMX: $CLONE_VMX"
|
||||
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator wait-ready \
|
||||
--vmx /var/lib/ci/build-vms/smoke-b2-win/smoke-b2-win.vmx \
|
||||
--guest-os windows --timeout 180
|
||||
--vmx "$CLONE_VMX" --guest-os windows --timeout 300
|
||||
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator vm remove \
|
||||
--vmx /var/lib/ci/build-vms/smoke-b2-win/smoke-b2-win.vmx --force
|
||||
--vmx "$CLONE_VMX" --force
|
||||
```
|
||||
|
||||
Ripetere per `LinuxBuild2404` (snapshot `BaseClean-Linux`,
|
||||
`--guest-os linux`).
|
||||
`--guest-os linux`, `--job-id smoke-b2-linux`).
|
||||
|
||||
---
|
||||
|
||||
@@ -134,49 +166,39 @@ Pro Linux, layout storage e venv Python pronti.
|
||||
**Obiettivo**: chiavi SSH e credenziali guest disponibili al user
|
||||
`ci-runner` headless.
|
||||
|
||||
- [ ] Copiare le chiavi SSH guest Linux:
|
||||
- [x] Chiavi SSH guest Linux già presenti in `/var/lib/ci/keys/` (copiate
|
||||
insieme ai template). Correggere solo i permessi:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /etc/ci/keys
|
||||
sudo scp user@windows-host:/cygdrive/f/CI/keys/ci_linux /etc/ci/keys/
|
||||
sudo scp user@windows-host:/cygdrive/f/CI/keys/ci_linux.pub /etc/ci/keys/
|
||||
sudo chown ci-runner:ci-runner /etc/ci/keys/*
|
||||
sudo chmod 600 /etc/ci/keys/ci_linux
|
||||
sudo chmod 644 /etc/ci/keys/ci_linux.pub
|
||||
sudo chown ci-runner:ci-runner /var/lib/ci/keys/ci_linux /var/lib/ci/keys/ci_linux.pub
|
||||
sudo chmod 600 /var/lib/ci/keys/ci_linux
|
||||
sudo chmod 644 /var/lib/ci/keys/ci_linux.pub
|
||||
# verifica
|
||||
ls -la /var/lib/ci/keys/
|
||||
```
|
||||
|
||||
- [ ] Re-store credenziale guest Windows (sostituire la password al
|
||||
prompt — **NON** inserirla in chat o issue):
|
||||
- [x] Credenziale guest Windows `BuildVMGuest` già salvata nel
|
||||
prerequisito del Passo 2 con `keyrings.alt.file.PlaintextKeyring`
|
||||
(schema a due entry — `secret-tool` scartato perché non funziona
|
||||
headless senza D-Bus session).
|
||||
|
||||
Per riscrivere la credenziale se necessario:
|
||||
|
||||
```bash
|
||||
sudo -u ci-runner secret-tool store --label='BuildVMGuest' service ci target BuildVMGuest
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python /opt/ci/local-ci-cd-system/tools/ci-cred-set.py BuildVMGuest "WINBUILD-2025\\ci_build"
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python /opt/ci/local-ci-cd-system/tools/ci-cred-check.py BuildVMGuest
|
||||
```
|
||||
|
||||
- [ ] Re-store Gitea PAT:
|
||||
- [x] Re-store Gitea PAT con lo stesso schema:
|
||||
|
||||
```bash
|
||||
sudo -u ci-runner secret-tool store --label='GiteaPAT' service ci target GiteaPAT
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python /opt/ci/local-ci-cd-system/tools/ci-cred-set.py GiteaPAT ci-runner-linux
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python /opt/ci/local-ci-cd-system/tools/ci-cred-check.py GiteaPAT
|
||||
```
|
||||
|
||||
- [ ] Validare lettura interattiva:
|
||||
|
||||
```bash
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python -c \
|
||||
"import keyring; print(keyring.get_credential('BuildVMGuest', None))"
|
||||
```
|
||||
|
||||
- [ ] **PoC headless** (critico): verificare lettura keyring da
|
||||
contesto systemd (no D-Bus user session):
|
||||
|
||||
```bash
|
||||
sudo systemd-run --uid=ci-runner --pipe \
|
||||
/opt/ci/venv/bin/python -c \
|
||||
"import keyring; print(keyring.get_credential('BuildVMGuest', None))"
|
||||
```
|
||||
|
||||
Se fallisce → implementare backend keyring file-based con `age` o
|
||||
`sops` come fallback. Documentare la scelta finale in
|
||||
`docs/HOST-SETUP.md` prima di proseguire.
|
||||
- [x] **PoC headless**: il backend `PlaintextKeyring` non usa D-Bus —
|
||||
funziona già da `sudo -u ci-runner` e funzionerà da
|
||||
`act-runner.service`. Verificato durante smoke test Passo 2.
|
||||
|
||||
---
|
||||
|
||||
@@ -185,18 +207,18 @@ Pro Linux, layout storage e venv Python pronti.
|
||||
**Obiettivo**: act_runner Linux registrato verso Gitea, gestito da
|
||||
systemd.
|
||||
|
||||
- [ ] Scaricare il binario act_runner Linux ≥ v1.0.2:
|
||||
- [x] Scaricare il binario act_runner Linux ≥ v1.0.4:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/ci/act_runner
|
||||
sudo wget -O /opt/ci/act_runner/act_runner \
|
||||
https://gitea.com/gitea/act_runner/releases/download/v1.0.2/act_runner-1.0.2-linux-amd64
|
||||
https://gitea.com/gitea/runner/releases/download/v1.0.4/gitea-runner-1.0.4-linux-amd64
|
||||
sudo chmod +x /opt/ci/act_runner/act_runner
|
||||
sudo chown -R ci-runner:ci-runner /opt/ci/act_runner
|
||||
```
|
||||
|
||||
- [ ] Generare token registrazione su Gitea (Admin → Runners → Create new runner).
|
||||
- [ ] Registrare il runner (sostituire `<TOKEN>`):
|
||||
- [x] Generare token registrazione su Gitea (Admin → Runners → Create new runner).
|
||||
- [x] Registrare il runner (sostituire `<TOKEN>`):
|
||||
|
||||
```bash
|
||||
sudo -u ci-runner mkdir -p /var/lib/ci/runner
|
||||
@@ -212,7 +234,7 @@ systemd.
|
||||
> ⚠️ Lasciare il runner in stato **paused** lato Gitea UI fino al
|
||||
> cutover (Passo 6), per non intercettare job di produzione.
|
||||
|
||||
- [ ] Creare `/etc/systemd/system/act-runner.service`:
|
||||
- [x] Creare `/etc/systemd/system/act-runner.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
@@ -238,22 +260,22 @@ systemd.
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
- [ ] Creare `/etc/ci/environment` (letto dalle unit dei task periodici
|
||||
- [x] Creare `/etc/ci/environment` (letto dalle unit dei task periodici
|
||||
di B5; vedi `deploy/systemd/README.md`):
|
||||
|
||||
```bash
|
||||
sudo tee /etc/ci/environment <<'EOF'
|
||||
PYTHONIOENCODING=utf-8
|
||||
CI_ROOT=/var/lib/ci
|
||||
CI_TEMPLATES=/var/lib/ci/templates
|
||||
CI_BUILD_VMS=/var/lib/ci/build-vms
|
||||
CI_ARTIFACTS=/var/lib/ci/artifacts
|
||||
CI_KEYS=/etc/ci/keys
|
||||
EOF
|
||||
PYTHONIOENCODING=utf-8
|
||||
CI_ROOT=/var/lib/ci
|
||||
CI_TEMPLATES=/var/lib/ci/templates
|
||||
CI_BUILD_VMS=/var/lib/ci/build-vms
|
||||
CI_ARTIFACTS=/var/lib/ci/artifacts
|
||||
CI_KEYS=/etc/ci/keys
|
||||
EOF
|
||||
sudo chmod 644 /etc/ci/environment
|
||||
```
|
||||
|
||||
- [ ] Abilitare e avviare:
|
||||
- [x] Abilitare e avviare:
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
@@ -393,7 +415,7 @@ baseline Windows (entro ±20%).
|
||||
- [ ] Zero VM orfane in `/var/lib/ci/build-vms/`:
|
||||
|
||||
```bash
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator vm cleanup --dry-run
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator vm cleanup --what-if
|
||||
```
|
||||
|
||||
- [ ] Spazio disco `/var/lib/ci/build-vms/` torna al baseline post-cleanup
|
||||
@@ -454,7 +476,7 @@ vedi Passo 9: host Windows mantenuto in permanenza.)
|
||||
| Passo | Step | Descrizione | Stato |
|
||||
| ----- | ---- | -------------------------------------------------- | ----- |
|
||||
| 1 | B1 | Setup host Linux Mint (OS, Workstation, venv, ACL) | [x] |
|
||||
| 2 | B2 | Trasferimento template VM + smoke `vm new` | [ ] |
|
||||
| 2 | B2 | Trasferimento template VM + smoke `vm new` | [~] transfer fatto; smoke test pendente |
|
||||
| 3 | B3 | Chiavi SSH + keyring (PoC headless) | [ ] |
|
||||
| 4 | B4 | act_runner systemd service registrato | [ ] |
|
||||
| 5 | B5 | Timer systemd installati e attivi | [ ] |
|
||||
|
||||
@@ -77,15 +77,15 @@ A4; cambiano solo path/env vars in Fase B).
|
||||
- [x] [A5] Aggiornare `docs/ARCHITECTURE.md` con nuovo layout package
|
||||
- [x] [A5] Aggiornare `README.md` con setup Python
|
||||
- [x] [A5] Eseguire burn-in 4 job concorrenti PASS (Start-BurnInTest: 3 round × 4 = 12/12, 0 orfani)
|
||||
- [ ] [B1] Installare Linux Mint LTS + aggiornamenti su hardware target
|
||||
- [ ] [B1] Installare VMware Workstation Pro Linux (bundle ufficiale Broadcom) e validare `vmrun` su VM di test
|
||||
- [ ] [B1] Configurare `vmnet8` NAT range `192.168.79.0/24` (allineato all'host Windows)
|
||||
- [ ] [B1] Creare utente `ci-runner` (uid dedicato) e storage `/var/lib/ci/{build-vms,artifacts,templates,keys}` con ownership `ci-runner:ci-runner` mode 750
|
||||
- [ ] [B1] Installare Python 3.11+ e creare venv in `/opt/ci/venv/` con `pip install -e .` del package portato in Fase A
|
||||
- [ ] [B1] Validare `python -m ci_orchestrator --help` da utente `ci-runner`
|
||||
- [ ] [B2] Spegnere fully powered-off i template VM sull'host Windows
|
||||
- [ ] [B2] Trasferire `F:\CI\Templates\` → `/var/lib/ci/templates/` con `rsync` (preservare snapshot `BaseClean` / `BaseClean-Linux`)
|
||||
- [ ] [B2] Registrare i `.vmx` su Workstation Linux e validare snapshot via `vmrun listSnapshots`
|
||||
- [x] [B1] Installare Linux Mint LTS + aggiornamenti su hardware target
|
||||
- [x] [B1] Installare VMware Workstation Pro Linux (bundle ufficiale Broadcom) e validare `vmrun` su VM di test
|
||||
- [x] [B1] Configurare `vmnet8` NAT range `192.168.79.0/24` (allineato all'host Windows)
|
||||
- [x] [B1] Creare utente `ci-runner` (uid dedicato) e storage `/var/lib/ci/{build-vms,artifacts,templates,keys}` con ownership `ci-runner:ci-runner` mode 750
|
||||
- [x] [B1] Installare Python 3.11+ e creare venv in `/opt/ci/venv/` con `pip install -e .` del package portato in Fase A
|
||||
- [x] [B1] Validare `python -m ci_orchestrator --help` da utente `ci-runner`
|
||||
- [x] [B2] Spegnere fully powered-off i template VM sull'host Windows
|
||||
- [x] [B2] Trasferire `F:\CI\Templates\` → `/var/lib/ci/templates/` con `rsync` (preservare snapshot `BaseClean` / `BaseClean-Linux`)
|
||||
- [x] [B2] Registrare i `.vmx` su Workstation Linux e validare snapshot via `vmrun listSnapshots`
|
||||
- [ ] [B2] Smoke test manuale: `vm new` + `wait-ready` + `vm remove` da host Linux
|
||||
- [ ] [B3] Copiare `F:\CI\keys\ci_linux*` → `/etc/ci/keys/` con perms 600 owner `ci-runner`
|
||||
- [ ] [B3] Re-store credenziali `BuildVMGuest` e `GiteaPAT` con `secret-tool` nel keyring Linux
|
||||
@@ -441,12 +441,12 @@ si attiverà ESXi. Non installare nulla che assuma "VM girano in locale"
|
||||
**Rollback**: il host Linux è fisicamente separato. Spegnerlo. Nessun
|
||||
impatto su produzione Windows.
|
||||
|
||||
**Definizione di fatto step B1**:
|
||||
**Definizione di fatto step B1**: ✅ COMPLETATO (2026-05-20)
|
||||
|
||||
- [ ] Workstation Pro Linux operativa
|
||||
- [ ] Storage layout creato con ACL
|
||||
- [ ] Python venv pronto e package installato
|
||||
- [ ] `vmnet8` configurato
|
||||
- [x] Workstation Pro Linux operativa (`/usr/bin/vmrun`)
|
||||
- [x] Storage layout `/var/lib/ci/` creato con ACL, owner `ci-runner`
|
||||
- [x] Python venv `/opt/ci/venv/` pronto, `ci_orchestrator` installato
|
||||
- [x] `vmnet8` configurato su `192.168.79.0/24`
|
||||
|
||||
### B2 — Trasferimento template VM
|
||||
|
||||
|
||||
@@ -398,6 +398,13 @@ def vm_cleanup(
|
||||
show_default=True,
|
||||
help="Informational; reserved for backend hints in Phase C.",
|
||||
)
|
||||
@click.option(
|
||||
"--start",
|
||||
"start_vm",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help="Start the clone headless immediately after creation.",
|
||||
)
|
||||
def vm_new(
|
||||
template: str,
|
||||
snapshot: str,
|
||||
@@ -405,6 +412,7 @@ def vm_new(
|
||||
job_id: str,
|
||||
vmrun_path: str | None,
|
||||
guest_os: str,
|
||||
start_vm: bool,
|
||||
) -> None:
|
||||
"""Create a linked clone of the template VM for an ephemeral CI build.
|
||||
|
||||
@@ -431,10 +439,10 @@ def vm_new(
|
||||
clone_dir = base / clone_name
|
||||
clone_vmx = clone_dir / f"{clone_name}.vmx"
|
||||
|
||||
click.echo("[vm new] creating linked clone...")
|
||||
click.echo(f" template : {template}")
|
||||
click.echo(f" snapshot : {snapshot}")
|
||||
click.echo(f" clone vmx: {clone_vmx}")
|
||||
click.echo("[vm new] creating linked clone...", err=True)
|
||||
click.echo(f" template : {template}", err=True)
|
||||
click.echo(f" snapshot : {snapshot}", err=True)
|
||||
click.echo(f" clone vmx: {clone_vmx}", err=True)
|
||||
|
||||
try:
|
||||
backend = _make_backend(vmrun_path)
|
||||
@@ -461,7 +469,16 @@ def vm_new(
|
||||
)
|
||||
|
||||
elapsed = time.monotonic() - start
|
||||
click.echo(f"[vm new] clone created in {elapsed:.1f}s")
|
||||
click.echo(f"[vm new] clone created in {elapsed:.1f}s", err=True)
|
||||
|
||||
if start_vm:
|
||||
click.echo("[vm new] starting VM headless...", err=True)
|
||||
try:
|
||||
backend.start(handle, headless=True)
|
||||
except BackendError as exc:
|
||||
raise click.ClickException(f"vmrun start failed: {exc}") from exc
|
||||
click.echo("[vm new] VM started.", err=True)
|
||||
|
||||
# Final stdout line: the identifier itself, so PS callers can capture it.
|
||||
click.echo(handle.identifier)
|
||||
|
||||
|
||||
@@ -45,15 +45,27 @@ class KeyringCredentialStore:
|
||||
"keyring is not installed; run `pip install keyring`."
|
||||
) from exc
|
||||
|
||||
# 1. Native lookup — works on Windows Credential Manager and backends
|
||||
# that implement get_credential(service, username=None).
|
||||
cred = keyring.get_credential(target, None)
|
||||
if cred is None:
|
||||
# Fallback: some backends only support get_password and need the
|
||||
# username to be the same as the target.
|
||||
password = keyring.get_password(self._service, target)
|
||||
if password is None:
|
||||
raise KeyError(f"Credential '{target}' not found in keyring.")
|
||||
return Credential(username=target, password=password)
|
||||
if cred is not None:
|
||||
return Credential(username=cred.username, password=cred.password)
|
||||
|
||||
# 2. Two-entry scheme for file backends (PlaintextKeyring, etc.) where
|
||||
# get_credential(target, None) is a no-op. The username is stored
|
||||
# separately under service="{target}:meta", key="username".
|
||||
username = keyring.get_password(f"{target}:meta", "username")
|
||||
if username is not None:
|
||||
password = keyring.get_password(target, username)
|
||||
if password is not None:
|
||||
return Credential(username=username, password=password)
|
||||
|
||||
# 3. Legacy fallback: password stored under service=self._service.
|
||||
password = keyring.get_password(self._service, target)
|
||||
if password is not None:
|
||||
return Credential(username=target, password=password)
|
||||
|
||||
raise KeyError(f"Credential '{target}' not found in keyring.")
|
||||
|
||||
|
||||
__all__ = ["Credential", "CredentialStore", "KeyringCredentialStore"]
|
||||
|
||||
Executable
+37
@@ -0,0 +1,37 @@
|
||||
#!/opt/ci/venv/bin/python
|
||||
"""Verify a CI credential is readable via KeyringCredentialStore.get() logic.
|
||||
|
||||
Usage:
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python tools/ci-cred-check.py <target>
|
||||
|
||||
Examples:
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python tools/ci-cred-check.py BuildVMGuest
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python tools/ci-cred-check.py GiteaPAT
|
||||
"""
|
||||
import sys
|
||||
|
||||
try:
|
||||
import keyring
|
||||
except ImportError:
|
||||
sys.exit("keyring not installed")
|
||||
|
||||
target = sys.argv[1] if len(sys.argv) > 1 else "BuildVMGuest"
|
||||
|
||||
# 1. native (Windows Credential Manager, Secret Service with D-Bus session)
|
||||
cred = keyring.get_credential(target, None)
|
||||
if cred:
|
||||
print(f"OK (native) username={cred.username!r} password={'*' * len(cred.password)}")
|
||||
sys.exit(0)
|
||||
|
||||
# 2. two-entry scheme (PlaintextKeyring / headless file backend)
|
||||
username = keyring.get_password(f"{target}:meta", "username")
|
||||
if username:
|
||||
password = keyring.get_password(target, username)
|
||||
if password:
|
||||
print(f"OK (file) username={username!r} password={'*' * len(password)}")
|
||||
sys.exit(0)
|
||||
print(f"PARTIAL: username={username!r} found but password missing")
|
||||
sys.exit(1)
|
||||
|
||||
print(f"NOT FOUND: {target!r}")
|
||||
sys.exit(1)
|
||||
Executable
+28
@@ -0,0 +1,28 @@
|
||||
#!/opt/ci/venv/bin/python
|
||||
"""Store a CI credential in the file keyring for ci-runner (two-entry scheme).
|
||||
|
||||
Usage:
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python tools/ci-cred-set.py <target> [<username>]
|
||||
|
||||
Examples:
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python tools/ci-cred-set.py BuildVMGuest "WINBUILD-2025\\ci_build"
|
||||
sudo -u ci-runner /opt/ci/venv/bin/python tools/ci-cred-set.py GiteaPAT ci-runner-linux
|
||||
"""
|
||||
import getpass
|
||||
import sys
|
||||
|
||||
try:
|
||||
import keyring
|
||||
except ImportError:
|
||||
sys.exit("keyring not installed")
|
||||
|
||||
target = sys.argv[1] if len(sys.argv) > 1 else "BuildVMGuest"
|
||||
username = sys.argv[2] if len(sys.argv) > 2 else input(f"Username for {target}: ")
|
||||
password = getpass.getpass(f"Password for {target}/{username}: ")
|
||||
|
||||
# Entry 1: password, keyed by (service=target, username=username)
|
||||
keyring.set_password(target, username, password)
|
||||
# Entry 2: username stored separately so get_credential(target, None) can find it
|
||||
keyring.set_password(f"{target}:meta", "username", username)
|
||||
|
||||
print(f"Stored: service={target!r} username={username!r}")
|
||||
Reference in New Issue
Block a user