docs(a5): document Python orchestrator in AGENTS.md, ARCHITECTURE.md, README.md
AGENTS.md: new 'Python development' section (venv paths, install/lint/typecheck/test commands, conventions, full PS->Python sub-command mapping, list of PS scripts that stay PowerShell, link to phase closeouts). README.md: expand 'Python orchestrator' with examples for all 10 sub-commands (wait-ready, vm new/remove/cleanup, build run, artifacts collect, monitor disk/runner, report job, job), local validation gates with 80% coverage gate, link to test_agents_errors.py. docs/ARCHITECTURE.md: new 'Python orchestrator (Phase A)' section with package layout tree, VmBackend Protocol contract, load_backend(config) factory, Phase C ESXi extension recipe, transport selection note, and CI validation gates.
This commit is contained in:
@@ -206,3 +206,116 @@ VMs have internet access via NAT — required for pip/nuget/apt during build.
|
||||
- Source code is transferred via zip (Compress-Archive → WinRM copy → Expand-Archive in guest) by default — no PAT is ever present inside the VM; VMs have NAT internet for pip/nuget at build time.
|
||||
- With `-UseGitClone` (§3.3, opt-in) il clone avviene dentro la VM via SSH alias `gitea-ci` (forwardato dall'host); la chiave privata SSH dell'host non viene esposta al guest — vedi BEST-PRACTICES.md per il threat model.
|
||||
- Linux build VMs usano la chiave SSH `F:\CI\keys\ci_linux` (ed25519, no passphrase), archiviata solo sull'host. Il guest non conosce credenziali WinRM.
|
||||
|
||||
---
|
||||
|
||||
## Python orchestrator (Phase A)
|
||||
|
||||
Phase A of [implementation-plan-A-B](../plans/implementation-plan-A-B.md)
|
||||
replaces the PowerShell orchestration scripts with a Python 3.11+ package
|
||||
under `src/ci_orchestrator/`. The PS scripts in `scripts/` are now thin
|
||||
3-line shims that forward to the Python CLI — the diagram above still
|
||||
applies, except that the boxes labelled `*.ps1` are entry-point names,
|
||||
not the implementation.
|
||||
|
||||
### Package layout
|
||||
|
||||
```
|
||||
src/ci_orchestrator/
|
||||
├── __init__.py
|
||||
├── __main__.py # click entry point: `python -m ci_orchestrator`
|
||||
├── config.py # Config dataclass + env/TOML loader (OS-aware paths)
|
||||
├── credentials.py # CredentialStore Protocol + KeyringCredentialStore
|
||||
│
|
||||
├── backends/
|
||||
│ ├── __init__.py # load_backend(config) factory (selector for Phase C)
|
||||
│ ├── protocol.py # VmBackend Protocol + VmHandle dataclass + VmState enum
|
||||
│ ├── workstation.py # WorkstationVmrunBackend (subprocess + vmrun -T ws)
|
||||
│ └── errors.py # BackendError / BackendNotAvailable / BackendOperationFailed
|
||||
│
|
||||
├── transport/
|
||||
│ ├── __init__.py
|
||||
│ ├── winrm.py # WinRM transport via pypsrp (replaces New-PSSession)
|
||||
│ ├── ssh.py # SSH/SFTP transport via paramiko (replaces ssh.exe/scp.exe)
|
||||
│ └── errors.py # TransportConnectError / TransportCommandError
|
||||
│
|
||||
└── commands/ # One module per `click` sub-command group
|
||||
├── wait.py # `wait-ready`
|
||||
├── vm.py # `vm new` / `vm remove` / `vm cleanup`
|
||||
├── build.py # `build run`
|
||||
├── artifacts.py # `artifacts collect`
|
||||
├── monitor.py # `monitor disk` / `monitor runner`
|
||||
├── report.py # `report job`
|
||||
└── job.py # `job` — full end-to-end orchestrator
|
||||
```
|
||||
|
||||
### `VmBackend` Protocol and the `load_backend(config)` factory
|
||||
|
||||
`backends/protocol.py` defines a hypervisor-agnostic Protocol:
|
||||
|
||||
```python
|
||||
class VmBackend(Protocol):
|
||||
def clone_linked(self, template, snapshot, name, destination=None) -> VmHandle: ...
|
||||
def start(self, handle, headless=True) -> None: ...
|
||||
def stop(self, handle, hard=False) -> None: ...
|
||||
def delete(self, handle) -> None: ...
|
||||
def get_ip(self, handle, timeout=0.0) -> str | None: ...
|
||||
def list_snapshots(self, handle) -> list[str]: ...
|
||||
def is_running(self, handle) -> bool: ...
|
||||
```
|
||||
|
||||
Names are deliberately neutral (no `vmrun_*` prefixes) and identifiers are
|
||||
opaque strings (`VmHandle.identifier`): a VMX path on Workstation, a
|
||||
managed object reference on ESXi. `is_running` MUST NOT use guest tools
|
||||
(see `AGENTS.md` error #10).
|
||||
|
||||
Backends are constructed via `backends.load_backend(config)`. Application
|
||||
code (most importantly `commands/job.py`) imports the factory only — never
|
||||
`WorkstationVmrunBackend` directly. The factory reads
|
||||
`config.backend.type` from `config.toml` (default: `"workstation"`).
|
||||
|
||||
### Phase C extension point — adding ESXi without forks
|
||||
|
||||
To add an ESXi backend in a future Phase C:
|
||||
|
||||
1. Add a new module `backends/esxi.py` implementing the `VmBackend`
|
||||
Protocol via `pyVmomi`. `VmHandle.identifier` becomes the managed
|
||||
object reference; `clone_linked` becomes a `CloneVM_Task` call;
|
||||
`start` becomes `PowerOnVM_Task`.
|
||||
2. Register it in `backends/__init__.py` `load_backend(config)` under a
|
||||
new `config.backend.type == "esxi"` branch.
|
||||
3. Add `[backend]` keys to `config.example.toml` (host, user, datacenter,
|
||||
datastore, …). The `Config` dataclass already namespaces backend
|
||||
options under `config.backend`.
|
||||
4. Transport layers (`winrm.py`, `ssh.py`) and the `commands/*` modules
|
||||
need no change — they consume `VmBackend` and `VmHandle` only.
|
||||
|
||||
The single non-trivial constraint is that `commands/vm.py` `vm new`
|
||||
currently builds the destination VMX path under `--clone-base-dir` (a
|
||||
host-local concept). For ESXi this becomes a datastore folder; the
|
||||
parameter rename can be deferred to Phase C as long as the backend
|
||||
ignores the value and uses `config.backend.datastore` instead.
|
||||
|
||||
### Transport selection
|
||||
|
||||
`commands/job.py` auto-detects the guest OS by parsing `guestOS = "..."`
|
||||
in the cloned VMX (`ubuntu-*` → Linux/SSH, anything else → Windows/WinRM).
|
||||
The Linux branch is wired to `transport/ssh.py` (paramiko) and the
|
||||
Windows branch to `transport/winrm.py` (pypsrp). Both transports raise
|
||||
typed errors (`TransportConnectError`, `TransportCommandError`) — never
|
||||
bare exceptions.
|
||||
|
||||
### Validation gates (CI)
|
||||
|
||||
Enforced by `gitea/workflows/lint.yml` job `python`:
|
||||
|
||||
```powershell
|
||||
python -m ruff check src tests\python
|
||||
python -m mypy --strict src
|
||||
python -m pytest tests\python -q --cov=ci_orchestrator --cov-fail-under=80
|
||||
```
|
||||
|
||||
Coverage gate is **80%** as of Phase A5. Regression tests for
|
||||
`AGENTS.md` errors #9–#12 live in
|
||||
[tests/python/test_agents_errors.py](../tests/python/test_agents_errors.py).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user