feat(web): service management panel over a sudo allow-list (stage D)

Adds operations/services.py: a fixed ManagedService allow-list (act-runner, act-runner-rust, vmware-host, ci-web) with find_service/service_states/control_service. control_service builds an argument list only (never shell=True), validates the action against ALLOWED_ACTIONS (ValueError->400), refuses stop on restart-only services (ci-web, allow_stop=False) without building a command, and defers ci-web's own restart via 'sudo -n systemd-run --on-active=2 systemctl restart ci-web.service' so the HTTP response returns first.

Route POST /services/{service_id}/{action} in web/routes.py mirrors vms_remove: require_user + require_csrf + two-step confirm gate + one audit entry per stage; unknown id / bad action / disallowed-stop short-circuit to 400 before any subprocess or audit; the vmware-host confirm step warns with the active-VM count. New /services read partial with state-aware Start/Stop/Restart buttons, polled every 15s.

deploy/sudoers.d/ci-web grants ci-runner exact-command NOPASSWD for exactly these nine systemctl lines plus the systemd-run deferral — no wildcards, no bare systemctl, no ci-web stop; setup-web-linux.sh installs it mode 0440 and validates with 'visudo -cf', rolling back on failure. Config gains [web].managed_services override; docs/WEB-UI.md documents the panel + security model.

Gates: 1044 tests pass (46 new security/unit tests), coverage 94.20%, ruff + mypy --strict clean. Not yet deployed: sudoers install + prod-venv reinstall + ci-web restart are a human privilege-boundary step.
This commit is contained in:
2026-07-24 12:11:36 +02:00
parent 9d7bbee35d
commit e56eae95c7
16 changed files with 1828 additions and 20 deletions
+22
View File
@@ -107,3 +107,25 @@ retention_days = 30
# redirect_url = "http://192.168.1.50:8800/auth/callback"
# client_secret_cred = "WebOidcClientSecret" # keyring KEY NAME, not the secret
# scopes = ["openid", "profile"]
#
# Optional: override the default systemd-unit allow-list the dashboard's
# service-management panel (Stage D) is allowed to start/stop/restart. Leave
# unset to use the built-in default (act-runner, act-runner-rust,
# vmware-host, ci-web -- see operations/services.py::MANAGED_SERVICES).
# A malformed row (missing/wrong-typed key) is skipped, never crashes
# startup. Every command reaching sudo is an exact-match systemctl/systemd-run
# invocation approved in deploy/sudoers.d/ci-web -- adding a unit here without
# also adding its NOPASSWD entries there just means every action for it fails
# (sudo prompts for a password that is never supplied, exit non-zero).
#
# [[web.managed_services]]
# id = "act-runner"
# unit = "act-runner.service"
# label = "Runner"
# allow_stop = true
#
# [[web.managed_services]]
# id = "ci-web"
# unit = "ci-web.service"
# label = "Web UI"
# allow_stop = false # restart-only -- stopping it would lock the operator out
+42
View File
@@ -0,0 +1,42 @@
# sudoers.d/ci-web — NOPASSWD grant for the ci-web dashboard's service panel (Stage D).
#
# Installed to /etc/sudoers.d/ci-web, mode 0440, owner root:root, by
# setup-web-linux.sh (which validates it with `visudo -cf` before/after
# installing and rolls back on any failure -- see that script's step and
# docs/WEB-UI.md's "Service management" section).
#
# SECURITY MODEL:
# * Exact-command NOPASSWD only -- every line below is a literal, fully
# qualified command with a literal unit name. No wildcards (`*`), no
# bare `systemctl` entry, no `ALL` in the command position. This is the
# entire reason ci-web (running as ci-runner, an unprivileged service
# account) can safely be granted passwordless sudo for exactly these
# nine command lines and nothing else -- ci-runner can never pass an
# arbitrary unit name or subcommand through sudo, even if the web app
# itself were compromised (defense in depth alongside the operations/
# services.py allow-list, which is the primary gate).
# * ci-web.service deliberately has NO `stop` entry here -- matching
# ManagedService("ci-web", ..., allow_stop=False) in
# operations/services.py. Stopping the service that renders this very
# sudoers-governed panel would lock the operator out with no way to
# start it again from the UI.
# * ci-web.service's "restart" is instead a *deferred* restart via
# `systemd-run --on-active=2 systemctl restart ci-web.service` -- a
# direct `systemctl restart ci-web.service` run from inside ci-web's
# own worker process would kill the HTTP response before it could be
# sent. The systemd-run invocation itself returns almost immediately
# (it only *schedules* the restart 2s out), so the confirming request
# completes normally.
#
# Before installing on a new host, verify the absolute paths below still
# match this host's actual binaries:
# command -v systemctl systemd-run
# and edit both this file and Cmnd_Alias below if either is not under
# /usr/bin (e.g. a distro that ships them under /bin or /usr/local/bin).
# Confirmed /usr/bin/systemctl and /usr/bin/systemd-run on this host.
Cmnd_Alias CI_WEB_SVC = \
/usr/bin/systemctl start act-runner.service, /usr/bin/systemctl stop act-runner.service, /usr/bin/systemctl restart act-runner.service, \
/usr/bin/systemctl start act-runner-rust.service, /usr/bin/systemctl stop act-runner-rust.service, /usr/bin/systemctl restart act-runner-rust.service, \
/usr/bin/systemctl start vmware-host.service, /usr/bin/systemctl stop vmware-host.service, /usr/bin/systemctl restart vmware-host.service, \
/usr/bin/systemd-run --on-active=2 systemctl restart ci-web.service
ci-runner ALL=(root) NOPASSWD: CI_WEB_SVC
+87 -1
View File
@@ -186,7 +186,93 @@ see `plans/archived/2026-05-23/B3-closeout.md`):
---
## 7. Troubleshooting
## 7. Service management (Stage D)
The dashboard's **Services** panel lets an authenticated web user start, stop,
or restart a fixed allow-list of systemd units on the host — the same
confirm-gated, CSRF-protected, audited mutating-action shape as VM
remove/cleanup, cache purge, and template backup (§5), applied to `sudo
systemctl`.
**Allow-list (default, `operations/services.py::MANAGED_SERVICES`):**
| Slug | Unit | Stop allowed? |
| --- | --- | --- |
| `act-runner` | `act-runner.service` | yes |
| `act-runner-rust` | `act-runner-rust.service` | yes |
| `vmware-host` | `vmware-host.service` | yes |
| `ci-web` | `ci-web.service` | **no** — restart-only |
Override it with `[web].managed_services` in `config.toml` (see
`config.example.toml`'s commented example) — a list of `{id, unit, label,
allow_stop}` tables. A malformed row is skipped, never crashes startup.
### Security model
- **Allow-list by slug, not by unit name.** The URL is
`POST /services/{service_id}/{action}` where `service_id` is one of the
slugs above (`find_service()`), never a raw systemd unit name — there is no
code path that builds a controllable unit from unvalidated request input.
- **Exact-command sudoers, no wildcards.** `deploy/sudoers.d/ci-web` grants
`ci-runner` passwordless `sudo` for exactly nine literal command lines (one
per unit × action, plus the `ci-web` restart special case) — no bare
`systemctl`, no `*`, no `ALL` in the command position. Even if the web app
itself were compromised, `ci-runner`'s sudo grant cannot be used to run any
command beyond those nine lines.
- **Arg list, never a shell.** `operations/services.py::control_service`
builds `["sudo", "-n", "systemctl", action, unit]` (or the `systemd-run`
variant below) as a Python list handed straight to `subprocess.run` —
`shell=True` is never used anywhere in this path.
- **`ci-web` is restart-only.** `allow_stop=False` — a `stop` request against
it is refused with a 400 *before* any subprocess or audit entry, both at
the route (`web/routes.py::services_control`) and defensively again inside
`control_service` itself. Rationale: stopping the service that renders this
very panel would lock the operator out with no way to start it again from
the UI.
- **`ci-web` restart is deferred.** Restarting `ci-web.service` from inside
its own worker would kill the HTTP response before it could be sent, so
that one case runs `sudo -n systemd-run --on-active=2 systemctl restart
ci-web.service` instead of a direct `systemctl restart` — the
`systemd-run` call itself returns almost immediately (it only schedules
the restart ~2s out), so the confirming request completes normally and the
dashboard briefly drops a couple of seconds later.
- **`vmware-host` stop/restart interrupts running jobs.** The confirm step
shows the current count of active VMs ("N active VM(s) — this interrupts
running jobs") when stopping or restarting `vmware-host.service`, computed
from the same VM backend the Live VMs panel uses — best-effort, degrades
to no warning if the backend is unavailable, never blocks the action.
- **Every action is audited exactly once per stage** (`service.start` /
`service.stop` / `service.restart`, `target` = the unit name), same as
every other mutating route — visible at `/audit`.
### Sudoers install
`setup-web-linux.sh` installs `deploy/sudoers.d/ci-web` to
`/etc/sudoers.d/ci-web` (mode `0440`, owner `root:root`) as step 6/7,
validating it with `visudo -cf` and **removing it again if validation
fails** — a broken `sudoers.d` fragment can lock out `sudo` entirely, so this
step never leaves an unvalidated file in place. Before installing on a new
host, confirm `systemctl`/`systemd-run` are actually at `/usr/bin` on that
host (`command -v systemctl systemd-run`) — edit both
`deploy/sudoers.d/ci-web` and the installed copy if not.
To install just the sudoers grant (e.g. after editing
`deploy/sudoers.d/ci-web`) without touching anything else:
```bash
sudo bash setup-web-linux.sh --skip-pip --skip-secrets
```
### Deploy note
After merging a Stage D change: re-run `pip install .` into the production
venv (per `CLAUDE.md`), re-run `setup-web-linux.sh` (installs/updates the
sudoers grant and restarts `ci-web.service`), then confirm the sudoers file
on the live host with `sudo visudo -c` before relying on the panel.
---
## 8. Troubleshooting
**Login loop (bounces back to `/login` after authenticating with Gitea)**
: The registered Redirect URI in Gitea does not exactly match
+10 -10
View File
@@ -1,6 +1,6 @@
# Web UI Redesign — Implementation Plan (Phase 1)
**Status:** Stages A, B, C complete + gates approved (989 tests pass, cov 94.12%, ruff+mypy clean). Stage D (service management, SECURITY-CRITICAL) not started. Uncommitted.
**Status:** Stages A–D COMPLETE + all gates approved (1044 tests pass, cov 94.20%, ruff+mypy clean). Committed on branch `feat/web-ui-redesign`. **Remaining: human deploy of Stage D** — `pip install .` into prod venv, install `deploy/sudoers.d/ci-web` (`visudo -c`), restart `ci-web.service` (privilege-boundary change, not auto-applied).
**Owner / executor:** Sonnet (implementation) under Opus supervision (review gate after every stage).
**Scope:** Presentation redesign + new read-only panels + service management. **No** time-series store, **no** light theme, **no** rebuild of Gitea job/log views.
**Target files root:** `src/ci_orchestrator/web/`, `src/ci_orchestrator/operations/`.
@@ -46,15 +46,15 @@ Legend: `[ ]` todo · `[~]` in progress · `[x]` done · `[!]` blocked
- [x] **C-GATE — supervisor review** (Opus: gates green 989 pass / cov 94.12% / ruff+mypy clean; subagent cut off mid-C2 by session limit — Opus finished wiring + fixed 2 left-over test bugs: empty-VM header assert, OIDC keyring stub)
### Stage D — Service management (SECURITY-CRITICAL, last)
- [ ] D1. `operations/services.py` — allow-list, `service_states()`, `control_service()`
- [ ] D2. POST `/services/{service_id}/{action}` — confirm + CSRF + audit + refusals
- [ ] D3. `ci-web` self-restart via detached `systemd-run` (response returns first)
- [ ] D4. Services panel partial (state-aware buttons, active-VM warning on stop)
- [ ] D5. `sudoers.d/ci-web` (exact-command NOPASSWD) + installer `visudo -c` validate
- [ ] D6. Config `[web].managed_services` override + `config.example.toml` + `WEB-UI.md`
- [ ] D7. Security regression tests (unknown id → 400 & systemctl never called; etc.)
- [ ] D8. Stage D tests green · ruff · mypy --strict · coverage ≥90
- [ ] **D-GATE — supervisor review + deploy note**
- [x] D1. `operations/services.py` — allow-list, `service_states()`, `control_service()`
- [x] D2. POST `/services/{service_id}/{action}` — confirm + CSRF + audit + refusals
- [x] D3. `ci-web` self-restart via detached `systemd-run` (response returns first)
- [x] D4. Services panel partial (state-aware buttons, active-VM warning on stop)
- [x] D5. `sudoers.d/ci-web` (exact-command NOPASSWD) + installer `visudo -c` validate
- [x] D6. Config `[web].managed_services` override + `config.example.toml` + `WEB-UI.md`
- [x] D7. Security regression tests (unknown id → 400 & systemctl never called; etc.)
- [x] D8. Stage D tests green · ruff · mypy --strict · coverage ≥90
- [x] **D-GATE — supervisor review + deploy note** (Opus: gates green 1044 pass / cov 94.20% / ruff+mypy clean; verified control_service arg-list+allow-list+stop-refusal+no-shell, route 400-before-subprocess for all 3 refusals + CSRF-first + confirm-gate, sudoers exact-command ↔ argv match with no ci-web stop, 46 security/unit tests, no live-host execution. **DEPLOY (human step, not done): on merge → `pip install .` into prod venv, install `deploy/sudoers.d/ci-web` to /etc/sudoers.d/ (mode 0440, `visudo -c`), `sudo systemctl restart ci-web.service`.**)
---
+37 -9
View File
@@ -16,7 +16,12 @@
# 4. Prompt to store WebSessionSecret / WebOidcClientSecret in the keyring
# (interactive prompt only — never accepts a secret as a CLI argument)
# 5. Print the manual Gitea OIDC application registration steps
# 6. systemctl daemon-reload && enable (+ restart if already running)
# 6. Install deploy/sudoers.d/ci-web to /etc/sudoers.d/ci-web (mode 0440,
# Stage D service-management panel's exact-command NOPASSWD grant),
# validated with `visudo -cf` -- removed again if validation fails
# (a broken sudoers.d file can lock out sudo, so this step never
# leaves an unvalidated file in place)
# 7. systemctl daemon-reload && enable (+ restart if already running)
#
# Usage: sudo bash setup-web-linux.sh [--host <lan-ip>] [options]
# See --help for the full option list.
@@ -31,6 +36,8 @@ VENV="$OPT_CI/venv"
REPO_DIR="$SCRIPT_DIR"
UNIT_SRC="$REPO_DIR/deploy/systemd/ci-web.service"
UNIT_DST="/etc/systemd/system/ci-web.service"
SUDOERS_SRC="$REPO_DIR/deploy/sudoers.d/ci-web"
SUDOERS_DST="/etc/sudoers.d/ci-web"
ENV_FILE="/etc/ci/web.env"
CONFIG_PATH="/var/lib/ci/config.toml"
WEB_PORT=8800
@@ -84,6 +91,7 @@ done
[[ $EUID -eq 0 ]] || die "run as root (sudo bash $(basename "$0") ...)"
[[ -f "$UNIT_SRC" ]] || die "unit template not found: $UNIT_SRC (run from the repo root)"
[[ -f "$SUDOERS_SRC" ]] || die "sudoers template not found: $SUDOERS_SRC (run from the repo root)"
id "$CI_USER" &>/dev/null || die "user '$CI_USER' does not exist -- run setup-host-linux.sh first"
[[ -x "$VENV/bin/python" ]] || die "$VENV/bin/python not found -- run setup-host-linux.sh first"
@@ -104,17 +112,17 @@ echo ""
# ── Step 1: pip install '[web]' extra (non-editable, per CLAUDE.md) ─────────
if [[ "$SKIP_PIP" == false ]]; then
info "[1/6] Installing ci_orchestrator[web] into $VENV (non-editable)"
info "[1/7] Installing ci_orchestrator[web] into $VENV (non-editable)"
sudo -u "$CI_USER" "$VENV/bin/pip" install -q "${REPO_DIR}[web]"
verify_out=$(sudo -u "$CI_USER" "$VENV/bin/python" -m ci_orchestrator web serve --help 2>&1) \
|| die "'ci web serve --help' failed after install -- see output above"
echo " Verify OK: ${verify_out%%$'\n'*}"
else
info "[1/6] pip install -- SKIPPED (--skip-pip)"
info "[1/7] pip install -- SKIPPED (--skip-pip)"
fi
# ── Step 2: render the systemd unit + optional CI_CONFIG env file ───────────
info "[2/6] Rendering ci-web.service (bind host: $WEB_HOST)"
info "[2/7] Rendering ci-web.service (bind host: $WEB_HOST)"
RENDERED="$(mktemp)"
trap 'rm -f "$RENDERED"' EXIT
sed "s#__CI_WEB_BIND_HOST__#${WEB_HOST}#" "$UNIT_SRC" > "$RENDERED"
@@ -143,7 +151,7 @@ else
fi
# ── Step 3: ensure a [web]/[web.oidc] template exists in config.toml ────────
info "[3/6] config.toml [web] / [web.oidc] template"
info "[3/7] config.toml [web] / [web.oidc] template"
WEB_MARKER="# --- appended by setup-web-linux.sh ---"
if [[ -f "$CONFIG_PATH" ]] && { grep -qF "$WEB_MARKER" "$CONFIG_PATH" || grep -q '^\[web\]' "$CONFIG_PATH"; }; then
warn "[web] section (or a previously appended template) already present in $CONFIG_PATH -- skip, not overwriting"
@@ -176,7 +184,7 @@ EOF
fi
# ── Step 4: keyring secrets (interactive prompt only, never argv) ───────────
info "[4/6] Keyring secrets (WebSessionSecret, WebOidcClientSecret)"
info "[4/7] Keyring secrets (WebSessionSecret, WebOidcClientSecret)"
if [[ "$SKIP_SECRETS" == true ]]; then
echo " SKIPPED (--skip-secrets)"
else
@@ -214,7 +222,7 @@ KeyringCredentialStore().get('$1')
fi
# ── Step 5: manual Gitea OIDC app registration instructions ─────────────────
info "[5/6] Gitea OIDC application registration (manual step)"
info "[5/7] Gitea OIDC application registration (manual step)"
cat <<EOF
In Gitea: Settings -> Applications -> "Manage OAuth2 Applications" -> Create
@@ -233,8 +241,28 @@ cat <<EOF
EOF
# ── Step 6: enable + (re)start the service ───────────────────────────────────
info "[6/6] systemctl daemon-reload + enable/restart ci-web.service"
# ── Step 6: sudoers.d/ci-web (Stage D service-panel NOPASSWD grant) ─────────
info "[6/7] Installing sudoers.d/ci-web (service-panel NOPASSWD grant)"
if [[ -f "$SUDOERS_DST" ]] && cmp -s "$SUDOERS_SRC" "$SUDOERS_DST"; then
warn "$SUDOERS_DST already up to date -- skip"
else
install -m 0440 -o root -g root "$SUDOERS_SRC" "$SUDOERS_DST"
if visudo -cf "$SUDOERS_DST"; then
echo " Wrote $SUDOERS_DST (validated: visudo -cf OK)"
else
rm -f "$SUDOERS_DST"
die "visudo -cf rejected $SUDOERS_DST -- REMOVED it (a broken" \
"sudoers.d file can lock out sudo, so this step never leaves an" \
"unvalidated file in place). Fix deploy/sudoers.d/ci-web and" \
"re-run this installer."
fi
fi
warn "Verify systemctl/systemd-run absolute paths on THIS host match the" \
"sudoers file: command -v systemctl systemd-run -- edit both" \
"deploy/sudoers.d/ci-web and $SUDOERS_DST if either is not /usr/bin."
# ── Step 7: enable + (re)start the service ───────────────────────────────────
info "[7/7] systemctl daemon-reload + enable/restart ci-web.service"
systemctl daemon-reload
if systemctl is-active --quiet ci-web.service; then
systemctl restart ci-web.service
+49
View File
@@ -21,6 +21,8 @@ from pathlib import Path
from typing import Any
from urllib.parse import urlsplit
from ci_orchestrator.operations.services import MANAGED_SERVICES, ManagedService
_ENV_KEYS = ("CI_ROOT", "CI_TEMPLATES", "CI_BUILD_VMS", "CI_ARTIFACTS", "CI_KEYS")
_IP_POOL_LEASE_DEFAULT_LINUX = "/var/lib/ci/ip-pool.json"
@@ -109,6 +111,12 @@ class WebConfig:
is deliberately no per-run Actions URL here: the exact Gitea Actions
run-URL path is unconfirmed (open risk), so only the top-level instance
URL is ever exposed.
``managed_services`` (Stage D) overrides the default systemd-unit
allow-list (``operations.services.MANAGED_SERVICES``) that the service
management panel is allowed to start/stop/restart. Left unset, the
default applies unchanged. See ``_parse_managed_services`` for the
permissive-parse contract (a malformed row is skipped, never raised).
"""
enabled: bool = False
@@ -119,6 +127,9 @@ class WebConfig:
session_secret_cred: str = "WebSessionSecret"
oidc: WebOidcConfig | None = None
gitea_url: str | None = None
managed_services: tuple[ManagedService, ...] = field(
default_factory=lambda: MANAGED_SERVICES
)
def gitea_base_url(web: WebConfig) -> str | None:
@@ -223,6 +234,43 @@ def _parse_web_oidc(oidc_section: dict[str, Any]) -> WebOidcConfig | None:
)
def _parse_managed_services(raw: Any) -> tuple[ManagedService, ...]:
"""Parse ``[web].managed_services`` -- an optional override of the systemd-unit
allow-list the service-management panel may control (default:
:data:`~ci_orchestrator.operations.services.MANAGED_SERVICES`).
Permissive by design, like :func:`_parse_web_oidc`: a malformed row
(missing key, wrong type) is skipped rather than raised, so one typo'd
entry in ``config.toml`` can't crash the whole app at startup. An
absent, empty, non-list value, or a list where every row was malformed,
all fall back to the built-in default unchanged.
"""
if not isinstance(raw, list) or not raw:
return MANAGED_SERVICES
services: list[ManagedService] = []
for row in raw:
if not isinstance(row, dict):
continue
service_id = row.get("id")
unit = row.get("unit")
label = row.get("label")
allow_stop = row.get("allow_stop")
if (
not isinstance(service_id, str)
or not service_id
or not isinstance(unit, str)
or not unit
or not isinstance(label, str)
or not label
or not isinstance(allow_stop, bool)
):
continue
services.append(ManagedService(id=service_id, unit=unit, label=label, allow_stop=allow_stop))
return tuple(services) if services else MANAGED_SERVICES
def _parse_web_config(web_section: dict[str, Any]) -> WebConfig:
"""Parse ``[web]`` (and nested ``[web.oidc]``). Absent section -> disabled defaults."""
if not web_section:
@@ -239,6 +287,7 @@ def _parse_web_config(web_section: dict[str, Any]) -> WebConfig:
audit_log=Path(audit_log) if audit_log else WebConfig().audit_log,
allowed_users=[str(u) for u in allowed_users] if isinstance(allowed_users, list) else [],
session_secret_cred=str(web_section.get("session_secret_cred", "WebSessionSecret")),
managed_services=_parse_managed_services(web_section.get("managed_services")),
oidc=_parse_web_oidc(web_section.get("oidc", {}) or {}),
gitea_url=str(gitea_url) if gitea_url else None,
)
+262
View File
@@ -0,0 +1,262 @@
"""Managed systemd service control (Stage D, web-UI redesign) -- SECURITY-CRITICAL.
Lets an authenticated web user start/stop/restart a **fixed allow-list** of
systemd units on the host via ``sudo``. Zero ``click``, zero ``print`` --
everything here returns a dataclass, matching the rest of ``operations/``.
**The allow-list is the whole security model.** :func:`find_service` only
ever returns a :class:`ManagedService` drawn from a known tuple (either the
:data:`MANAGED_SERVICES` default or ``config.web.managed_services`` -- see
``config.py``); there is no code path that builds a :class:`ManagedService`
from unvalidated request input. :func:`control_service` never receives a
bare unit-name string from the caller -- only an already-resolved
``ManagedService``, so a crafted ``service_id``/``action`` can reach at most
a 400 (``web/routes.py``), never an arbitrary ``systemctl <verb> <unit>``.
**Never ``shell=True``.** Every command run by this module is an argument
list handed straight to :func:`subprocess.run` -- no user-supplied string
ever reaches a shell or the unit-name position of ``systemctl``/``sudo``.
**``ci-web`` is restart-only** (``allow_stop=False``): stopping the service
that renders this very page would lock the operator out with no way to
start it again from the UI. A ``stop`` request against such a service is a
policy refusal, not a subprocess failure -- :func:`control_service` returns
``ServiceActionResult(ok=False, command=[])`` *without* building or running
any command (documented choice, see that function's docstring for why this
is a returned result rather than a raised exception).
**``ci-web`` restart is deferred.** Restarting ``ci-web.service`` from
inside its own worker process would kill the HTTP response before it can be
sent. For ``service.id == "ci-web"`` and ``action == "restart"`` the built
command is a detached ``systemd-run --on-active=2 systemctl restart
ci-web.service`` instead of a direct ``systemctl restart`` -- the scheduling
call itself returns almost instantly, well before the 2s-deferred restart
actually happens, so the HTTP response reaches the browser first.
``service_states`` reuses ``operations.host.runner_status`` -- the exact
same argument-list/timeout/parse ``systemctl`` invocation already used for
the runner-status card -- rather than a second read-only systemctl code
path.
"""
from __future__ import annotations
import subprocess
from dataclasses import dataclass
from ci_orchestrator.operations.host import runner_status
#: Timeout for a real (non-dry-run) start/stop/restart invocation. Generous
#: relative to the read-only status query's timeout (``host.py``'s 10s)
#: since a real ``systemctl start`` can legitimately take a few seconds.
_CONTROL_TIMEOUT = 15.0
#: How many characters of ``stderr`` to keep in a failure ``detail`` -- long
#: enough to be useful in the audit log / result partial, short enough to
#: never balloon an audit-log line.
_STDERR_TAIL_CHARS = 500
ALLOWED_ACTIONS: tuple[str, ...] = ("start", "stop", "restart")
@dataclass(frozen=True)
class ManagedService:
"""One systemd unit the web dashboard is allowed to control.
``id`` is the stable slug used in the URL (``/services/{id}/{action}``)
and as the audit-log ``target`` prefix -- never the raw unit name, so a
typo'd or renamed unit can't silently widen what a URL segment controls.
"""
id: str
unit: str
label: str
allow_stop: bool
#: The default allow-list (overridable via ``config.web.managed_services`` --
#: see ``config.py``). ``ci-web`` deliberately has ``allow_stop=False``:
#: restart only, so an operator can never remove their own access to the
#: panel that would let them start it again.
MANAGED_SERVICES: tuple[ManagedService, ...] = (
ManagedService("act-runner", "act-runner.service", "Runner", True),
ManagedService("act-runner-rust", "act-runner-rust.service", "Runner (Rust)", True),
ManagedService("vmware-host", "vmware-host.service", "VMware host", True),
ManagedService("ci-web", "ci-web.service", "Web UI", False),
)
@dataclass(frozen=True)
class ServiceState:
"""Current ``systemctl`` state of one managed service."""
service: ManagedService
active: bool
sub_state: str
@dataclass(frozen=True)
class ServiceActionResult:
"""Outcome of a (real or dry-run) start/stop/restart request.
``command`` is the exact argv that ran (or would run, on ``dry_run``) --
never a shell string. Empty (``[]``) only for the "stop refused" case,
where no command is ever built.
"""
service_id: str
action: str
ok: bool
detail: str
command: list[str]
def find_service(
service_id: str, services: tuple[ManagedService, ...] = MANAGED_SERVICES
) -> ManagedService | None:
"""Look up ``service_id`` in the allow-list. ``None`` if not found.
The sole gate between a URL path segment and a real ``ManagedService`` --
callers (``web/routes.py``) MUST treat a ``None`` return as a refusal
(400) and never fall back to constructing a ``ManagedService`` by hand.
"""
for service in services:
if service.id == service_id:
return service
return None
def service_states(
services: tuple[ManagedService, ...] = MANAGED_SERVICES,
) -> list[ServiceState]:
"""``systemctl`` active/sub-state for every service in ``services``.
Delegates the actual query to :func:`~ci_orchestrator.operations.host.
runner_status`, which already degrades a unit to
``active=False, sub_state="unknown"`` on any failure (missing
``systemctl``, timeout, ``OSError``) rather than raising -- this
function inherits that guarantee unchanged.
"""
statuses = {status.name: status for status in runner_status([s.unit for s in services])}
out: list[ServiceState] = []
for service in services:
status = statuses.get(service.unit)
if status is None: # pragma: no cover -- runner_status always returns one entry per input
out.append(ServiceState(service=service, active=False, sub_state="unknown"))
else:
out.append(ServiceState(service=service, active=status.active, sub_state=status.sub_state))
return out
def _build_command(service: ManagedService, action: str) -> list[str]:
"""The exact argv for ``action`` against ``service`` -- an arg list, never a shell string.
Special case: ``ci-web`` + ``restart`` -> a detached ``systemd-run``
deferral (see the module docstring) instead of a direct ``systemctl
restart`` -- this is the one case where the running process would
otherwise kill its own response mid-flight.
"""
if service.id == "ci-web" and action == "restart":
return [
"sudo",
"-n",
"systemd-run",
"--on-active=2",
"systemctl",
"restart",
service.unit,
]
return ["sudo", "-n", "systemctl", action, service.unit]
def control_service(
service: ManagedService, action: str, *, dry_run: bool = False
) -> ServiceActionResult:
"""Start/stop/restart ``service``. ``service`` MUST come from :func:`find_service`.
Rules (enforced in this order):
1. ``action`` not in :data:`ALLOWED_ACTIONS` -> raises :class:`ValueError`
(the caller, ``web/routes.py``, maps this to a 400 -- a programmer/API
error, not a user-facing operational outcome).
2. ``action == "stop"`` and ``not service.allow_stop`` -> returns a
refusal :class:`ServiceActionResult` (``ok=False, command=[]``)
*without* building or running any command. A returned result rather
than a raised exception: unlike rule 1 (a malformed request), this is
an expected, policy-driven outcome the caller may want to render
identically to any other "ok=False" result rather than special-case
as an exception.
3. ``dry_run=True`` -> returns the would-run ``command`` without
executing it (``ok=True``).
4. Otherwise, run the command with a timeout and ``check=False``; a
non-zero exit or timeout/OSError degrades to ``ok=False`` with a
stderr tail (or the exception text) in ``detail`` -- never raises.
"""
if action not in ALLOWED_ACTIONS:
raise ValueError(f"unsupported service action: {action!r}")
if action == "stop" and not service.allow_stop:
return ServiceActionResult(
service_id=service.id,
action=action,
ok=False,
detail=f"stop is disabled for '{service.label}' ({service.unit} is restart-only)",
command=[],
)
command = _build_command(service, action)
if dry_run:
return ServiceActionResult(
service_id=service.id,
action=action,
ok=True,
detail=f"would run: {' '.join(command)}",
command=command,
)
try:
result = subprocess.run(
command,
capture_output=True,
text=True,
check=False,
timeout=_CONTROL_TIMEOUT,
)
except (subprocess.TimeoutExpired, OSError) as exc:
return ServiceActionResult(
service_id=service.id,
action=action,
ok=False,
detail=f"failed to invoke {command[0]}: {exc}",
command=command,
)
if result.returncode == 0:
if service.id == "ci-web" and action == "restart":
detail = f"restart of {service.unit} scheduled via systemd-run (deferred ~2s)"
else:
detail = f"{action} succeeded for {service.unit}"
return ServiceActionResult(
service_id=service.id, action=action, ok=True, detail=detail, command=command
)
stderr_tail = (result.stderr or "").strip()[-_STDERR_TAIL_CHARS:]
detail = (
f"{action} failed for {service.unit} (exit {result.returncode}): {stderr_tail}"
if stderr_tail
else f"{action} failed for {service.unit} (exit {result.returncode})"
)
return ServiceActionResult(service_id=service.id, action=action, ok=False, detail=detail, command=command)
__all__ = [
"ALLOWED_ACTIONS",
"MANAGED_SERVICES",
"ManagedService",
"ServiceActionResult",
"ServiceState",
"control_service",
"find_service",
"service_states",
]
+37
View File
@@ -61,6 +61,7 @@ from ci_orchestrator.operations.inventory import (
list_backups,
list_templates,
)
from ci_orchestrator.operations.services import ServiceState, service_states
from ci_orchestrator.operations.vms import LiveVm, list_live_vms
from ci_orchestrator.web import filters
from ci_orchestrator.web.auth import _read_secret, configure_auth, require_user
@@ -202,6 +203,20 @@ def _fetch_artifacts(config: Config) -> list[FileEntry] | None:
return None
def _fetch_service_states(config: Config) -> list[ServiceState]:
"""Managed systemd unit states (Stage D), or ``[]`` if the query could not run at all.
``service_states`` itself already degrades a single unit to
``active=False, sub_state="unknown"`` on a ``systemctl`` hiccup (see
``operations/host.py::runner_status``) -- this ``except Exception`` is a
defensive second layer, matching every other ``_fetch_*`` wrapper here.
"""
try:
return service_states(config.web.managed_services)
except Exception: # degrade, never 500
return []
# ────────────────────────────────────────────────────────── status header
@@ -335,6 +350,7 @@ def dashboard(
"template_names": template_names,
"backups": _fetch_backups(config),
"artifacts": _fetch_artifacts(config),
"services": _fetch_service_states(config),
"health": _health_summary(
vms,
backend_available=backend_available,
@@ -415,6 +431,23 @@ def files_partial(request: Request) -> HTMLResponse:
return templates.TemplateResponse(request, "partials/files.html", context)
def services_partial(request: Request) -> HTMLResponse:
"""``GET /services`` (Stage D) -- polled every 15s by the dashboard via htmx.
Needs a fresh ``csrf_token`` on every render (unlike ``resources``/
``vmware``/``templates``/``files``, which are pure read-only cards) --
the panel's start/stop/restart forms are mutating, same as
``vms_partial``/``host_partial``.
"""
config: Config = request.app.state.config
templates: Jinja2Templates = request.app.state.templates
context = {
"services": _fetch_service_states(config),
"csrf_token": issue_csrf(request),
}
return templates.TemplateResponse(request, "partials/services.html", context)
# ─────────────────────────────────────────────────────────────────── factory
@@ -470,6 +503,9 @@ def create_app(config: Config) -> FastAPI:
app.get("/files", response_class=HTMLResponse, dependencies=[Depends(require_user)])(
files_partial
)
app.get("/services", response_class=HTMLResponse, dependencies=[Depends(require_user)])(
services_partial
)
register_action_routes(app)
# SessionMiddleware is ALWAYS installed (W6) -- CSRF needs
@@ -509,6 +545,7 @@ __all__ = [
"get_backend",
"healthz",
"resources_partial",
"services_partial",
"templates_partial",
"vmware_partial",
]
+112
View File
@@ -49,6 +49,7 @@ from ci_orchestrator.operations.audit import AuditEntry, append_audit, read_audi
from ci_orchestrator.operations.cache import purge_cargo_cache
from ci_orchestrator.operations.host import backup_template
from ci_orchestrator.operations.inventory import list_templates
from ci_orchestrator.operations.services import ALLOWED_ACTIONS, control_service, find_service
from ci_orchestrator.operations.vms import RemovalResult, cleanup_orphans, remove_vm
from ci_orchestrator.web.app import get_backend
from ci_orchestrator.web.auth import require_user
@@ -200,13 +201,23 @@ def _confirm_partial(
summary: str,
action_url: str,
extra_fields: dict[str, str],
warning: str | None = None,
) -> HTMLResponse:
"""Render the shared confirm partial.
``warning`` (Stage D) is an optional extra callout rendered above the
Confirm button -- used by the service-control route to surface "N
active VM(s) -- this interrupts running jobs" when confirming a
stop/restart of ``vmware-host.service``. ``None`` (every other caller)
renders nothing extra, unchanged from before Stage D.
"""
context = {
"action_label": action_label,
"target": target,
"summary": summary,
"action_url": action_url,
"extra_fields": extra_fields,
"warning": warning,
"csrf_token": issue_csrf(request),
}
return _render_action(request, "partials/confirm.html", context)
@@ -470,6 +481,100 @@ async def host_backup(
)
# ───────────────────────────────────────────────────── POST /services/{id}/{action}
#: The ``ManagedService.id`` whose stop/restart confirm step warns about
#: active VMs -- stopping the VMware host stack kills every running build.
_VMWARE_HOST_SERVICE_ID = "vmware-host"
def _active_vm_count(backend: VmBackend | None) -> int:
"""Best-effort running-VM count for the ``vmware-host`` confirm warning.
Degrades to ``0`` on a missing backend or any hiccup listing VMs --
this is advisory copy in a confirm screen, never a value a route should
500 over.
"""
if backend is None:
return 0
try:
return len(backend.list_running())
except Exception: # a route must never 500 on a backend hiccup
return 0
async def services_control(
service_id: str,
action: str,
request: Request,
user: str = Depends(require_user),
_csrf: None = Depends(require_csrf),
backend: VmBackend | None = Depends(get_backend),
) -> HTMLResponse:
"""Start/stop/restart one managed systemd unit, behind the two-step confirm gate.
Mirrors :func:`vms_remove`'s shape exactly. Three refusals short-circuit
**before** ``control_service`` (and therefore any subprocess) or the
audit log ever run -- an unknown ``service_id``, an unsupported
``action``, and a ``stop`` request against a restart-only service (see
``operations/services.py``'s module docstring on why ``ci-web`` is
restart-only). Each is a plain validation refusal, not an action, so
none of them is audited -- same rule as the path-containment check in
:func:`vms_remove`.
"""
config: Config = request.app.state.config
services = config.web.managed_services
service = find_service(service_id, services)
if service is None:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="unknown service id")
if action not in ALLOWED_ACTIONS:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="unsupported action")
if action == "stop" and not service.allow_stop:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"stop is disabled for '{service.label}' (restart-only)",
)
action_label = f"{service.label}: {action}"
target = service.unit
audit_action = f"service.{action}"
form = await request.form()
execute = _form_str(form, _STAGE_FIELD) == _STAGE_EXECUTE
if execute:
result = control_service(service, action, dry_run=False)
_audit(
config,
user=user,
action=audit_action,
target=target,
result="ok" if result.ok else "error",
detail=result.detail,
)
return _result_partial(request, action_label=action_label, ok=result.ok, detail=result.detail)
result = control_service(service, action, dry_run=True)
_audit(config, user=user, action=audit_action, target=target, result="dry-run", detail=result.detail)
warning: str | None = None
if service.id == _VMWARE_HOST_SERVICE_ID and action != "start":
vm_count = _active_vm_count(backend)
if vm_count > 0:
warning = f"{vm_count} active VM(s) -- this interrupts running jobs."
return _confirm_partial(
request,
action_label=action_label,
target=target,
summary=result.detail,
action_url=f"/services/{service_id}/{action}",
extra_fields={},
warning=warning,
)
# ──────────────────────────────────────────────────────────────── GET /audit
@@ -492,6 +597,12 @@ def register_action_routes(app: FastAPI) -> None:
app.add_api_route("/vms/cleanup", vms_cleanup, methods=["POST"], response_class=HTMLResponse)
app.add_api_route("/cache/purge", cache_purge, methods=["POST"], response_class=HTMLResponse)
app.add_api_route("/host/backup", host_backup, methods=["POST"], response_class=HTMLResponse)
app.add_api_route(
"/services/{service_id}/{action}",
services_control,
methods=["POST"],
response_class=HTMLResponse,
)
app.add_api_route("/audit", audit_view, methods=["GET"], response_class=HTMLResponse)
@@ -501,6 +612,7 @@ __all__ = [
"encode_vmx_id",
"host_backup",
"register_action_routes",
"services_control",
"vms_cleanup",
"vms_remove",
]
+6
View File
@@ -305,6 +305,12 @@ input[type="text"], input[type="number"], select {
/* One-line field description above a form control (B6 backup dropdown). */
.field-hint { font-size: 0.8rem; color: var(--text-dim); margin: 0 0 0.5rem; }
/* Multiple one-button action forms side-by-side in one table cell (Stage D
* services panel: Start/Stop/Restart). Overrides the default form's
* margin-top so the row stays compact inside a <td>. */
.service-actions { display: flex; flex-wrap: wrap; align-items: center; gap: 0.5rem; }
.service-actions form { margin-top: 0; }
/* Muted inline text (e.g. a relative-time suffix next to a file name). */
.text-dim { color: var(--text-dim); }
@@ -13,5 +13,6 @@
{% include "partials/vmware.html" %}
{% include "partials/templates.html" %}
{% include "partials/files.html" %}
{% include "partials/services.html" %}
</div>
{% endblock %}
@@ -2,6 +2,9 @@
<h3>Confirm: {{ action_label }}</h3>
<p>Target: <code>{{ target }}</code></p>
<p>{{ summary }}</p>
{% if warning %}
<p class="notice notice-warn" role="alert">{{ warning }}</p>
{% endif %}
<form method="post" action="{{ action_url }}" hx-post="{{ action_url }}" hx-target="#action-slot" hx-swap="innerHTML">
<input type="hidden" name="csrf" value="{{ csrf_token }}">
<input type="hidden" name="stage" value="execute">
@@ -0,0 +1,57 @@
<section class="card" id="services-card" hx-get="/services" hx-trigger="every 15s" hx-swap="outerHTML">
<h2><svg class="icon" aria-hidden="true"><use href="#i-gear"></use></svg>Services</h2>
{% if services %}
<div class="table-wrap">
<table>
<thead>
<tr>
<th>Service</th>
<th>Unit</th>
<th>Status</th>
<th>Sub-state</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for state in services %}
<tr class="{{ 'row-alert' if not state.active else '' }}">
<td>{{ state.service.label }}</td>
<td><code>{{ state.service.unit }}</code></td>
<td>
{% if state.active %}
<span class="badge badge-success"><svg class="icon" aria-hidden="true"><use href="#i-check"></use></svg> active</span>
{% else %}
<span class="badge badge-danger"><svg class="icon" aria-hidden="true"><use href="#i-alert"></use></svg> down</span>
{% endif %}
</td>
<td>{{ state.sub_state }}</td>
<td class="service-actions">
{% if not state.active %}
<form method="post" action="/services/{{ state.service.id }}/start"
hx-post="/services/{{ state.service.id }}/start" hx-target="#action-slot" hx-swap="innerHTML">
<input type="hidden" name="csrf" value="{{ csrf_token }}">
<button type="submit" class="btn btn-primary" aria-label="Start {{ state.service.label }}">Start</button>
</form>
{% endif %}
{% if state.active and state.service.allow_stop %}
<form method="post" action="/services/{{ state.service.id }}/stop"
hx-post="/services/{{ state.service.id }}/stop" hx-target="#action-slot" hx-swap="innerHTML">
<input type="hidden" name="csrf" value="{{ csrf_token }}">
<button type="submit" class="btn btn-danger" aria-label="Stop {{ state.service.label }}">Stop</button>
</form>
{% endif %}
<form method="post" action="/services/{{ state.service.id }}/restart"
hx-post="/services/{{ state.service.id }}/restart" hx-target="#action-slot" hx-swap="innerHTML">
<input type="hidden" name="csrf" value="{{ csrf_token }}">
<button type="submit" class="btn" aria-label="Restart {{ state.service.label }}">Restart</button>
</form>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<p class="notice notice-warn" role="alert">Service status unavailable.</p>
{% endif %}
</section>
+86
View File
@@ -10,6 +10,7 @@ from __future__ import annotations
from pathlib import Path
from ci_orchestrator.config import WebConfig, WebOidcConfig, gitea_base_url, load_config
from ci_orchestrator.operations.services import MANAGED_SERVICES, ManagedService
def _paths_section() -> str:
@@ -251,3 +252,88 @@ def test_gitea_base_url_none_when_issuer_unparseable() -> None:
)
assert gitea_base_url(web) is None
# ── [web].managed_services (Stage D) ────────────────────────────────────────
def test_managed_services_absent_defaults_to_builtin_allowlist() -> None:
assert WebConfig().managed_services == MANAGED_SERVICES
def test_managed_services_absent_from_toml_defaults_to_builtin_allowlist(tmp_path: Path) -> None:
cfg = load_config(config_path=_write_toml(tmp_path), env={})
assert cfg.web.managed_services == MANAGED_SERVICES
def test_managed_services_override_parses_rows(tmp_path: Path) -> None:
extra = (
"\n[[web.managed_services]]\n"
'id = "act-runner"\n'
'unit = "act-runner.service"\n'
'label = "Runner"\n'
"allow_stop = true\n"
"\n[[web.managed_services]]\n"
'id = "ci-web"\n'
'unit = "ci-web.service"\n'
'label = "Web UI"\n'
"allow_stop = false\n"
)
cfg = load_config(config_path=_write_toml(tmp_path, extra), env={})
assert cfg.web.managed_services == (
ManagedService("act-runner", "act-runner.service", "Runner", True),
ManagedService("ci-web", "ci-web.service", "Web UI", False),
)
def test_managed_services_malformed_row_is_skipped_not_raised(tmp_path: Path) -> None:
extra = (
"\n[[web.managed_services]]\n"
'id = "act-runner"\n'
'unit = "act-runner.service"\n'
'label = "Runner"\n'
"allow_stop = true\n"
"\n[[web.managed_services]]\n"
# Missing 'unit' -- this row must be skipped, not crash config loading.
'id = "broken"\n'
'label = "Broken"\n'
"allow_stop = true\n"
)
cfg = load_config(config_path=_write_toml(tmp_path, extra), env={})
assert cfg.web.managed_services == (
ManagedService("act-runner", "act-runner.service", "Runner", True),
)
def test_managed_services_all_rows_malformed_falls_back_to_default(tmp_path: Path) -> None:
extra = (
"\n[[web.managed_services]]\n"
'id = "broken"\n'
"allow_stop = true\n"
)
cfg = load_config(config_path=_write_toml(tmp_path, extra), env={})
assert cfg.web.managed_services == MANAGED_SERVICES
def test_managed_services_wrong_type_for_allow_stop_is_skipped(tmp_path: Path) -> None:
extra = (
"\n[[web.managed_services]]\n"
'id = "act-runner"\n'
'unit = "act-runner.service"\n'
'label = "Runner"\n'
'allow_stop = "yes"\n' # string, not bool -- must be rejected
)
cfg = load_config(config_path=_write_toml(tmp_path, extra), env={})
assert cfg.web.managed_services == MANAGED_SERVICES
def test_managed_services_empty_list_falls_back_to_default(tmp_path: Path) -> None:
extra = "\n[web]\nmanaged_services = []\n"
cfg = load_config(config_path=_write_toml(tmp_path, extra), env={})
assert cfg.web.managed_services == MANAGED_SERVICES
+369
View File
@@ -0,0 +1,369 @@
"""Tests for the pure ``operations.services`` layer (Stage D of the web-UI
redesign) -- SECURITY-CRITICAL: this module builds the exact argv that ends
up behind ``sudo`` on the live host.
No real ``systemctl``/``sudo``/``systemd-run`` subprocess is ever invoked --
every ``subprocess.run`` call is monkeypatched at the name this module
imported it under (mirrors ``test_operations_host.py``'s convention).
"""
from __future__ import annotations
import subprocess
from typing import Any
import pytest
import ci_orchestrator.operations.host as host_module
import ci_orchestrator.operations.services as services_module
from ci_orchestrator.operations.services import (
ALLOWED_ACTIONS,
MANAGED_SERVICES,
ManagedService,
ServiceState,
control_service,
find_service,
service_states,
)
class _RunResult:
def __init__(self, stdout: str = "", stderr: str = "", returncode: int = 0) -> None:
self.stdout = stdout
self.stderr = stderr
self.returncode = returncode
def _spy(real: Any) -> Any:
"""Wrap ``real`` so ``.calls`` records every ``(args, kwargs)`` call, still delegating."""
def wrapper(*args: Any, **kwargs: Any) -> Any:
wrapper.calls.append((args, kwargs)) # type: ignore[attr-defined]
return real(*args, **kwargs)
wrapper.calls = [] # type: ignore[attr-defined]
return wrapper
# ── MANAGED_SERVICES / find_service ─────────────────────────────────────────
def test_managed_services_shape() -> None:
ids = {s.id for s in MANAGED_SERVICES}
assert ids == {"act-runner", "act-runner-rust", "vmware-host", "ci-web"}
by_id = {s.id: s for s in MANAGED_SERVICES}
assert by_id["ci-web"].allow_stop is False
assert by_id["ci-web"].unit == "ci-web.service"
assert by_id["act-runner"].allow_stop is True
assert by_id["vmware-host"].allow_stop is True
def test_find_service_returns_allowlist_member() -> None:
svc = find_service("act-runner")
assert svc is not None
assert svc.unit == "act-runner.service"
def test_find_service_unknown_id_returns_none() -> None:
assert find_service("does-not-exist") is None
def test_find_service_uses_custom_services_tuple() -> None:
custom = (ManagedService("x", "x.service", "X", True),)
assert find_service("x", custom) is not None
# Not present in the default list, only in the custom one.
assert find_service("x", MANAGED_SERVICES) is None
# ── service_states ───────────────────────────────────────────────────────────
def test_service_states_reflects_active_and_inactive(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(host_module.shutil, "which", lambda _n: "/usr/bin/systemctl")
def _run(cmd: list[str], **_kw: Any) -> _RunResult:
unit = cmd[-1]
if cmd[1] == "is-active":
return _RunResult(stdout="active\n" if unit == "act-runner.service" else "inactive\n")
return _RunResult(stdout="running\n" if unit == "act-runner.service" else "dead\n")
monkeypatch.setattr(host_module.subprocess, "run", _run)
services = (
ManagedService("act-runner", "act-runner.service", "Runner", True),
ManagedService("ci-web", "ci-web.service", "Web UI", False),
)
states = service_states(services)
assert states == [
ServiceState(service=services[0], active=True, sub_state="running"),
ServiceState(service=services[1], active=False, sub_state="dead"),
]
def test_service_states_degrades_to_unknown_when_systemctl_missing(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(host_module.shutil, "which", lambda _n: None)
states = service_states(MANAGED_SERVICES)
assert len(states) == len(MANAGED_SERVICES)
assert all(not s.active and s.sub_state == "unknown" for s in states)
def test_service_states_degrades_to_unknown_on_subprocess_timeout(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(host_module.shutil, "which", lambda _n: "/usr/bin/systemctl")
def _boom(*_a: Any, **_kw: Any) -> Any:
raise subprocess.TimeoutExpired(cmd="systemctl", timeout=10.0)
monkeypatch.setattr(host_module.subprocess, "run", _boom)
states = service_states(MANAGED_SERVICES)
assert all(not s.active and s.sub_state == "unknown" for s in states)
def test_service_states_never_uses_shell_true(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(host_module.shutil, "which", lambda _n: "/usr/bin/systemctl")
seen_kwargs: list[dict[str, Any]] = []
def _run(cmd: list[str], **kw: Any) -> _RunResult:
seen_kwargs.append(kw)
return _RunResult(stdout="active\n")
monkeypatch.setattr(host_module.subprocess, "run", _run)
service_states(MANAGED_SERVICES)
assert seen_kwargs # sanity: the fake was actually invoked
assert all(not kw.get("shell") for kw in seen_kwargs)
# ── control_service: validation ──────────────────────────────────────────────
def test_control_service_unsupported_action_raises_value_error(
monkeypatch: pytest.MonkeyPatch,
) -> None:
spy = _spy(subprocess.run)
monkeypatch.setattr(services_module.subprocess, "run", spy)
service = find_service("act-runner")
assert service is not None
with pytest.raises(ValueError):
control_service(service, "reboot")
assert spy.calls == [] # type: ignore[attr-defined]
def test_control_service_stop_refused_for_disallowed_service_builds_no_command(
monkeypatch: pytest.MonkeyPatch,
) -> None:
spy = _spy(subprocess.run)
monkeypatch.setattr(services_module.subprocess, "run", spy)
ci_web = find_service("ci-web")
assert ci_web is not None
result = control_service(ci_web, "stop")
assert result.service_id == "ci-web"
assert result.action == "stop"
assert result.ok is False
assert result.command == []
assert "restart-only" in result.detail or "disabled" in result.detail
assert spy.calls == [] # type: ignore[attr-defined]
def test_control_service_stop_refused_even_in_dry_run(monkeypatch: pytest.MonkeyPatch) -> None:
"""The stop-refusal check must win over dry_run -- no command is built either way."""
spy = _spy(subprocess.run)
monkeypatch.setattr(services_module.subprocess, "run", spy)
ci_web = find_service("ci-web")
assert ci_web is not None
result = control_service(ci_web, "stop", dry_run=True)
assert result.ok is False
assert result.command == []
assert spy.calls == [] # type: ignore[attr-defined]
# ── control_service: dry_run argv, incl. the ci-web restart special case ───
@pytest.mark.parametrize("action", ALLOWED_ACTIONS)
def test_control_service_dry_run_argv_for_every_normal_service(action: str) -> None:
for service in MANAGED_SERVICES:
if service.id == "ci-web":
continue # covered separately below (special-cased argv)
if action == "stop" and not service.allow_stop:
continue # covered by the refusal tests above
result = control_service(service, action, dry_run=True)
assert result.ok is True
assert result.command == ["sudo", "-n", "systemctl", action, service.unit]
assert result.detail.startswith("would run:")
def test_control_service_dry_run_ci_web_start_and_stop_use_plain_systemctl() -> None:
ci_web = find_service("ci-web")
assert ci_web is not None
start_result = control_service(ci_web, "start", dry_run=True)
assert start_result.command == ["sudo", "-n", "systemctl", "start", "ci-web.service"]
def test_control_service_dry_run_ci_web_restart_uses_systemd_run_deferred() -> None:
ci_web = find_service("ci-web")
assert ci_web is not None
result = control_service(ci_web, "restart", dry_run=True)
assert result.ok is True
assert result.command == [
"sudo",
"-n",
"systemd-run",
"--on-active=2",
"systemctl",
"restart",
"ci-web.service",
]
assert "systemd-run" in result.detail
assert "--on-active=2" in result.detail
def test_control_service_dry_run_never_executes_anything(monkeypatch: pytest.MonkeyPatch) -> None:
spy = _spy(subprocess.run)
monkeypatch.setattr(services_module.subprocess, "run", spy)
service = find_service("act-runner")
assert service is not None
control_service(service, "restart", dry_run=True)
assert spy.calls == [] # type: ignore[attr-defined]
# ── control_service: real execution ──────────────────────────────────────────
def test_control_service_real_run_success_builds_arg_list(monkeypatch: pytest.MonkeyPatch) -> None:
seen: list[tuple[Any, ...]] = []
def _run(cmd: list[str], **kw: Any) -> _RunResult:
seen.append(tuple(cmd))
assert not kw.get("shell")
return _RunResult(returncode=0)
monkeypatch.setattr(services_module.subprocess, "run", _run)
service = find_service("act-runner")
assert service is not None
result = control_service(service, "restart", dry_run=False)
assert result.ok is True
assert seen == [("sudo", "-n", "systemctl", "restart", "act-runner.service")]
assert seen[0][0:3] == ("sudo", "-n", "systemctl")
assert result.command == ["sudo", "-n", "systemctl", "restart", "act-runner.service"]
def test_control_service_ci_web_restart_real_run_uses_systemd_run(
monkeypatch: pytest.MonkeyPatch,
) -> None:
seen: list[list[str]] = []
def _run(cmd: list[str], **_kw: Any) -> _RunResult:
seen.append(cmd)
return _RunResult(returncode=0)
monkeypatch.setattr(services_module.subprocess, "run", _run)
ci_web = find_service("ci-web")
assert ci_web is not None
result = control_service(ci_web, "restart", dry_run=False)
assert result.ok is True
assert "systemd-run" in seen[0]
assert "--on-active=2" in seen[0]
assert "scheduled" in result.detail
def test_control_service_real_run_nonzero_exit_is_not_ok(monkeypatch: pytest.MonkeyPatch) -> None:
def _run(cmd: list[str], **_kw: Any) -> _RunResult:
return _RunResult(returncode=1, stderr="Unit act-runner.service not found.\n")
monkeypatch.setattr(services_module.subprocess, "run", _run)
service = find_service("act-runner")
assert service is not None
result = control_service(service, "start", dry_run=False)
assert result.ok is False
assert "not found" in result.detail
assert "1" in result.detail
def test_control_service_real_run_nonzero_exit_no_stderr_still_reports_exit_code(
monkeypatch: pytest.MonkeyPatch,
) -> None:
def _run(cmd: list[str], **_kw: Any) -> _RunResult:
return _RunResult(returncode=1, stderr="")
monkeypatch.setattr(services_module.subprocess, "run", _run)
service = find_service("act-runner")
assert service is not None
result = control_service(service, "start", dry_run=False)
assert result.ok is False
assert "exit 1" in result.detail
def test_control_service_timeout_degrades_to_error_never_raises(
monkeypatch: pytest.MonkeyPatch,
) -> None:
def _boom(*_a: Any, **_kw: Any) -> Any:
raise subprocess.TimeoutExpired(cmd="sudo", timeout=15.0)
monkeypatch.setattr(services_module.subprocess, "run", _boom)
service = find_service("act-runner")
assert service is not None
result = control_service(service, "restart", dry_run=False)
assert result.ok is False
assert "sudo" in result.detail or "restart" in result.detail.lower()
def test_control_service_oserror_degrades_to_error_never_raises(
monkeypatch: pytest.MonkeyPatch,
) -> None:
def _boom(*_a: Any, **_kw: Any) -> Any:
raise OSError("sudo: command not found")
monkeypatch.setattr(services_module.subprocess, "run", _boom)
service = find_service("act-runner")
assert service is not None
result = control_service(service, "restart", dry_run=False)
assert result.ok is False
assert "sudo: command not found" in result.detail
def test_control_service_stderr_tail_is_truncated(monkeypatch: pytest.MonkeyPatch) -> None:
huge_stderr = "x" * 5000
def _run(cmd: list[str], **_kw: Any) -> _RunResult:
return _RunResult(returncode=1, stderr=huge_stderr)
monkeypatch.setattr(services_module.subprocess, "run", _run)
service = find_service("act-runner")
assert service is not None
result = control_service(service, "start", dry_run=False)
assert len(result.detail) < len(huge_stderr) + 100
+648
View File
@@ -0,0 +1,648 @@
"""Tests for the Stage D service-management web route -- SECURITY-CRITICAL:
this route lets an authenticated web user start/stop/restart systemd
services on the host via ``sudo``.
Same conventions as ``test_web_actions.py``: no real ``vmrun``/``systemctl``/
``sudo``/``systemd-run`` subprocess is ever invoked. ``operations.services.
control_service`` is either spied on (still delegates to the real logic, so
the confirm-gate/CSRF/audit plumbing is exercised for real) or has its
underlying ``subprocess.run`` monkeypatched out from under it.
"""
from __future__ import annotations
import re
from pathlib import Path
from typing import Any
import pytest
from fastapi.testclient import TestClient
import ci_orchestrator.operations.services as services_module
import ci_orchestrator.web.routes as routes_module
from ci_orchestrator.config import (
BackendConfig,
CargoCacheConfig,
Config,
Paths,
WebConfig,
WebOidcConfig,
)
from ci_orchestrator.credentials import Credential, KeyringCredentialStore
from ci_orchestrator.operations.audit import read_audit
from ci_orchestrator.operations.services import (
MANAGED_SERVICES,
ManagedService,
control_service,
)
from ci_orchestrator.web.app import create_app, get_backend
_SESSION_CRED_NAME = "TestWebSessionSecretD"
_OIDC_CRED_NAME = "TestWebOidcClientSecretD"
_FAKE_SESSION_SECRET = "sess-secret-do-not-leak-stage-d"
_FAKE_OIDC_CLIENT_SECRET = "oidc-secret-do-not-leak-stage-d"
@pytest.fixture(autouse=True)
def _no_real_systemctl(monkeypatch: pytest.MonkeyPatch) -> None:
"""Safety net: no test in this file may invoke a real ``systemctl``.
Every ``GET /services`` (including the ones ``_get_csrf`` issues just to
fetch a token) calls ``service_states()`` -> ``runner_status()``, which
otherwise would shell out for real. Defaulting ``shutil.which`` to
"systemctl not found" makes that read-only status query a guaranteed
no-subprocess no-op everywhere in this file; the handful of tests that
need a specific active/inactive shape re-patch both ``shutil.which`` and
``subprocess.run`` locally, which simply overrides this default for the
duration of that one test.
"""
import ci_orchestrator.operations.host as host_module
monkeypatch.setattr(host_module.shutil, "which", lambda _n: None)
# ── shared fixtures / fakes ─────────────────────────────────────────────────
def _make_config(
root: Path,
*,
managed_services: tuple[ManagedService, ...] | None = None,
active: bool = False,
allowed_users: list[str] | None = None,
) -> Config:
oidc = None
if active:
oidc = WebOidcConfig(
issuer="https://gitea.example.lan",
client_id="ci-web",
redirect_url="http://127.0.0.1:8800/auth/callback",
client_secret_cred=_OIDC_CRED_NAME,
)
web_kwargs: dict[str, Any] = {
"audit_log": root / "web-audit.jsonl",
"session_secret_cred": _SESSION_CRED_NAME,
"enabled": active,
"allowed_users": allowed_users if allowed_users is not None else [],
"oidc": oidc,
}
if managed_services is not None:
web_kwargs["managed_services"] = managed_services
return Config(
paths=Paths(
root=root,
templates=root / "templates",
build_vms=root / "build-vms",
artifacts=root / "artifacts",
keys=root / "keys",
),
backend=BackendConfig(),
cargo_cache=CargoCacheConfig(enabled=True, root=root / "cache" / "cargo", retention_days=30),
web=WebConfig(**web_kwargs),
)
def _install_fake_keyring(monkeypatch: pytest.MonkeyPatch) -> None:
secrets = {
_SESSION_CRED_NAME: _FAKE_SESSION_SECRET,
_OIDC_CRED_NAME: _FAKE_OIDC_CLIENT_SECRET,
}
def fake_get(self: KeyringCredentialStore, target: str) -> Credential:
if target not in secrets:
raise KeyError(target)
return Credential(username=target, password=secrets[target])
monkeypatch.setattr(KeyringCredentialStore, "get", fake_get)
class _FakeBackend:
"""Stub ``VmBackend`` reporting a fixed number of running VMs."""
def __init__(self, running_count: int = 0) -> None:
self._running_count = running_count
def list_running(self) -> list[Any]:
return [object()] * self._running_count
class _BoomBackend:
"""A backend whose ``list_running`` always raises -- must degrade to 0, never 500."""
def list_running(self) -> list[Any]:
raise RuntimeError("backend hiccup")
def _spy(real: Any) -> Any:
def wrapper(*args: Any, **kwargs: Any) -> Any:
wrapper.calls.append((args, kwargs)) # type: ignore[attr-defined]
return real(*args, **kwargs)
wrapper.calls = [] # type: ignore[attr-defined]
return wrapper
def _csrf_from(text: str) -> str:
match = re.search(r'name="csrf" value="([^"]+)"', text)
assert match is not None, f"no csrf field found in: {text[:300]!r}"
return match.group(1)
def _get_csrf(client: TestClient, route: str = "/services") -> str:
resp = client.get(route)
assert resp.status_code == 200
return _csrf_from(resp.text)
def _last_audit_entry(audit_log: Path) -> Any:
entries = read_audit(audit_log, last=100)
assert entries, "expected at least one audit entry"
return entries[-1]
def _quiet_subprocess_ok(monkeypatch: pytest.MonkeyPatch) -> list[list[str]]:
"""Fake every real ``systemctl``/``sudo``/``systemd-run`` call as succeeding.
Returns the list argv's seen, in call order, for assertion.
"""
seen: list[list[str]] = []
class _R:
returncode = 0
stdout = ""
stderr = ""
def _run(cmd: list[str], **_kw: Any) -> _R:
seen.append(list(cmd))
return _R()
monkeypatch.setattr(services_module.subprocess, "run", _run)
return seen
# ── 1. unknown service_id -> 400, never calls control_service ──────────────
def test_unknown_service_id_is_400_and_control_service_never_called(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
spy = _spy(control_service)
monkeypatch.setattr(routes_module, "control_service", spy)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/does-not-exist/restart", data={"csrf": csrf, "stage": "execute"})
assert resp.status_code == 400
assert spy.calls == [] # type: ignore[attr-defined]
assert not config.web.audit_log.exists()
# ── 2. unsupported action -> 400, never calls control_service ──────────────
def test_unsupported_action_is_400_and_control_service_never_called(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
spy = _spy(control_service)
monkeypatch.setattr(routes_module, "control_service", spy)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/act-runner/reboot", data={"csrf": csrf, "stage": "execute"})
assert resp.status_code == 400
assert spy.calls == [] # type: ignore[attr-defined]
assert not config.web.audit_log.exists()
# ── 3. stop on a restart-only service (ci-web) -> 400, never called ────────
def test_stop_on_ci_web_is_400_and_control_service_never_called(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
spy = _spy(control_service)
monkeypatch.setattr(routes_module, "control_service", spy)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/ci-web/stop", data={"csrf": csrf, "stage": "execute"})
assert resp.status_code == 400
assert spy.calls == [] # type: ignore[attr-defined]
assert not config.web.audit_log.exists()
def test_stop_on_ci_web_preview_is_also_400_not_a_dry_run(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""The refusal must win even on a preview (no ``stage=execute``) POST."""
spy = _spy(control_service)
monkeypatch.setattr(routes_module, "control_service", spy)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/ci-web/stop", data={"csrf": csrf})
assert resp.status_code == 400
assert spy.calls == [] # type: ignore[attr-defined]
# ── 4. CSRF enforcement ──────────────────────────────────────────────────────
def test_post_without_csrf_field_is_403(tmp_path: Path) -> None:
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
client.get("/services") # establish a session
resp = client.post("/services/act-runner/restart", data={})
assert resp.status_code == 403
def test_post_with_wrong_csrf_is_403(tmp_path: Path) -> None:
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
client.get("/services")
resp = client.post("/services/act-runner/restart", data={"csrf": "totally-not-the-token"})
assert resp.status_code == 403
def test_wrong_csrf_never_calls_control_service(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
spy = _spy(control_service)
monkeypatch.setattr(routes_module, "control_service", spy)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
client.get("/services")
resp = client.post(
"/services/act-runner/restart", data={"csrf": "bogus", "stage": "execute"}
)
assert resp.status_code == 403
assert spy.calls == [] # type: ignore[attr-defined]
# ── 5. two-step confirm gate ─────────────────────────────────────────────────
def test_preview_returns_confirm_partial_dry_run_and_does_not_execute(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
spy = _spy(control_service)
monkeypatch.setattr(routes_module, "control_service", spy)
seen = _quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/act-runner/restart", data={"csrf": csrf})
assert resp.status_code == 200
assert "Confirm" in resp.text
assert len(spy.calls) == 1 # type: ignore[attr-defined]
assert spy.calls[0][1]["dry_run"] is True # type: ignore[attr-defined]
assert seen == [] # dry_run never reaches subprocess.run
entry = _last_audit_entry(config.web.audit_log)
assert entry.result == "dry-run"
assert entry.action == "service.restart"
assert entry.target == "act-runner.service"
def test_execute_runs_real_control_service_and_audits_ok(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
spy = _spy(control_service)
monkeypatch.setattr(routes_module, "control_service", spy)
seen = _quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post(
"/services/act-runner/restart", data={"csrf": csrf, "stage": "execute"}
)
assert resp.status_code == 200
assert len(spy.calls) == 1 # type: ignore[attr-defined]
assert spy.calls[0][1]["dry_run"] is False # type: ignore[attr-defined]
assert seen == [["sudo", "-n", "systemctl", "restart", "act-runner.service"]]
entry = _last_audit_entry(config.web.audit_log)
assert entry.result == "ok"
assert entry.action == "service.restart"
assert entry.target == "act-runner.service"
def test_execute_failure_audits_error_and_never_500s(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
class _R:
returncode = 1
stdout = ""
stderr = "Unit act-runner.service not found."
monkeypatch.setattr(services_module.subprocess, "run", lambda *_a, **_kw: _R())
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post(
"/services/act-runner/restart", data={"csrf": csrf, "stage": "execute"}
)
assert resp.status_code == 200
assert "error" in resp.text.lower()
entry = _last_audit_entry(config.web.audit_log)
assert entry.result == "error"
# ── 6. ci-web restart -> deferred systemd-run, never a direct restart ──────
def test_ci_web_restart_execute_uses_systemd_run_not_direct_systemctl(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
seen = _quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/ci-web/restart", data={"csrf": csrf, "stage": "execute"})
assert resp.status_code == 200
assert len(seen) == 1
argv = seen[0]
assert "systemd-run" in argv
assert "--on-active=2" in argv
assert argv[0:3] == ["sudo", "-n", "systemd-run"]
entry = _last_audit_entry(config.web.audit_log)
assert entry.result == "ok"
assert entry.action == "service.restart"
assert entry.target == "ci-web.service"
def test_ci_web_restart_preview_never_touches_subprocess(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
seen = _quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/ci-web/restart", data={"csrf": csrf})
assert resp.status_code == 200
assert "Confirm" in resp.text
assert seen == []
# ── 7. vmware-host confirm step: active-VM count warning ───────────────────
def test_vmware_host_confirm_shows_active_vm_warning(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
app.dependency_overrides[get_backend] = lambda: _FakeBackend(running_count=3)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/vmware-host/stop", data={"csrf": csrf})
assert resp.status_code == 200
assert "3 active VM" in resp.text
assert "interrupts running jobs" in resp.text
def test_vmware_host_confirm_no_warning_when_no_active_vms(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
app.dependency_overrides[get_backend] = lambda: _FakeBackend(running_count=0)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/vmware-host/restart", data={"csrf": csrf})
assert resp.status_code == 200
assert "interrupts running jobs" not in resp.text
def test_vmware_host_confirm_start_never_shows_warning_even_with_active_vms(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
app.dependency_overrides[get_backend] = lambda: _FakeBackend(running_count=5)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/vmware-host/start", data={"csrf": csrf})
assert resp.status_code == 200
assert "interrupts running jobs" not in resp.text
def test_vmware_host_confirm_backend_unavailable_degrades_to_no_warning(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Backend missing (``vmrun`` not found) -> 0 active VMs, no warning, no 500."""
_quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
app.dependency_overrides[get_backend] = lambda: None
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/vmware-host/stop", data={"csrf": csrf})
assert resp.status_code == 200
assert "interrupts running jobs" not in resp.text
def test_vmware_host_confirm_backend_exception_degrades_to_no_warning(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
app.dependency_overrides[get_backend] = lambda: _BoomBackend()
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/vmware-host/restart", data={"csrf": csrf})
assert resp.status_code == 200
assert "interrupts running jobs" not in resp.text
def test_non_vmware_host_service_confirm_never_shows_vm_warning(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path)
app = create_app(config)
app.dependency_overrides[get_backend] = lambda: _FakeBackend(running_count=7)
client = TestClient(app)
csrf = _get_csrf(client)
resp = client.post("/services/act-runner/restart", data={"csrf": csrf})
assert resp.status_code == 200
assert "interrupts running jobs" not in resp.text
# ── 8. custom config.web.managed_services override is actually wired in ────
def test_route_uses_configured_managed_services_override(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
seen = _quiet_subprocess_ok(monkeypatch)
custom = (ManagedService("custom-svc", "custom.service", "Custom", True),)
config = _make_config(tmp_path, managed_services=custom)
app = create_app(config)
client = TestClient(app)
csrf = _get_csrf(client)
# The default allow-list's "act-runner" must NOT be reachable once overridden.
resp_default = client.post(
"/services/act-runner/restart", data={"csrf": csrf, "stage": "execute"}
)
assert resp_default.status_code == 400
resp_custom = client.post(
"/services/custom-svc/restart", data={"csrf": csrf, "stage": "execute"}
)
assert resp_custom.status_code == 200
assert seen == [["sudo", "-n", "systemctl", "restart", "custom.service"]]
# ── 9. unauthenticated POST -> redirect to /login, never executes ──────────
def test_unauthenticated_post_redirects_to_login_and_never_runs(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_install_fake_keyring(monkeypatch)
seen = _quiet_subprocess_ok(monkeypatch)
config = _make_config(tmp_path, active=True, allowed_users=["simone"])
app = create_app(config)
client = TestClient(app, follow_redirects=False)
resp = client.post(
"/services/act-runner/restart", data={"csrf": "whatever", "stage": "execute"}
)
assert resp.status_code in (302, 303)
assert "/login" in resp.headers["location"]
assert seen == []
assert not config.web.audit_log.exists()
# ── 10. GET /services partial: state-aware buttons ──────────────────────────
def test_services_partial_all_inactive_shows_start_never_stop(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
# No real systemctl call: force runner_status() to degrade every unit to
# inactive/unknown, deterministically, regardless of the test host.
import ci_orchestrator.operations.host as host_module
monkeypatch.setattr(host_module.shutil, "which", lambda _n: None)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
resp = client.get("/services")
assert resp.status_code == 200
assert "Start" in resp.text
assert ">Stop<" not in resp.text
assert "Restart" in resp.text
def test_services_partial_active_shows_stop_except_for_ci_web(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
import ci_orchestrator.operations.host as host_module
monkeypatch.setattr(host_module.shutil, "which", lambda _n: "/usr/bin/systemctl")
class _R:
def __init__(self, stdout: str) -> None:
self.stdout = stdout
self.returncode = 0
def _run(cmd: list[str], **_kw: Any) -> _R:
if cmd[1] == "is-active":
return _R("active\n")
return _R("running\n")
monkeypatch.setattr(host_module.subprocess, "run", _run)
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
resp = client.get("/services")
assert resp.status_code == 200
assert 'aria-label="Stop Runner"' in resp.text
assert 'aria-label="Stop VMware host"' in resp.text
assert 'aria-label="Stop Web UI"' not in resp.text # ci-web: allow_stop=False
def test_dashboard_includes_services_panel(tmp_path: Path) -> None:
config = _make_config(tmp_path)
app = create_app(config)
client = TestClient(app)
resp = client.get("/")
assert resp.status_code == 200
assert "Services" in resp.text
assert 'id="services-card"' in resp.text
# ── extra: find_service / MANAGED_SERVICES sanity from the route's angle ───
def test_find_service_via_default_allowlist_matches_module_constant() -> None:
from ci_orchestrator.operations.services import find_service
assert find_service("act-runner") is not None
assert find_service("nonexistent-slug") is None
for service in MANAGED_SERVICES:
assert find_service(service.id) == service