local-ci-cd-system/docs/WEB-UI.md

Web UI — Management Dashboard

Server-rendered dashboard (ci web serve) for the local CI/CD system: live VMs, cargo cache, disk, and act_runner status, plus confirm-gated destructive actions (VM remove/cleanup, cache purge, template backup) with an audit trail. In-process only — it imports operations/ and backends.load_backend directly and never shells out to the CLI.

See plans/web-ui-management-plan.md (steps W0–W7) for the design history and config.example.toml's [web] / [web.oidc] section for every config key referenced below.

---

0. Current deployment on this host

Live since 2026-07-23: ci-web.service is enabled + running, bound to the LAN (192.168.10.11:8800), authenticating against https://gitea.emulab.it via OIDC. Only the allowlisted Gitea user(s) in /var/lib/ci/config.toml's [web].allowed_users can log in. Manage it exactly as described in §4 below (systemctl status/restart ci-web.service, journalctl -u ci-web.service -f). WebSessionSecret and WebOidcClientSecret are already stored in the ci-runner keyring — no action needed unless they're rotated (§3/§6).

---

1. Prerequisites

• The Linux host already bootstrapped by setup-host-linux.sh (ci-runner service user, /opt/ci/venv, /var/lib/ci/config.toml).
• A reachable Gitea instance to authenticate against (OIDC/OAuth2) — only required if you intend to bind the dashboard on the LAN; a loopback-only install can skip Gitea entirely (see §5, "open mode").
• Outbound network from the host to Gitea's /.well-known/openid-configuration endpoint, if OIDC is enabled.

Nothing here needs internet access — ci_orchestrator[web]'s dependencies (FastAPI, Jinja2, Authlib, uvicorn, ...) install from the same package index already used for the core install, and every frontend asset (web/static/app.css, web/static/htmx.min.js) is vendored in the repo, no CDN.

---

2. Running the installer

cd /opt/ci/local-ci-cd-system
sudo bash setup-web-linux.sh --host 127.0.0.1        # loopback-only, no OIDC needed
# or, once OIDC is registered (see §3):
sudo bash setup-web-linux.sh --host 192.168.1.50      # LAN

setup-web-linux.sh is idempotent — every mutating step checks its own state first and skips (with a message) when already satisfied, so re-running it (e.g. after git pull + a code change) is always safe:

1. pip install '<repo>[web]' into /opt/ci/venv, non-editable — per CLAUDE.md, the production venv can't resolve editable installs when act_runner/systemd launches Python from a transient working directory.
2. Renders deploy/systemd/ci-web.service into /etc/systemd/system/, substituting the bind IP you passed via --host.
3. Appends a commented [web] / [web.oidc] template to /var/lib/ci/config.toml if no such section exists yet — it never overwrites an existing one, and never writes a secret value.
4. Prompts (interactively, one at a time) to store WebSessionSecret and WebOidcClientSecret in the keyring, if they aren't already set. The secret is only ever typed at the terminal prompt — the installer never accepts it as a command-line argument and never echoes or logs it.
5. Prints the manual Gitea OIDC application registration steps (see §3).
6. systemctl daemon-reload then enables + starts (or restarts, if already running) ci-web.service.

Run setup-web-linux.sh --help for the full flag list (--skip-pip, --skip-secrets, --config-path).

After step 3/4, edit /var/lib/ci/config.toml: uncomment the appended [web] / [web.oidc] block, fill in issuer, client_id, allowed_users, then restart the service (systemctl restart ci-web.service).

---

3. Gitea OAuth2 application registration

The dashboard authenticates against Gitea via OIDC (Authlib + Gitea's /.well-known/openid-configuration discovery document). Register the application before enabling [web.oidc]:

1. In Gitea: Settings → Applications → "Manage OAuth2 Applications" → Create a new OAuth2 Application.
2. Application Name: anything recognizable, e.g. ci-web.
3. Redirect URI: http://<LAN-IP>:8800/auth/callback — this must match [web.oidc].redirect_url exactly (scheme, host, port, and path). A mismatch is the #1 cause of a login redirect loop (see §6).
4. Gitea shows a Client ID and Client Secret exactly once:
   • Client ID → [web.oidc].client_id in config.toml (not a secret, plain text is fine).
   • Client Secret → never put it in config.toml. Store it in the keyring instead:

     sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator creds set \
         --target WebOidcClientSecret --user web-oidc-client
     

     config.toml only ever holds the keyring key name (client_secret_cred = "WebOidcClientSecret"), never the value.
5. Add the Gitea username(s) allowed to log in to [web].allowed_users. The allowlist is fail-closed: an empty list denies everyone, it never means "allow all".

---

4. Operating

# Start / stop / restart
sudo systemctl start ci-web.service
sudo systemctl stop ci-web.service
sudo systemctl restart ci-web.service     # after a config.toml edit or a code upgrade

# Status
systemctl status ci-web.service

# Logs (uvicorn access log + app stdout/stderr go to the journal)
journalctl -u ci-web.service -f
journalctl -u ci-web.service -n 100 --no-pager

# Health check (unauthenticated, always 200)
curl -s http://127.0.0.1:8800/healthz

Once code changes: sudo bash setup-web-linux.sh --host <same-as-before> reinstalls the [web] extra and restarts the service in one step (steps 2–6 no-op if nothing else changed).

---

5. Firewall / security

• Bind only a specific LAN IP, never 0.0.0.0. Neither ci web serve nor setup-web-linux.sh will ever write 0.0.0.0 — the installer prompts for (or accepts via --host) one explicit address, defaulting to 127.0.0.1.
• Open mode (no [web.oidc] configured) is localhost-only, enforced at startup. ci web serve refuses to start if you bind a non-loopback address while [web.oidc] is absent — an unauthenticated dashboard that can destroy VMs and purge caches must never be reachable from the LAN. Configure [web.oidc] first, or bind 127.0.0.1.
• Every mutating route (VM remove/cleanup, cache purge, template backup) is behind a synchronizer-token CSRF check and a two-step confirm gate (a preview/dry-run, then an explicit "Confirm" that actually executes) — there is no one-click destroy.
• Every mutating action appends exactly one entry to the audit log ([web].audit_log, default /var/lib/ci/logs/web-audit.jsonl), viewable at /audit in the dashboard.
• If you run a host firewall (ufw/nftables/iptables), only open the chosen port (default 8800) to the specific LAN/VLAN that needs dashboard access, not the whole network.

---

6. Keyring / headless note

ci-web.service runs as ci-runner under systemd — there is no desktop session and no D-Bus, so the GNOME Secret Service backend (secret-tool) does not work here. This host uses keyrings.alt.file.PlaintextKeyring instead (already the credential backend used for guest/Gitea credentials — see plans/archived/2026-05-23/B3-closeout.md):

• Config: /home/ci-runner/.local/share/python_keyring/keyringrc.cfg ([backend] default-keyring=keyrings.alt.file.PlaintextKeyring)
• Store: /home/ci-runner/.local/share/python_keyring/keyring_pass.cfg — plain text, readable only by ci-runner (file permissions), no D-Bus session required. Verified to work both from sudo -u ci-runner ... and from inside a plain systemd service.
• This is why ci-web.service's hardening uses ProtectHome=read-only and not ProtectHome=yes/true: the latter hides /home entirely from the unit's mount namespace, which would break every OIDC/session secret lookup and make login fail even though the keyring itself is fine. read-only keeps /home/ci-runner readable (the app never writes to the keyring at request time — secrets are set once, out-of-band, by ci creds set) while still blocking writes.
• WebSessionSecret / WebOidcClientSecret are keyring key names in config.toml (session_secret_cred, client_secret_cred) — the actual values live only in the keyring, set via ci creds set --target <name> --user <label> (interactive prompt, hide_input + confirmation; never a CLI argument, never logged).

---

7. Service management (Stage D)

The dashboard's Services panel lets an authenticated web user start, stop, or restart a fixed allow-list of systemd units on the host — the same confirm-gated, CSRF-protected, audited mutating-action shape as VM remove/cleanup, cache purge, and template backup (§5), applied to sudo systemctl.

Allow-list (default, operations/services.py::MANAGED_SERVICES):

Slug	Unit	Stop allowed?
act-runner	act-runner.service	yes
act-runner-rust	act-runner-rust.service	yes
vmware-host	vmware-host.service	yes
ci-web	ci-web.service	no — restart-only

Override it with [web].managed_services in config.toml (see config.example.toml's commented example) — a list of {id, unit, label, allow_stop} tables. A malformed row is skipped, never crashes startup.

Security model

• Allow-list by slug, not by unit name. The URL is POST /services/{service_id}/{action} where service_id is one of the slugs above (find_service()), never a raw systemd unit name — there is no code path that builds a controllable unit from unvalidated request input.
• Exact-command sudoers, no wildcards. deploy/sudoers.d/ci-web grants ci-runner passwordless sudo for exactly nine literal command lines (one per unit × action, plus the ci-web restart special case) — no bare systemctl, no *, no ALL in the command position. Even if the web app itself were compromised, ci-runner's sudo grant cannot be used to run any command beyond those nine lines.
• Arg list, never a shell. operations/services.py::control_service builds ["sudo", "-n", "systemctl", action, unit] (or the systemd-run variant below) as a Python list handed straight to subprocess.run — shell=True is never used anywhere in this path.
• ci-web is restart-only. allow_stop=False — a stop request against it is refused with a 400 before any subprocess or audit entry, both at the route (web/routes.py::services_control) and defensively again inside control_service itself. Rationale: stopping the service that renders this very panel would lock the operator out with no way to start it again from the UI.
• ci-web restart is deferred. Restarting ci-web.service from inside its own worker would kill the HTTP response before it could be sent, so that one case runs sudo -n systemd-run --on-active=2 systemctl restart ci-web.service instead of a direct systemctl restart — the systemd-run call itself returns almost immediately (it only schedules the restart ~2s out), so the confirming request completes normally and the dashboard briefly drops a couple of seconds later.
• vmware-host stop/restart interrupts running jobs. The confirm step shows the current count of active VMs ("N active VM(s) — this interrupts running jobs") when stopping or restarting vmware-host.service, computed from the same VM backend the Live VMs panel uses — best-effort, degrades to no warning if the backend is unavailable, never blocks the action.
• Every action is audited exactly once per stage (service.start / service.stop / service.restart, target = the unit name), same as every other mutating route — visible at /audit.

Sudoers install

setup-web-linux.sh installs deploy/sudoers.d/ci-web to /etc/sudoers.d/ci-web (mode 0440, owner root:root) as step 6/7, validating it with visudo -cf and removing it again if validation fails — a broken sudoers.d fragment can lock out sudo entirely, so this step never leaves an unvalidated file in place. Before installing on a new host, confirm systemctl/systemd-run are actually at /usr/bin on that host (command -v systemctl systemd-run) — edit both deploy/sudoers.d/ci-web and the installed copy if not.

To install just the sudoers grant (e.g. after editing deploy/sudoers.d/ci-web) without touching anything else:

sudo bash setup-web-linux.sh --skip-pip --skip-secrets

Deploy note

After merging a Stage D change: re-run pip install . into the production venv (per CLAUDE.md), re-run setup-web-linux.sh (installs/updates the sudoers grant and restarts ci-web.service), then confirm the sudoers file on the live host with sudo visudo -c before relying on the panel.

---

8. Troubleshooting

Login loop (bounces back to /login after authenticating with Gitea)

The registered Redirect URI in Gitea does not exactly match [web.oidc].redirect_url in config.toml — check scheme (http vs https), host/IP, port, and the /auth/callback path character-for-character.

403 after a successful Gitea login

Your Gitea username is not in [web].allowed_users (or the list is empty — the allowlist is fail-closed by design). Add it and restart the service.

**"keyring key '...' not found -- run ci creds set ..." in the journal /

on /login**

The session secret or the OIDC client secret has not been stored yet (or was stored for a different Unix user than ci-runner, or under a different _cred name than the one in config.toml). Re-run:

sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator creds set \
    --target WebSessionSecret --user web-session

substituting the exact key name printed in the error. See §6 for why this must run as ci-runner, not as your own login.

Dashboard is blank / cards show "unavailable"

The VM backend (vmrun) or shutil.disk_usage/systemctl could not be reached from the host — the dashboard degrades a card to an "unavailable" notice instead of a 500. Check vmware-host.service is up (systemctl is-active vmware-host.service) and that /healthz still returns 200.

**ci web serve refuses to start with "refusing to bind ... without

authentication"**

You passed a non-loopback --host (or [web].bind_host in config.toml) while [web.oidc] is unset. This is the guard described in §5 — configure [web.oidc] (see §3) or bind 127.0.0.1.

Confirm/Cleanup/Purge form returns 403

The CSRF token expired (the session cookie was cleared, e.g. by /logout, or the browser tab was left open across a service restart that rotated the session secret in open mode). Reload the page and retry — the confirm form always mints a fresh token.