feat(b5): add systemd unit files for CI periodic tasks (Linux host)
Implements the repo-side artifacts of Phase B5 (plans/implementation-plan-A-B.md). Creates 5 .service + .timer pairs under deploy/systemd/ that replicate the periodic tasks registered on Windows by scripts/Register-CIScheduledTasks.ps1: - ci-cleanup-orphans -> python -m ci_orchestrator vm cleanup --max-age-hours 6 (every 6h + boot) - ci-retention-policy -> pwsh Invoke-RetentionPolicy.ps1 (daily 03:00) - ci-watch-disk-space -> python -m ci_orchestrator monitor disk (every 15min) - ci-watch-runner-health -> python -m ci_orchestrator monitor runner (every 15min) - ci-backup-template -> pwsh Backup-CITemplate.ps1 (weekly Sun 02:00, NEW) Includes deploy/systemd/README.md with mapping table, install/test/rollback instructions, and PowerShell Core install note for the two PS-only scripts. Plan checklist updated; B5-closeout.md added. Hardware install steps (systemctl enable + smoke) remain user responsibility on the Linux host.
This commit is contained in:
@@ -0,0 +1,105 @@
|
||||
# systemd units — Local CI/CD System (Linux host)
|
||||
|
||||
Coppie `*.service` + `*.timer` che replicano i task periodici registrati su
|
||||
Windows da `scripts/Register-CIScheduledTasks.ps1`. Pensate per essere
|
||||
installate su un host Linux Mint / Ubuntu LTS, con il package
|
||||
`ci_orchestrator` installato in `/opt/ci/venv/`.
|
||||
|
||||
## Mapping Windows Task Scheduler → systemd timer
|
||||
|
||||
| Windows Task Scheduler | systemd unit | Cadenza | Comando |
|
||||
| ----------------------- | ---------------------------------- | ----------------------- | --------------------------------------------------------------- |
|
||||
| `CI-CleanupOrphans` | `ci-cleanup-orphans.timer` | every 6h + at boot | `python -m ci_orchestrator vm cleanup --max-age-hours 6` |
|
||||
| `CI-RetentionPolicy` | `ci-retention-policy.timer` | daily 03:00 + at boot | `pwsh Invoke-RetentionPolicy.ps1` (vedi nota PowerShell) |
|
||||
| `CI-DiskSpaceAlert` | `ci-watch-disk-space.timer` | every 15 min | `python -m ci_orchestrator monitor disk` |
|
||||
| `CI-RunnerHealth` | `ci-watch-runner-health.timer` | every 15 min | `python -m ci_orchestrator monitor runner` |
|
||||
| *(nuovo)* | `ci-backup-template.timer` | weekly Sun 02:00 | `pwsh Backup-CITemplate.ps1` (vedi nota PowerShell) |
|
||||
|
||||
## Nota PowerShell su Linux
|
||||
|
||||
Due unit (`ci-retention-policy`, `ci-backup-template`) invocano script
|
||||
PowerShell che, per scelta documentata in `AGENTS.md` ("Mappatura script
|
||||
PowerShell → Python"), restano in PowerShell. Su Linux questo richiede
|
||||
**PowerShell Core (`pwsh`)**:
|
||||
|
||||
```bash
|
||||
# Ubuntu / Mint LTS
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y wget apt-transport-https software-properties-common
|
||||
wget -q https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/packages-microsoft-prod.deb
|
||||
sudo dpkg -i packages-microsoft-prod.deb
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y powershell
|
||||
pwsh --version # verifica
|
||||
```
|
||||
|
||||
Se preferisci non installare `pwsh`, porta `Invoke-RetentionPolicy.ps1` e
|
||||
`Backup-CITemplate.ps1` come sub-comandi `python -m ci_orchestrator
|
||||
retention run` / `template backup` (out of scope B5 per il piano corrente
|
||||
— vedi `plans/implementation-plan-A-B.md`).
|
||||
|
||||
## Prerequisiti
|
||||
|
||||
1. Utente di sistema `ci-runner` con UID dedicato (creato in B1).
|
||||
2. Venv Python attivo in `/opt/ci/venv/` con `ci_orchestrator` installato:
|
||||
```bash
|
||||
sudo -u ci-runner /opt/ci/venv/bin/pip install -e /opt/ci/local-ci-cd-system
|
||||
```
|
||||
3. Variabili d'ambiente `CI_*` definite in `/etc/ci/environment` (caricato
|
||||
tramite `EnvironmentFile=` in ogni unit `.service`):
|
||||
```
|
||||
CI_ROOT=/var/lib/ci
|
||||
CI_TEMPLATES=/var/lib/ci/templates
|
||||
CI_BUILD_VMS=/var/lib/ci/build-vms
|
||||
CI_ARTIFACTS=/var/lib/ci/artifacts
|
||||
CI_KEYS=/etc/ci/keys
|
||||
PYTHONIOENCODING=utf-8
|
||||
```
|
||||
4. `Invoke-RetentionPolicy.ps1` e `Backup-CITemplate.ps1` presenti in
|
||||
`/opt/ci/local-ci-cd-system/scripts/` (clone del repo).
|
||||
|
||||
## Installazione
|
||||
|
||||
```bash
|
||||
# 1. Copia le unit in /etc/systemd/system/
|
||||
sudo cp deploy/systemd/ci-*.service deploy/systemd/ci-*.timer /etc/systemd/system/
|
||||
|
||||
# 2. Reload systemd
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
# 3. Abilita + avvia tutti i timer
|
||||
for t in ci-cleanup-orphans ci-retention-policy ci-watch-disk-space \
|
||||
ci-watch-runner-health ci-backup-template; do
|
||||
sudo systemctl enable --now "${t}.timer"
|
||||
done
|
||||
|
||||
# 4. Verifica
|
||||
systemctl list-timers --all 'ci-*'
|
||||
```
|
||||
|
||||
## Test e troubleshooting
|
||||
|
||||
```bash
|
||||
# Trigger manuale di un singolo task (esegue il .service una volta)
|
||||
sudo systemctl start ci-cleanup-orphans.service
|
||||
|
||||
# Log dell'ultima esecuzione
|
||||
journalctl -u ci-cleanup-orphans.service -n 50 --no-pager
|
||||
|
||||
# Live tail di tutti i task CI
|
||||
journalctl -u 'ci-*' -f
|
||||
```
|
||||
|
||||
## Rollback
|
||||
|
||||
```bash
|
||||
for t in ci-cleanup-orphans ci-retention-policy ci-watch-disk-space \
|
||||
ci-watch-runner-health ci-backup-template; do
|
||||
sudo systemctl disable --now "${t}.timer"
|
||||
sudo rm -f "/etc/systemd/system/${t}.service" "/etc/systemd/system/${t}.timer"
|
||||
done
|
||||
sudo systemctl daemon-reload
|
||||
```
|
||||
|
||||
A questo punto il fallback è ri-attivare i task su Windows con
|
||||
`scripts/Register-CIScheduledTasks.ps1`.
|
||||
@@ -0,0 +1,26 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — weekly backup of VMware templates
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=ci-runner
|
||||
Group=ci-runner
|
||||
EnvironmentFile=/etc/ci/environment
|
||||
WorkingDirectory=/opt/ci/local-ci-cd-system
|
||||
# NOTE: requires PowerShell Core (`pwsh`) on the Linux host.
|
||||
# See deploy/systemd/README.md "Nota PowerShell su Linux".
|
||||
ExecStart=/usr/bin/pwsh -NoProfile -NonInteractive -File scripts/Backup-CITemplate.ps1
|
||||
TimeoutStartSec=4h
|
||||
IOSchedulingClass=best-effort
|
||||
IOSchedulingPriority=7
|
||||
Nice=19
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
NoNewPrivileges=true
|
||||
ReadWritePaths=/var/lib/ci /var/backups/ci
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,13 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — weekly timer for ci-backup-template.service
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
|
||||
[Timer]
|
||||
# Sunday 02:00 + random delay 1h to avoid clashing with other CI tasks.
|
||||
OnCalendar=Sun *-*-* 02:00:00
|
||||
RandomizedDelaySec=1h
|
||||
Persistent=true
|
||||
Unit=ci-backup-template.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -0,0 +1,23 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — destroy orphaned build VMs (max age 6h)
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=ci-runner
|
||||
Group=ci-runner
|
||||
EnvironmentFile=/etc/ci/environment
|
||||
ExecStart=/opt/ci/venv/bin/python -m ci_orchestrator vm cleanup --max-age-hours 6
|
||||
TimeoutStartSec=1h
|
||||
Nice=10
|
||||
# Hardening
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
NoNewPrivileges=true
|
||||
ReadWritePaths=/var/lib/ci /var/log/ci
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,16 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — periodic timer for ci-cleanup-orphans.service
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
|
||||
[Timer]
|
||||
# Equivalent to Windows Task Scheduler:
|
||||
# New-ScheduledTaskTrigger -Once -At '00:00' -RepetitionInterval 6h
|
||||
# New-ScheduledTaskTrigger -AtStartup
|
||||
OnBootSec=2min
|
||||
OnUnitActiveSec=6h
|
||||
Persistent=true
|
||||
AccuracySec=1min
|
||||
Unit=ci-cleanup-orphans.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -0,0 +1,24 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — purge old artifacts/logs/IP leases (retention policy)
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=ci-runner
|
||||
Group=ci-runner
|
||||
EnvironmentFile=/etc/ci/environment
|
||||
WorkingDirectory=/opt/ci/local-ci-cd-system
|
||||
# NOTE: requires PowerShell Core (`pwsh`) on the Linux host.
|
||||
# See deploy/systemd/README.md "Nota PowerShell su Linux".
|
||||
ExecStart=/usr/bin/pwsh -NoProfile -NonInteractive -File scripts/Invoke-RetentionPolicy.ps1
|
||||
TimeoutStartSec=1h
|
||||
Nice=15
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
NoNewPrivileges=true
|
||||
ReadWritePaths=/var/lib/ci /var/log/ci
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,14 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — daily timer for ci-retention-policy.service
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
|
||||
[Timer]
|
||||
# Equivalent to Windows Task Scheduler:
|
||||
# New-ScheduledTaskTrigger -Daily -At '03:00' -RandomDelay 30min
|
||||
OnCalendar=*-*-* 03:00:00
|
||||
RandomizedDelaySec=30min
|
||||
Persistent=true
|
||||
Unit=ci-retention-policy.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -0,0 +1,20 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — disk space monitor (alert via systemd journal)
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=ci-runner
|
||||
Group=ci-runner
|
||||
EnvironmentFile=/etc/ci/environment
|
||||
ExecStart=/opt/ci/venv/bin/python -m ci_orchestrator monitor disk
|
||||
TimeoutStartSec=5min
|
||||
Nice=15
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
NoNewPrivileges=true
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,15 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — periodic timer for ci-watch-disk-space.service
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
|
||||
[Timer]
|
||||
# Equivalent to Windows Task Scheduler:
|
||||
# New-ScheduledTaskTrigger -Once -At '00:00' -RepetitionInterval 15min
|
||||
OnBootSec=5min
|
||||
OnUnitActiveSec=15min
|
||||
Persistent=true
|
||||
AccuracySec=30s
|
||||
Unit=ci-watch-disk-space.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -0,0 +1,24 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — auto-restart act_runner if stopped (rate-limited)
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
After=network-online.target act-runner.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=ci-runner
|
||||
Group=ci-runner
|
||||
EnvironmentFile=/etc/ci/environment
|
||||
# `monitor runner` enforces internal rate-limit: max 3 restarts/h
|
||||
# (matches Windows CI-RunnerHealth scheduled task semantics).
|
||||
ExecStart=/opt/ci/venv/bin/python -m ci_orchestrator monitor runner
|
||||
TimeoutStartSec=5min
|
||||
Nice=15
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
# NoNewPrivileges intentionally NOT set: monitor may need to call
|
||||
# `systemctl restart act-runner.service` via polkit.
|
||||
ReadWritePaths=/var/lib/ci /var/log/ci
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,15 @@
|
||||
[Unit]
|
||||
Description=Local CI/CD — periodic timer for ci-watch-runner-health.service
|
||||
Documentation=file:///opt/ci/local-ci-cd-system/deploy/systemd/README.md
|
||||
|
||||
[Timer]
|
||||
# Equivalent to Windows Task Scheduler:
|
||||
# New-ScheduledTaskTrigger -Once -At '00:00' -RepetitionInterval 15min
|
||||
OnBootSec=3min
|
||||
OnUnitActiveSec=15min
|
||||
Persistent=true
|
||||
AccuracySec=30s
|
||||
Unit=ci-watch-runner-health.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
Reference in New Issue
Block a user