From 9688e7e2f8e60c3ba7e706d226e3253da3818d72 Mon Sep 17 00:00:00 2001 From: Simone Date: Thu, 21 May 2026 23:35:43 +0200 Subject: [PATCH] fix(backup): bring act-runner back up on every termination path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit If ci-backup-template.service is killed mid-run (SIGTERM), the Python finally block that restarts act-runner.service is skipped. A subsequent backup then finds the runner already inactive, _stop_runner returns False, and the finally restart is gated off — leaving the runner down indefinitely (incident: 21 mag 2026, runner stayed off ~80 min). Add ExecStopPost=-/bin/systemctl start act-runner.service so systemd guarantees the runner is restarted regardless of how the unit exits (success, failure, signal). The `-` prefix preserves the unit's own exit status. Co-Authored-By: Claude Sonnet 4.6 --- deploy/systemd/ci-backup-template.service | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/deploy/systemd/ci-backup-template.service b/deploy/systemd/ci-backup-template.service index dcf5311..460758a 100644 --- a/deploy/systemd/ci-backup-template.service +++ b/deploy/systemd/ci-backup-template.service @@ -9,6 +9,11 @@ User=root EnvironmentFile=/etc/ci/environment WorkingDirectory=/opt/ci/local-ci-cd-system ExecStart=/opt/ci/venv/bin/python -m ci_orchestrator template backup --all-templates +# Safety net: bring act-runner.service back up on any exit path (success, +# failure, signal). The Python `finally` block can be skipped if the +# service is killed mid-run, and a runner left stopped by a killed +# backup would cause the next backup to skip the restart too. +ExecStopPost=-/bin/systemctl start act-runner.service TimeoutStartSec=4h IOSchedulingClass=best-effort IOSchedulingPriority=7