Thread entry furyrs#12 measured where a 133s Rust job actually goes: ~120s is compiling the ~490 external dependencies, which a committed Cargo.lock keeps bit-identical between runs. The workspace crates cost 7s. #13 settled the policy -- reuse dependency artifacts, never workspace ones, and keep v* tags cold. The plan departs from #12's cargo-chef shape for a concrete reason: the orchestrator receives build-command as an opaque string (furyrs runs six cargo invocations across three feature sets) and `cargo chef cook` must be handed the same flags as the real build. Rather than guess them, invert the order -- build normally, then prune the workspace members' artifacts out of target/ and publish the remainder. No knowledge of the command, one entry holding every feature graph, no new tool in the template. That makes the safety property checkable rather than argued: after the prune, an assertion looks for any surviving workspace fingerprint and the job refuses to publish if it finds one. A missing entry costs 120s; a poisoned one costs a wrong verdict, which is the worst failure a build system has because it is silent. Store is one immutable file per key with an atomic rename, so there is no shared mutable directory and no advisory locking -- the objection furyrs raised about VMware Shared Folders under concurrency. Retention of the store is in the same plan: adding an unbounded directory without it would repeat TODO 2.9 at a larger scale.
53 KiB
Cargo Dependency Cache — Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Cut ~120s off every Rust CI job on LinuxRust2404 by reusing the compiled dependency artifacts between runs, while making it structurally impossible for a workspace crate's artifacts to survive into the cache.
Architecture: After a successful build, prune every workspace-member artifact out of the guest's target/, assert none survived, and publish what remains to a host-side content-addressed store keyed by sha256(Cargo.lock) + rustc version + host triple. The next job with the same key restores that tarball into its clone before building. Entries are immutable — one file per key, published by atomic rename(2) from a single writer — so there is no shared mutable directory and no locking.
Tech Stack: Python 3.12 (click, paramiko via the existing SshTransport), POSIX shell in the guest, tar + zstd/gzip, pytest.
Global Constraints
click.echoeverywhere — no bareprint().- No
subprocess.run(..., shell=True). - No global variables — state lives in
Configand objects passed explicitly. - All public functions and exported helpers have type hints.
mypy --strict srcmust pass. ruff check src tests/pythonmust pass.- Coverage gate: ≥90% on
ci_orchestrator. - Application code imports
backends.load_backend, neverWorkstationVmrunBackend. - Do not touch the Windows/WinRM branch in any commit that touches the Linux/SSH branch (
AGENTS.md§"Errori frequenti" #4). - Cache store lives under
config.paths.root(/var/lib/cion the Linux host), owned byci-runner.
Design decisions, and where they depart from thread entry #12
Entry #12 proposed the cargo-chef principle: pre-build a dependency-only
target/ from a skeleton workspace, key it on `digest(Cargo.lock) + toolchain
- feature set`. Two things forced a different mechanism.
1. The build command is opaque to us. The orchestrator receives
build-command as a single string and runs it. furyrs's pipeline is five
distinct cargo invocations with three different feature sets. cargo chef cook
must be handed the same flags as the real build to produce the right dependency
graph — we do not have them, and parsing an arbitrary shell string to recover
them is a guess that fails silently (a wrong cook produces a cache that misses
every time, costing time instead of saving it).
2. Pruning after the fact needs no such knowledge. Whatever cargo
invocations ran, every artifact they produced is in target/. Delete the
workspace members' artifacts and what remains is exactly the dependency set —
for all feature variants at once, in one entry. This is the mechanism
Swatinem/rust-cache uses in the wild.
Consequences, all favourable:
| #12's proposal | This plan | |
|---|---|---|
| Needs to know the cargo flags | yes | no |
| Entries per job | one per feature set | one, holding every variant cargo actually built |
feature set in the cache key |
required | dropped — cargo already encodes the feature set in each artifact's own -<hash> suffix, so variants coexist without colliding |
| New tool in the template | cargo-chef |
none |
The safety property furyrs asked for is unchanged and is now checkable: the cache must contain no workspace-crate artifact. The prune step is followed by an assertion that no fingerprint directory for any workspace member survives. If one does, the job refuses to publish and says so. Fail closed: a missing cache entry costs 120 seconds, a poisoned one costs a wrong verdict.
Scope gate. The cache engages only when Cargo.lock exists in the build
workdir. Non-Rust jobs are untouched, with no flag to set.
Cold on release tags. refs/tags/v* builds run with the cache disabled, per
furyrs's constraint in #13: those verify a release artifact, not a change, and
the clean room is the point.
File Structure
| File | Responsibility |
|---|---|
src/ci_orchestrator/cargo_cache.py |
Create. Pure host-side logic: key derivation from parts, store paths, lookup, atomic publish, age-based prune. No SSH, no click, no I/O beyond the store directory. |
src/ci_orchestrator/assets/cargo-cache-prune.sh |
Create. Guest-side prune + fail-closed assertion. Shipped per job over SCP, not baked into the template, so changing it needs no template rebuild. It lives inside the package and is read through importlib.resources, following winimage/assets/: the production venv is installed non-editable, so anything resolved relative to the repo root is absent at runtime. |
src/ci_orchestrator/assets/__init__.py |
Create. Empty — makes the asset directory an importable package so importlib.resources can find it. |
tests/python/test_cargo_cache.py |
Create. Unit tests for the store module. |
src/ci_orchestrator/config.py |
Modify. [cargo_cache] section → CargoCacheConfig. |
src/ci_orchestrator/commands/build.py |
Modify. _linux_build gains restore-before / save-after. Windows branch untouched. |
src/ci_orchestrator/commands/job.py |
Modify. --cargo-cache/--no-cargo-cache, plumbed to _linux_build. |
src/ci_orchestrator/commands/retention.py |
Modify. Purge cache entries older than the retention window. |
.gitea/actions/local-ci-build/action.yml |
Modify. cargo-cache input (auto/on/off), auto = off for refs/tags/v*. |
docs/RUNBOOK-PhaseC.md |
Modify. Operator section: where the store lives, how to read a miss, how to purge. |
Task 0: Measure before building any of it
No production code. Entry #12 set the standard — the registry cache was abandoned because measurement showed it bought 9.3s for +800 MB. Do the same here before committing to the transfer cost.
Files:
-
Create:
plans/cargo-dependency-cache-measurements.md -
Step 1: Clone the Rust template and run furyrs's pipeline once, cold
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator vm new \
--template /var/lib/ci/templates/LinuxRust2404/LinuxRust2404.vmx \
--snapshot BaseClean-LinuxRust --job-id cachemeasure
# note the clone VMX path it prints, then:
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator wait-ready \
--vmx <clone.vmx> --guest-os linux
Inside the guest, clone furyrs and run its ci.yml build command to completion.
- Step 2: Record the five numbers that decide the design
ssh -i /var/lib/ci/keys/ci_linux ci_build@<ip> '
du -sb /opt/ci/build/target # 1. target/ raw
time tar -C /opt/ci/build -c target | wc -c # 2. tar cost + size
time tar -C /opt/ci/build -c target | zstd -3 | wc -c # 3. zstd -3 cost + size
command -v zstd || echo "ZSTD ABSENT" # 4. is zstd even present?
nproc; free -m'
Then, host side, time the transfer of a file that size over SCP to the guest.
- Step 3: Write the numbers down and state the go/no-go
Record in plans/cargo-dependency-cache-measurements.md: raw size, pruned size,
compressed size, compress time, transfer time, extract time. The feature is
worth building if restore + extract is comfortably under the ~120s it saves.
If zstd is absent, note it — Task 3 falls back to gzip and Task 3 Step 7
adds zstd to the toolchain script for the next rebuild.
- Step 4: Destroy the clone
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator vm remove --vmx <clone.vmx>
- Step 5: Commit
git add plans/cargo-dependency-cache-measurements.md
git commit -m "docs(cache): measure the cargo target/ cache before building it"
Task 1: The store — key derivation and atomic publish
Files:
- Create:
src/ci_orchestrator/cargo_cache.py - Test:
tests/python/test_cargo_cache.py
Interfaces:
-
Consumes: nothing from earlier tasks.
-
Produces, all used by Task 3 and Task 6:
CACHE_FORMAT: int— bump to invalidate every entry at once.cache_key(*, lock_digest: str, rustc_version: str, host_triple: str) -> strclass CargoCacheStorewith__init__(self, root: Path, *, suffix: str = ".tar.zst") -> None,path_for(self, key: str) -> Path,lookup(self, key: str) -> Path | None,publish(self, key: str, source: Path) -> Path,prune_older_than(self, cutoff: datetime, *, what_if: bool = False) -> list[Path]
-
Step 1: Write the failing tests
"""Tests for ci_orchestrator.cargo_cache — key derivation and the entry store."""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from pathlib import Path
import pytest
from ci_orchestrator.cargo_cache import CACHE_FORMAT, CargoCacheStore, cache_key
# ── key derivation ───────────────────────────────────────────────────────────
def test_key_is_stable_for_identical_inputs() -> None:
a = cache_key(lock_digest="abc", rustc_version="rustc 1.97.1", host_triple="x86_64")
b = cache_key(lock_digest="abc", rustc_version="rustc 1.97.1", host_triple="x86_64")
assert a == b
def test_key_changes_with_lock_digest() -> None:
a = cache_key(lock_digest="abc", rustc_version="rustc 1.97.1", host_triple="x86_64")
b = cache_key(lock_digest="def", rustc_version="rustc 1.97.1", host_triple="x86_64")
assert a != b
def test_key_changes_with_toolchain() -> None:
a = cache_key(lock_digest="abc", rustc_version="rustc 1.97.1", host_triple="x86_64")
b = cache_key(lock_digest="abc", rustc_version="rustc 1.98.0", host_triple="x86_64")
assert a != b
def test_key_is_filesystem_safe() -> None:
key = cache_key(
lock_digest="a/b+c=", rustc_version="rustc 1.97.1 (x /y)", host_triple="x86_64"
)
assert key.isalnum() or all(c.isalnum() or c == "-" for c in key)
assert "/" not in key
def test_key_carries_the_format_version() -> None:
key = cache_key(lock_digest="abc", rustc_version="r", host_triple="t")
assert key.startswith(f"v{CACHE_FORMAT}-")
# ── store ────────────────────────────────────────────────────────────────────
def test_lookup_returns_none_when_absent(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
assert store.lookup("v1-deadbeef") is None
def test_publish_then_lookup_finds_the_entry(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
src = tmp_path / "incoming.tar.zst"
src.write_bytes(b"payload")
published = store.publish("v1-deadbeef", src)
assert store.lookup("v1-deadbeef") == published
assert published.read_bytes() == b"payload"
def test_publish_consumes_the_source_file(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
src = tmp_path / "incoming.tar.zst"
src.write_bytes(b"payload")
store.publish("v1-deadbeef", src)
assert not src.exists(), "publish must move, not copy — a copy doubles peak disk"
def test_publish_overwrites_an_existing_key_atomically(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
first = tmp_path / "a.tar.zst"
first.write_bytes(b"first")
store.publish("v1-key", first)
second = tmp_path / "b.tar.zst"
second.write_bytes(b"second")
published = store.publish("v1-key", second)
assert published.read_bytes() == b"second"
def test_publish_leaves_no_temp_files_behind(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
src = tmp_path / "incoming.tar.zst"
src.write_bytes(b"payload")
store.publish("v1-key", src)
leftovers = list((tmp_path / ".incoming").glob("*")) if (tmp_path / ".incoming").is_dir() else []
assert leftovers == []
def test_publish_missing_source_raises(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
with pytest.raises(FileNotFoundError):
store.publish("v1-key", tmp_path / "nope.tar.zst")
def test_lookup_rejects_a_key_with_a_path_separator(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
with pytest.raises(ValueError, match="unsafe cache key"):
store.lookup("../../etc/passwd")
# ── prune ────────────────────────────────────────────────────────────────────
def _age(path: Path, days: int) -> None:
old = (datetime.now(tz=UTC) - timedelta(days=days)).timestamp()
import os
os.utime(path, (old, old))
def test_prune_removes_entries_older_than_the_cutoff(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
src = tmp_path / "old.tar.zst"
src.write_bytes(b"x")
entry = store.publish("v1-old", src)
_age(entry, days=40)
removed = store.prune_older_than(datetime.now(tz=UTC) - timedelta(days=30))
assert removed == [entry]
assert not entry.exists()
def test_prune_keeps_fresh_entries(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
src = tmp_path / "new.tar.zst"
src.write_bytes(b"x")
entry = store.publish("v1-new", src)
removed = store.prune_older_than(datetime.now(tz=UTC) - timedelta(days=30))
assert removed == []
assert entry.exists()
def test_prune_what_if_reports_without_deleting(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path)
src = tmp_path / "old.tar.zst"
src.write_bytes(b"x")
entry = store.publish("v1-old", src)
_age(entry, days=40)
removed = store.prune_older_than(
datetime.now(tz=UTC) - timedelta(days=30), what_if=True
)
assert removed == [entry]
assert entry.exists()
def test_prune_on_a_missing_store_is_a_noop(tmp_path: Path) -> None:
store = CargoCacheStore(tmp_path / "never-created")
assert store.prune_older_than(datetime.now(tz=UTC)) == []
- Step 2: Run the tests to verify they fail
Run: .venv/bin/python -m pytest tests/python/test_cargo_cache.py -q
Expected: collection error — ModuleNotFoundError: No module named 'ci_orchestrator.cargo_cache'
- Step 3: Write the implementation
"""Content-addressed store for compiled Rust dependency artifacts.
A cache entry is one immutable file per key. Publishing is a ``rename(2)``
within the store directory, so a reader either sees the previous entry or the
new one and never a half-written file — no advisory locking, which is exactly
what a VMware shared folder cannot give us with several jobs in flight.
Nothing here knows about SSH or guests: it takes a finished tarball and files
it, or hands back the path of an existing one. That keeps the interesting
logic testable without a VM.
"""
from __future__ import annotations
import hashlib
import os
import re
from datetime import UTC, datetime
from pathlib import Path
# Bump to invalidate every entry at once — e.g. when the prune script changes
# what it strips, making older entries untrustworthy.
CACHE_FORMAT = 1
_SAFE_KEY = re.compile(r"^v\d+-[0-9a-f]{32,64}$")
_INCOMING = ".incoming"
def cache_key(*, lock_digest: str, rustc_version: str, host_triple: str) -> str:
"""Derive the entry key from the inputs that make artifacts incompatible.
``lock_digest`` is the sha256 of ``Cargo.lock``: it changes exactly when
the dependency set changes, which is the property that makes this cache
sound. The feature set is deliberately *not* part of the key — cargo
encodes it in each artifact's own hash suffix, so several feature graphs
coexist inside one entry instead of fragmenting it.
"""
digest = hashlib.sha256(
"\0".join((lock_digest, rustc_version, host_triple)).encode("utf-8")
).hexdigest()
return f"v{CACHE_FORMAT}-{digest}"
class CargoCacheStore:
"""A directory of immutable cache entries, one file per key."""
def __init__(self, root: Path, *, suffix: str = ".tar.zst") -> None:
self.root = Path(root)
self.suffix = suffix
# ── paths ────────────────────────────────────────────────────────────
def _checked(self, key: str) -> str:
if not _SAFE_KEY.match(key):
raise ValueError(f"unsafe cache key: {key!r}")
return key
def path_for(self, key: str) -> Path:
return self.root / f"{self._checked(key)}{self.suffix}"
# ── read ─────────────────────────────────────────────────────────────
def lookup(self, key: str) -> Path | None:
"""Return the entry path for ``key``, or ``None`` on a miss."""
candidate = self.path_for(key)
return candidate if candidate.is_file() else None
# ── write ────────────────────────────────────────────────────────────
def publish(self, key: str, source: Path) -> Path:
"""Move ``source`` into the store under ``key``. Returns the entry path.
The move goes through a staging directory on the same filesystem so the
final step is a rename: concurrent publishers of the same key both
produce a valid entry and the last one wins, with no window in which a
reader can observe a partial file.
"""
source = Path(source)
if not source.is_file():
raise FileNotFoundError(f"nothing to publish: {source}")
target = self.path_for(key)
staging = self.root / _INCOMING
staging.mkdir(parents=True, exist_ok=True)
tmp = staging / f"{key}.{os.getpid()}{self.suffix}"
try:
os.replace(source, tmp)
except OSError:
# Different filesystem — fall back to a copy into staging, which
# still keeps the rename below atomic.
import shutil
shutil.copy2(source, tmp)
source.unlink(missing_ok=True)
os.replace(tmp, target)
return target
# ── maintenance ──────────────────────────────────────────────────────
def prune_older_than(
self, cutoff: datetime, *, what_if: bool = False
) -> list[Path]:
"""Delete entries last modified before ``cutoff``. Returns what went."""
if not self.root.is_dir():
return []
removed: list[Path] = []
for entry in sorted(self.root.glob(f"*{self.suffix}")):
if not entry.is_file():
continue
mtime = datetime.fromtimestamp(entry.stat().st_mtime, tz=UTC)
if mtime >= cutoff:
continue
removed.append(entry)
if not what_if:
entry.unlink(missing_ok=True)
return removed
__all__ = ["CACHE_FORMAT", "CargoCacheStore", "cache_key"]
- Step 4: Run the tests to verify they pass
Run: .venv/bin/python -m pytest tests/python/test_cargo_cache.py -q
Expected: all pass.
- Step 5: Lint and type-check
Run: .venv/bin/python -m ruff check src tests/python && .venv/bin/python -m mypy --strict src
Expected: All checks passed! and Success: no issues found.
- Step 6: Commit
git add src/ci_orchestrator/cargo_cache.py tests/python/test_cargo_cache.py
git commit -m "feat(cache): content-addressed store for cargo dependency artifacts"
Task 2: The guest-side prune, and the assertion that makes it safe
Files:
- Create:
src/ci_orchestrator/assets/cargo-cache-prune.sh - Create:
src/ci_orchestrator/assets/__init__.py(empty)
Interfaces:
-
Consumes: nothing.
-
Produces: a script invoked by Task 3 as
bash cargo-cache-prune.sh <target-dir> <manifest-path>, exit 0 on a clean prune, exit 2 when a workspace artifact survived (caller must not publish), exit 1 on a usage/environment error. Task 3 locates it withimportlib.resources.as_file(resources.files("ci_orchestrator.assets").joinpath("cargo-cache-prune.sh")). -
Step 0: Create the asset package
mkdir -p src/ci_orchestrator/assets
touch src/ci_orchestrator/assets/__init__.py
[tool.hatch.build.targets.wheel] already declares packages = ["src/ci_orchestrator"], so a sub-package with a data file is included in the
wheel without further configuration — the same route
winimage/assets/autounattend.win2025.xml takes.
- Step 1: Write the script
#!/usr/bin/env bash
# Strip every workspace-crate artifact out of target/, then prove none survived.
#
# The cache is only sound while it contains dependency artifacts alone. A
# workspace crate's artifact carries the code under test, and reusing one can
# turn a red build green -- the worst failure a CI system has, because it is
# silent. So this script ends by asserting its own work and exits non-zero if
# the assertion fails; the caller must treat that as "do not publish".
set -euo pipefail
TARGET_DIR="${1:?usage: cargo-cache-prune.sh <target-dir> <manifest-path>}"
MANIFEST="${2:?usage: cargo-cache-prune.sh <target-dir> <manifest-path>}"
[ -d "$TARGET_DIR" ] || { echo "prune: no such target dir: $TARGET_DIR" >&2; exit 1; }
command -v cargo >/dev/null || { echo "prune: cargo not on PATH" >&2; exit 1; }
# --no-deps restricts the listing to workspace members, which is precisely the
# set whose artifacts must never be cached.
members="$(cargo metadata --no-deps --format-version 1 --manifest-path "$MANIFEST" \
| python3 -c 'import json,sys; print("\n".join(p["name"] for p in json.load(sys.stdin)["packages"]))')"
if [ -z "$members" ]; then
echo "prune: cargo metadata listed no workspace members" >&2
exit 1
fi
# Incremental state is large, machine-specific and never reusable across
# clones; examples and docs are not dependencies of anything we build.
rm -rf "$TARGET_DIR"/*/incremental "$TARGET_DIR"/*/examples "$TARGET_DIR"/doc 2>/dev/null || true
prune_one() {
name="$1"
snake="${name//-/_}"
# Fingerprints and build-script output live in per-crate directories.
find "$TARGET_DIR" -maxdepth 3 \
\( -path "*/.fingerprint/$name-*" \
-o -path "*/.fingerprint/$snake-*" \
-o -path "*/build/$name-*" \
-o -path "*/build/$snake-*" \) \
-prune -exec rm -rf {} + 2>/dev/null || true
# Compiled outputs: rlib/rmeta/d in deps/, plus binaries at the profile root.
find "$TARGET_DIR" -maxdepth 3 \
\( -name "$snake-*" -o -name "lib$snake-*" -o -name "lib$snake.*" \
-o -name "$name" -o -name "$name-*" \) \
-exec rm -rf {} + 2>/dev/null || true
}
while IFS= read -r name; do
[ -n "$name" ] || continue
prune_one "$name"
done <<EOF
$members
EOF
# The assertion. A surviving fingerprint means cargo would consider that
# workspace crate already built and skip recompiling it on restore.
leaked=""
while IFS= read -r name; do
[ -n "$name" ] || continue
snake="${name//-/_}"
hit="$(find "$TARGET_DIR" -maxdepth 3 \
\( -path "*/.fingerprint/$name-*" -o -path "*/.fingerprint/$snake-*" \) \
-print -quit 2>/dev/null || true)"
[ -z "$hit" ] || leaked="$leaked $name"
done <<EOF
$members
EOF
if [ -n "$leaked" ]; then
echo "prune: REFUSING to publish -- workspace artifacts survived:$leaked" >&2
exit 2
fi
echo "prune: ok -- $(printf '%s\n' "$members" | wc -l) workspace member(s) stripped"
- Step 2: Make it executable and shellcheck it
chmod +x src/ci_orchestrator/assets/cargo-cache-prune.sh
shellcheck src/ci_orchestrator/assets/cargo-cache-prune.sh || true # advisory; SC2044 on find is expected
- Step 3: Prove the assertion actually fires
This is the one guard the whole feature's safety rests on, so exercise it against a real cargo workspace rather than trusting the code reading. In a LinuxRust2404 clone (reuse the Task 0 procedure):
# after a real build of the furyrs workspace:
bash cargo-cache-prune.sh /opt/ci/build/target /opt/ci/build/Cargo.toml
echo "exit=$?" # expect 0, and a "prune: ok" line
# now re-create one workspace fingerprint by hand and re-run
mkdir -p /opt/ci/build/target/debug/.fingerprint/fury-core-deadbeef
bash cargo-cache-prune.sh /opt/ci/build/target /opt/ci/build/Cargo.toml
echo "exit=$?" # expect 2, with "REFUSING to publish"
Both outcomes must be observed before this task is considered done. If the second run exits 0, the prune glob is deleting the directory it is supposed to detect — widen the assertion, do not narrow the test.
- Step 4: Verify the dependency artifacts survived
ls /opt/ci/build/target/*/deps/ | head # expect many libserde-*, libtokio-* etc.
du -sh /opt/ci/build/target # record; compare against Task 0's raw size
Expected: dependency .rlib/.rmeta files are still present. A prune that
emptied deps/ is a bug — record the pruned size in
plans/cargo-dependency-cache-measurements.md.
- Step 5: Commit
git add src/ci_orchestrator/assets/ plans/cargo-dependency-cache-measurements.md
git commit -m "feat(cache): guest-side prune of workspace artifacts, with a fail-closed assert"
Task 3: Restore before the build, publish after it
Files:
- Modify:
src/ci_orchestrator/commands/build.py(_linux_buildonly — leave_windows_buildalone) - Test:
tests/python/test_commands_build.py
Interfaces:
-
Consumes:
CargoCacheStore,cache_key,CACHE_FORMATfrom Task 1;ci_orchestrator/assets/cargo-cache-prune.shfrom Task 2. -
Produces:
_linux_build(..., cargo_cache_root: Path | None = None)—Nonedisables the feature entirely, which is the default so no existing caller changes behaviour. -
Step 1: Write the failing tests
Append to tests/python/test_commands_build.py. FakeSsh already exists in
that file; if its recorded commands are not exposed as .commands, add that
list first in the same style as the existing fake.
# ── cargo dependency cache ───────────────────────────────────────────────────
def test_linux_build_without_cache_root_runs_no_cache_commands(monkeypatch, tmp_path):
fake = _install_fake_ssh(monkeypatch)
_linux_build(**_linux_kwargs(clone_url="https://example/r.git"))
assert not any("cargo-cache" in c for c in fake.commands)
def test_linux_build_skips_cache_when_no_cargo_lock(monkeypatch, tmp_path):
fake = _install_fake_ssh(monkeypatch)
fake.result_for("test -f", returncode=1) # no Cargo.lock in the workdir
_linux_build(**_linux_kwargs(clone_url="https://example/r.git",
cargo_cache_root=tmp_path))
assert not any("cargo-cache-prune" in c for c in fake.commands)
def test_linux_build_restores_an_existing_entry(monkeypatch, tmp_path):
from ci_orchestrator.cargo_cache import CargoCacheStore, cache_key
fake = _install_fake_ssh(monkeypatch)
fake.result_for("sha256sum", stdout="deadbeef Cargo.lock\n")
fake.result_for("rustc -V", stdout="rustc 1.97.1 (abc 2026-01-01)\n")
fake.result_for("rustc -vV", stdout="host: x86_64-unknown-linux-gnu\n")
store = CargoCacheStore(tmp_path)
key = cache_key(lock_digest="deadbeef",
rustc_version="rustc 1.97.1 (abc 2026-01-01)",
host_triple="x86_64-unknown-linux-gnu")
seed = tmp_path / "seed.tar.zst"
seed.write_bytes(b"entry")
store.publish(key, seed)
_linux_build(**_linux_kwargs(clone_url="https://example/r.git",
cargo_cache_root=tmp_path))
assert any("tar" in c and "target" in c for c in fake.commands)
assert fake.copied, "a cache hit must upload the entry to the guest"
def test_linux_build_publishes_on_a_miss(monkeypatch, tmp_path):
from ci_orchestrator.cargo_cache import CargoCacheStore, cache_key
fake = _install_fake_ssh(monkeypatch)
fake.result_for("sha256sum", stdout="cafe Cargo.lock\n")
fake.result_for("rustc -V", stdout="rustc 1.97.1 (abc 2026-01-01)\n")
fake.result_for("rustc -vV", stdout="host: x86_64-unknown-linux-gnu\n")
fake.fetch_writes = b"fresh-entry"
_linux_build(**_linux_kwargs(clone_url="https://example/r.git",
cargo_cache_root=tmp_path))
key = cache_key(lock_digest="cafe",
rustc_version="rustc 1.97.1 (abc 2026-01-01)",
host_triple="x86_64-unknown-linux-gnu")
assert CargoCacheStore(tmp_path).lookup(key) is not None
def test_linux_build_does_not_publish_when_the_prune_assertion_fails(monkeypatch, tmp_path):
from ci_orchestrator.cargo_cache import CargoCacheStore, cache_key
fake = _install_fake_ssh(monkeypatch)
fake.result_for("sha256sum", stdout="cafe Cargo.lock\n")
fake.result_for("rustc -V", stdout="rustc 1.97.1 (abc 2026-01-01)\n")
fake.result_for("rustc -vV", stdout="host: x86_64-unknown-linux-gnu\n")
fake.result_for("cargo-cache-prune", returncode=2,
stderr="prune: REFUSING to publish -- workspace artifacts survived: fury-core")
_linux_build(**_linux_kwargs(clone_url="https://example/r.git",
cargo_cache_root=tmp_path))
key = cache_key(lock_digest="cafe",
rustc_version="rustc 1.97.1 (abc 2026-01-01)",
host_triple="x86_64-unknown-linux-gnu")
assert CargoCacheStore(tmp_path).lookup(key) is None, (
"a failed prune assertion must never reach the store"
)
def test_cache_failure_does_not_fail_the_build(monkeypatch, tmp_path):
fake = _install_fake_ssh(monkeypatch)
fake.result_for("sha256sum", returncode=1, stderr="boom")
# Must not raise: the cache is an optimisation, never a failure mode.
_linux_build(**_linux_kwargs(clone_url="https://example/r.git",
cargo_cache_root=tmp_path))
- Step 2: Run the tests to verify they fail
Run: .venv/bin/python -m pytest tests/python/test_commands_build.py -q -k cargo_cache
Expected: FAIL — _linux_build() got an unexpected keyword argument 'cargo_cache_root'
- Step 3: Add the cache helpers to
build.py
Insert above _linux_build:
_PRUNE_ASSET = "cargo-cache-prune.sh"
_GUEST_PRUNE = "/tmp/cargo-cache-prune.sh"
_GUEST_TARBALL = "/tmp/cargo-cache-entry.tar"
def _cargo_cache_probe(t: SshTransport, workdir: str) -> tuple[str, str] | None:
"""Return ``(key, compressor)`` for this workspace, or ``None`` to skip.
``None`` means "not a Rust build, or we could not read what we need" — the
cache is an optimisation and every uncertainty here resolves to skipping it.
"""
if t.run(f"test -f {_sh_quote(workdir + '/Cargo.lock')}", check=False).returncode != 0:
return None
lock = t.run(f"sha256sum {_sh_quote(workdir + '/Cargo.lock')}", check=False)
version = t.run("rustc -V", check=False)
triple = t.run("rustc -vV | sed -n 's/^host: //p'", check=False)
if not (lock.ok and version.ok and triple.ok):
click.echo("[cache] could not read Cargo.lock digest or rustc version; skipping")
return None
compressor = "zstd" if t.run("command -v zstd", check=False).ok else "gzip"
key = cache_key(
lock_digest=lock.stdout.split()[0],
rustc_version=version.stdout.strip(),
host_triple=triple.stdout.strip(),
)
return key, compressor
def _cargo_cache_restore(
t: SshTransport, store: CargoCacheStore, key: str, workdir: str, compressor: str
) -> bool:
"""Upload and unpack an existing entry. Returns True on a hit."""
entry = store.lookup(key)
if entry is None:
click.echo(f"[cache] miss for {key} — building cold")
return False
size_mb = entry.stat().st_size / 1_048_576
click.echo(f"[cache] hit for {key} ({size_mb:.0f} MB) — restoring")
guest_entry = f"{_GUEST_TARBALL}.{'zst' if compressor == 'zstd' else 'gz'}"
t.copy(str(entry), guest_entry)
flag = "--zstd" if compressor == "zstd" else "-z"
t.run(
f"tar {flag} -xf {_sh_quote(guest_entry)} -C {_sh_quote(workdir)} && "
f"rm -f {_sh_quote(guest_entry)}"
)
return True
def _cargo_cache_publish(
t: SshTransport, store: CargoCacheStore, key: str, workdir: str, compressor: str
) -> None:
"""Prune workspace artifacts, then file the remainder under ``key``.
A non-zero prune means the fail-closed assertion tripped: say so loudly and
publish nothing. A poisoned entry would be far more expensive than a miss.
"""
# as_file() because the production venv is non-editable: the script is
# package data, not a path relative to a source checkout.
asset = resources.files("ci_orchestrator.assets").joinpath(_PRUNE_ASSET)
with resources.as_file(asset) as prune_script:
t.copy(str(prune_script), _GUEST_PRUNE)
pruned = t.run(
f"bash {_GUEST_PRUNE} {_sh_quote(workdir + '/target')} "
f"{_sh_quote(workdir + '/Cargo.toml')}",
check=False,
)
click.echo(f"[cache] {pruned.stdout.strip() or pruned.stderr.strip()}")
if pruned.returncode != 0:
click.echo("[cache] prune refused — publishing nothing", err=True)
return
ext = "zst" if compressor == "zstd" else "gz"
flag = "--zstd" if compressor == "zstd" else "-z"
guest_tar = f"{_GUEST_TARBALL}.{ext}"
packed = t.run(
f"tar {flag} -cf {_sh_quote(guest_tar)} -C {_sh_quote(workdir)} target",
check=False,
)
if not packed.ok:
click.echo("[cache] packing failed — publishing nothing", err=True)
return
with tempfile.TemporaryDirectory() as td:
local = Path(td) / f"entry.tar.{ext}"
t.fetch(guest_tar, local)
published = store.publish(key, local)
click.echo(
f"[cache] published {key} "
f"({published.stat().st_size / 1_048_576:.0f} MB)"
)
Add to the imports at the top of build.py:
from importlib import resources
from ci_orchestrator.cargo_cache import CargoCacheStore, cache_key
- Step 4: Call them from
_linux_build
Add the parameter to the signature (keyword-only, defaulted, so no existing caller changes):
use_xvfb: bool = False,
cargo_cache_root: Path | None = None,
) -> None:
Immediately after the source-mtime normalisation t.run(find ... touch ...)
and before env_prefix = ..., insert:
# Cargo dependency cache. Everything here is best-effort: a cache
# problem must never turn a good build red, so failures degrade to a
# cold build with a message.
cache_key_value: str | None = None
compressor = "gzip"
store: CargoCacheStore | None = None
if cargo_cache_root is not None:
try:
store = CargoCacheStore(cargo_cache_root)
probed = _cargo_cache_probe(t, workdir)
if probed is not None:
cache_key_value, compressor = probed
_cargo_cache_restore(t, store, cache_key_value, workdir, compressor)
except (TransportError, OSError, ValueError) as exc:
click.echo(f"[cache] restore skipped: {exc}", err=True)
cache_key_value = None
Immediately after the build-failure check (if result.returncode != 0: raise ...)
and before the if skip_artifact: block, insert:
# Only publish on a miss: in the steady state this uploads nothing.
if store is not None and cache_key_value and store.lookup(cache_key_value) is None:
try:
_cargo_cache_publish(t, store, cache_key_value, workdir, compressor)
except (TransportError, OSError, ValueError) as exc:
click.echo(f"[cache] publish skipped: {exc}", err=True)
- Step 5: Run the tests to verify they pass
Run: .venv/bin/python -m pytest tests/python/test_commands_build.py -q
Expected: all pass, including the pre-existing tests (the Windows branch and
every non-Rust Linux path must be untouched).
- Step 6: Lint, type-check, full suite with the coverage gate
Run:
.venv/bin/python -m ruff check src tests/python
.venv/bin/python -m mypy --strict src
.venv/bin/python -m pytest tests/python -q --cov=ci_orchestrator --cov-fail-under=90
Expected: clean, clean, and coverage ≥90%.
- Step 7: If Task 0 found
zstdabsent, add it to the toolchain script
Only if Step 4 of Task 0 recorded ZSTD ABSENT. In
template/Install-CIToolchain-LinuxRust2404.sh, add zstd to the apt package
list, next to the existing build tooling. It takes effect at the next template
rebuild; until then the gzip fallback above carries the feature.
- Step 8: Commit
git add src/ci_orchestrator/commands/build.py tests/python/test_commands_build.py \
template/Install-CIToolchain-LinuxRust2404.sh
git commit -m "feat(cache): restore and publish the cargo dependency cache around Linux builds"
Task 4: CLI flag and config section
Files:
- Modify:
src/ci_orchestrator/config.py - Modify:
src/ci_orchestrator/commands/job.py - Modify:
config.example.toml - Test:
tests/python/test_config.py,tests/python/test_commands_job.py
Interfaces:
-
Consumes:
_linux_build(..., cargo_cache_root=...)from Task 3. -
Produces:
Config.cargo_cache: CargoCacheConfigwith fieldsenabled: bool = True,root: Path(default<paths.root>/cache/cargo),retention_days: int = 30; andci job --cargo-cache/--no-cargo-cache. -
Step 1: Write the failing config test
Append to tests/python/test_config.py:
def test_cargo_cache_defaults_under_paths_root(tmp_path: Path) -> None:
cfg_file = tmp_path / "config.toml"
cfg_file.write_text(
'[paths]\nroot = "/var/lib/ci"\ntemplates = "/var/lib/ci/templates"\n'
'build_vms = "/var/lib/ci/build-vms"\nartifacts = "/var/lib/ci/artifacts"\n'
'keys = "/var/lib/ci/keys"\n',
encoding="utf-8",
)
cfg = load_config(config_path=cfg_file, env={})
assert cfg.cargo_cache.enabled is True
assert cfg.cargo_cache.root == Path("/var/lib/ci/cache/cargo")
def test_cargo_cache_section_overrides_the_defaults(tmp_path: Path) -> None:
cfg_file = tmp_path / "config.toml"
cfg_file.write_text(
'[paths]\nroot = "/var/lib/ci"\ntemplates = "/var/lib/ci/templates"\n'
'build_vms = "/var/lib/ci/build-vms"\nartifacts = "/var/lib/ci/artifacts"\n'
'keys = "/var/lib/ci/keys"\n'
'\n[cargo_cache]\nenabled = false\nroot = "/mnt/fast/cargo"\n'
"retention_days = 7\n",
encoding="utf-8",
)
cfg = load_config(config_path=cfg_file, env={})
assert cfg.cargo_cache.enabled is False
assert cfg.cargo_cache.root == Path("/mnt/fast/cargo")
assert cfg.cargo_cache.retention_days == 7
- Step 2: Run it to verify it fails
Run: .venv/bin/python -m pytest tests/python/test_config.py -q -k cargo_cache
Expected: FAIL — AttributeError: 'Config' object has no attribute 'cargo_cache'
- Step 3: Add the dataclass and loader wiring
In config.py, beside the other section dataclasses:
@dataclass(frozen=True)
class CargoCacheConfig:
"""Where compiled Rust dependency artifacts are filed between jobs."""
enabled: bool = True
root: Path = Path("/var/lib/ci/cache/cargo")
retention_days: int = 30
Add the field to Config (after ip_pool, so no positional call breaks):
cargo_cache: CargoCacheConfig = field(default_factory=CargoCacheConfig)
In load_config, after the ip_pool block:
cc_raw = data.get("cargo_cache", {}) if isinstance(data, dict) else {}
cargo_cache = CargoCacheConfig(
enabled=bool(cc_raw.get("enabled", True)),
root=Path(cc_raw.get("root", paths.root / "cache" / "cargo")),
retention_days=int(cc_raw.get("retention_days", 30)),
)
and pass cargo_cache=cargo_cache into the Config(...) construction.
- Step 4: Add the CLI flag to
job.py
Add the option beside --use-shared-cache:
@click.option(
"--cargo-cache/--no-cargo-cache",
"cargo_cache",
default=True,
show_default=True,
help="Reuse compiled Rust dependency artifacts between jobs (Linux only; "
"engages only when the workspace has a Cargo.lock).",
)
Add cargo_cache: bool, to the job(...) signature, and in the Linux branch
of Phase 4 pass it down:
use_xvfb=use_xvfb,
cargo_cache_root=(
config.cargo_cache.root
if cargo_cache and config.cargo_cache.enabled
else None
),
- Step 5: Write the failing job test, then run the suite
Append to tests/python/test_commands_job.py, in the style of the existing
_linux_build call-capture tests:
def test_job_passes_cache_root_to_linux_build(monkeypatch, tmp_path):
captured = _capture_linux_build(monkeypatch)
_run_job(monkeypatch, tmp_path, extra_args=[])
assert captured["cargo_cache_root"] is not None
def test_job_no_cargo_cache_disables_it(monkeypatch, tmp_path):
captured = _capture_linux_build(monkeypatch)
_run_job(monkeypatch, tmp_path, extra_args=["--no-cargo-cache"])
assert captured["cargo_cache_root"] is None
Run: .venv/bin/python -m pytest tests/python -q --cov=ci_orchestrator --cov-fail-under=90
Expected: all pass, coverage ≥90%.
- Step 6: Document the section in
config.example.toml
# Compiled Rust dependency artifacts, reused between jobs. Engages only for
# Linux guests whose build workdir contains a Cargo.lock; workspace crates are
# stripped before an entry is stored, so a cached entry can never carry the
# code under test.
[cargo_cache]
enabled = true
root = "/var/lib/ci/cache/cargo"
retention_days = 30
- Step 7: Lint, type-check, commit
.venv/bin/python -m ruff check src tests/python && .venv/bin/python -m mypy --strict src
git add src/ci_orchestrator/config.py src/ci_orchestrator/commands/job.py \
config.example.toml tests/python/test_config.py tests/python/test_commands_job.py
git commit -m "feat(cache): --cargo-cache flag and [cargo_cache] config section"
Task 5: Action input, and cold builds on release tags
Files:
- Modify:
.gitea/actions/local-ci-build/action.yml
Interfaces:
-
Consumes:
ci job --cargo-cache/--no-cargo-cachefrom Task 4. -
Produces: action input
cargo-cachewith valuesauto(default),on,off. -
Step 1: Add the input
In the inputs: block, after use-shared-cache:
cargo-cache:
description: >
Reuse compiled Rust dependency artifacts between jobs: 'auto' (default)
caches on branches and pull requests but builds release tags (refs/tags/v*)
cold, 'on' always caches, 'off' never does. Only ever engages for Linux
guests whose workspace has a Cargo.lock. Workspace crates are stripped
from an entry before it is stored, so a restored cache cannot carry the
code under test -- but a release build verifies an artifact rather than a
change, so 'auto' gives it the clean room.
required: false
default: 'auto'
- Step 2: Pass it into the env block
Beside INPUT_USE_SHARED_CACHE:
INPUT_CARGO_CACHE: ${{ inputs.cargo-cache }}
GITHUB_REF_FOR_CACHE: ${{ github.ref }}
- Step 3: Resolve it next to the other input resolution
In the same PowerShell step that resolves $resolvedSnapshotName:
# Release tags build cold: they verify a release artifact, not a
# change, so the clean room is worth the two minutes (furyrs#13).
$cargoCacheMode = if ($env:INPUT_CARGO_CACHE) { $env:INPUT_CARGO_CACHE } else { 'auto' }
switch ($cargoCacheMode) {
'on' { $useCargoCache = $true }
'off' { $useCargoCache = $false }
'auto' {
$useCargoCache = -not ($env:GITHUB_REF_FOR_CACHE -like 'refs/tags/v*')
}
default {
throw "cargo-cache must be one of 'auto', 'on', 'off' — got '$cargoCacheMode'."
}
}
Write-Host "[job] Cargo cache: $(if ($useCargoCache) { 'on' } else { 'off (cold)' })"
- Step 4: Add the flag to the argument list
Where the other ci job switches are appended:
if (-not $useCargoCache) { $jobArgs += '--no-cargo-cache' }
- Step 5: Verify the action parses and the branches behave
.venv/bin/python -c "import yaml,sys; yaml.safe_load(open('.gitea/actions/local-ci-build/action.yml')); print('yaml ok')"
pwsh -NoProfile -Command "
\$env:INPUT_CARGO_CACHE='auto'; \$env:GITHUB_REF_FOR_CACHE='refs/tags/v1.2.3'
# paste the switch block; expect: off (cold)
"
Expected: yaml ok, and the auto + tag combination selects the cold path.
If pwsh is not installed on this host (Phase C removed it), verify on the
Windows runner or by reading the branch — but record which you did.
- Step 6: Commit, then publish — the action only runs from the server
git add .gitea/actions/local-ci-build/action.yml
git commit -m "feat(action): cargo-cache input, cold on release tags"
git push origin main
sudo -u ci-runner /opt/ci/venv/bin/python tools/ci-action-published.py
Expected: OK — .gitea/ matches origin/main. Committing is not deploying for
anything under .gitea/; this check is the proof.
Task 6: Retention — the store must not grow without bound
Files:
- Modify:
src/ci_orchestrator/commands/retention.py - Test:
tests/python/test_commands_retention.py
Interfaces:
- Consumes:
CargoCacheStore.prune_older_thanfrom Task 1,Config.cargo_cache.retention_daysfrom Task 4.
TODO §2.9 records that retention run already warns about low disk while
freeing 0.0 GB, because it only governs artifacts, logs and IP leases. Adding a
new unbounded directory without adding it to retention repeats that mistake at
a larger scale — an entry is hundreds of megabytes and one is minted per
Cargo.lock change.
- Step 1: Write the failing test
def test_retention_purges_stale_cargo_cache_entries(tmp_path, monkeypatch):
import os
from datetime import UTC, datetime, timedelta
from ci_orchestrator.cargo_cache import CargoCacheStore
store_root = tmp_path / "cache" / "cargo"
store_root.mkdir(parents=True)
old = store_root / ("v1-" + "a" * 40 + ".tar.zst")
old.write_bytes(b"x")
stamp = (datetime.now(tz=UTC) - timedelta(days=90)).timestamp()
os.utime(old, (stamp, stamp))
_run_retention(monkeypatch, tmp_path, retention_days=30)
assert not old.exists()
def test_retention_what_if_keeps_cargo_cache_entries(tmp_path, monkeypatch):
import os
from datetime import UTC, datetime, timedelta
store_root = tmp_path / "cache" / "cargo"
store_root.mkdir(parents=True)
old = store_root / ("v1-" + "a" * 40 + ".tar.zst")
old.write_bytes(b"x")
stamp = (datetime.now(tz=UTC) - timedelta(days=90)).timestamp()
os.utime(old, (stamp, stamp))
_run_retention(monkeypatch, tmp_path, retention_days=30, what_if=True)
assert old.exists()
- Step 2: Run it to verify it fails
Run: .venv/bin/python -m pytest tests/python/test_commands_retention.py -q -k cargo
Expected: FAIL — the entry still exists.
- Step 3: Implement
In retention_run, after the existing artifact and log purges:
if config.cargo_cache.enabled:
cache_cutoff = datetime.now(tz=UTC) - timedelta(
days=min(effective_days, config.cargo_cache.retention_days)
)
removed = CargoCacheStore(config.cargo_cache.root).prune_older_than(
cache_cutoff, what_if=what_if
)
freed_mb = sum(p.stat().st_size for p in removed if p.exists()) / 1_048_576
verb = "WhatIf: would purge" if what_if else "Purged"
click.echo(
f"[RetentionPolicy] {verb} {len(removed)} cargo cache entr"
f"{'y' if len(removed) == 1 else 'ies'}"
+ (f" ({freed_mb:.0f} MB)" if what_if else "")
)
Add the import: from ci_orchestrator.cargo_cache import CargoCacheStore.
- Step 4: Run tests, lint, type-check
.venv/bin/python -m pytest tests/python -q --cov=ci_orchestrator --cov-fail-under=90
.venv/bin/python -m ruff check src tests/python && .venv/bin/python -m mypy --strict src
- Step 5: Commit
git add src/ci_orchestrator/commands/retention.py tests/python/test_commands_retention.py
git commit -m "feat(cache): purge stale cargo cache entries in retention run"
Task 7: End-to-end validation on the real workspace, and report the numbers
Files:
-
Modify:
plans/cargo-dependency-cache-measurements.md -
Modify:
docs/RUNBOOK-PhaseC.md -
Modify:
TODO.md -
Step 1: Cold run — populates the store
The build command is the build-command: block of
/home/simone/_Workspace/furyrs/.gitea/workflows/ci.yml (line 60 onward) —
copy it verbatim into a file rather than retyping it, because it carries the
/etc/profile.d/rust.sh sourcing and the --locked flags that make the run
representative:
sed -n '/^ build-command: |/,/^ [a-z-]*:/p' \
/home/simone/_Workspace/furyrs/.gitea/workflows/ci.yml \
| sed '1d;$d;s/^ //' > /tmp/furyrs-build.sh
sudo -u ci-runner /opt/ci/venv/bin/python -m ci_orchestrator job \
--job-id cache-e2e-cold \
--repo-url https://gitea.emulab.it/Simone/furyrs.git --branch main \
--template-path /var/lib/ci/templates/LinuxRust2404/LinuxRust2404.vmx \
--snapshot-name BaseClean-LinuxRust --guest-os linux \
--build-command "$(cat /tmp/furyrs-build.sh)" --skip-artifact
At the time of writing that block runs, in order: cargo fmt --all -- --check,
cargo test --workspace --locked, cargo test -p fury-core --features ddr5-scaffold --locked, cargo clippy -p fury-cli --all-targets --features direct-backend --locked -- -D warnings, cargo clippy --workspace --all-targets --locked -- -D warnings, cargo clippy -p fury-gui --features gui --bin fury-gui --locked -- -D warnings, and cargo audit. Three distinct
feature sets, which is exactly the case the single-entry design has to handle.
Expected in the log: [cache] miss for v1-… — building cold, then
[cache] prune: ok — N workspace member(s) stripped, then
[cache] published v1-… (NNN MB). Record the total job time.
- Step 2: Warm run — same commit
Re-run the identical command with --job-id cache-e2e-warm.
Expected: [cache] hit for v1-… (NNN MB) — restoring, no published line
(publish happens on a miss only), and a materially shorter build phase. Record
the total.
- Step 3: The run that actually matters — warm, with modified source
Push a trivial source change to a scratch branch of furyrs and run against it. This is the real CI scenario; #12 measured 6.8s for the build step here.
Expected: cache hit, workspace crates recompiled, dependencies not.
- Step 4: Prove the cache cannot hide a regression
Introduce a deliberate compile error in a workspace crate on a scratch branch and run warm.
Expected: the build fails. If it passes, stop and do not ship: the prune is leaking workspace artifacts and the assertion in Task 2 is not catching it. This single test is what the whole safety argument reduces to.
- Step 5: Confirm the release-tag path is cold
Run once with --no-cargo-cache and confirm the log shows a cold build and no
[cache] hit line.
- Step 6: Write the numbers into the measurements file
Cold total, warm-identical total, warm-modified total, entry size on disk, restore time, publish time. State the net saving against the 133s baseline from #12/#13.
- Step 7: Document for the operator
Add to docs/RUNBOOK-PhaseC.md §6 (Maintenance): where the store lives
(/var/lib/ci/cache/cargo), how to read [cache] hit/miss/REFUSING, how to
invalidate everything (bump CACHE_FORMAT), and how to purge by hand
(rm /var/lib/ci/cache/cargo/*.tar.zst — entries are immutable, deleting one is
always safe and costs one cold build).
- Step 8: Close §7 in TODO.md and the thread
Mark the cargo-cache entry done in TODO.md with the measured numbers, then
append a CI → furyrs entry to
/home/simone/_Workspace/furyrs/docs/ci-rust-integration.md closing #7: what
was built, the deviation from #12's cargo-chef shape and why, the measured
saving, that release tags are cold as they asked, and that cargo-cache: 'off'
is available as a per-workflow override.
- Step 9: Commit and publish
git add plans/cargo-dependency-cache-measurements.md docs/RUNBOOK-PhaseC.md TODO.md
git commit -m "docs(cache): end-to-end numbers for the cargo dependency cache"
git push origin main
sudo -u ci-runner /opt/ci/venv/bin/python tools/ci-action-published.py
Rollback
Every piece is opt-out at a different level, cheapest first:
- Per workflow:
cargo-cache: 'off'in the composite action inputs. - Per host:
enabled = falseunder[cargo_cache]in/var/lib/ci/config.toml. - Everything at once: bump
CACHE_FORMATincargo_cache.py— every existing entry becomes unreachable and every job goes cold, with no file to delete. - Disk:
rm /var/lib/ci/cache/cargo/*.tar.zst. Entries are immutable and independent; deleting any subset costs exactly one cold build each.
No step needs a template rebuild, so none of them has to be coordinated with furyrs's pin-bump window (thread #4).