From b5c18a6d70ece92c80948f4a2ddaf649b53d9f54 Mon Sep 17 00:00:00 2001 From: Simone Date: Wed, 22 Jul 2026 20:55:31 +0200 Subject: [PATCH] feat(cache): guest-side prune, asserted against an independent oracle Measured first, as thread entry #12 requires: a cold furyrs pipeline is 134.7s, target/ is 3.0 GiB raw and 2.5 GiB pruned, and the stored entry is 645 MiB. A warm job pays 8.1s (upload + extract) to skip ~120s of dependency compilation. Publishing costs 15.7s but only on a miss. Build it. The assertion in the plan turned out to be worthless and is replaced. It re-scanned for */.fingerprint/-* after deleting exactly those paths, so asked to catch a planted fury-core fingerprint it exited 0 -- the prune had removed the evidence before the check looked. An assertion sharing its globs with the deletion can only prove that rm ran; it cannot see the failure that matters, which is a glob that never matched the artifact in the first place. The replacement asks a different oracle. Cargo records in each .d depfile the sources an artifact was built from: dependencies point into the registry, anything built from workspace source points back into the workspace. After a correct prune, zero surviving depfiles reference workspace source outside target/. Verified negatively too -- an artifact named totallyunrelated-abc123.d that no prune glob matches, declaring a dependency on fury-core/src/lib.rs, makes the script exit 2 and refuse to publish. zstd is already present on the template, so no rebuild is needed; the gzip fallback stays for the plain LinuxBuild2404. --- plans/cargo-dependency-cache-measurements.md | 93 +++++++++++++++++++ src/ci_orchestrator/assets/__init__.py | 0 .../assets/cargo-cache-prune.sh | 87 +++++++++++++++++ 3 files changed, 180 insertions(+) create mode 100644 plans/cargo-dependency-cache-measurements.md create mode 100644 src/ci_orchestrator/assets/__init__.py create mode 100755 src/ci_orchestrator/assets/cargo-cache-prune.sh diff --git a/plans/cargo-dependency-cache-measurements.md b/plans/cargo-dependency-cache-measurements.md new file mode 100644 index 0000000..836c3c4 --- /dev/null +++ b/plans/cargo-dependency-cache-measurements.md @@ -0,0 +1,93 @@ +# Cargo dependency cache — measurements + +Measured 2026-07-22 on a `LinuxRust2404` linked clone (4 vCPU, 3.9 GB RAM, +`rustc 1.97.1`, `zstd` present at `/usr/bin/zstd`), running furyrs `b8ea567` +with the `build-command` block from its `.gitea/workflows/ci.yml` verbatim. + +Entry #12 of the furyrs thread set the rule: measure before building. There it +killed the registry cache — 9.3s of saving for +800 MB on the template. Here +the numbers point the other way, decisively. + +## Baseline + +| | | +|---|---| +| Cold pipeline, empty `target/` | **134.7s** | + +Matches the 133s furyrs reported for job 1021 and the 126.6s measured in #12, +so the clone is representative. + +## Sizes + +| | Bytes | Human | +|---|---:|---:| +| `target/` raw | 3,206,670,175 | 3.0 GiB | +| `tar` of raw `target/` | 3,215,503,360 | 3.0 GiB | +| `tar` + `zstd -3 -T0`, raw | 810,108,028 | 773 MiB | +| **`target/` after prune** | 2,719,971,105 | **2.5 GiB** | +| **entry as stored** (`tar --zstd`, pruned) | 676,585,779 | **645 MiB** | + +Pruning strips 487 MB (15%) of raw `target/`, which is the workspace crates' +own artifacts — 42 fingerprint directories across the six members +(`fury-core` 11, `fury-gui` 12, `fury-ipc` 6, `fury-backend` 5, `fury-cli` 5, +`furyd` 3). 1,928 dependency artifacts and 903 dependency fingerprints survive. + +## Costs + +| Step | Seconds | Paid by | +|---|---:|---| +| prune + assert | 0.21 | publish | +| pack (`tar --zstd`) | 10.88 | publish | +| fetch guest→host (140 MiB/s) | 4.62 | publish | +| **publish total** | **15.7** | **only on a cache miss** | +| upload host→guest (179 MiB/s) | 3.60 | restore | +| extract | 4.52 | restore | +| **restore total** | **8.1** | **every warm job** | + +## Verdict: build it + +A warm job pays **8.1s** to skip roughly **120s** of dependency compilation — +the ~490 external crates that a committed `Cargo.lock` keeps bit-identical +between runs. Net saving ≈ 112s on a 134.7s job, about **83%**. + +The publish path costs 15.7s but runs only on a miss, i.e. once per +`Cargo.lock` change, so it is amortised across every run in between. + +Transfer is not the bottleneck anyone feared: the virtual NIC moves the entry +at 140–180 MiB/s, so 645 MiB crosses in under 4 seconds each way. + +## Notes that changed the implementation + +**`zstd` is already in the template.** `/usr/bin/zstd` is present on +`LinuxRust2404`, so the gzip fallback is dead code in practice and no template +rebuild is needed. Keep the fallback anyway — it costs one branch and keeps the +feature working on a plain `LinuxBuild2404`. + +**The name-based assertion was worthless and was replaced.** The plan's +original check re-scanned for `*/.fingerprint/-*` after deleting +exactly those paths. Asked to catch a planted `fury-core-deadbeefcafe` +fingerprint, it exited **0** — the prune had deleted the evidence before the +assertion looked. An assertion built from the same globs as the deletion can +only prove that `rm` ran; it is blind to the failure that matters, a glob that +never matched the artifact at all. + +The replacement asks a different oracle. Cargo records in each `.d` depfile the +source files an artifact was built *from*: a dependency points into +`/opt/rust/cargo/registry/...`, anything built from workspace source points +back into the workspace. After a correct prune, **zero** surviving depfiles +reference workspace source outside `target/`. Verified negatively as well — an +artifact named `totallyunrelated-abc123.d` (which no prune glob matches) +declaring a dependency on `fury-core/src/lib.rs` makes the script exit **2** +and refuse to publish. + +| Check | Clean tree | Planted fingerprint | Planted foreign artifact | +|---|---|---|---| +| name-based (original) | 0 | **0 — missed it** | 0 — missed it | +| depfile-based (shipped) | 0 | 0 (prune removed it, correctly) | **2 — refused** | + +**Feature variants coexist in one entry, as predicted.** `deps/` holds +`libclap-943062c20e623534.rmeta` and `libclap-abe395d6778d44ce.rmeta` side by +side — the same crate compiled under two different feature resolutions. This is +why the feature set is not part of the cache key: cargo already separates the +variants by hash, so one entry serves all of furyrs's four feature +configurations instead of fragmenting into four. diff --git a/src/ci_orchestrator/assets/__init__.py b/src/ci_orchestrator/assets/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/ci_orchestrator/assets/cargo-cache-prune.sh b/src/ci_orchestrator/assets/cargo-cache-prune.sh new file mode 100755 index 0000000..52b3adb --- /dev/null +++ b/src/ci_orchestrator/assets/cargo-cache-prune.sh @@ -0,0 +1,87 @@ +#!/usr/bin/env bash +# Strip every workspace-crate artifact out of target/, then prove none survived. +# +# The cache is only sound while it contains dependency artifacts alone. A +# workspace crate's artifact carries the code under test, and reusing one can +# turn a red build green -- the worst failure a CI system has, because it is +# silent. So this script ends by asserting its own work and exits non-zero if +# the assertion fails; the caller must treat that as "do not publish". +# +# Exit codes: 0 clean prune, 1 usage/environment error, 2 assertion tripped. +set -euo pipefail + +TARGET_DIR="${1:?usage: cargo-cache-prune.sh }" +MANIFEST="${2:?usage: cargo-cache-prune.sh }" + +[ -d "$TARGET_DIR" ] || { echo "prune: no such target dir: $TARGET_DIR" >&2; exit 1; } +command -v cargo >/dev/null || { echo "prune: cargo not on PATH" >&2; exit 1; } + +# --no-deps restricts the listing to workspace members, which is precisely the +# set whose artifacts must never be cached. +members="$(cargo metadata --no-deps --format-version 1 --manifest-path "$MANIFEST" \ + | python3 -c 'import json,sys; print("\n".join(p["name"] for p in json.load(sys.stdin)["packages"]))')" + +if [ -z "$members" ]; then + echo "prune: cargo metadata listed no workspace members" >&2 + exit 1 +fi + +# Incremental state is large, machine-specific and never reusable across +# clones; examples and docs are not dependencies of anything we build. +rm -rf "$TARGET_DIR"/*/incremental "$TARGET_DIR"/*/examples "$TARGET_DIR"/doc 2>/dev/null || true + +prune_one() { + name="$1" + snake="${name//-/_}" + # Fingerprints and build-script output live in per-crate directories. + find "$TARGET_DIR" -maxdepth 3 \ + \( -path "*/.fingerprint/$name-*" \ + -o -path "*/.fingerprint/$snake-*" \ + -o -path "*/build/$name-*" \ + -o -path "*/build/$snake-*" \) \ + -prune -exec rm -rf {} + 2>/dev/null || true + # Compiled outputs: rlib/rmeta/d in deps/, plus binaries at the profile root. + find "$TARGET_DIR" -maxdepth 3 \ + \( -name "$snake-*" -o -name "lib$snake-*" -o -name "lib$snake.*" \ + -o -name "$name" -o -name "$name-*" \) \ + -exec rm -rf {} + 2>/dev/null || true +} + +while IFS= read -r name; do + [ -n "$name" ] || continue + prune_one "$name" +done </dev/null \ + | xargs -0 -r grep -h -oE "${ws_root}/[^ :]*" 2>/dev/null \ + | grep -v "^${abs_target}/" \ + | sort -u || true)" + +if [ -n "$leaked_paths" ]; then + echo "prune: REFUSING to publish -- artifacts still reference workspace source:" >&2 + printf '%s\n' "$leaked_paths" | sed 's/^/ /' | head -20 >&2 + exit 2 +fi + +echo "prune: ok -- $(printf '%s\n' "$members" | wc -l) workspace member(s) stripped," \ + "no surviving artifact references workspace source"