Files
local-ci-cd-system/plans/cargo-dependency-cache-measurements.md
T
Simone b5c18a6d70 feat(cache): guest-side prune, asserted against an independent oracle
Measured first, as thread entry #12 requires: a cold furyrs pipeline is 134.7s,
target/ is 3.0 GiB raw and 2.5 GiB pruned, and the stored entry is 645 MiB. A
warm job pays 8.1s (upload + extract) to skip ~120s of dependency compilation.
Publishing costs 15.7s but only on a miss. Build it.

The assertion in the plan turned out to be worthless and is replaced. It
re-scanned for */.fingerprint/<member>-* after deleting exactly those paths, so
asked to catch a planted fury-core fingerprint it exited 0 -- the prune had
removed the evidence before the check looked. An assertion sharing its globs
with the deletion can only prove that rm ran; it cannot see the failure that
matters, which is a glob that never matched the artifact in the first place.

The replacement asks a different oracle. Cargo records in each .d depfile the
sources an artifact was built from: dependencies point into the registry,
anything built from workspace source points back into the workspace. After a
correct prune, zero surviving depfiles reference workspace source outside
target/. Verified negatively too -- an artifact named totallyunrelated-abc123.d
that no prune glob matches, declaring a dependency on fury-core/src/lib.rs,
makes the script exit 2 and refuse to publish.

zstd is already present on the template, so no rebuild is needed; the gzip
fallback stays for the plain LinuxBuild2404.
2026-07-22 20:55:31 +02:00

4.1 KiB
Raw Blame History

Cargo dependency cache — measurements

Measured 2026-07-22 on a LinuxRust2404 linked clone (4 vCPU, 3.9 GB RAM, rustc 1.97.1, zstd present at /usr/bin/zstd), running furyrs b8ea567 with the build-command block from its .gitea/workflows/ci.yml verbatim.

Entry #12 of the furyrs thread set the rule: measure before building. There it killed the registry cache — 9.3s of saving for +800 MB on the template. Here the numbers point the other way, decisively.

Baseline

Cold pipeline, empty target/ 134.7s

Matches the 133s furyrs reported for job 1021 and the 126.6s measured in #12, so the clone is representative.

Sizes

Bytes Human
target/ raw 3,206,670,175 3.0 GiB
tar of raw target/ 3,215,503,360 3.0 GiB
tar + zstd -3 -T0, raw 810,108,028 773 MiB
target/ after prune 2,719,971,105 2.5 GiB
entry as stored (tar --zstd, pruned) 676,585,779 645 MiB

Pruning strips 487 MB (15%) of raw target/, which is the workspace crates' own artifacts — 42 fingerprint directories across the six members (fury-core 11, fury-gui 12, fury-ipc 6, fury-backend 5, fury-cli 5, furyd 3). 1,928 dependency artifacts and 903 dependency fingerprints survive.

Costs

Step Seconds Paid by
prune + assert 0.21 publish
pack (tar --zstd) 10.88 publish
fetch guest→host (140 MiB/s) 4.62 publish
publish total 15.7 only on a cache miss
upload host→guest (179 MiB/s) 3.60 restore
extract 4.52 restore
restore total 8.1 every warm job

Verdict: build it

A warm job pays 8.1s to skip roughly 120s of dependency compilation — the ~490 external crates that a committed Cargo.lock keeps bit-identical between runs. Net saving ≈ 112s on a 134.7s job, about 83%.

The publish path costs 15.7s but runs only on a miss, i.e. once per Cargo.lock change, so it is amortised across every run in between.

Transfer is not the bottleneck anyone feared: the virtual NIC moves the entry at 140–180 MiB/s, so 645 MiB crosses in under 4 seconds each way.

Notes that changed the implementation

zstd is already in the template. /usr/bin/zstd is present on LinuxRust2404, so the gzip fallback is dead code in practice and no template rebuild is needed. Keep the fallback anyway — it costs one branch and keeps the feature working on a plain LinuxBuild2404.

The name-based assertion was worthless and was replaced. The plan's original check re-scanned for */.fingerprint/<member>-* after deleting exactly those paths. Asked to catch a planted fury-core-deadbeefcafe fingerprint, it exited 0 — the prune had deleted the evidence before the assertion looked. An assertion built from the same globs as the deletion can only prove that rm ran; it is blind to the failure that matters, a glob that never matched the artifact at all.

The replacement asks a different oracle. Cargo records in each .d depfile the source files an artifact was built from: a dependency points into /opt/rust/cargo/registry/..., anything built from workspace source points back into the workspace. After a correct prune, zero surviving depfiles reference workspace source outside target/. Verified negatively as well — an artifact named totallyunrelated-abc123.d (which no prune glob matches) declaring a dependency on fury-core/src/lib.rs makes the script exit 2 and refuse to publish.

Check Clean tree Planted fingerprint Planted foreign artifact
name-based (original) 0 0 — missed it 0 — missed it
depfile-based (shipped) 0 0 (prune removed it, correctly) 2 — refused

Feature variants coexist in one entry, as predicted. deps/ holds libclap-943062c20e623534.rmeta and libclap-abe395d6778d44ce.rmeta side by side — the same crate compiled under two different feature resolutions. This is why the feature set is not part of the cache key: cargo already separates the variants by hash, so one entry serves all of furyrs's four feature configurations instead of fragmenting into four.