feat(cache): guest-side prune, asserted against an independent oracle
Measured first, as thread entry #12 requires: a cold furyrs pipeline is 134.7s, target/ is 3.0 GiB raw and 2.5 GiB pruned, and the stored entry is 645 MiB. A warm job pays 8.1s (upload + extract) to skip ~120s of dependency compilation. Publishing costs 15.7s but only on a miss. Build it. The assertion in the plan turned out to be worthless and is replaced. It re-scanned for */.fingerprint/<member>-* after deleting exactly those paths, so asked to catch a planted fury-core fingerprint it exited 0 -- the prune had removed the evidence before the check looked. An assertion sharing its globs with the deletion can only prove that rm ran; it cannot see the failure that matters, which is a glob that never matched the artifact in the first place. The replacement asks a different oracle. Cargo records in each .d depfile the sources an artifact was built from: dependencies point into the registry, anything built from workspace source points back into the workspace. After a correct prune, zero surviving depfiles reference workspace source outside target/. Verified negatively too -- an artifact named totallyunrelated-abc123.d that no prune glob matches, declaring a dependency on fury-core/src/lib.rs, makes the script exit 2 and refuse to publish. zstd is already present on the template, so no rebuild is needed; the gzip fallback stays for the plain LinuxBuild2404.
This commit is contained in:
+87
@@ -0,0 +1,87 @@
|
||||
#!/usr/bin/env bash
|
||||
# Strip every workspace-crate artifact out of target/, then prove none survived.
|
||||
#
|
||||
# The cache is only sound while it contains dependency artifacts alone. A
|
||||
# workspace crate's artifact carries the code under test, and reusing one can
|
||||
# turn a red build green -- the worst failure a CI system has, because it is
|
||||
# silent. So this script ends by asserting its own work and exits non-zero if
|
||||
# the assertion fails; the caller must treat that as "do not publish".
|
||||
#
|
||||
# Exit codes: 0 clean prune, 1 usage/environment error, 2 assertion tripped.
|
||||
set -euo pipefail
|
||||
|
||||
TARGET_DIR="${1:?usage: cargo-cache-prune.sh <target-dir> <manifest-path>}"
|
||||
MANIFEST="${2:?usage: cargo-cache-prune.sh <target-dir> <manifest-path>}"
|
||||
|
||||
[ -d "$TARGET_DIR" ] || { echo "prune: no such target dir: $TARGET_DIR" >&2; exit 1; }
|
||||
command -v cargo >/dev/null || { echo "prune: cargo not on PATH" >&2; exit 1; }
|
||||
|
||||
# --no-deps restricts the listing to workspace members, which is precisely the
|
||||
# set whose artifacts must never be cached.
|
||||
members="$(cargo metadata --no-deps --format-version 1 --manifest-path "$MANIFEST" \
|
||||
| python3 -c 'import json,sys; print("\n".join(p["name"] for p in json.load(sys.stdin)["packages"]))')"
|
||||
|
||||
if [ -z "$members" ]; then
|
||||
echo "prune: cargo metadata listed no workspace members" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Incremental state is large, machine-specific and never reusable across
|
||||
# clones; examples and docs are not dependencies of anything we build.
|
||||
rm -rf "$TARGET_DIR"/*/incremental "$TARGET_DIR"/*/examples "$TARGET_DIR"/doc 2>/dev/null || true
|
||||
|
||||
prune_one() {
|
||||
name="$1"
|
||||
snake="${name//-/_}"
|
||||
# Fingerprints and build-script output live in per-crate directories.
|
||||
find "$TARGET_DIR" -maxdepth 3 \
|
||||
\( -path "*/.fingerprint/$name-*" \
|
||||
-o -path "*/.fingerprint/$snake-*" \
|
||||
-o -path "*/build/$name-*" \
|
||||
-o -path "*/build/$snake-*" \) \
|
||||
-prune -exec rm -rf {} + 2>/dev/null || true
|
||||
# Compiled outputs: rlib/rmeta/d in deps/, plus binaries at the profile root.
|
||||
find "$TARGET_DIR" -maxdepth 3 \
|
||||
\( -name "$snake-*" -o -name "lib$snake-*" -o -name "lib$snake.*" \
|
||||
-o -name "$name" -o -name "$name-*" \) \
|
||||
-exec rm -rf {} + 2>/dev/null || true
|
||||
}
|
||||
|
||||
while IFS= read -r name; do
|
||||
[ -n "$name" ] || continue
|
||||
prune_one "$name"
|
||||
done <<EOF
|
||||
$members
|
||||
EOF
|
||||
|
||||
# ── The assertion ────────────────────────────────────────────────────────────
|
||||
#
|
||||
# This deliberately does NOT re-use the name patterns above. An assertion built
|
||||
# from the same globs as the deletion can only prove that `rm` ran; it is blind
|
||||
# to the failure that actually matters, which is a glob that never matched the
|
||||
# artifact in the first place. Asked to catch a planted fingerprint, such a
|
||||
# check "passes" because the prune deleted the evidence before looking.
|
||||
#
|
||||
# So ask a different oracle: cargo records in each `.d` depfile the source
|
||||
# files an artifact was built FROM. A dependency's depfile points into the
|
||||
# registry; anything built from workspace source points back into the
|
||||
# workspace. That is the property we actually care about, and it holds however
|
||||
# the artifact happens to be named.
|
||||
ws_root="$(cd "$(dirname "$MANIFEST")" && pwd)"
|
||||
abs_target="$(cd "$TARGET_DIR" && pwd)"
|
||||
|
||||
# Every depfile names its own output under target/, so that prefix is excluded;
|
||||
# what remains is genuine source input.
|
||||
leaked_paths="$(find "$abs_target" -name '*.d' -print0 2>/dev/null \
|
||||
| xargs -0 -r grep -h -oE "${ws_root}/[^ :]*" 2>/dev/null \
|
||||
| grep -v "^${abs_target}/" \
|
||||
| sort -u || true)"
|
||||
|
||||
if [ -n "$leaked_paths" ]; then
|
||||
echo "prune: REFUSING to publish -- artifacts still reference workspace source:" >&2
|
||||
printf '%s\n' "$leaked_paths" | sed 's/^/ /' | head -20 >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
echo "prune: ok -- $(printf '%s\n' "$members" | wc -l) workspace member(s) stripped," \
|
||||
"no surviving artifact references workspace source"
|
||||
Reference in New Issue
Block a user