Repository navigation
Scaling up: history replay, an edit fuzzer, closed bugs as queries, and four new findings - #17
Merged
Merged
Conversation
zmaril
added this pull request to stack #19
October 7, 2026 20:17
rustc/replay.py replays a crate's git history through incremental compilation: each commit built incrementally on the last and again from scratch, compared (P6). fixtures/sink is a five-crate workspace with as many stable language features as fit, and rustc/fuzz.py makes random edits to it and checks every incremental rebuild against a clean build, the binaries' output included; rustc/fuzz-replay.py replays a finding. Both found a third bug: an incremental rebuild republishes the previous session's metadata when an edit moves no span, because the source map's file hashes and lengths are not tracked (regression from #114669). The replay also caught my first fix for the literal bug over-deduplicating on a serde commit; the fix now records whether an allocation was deduplicated when created. docs/properties.md surveys 1,000 fixed rustc bugs for checkable properties, with every citation checked against the issue text. docs/scale.md has the numbers. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Mnrg9JXj9X1ht6Qkz2ybh
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Mnrg9JXj9X1ht6Qkz2ybh
… each fix 27 rust-lang/rust bugs that violated P1 to P7, P5 under threads and dependency tracking, each reproduced on a toolchain from before its fix and shown fixed on one after, without mirth: docs/motivating.md, with the reproductions in docs/motivating/bugs.json, the runner in run.py and every output in out/. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Mnrg9JXj9X1ht6Qkz2ybh
… too ur/rustc/RoundTrip.rsc writes the three bugs' patterns as classifiers for ur rewrite --classify, generalized as far as they still find their bug. On rustc's compiler/ they run in 0.9 s; docs/ur-queries.md has every site and its triage. No new bug: the other sites never reach .rmeta, read tracked options, or deduplicate inside. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Mnrg9JXj9X1ht6Qkz2ybh
rustc/artifacts.py collects, from cargo's JSON messages, every rlib's members (object files named without their incremental session suffix, which differs between sessions while the objects are identical), every executable, and each crate's diagnostics. The fuzzer compares all three between an incremental rebuild and a clean build; the replay compares rlibs and diagnostics. Two clean builds, and a touch rebuild against a clean one, agree on all of them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Mnrg9JXj9X1ht6Qkz2ybh
…d do Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Mnrg9JXj9X1ht6Qkz2ybh
…ound ur/rustc/ClosedBugs.rsc writes eleven closed rustc bugs' patterns as queries; ur/verify-closed.py fetches each fix's pre-fix files and checks the query finds the site, which all eleven do. Over today's compiler, the query from #66955 (untracked --remap-path-prefix), joined with the [UNTRACKED] options, led to rustc/audit-options.py, which builds a crate incrementally without and with each option against a clean build: -Zemit-stack-sizes, -Zcodegen-source-order and -Zbuild-sdylib-interface change output that incremental compilation reuses. Report draft for rust-lang/rust#84232 in docs/hunt/issue-untracked-options.md; triage in docs/ur-queries.md. The final numbers of the replay and the fuzzer are in docs/scale.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Mnrg9JXj9X1ht6Qkz2ybh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #13.
New bugs, all reproducible on the official nightly (draft reports in
docs/hunt/):-Zemit-stack-sizes,-Zcodegen-source-orderand-Zbuild-sdylib-interfaceare[UNTRACKED]but change output that incremental compilation reusesrustccommandsEach has a candidate fix where one makes sense, and the stale-metadata bug has a regression test. Two corrections to earlier work:
What was built:
rustc/replay.py: replays a crate's history, building each commit incrementally and from scratch, and compares them. Ten crates, 4,862 commits that both builds compiled; nothing differs on the fixed compiler.fixtures/sink: about 1,200 lines in five crates, with the stable language features that fit and 60 runtime checks.rustc/fuzz.py:.rmeta, proc-macro metadata, rlib object code, binaries and their output, and diagnostics (rustc/artifacts.py);ur/rustc/RoundTrip.rscandClosedBugs.rsc:ur/verify-closed.pyfetches each fix's pre-fix files to check;compiler/in about 2 s; triage is indocs/ur-queries.md;rustc/audit-options.py: builds incrementally without and then with each untracked option, compared with a clean build.What was written up:
docs/motivating.md: the 27 real rustc bugs behind P1–P7, P5 under threads, and tracking/reuse. Each one is reproduced on a toolchain from before its fix and shown fixed after it (outputs indocs/motivating/out/).docs/properties.md: 29 properties surveyed from 1,000 fixed C-bug issues. Every citation was checked against the issue text, and 107 of 156 held.docs/shadow-mode.md: checking reuse inside rustc. Nothing like it exists:-Zincremental-verify-ichre-hashes cached values but never recomputes them or checks work products.Saved ideas are filed as #14 (fault injection), #15 (mutation testing) and #16 (coverage-steered fuzzing).
🤖 Generated with Claude Code
https://claude.ai/code/session_018Mnrg9JXj9X1ht6Qkz2ybh