Check reuse and report untracked reads inside rustc; findings 6 and 7

README.md

@@ -37,6 +37,7 @@ fixture, the third from fuzzing edits and replaying ten crates' git histories.
 - [`docs/motivating.md`](docs/motivating.md): the real rustc bugs behind each property, each reproduced before and after its fix
 - [`docs/ur-queries.md`](docs/ur-queries.md): the bugs' patterns, and closed bugs' patterns, as Ur queries over rustc's source
 - [`docs/shadow-mode.md`](docs/shadow-mode.md): checking reuse inside rustc, and what exists today
+- [`docs/untracked-reads.md`](docs/untracked-reads.md): reporting reads of untracked state inside rustc
 - [`docs/plan.md`](docs/plan.md): the plan the work followed, with the properties
 
 ## An instrumented compiler

docs/hunt.md

@@ -26,9 +26,12 @@ with `-Zthreads=8`. `rustc/check.sh wide` runs the ordinary checks.
 |---|---|---|
 | 1 | incremental rebuilds encode `Generics::param_def_id_to_index` in a different order from clean builds | **looks new**; root cause found; fix and regression test written |
 | 2 | incremental rebuilds encode a string literal twice where clean builds encode it once | **looks new**; root cause found, regression from #116707 (1.90); fix and regression test written |
-| 3 | with `-Zthreads=8`, two traits with `-> impl Trait` methods give different metadata from run to run | known: [#162202](https://github.com/rust-lang/rust/issues/162202) |
+| 3 | with `-Zthreads=8`, two traits with `-> impl Trait` methods give different metadata from run to run | known: [#162202](https://github.com/rust-lang/rust/issues/162202); a testing workaround, [`hunt/threads-def-order-stopgap.patch`](hunt/threads-def-order-stopgap.patch), makes the order deterministic |
 | 4 | incremental rebuilds republish the previous session's metadata when an edit moves no span, so its source map describes old files | **looks new**; found later by the fuzzer and the history replay ([`scale.md`](scale.md)); root cause found, regression from #114669 (1.90); fix and regression test written |
-| 5 | `-Zemit-stack-sizes`, `-Zcodegen-source-order` and `-Zbuild-sdylib-interface` are untracked but change output that incremental compilation reuses | **looks new**; found by a query written from a closed bug and an option audit ([`ur-queries.md`](ur-queries.md)); report drafted |
+| 5 | six untracked options change results incremental compilation reuses; with `-Zno-leak-check`, a rebuild accepts a program a clean build rejects | **looks new**; the first three found by a query written from a closed bug and an option audit ([`ur-queries.md`](ur-queries.md)), `-Zno-leak-check`, `-C extra-filename` and `-Zfuture-incompat-test` by reporting untracked reads ([`untracked-reads.md`](untracked-reads.md)); report drafted |
+| 6 | reused object code keeps the previous checksum of an edited source file in its debuginfo, and with `-Zembed-source` the previous file; with optimizations, ThinLTO symbol names then differ from a clean build | **looks new**; found by the fuzzer at `-Copt-level=2`; root cause found, since 1.44 (#69718); report drafted and a regression test (`hunt/tests/incr-debuginfo-embedded-source`, failing: no fix) |
+| 7 | warnings from inline assembly are not shown again when an incremental rebuild reuses the codegen unit | **looks new**; found while checking reused codegen units ([`shadow-mode.md`](shadow-mode.md)); on 1.60.0 through the nightly; report drafted ([draft](hunt/issue-asm-warnings-reused-cgu.md)) and a regression test (`hunt/tests/incr-asm-warning-reused`, failing: no fix) |
+| 8 | with `-Zmir-opt-level=3` and debuginfo, an incremental rebuild encodes an allocation twice where a clean build encodes it once | **looks new**; found by the fuzzer at `-Zmir-opt-level=4` and named by the reuse check, reduced to three lines; on 1.60.0 through the nightly; report drafted ([draft](hunt/issue-inlined-alloc-identity.md)), no fix |
 
 Findings 1 and 2 are single-threaded: an ordinary `cargo build`, an edit, another
 `cargo build`, and the metadata differs from a clean build of the edited source. Both come
@@ -37,12 +40,14 @@ incremental cache unchanged. All three reproduce with the official `nightly-2026
 without mirth: `docs/hunt/repro.sh` runs them. None was searched for: P5 and P6 reported
 them on the first run of the new fixture.
 
-Draft bug reports for 1, 2, 4 and 5, written to be filed upstream, are
+Draft bug reports for 1, 2, 4, 5, 6, 7 and 8, written to be filed upstream, are
 [`hunt/issue-generics-order.md`](hunt/issue-generics-order.md),
 [`hunt/issue-literal-dedup.md`](hunt/issue-literal-dedup.md) and
 [`hunt/issue-stale-metadata-reuse.md`](hunt/issue-stale-metadata-reuse.md) and
 [`hunt/issue-untracked-options.md`](hunt/issue-untracked-options.md), the last a comment for
-rust-lang/rust#84232. Each has a candidate fix
+rust-lang/rust#84232; for 6 it is
+[`hunt/issue-stale-debuginfo-source.md`](hunt/issue-stale-debuginfo-source.md), which has no
+fix, since every fix costs codegen reuse and the choice is the maintainers'. Each has a candidate fix
 (`hunt/*.patch`) and a regression test in the style of rustc's `tests/run-make`
 (`hunt/tests/`), which fails on the pinned compiler and passes with the fix.
 

docs/hunt/debuginfo-checksum-stopgap.patch

@@ -0,0 +1,20 @@
+--- a/compiler/rustc_codegen_llvm/src/debuginfo/metadata.rs	2026-10-07 15:55:46.835604483 +0000
++++ b/compiler/rustc_codegen_llvm/src/debuginfo/metadata.rs	2026-10-07 15:55:46.835604483 +0000
+@@ -617,8 +617,15 @@
+             rustc_span::SourceFileHashAlgorithm::Sha256 => llvm::ChecksumKind::SHA256,
+             rustc_span::SourceFileHashAlgorithm::Blake3 => llvm::ChecksumKind::None,
+         };
+-        rustc_data_structures::untracked::untracked_read("source file contents");
+-        let hash_value = hex_encode(source_file.src_hash.hash_bytes());
++        // mirth: testing only. A reused codegen unit keeps the checksum of the file as it was
++        // when the unit was compiled (hunt.md, finding 6); leave it out of incremental sessions
++        // so that the difference does not hide others.
++        let (hash_kind, hash_value) = if cx.sess().opts.incremental.is_some() {
++            (llvm::ChecksumKind::None, String::new())
++        } else {
++            rustc_data_structures::untracked::untracked_read("source file contents");
++            (hash_kind, hex_encode(source_file.src_hash.hash_bytes()))
++        };
+
+         let mut source = None;
+         let external_src;

docs/hunt/issue-asm-warnings-reused-cgu.md

@@ -0,0 +1,82 @@
+# Warnings from inline assembly (and optimization remarks) disappear when an incremental rebuild reuses the codegen unit
+
+<!-- Draft issue for rust-lang/rust. Seen on 1.60.0 through nightly-2026-10-06. -->
+
+The assembler's warnings about an `asm!` block are reported while LLVM compiles the codegen
+unit that contains it. When an incremental rebuild reuses that unit's object code, LLVM does
+not compile it, and the warning is not shown. Unlike warnings from queries, which incremental
+compilation stores and shows again, warnings from LLVM are not stored. A clean build of the
+same source shows the warning.
+
+### Reproduction
+
+```rust
+// main.rs
+mod m;
+fn main() {
+    m::f();
+}
+```
+
+```rust
+// m.rs
+pub fn f() {
+    unsafe { std::arch::asm!(".warning \"from the assembler\"") }
+}
+```
+
+```sh
+rustc --edition 2021 -C codegen-units=4 -C incremental=incr       -o rebuilt main.rs   # the warning
+echo "// a comment at the end" >> m.rs
+rustc --edition 2021 -C codegen-units=4 -C incremental=incr       -o rebuilt main.rs   # nothing
+rustc --edition 2021 -C codegen-units=4 -C incremental=clean-incr -o clean   main.rs   # the warning
+```
+
+The first build and the clean build print:
+
+```text
+warning: from the assembler
+ --> m.rs:2:31
+  |
+2 |     unsafe { std::arch::asm!(".warning \"from the assembler\"") }
+  |                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+```
+
+The rebuild prints nothing. I expected the rebuild to print the same warning as the clean
+build, as it does for warnings from rustc itself.
+
+The same happens on 1.60.0, 1.65.0, 1.70.0, 1.75.0 and 1.90.0, and at `-C opt-level=2`.
+
+### Why
+
+Diagnostics emitted inside a query are saved as side effects and replayed when the query's
+result is reused. Diagnostics from the LLVM backend are emitted through the codegen
+coordinator's `SharedEmitter` while a module is compiled, and a work product (the `.o`, and
+the `.bc` for ThinLTO) records nothing of them. A reused unit is never compiled, so its
+warnings are lost, for as long as the unit stays reused.
+
+Optimization remarks go the same way. With this `m.rs` instead:
+
+```rust
+pub fn f(n: usize) -> usize {
+    (0..n).map(|i| i * 3).sum()
+}
+```
+
+and `main.rs` printing `m::f(std::env::args().count())`, the same three builds with
+`-C opt-level=2 -C remark=all -C debuginfo=1` print 14 remarks located in `m.rs` the first
+time and in the clean build, and none in the rebuild. Remarks are arguably a debugging aid,
+but they are what `-C remark` is for, and an incremental build silently drops them for every
+reused unit.
+
+### Possible fixes
+
+Record the diagnostics emitted while compiling a module with its work product and emit them
+again when the work product is reused, or treat a unit whose compilation warned as not
+reusable.
+
+### How it was found
+
+[mirth](https://github.com/PowderworksCode/mirth) checks reused codegen units against a fresh
+codegen of the same unit. The only differences on its test workspace were `srcloc` cookies on
+inline assembly, which led to looking at what inline-assembly diagnostics do across a reuse.

docs/hunt/issue-inlined-alloc-identity.md

@@ -0,0 +1,68 @@
+# With `-Zmir-opt-level=3` and debuginfo, an incremental rebuild encodes an allocation twice where a clean build encodes it once
+
+<!-- Draft issue for rust-lang/rust. Seen on 1.60.0 through nightly-2026-10-06. Related to the
+string-literal case in hunt/issue-literal-dedup.md: an allocation shared in a clean session is
+not shared after a round trip through the incremental cache. -->
+
+An incremental rebuild produces different `.rmeta` from a clean build of the same source: the
+rebuild's metadata has one more entry in its allocation table.
+
+### Reproduction
+
+```sh
+cat > lib.rs <<'EOF'
+pub fn first_n<const N: usize>(v: &[u8]) -> Option<[u8; N]> {
+    v.get(..N)?.try_into().ok()
+}
+EOF
+F="--edition 2021 --crate-type lib --crate-name x --emit=metadata,link -Zmir-opt-level=3 -Cdebuginfo=2"
+rustc $F -C incremental=incr  --out-dir rebuilt lib.rs
+cat >> lib.rs <<'EOF'
+pub fn first_m<const N: usize>(v: &[u8]) -> Option<[u8; N]> {
+    v.get(..N)?.try_into().ok()
+}
+EOF
+rustc $F -C incremental=incr  --out-dir rebuilt lib.rs
+rustc $F -C incremental=clean --out-dir clean   lib.rs
+cmp rebuilt/libx.rmeta clean/libx.rmeta   # differ
+```
+
+I expected the two to be identical. With `-Zmeta-stats`, the whole difference is in
+`interpret-alloc-index` (18 bytes on a larger crate). Two clean builds agree; without
+`-Cdebuginfo=2`, or at `-Zmir-opt-level=2` and below (so also at `-Copt-level=3`), the rebuild
+agrees with the clean build. It reproduces on 1.60.0, 1.65.0, 1.70.0, 1.75.0, 1.80.0, 1.85.0,
+1.90.0 and `nightly-2026-10-06` (with `RUSTC_BOOTSTRAP=1` on stable).
+
+### What differs
+
+The optimized MIR of `first_n` at this level has a constant `Option::<&[u8]>::None` held in a
+16-byte allocation (`_4 = const Option::<&[u8]>::None;` with `--emit=mir`). In a clean session
+`first_n` and `first_m` refer to the same allocation, so the metadata encodes it once. In the
+rebuild, `first_n` is green and its optimized MIR is decoded from the incremental cache, where
+its allocations were encoded as plain memory; decoding gives it a new `AllocId`, while
+`first_m`, computed fresh, refers to the shared one. Metadata then encodes both.
+
+Where the shared allocation comes from I have not confirmed. Interning a constant for
+propagation does not deduplicate (`intern_with_temp_alloc`), so the sharing more likely comes
+from MIR inlined from `core` (here `<[u8]>::get`): allocations decoded from a crate's metadata
+are decoded once per session and shared by every function that inlines that MIR, but the
+incremental cache stores a copy, not a reference to the upstream crate's allocation. That would
+also explain why debuginfo matters (the constant appears in inlined debuginfo) and why only
+`-Zmir-opt-level=3` and above (more inlining).
+
+The string-literal case ([report](issue-literal-dedup.md)) is the same kind of loss: an
+allocation shared by deduplication is not shared again after a round trip through the cache.
+
+### How it was found
+
+[mirth](https://github.com/PowderworksCode/mirth)'s fuzzer, adding `-Zmir-opt-level=4` to every
+build, reported incremental rebuilds whose metadata differed from clean builds after an edit
+that duplicated this function in its test workspace. A patch that checks reused results inside
+rustc ([`verify-reuse.patch`](verify-reuse.patch)) named the two MIR bodies and the allocation:
+
+```text
+rustc-verify-reuse: allocation shared differently: query `optimized_mir` for DefId(0:125 ~ sink_core[7fc5]::consts::first_n_fuzz19), computed this session uses alloc43 where a fresh computation uses alloc43, but query `optimized_mir` for DefId(0:122 ~ sink_core[7fc5]::consts::first_n), green uses alloc42 for it
+  allocation: memory, 16 bytes, align 8, [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
+```
+
+The test workspace was then reduced automatically to the three lines above.

docs/hunt/issue-stale-debuginfo-source.md

@@ -0,0 +1,142 @@
+# Incremental rebuilds reuse object code whose debuginfo has the previous checksum (and embedded source) of an edited file
+
+<!-- Draft issue for rust-lang/rust. Since 1.44 (#69718, checksums); -Zembed-source since #126985. -->
+
+A codegen unit's debuginfo names each source file with a checksum of its contents
+(`DIFile`, added in #69718 so that "a debugger can verify that the source code matches the
+executable"), and with `-Zembed-source` the contents themselves. Both are read straight from
+the session's source map (`file_metadata` in `rustc_codegen_llvm/src/debuginfo/metadata.rs`),
+which nothing tracks. When an edit leaves a codegen unit green (a comment added at the end
+of a file, for instance), the incremental rebuild reuses the unit's object code, and with
+it the previous session's checksum and source. A clean build of the same source has the
+new ones.
+
+### Reproduction
+
+With `-Zembed-source`, the stale file is visible directly:
+
+```sh
+printf 'pub fn f(x: u32) -> u32 {\n    x ^ 7\n}\n' > lib.rs
+F="--crate-type lib -Cdebuginfo=2 -Zdwarf-version=5 -Zembed-source=yes -Cembed-bitcode=no"
+rustc $F -C incremental=incr  --out-dir rebuilt lib.rs
+echo "// added after the first build" >> lib.rs
+rustc $F -C incremental=incr  --out-dir rebuilt lib.rs
+rustc $F -C incremental=clean --out-dir clean   lib.rs
+for d in rebuilt clean; do (mkdir $d/x && cd $d/x && ar x ../liblib.rlib && llvm-dwarfdump --debug-line *.o | grep source:); done
+```
+
+On `nightly-2026-10-06` the rebuilt object embeds `lib.rs` as it was before the edit:
+
+```text
+rebuilt:  source: "pub fn f(x: u32) -> u32 {\n    x ^ 7\n}\n"
+clean:    source: "pub fn f(x: u32) -> u32 {\n    x ^ 7\n}\n// added after the first build\n"
+```
+
+Without `-Zembed-source` only the checksum is stale. It is in the LLVM IR, so it shows in the
+bitcode `rustc` embeds in rlib objects by default: the same steps without
+`-Zdwarf-version=5 -Zembed-source=yes -Cembed-bitcode=no` give objects whose embedded
+bitcode, disassembled, differs only in the MD5 of `lib.rs` (the old one in the rebuilt
+object, the current one in the clean build) and in the module hash computed from it. (DWARF line tables on Linux carry no MD5s, because the compile unit's own file entry
+has none and LLVM emits them for all files or none.)
+
+It also changes executables built with optimizations, through ThinLTO: the `.llvm.<hash>`
+suffix of a promoted symbol is a hash of its module's bitcode, checksum included. With this
+`main.rs`:
+
+```rust
+mod a {
+    #[inline(never)]
+    pub fn get(v: &[u32], i: usize) -> u32 {
+        v[i] * 3
+    }
+}
+
+mod b {
+    pub fn run(v: &[u32]) -> u32 {
+        crate::a::get(v, 1) + v[0]
+    }
+}
+
+fn main() {
+    let v: Vec<u32> = std::env::args().map(|a| a.len() as u32).collect();
+    println!("{}", b::run(&v));
+}
+```
+
+```sh
+F="--edition 2021 -Copt-level=2 -Cdebuginfo=2 -Ccodegen-units=4"
+rustc $F -C incremental=incr  -o rebuilt main.rs
+echo "// a comment at the end" >> main.rs
+rustc $F -C incremental=incr  -o rebuilt main.rs
+rustc $F -C incremental=clean -o clean   main.rs
+diff <(nm rebuilt | awk '{print $3}' | sort) <(nm clean | awk '{print $3}' | sort)
+```
+
+```text
+< anon.a976517f1c32298f1e683e92c2a4747a.0.llvm.15415811769069898958
+< anon.a976517f1c32298f1e683e92c2a4747a.1.llvm.15415811769069898958
+---
+> anon.a976517f1c32298f1e683e92c2a4747a.0.llvm.4724874333102837038
+> anon.a976517f1c32298f1e683e92c2a4747a.1.llvm.4724874333102837038
+```
+
+The code, data and debug sections are identical; only those names differ. Two clean builds
+agree, and with `-Cdebuginfo=0` the rebuild agrees with the clean build.
+
+I expected the rebuilt objects to equal the clean build's, as they do when the same edit
+is made with debuginfo off.
+
+The Cranelift backend reads the same fields (`debuginfo/line_info.rs` in
+`rustc_codegen_cranelift`) and gives the same result: with `-Zcodegen-backend=cranelift` added
+to the first reproduction, the rebuilt object embeds the old `lib.rs` and the clean one the
+current file.
+
+### Consequences
+
+- With `-Zembed-source`, a debugger that shows embedded source shows a file that no longer
+  exists. With several codegen units, one binary can embed different versions of the same
+  file, depending on which units were reused.
+- The checksum exists so a debugger can tell whether the source on disk matches the binary.
+  A reused unit claims the old file, so a debugger that checks it would reject the current
+  file for code built from it. CodeView (MSVC targets) carries these checksums
+  (#113707 made them SHA256); I have not tried this on Windows.
+- Incremental builds are not reproducible against clean builds once ThinLTO runs (any
+  `opt-level` above 0 with more than one codegen unit), which is the default for a profile
+  with optimizations and incremental compilation on.
+
+### Since when
+
+Checksums came with #69718 (1.44). Bitcode embedded in rlibs by default came in 1.45;
+from 1.45 on, the reproduction above without `-Zembed-source` gives a rebuilt object with the
+old MD5 (checked on 1.44.0, 1.45.0, 1.46.0, 1.47.0, 1.55.0, 1.90.0 and the nightly). The
+executable-level difference depends on what ThinLTO promotes; it appears on 1.47.0, 1.51.0,
+1.60.0 to 1.90.0 and the nightly, and not on 1.55.0. `-Zembed-source` came with #126985.
+
+### Possible fixes
+
+The codegen unit's dependency node would have to depend on the contents of each file its
+debuginfo names. That makes every unit with code from a file red whenever the file changes,
+comments included, which costs codegen reuse; with `-Zembed-source` there is no way around
+it. For the checksum alone, a cheaper option might be to leave it out of debuginfo in
+incremental sessions, or to fix it up when a unit is reused, but either changes what
+incremental builds emit. A query fingerprinting one source file's contents, read by
+`file_metadata`, would make the dependency explicit, as the candidate fix for the same
+problem in reused metadata does ([report](issue-stale-metadata-reuse.md)).
+
+### A test
+
+`tests/run-make/incr-debuginfo-embedded-source` (in mirth at
+`docs/hunt/tests/incr-debuginfo-embedded-source/rmake.rs`) builds a binary with
+`-Zembed-source`, adds a comment at the end of `main.rs`, rebuilds incrementally and builds
+clean, and checks that both embed the edited file. On `ea137335b` it fails:
+
+```text
+the incremental rebuild embeds a different main.rs from a clean build: ["fn main() {\n    println!(\"{}\", 7);\n}\n"]
+```
+
+### How it was found
+
+[mirth](https://github.com/PowderworksCode/mirth) fuzzes edits on a five-crate workspace
+and compares every incremental rebuild's artifacts with a clean build's. At `-Copt-level=2`
+a third of the rebuilds gave binaries that differed only in `.llvm.<hash>` suffixes; the
+modules behind them had pre-LTO bitcode that differed only in a `DIFile` checksum.