More properties on the same corpus; finding 8 reduced and confirmed

README.md

@@ -38,6 +38,7 @@ fixture, the third from fuzzing edits and replaying ten crates' git histories.
 - [`docs/ur-queries.md`](docs/ur-queries.md): the bugs' patterns, and closed bugs' patterns, as Ur queries over rustc's source
 - [`docs/shadow-mode.md`](docs/shadow-mode.md): checking reuse inside rustc, and what exists today
 - [`docs/untracked-reads.md`](docs/untracked-reads.md): reporting reads of untracked state inside rustc
+- [`docs/props.md`](docs/props.md): MIR validation, optimization levels and the new trait solver on the same corpus
 - [`docs/plan.md`](docs/plan.md): the plan the work followed, with the properties
 
 ## An instrumented compiler

docs/hunt.md

@@ -31,7 +31,7 @@ with `-Zthreads=8`. `rustc/check.sh wide` runs the ordinary checks.
 | 5 | six untracked options change results incremental compilation reuses; with `-Zno-leak-check`, a rebuild accepts a program a clean build rejects | **looks new**; the first three found by a query written from a closed bug and an option audit ([`ur-queries.md`](ur-queries.md)), `-Zno-leak-check`, `-C extra-filename` and `-Zfuture-incompat-test` by reporting untracked reads ([`untracked-reads.md`](untracked-reads.md)); report drafted |
 | 6 | reused object code keeps the previous checksum of an edited source file in its debuginfo, and with `-Zembed-source` the previous file; with optimizations, ThinLTO symbol names then differ from a clean build | **looks new**; found by the fuzzer at `-Copt-level=2`; root cause found, since 1.44 (#69718); report drafted and a regression test (`hunt/tests/incr-debuginfo-embedded-source`, failing: no fix) |
 | 7 | warnings from inline assembly are not shown again when an incremental rebuild reuses the codegen unit | **looks new**; found while checking reused codegen units ([`shadow-mode.md`](shadow-mode.md)); on 1.60.0 through the nightly; report drafted ([draft](hunt/issue-asm-warnings-reused-cgu.md)) and a regression test (`hunt/tests/incr-asm-warning-reused`, failing: no fix) |
-| 8 | with `-Zmir-opt-level=3` and debuginfo, an incremental rebuild encodes an allocation twice where a clean build encodes it once | **looks new**; found by the fuzzer at `-Zmir-opt-level=4` and named by the reuse check, reduced to three lines; on 1.60.0 through the nightly; report drafted ([draft](hunt/issue-inlined-alloc-identity.md)), no fix |
+| 8 | with `-Zmir-opt-level=3`, an incremental rebuild encodes an allocation from inlined `core` MIR twice where a clean build encodes it once | **looks new**; found by the fuzzer at `-Zmir-opt-level=4` and named by the reuse check, reduced to one line; on 1.60.0 through the nightly; report drafted ([draft](hunt/issue-inlined-alloc-identity.md)), no fix |
 
 Findings 1 and 2 are single-threaded: an ordinary `cargo build`, an edit, another
 `cargo build`, and the metadata differs from a clean build of the edited source. Both come

docs/hunt/issue-inlined-alloc-identity.md

@@ -1,4 +1,4 @@
-# With `-Zmir-opt-level=3` and debuginfo, an incremental rebuild encodes an allocation twice where a clean build encodes it once
+# With `-Zmir-opt-level=3`, an incremental rebuild encodes an allocation from inlined `core` MIR twice where a clean build encodes it once
 
 <!-- Draft issue for rust-lang/rust. Seen on 1.60.0 through nightly-2026-10-06. Related to the
 string-literal case in hunt/issue-literal-dedup.md: an allocation shared in a clean session is
@@ -10,48 +10,69 @@ rebuild's metadata has one more entry in its allocation table.
 ### Reproduction
 
 ```sh
-cat > lib.rs <<'EOF'
-pub fn first_n<const N: usize>(v: &[u8]) -> Option<[u8; N]> {
-    v.get(..N)?.try_into().ok()
-}
-EOF
-F="--edition 2021 --crate-type lib --crate-name x --emit=metadata,link -Zmir-opt-level=3 -Cdebuginfo=2"
+echo 'pub fn f<T>(v: &[T]) -> Option<&[T]> { v.get(..3) }' > lib.rs
+F="--edition 2021 --crate-type lib --crate-name x --emit=metadata,link -Zmir-opt-level=3"
 rustc $F -C incremental=incr  --out-dir rebuilt lib.rs
-cat >> lib.rs <<'EOF'
-pub fn first_m<const N: usize>(v: &[u8]) -> Option<[u8; N]> {
-    v.get(..N)?.try_into().ok()
-}
-EOF
+echo 'pub fn g<T>(v: &[T]) -> Option<&[T]> { v.get(..3) }' >> lib.rs
 rustc $F -C incremental=incr  --out-dir rebuilt lib.rs
 rustc $F -C incremental=clean --out-dir clean   lib.rs
 cmp rebuilt/libx.rmeta clean/libx.rmeta   # differ
 ```
 
 I expected the two to be identical. With `-Zmeta-stats`, the whole difference is in
-`interpret-alloc-index` (18 bytes on a larger crate). Two clean builds agree; without
-`-Cdebuginfo=2`, or at `-Zmir-opt-level=2` and below (so also at `-Copt-level=3`), the rebuild
-agrees with the clean build. It reproduces on 1.60.0, 1.65.0, 1.70.0, 1.75.0, 1.80.0, 1.85.0,
-1.90.0 and `nightly-2026-10-06` (with `RUSTC_BOOTSTRAP=1` on stable).
+`interpret-alloc-index`: the rebuild's metadata has an extra allocation. Two clean builds agree.
+
+| variation | result |
+|---|---|
+| as above, on `nightly-2026-10-06` | differs |
+| `-Zmir-opt-level=2`, or `-Copt-level=3` alone | same |
+| `-Zmir-opt-level=3 -Zinline-mir=no` | same |
+| the same result from a generic local `#[inline]` helper instead of `get` (`if v.len() >= 3 { Some(&v[..3]) } else { None }`) | same |
+| non-generic (`v: &[u8]`) | same |
+
+An earlier form of the reproduction (`v.get(..N)?.try_into().ok()` with a const parameter and
+`-Cdebuginfo=2`) also differs on 1.60.0, 1.65.0, 1.70.0, 1.75.0, 1.80.0, 1.85.0 and 1.90.0
+(with `RUSTC_BOOTSTRAP=1`).
 
 ### What differs
 
-The optimized MIR of `first_n` at this level has a constant `Option::<&[u8]>::None` held in a
-16-byte allocation (`_4 = const Option::<&[u8]>::None;` with `--emit=mir`). In a clean session
-`first_n` and `first_m` refer to the same allocation, so the metadata encodes it once. In the
-rebuild, `first_n` is green and its optimized MIR is decoded from the incremental cache, where
-its allocations were encoded as plain memory; decoding gives it a new `AllocId`, while
-`first_m`, computed fresh, refers to the shared one. Metadata then encodes both.
-
-Where the shared allocation comes from I have not confirmed. Interning a constant for
-propagation does not deduplicate (`intern_with_temp_alloc`), so the sharing more likely comes
-from MIR inlined from `core` (here `<[u8]>::get`): allocations decoded from a crate's metadata
-are decoded once per session and shared by every function that inlines that MIR, but the
-incremental cache stores a copy, not a reference to the upstream crate's allocation. That would
-also explain why debuginfo matters (the constant appears in inlined debuginfo) and why only
-`-Zmir-opt-level=3` and above (more inlining).
+At this level `f`'s optimized MIR returns a constant held in an allocation, because `T` is
+generic and the value cannot be a scalar:
+
+```text
+_0 = const Indirect { alloc_id: alloc1, offset: Size(0 bytes) }: Option<&[T]>;
+```
+
+In a clean session `f` and `g` refer to the same allocation, so the metadata encodes it once.
+In the rebuild, `f` is green and its optimized MIR is decoded from the incremental cache,
+where the allocation was encoded as plain memory; decoding gives it a new `AllocId`, while
+`g`, computed fresh, refers to the shared one. Metadata then encodes both.
+
+The sharing comes from inlining: the constant arrives with the MIR of
+`<[T]>::get` and its `SliceIndex` impl, inlined from `core`. Allocations decoded from a crate's
+metadata are decoded once per session and shared by every function that inlines that MIR, but
+the incremental cache stores a copy rather than a reference to `core`'s allocation. This fits
+every variation above: no inlining, no difference; a generic local helper (whose MIR is not
+decoded from another crate), no difference; a non-generic function (where the value is a
+scalar), no difference.
+
+A change to the incremental cache confirms it
+([`upstream-alloc-reference.patch`](upstream-alloc-reference.patch), experimental): when an
+allocation that was decoded from another crate's metadata is written to the cache, it is
+written as that crate's stable id and the allocation's index there, with its contents, and
+decoding it gives the same `AllocId` as decoding that crate's allocation (if the contents
+agree). With it, both reproductions above build the same incrementally as clean.
+
+It does not fix everything the fuzzer found: one of its cases on the test workspace, at
+`-Zmir-opt-level=4`, still has an extra allocation with the change, so there is at least one
+more source (perhaps MIR inlined from a function of the same crate, whose own MIR came from
+the cache). With the change, the reuse check also reports a string constant whose cached and
+fresh encodings differ, which may be the change's own doing. Not investigated further.
 
 The string-literal case ([report](issue-literal-dedup.md)) is the same kind of loss: an
-allocation shared by deduplication is not shared again after a round trip through the cache.
+allocation shared in a clean session is not shared again after a round trip through the cache.
+A fix along the same lines would encode, in the incremental cache, an allocation that came
+from another crate's metadata as a reference to it rather than as a copy.
 
 ### How it was found
 
@@ -65,4 +86,4 @@ rustc-verify-reuse: allocation shared differently: query `optimized_mir` for Def
   allocation: memory, 16 bytes, align 8, [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
 ```
 
-The test workspace was then reduced automatically to the three lines above.
+The test workspace was then reduced automatically, and by hand to the one line above.

docs/hunt/repro.sh

@@ -81,12 +81,12 @@ r1=$("$rustc" --crate-type lib -Cincremental="$d/i11" --out-dir "$d/nl" "$d/nl/l
 r2=$("$rustc" --crate-type lib -Cincremental="$d/i12" --out-dir "$d/nl" "$d/nl/lib.rs" > /dev/null 2>&1; echo $?)
 if [ "$r1" = "$r2" ]; then echo same; else echo "DIFFER (the rebuild exits $r1, a clean build $r2)"; fi
 
-echo -n "inlined-alloc, -Zmir-opt-level=3 with debuginfo, a duplicated generic fn, incremental vs clean: "
+echo -n "inlined-alloc, -Zmir-opt-level=3, a generic fn duplicated, incremental vs clean: "
 mkdir -p "$d/ia"
-printf 'pub fn first_n<const N: usize>(v: &[u8]) -> Option<[u8; N]> {\n    v.get(..N)?.try_into().ok()\n}\n' > "$d/ia/lib.rs"
-iaf="--edition 2021 --crate-type lib --crate-name x --emit=metadata,link -Zmir-opt-level=3 -Cdebuginfo=2"
+echo 'pub fn f<T>(v: &[T]) -> Option<&[T]> { v.get(..3) }' > "$d/ia/lib.rs"
+iaf="--edition 2021 --crate-type lib --crate-name x --emit=metadata,link -Zmir-opt-level=3"
 "$rustc" $iaf -Cincremental="$d/i13" --out-dir "$d/ia/o1" "$d/ia/lib.rs" 2> /dev/null
-printf 'pub fn first_m<const N: usize>(v: &[u8]) -> Option<[u8; N]> {\n    v.get(..N)?.try_into().ok()\n}\n' >> "$d/ia/lib.rs"
+echo 'pub fn g<T>(v: &[T]) -> Option<&[T]> { v.get(..3) }' >> "$d/ia/lib.rs"
 "$rustc" $iaf -Cincremental="$d/i13" --out-dir "$d/ia/o1" "$d/ia/lib.rs" 2> /dev/null
 "$rustc" $iaf -Cincremental="$d/i14" --out-dir "$d/ia/o2" "$d/ia/lib.rs" 2> /dev/null
 cmp -s "$d/ia/o1/libx.rmeta" "$d/ia/o2/libx.rmeta" && echo same || echo DIFFER

docs/hunt/upstream-alloc-reference.patch

@@ -0,0 +1,148 @@
+--- a/compiler/rustc_middle/src/mir/interpret/mod.rs
++++ b/compiler/rustc_middle/src/mir/interpret/mod.rs
+@@ -18,7 +18,7 @@
+ use rustc_data_structures::sharded::ShardedHashMap;
+ use rustc_data_structures::sync::{AtomicU64, Lock};
+ use rustc_hir::def::DefKind;
+-use rustc_hir::def_id::{DefId, LocalDefId};
++use rustc_hir::def_id::{DefId, LocalDefId, StableCrateId};
+ use rustc_macros::{StableHash, TyDecodable, TyEncodable, TypeFoldable, TypeVisitable};
+ use rustc_serialize::{Decodable, Encodable};
+ use rustc_span::bug;
+@@ -103,6 +103,11 @@
+     /// must be deduplicated again when decoded: otherwise the same allocation decoded from the
+     /// incremental cache and created afresh would get two `AllocId`s.
+     DedupAlloc,
++    /// In the incremental cache: memory decoded from another crate's metadata, by that crate's
++    /// stable id and the allocation's index there, followed by its contents. Decoding it
++    /// gives the same `AllocId` as decoding that crate's allocation, as a fresh computation
++    /// that inlined MIR from that crate would refer to.
++    Upstream,
+ }
+
+ pub fn specialized_encode_alloc_id<'tcx, E: TyEncoder<'tcx>>(
+@@ -115,7 +120,16 @@
+             trace!("encoding {:?} with {:#?}", alloc_id, alloc);
+             let deduplicated = tcx.alloc_map.dedup.lock().get(&(GlobalAlloc::Memory(alloc), CTFE_ALLOC_SALT))
+                 == Some(&alloc_id);
+-            if deduplicated {
++            let upstream = if E::CLEAR_CROSS_CRATE {
++                None
++            } else {
++                tcx.alloc_map.upstream_of.lock().get(&alloc_id).copied()
++            };
++            if let Some((krate, index)) = upstream {
++                AllocDiscriminant::Upstream.encode(encoder);
++                krate.encode(encoder);
++                index.encode(encoder);
++            } else if deduplicated {
+                 AllocDiscriminant::DedupAlloc.encode(encoder);
+             } else {
+                 AllocDiscriminant::Alloc.encode(encoder);
+@@ -161,6 +175,9 @@
+     decoding_state: Vec<Lock<State>>,
+     // The offsets of each allocation in the data stream.
+     data_offsets: Vec<u64>,
++    // For a crate's metadata, the crate's stable id: its allocations are recorded by it and
++    // their index, so that the incremental cache can refer to them.
++    upstream: Option<StableCrateId>,
+ }
+
+ impl AllocDecodingState {
+@@ -173,7 +190,12 @@
+         let decoding_state =
+             std::iter::repeat_with(|| Lock::new(State::Empty)).take(data_offsets.len()).collect();
+
+-        Self { decoding_state, data_offsets }
++        Self { decoding_state, data_offsets, upstream: None }
++    }
++
++    /// The decoding state for the allocations in `krate`'s metadata.
++    pub fn new_upstream(data_offsets: Vec<u64>, krate: StableCrateId) -> Self {
++        Self { upstream: Some(krate), ..Self::new(data_offsets) }
+     }
+ }
+
+@@ -224,7 +246,30 @@
+                 trace!("creating memory alloc ID");
+                 let alloc = <ConstAllocation<'tcx> as Decodable<_>>::decode(decoder);
+                 trace!("decoded alloc {:?}", alloc);
+-                decoder.interner().reserve_and_set_memory_alloc(alloc)
++                let tcx = decoder.interner();
++                // An allocation of a crate's metadata that the incremental cache already
++                // decoded, by reference, gets the same `AllocId` if its contents agree.
++                let known = self.state.upstream.and_then(|krate| {
++                    let id = *tcx.alloc_map.upstream.lock().get(&(krate, idx as u32))?;
++                    (tcx.try_get_global_alloc(id) == Some(GlobalAlloc::Memory(alloc))).then_some(id)
++                });
++                known.unwrap_or_else(|| tcx.reserve_and_set_memory_alloc(alloc))
++            }
++            AllocDiscriminant::Upstream => {
++                trace!("creating memory alloc ID from another crate's metadata");
++                let krate = StableCrateId::decode(decoder);
++                let index = u32::decode(decoder);
++                let alloc = <ConstAllocation<'tcx> as Decodable<_>>::decode(decoder);
++                let tcx = decoder.interner();
++                let known = tcx.alloc_map.upstream.lock().get(&(krate, index)).copied().filter(
++                    |&id| tcx.try_get_global_alloc(id) == Some(GlobalAlloc::Memory(alloc)),
++                );
++                known.unwrap_or_else(|| {
++                    let id = tcx.reserve_and_set_memory_alloc(alloc);
++                    tcx.alloc_map.upstream.lock().insert((krate, index), id);
++                    tcx.alloc_map.upstream_of.lock().insert(id, (krate, index));
++                    id
++                })
+             }
+             AllocDiscriminant::DedupAlloc => {
+                 trace!("creating deduplicated memory alloc ID");
+@@ -259,6 +304,14 @@
+             }
+         });
+
++        if let Some(krate) = self.state.upstream
++            && let Some(GlobalAlloc::Memory(_)) = decoder.interner().try_get_global_alloc(alloc_id)
++        {
++            let tcx = decoder.interner();
++            tcx.alloc_map.upstream.lock().entry((krate, idx as u32)).or_insert(alloc_id);
++            tcx.alloc_map.upstream_of.lock().entry(alloc_id).or_insert((krate, idx as u32));
++        }
++
+         *entry = State::Done(alloc_id);
+
+         alloc_id
+@@ -442,6 +495,11 @@
+     /// the actual guarantees.
+     dedup: Lock<FxHashMap<(GlobalAlloc<'tcx>, usize), AllocId>>,
+
++    /// Memory decoded from another crate's metadata, by that crate's stable id and the
++    /// allocation's index there, in both directions (see `AllocDiscriminant::Upstream`).
++    upstream: Lock<FxHashMap<(StableCrateId, u32), AllocId>>,
++    upstream_of: Lock<FxHashMap<AllocId, (StableCrateId, u32)>>,
++
+     /// The `AllocId` to assign to the next requested ID.
+     /// Always incremented; never gets smaller.
+     next_id: AtomicU64,
+@@ -452,6 +510,8 @@
+         AllocMap {
+             to_alloc: Default::default(),
+             dedup: Default::default(),
++            upstream: Default::default(),
++            upstream_of: Default::default(),
+             next_id: AtomicU64::new(1),
+         }
+     }
+--- a/compiler/rustc_metadata/src/rmeta/decoder.rs
++++ b/compiler/rustc_metadata/src/rmeta/decoder.rs
+@@ -1969,8 +1969,10 @@
+             .decode(&blob)
+             .map(|trait_impls| (trait_impls.trait_id, trait_impls.impls))
+             .collect();
+-        let alloc_decoding_state =
+-            AllocDecodingState::new(root.interpret_alloc_index.decode(&blob).collect());
++        let alloc_decoding_state = AllocDecodingState::new_upstream(
++            root.interpret_alloc_index.decode(&blob).collect(),
++            root.stable_crate_id,
++        );
+
+         // Pre-decode the DefPathHash->DefIndex table. This is a cheap operation
+         // that does not copy any data. It just does some data verification.