Flag universe: covering arrays, transition walks, findings 9–11

docs/flags.md

@@ -0,0 +1,176 @@
+# The flag universe
+
+How many rustc configurations are there, which combinations does rustc refuse, and how many
+builds cover every pair or triple of option values? Pinned compiler: nightly-2026-10-06
+(ea137335b) with mirth's local patches (`rustc-verify5`), x86_64-unknown-linux-gnu.
+
+## Enumeration
+
+`rustc/flag-universe.py` reads `compiler/rustc_session/src/options.rs`:
+
+| | options | enumerable | free-form (left out) |
+|---|---|---|---|
+| `-C` | 51 | 26 | 25 |
+| `-Z` | 235 | 190 | 45 |
+| total | 286 | 216 | 70 |
+
+An option's domain is absence plus: `yes`/`no` for a boolean, present for an option without a
+value, the backticked values in its parser's description for an enumerated one, `1` and `16`
+for a number. Strings, paths, lists, target features, passes and the like are left out,
+except 19 options with hand-picked samples (`SAMPLES` in `flag-universe.py`): the first tables
+below were made before these were added, and left out `-Copt-level` (its parser takes a
+string), so the walks there ran at opt-level 0. With the samples, 235 options are enumerable,
+556 values were tried alone and 508 accepted; the pairs were not tried again.
+
+Every value was tried alone on a one-function lib with `--emit=metadata`: 499 values, 452
+accepted, 212 options with at least one accepted value. Then every pair of accepted values of
+different options, and every value rejected alone against every accepted value of every other
+option: 122,682 compilations.
+
+## Which combinations rustc refuses
+
+Among values accepted alone, only **4 pairs** are rejected together, and they are one rule:
+`-Cembed-bitcode=no` with any `-Clto` other than `no`/`off`.
+
+The 47 values rejected alone fall into four groups:
+
+- **Not for this target or environment** (left out): `-Zfixed-x18`, `-Zpacked-stack`,
+  `-Zreg-struct-return`, `-Zregparm`, `-Zinstrument-mcount=fentry-*`, the hwaddress,
+  kernel-address, kernel-hwaddress, memtag and shadow-call-stack sanitizers,
+  `-Cinstrument-coverage` (no `profiler_builtins` in our sysroot), out-of-range numbers
+  (`-Cdwarf-version=1`, `-Zregparm=16`), removed options (`-Zno-parallel-backend`), and a
+  description whose backticked words are not values (`-Zcross-crate-inline-threshold=no`).
+- **Target modifiers** that differ from the sysroot's: `-Zretpoline`,
+  `-Zretpoline-external-thunk`, `-Zindirect-branch-cs-prefix`, the memory, thread, dataflow
+  and safestack sanitizers. Accepted once `-Cunsafe-allow-abi-mismatch` names them, which every
+  row below passes.
+- **Needs another option** (pair probe and by hand):
+
+  | value | needs |
+  |---|---|
+  | `-Zsplit-lto-unit=yes` | `-Clto` = yes/on/thin/fat |
+  | `-Zvirtual-function-elimination=yes` | `-Clto` = yes/on/fat |
+  | `-Zsanitizer=cfi` | `-Clto` = yes/on/fat (thin is not enough) and `-Ccodegen-units=1` |
+  | `-Zsanitizer=kcfi` | `-Cpanic=abort` |
+  | `-Zsanitizer-cfi-{diag,recover,minimal-runtime}`, `-Zsanitizer-cfi-canonical-jump-tables=no` | `-Zsanitizer=cfi` |
+  | `-Zsanitizer-cfi-minimal-runtime=yes` | also `-Zsanitizer-cfi-recover=yes` or `-Zsanitizer-cfi-diag=yes` |
+  | `-Zsanitizer-cfi-{generalize-pointers,normalize-integers}` | `-Zsanitizer=cfi` or `kcfi` |
+  | `-Zsanitizer-kcfi-arity=yes` | `-Zsanitizer=kcfi` |
+  | `-Cforce-frame-pointers=non-leaf`, `-Cpanic=immediate-abort` | `-Zunstable-options` |
+  | `-Zdump-dep-graph=yes` | `-Zquery-dep-graph=yes` |
+
+- **Stops compilation early** (left out of walks, and they made the pair probe report false
+  "needs"): `-Chelp`, `-Zhelp`, `-Zparse-crate-root-only=yes`, `-Zno-analysis`,
+  `-Zlink-only`, `-Zimplicit-sysroot-deps=no` (needs `#![no_std]`).
+
+So the declared constraints are few: 1 pairwise exclusion and 17 implications. They are in
+`rustc/flag-model.py`, which writes a [PICT](https://github.com/microsoft/pict) model.
+
+## Covering array sizes
+
+Values per option are absence plus the accepted values. "untracked" are options marked
+`[UNTRACKED]` (no effect on the incremental hash), "tracked" the rest.
+
+| subset | options | all combinations | t=2 rows (lower bound) | t=3 rows (lower bound) |
+|---|---|---|---|---|
+| untracked | 57 | 10^27.4 | 42 (36) | 230 (144) |
+| tracked | 149 | 10^74.4 | 152 (126) | 1,509 (1,008) |
+| all | 206 | 10^102.0 | 166 (126) | 1,746 (1,134) |
+
+The lower bound is the product of the largest two (three) domains. Without the constraints
+the sizes are about the same (all, t=2: 145; tracked, t=3: 1,275); constraints add rows
+because the constrained values can only appear in some rows.
+
+Every row of the all t=2 table and the tracked t=3 table was compiled (metadata, and full
+codegen to an rlib) on the trivial crate: **0 rejected** out of 1,675, with 99–137 options set
+per row. The constraint list is complete for these tables.
+
+### Transitions
+
+An incremental bug needs a change between two sessions. Modeling each option twice (before
+and after, 412 parameters for all options) and covering pairs of those covers every
+single-option change `x: v → w` and every "x = v before, y = w after":
+
+| subset | parameters | t=2 rows | t=3 rows |
+|---|---|---|---|
+| untracked | 114 | 54 | 343 |
+| all | 412 | 224 | |
+
+Each row is a clean build with A, then a rebuild with B, compared with a clean build with B.
+
+## Walking transitions on sink
+
+`rustc/flag-walk.py` takes a table from `flag-model.py --transitions --cargo` and, per row,
+builds `fixtures/sink` clean with the A options, rebuilds with the B options, builds clean
+with the B options, and compares (metadata, object code, binaries, diagnostics, the
+program's output). A difference is checked against up to 12 more clean builds first, and
+reported as P5 (nondeterminism) if clean builds differ among themselves.
+
+A real workspace adds constraints the trivial crate does not show (`CARGO_DROP` and
+`CARGO_NEEDS` in `flag-model.py`): `-Clto` is rejected for rlibs and dylibs, Cargo's target
+probe fails on values that need another option, there are no sanitizer runtimes here,
+`-Cprefer-dynamic` with `-Cpanic=abort` or LTO cannot link, and so on. Single values on sink:
+428 of 466 build; no single option, set the same in both sessions, changes a rebuild.
+
+| walk | rows | compared | findings |
+|---|---|---|---|
+| all options, pairwise transitions | 207 | 190 | 2 rows: P5, finding 10 |
+| untracked options, three-way transitions | 345 | 308 | 37 rows: one ICE, finding 11 |
+
+Three new compiler bugs came out of minimizing rows: 10 and 11 from rows that differed or
+crashed, 9 from rows that would not link:
+
+- **9**: with `-Cno-prepopulate-passes -Zshare-generics=no -Zthinlto=yes`, a dylib fails to
+  link ([facts](hunt/no-prepopulate-link.md)). Not incremental.
+- **10**: with `-g -Clto=thin` and incremental compilation, ThinLTO's input is in codegen
+  completion order, so clean builds differ from run to run
+  ([facts](hunt/thinlto-module-order.md)).
+- **11**: a session with `-Zprint-type-sizes` leaves a dependency that makes the next
+  session after an edit panic ([facts](hunt/print-type-sizes-trimmed-paths.md)).
+
+Not bugs: `-Csplit-debuginfo=packed|unpacked` objects name `.dwo` files by session (the
+walk skips object and binary comparison there); `-Zlint-llvm-ir` aborts on a known LLVM lint
+finding ([#59793](https://github.com/rust-lang/rust/issues/59793)). With `-Zthreads=4`, `-Zmir-opt-bisect-limit`
+makes metadata differ from run to run: the limit counts pass runs across the session, so which
+bodies stay under it depends on thread timing (excluded from the models). Rarely, a clean build
+reports an extra empty diagnostic: LLVM's `-Zprint-llvm-passes` listing, written from codegen
+threads, splits a `-Ztime-passes-format=json` line on stderr and Cargo reads the JSON half as
+a message (both left out of `--cargo` models).
+
+## Staying at the frontier
+
+A walk that keeps hitting a known bug finds nothing behind it, and excluding the bug from the
+model loses the coverage. So a found bug is patched locally and the walk resumes:
+
+1. `flag-walk.py --pause-on-finding` stops taking rows at the first finding and writes
+   `PAUSED` (the row and what it found); rows in flight finish.
+2. Minimize (`flag-min.py` for failing rows, delta debugging by hand for differences),
+   reproduce with plain rustc, write the facts (`docs/hunt/`).
+3. Patch `~/mirth-work/rust` (a stopgap in `docs/hunt/*-stopgap.patch`; the reproduction in
+   `docs/hunt/repro.sh` must change), build stage 1, freeze it as a new toolchain.
+4. Rerun with `--rustc <new> --recheck`: the rows with findings run first, then the rest.
+   Done rows are never repeated.
+
+`rustc/flag-campaign.sh <dir>` strings walks together this way: it exits 3 when a walk pauses,
+reads the compiler from the `<dir>/rustc` symlink, and resumes on the next run. With the
+stopgaps for findings 9–12 (`rustc-verify6`), the 37 rows of the untracked three-way walk
+that hit finding 11 all compare equal, and the reproductions of 9–12 no longer fail.
+
+## Cost
+
+`fixtures/sink` builds clean in about 2 seconds (dev profile), so one transition row (clean
+with A, rebuild with B, clean with B) is a few seconds, and the pairwise transitions walk over
+all options (224 rows) is minutes. With fuzzer edits on each row, as in the flag battery
+(120 edits per configuration), it is about 27,000 rebuilds, a few hours at the fuzzer's rate
+of about 2 edits a second. Three-way over all options at roughly 1,600 rows and 120 edits
+each is about a day.
+
+## Reproduce
+
+    rustc/flag-universe.py --rustc <rustc> --source <rust checkout> --work <dir> --jobs 10
+    rustc/flag-model.py <dir> all model.txt              # or untracked / tracked, --transitions
+    pict model.txt /o:2 /r:1 > rows.tsv
+    rustc/flag-rows.py <dir> rows.tsv <rustc> --emit=metadata
+    rustc/flag-model.py <dir> untracked tr.txt --transitions --cargo
+    pict tr.txt /o:3 /r:1 > tr.tsv
+    rustc/flag-walk.py --rustc <rustc> --fixture fixtures/sink --flags <dir> --table tr.tsv --work <walk dir>

docs/hunt.md

@@ -32,6 +32,12 @@ with `-Zthreads=8`. `rustc/check.sh wide` runs the ordinary checks.
 | 6 | reused object code keeps the previous checksum of an edited source file in its debuginfo, and with `-Zembed-source` the previous file; with optimizations, ThinLTO symbol names then differ from a clean build | **looks new**; found by the fuzzer at `-Copt-level=2`; root cause found, since 1.44 (#69718); report drafted and a regression test (`hunt/tests/incr-debuginfo-embedded-source`, failing: no fix) |
 | 7 | warnings from inline assembly are not shown again when an incremental rebuild reuses the codegen unit | **looks new**; found while checking reused codegen units ([`shadow-mode.md`](shadow-mode.md)); on 1.60.0 through the nightly; report drafted ([draft](hunt/issue-asm-warnings-reused-cgu.md)) and a regression test (`hunt/tests/incr-asm-warning-reused`, failing: no fix) |
 | 8 | with `-Zmir-opt-level=3`, an incremental rebuild encodes an allocation from inlined `core` MIR twice where a clean build encodes it once | **looks new**; found by the fuzzer at `-Zmir-opt-level=4` and named by the reuse check, reduced to one line; on 1.60.0 through the nightly; report drafted ([draft](hunt/issue-inlined-alloc-identity.md)), no fix |
+| 9 | a dylib fails to link (undefined hidden symbols) with `-Cno-prepopulate-passes -Zshare-generics=no` and local ThinLTO | **looks new**, not incremental, unstable flags only; found by the flag transitions walk ([`flags.md`](flags.md)); since 1.78; [facts](hunt/no-prepopulate-link.md), local stopgap: no local ThinLTO with `-Cno-prepopulate-passes` ([patch](hunt/no-prepopulate-thinlto-stopgap.patch)) |
+| 10 | with `-g -Clto=thin` and incremental compilation, clean builds give different object files from run to run, and since 1.90 (rust-lld) different binaries | **looks new**; found by the flag transitions walk ([`flags.md`](flags.md)), first as a rebuild differing from a clean build; objects differ since at least 1.60; cause found (ThinLTO input in codegen completion order); [facts](hunt/thinlto-module-order.md), local stopgap: inputs sorted by name ([patch](hunt/thinlto-order-stopgap.patch)) |
+| 11 | an incremental rebuild after an edit panics ("`trimmed_def_paths` called, diagnostics were expected but none were emitted") when the previous session had `-Zprint-type-sizes` and the crate has an `async fn` awaiting another | **looks new**; found by the three-way walk over untracked option transitions ([`flags.md`](flags.md)); since 1.79; cause found (an awaited type formatted with trimmed paths inside `layout_of`); [facts](hunt/print-type-sizes-trimmed-paths.md), local stopgap: the field type formatted without trimmed paths ([patch](hunt/print-type-sizes-trimmed-stopgap.patch)) |
+| 12 | rustc segfaults in LLVM's DWARF emission with `-g -Crelocation-model=rwpi` on x86_64 when the crate has a writable static | **looks new**; stable flags; found by trying `-Crelocation-model` values on sink ([`flags.md`](flags.md)); since 1.60 (LLVM 14); [facts](hunt/rwpi-debuginfo-segfault.md), local stopgap: `rwpi` and `ropi-rwpi` rejected off ARM ([patch](hunt/rwpi-stopgap.patch)) |
+| 13 | LLVM's machine outliner (`-Cllvm-args=-enable-machine-outliner`) segfaults with retpolines at `-Copt-level` 1 and up, and fails in other combinations (`-Zcf-protection` with `-Zpatchable-function-entry`, the large code model) | in LLVM; unstable or raw LLVM flags; found by the flag walk with `flag-min.py`; since at least 1.71; [facts](hunt/llvm-retpoline.md); excluded from the models |
+| 14 | `-Ccode-model=large` with retpolines: a dylib cannot link (absolute relocation to the retpoline thunk) | in LLVM; unstable flags; found as 13; [facts](hunt/llvm-retpoline.md); excluded from the models |
 
 Findings 1 and 2 are single-threaded: an ordinary `cargo build`, an edit, another
 `cargo build`, and the metadata differs from a clean build of the edited source. Both come

docs/hunt/llvm-retpoline.md

@@ -0,0 +1,52 @@
+# Findings 13 and 14: retpoline in LLVM, with the machine outliner and the large code model
+
+Facts for reports; the reports themselves are for a person to write (rust-lang/rust's LLM
+policy). Both are in LLVM's x86 backend, reached through rustc options; LLVM has its own
+tracker. No local patch: these are excluded from the flag models instead (they would need an
+LLVM build).
+
+## 13: SIGSEGV with `-Cllvm-args=-enable-machine-outliner` and retpolines
+
+**Repro.**
+
+    cat > a.rs <<'RS'
+    pub fn f(v: &[u32]) -> u32 { v.iter().map(|x| x * 3).sum() }
+    pub fn g(v: &[u32]) -> u32 { v.iter().map(|x| x * 5).sum::<u32>() + f(v) }
+    pub fn h(b: Box<dyn Fn(u32) -> u32>) -> u32 { b(1) + b(2) }
+    RS
+    rustc --crate-type lib -Copt-level=1 -Cllvm-args=-enable-machine-outliner \
+        -Zretpoline=yes -Zunstable-options -Cunsafe-allow-abi-mismatch=retpoline a.rs
+
+**Expected.** An rlib. **Actual.** `rustc interrupted by SIGSEGV`, exit 139.
+
+**Conditions.** Any `-Copt-level` from 1 (also `s`, `z`); without retpolines it builds. Before
+`-Zretpoline` existed, `-Ctarget-feature=+retpoline-indirect-calls,+retpoline-indirect-branches`
+gives the same crash.
+
+**Versions.** 1.71.0, 1.87.0 (target features), 1.93.0, 1.98.1, nightly-2026-10-06
+(`-Zretpoline`).
+
+**The outliner fails in other combinations too** (minimized from walk rows with
+`rustc/flag-min.py`, on sink):
+
+- `-Cllvm-args=-enable-machine-outliner -Copt-level=3 -Zcf-protection=full
+  -Zpatchable-function-entry=4,2 -Cdebuginfo=none`: SIGSEGV.
+- `-Cllvm-args=-enable-machine-outliner -Copt-level=2 -Ccode-model=large` (with a few more
+  options): `error: symbol '.L6$pb' can not be undefined in a subtraction expression`.
+
+The models now leave the outliner out entirely (`DROP` in `rustc/flag-model.py`).
+
+## 14: `-Ccode-model=large` with retpolines cannot link a dylib
+
+**Repro.**
+
+    echo 'pub fn h(b: &dyn Fn(u32) -> u32) -> u32 { b(1) }' > c.rs
+    rustc --crate-type dylib -Ccode-model=large -Zretpoline=yes \
+        -Zunstable-options -Cunsafe-allow-abi-mismatch=retpoline c.rs
+
+**Expected.** `libc.so`. **Actual.** `rust-lld: error: relocation R_X86_64_64 cannot be used
+against symbol '__llvm_retpoline_r11'; recompile with -fPIC`: the call to the retpoline thunk
+is emitted as an absolute address in position-independent code.
+
+**How mirth found them.** The pairwise walk over all options with the sample values
+(`docs/flags.md`), then `rustc/flag-min.py` on the rows that failed.

docs/hunt/no-prepopulate-link.md

@@ -0,0 +1,39 @@
+# Finding 9: a dylib fails to link with -Cno-prepopulate-passes and ThinLTO without shared generics
+
+Facts for a report; the report itself is for a person to write (rust-lang/rust's LLM policy).
+
+**Repro.** `docs/hunt/repro.sh`, "no-prepopulate-link":
+
+    echo 'pub fn f(a: &mut u8, b: &mut u8) { core::mem::swap(a, b) }' > a.rs
+    rustc --edition 2021 --crate-type dylib -Ccodegen-units=16 \
+        -Cno-prepopulate-passes -Zshare-generics=no -Zthinlto=yes a.rs
+
+**Expected.** `liba.so`.
+
+**Actual.** `rust-lld: error: undefined hidden symbol` for
+`<usize>::unchecked_add::precondition_check`, `core::ptr::swap_nonoverlapping::<u8>`,
+`<*const ()>::is_aligned_to` and `core::ub_checks::maybe_is_nonoverlapping::runtime`.
+
+**Conditions.** All three flags are needed; any number of codegen units from 2. A cdylib
+fails the same way. A bin and a staticlib link. `-Copt-level=1` with
+`-Cno-prepopulate-passes -Zshare-generics=no` (local ThinLTO is on by default there) fails
+too. With `-Clto=thin` it links. Not specific to incremental compilation.
+
+**Variant.** `-Clink-dead-code=yes` in place of `-Zshare-generics=no` fails too, and then a
+binary and a cdylib fail as well (a dylib links): `rustc --crate-type bin
+-Ccodegen-units=16 -Clink-dead-code=yes -Cno-prepopulate-passes -Zthinlto=yes` on a `main`
+calling `core::mem::swap`. Found by minimizing the link failures of the pairwise walk
+(`rustc/flag-min.py`).
+
+**Versions.** Stable releases with `RUSTC_BOOTSTRAP=1`: links on 1.53.0 through 1.77.0,
+fails on 1.78.0 through 1.98.1 and nightly-2026-10-06. 1.78 is when these `ub_checks`
+helpers appeared in `core`, so older releases may only lack a function that shows it.
+
+**Cause, as far as followed.** With `-Csave-temps`, the defining codegen unit's copy is
+`define hidden` in `thin-lto-input` and `define internal` from `thin-lto-after-internalize`
+on, while the calling unit still only `declare`s it after `thin-lto-after-import`: ThinLTO
+internalized a definition another module needs and the import did not happen. Why
+`-Cno-prepopulate-passes` changes this was not followed further.
+
+**How mirth found it.** The transitions walk on `fixtures/sink` (`docs/flags.md`), whose
+`sink-dy` crate is a dylib, then a delta minimization of the row's 130 flags.

docs/hunt/no-prepopulate-thinlto-stopgap.patch

@@ -0,0 +1,15 @@
+--- a/compiler/rustc_session/src/session.rs
++++ b/compiler/rustc_session/src/session.rs
+@@ -1036,6 +1036,12 @@
+             return config::Lto::No;
+         }
+
++        // Without the default passes, local ThinLTO internalizes definitions that another
++        // codegen unit still references, and linking fails.
++        if self.opts.cg.no_prepopulate_passes {
++            return config::Lto::No;
++        }
++
+         // If `-Z thinlto` specified process that, but note that this is mostly
+         // a deprecated option now that `-C lto=thin` exists.
+         if let Some(enabled) = self.opts.unstable_opts.thinlto {