The C cases link without DWARF: the testcases ROOT goes from 305 to 209 MiB, and the checkout-path issue records the paths every guest binary carries

[Japabu]

The C test programs are now linked without DWARF, the same way [profile.toyos] links every Rust guest binary. They keep .symtab. On the T14 testcases image, ROOT shrinks from 305.00 to 209.00 MiB and the whole image from 410 to 314 MiB. The checkout-path issue was also corrected: every guest binary carries absolute host paths, including snake, which the issue said carried none.

What changed and why

• tests/common/compile.rs: link_toyos passes -Wl,--strip-debug. Every C program the harness links goes through it: the 137 corpus cases and tests/netcase's three libc_sockets programs. libtoyos_c.a brought its Rust libraries' DWARF into each case, 0.70 MiB per case and 96.48 MiB in total, and nothing read it. The .symtab/.strtab sum (44.05 MiB) and the .text sum (56.31 MiB) are the same to the hundredth of a MiB before and after. So the linker dropped only the debug sections. One link step for every C program, so no second path. The strip is at the link, the same place rustc strips a Rust binary (strip = "debuginfo"). This also makes src/qemu.rs's header true for C programs: "every guest profile strips it at the link".
• issues/two-checkouts-of-one-tree-build-different-guest-bytes.md: now matches what was measured. Paths are named here by kind, never by their characters.
  • Every guest binary carries the path of the checkout that built the sysroot store's std and C library, which can be a different checkout from the one building. These are mostly panic locations of toyos, toyos-abi and toyos-osrelease. That is all 250 ELF files on the testcases ROOT (23 in each C case once stripped) and all 27 programs on the cargo run ROOT (6 in snake).
  • 45 test_rs_* binaries and libtls_cranelift.so carry the building checkout's own path, because tests/toyos-rust-tests is a workspace of its own.
  • Every guest binary, the kernel (7) and the bootloader (40) carry the cargo home's path.
  • The kernel and snake carry no path into the checkout that built them. That is what the issue's two-checkout comparison varied, so its symbol counts still stand as recorded. They were not re-measured here.
  • The issue now names the command, LC_ALL=C grep -a -o "$HOME/" <binary> | wc -l. It gives 42 for snake (6 + 36), 7 for the kernel and 40 for the bootloader, run on this worktree's target/ files.
  • The paths themselves are not fixed in this branch, as the brief directs.

Does anything read a C case's DWARF? No.

• libc's #[panic_handler] prints libc panic: {info} and exits 134. It prints no backtrace. A C case carries no gimli or addr2line. With the strip, its only home-directory paths are 23 into the sysroot-building checkout and one into dlmalloc in the cargo home, all in .rodata. The 24th checkout path was .debug_str's compilation directory, which the strip removed.
• A search for gimli|addr2line|debug_line|debug_info|.debug_|dwarf outside rust/ finds no reader. Its only hits are src/qemu.rs's "There is no DWARF" header and PT_GNU_EH_FRAME comments.
• The kernel's crash report and symbolize name frames from .symtab (kernel/src/loader/symbols.rs, src/qemu.rs's header). The kernel's dlopen binding reads the executable's .symtab. Both are kept.
• ccheck compares stdout with .expect. The corpus's one backtrace case, 112_backtrace, is declined in NOT_RUN: it is a meta-test of tcc's -b runtime.

Measured: ROOT before and after

The method is #804's. The testcases image is staged with --metal-readback … boot:testcases --nocapture, and ROOT's files are taken from that run's own root: adding lines (389 files, the same list both times). Each file is read back off the image through toyos_build::image::root_file_on, and its ELF section headers are summed. ROOT's extent comes from the image's slot and partition tables. The tool, the per-kind sums and the path counts are in the comment on this pull request.

	before (a944746d5)	after (bb036abef)
image	429,916,160 B (410 MiB)	329,252,864 B (314 MiB)
ROOT partition	305.00 MiB	209.00 MiB
ROOT's files	302.89 MiB	206.35 MiB
C cases (137)	224.17 MiB, 96.48 DWARF	127.63 MiB, 0.00 DWARF
C cases' .symtab+.strtab	44.05 MiB	44.05 MiB
C cases' .text	56.31 MiB	56.31 MiB
Rust tests, cdylibs, shipped programs	59.03 / 11.22 / 8.44 MiB	unchanged

Flash time: not separable on this base. This base has no phase timing (#804 adds it), so the T14 run below records no flash_secs. #804's reading at 52bb3de20 flashed 430,964,736 bytes in 66.6 s of dd, about 0.155 s per MB; the 100.7 MB cut is therefore about 15.6 s per flash by that rate. That is an estimate, not a reading, and the earlier "about 27 s" in this body was arithmetic on a wrong rate. What the T14 run did measure: the testcases boot's driver time outside the boot itself (wall time minus back_secs) was 90 s, against 101 s for #804's unstripped image the same afternoon; one boot against one boot, so no timing verdict.

Gates (head bb036abef)

gate	exit
cargo run -- --ci host (78 steps), at 8d9179496 and again at bb036abef	0 and 0
cargo run -- --build-only (at 38c8fac04: the same tree minus the issue file)	0
cargo test at 8d9179496 (the guest suite, once; bb036abef changes only three lines of the issue file, which no guest test reads: 38 passed of 38; uptime load averages 55.56/38.78/31.24 before and 78.89/67.67/48.02 after)	0
cargo test --test toyos-build -- --metal --metal-readback <scratch>/cstrip/metal/ boot:testcases (stages only; touches no machine)	2, the Staged verdict by design

Closing lines of each gate's log: host 15:03:30 [ci] Host: 78 step(s), all green / EXIT=0 (at bb036abef); guest suite test result: ok. 38 passed, 38 total (302.9s; …) / EXIT=0 (at 8d9179496), with libc_sockets among the 38; staging [toyos] Compiling 137 C tests, and attempting 24 declared ones... then [metal] staged 2 image(s) / EXIT=2.

What the guest suite reaches, and what it does not. At this base the QEMU suite does not run the C corpus. Its shared boots are metal-only now: the 38 tests include no testcases shared boot, and the suite's log has no Compiling … C tests line. The suite reaches this change through libc_sockets (PASS), which links tests/netcase's three C programs through link_toyos. The corpus itself was compiled and linked through the new flag on the host when the image was staged. check_not_run's declared stages all held, or the staging would have panicked.

T14 boot:testcases at bb036abef, run by the orchestrator. Images as staged here, each hash checked against request.txt before its flash (testcases 63f1eb53…04713855, testcases-watchdog d24014d0…92844a928); each toyos-metal --image <img> --readback <dir> --fat32-check exited 0 with verdict passed. Judge cargo test --test toyos-build -- --metal --metal-readback <dir> boot:testcases: EXIT=0, [metal] 248 passed, 0 failed, 2 boot(s).

• The C corpus by name: the boot log pairs each case's spawn (spawn: /system/bin/<case>) with the comparator's verdict (ccheck: <code>): 137 cases, every one ccheck: 0 (its output matched its .expect), and the 137 names with their codes are identical to those of toyos-metal writes where each boot's wall time went into boot.txt, and the judge prints it #804's unstripped testcases boot at 52bb3de20 (main 8dbccd3ab, the same corpus as a944746d5): no member went red.
• back_secs 134 (testcases) and 77 (testcases-watchdog), against 104 and 65 in toyos-metal writes where each boot's wall time went into boot.txt, and the judge prints it #804's boots: slower, from one boot each; stripping DWARF from programs the boot runs does not plausibly cost 30 s, but no measurement here says why.

Negative control. The before arm is the base a944746d5 with the change absent. It was staged and measured in this session with the same tool (96.48 MiB DWARF, ROOT 305.00 MiB).

No mutation. The change is one flag. Its effect is the measured byte delta, and the base arm above is the arm without it.

Images staged for the T14, sha256:

• testcases: 63f1eb5363929aa0d7cc01fb129a91f125961d46eb1a7af74f52528004713855
• testcases-watchdog: d24014d01707ec11dcace9ba93472aeb2d71b38015b56d378d2b08492844a928

Net lines (git diff --shortstat origin/main...HEAD): 2 files, +31 −4. Production: none. Harness: tests/common/compile.rs +6, one flag and its doc. The issue: +25 −4.

What I am unsure of

• The flash saving is derived, not measured (above); toyos-metal writes where each boot's wall time went into boot.txt, and the judge prints it #804's flash_secs gives it on the first testcases boot after both land.
• The cargo run ROOT's path counts were taken at 38c8fac04. Its guest sources are a944746d5's, because this branch touches no guest source.

🤖 Generated with Claude Code

https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C

[Japabu]

The measurement behind the body: method, tool and per-kind sums. Paths to the home directory are masked.

ROOT's file list is the staging run's own root: adding lines for the testcases image (--nocapture; 389 files before and after, the same list). Each file is read back off the staged image through toyos_build::image::root_file_on, ROOT's extent comes from image::slot_table_of and image::partition_extent, and each ELF's section headers are summed by kind with object 0.38.1 (the version Cargo.lock already resolves). The tool also counts occurrences of the home directory's prefix per file and classifies each as the cargo home, this worktree, or another checkout.

Sums

before (a944746d5)

# image_bytes	429916160	root_start	109051904	root_bytes	319815680	root_mib	305.00
all files MiB=302.89
c_cases	files=137	MiB=224.17	dwarf_MiB=96.48	symtab_strtab_MiB=44.05	text_MiB=56.31
cdylibs	files=4	MiB=11.22	dwarf_MiB=0.00	symtab_strtab_MiB=2.71	text_MiB=6.50
etc	files=2	MiB=0.01	dwarf_MiB=0.00	symtab_strtab_MiB=0.00	text_MiB=0.00
expect	files=137	MiB=0.02	dwarf_MiB=0.00	symtab_strtab_MiB=0.00	text_MiB=0.00
rs_tests	files=100	MiB=59.03	dwarf_MiB=0.00	symtab_strtab_MiB=14.81	text_MiB=31.95
shipped	files=9	MiB=8.44	dwarf_MiB=0.00	symtab_strtab_MiB=2.21	text_MiB=4.40

after (8d9179496)

# image_bytes	329252864	root_start	109051904	root_bytes	219152384	root_mib	209.00
all files MiB=206.35
c_cases	files=137	MiB=127.63	dwarf_MiB=0.00	symtab_strtab_MiB=44.05	text_MiB=56.31
cdylibs	files=4	MiB=11.22	dwarf_MiB=0.00	symtab_strtab_MiB=2.71	text_MiB=6.50
etc	files=2	MiB=0.01	dwarf_MiB=0.00	symtab_strtab_MiB=0.00	text_MiB=0.00
expect	files=137	MiB=0.02	dwarf_MiB=0.00	symtab_strtab_MiB=0.00	text_MiB=0.00
rs_tests	files=100	MiB=59.03	dwarf_MiB=0.00	symtab_strtab_MiB=14.81	text_MiB=31.95
shipped	files=9	MiB=8.44	dwarf_MiB=0.00	symtab_strtab_MiB=2.21	text_MiB=4.40

Paths under the home directory, a944746d5 testcases ROOT (before)

c_cases files=137 | std-building checkout: files=137 hits=3288 | building checkout: files=0 hits=0 | cargo home: files=137 hits=137
cdylibs files=4 | std-building checkout: files=4 hits=24 | building checkout: files=1 hits=11 | cargo home: files=4 hits=230
rs_tests files=100 | std-building checkout: files=100 hits=733 | building checkout: files=45 hits=103 | cargo home: files=100 hits=1815
shipped files=9 | std-building checkout: files=9 hits=67 | building checkout: files=0 hits=0 | cargo home: files=9 hits=178

Paths under the home directory, cargo run -- --build-only ROOT (27 ELF programs)

bin/acpiserver	cargo_home=18	building_checkout=0	std_building_checkout=6
bin/calc	cargo_home=32	building_checkout=0	std_building_checkout=6
bin/compositor	cargo_home=134	building_checkout=0	std_building_checkout=10
bin/diskserver	cargo_home=18	building_checkout=0	std_building_checkout=6
bin/doom	cargo_home=62	building_checkout=0	std_building_checkout=6
bin/editor	cargo_home=18	building_checkout=0	std_building_checkout=6
bin/filepicker	cargo_home=18	building_checkout=0	std_building_checkout=7
bin/files	cargo_home=134	building_checkout=0	std_building_checkout=7
bin/fileserver	cargo_home=18	building_checkout=0	std_building_checkout=6
bin/host	cargo_home=18	building_checkout=0	std_building_checkout=7
bin/inspect	cargo_home=18	building_checkout=0	std_building_checkout=6
bin/logkeeper	cargo_home=18	building_checkout=0	std_building_checkout=7
bin/netstack	cargo_home=44	building_checkout=0	std_building_checkout=6
bin/paint	cargo_home=18	building_checkout=0	std_building_checkout=6
bin/pkg	cargo_home=33	building_checkout=0	std_building_checkout=7
bin/proctest	cargo_home=18	building_checkout=0	std_building_checkout=9
bin/shell	cargo_home=18	building_checkout=0	std_building_checkout=10
bin/snake	cargo_home=36	building_checkout=0	std_building_checkout=6
bin/soundserver	cargo_home=25	building_checkout=0	std_building_checkout=6
bin/sshserver	cargo_home=263	building_checkout=0	std_building_checkout=11
bin/supervisor	cargo_home=21	building_checkout=0	std_building_checkout=9
bin/swap	cargo_home=18	building_checkout=0	std_building_checkout=6
bin/symbolize	cargo_home=21	building_checkout=0	std_building_checkout=6
bin/terminal	cargo_home=18	building_checkout=0	std_building_checkout=9
bin/toybox	cargo_home=21	building_checkout=0	std_building_checkout=12
bin/trace	cargo_home=18	building_checkout=0	std_building_checkout=6
bin/update	cargo_home=29	building_checkout=0	std_building_checkout=6

The tool (scratch crate; object 0.38.1 and toyos-build by path, the root manifest's [patch.crates-io] copied)

//! rootelf <image> <files-list> <needle> <out.tsv> <paths-local.txt> <this-worktree>
//! Reads ROOT's extent off the image's partition table, then each listed file
//! back off ROOT through `toyos_build::image::root_file_on`, and sums its ELF
//! sections by kind; counts occurrences of <needle> per file and per section.
use object::read::elf::{ElfFile64, SectionHeader};
use object::{Object, ObjectSection};
use std::collections::BTreeMap;
use std::io::Write;
use std::path::Path;

fn main() {
    let a: Vec<String> = std::env::args().collect();
    let image = Path::new(&a[1]);
    let names: Vec<String> = std::fs::read_to_string(&a[2]).unwrap().lines().map(str::to_string).collect();
    let needle = a[3].as_bytes();
    let mut out = std::fs::File::create(&a[4]).unwrap();
    let mut local = std::fs::File::create(&a[5]).unwrap();

    let mut f = std::fs::File::open(image).unwrap();
    let table = toyos_build::image::slot_table_of(&mut f).unwrap();
    let slot = table.slot(table.marked).unwrap();
    let (start, len) = toyos_build::image::partition_extent(&mut f, slot.root).unwrap();
    let total = std::fs::metadata(image).unwrap().len();
    writeln!(out, "# image_bytes\t{total}\troot_start\t{start}\troot_bytes\t{len}\troot_mib\t{:.2}", len as f64 / 1048576.0).unwrap();
    writeln!(out, "name\tbytes\tdebug\tsymtab_strtab\ttext\trodata_eh\tother_sections\tneedle_hits\tneedle_sections\tclasses").unwrap();

    let mut distinct: BTreeMap<String, usize> = BTreeMap::new();
    for name in &names {
        let (_, bytes) = toyos_build::image::root_file_on(image, name).unwrap();
        let hits: Vec<usize> = bytes.windows(needle.len()).enumerate().filter(|(_, w)| *w == needle).map(|(i, _)| i).collect();
        for &h in &hits {
            let end = bytes[h..].iter().position(|&b| b == 0 || !(0x20..0x7f).contains(&b)).map_or(bytes.len(), |e| h + e);
            *distinct.entry(String::from_utf8_lossy(&bytes[h..end]).into_owned()).or_default() += 1;
        }
        let Ok(elf) = ElfFile64::<object::Endianness>::parse(&*bytes) else {
            writeln!(out, "{name}\t{}\t-\t-\t-\t-\t-\t{}\tnot-elf", bytes.len(), hits.len()).unwrap();
            continue;
        };
        let (mut dbg, mut sym, mut text, mut ro, mut other) = (0u64, 0u64, 0u64, 0u64, 0u64);
        let mut in_sections: BTreeMap<String, usize> = BTreeMap::new();
        for s in elf.sections() {
            let n = s.name().unwrap_or("");
            let hdr = s.elf_section_header();
            if hdr.sh_type(elf.endian()) == object::elf::SHT_NOBITS {
                continue;
            }
            let size = hdr.sh_size(elf.endian());
            let off = hdr.sh_offset(elf.endian()) as usize;
            for &h in &hits {
                if h >= off && h < off + size as usize {
                    *in_sections.entry(n.to_string()).or_default() += 1;
                }
            }
            if n.starts_with(".debug") {
                dbg += size;
            } else if n == ".symtab" || n == ".strtab" {
                sym += size;
            } else if n == ".text" {
                text += size;
            } else if n.starts_with(".rodata") || n.starts_with(".eh_frame") || n == ".gcc_except_table" {
                ro += size;
            } else {
                other += size;
            }
        }
        let secs: Vec<String> = in_sections.iter().map(|(k, v)| format!("{k}:{v}")).collect();
        // Which tree each hit points into: the cargo home, this worktree, or another checkout.
        let own = a[6].as_bytes();
        let (mut cargo, mut mine, mut elsewhere) = (0, 0, 0);
        for &h in &hits {
            let rest = &bytes[h..(h + 200).min(bytes.len())];
            if rest[needle.len()..].starts_with(b".cargo/") {
                cargo += 1;
            } else if rest.starts_with(own) {
                mine += 1;
            } else {
                elsewhere += 1;
            }
        }
        writeln!(out, "{name}\t{}\t{dbg}\t{sym}\t{text}\t{ro}\t{other}\t{}\t{}\tcargo_home={cargo}\tthis_worktree={mine}\tother_checkout={elsewhere}", bytes.len(), hits.len(), secs.join(",")).unwrap();
    }
    for (s, n) in distinct {
        writeln!(local, "{n}\t{s}").unwrap();
    }
}

[Japabu]

Review, round 1, head bb036abef (base a944746d5)

Net lines (git diff --shortstat origin/main...bb036abef): 2 files, +31 −4. Production: 0. Harness (tests/common/compile.rs): +6, one flag and its doc. Tracker (issues/two-checkouts-of-one-tree-build-different-guest-bytes.md): +25 −4.

I checked the issue's counts against the implementer's root-elf-final.tsv (testcases ROOT, after) and they hold: 250 ELF files, all carrying the sysroot-building checkout's path; 137 C cases at 23 each; bin/fileserver 6; 45 test_rs_* binaries plus lib/libtls_cranelift.so carrying the building checkout's path. The log timestamps put ci-host.log and guest.log after 8d9179496 and ci-host-final.log after bb036abef. Each log ends on EXIT=0, apart from the staging logs, which end on EXIT=2, the Staged verdict. Neither the PR body nor its comment contains a home path.

BLOCKER

• PR body, "Gates" — Evidence: no log at all. The body gives each gate's command and exit code but not a single line of any log. Everything it says passed (Host: 78 step(s), all green, 38 passed, 38 total, PASS libc_sockets, Compiling 137 C tests, and attempting 24 declared ones) rests only on logs in a local scratch directory. Fix: put each gate's closing lines and exit code in the body, from ci-host-final.log, guest.log and stage-final.log.

• tests/common/compile.rs:123 — Evidence: the hardware reading is missing. On this base the C corpus runs only on the T14. QEMU does not run it: the suite has no testcases shared boot, and libc_sockets reaches only tests/netcase's three programs. So no run has yet shown that the 137 stripped cases run and match their .expect. The orchestrator's boot:testcases reading at bb036abef closes this blocker only if it shows all of the following:

  1. Head and image: the head is bb036abef, and the image's sha256 is stated (the body's 63f1eb53…, or the orchestrator's own staging of this head).
  2. Exit code: the run's exit code and its verdict line.
  3. Every boot the word stages is green: both testcases and testcases-watchdog.
  4. All 137 C corpus members PASS, by name: each output matched by ccheck against its .expect, and the declared NOT_RUN / check_not_run stages hold as declared.
  5. No member that passed at a944746d5 goes red.
  6. flash_secs from boot.txt: this replaces the body's derived "about 27 s" with a reading.

  A red member means a defect; it is not re-run.

NOTE

• PR body, "At toyos-metal writes where each boot's wall time went into boot.txt, and the judge prints it #804's measured flash rate … about 27 s per flash" — an estimate that the T14 reading's flash_secs should replace before landing (prose).

SEND BACK

[Japabu]

Review, round 2, head bb036abef (base a944746d5; unchanged since round 1)

Net lines (git diff --shortstat origin/main...bb036abef): 2 files, +31 −4. Production: 0. Harness (tests/common/compile.rs): +6. Tracker: +25 −4. 8d9179496..bb036abef and 38c8fac04..bb036abef touch only the issue file, so the guest suite at 8d9179496 and the build at 38c8fac04 cover the same harness and guest sources as this head.

Round 1's blockers

• Evidence: no log in the body. CLOSED. The body now quotes each gate's closing lines and exit code. They match the logs: ci-host-final.log ends 15:03:30 [ci] Host: 78 step(s), all green / EXIT=0 (the log was written after bb036abef was committed). guest.log has test result: ok. 38 passed, 38 total (302.9s; …), then PASS libc_sockets (34s) and EXIT=0, with uptime readings that bracket 8d9179496's commit time. stage-final.log has Compiling 137 C tests, and attempting 24 declared ones... and [metal] staged 2 image(s), then EXIT=2.
• Evidence: the hardware reading. CLOSED. Each of round 1's conditions holds:
  1. Head and image. request.txt stages both images at bb036abef. The driver log records testcases boot rc=0 sha=63f1eb53…04713855 and testcases-watchdog boot rc=0 sha=d24014d0…92844a928, and both hashes equal request.txt's.
  2. Exit code. The driver log has judge EXIT=0 [metal] 248 passed, 0 failed, 2 boot(s), and judge.log ends on the same verdict line.
  3. Both boots green. verdict.txt reads passed in both readbacks. judge.log reports 225 of 225 with both markers.
  4. All 137 C corpus members, by name. In t14-cstrip-testcases.log I paired each spawn: /system/bin/<case> pid=P with the test-runner pid=P] ccheck: <code> that the same pid printed. That gives 137 distinct names and 137 × ccheck: 0. Each case's ===TEST_END <case> exit=0=== is present, and so is each exit: test_c_<case> … code=0. In ccheck.rs, 0 is MATCHED: the case exited 0 and its output equalled its .expect. NOT_RUN / check_not_run held at staging (attempting 24 declared ones, and no panic before staged 2 image(s)).
  5. No member went red. The same pid pairing on t14-s0r2-testcases.log gives an identical sorted list of (name, code) pairs. That boot is at 52bb3de20, whose compile.rs has no strip flag, and its corpus tree is the same as a944746d5's.
  6. flash_secs. This no longer blocks. bb036abef's src/metal.rs records only back_secs; flash_secs first exists at 52bb3de20 (toyos-metal writes where each boot's wall time went into boot.txt, and the judge prints it #804), so this base cannot produce the reading. The body now labels the flash saving an estimate and draws no timing verdict from it. Its arithmetic checks out: 100,663,296 B at toyos-metal writes where each boot's wall time went into boot.txt, and the judge prints it #804's 430,964,736 B / 66.6 s of dd comes to about 15.6 s. Nothing the change claims depends on that number. The reading this change actually claims, the bytes, is measured. My root-elf-before.tsv / root-elf-final.tsv sums reproduce the body's table: 137 C cases, 224.17 → 127.63 MiB; DWARF 96.48 → 0.00; .symtab+.strtab 44.05 = 44.05; .text 56.31 = 56.31; ROOT 305.00 → 209.00 MiB; image 429,916,160 → 329,252,864 B.

BLOCKER

None.

NOTE

None.

LAND

[Japabu]

CI at bb036abef, read from each job's log: host 79 steps all green; toolchain restored (llvm a2cc063281d9f279, compiler 6c76c19e5ffcc869, freestanding 0f5a5faab7bbc78c, sysroot fcaf4949f5310fa6); guest suite 41/41 ok. Merges clean with main d6298c83e. Queued.