Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,406 changes: 1,334 additions & 72 deletions Cargo.lock

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ members = [
"toyos-net-shard/testnet",
"toyos-net-udp",
"toyos-net-wire",
"toyos-osrelease",
"toyos-quiesce",
"toyos-rootimage",
"toyos-swap",
Expand Down Expand Up @@ -141,6 +142,7 @@ toyos-keymap = { path = "toyos-keymap" }
# on, so the judges reading the log and `logkeeper` writing it read one declaration.
toyos-logstream = { path = "toyos-logstream" }
toyos-manifest = { path = "toyos-manifest" }
toyos-osrelease = { path = "toyos-osrelease" }
# The shutdown's stop record: the kernel renders it and this crate reads it back
# off a stick, so one declaration is what both sides spell it with.
toyos-quiesce = { path = "toyos-quiesce" }
Expand Down Expand Up @@ -176,6 +178,9 @@ flate2 = { version = "1", default-features = false, features = ["rust_backend"]
# takes (`issues/the-internet-clients-work-unchanged.md`), whose C
# and assembly the host's `cc` compiles.
ureq = { version = "3", default-features = false, features = ["rustls"] }
# The commit an image is built from, its time and whether the tree is that
# commit's, where `git` is a host's C tool (`src/build.rs`'s `release`).
gix = { version = "0.88", default-features = false, features = ["status", "sha1"] }

[dev-dependencies]
# `USER_TOP`, so the harness judges a held `rsp` against the bound the kernel
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
status: open
kind: defect
opened: 2026-10-04
---

# A supervisor panic before logkeeper starts reaches no console

`/system/bin/supervisor` makes its own lines records in its log ring first thing
(`Log::open` in `userland/supervisor/src/main.rs`), and only `logkeeper`
drains a ring. So a panic before `logkeeper` runs leaves its message in a
ring nobody reads. The refusals of a ROOT without
`/system/etc/os-release`, or without a readable `/system/etc/system.manifest`,
are such panics.

Measured with ROOT carrying no os-release (`machine_shutdown`, x86-64, QEMU):
the console's last kernel record is `exit: supervisor pid=0 code=101`, the
panic's words appear nowhere, and the harness waits out its 63 s boot ceiling
for `===READY===`. It does not treat the supervisor's end as a death.

**Exit**: a supervisor panic before `logkeeper` serves puts its message on the
console, and the harness ends a boot whose supervisor exited, naming the exit.
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
status: open
kind: defect
opened: 2026-10-04
---

# An image's signed version and its FAT dates are the host's clock

Two builds of one tree write different images, so ToyOS rebuilding itself
cannot reproduce the host's bytes. Two sites in `src/image.rs` read the clock:

- `version_now()`, the signed header's anti-rollback version for a build that
names none (`src/build.rs`'s plans call it).
- `build_time()`, which dates every file and directory `populate` writes onto
the ESP and LOG volumes.

ROOT is a function of its files (`root_uuid` hashes them), and
`/system/etc/os-release` records only source facts, its time being the
commit's (`src/build.rs`'s `release`).

**Exit**: both derive from the commit's committer time, as
`TOYOS_COMMIT_TIME` does, or this file records why the signed version cannot.
3 changes: 1 addition & 2 deletions issues/libc-refuses-what-toyos-cannot-yet-answer.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ These answer failure in their POSIX form and do nothing
- `getrlimit` and `setrlimit`, `ENOSYS`: no call answers a process's limits.
LLVM's `getDefaultStackSize` (`llvm/lib/Support/ProgramStack.cpp`) ignores
the failure and reads the `rlimit` it did not get.
- `gethostname` and `uname`, `ENOSYS`: no host name, release or version is
published to a process.
- `gethostname`, `ENOSYS`: no host name is published to a process.
- `realpath`, `ENOSYS`: the kernel resolves a path by rules of its own (`..`
read off the text, only the last name's link followed, a relative link read
against its mount), and no call answers where a path leads.
Expand Down
7 changes: 7 additions & 0 deletions kernel/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,13 @@ pub(crate) unsafe extern "C" fn kernel_main(kernel_args: &KernelArgs) -> ! {
"boot: cmdline {:#x}+{}",
kernel_args.cmdline_addr, kernel_args.cmdline_len
);
// ROOT's name hashes every file it carries, `/system/etc/os-release` among
// them: the build, named before anything is mounted.
match toyos_abi::boot::root_uuid(cmdline).map(bcachefs::FsUuid::parse) {
Some(Some(root)) => log!("boot: root={root}"),
Some(None) => log!("boot: root= names no filesystem this kernel can parse"),
None => log!("boot: the cmdline carries no root="),
}
// Before `mm::init`, which may hand the parameter's memory out. This record
// is how a slot that died or was refused reaches the next boot's `/log`.
match params::slot(cmdline) {
Expand Down
106 changes: 105 additions & 1 deletion src/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -741,6 +741,54 @@ fn render_manifest(config: &SystemConfig) -> Vec<u8> {
.unwrap_or_else(|e| panic!("system.toml cannot be rendered as a manifest: {e:?}"))
}

/// Which build `root`'s tree makes for `arch` against the sysroot whose key
/// for it is `toolchain`, as `/system/etc/os-release` records it, read with
/// gitoxide (`issues/the-build-runs-host-tools-outside-rust-and-qemu.md`,
/// row 21). Untracked files are dirty whatever `status.showUntrackedFiles`
/// says; submodules are not read, because the fork's state is the toolchain
/// key's.
fn release(root: &Path, toolchain: &crate::keystore::Key, arch: Arch) -> toyos_osrelease::Release {
let repo = gix::open(root).unwrap_or_else(|e| panic!("{} is no git checkout: {e}", root.display()));
let head = repo.head_commit().unwrap_or_else(|e| panic!("{}'s HEAD names no commit: {e}", root.display()));
let commit = toyos_osrelease::Hex::parse(&head.id.to_string()).expect("a SHA-1 commit is forty hex digits");
let time = head.time().unwrap_or_else(|e| panic!("commit {} names no committer time: {e}", head.id));
let committed = u64::try_from(time.seconds)
.unwrap_or_else(|_| panic!("commit {} was committed before 1970: {}", head.id, time.seconds));
// Set whole: the platform `status` makes has no walk at all for a
// checkout configured to show no untracked files.
let walk = repo
.dirwalk_options()
.unwrap_or_else(|e| panic!("the status of {}: {e}", root.display()))
.emit_untracked(gix::dir::walk::EmissionMode::CollapseDirectory);
let changes = repo
.status(gix::progress::Discard)
.and_then(|status| {
status
.index_worktree_options_mut(|options| options.dirwalk_options = Some(walk))
.index_worktree_submodules(None)
.into_iter(None)
})
.unwrap_or_else(|e| panic!("the status of {}: {e}", root.display()));
// An index entry whose stat alone moved, and an ignored file the walk
// passed, summarise to nothing.
let dirty = changes.map(|item| item.unwrap_or_else(|e| panic!("the status of {}: {e}", root.display()))).any(
|item| match item {
gix::status::Item::IndexWorktree(change) => change.summary().is_some(),
gix::status::Item::TreeIndex(_) => true,
},
);
toyos_osrelease::Release {
commit,
tree: if dirty { toyos_osrelease::Tree::Dirty } else { toyos_osrelease::Tree::Clean },
toolchain: toyos_osrelease::Hex::parse(toolchain.as_str()).expect("a key is sixteen hex digits"),
arch: match arch {
Arch::X86_64 => toyos_osrelease::Arch::X86_64,
Arch::Aarch64 => toyos_osrelease::Arch::Aarch64,
},
committed,
}
}

fn build_and_assemble(
root: &Path,
config: &SystemConfig,
Expand All @@ -749,7 +797,11 @@ fn build_and_assemble(
quiet: bool,
arch: Arch,
) -> Vec<u8> {
let mut root_files: Vec<(String, Vec<u8>)> = Vec::new();
// Before anything is built, so a file edited during the build is not
// credited to the state it was read in.
let release = release(root, env.sysroot.identity.of_target(arch.userland()), arch);
let mut root_files: Vec<(String, Vec<u8>)> =
vec![(toyos_osrelease::PATH.to_string(), release.to_string().into_bytes())];
build_programs(root, config, env, quiet, arch, &mut root_files);
root_files.push((toyos_manifest::PATH.to_string(), render_manifest(config)));

Expand Down Expand Up @@ -2304,6 +2356,58 @@ fn collect_hosted_rustc(root: &Path, toolchain: &Path, root_files: &mut Vec<(Str
mod tests {
use super::*;

/// **The release a tree records is its commit, at that commit's own time,
/// and dirty from the first file that is not that commit's**: an
/// untracked file counts, and so does an edit to a tracked one, staged or
/// not, whatever the checkout's own status shows. The time is the
/// committer's, in a zone not UTC's, and never the author's or the host
/// clock's.
#[test]
fn a_tree_records_its_commit_its_commit_time_and_whether_it_is_dirty() {
let (_dir, _origin, work) = crate::gitfixture::repo("release");
fs::write(work.join("f"), "next\n").unwrap();
crate::gitfixture::sh(&work, &["add", "f"]);
let committed = Command::new("git")
.args(["commit", "-qm", "next"])
.env("GIT_AUTHOR_DATE", "@1000000000 +0000")
.env("GIT_COMMITTER_DATE", "@1791089159 +0200")
.current_dir(&work)
.status()
.unwrap();
assert!(committed.success());
// The ref as `git` wrote it, for an oracle that is not gitoxide.
let head = fs::read_to_string(work.join(".git/refs/heads/wt")).unwrap();
let key = crate::keystore::Key::parse("0123456789abcdef").unwrap();

let clean = release(&work, &key, Arch::Aarch64);
assert_eq!(clean.commit.as_str(), head.trim());
assert_eq!(clean.committed, 1_791_089_159);
assert_eq!(clean.tree, toyos_osrelease::Tree::Clean);
assert_eq!(clean.toolchain.as_str(), key.as_str());
assert_eq!(clean.arch, toyos_osrelease::Arch::Aarch64);

fs::write(work.join("untracked"), "x").unwrap();
assert_eq!(release(&work, &key, Arch::Aarch64).tree, toyos_osrelease::Tree::Dirty);
fs::remove_file(work.join("untracked")).unwrap();
assert_eq!(release(&work, &key, Arch::Aarch64).tree, toyos_osrelease::Tree::Clean);
fs::write(work.join("f"), "edited\n").unwrap();
assert_eq!(release(&work, &key, Arch::Aarch64).tree, toyos_osrelease::Tree::Dirty);
// An ignored file is the build's own output, not a change to the tree.
fs::write(work.join("f"), "next\n").unwrap();
fs::create_dir(work.join("target")).unwrap();
fs::write(work.join("target/out"), "x").unwrap();
assert_eq!(release(&work, &key, Arch::Aarch64).tree, toyos_osrelease::Tree::Clean);
// A checkout that hides untracked files from its own status.
crate::gitfixture::sh(&work, &["config", "status.showUntrackedFiles", "no"]);
fs::write(work.join("untracked"), "x").unwrap();
assert_eq!(release(&work, &key, Arch::Aarch64).tree, toyos_osrelease::Tree::Dirty);
fs::remove_file(work.join("untracked")).unwrap();
// A change staged, with the files as staged: only HEAD's tree differs.
fs::write(work.join("f"), "staged\n").unwrap();
crate::gitfixture::sh(&work, &["add", "f"]);
assert_eq!(release(&work, &key, Arch::Aarch64).tree, toyos_osrelease::Tree::Dirty);
}

/// `console` is reached by `console/system.toml` alone and the supervisor by no
/// `[programs]` row, so a reader that drops a mode or the supervisor loses one.
#[test]
Expand Down
20 changes: 20 additions & 0 deletions src/image.rs
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,26 @@ fn cmdline_of(path: &Path) -> Result<String, String> {
String::from_utf8(bytes).map_err(|e| format!("the boot parameter on {} is not text: {e}", path.display()))
}

/// The ROOT of the image's marked slot, read back off the image at `path`:
/// its name, and the file `name` it carries, read with the driver the kernel
/// mounts it with.
pub fn root_file_on(path: &Path, name: &str) -> Result<(FsUuid, Vec<u8>), String> {
let mut file = std::fs::File::open(path).map_err(|e| format!("opening {}: {e}", path.display()))?;
let table = slot_table_of(&mut file).map_err(|why| format!("{}: {why}", path.display()))?;
let slot = table.slot(table.marked).expect("a table marks a slot it carries");
let (start, len) = partition_extent(&mut file, slot.root).map_err(|why| format!("{}: {why}", path.display()))?;
let len = usize::try_from(len).map_err(|_| format!("a {len}-byte ROOT on {}", path.display()))?;
let mut volume = vec![0u8; len];
file.seek(SeekFrom::Start(start))
.and_then(|_| file.read_exact(&mut volume))
.map_err(|e| format!("reading {}'s ROOT at byte {start}: {e}", path.display()))?;
let uuid = root_uuid_of(&volume);
let fs = bcachefs::Mounted::<_, bcachefs::ReadOnly>::open(VecBlockIO::from_vec(volume))
.map_err(|e| format!("{}'s ROOT does not mount: {e:?}", path.display()))?;
let bytes = fs.read_file(name).map_err(|e| format!("{}'s ROOT: reading {name}: {e:?}", path.display()))?;
Ok((uuid, bytes))
}

/// The slot table on the disk image `file`, as the loader reads it.
pub fn slot_table_of(file: &mut std::fs::File) -> Result<toyos_update::slots::Table, String> {
table_on(file).map(|(table, _, _)| table)
Expand Down
15 changes: 14 additions & 1 deletion src/release.rs
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,16 @@ impl Layer {
}
}

const TAG: &str = "toolchain-linux-x86_64-";

/// The release of the sysroot `key` names.
fn tag(key: &Key) -> String {
format!("toolchain-linux-x86_64-{key}")
format!("{TAG}{key}")
}

/// The sysroot key a [`manifest`]'s `text` names, read back off its first line.
pub(crate) fn named_key(text: &str) -> Option<Key> {
text.lines().next()?.strip_prefix("toolchain ")?.strip_prefix(TAG).and_then(Key::parse)
}

pub(crate) fn sha256_hex(bytes: &[u8]) -> String {
Expand Down Expand Up @@ -863,6 +870,12 @@ mod tests {
assert_eq!(fs::read_to_string(crate::toolchain::witness_path(&rust_dir)).unwrap(), witness);
let toolchain = fs::read_to_string(crate::toolchain::manifest_path(&rust_dir)).unwrap();
assert_eq!(toolchain, manifest("toolchain-linux-x86_64-0123456789abcdef"));
// What an image built against it records as its toolchain: the key its
// release is tagged with, as a build that made its own sysroot records.
let installed = crate::sysroot::Sysroot::installed(crate::toolchain::stage2(&rust_dir), &toolchain);
for arch in [crate::arch::Arch::X86_64, crate::arch::Arch::Aarch64] {
assert_eq!(installed.identity.of_target(arch.userland()).as_str(), "0123456789abcdef");
}
fs::create_dir_all(rust_dir.join("build/sysroots/not-a-key")).unwrap();
assert!(lay_out(&root).unwrap_err().contains("named by no key"));
}
Expand Down
31 changes: 24 additions & 7 deletions src/sysroot.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,16 +60,24 @@ use crate::toolchain::{self, host_triple, GuestTarget, Owner, Role, GUEST_TARGET
use whole_toolchain::{whole, Whole};

/// The per-worktree sources that end up inside a sysroot: std links `toyos-abi`
/// and `toyos`, and `libtoyos_c.a` is `userland/libc` with `toyos-elf`.
pub const SYSROOT_SOURCES: [&str; 5] =
["toyos-abi/src", "toyos/src", "toyos-elf/src", "userland/libc/src", "userland/libc/include"];
/// and `toyos`, and `libtoyos_c.a` is `userland/libc` with `toyos-elf` and
/// `toyos-osrelease`.
pub const SYSROOT_SOURCES: [&str; 6] = [
"toyos-abi/src",
"toyos/src",
"toyos-elf/src",
"toyos-osrelease/src",
"userland/libc/src",
"userland/libc/include",
];

/// Their manifests, and the lockfile and cargo configuration libc is built
/// under: the features, versions and flags of the same build.
pub(crate) const SYSROOT_MANIFESTS: [&str; 6] = [
pub(crate) const SYSROOT_MANIFESTS: [&str; 7] = [
"toyos-abi/Cargo.toml",
"toyos/Cargo.toml",
"toyos-elf/Cargo.toml",
"toyos-osrelease/Cargo.toml",
"userland/libc/Cargo.toml",
"userland/libc/Cargo.lock",
"userland/.cargo/config.toml",
Expand Down Expand Up @@ -141,10 +149,12 @@ pub struct Sysroot {
impl Sysroot {
/// A checkout whose toolchain arrived as an artifact has one sysroot, the
/// artifact's, which `toolchain::check_installed_toolchain` has matched to
/// these sources; `release`, the `TOOLCHAIN` it was published with, is the
/// identity of all of it.
/// these sources; the sysroot key named by `release`, the `TOOLCHAIN` it
/// was published with, is the identity of all of it: the key a build of
/// these sources computes where it builds its own.
pub(crate) fn installed(stage2: PathBuf, release: &str) -> Self {
let id = Key::of(release.as_bytes());
let id = crate::release::named_key(release)
.unwrap_or_else(|| panic!("the installed TOOLCHAIN names no sysroot key:\n{release}"));
Self { dir: stage2, primary_compiler: true, identity: Identity::new(id.clone(), &id, &id), _using: None }
}

Expand Down Expand Up @@ -199,6 +209,13 @@ impl Identity {
Self::new(of(compiler), &of(freestanding), &of(key))
}

/// The key of `triple`'s libraries, which reads the compiler's key and the
/// freestanding libraries' too: everything an image of that target's
/// programs was compiled with.
pub fn of_target(&self, triple: &str) -> &Key {
self.libraries.get(triple).unwrap_or_else(|| panic!("{triple} is no guest target of this sysroot"))
}

/// What this leaves stale of a target directory whose `.deps-stamp` says
/// `stamp`, if it has one: nothing, if it is this.
pub fn stale(&self, stamp: Option<&str>) -> Option<Stale> {
Expand Down
26 changes: 26 additions & 0 deletions tests/common/qemu.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2251,6 +2251,32 @@ fn spawn_and_wait_ready(mut qemu: Command, options: &BootOptions, files: Files)
} else {
wait_for_ready(&mut child, &rx, options, &uart_log)
};
// The ready marker is test-runner's, which the supervisor starts after its
// first line, so a boot that reached it has said its build, and the kernel
// named the ROOT that carries it before that. Both are read off the image
// this boot was handed, never off the checkout, which may have moved since.
if options.ready_marker == DEFAULT_READY && !options.mute {
let (root, bytes) = toyos_build::image::root_file_on(&boot_image, toyos_osrelease::PATH)
.unwrap_or_else(|why| panic!("[qemu] {why}"));
let release = toyos_osrelease::parse(&bytes)
.unwrap_or_else(|why| panic!("[qemu] {} on {} is not the build's: {why:?}", toyos_osrelease::PATH, boot_image.display()));
let named = format!("boot: root={root}");
let said = format!(
"{}{} {}, committed {} UTC, toolchain {}, {}",
toyos_osrelease::SAID,
release.commit.as_str(),
release.tree,
toyos_wallclock::Civil::from_unix_secs(release.committed),
release.toolchain.as_str(),
release.arch.machine()
);
for line in [&named, &said] {
if !boot_log.lines().any(|l| l.contains(line.as_str())) {
let _ = child.kill();
panic!("[qemu] the boot never said `{line}`, of the image this run built\nconsole:\n{boot_log}");
}
}
}

QemuInstance {
child,
Expand Down
4 changes: 2 additions & 2 deletions tests/testcases/LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ against tinycc upstream at 64552b3faa39ee7948a9ea21bfcc11045b90c70d
(repo.or.cz/tinycc.git, 2026-08-05), allowing for the `-` → `_` renames this
project made to some filenames.

tinycc/ 326 files: 239 byte-identical to tinycc's `tests/tests2`,
tinycc/ 328 files: 239 byte-identical to tinycc's `tests/tests2`,
17 tinycc files this project modified,
70 not upstream at all — 69 cases written here, plus
72 not upstream at all — 71 cases written here, plus
`fred.txt`, which is output `40_stdio.c` writes when it
runs and which was committed by accident.

Expand Down
Loading
Loading