Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
8246c06
The toolchain is a content-addressed store; buildlock, keystore, work…
Japabu Sep 29, 2026
7f8ff2a
The store holds whole keys only; lockfiles key as staged; the toolcha…
Japabu Sep 29, 2026
60a694b
Merge origin/main into wt/toyos-castore
Japabu Sep 29, 2026
2ff7023
Merge origin/main into wt/toyos-castore
Japabu Sep 29, 2026
65b59d4
Answer the review of #629: the store's races, holds and keys; no fork…
Japabu Sep 29, 2026
2accc44
Merge origin/main (#624) into wt/toyos-castore
Japabu Sep 29, 2026
bda47ba
Issues follow the store; the config fixtures lose the key nothing reads
Japabu Sep 29, 2026
4424492
A compiler test passes its fork checkout, not a clone of it
Japabu Sep 29, 2026
ef162eb
A worktree's own fork checkout at its pin, clean, builds as the pin
Japabu Sep 29, 2026
9692f3d
A test holds the maker to letting go of its key before it collects
Japabu Sep 29, 2026
8ad36aa
The shared checkout's submodules are its own clones; no linked worktr…
Japabu Sep 30, 2026
886cee6
The layout the store replaces leaves a backtrace in 20 worktrees on t…
Japabu Sep 30, 2026
83f813d
Merge origin/main into wt/toyos-castore
Japabu Sep 30, 2026
fc4983f
Merge origin/main into wt/toyos-castore
Japabu Sep 30, 2026
a0a345d
Answer the second review of #629: locks through the name, dirty submo…
Japabu Sep 30, 2026
013e3e5
Merge origin/main (#632) into wt/toyos-castore
Japabu Sep 30, 2026
864594b
Merge origin/main (#536) into wt/toyos-castore
Japabu Sep 30, 2026
bf23d5a
Answer the third review of #629: submodules built at their gitlinks, …
Japabu Sep 30, 2026
e8dfe49
Answer the fourth review of #629: every tool OpenSSL's make runs has …
Japabu Sep 30, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,14 +62,15 @@ jobs:
fetch-depth: 0

# CMake and Ninja are what bootstrap builds LLVM and clang with, pinned to
# the versions Ubuntu 24.04 released (src/main.rs's `ALSO_USED`).
- name: disk, QEMU, CMake and Ninja
# the versions Ubuntu 24.04 released; Perl and make build the OpenSSL of
# the toolchain's cargo (src/main.rs's `ALSO_USED`).
- name: disk, QEMU, CMake, Ninja, Perl and make
run: |
sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc \
/usr/local/share/boost /usr/local/.ghcup
sudo apt-get update -qq
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y -qq qemu-system-x86 zstd \
cmake=3.28.3-1build7 ninja-build=1.11.1-2
cmake=3.28.3-1build7 ninja-build=1.11.1-2 perl make

- env:
GH_TOKEN: ${{ github.token }}
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ An operating system built from scratch in Rust, held to a production-grade engin
| `kernel/CLAUDE.md` | the caveats that bite kernel work |
| `userland/CLAUDE.md` | the server doctrine, and the caveats that bite userland work |
| `tests/CLAUDE.md` | the caveats that bite the harness |
| `src/CLAUDE.md` | boot modes, the locks, worktrees — the operational file |
| `src/CLAUDE.md` | boot modes, the toolchain store, worktrees — the operational file |
| `issues/README.md` | the issue tracker: one file per issue, typed by kind; `ls` is the index |
| `.claude/agents/reviewer.md` | the review prompt the orchestrator spawns a reviewer with |

Expand Down Expand Up @@ -82,7 +82,7 @@ The root `Cargo.toml`'s `[workspace]` `members` and `exclude` lists account for

## Workflow

**One agent, one worktree, one branch.** `cargo run -- --worktree add <path>` makes one; never `git worktree add` by hand — the naive path clones the rust fork's history and takes the machine-global toolchain name from every other checkout. The primary checkout is not a workspace: it owns `rust/`, the rustup link and `main`; `cargo run -- --sync` moves it onto whatever GitHub merged.
**One agent, one worktree, one branch.** `git worktree add --no-track -b wt/<name> <path> origin/main` makes one and `git worktree remove <path>` takes it; never `git submodule update` in one — that clones the rust fork's history again. The primary checkout is not a workspace: it owns `rust/`, the rustup link and `main`; `cargo run -- --sync` moves it onto whatever GitHub merged.

- Stay on the current task. File what you find in `issues/` and do not go fix it; one file per issue, its README has the shape.
- If something blocks, stop and report it. Don't work around it.
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ default-run = "toyos-build"
fatfs = "0.3.6"
fontdue = "0.9"
gpt = "3.1.0"
# `statvfs` for `worktree::free_bytes`, and the unprivileged ICMP datagram
# `flock` for `src/dirlock.rs`, and the unprivileged ICMP datagram
# socket `icmp::echo` asks a metal boot over, which is the platform call that
# replaces a `ping` binary.
libc = "0.2"
Expand Down
16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ One command, and a complete OS boots. No Make, no Docker, no LLVM to install,
no cross-toolchain to assemble. Everything that boots is
built by a toolchain in this repository.

Rust and QEMU, plus the four things `rustc`'s own bootstrap needs on every
Rust and QEMU, plus what `rustc`'s own bootstrap needs on every
platform — [Prerequisites](#prerequisites) says exactly what they are and why
nothing that boots goes near them.

Expand Down Expand Up @@ -224,20 +224,24 @@ same font the kernel blits.
- QEMU
- A C compiler on `PATH` as `cc`, and a Python 3
- CMake and Ninja
- Perl and `make`

`rustc` links every **host** binary through `cc`, which rustup does not
install. `rust/x`, the entry point to rustc's own bootstrap, is a shell script
whose whole job is to find a Python to run `bootstrap.py` with — so a clean
clone needs one, and so does every toolchain change. And that bootstrap builds
LLVM and clang from source with CMake and Ninja, whenever the LLVM commit
`rust/` names has not been built on the machine before.
`rust/` names has not been built on the machine before. The toolchain's cargo
is the fork's own, built with its compiler, and it carries its own OpenSSL,
which `openssl-src` configures with Perl and builds with `make`.

Nothing in the OS goes near any of them. `bootloader/`, `kernel/` and
`userland/` all link with the toolchain's `rust-lld`, and no image contains a C
toolchain or a Python. On macOS `cc` and Python arrive with the Xcode Command
Line Tools, and CMake and Ninja come from Homebrew (`brew install cmake
ninja`); on Debian and Ubuntu they are `build-essential`, `python3`, `cmake`
and `ninja-build`.
toolchain or a Python. On macOS `cc`, Python and `make` arrive with the Xcode
Command Line Tools and Perl with macOS itself, and CMake and Ninja come from
Homebrew (`brew install cmake ninja`); on Debian and Ubuntu they are
`build-essential`, which brings `make` and Perl, `python3`, `cmake` and
`ninja-build`.

`cargo run` names anything it needs and cannot find, before it does anything
else — including the Python that only the toolchain bootstrap runs, which
Expand Down
10 changes: 2 additions & 8 deletions examples/imgstat.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,15 +94,9 @@ fn main() {
}
}

/// Which of the four things on ROOT an entry is.
///
/// The order matters: `bin/rustc` is the toolchain's, not userland's.
/// Which of the things on ROOT an entry is.
fn group_of(name: &str) -> &'static str {
if name.starts_with("lib/") {
"hosted rustc lib/"
} else if name.starts_with("bin/rustc") {
"hosted rustc bin/"
} else if name.starts_with("bin/") {
if name.starts_with("bin/") {
"userland bin/"
} else if name.starts_with("share/") {
"assets share/"
Expand Down
4 changes: 1 addition & 3 deletions issues/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,9 +135,7 @@ with it.
slug as well as the path.** The slug is the identity, and a pointer written as a
bare name is invisible to a path search. Search the *tree* rather than the
checkout (`git grep <rev>`): `rg` skips dotfile directories without `--hidden`,
and `.github/` holds citations too. Then read where the hits are. One in a comment
under `toyos-abi/src`, `toyos/src` or a published crate changes no identity
(`src/identity.rs`), so it owes no version and builds no sysroot. One in
and `.github/` holds citations too. Then read where the hits are. One in
`src/redlist.rs` is a disabled test's `issue`: the row goes with the file.

## Two area notes, carried over from the file this replaced
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ kind: tooling
opened: 2026-09-29
---

# A killed keystore writer leaves an orphan temp in `target/`
# A killed store record writer leaves an orphan temp in `target/`

`record_by` in `src/keystore.rs` writes a uniquely named temp beside the record
`record` in `src/store.rs` writes a uniquely named temp beside the record
and renames it over. A writer killed between the write and the rename leaves
that temp behind, and because its name is unique nothing later overwrites it: one
orphan per kill.
Expand Down
16 changes: 16 additions & 0 deletions issues/build/a-lock-wait-in-the-build-has-no-ceiling.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
status: open
kind: tooling
opened: 2026-09-29
---

# A lock wait in the build has no ceiling

Every blocking lock in `src/dirlock.rs` says what it waits for every 30 s and
waits for as long as its holder lives. A killed holder releases it, but a live
one that hangs — a bootstrap stuck in a fetch, a maker blocked on a terminal —
holds every build waiting for that key, that checkout or that store forever,
each of them saying so every 30 s and none of them failing.

Exit: a wait whose holder has made no progress past a bound its kind of hold
declares fails loudly, naming the holder's pid and what it holds.
13 changes: 6 additions & 7 deletions issues/build/cargo-package-fails-in-a-linked-worktree.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,19 @@ opened: 2026-09-26
# `cargo package`/`cargo publish --dry-run` cannot run inside a linked worktree

`cargo package -p <any workspace member>` (tried `toyos-abi` and `toyos-ld`)
reds with a bare `error: No such file or directory (os error 2)` in a worktree
made by `cargo run -- --worktree add`, right after cargo's own trace logs
reds with a bare `error: No such file or directory (os error 2)` in a worktree,
right after cargo's own trace logs
`found a git repo` and `found (git) Cargo.toml`, inside
`cargo::ops::cargo_package::vcs::check_repo_state` — before it prints anything
about a dirty tree, and regardless of `--allow-dirty` or a fully clean working
tree (confirmed with `git stash`). The identical command against the identical
crate exits 0 in the **primary** checkout. Every worktree carries dozens of
submodule entries (`userland/*`, `rust`) registered in the shared `.git/config`
but not checked out on disk (`rust`'s own empty stub included, per
`src/worktree.rs`) — a repo shape only a linked worktree has, and the likely
reason cargo's git-repo-state walk (`git2`) chokes there and not in the
primary.
but not checked out on disk (`rust`'s own empty stub included) — a repo shape
only a linked worktree has, and the likely reason cargo's git-repo-state walk
(`git2`) chokes there and not in the primary.

Reproduce: from any `--worktree add` checkout, `cargo package -p toyos-abi
Reproduce: from any linked worktree, `cargo package -p toyos-abi
--no-verify --allow-dirty` exits 101 with that message; the same command in
the primary checkout exits 0.

Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@ kind: tooling
opened: 2026-09-29
---

# No test covers the pid in a keystore record's temp name
# No test covers the pid in a store record's temp name

`record_by` in `src/keystore.rs` names its temp file
`<record>.<pid>.<counter>.new` so that concurrent writers of one record never
`record` in `src/store.rs` names its temp file
`<record>.<pid>-<counter>.new` so that concurrent writers of one record never
share one. The counter distinguishes threads of one process; the pid is what
distinguishes two processes in one worktree, and nothing tests it: dropping
`std::process::id()` from the format leaves `cargo test --lib keystore` green
`std::process::id()` from the format leaves `cargo test --lib store::` green
(exit 0, measured by that mutation and that command)
while two processes writing the same record share a temp name again.

Exit condition: a test arm that races a child process against the parent on one
record, using the re-exec pattern of `rerun` in `src/buildlock.rs`, and goes red
record, using the re-exec pattern of `rerun` in `src/dirlock.rs`, and goes red
when the pid is dropped from the format.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
status: open
kind: tooling
opened: 2026-09-30
---

# Stub worktrees build their toolchains one at a time

A linked worktree whose `rust/` is the stub builds every toolchain its key
lacks in the host's one shared checkout, `<primary>/rust/build/fork/`, held
exclusively for the whole build (`sysroot::Fork::checkout`). A second stub
worktree needing a key of its own waits behind the first: behind a compiler
build, which took 22:05 there (`Build completed successfully in 0:22:05`), or a
sysroot's std build, which took 6:29 (`0:06:29`), both from one
`cargo run -- --build-only` of this store's branch on this host. The old
layout built std in each worktree's own `rust/`, side by side. What the queue
costs with several stub worktrees building at once has not been measured.

Exit: the wait of several stub worktrees whose keys differ is measured on the
host and the owner accepts that number, or their builds no longer share one
checkout.
3 changes: 0 additions & 3 deletions issues/build/the-ack-delay-abi-doc-names-one-cpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ sentence userland reads to learn what action 12 does describes a selection the
kernel does not make, omits the answer it returns, and says nothing about how
long what it leaves behind lasts.

`toyos-abi/src` is one of `toolchain::SYSROOT_SOURCES`, so the correction is a
single-commit branch of its own.

**Exit condition.** The doc names what the kernel does: each other CPU in turn,
the smallest of those waits returned, and the arming left standing against every
other CPU until a disarm or the end of the two-second window, whichever comes
Expand Down
Loading
Loading