File grants: one port per file-server role, a connection is the grant it was minted, and each instance has a quarter of each bound

issues/a-file-servers-shares-are-per-instance-and-one-session-launches-instances.md

@@ -0,0 +1,37 @@
+---
+status: assigned
+kind: defect
+opened: 2026-10-04
+---
+
+# A file server's shares are per instance, and one session launches instances
+
+Held by the orchestrator.
+
+`userland/fileserver/src/main.rs` gives each instance a grant names a quarter
+of each machine-wide bound: 32 of `MAX_SERVED`'s 128 served connections, 8 of
+`MAX_HANDSHAKES`' 32 waiting on their hello, 16 of `MAX_STREAMS`' 64 streams.
+An instance is one start and the children it spawns directly, and the
+supervisor mints a fresh one on every start. So four instances at their shares
+take each bound whole, and every other program's first connection or `STREAM`
+on that server is refused `ResourceExhausted`.
+
+Four instances are cheap to have: a program whose row lists `starts` (the
+compositor, terminal, shell, toybox and sshserver rows of `system.toml`) gets
+a new instance on every launch, and a shell launches shells. One session can
+therefore take DATA's server from every other session.
+
+The shares are sized against what was measured, not against a session's need.
+With the shares in place, every boot of the guest suite, the shipping
+desktop booted under QEMU, and every shipping Rust job run one after another in
+one `tests/testcases` boot never had an instance past 3 served connections, 1
+waiting on its hello or 0 streams, nor more than 5 of DATA's connections served
+at once; the desktop at rest holds 3. A parallel build (cargo with rustc as its
+direct children) is the obvious unmeasured load.
+
+## Exit condition
+
+A server's bounds are shared per session, so that one session holding all its
+programs may hold — through as many launches as its `starts` allow — leaves
+another session's first connection and first stream on that server answered,
+shown by a test that does exactly that.

issues/a-refused-handle-send-leaves-its-handles-with-callers-that-think-them-moved.md

@@ -0,0 +1,35 @@
+---
+status: open
+kind: defect
+opened: 2026-10-04
+---
+
+# A refused handle send leaves its handles with callers that think them moved
+
+When the kernel refuses `SYS_HANDLE_SEND`, it puts every handle back at its own
+number (`sys_handle_send` in `kernel/src/syscall/ipc.rs`). `Connection::send_with_handles`
+and its siblings in `toyos/src/ipc.rs` return that refusal and a refused frame
+as one error, so a caller cannot tell whether the handles are still its own.
+These callers keep what was refused and never close it:
+
+- `handshake` in `userland/diskserver/src/session.rs`: the region's dup, on a
+  service that has closed.
+- `open_stream` in `userland/soundserver/src/client.rs`: the client's region
+  and the signal pipe's read end. Its comment says both are moved whether or
+  not the send succeeds.
+- `NetstackConn::request_with_handles` in `toyos/src/net.rs`: its doc says a
+  refused send drops the batch. Every caller that passes handles it gave up
+  keeps them.
+
+`toyos::fs::hello`, the compositor's `copy_begin` and the supervisor's
+`serve_launch` each call `handle_send` themselves and close on its refusal.
+`issues/a-refused-handle-move-leaves-the-compositor-holding-it.md` is the
+compositor's `deliver_with_handles` case of the same thing.
+
+Owner: the SDK's connection, `toyos/src/ipc.rs`.
+
+**Exit**: no caller of a handle send can keep a handle the kernel refused to
+move. That holds when the send consumes handles it owns and closes them on the
+handle send's refusal, and when the three sites above use it. Its close also
+closes `issues/a-refused-handle-move-leaves-the-compositor-holding-it.md`: the
+fix that meets this exit deletes both files.

tests/proctreecase/system.toml

@@ -12,6 +12,9 @@
 # refused outright; `toybox`, which opens no session, starts `swap` in the
 # session it was launched in: refused in the machine's, started in the one the
 # shell opens.
+#
+# And `fs_share`: the shell test-runner's row lists is launched with grants of
+# its own, so it is another instance on DATA's server.
 
 [boot]
 start = ["logkeeper", "diskserver", "fileserver", "test-runner"]

tests/toyos-rust-tests/src/bin/fs_escape.rs

@@ -15,7 +15,7 @@ use std::fs;
 use std::io::ErrorKind;
 use std::os::toyos::fs::symlink;
 
-use toyos::fs::{window_put, Reply, Request, HELLO, OPEN, O_READ, REPLY, WINDOW_BYTES};
+use toyos::fs::{hello, window_put, Reply, Request, OPEN, O_READ, REPLY, WINDOW_BYTES};
 use toyos::shm::SharedMemory;
 use toyos::volatile::Window;
 use toyos_abi::syscall::SyscallError;
@@ -43,14 +43,12 @@ fn wire_open(rel: &[u8]) -> Reply {
     let names = toyos::endow::namespace().expect("this program was endowed a namespace");
     let conn = names.open("fs:/home").expect("this program holds fs:/home");
     let window = SharedMemory::create(WINDOW_BYTES).expect("a window");
-    let lent = window.share().expect("the window, shared");
-    conn.send_with_handles(&[lent], HELLO, &Request::new()).expect("hello");
+    hello(&conn, &window).expect("fs:/home answers its hello");
     let answer = |conn: &toyos::ipc::Connection| -> Reply {
         let header = conn.recv_header().expect("a reply");
         assert_eq!(header.msg_type, REPLY, "a reply frame");
         conn.recv_payload(&header).expect("a reply's words")
     };
-    assert_eq!(answer(&conn).status, 0, "fs:/home answers its hello");
     // SAFETY: the region is `WINDOW_BYTES` long and outlives this use.
     window_put(unsafe { Window::new(window.as_ptr(), WINDOW_BYTES) }, 0, rel);
     conn.send(OPEN, &Request { len: rel.len() as u64, flags: O_READ, ..Request::new() }).expect("open");

tests/toyos-rust-tests/src/bin/fs_share.rs

@@ -0,0 +1,159 @@
+//! One program holding all a file server lets it hold leaves the server
+//! answering another.
+//!
+//! This job holds test-runner's grants, so it and every child it spawns
+//! directly are one instance. On DATA's server:
+//!
+//! - each of DATA's directories is its grant's root: a file made under
+//!   `/home` is under none of the others;
+//! - it takes every stream the server will give it, until one is refused, and
+//!   every connection the server will serve it, until one is refused;
+//! - holding all that, it launches a shell, whose row test-runner's lists, so
+//!   the shell is an instance of its own: it opens a file under `/home` and
+//!   streams a child's output into it, and the bytes are read back here;
+//! - last, it opens more connections than any one instance may have waiting
+//!   on their hello: the ones past its share are answered `ResourceExhausted`
+//!   and let go as the server takes them, so the first to end is not the
+//!   first opened, which the server would otherwise let go first, at its
+//!   handshake timeout; a hello on it, which finds the server gone, reads
+//!   why, and every later one keeps none of the windows it could not lend.
+//!   Last, because the server reaps the ones this job drops only when it
+//!   next reads them, and until then they are this instance's share.
+
+use std::fs;
+use std::process::Command;
+
+use toyos::endow;
+use toyos::fs::{hello, Dir, O_CREATE, O_WRITE, WINDOW_BYTES};
+use toyos::ipc::Connection;
+use toyos::poller::{Poller, READABLE};
+use toyos::shm::SharedMemory;
+use toyos_abi::syscall::SyscallError;
+use toyos_abi::RawHandle;
+
+const DIR: &str = "/home/fs_share";
+const OTHER: &str = "/home/fs_share/other";
+const MARK: &str = "/home/fs_share/mark";
+const SAID: &str = "answered";
+
+/// More connections than one instance's share of a server's handshakes: the
+/// server's own machine-wide bound on them.
+const UNANSWERED: usize = 32;
+
+/// Far past any bound a server keeps, so a server that refuses nothing ends the
+/// loop rather than the machine.
+const CEILING: usize = 1024;
+
+/// How long the first unanswered connection may take to end: a hang ceiling,
+/// never a measure.
+const HANG_NS: u64 = 60_000_000_000;
+
+fn main() {
+    let names = endow::namespace().expect("this job was endowed a namespace");
+    // std's own connection to /home, made before anything is held, is what
+    // the shell's file is read back through.
+    fs::create_dir_all(DIR).expect("make the test's directory");
+    let _ = fs::remove_file(OTHER);
+    // Every arm runs, so one run says each one that is red.
+    let mut red = Vec::new();
+
+    fs::write(MARK, b"home's").expect("write a file under /home");
+    for elsewhere in ["/apps", "/config", "/state"] {
+        let path = format!("{elsewhere}/fs_share/mark");
+        match fs::metadata(&path) {
+            Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
+            other => red.push(format!("{MARK} is also {path}: {other:?}")),
+        }
+    }
+
+    // Every stream the server gives this instance.
+    let mut dir = Dir::connect(names, "fs:/home").expect("a client of /home");
+    let file = dir.open("fs_share/streams", O_WRITE | O_CREATE).expect("open a file to stream into");
+    let mut streams = Vec::new();
+    let refused = loop {
+        if streams.len() == CEILING {
+            break None;
+        }
+        match dir.stream(file.fid, file.generation, 0) {
+            Ok(pipe) => streams.push(pipe),
+            Err(e) => break Some(e),
+        }
+    };
+    match refused {
+        Some(SyscallError::ResourceExhausted) => println!("  this instance holds {} streams", streams.len()),
+        other => red.push(format!("after {} streams, the next was answered {other:?}", streams.len())),
+    }
+
+    // Every connection the server serves this instance, all lent one window.
+    let window = SharedMemory::create(WINDOW_BYTES).expect("a window");
+    let mut served = Vec::new();
+    let refused = loop {
+        if served.len() == CEILING {
+            break None;
+        }
+        let conn = names.open("fs:/home").expect("connect to /home");
+        match hello(&conn, &window) {
+            Ok(_) => served.push(conn),
+            Err(e) => break Some(e),
+        }
+    };
+    match refused {
+        Some(SyscallError::ResourceExhausted) => {
+            println!("  this instance is served {} more connections", served.len())
+        }
+        other => red.push(format!("after {} connections, the next hello was answered {other:?}", served.len())),
+    }
+
+    // Another instance, while this one holds all that.
+    let shell = Command::new("/system/bin/shell")
+        .args(["-c", &format!("/system/bin/toybox echo {SAID} > {OTHER}")])
+        .output()
+        .expect("launch a shell");
+    if !shell.status.success() {
+        red.push(format!("the other instance's shell failed: {shell:?}"));
+    }
+    match fs::read_to_string(OTHER) {
+        Ok(text) if text.trim_end() == SAID => println!("  another instance connected and streamed"),
+        other => red.push(format!("another instance's stream wrote {other:?}")),
+    }
+
+    // Connections that never say hello.
+    let opened: Vec<Connection> =
+        (0..UNANSWERED).map(|_| names.open("fs:/home").expect("connect to /home")).collect();
+    let poller = Poller::new(UNANSWERED as u32);
+    for (i, conn) in opened.iter().enumerate() {
+        poller.watch(conn, READABLE, i as u64);
+    }
+    let mut ended = Vec::new();
+    // Two, so the server had let the first of them go whole before the second
+    // was answered: it takes one connection at a time.
+    poller.wait(2, HANG_NS, |token| ended.push(token as usize));
+    ended.sort_unstable();
+    match ended.first() {
+        None => red.push(format!("none of {UNANSWERED} unanswered connections ended")),
+        Some(0) => red.push(format!("the first unanswered connection ended first, with {ended:?}")),
+        Some(_) => println!("  unanswered connections past the share ended first: {ended:?}"),
+    }
+    // So this hello finds the server gone, and reads what it was answered.
+    if let Some(&first) = ended.first() {
+        match hello(&opened[first], &window) {
+            Err(SyscallError::ResourceExhausted) => println!("  a hello on a connection let go is told why"),
+            other => red.push(format!("a hello on connection {first}, let go, was answered {:?}", other.map(|_| ()))),
+        }
+        // As many hellos as this process has handle slots: one that kept the
+        // window it could not lend would fill the table before the last.
+        let refused = (0..RawHandle::MAX_SLOTS)
+            .map(|_| hello(&opened[first], &window).map(|_| ()))
+            .enumerate()
+            .find(|(_, answer)| *answer != Err(SyscallError::Gone));
+        match refused {
+            None => println!("  {} hellos on a connection let go kept no window", RawHandle::MAX_SLOTS),
+            Some((i, answer)) => red.push(format!("hello {i} on connection {first}, let go, was answered {answer:?}")),
+        }
+    }
+
+    // A table the last arm filled has no room left to report it in.
+    drop((poller, opened, served, streams));
+    assert!(red.is_empty(), "fs_share:\n  {}", red.join("\n  "));
+    println!("fs_share: PASS");
+}

tests/toyos-rust-tests/src/bin/fs_stream_offset.rs

@@ -8,7 +8,7 @@
 //! program's `/apps`, `/config`, `/home` and `/state` are on.
 
 use toyos::fs::{
-    window_put, Reply, Request, HELLO, MAX_FILE_BYTES, OPEN, O_CREATE, O_WRITE, REPLY, STREAM, WINDOW_BYTES,
+    hello, window_put, Reply, Request, MAX_FILE_BYTES, OPEN, O_CREATE, O_WRITE, REPLY, STREAM, WINDOW_BYTES,
 };
 use toyos::ipc::Connection;
 use toyos::shm::SharedMemory;
@@ -36,9 +36,7 @@ fn main() {
     let names = toyos::endow::namespace().expect("this program was endowed a namespace");
     let conn = names.open("fs:/home").expect("this program holds fs:/home");
     let window = SharedMemory::create(WINDOW_BYTES).expect("a window");
-    conn.send_with_handles(&[window.share().expect("the window, shared")], HELLO, &Request::new())
-        .expect("hello");
-    assert_eq!(answer(&conn).status, 0, "fs:/home answers its hello");
+    hello(&conn, &window).expect("fs:/home answers its hello");
 
     let opened = open(&conn, &window);
     assert_eq!(opened.status, 0, "{} opened to write", core::str::from_utf8(NAME).unwrap_or(""));

tests/toyos.rs

@@ -127,6 +127,10 @@ const RUST_SKIP: &[&str] = &[
     // A kernel primitive with no use for any one boot's devices: the
     // `port_badge` metal row runs it on tests/proctreecase.
     "port_badge",
+    // Needs a launcher whose row lists a program that streams into a file, so
+    // that another instance asks DATA's server while this one holds its
+    // shares: the `fs_share` metal row runs it on tests/proctreecase.
+    "fs_share",
     // It asserts nothing at all: it holds a `tests/lanleasecase` boot open for
     // twenty seconds. On a shared boot it would be twenty seconds of nothing.
     "lan_hold",
@@ -686,6 +690,12 @@ const METAL: &[(&str, metal::Metal)] = &[
         "port_badge",
         metal::Metal { arms: PROCTREECASE, judge: |b| b[0].job_passed("test_rs_port_badge") },
     ),
+    (
+        // One instance holding all a file server lets it hold leaves the
+        // server answering another.
+        "fs_share",
+        metal::Metal { arms: PROCTREECASE, judge: |b| b[0].job_passed("test_rs_fs_share") },
+    ),
     // ---- one image: tests/metalcase ----
     (
         "metal_sim_scanout_wc",
@@ -910,13 +920,20 @@ const USB_RESET_BOOTS: &[metal::Arm] = &[
 const METALCASE: &[metal::Arm] = &[metal::once("metalcase", "tests/metalcase", &[], &[])];
 
 /// A launcher and a declared `cat` and shell, which `process_tree`'s subtree
-/// launches, a `toybox` row holding `roster`, which `launch_toctou` races, and
-/// the rows `launch_authority` is refused and started.
+/// launches, a `toybox` row holding `roster`, which `launch_toctou` races, the
+/// rows `launch_authority` is refused and started, and the shell `fs_share`
+/// asks DATA's server through.
 const PROCTREECASE: &[metal::Arm] = &[metal::once(
     "proctreecase",
     "tests/proctreecase",
     &[],
-    &["test_rs_process_tree", "test_rs_launch_toctou", "test_rs_launch_authority", "test_rs_port_badge"],
+    &[
+        "test_rs_process_tree",
+        "test_rs_launch_toctou",
+        "test_rs_launch_authority",
+        "test_rs_port_badge",
+        "test_rs_fs_share",
+    ],
 )];
 
 /// netstack in front of the T14's I219 with its lease probe armed: netstack's exit code

toyos-abi/src/syscall.rs

@@ -173,8 +173,7 @@ pub const SYS_NAMESPACE_OPEN: u64 = 102;
 ///
 /// The batch is queued on the connection, not interleaved with its bytes, so
 /// **handles are sent before the frame that announces them** and a receiver
-/// that has the frame already has the handles. The SDK's
-/// `Connection::send_with_handles` is that ordering written once.
+/// that has the frame already has the handles.
 pub const SYS_HANDLE_SEND: u64 = 103;
 /// Take the oldest batch of handles the peer sent. See [`handle_recv`].
 pub const SYS_HANDLE_RECV: u64 = 104;