diff --git a/issues/a-file-servers-shares-are-per-instance-and-one-session-launches-instances.md b/issues/a-file-servers-shares-are-per-instance-and-one-session-launches-instances.md new file mode 100644 index 00000000000..a2a68cbf184 --- /dev/null +++ b/issues/a-file-servers-shares-are-per-instance-and-one-session-launches-instances.md @@ -0,0 +1,37 @@ +--- +status: assigned +kind: defect +opened: 2026-10-04 +--- + +# A file server's shares are per instance, and one session launches instances + +Held by the orchestrator. + +`userland/fileserver/src/main.rs` gives each instance a grant names a quarter +of each machine-wide bound: 32 of `MAX_SERVED`'s 128 served connections, 8 of +`MAX_HANDSHAKES`' 32 waiting on their hello, 16 of `MAX_STREAMS`' 64 streams. +An instance is one start and the children it spawns directly, and the +supervisor mints a fresh one on every start. So four instances at their shares +take each bound whole, and every other program's first connection or `STREAM` +on that server is refused `ResourceExhausted`. + +Four instances are cheap to have: a program whose row lists `starts` (the +compositor, terminal, shell, toybox and sshserver rows of `system.toml`) gets +a new instance on every launch, and a shell launches shells. One session can +therefore take DATA's server from every other session. + +The shares are sized against what was measured, not against a session's need. +With the shares in place, every boot of the guest suite, the shipping +desktop booted under QEMU, and every shipping Rust job run one after another in +one `tests/testcases` boot never had an instance past 3 served connections, 1 +waiting on its hello or 0 streams, nor more than 5 of DATA's connections served +at once; the desktop at rest holds 3. A parallel build (cargo with rustc as its +direct children) is the obvious unmeasured load. + +## Exit condition + +A server's bounds are shared per session, so that one session holding all its +programs may hold — through as many launches as its `starts` allow — leaves +another session's first connection and first stream on that server answered, +shown by a test that does exactly that. diff --git a/issues/a-refused-handle-send-leaves-its-handles-with-callers-that-think-them-moved.md b/issues/a-refused-handle-send-leaves-its-handles-with-callers-that-think-them-moved.md new file mode 100644 index 00000000000..2f3505c7058 --- /dev/null +++ b/issues/a-refused-handle-send-leaves-its-handles-with-callers-that-think-them-moved.md @@ -0,0 +1,35 @@ +--- +status: open +kind: defect +opened: 2026-10-04 +--- + +# A refused handle send leaves its handles with callers that think them moved + +When the kernel refuses `SYS_HANDLE_SEND`, it puts every handle back at its own +number (`sys_handle_send` in `kernel/src/syscall/ipc.rs`). `Connection::send_with_handles` +and its siblings in `toyos/src/ipc.rs` return that refusal and a refused frame +as one error, so a caller cannot tell whether the handles are still its own. +These callers keep what was refused and never close it: + +- `handshake` in `userland/diskserver/src/session.rs`: the region's dup, on a + service that has closed. +- `open_stream` in `userland/soundserver/src/client.rs`: the client's region + and the signal pipe's read end. Its comment says both are moved whether or + not the send succeeds. +- `NetstackConn::request_with_handles` in `toyos/src/net.rs`: its doc says a + refused send drops the batch. Every caller that passes handles it gave up + keeps them. + +`toyos::fs::hello`, the compositor's `copy_begin` and the supervisor's +`serve_launch` each call `handle_send` themselves and close on its refusal. +`issues/a-refused-handle-move-leaves-the-compositor-holding-it.md` is the +compositor's `deliver_with_handles` case of the same thing. + +Owner: the SDK's connection, `toyos/src/ipc.rs`. + +**Exit**: no caller of a handle send can keep a handle the kernel refused to +move. That holds when the send consumes handles it owns and closes them on the +handle send's refusal, and when the three sites above use it. Its close also +closes `issues/a-refused-handle-move-leaves-the-compositor-holding-it.md`: the +fix that meets this exit deletes both files. diff --git a/issues/one-program-can-take-every-file-servers-client-slot.md b/issues/one-program-can-take-every-file-servers-client-slot.md deleted file mode 100644 index e46ffdd4c99..00000000000 --- a/issues/one-program-can-take-every-file-servers-client-slot.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -status: assigned -kind: defect -opened: 2026-09-27 ---- - -# One program can take every file server's client slot - -Held by the orchestrator. - -`userland/fsd/src/main.rs`'s `MAX_SERVED` bounds the clients one file server -serves at once, machine-wide: the 129th hello is refused `ResourceExhausted`. -Any program holding `fs:/home` can open 128 connections and hold them, and -from then on every other program's first connection to DATA's directories is -refused, by name instead of by a hang. `test_rs_fs_client_bound` does exactly -this for the length of its run. - -`MAX_STREAMS` is the same bound on streams: 64 a server, machine-wide, which -one program's clients can take whole, and every other program's `STREAM` — a -child's output into a served file — is then refused. - -## Exit condition - -A program's connections and streams each count against a share of their own, -so one program holding all it may hold leaves another's first connection and -first stream answered, shown by a guest test that holds one program at both -bounds while another connects and streams. diff --git a/tests/proctreecase/system.toml b/tests/proctreecase/system.toml index 57bbdf480be..c24edd7bfe8 100644 --- a/tests/proctreecase/system.toml +++ b/tests/proctreecase/system.toml @@ -12,6 +12,9 @@ # refused outright; `toybox`, which opens no session, starts `swap` in the # session it was launched in: refused in the machine's, started in the one the # shell opens. +# +# And `fs_share`: the shell test-runner's row lists is launched with grants of +# its own, so it is another instance on DATA's server. [boot] start = ["logkeeper", "diskserver", "fileserver", "test-runner"] diff --git a/tests/toyos-rust-tests/src/bin/fs_escape.rs b/tests/toyos-rust-tests/src/bin/fs_escape.rs index c8ee7e55598..be542244299 100644 --- a/tests/toyos-rust-tests/src/bin/fs_escape.rs +++ b/tests/toyos-rust-tests/src/bin/fs_escape.rs @@ -15,7 +15,7 @@ use std::fs; use std::io::ErrorKind; use std::os::toyos::fs::symlink; -use toyos::fs::{window_put, Reply, Request, HELLO, OPEN, O_READ, REPLY, WINDOW_BYTES}; +use toyos::fs::{hello, window_put, Reply, Request, OPEN, O_READ, REPLY, WINDOW_BYTES}; use toyos::shm::SharedMemory; use toyos::volatile::Window; use toyos_abi::syscall::SyscallError; @@ -43,14 +43,12 @@ fn wire_open(rel: &[u8]) -> Reply { let names = toyos::endow::namespace().expect("this program was endowed a namespace"); let conn = names.open("fs:/home").expect("this program holds fs:/home"); let window = SharedMemory::create(WINDOW_BYTES).expect("a window"); - let lent = window.share().expect("the window, shared"); - conn.send_with_handles(&[lent], HELLO, &Request::new()).expect("hello"); + hello(&conn, &window).expect("fs:/home answers its hello"); let answer = |conn: &toyos::ipc::Connection| -> Reply { let header = conn.recv_header().expect("a reply"); assert_eq!(header.msg_type, REPLY, "a reply frame"); conn.recv_payload(&header).expect("a reply's words") }; - assert_eq!(answer(&conn).status, 0, "fs:/home answers its hello"); // SAFETY: the region is `WINDOW_BYTES` long and outlives this use. window_put(unsafe { Window::new(window.as_ptr(), WINDOW_BYTES) }, 0, rel); conn.send(OPEN, &Request { len: rel.len() as u64, flags: O_READ, ..Request::new() }).expect("open"); diff --git a/tests/toyos-rust-tests/src/bin/fs_share.rs b/tests/toyos-rust-tests/src/bin/fs_share.rs new file mode 100644 index 00000000000..d47331c7577 --- /dev/null +++ b/tests/toyos-rust-tests/src/bin/fs_share.rs @@ -0,0 +1,159 @@ +//! One program holding all a file server lets it hold leaves the server +//! answering another. +//! +//! This job holds test-runner's grants, so it and every child it spawns +//! directly are one instance. On DATA's server: +//! +//! - each of DATA's directories is its grant's root: a file made under +//! `/home` is under none of the others; +//! - it takes every stream the server will give it, until one is refused, and +//! every connection the server will serve it, until one is refused; +//! - holding all that, it launches a shell, whose row test-runner's lists, so +//! the shell is an instance of its own: it opens a file under `/home` and +//! streams a child's output into it, and the bytes are read back here; +//! - last, it opens more connections than any one instance may have waiting +//! on their hello: the ones past its share are answered `ResourceExhausted` +//! and let go as the server takes them, so the first to end is not the +//! first opened, which the server would otherwise let go first, at its +//! handshake timeout; a hello on it, which finds the server gone, reads +//! why, and every later one keeps none of the windows it could not lend. +//! Last, because the server reaps the ones this job drops only when it +//! next reads them, and until then they are this instance's share. + +use std::fs; +use std::process::Command; + +use toyos::endow; +use toyos::fs::{hello, Dir, O_CREATE, O_WRITE, WINDOW_BYTES}; +use toyos::ipc::Connection; +use toyos::poller::{Poller, READABLE}; +use toyos::shm::SharedMemory; +use toyos_abi::syscall::SyscallError; +use toyos_abi::RawHandle; + +const DIR: &str = "/home/fs_share"; +const OTHER: &str = "/home/fs_share/other"; +const MARK: &str = "/home/fs_share/mark"; +const SAID: &str = "answered"; + +/// More connections than one instance's share of a server's handshakes: the +/// server's own machine-wide bound on them. +const UNANSWERED: usize = 32; + +/// Far past any bound a server keeps, so a server that refuses nothing ends the +/// loop rather than the machine. +const CEILING: usize = 1024; + +/// How long the first unanswered connection may take to end: a hang ceiling, +/// never a measure. +const HANG_NS: u64 = 60_000_000_000; + +fn main() { + let names = endow::namespace().expect("this job was endowed a namespace"); + // std's own connection to /home, made before anything is held, is what + // the shell's file is read back through. + fs::create_dir_all(DIR).expect("make the test's directory"); + let _ = fs::remove_file(OTHER); + // Every arm runs, so one run says each one that is red. + let mut red = Vec::new(); + + fs::write(MARK, b"home's").expect("write a file under /home"); + for elsewhere in ["/apps", "/config", "/state"] { + let path = format!("{elsewhere}/fs_share/mark"); + match fs::metadata(&path) { + Err(e) if e.kind() == std::io::ErrorKind::NotFound => {} + other => red.push(format!("{MARK} is also {path}: {other:?}")), + } + } + + // Every stream the server gives this instance. + let mut dir = Dir::connect(names, "fs:/home").expect("a client of /home"); + let file = dir.open("fs_share/streams", O_WRITE | O_CREATE).expect("open a file to stream into"); + let mut streams = Vec::new(); + let refused = loop { + if streams.len() == CEILING { + break None; + } + match dir.stream(file.fid, file.generation, 0) { + Ok(pipe) => streams.push(pipe), + Err(e) => break Some(e), + } + }; + match refused { + Some(SyscallError::ResourceExhausted) => println!(" this instance holds {} streams", streams.len()), + other => red.push(format!("after {} streams, the next was answered {other:?}", streams.len())), + } + + // Every connection the server serves this instance, all lent one window. + let window = SharedMemory::create(WINDOW_BYTES).expect("a window"); + let mut served = Vec::new(); + let refused = loop { + if served.len() == CEILING { + break None; + } + let conn = names.open("fs:/home").expect("connect to /home"); + match hello(&conn, &window) { + Ok(_) => served.push(conn), + Err(e) => break Some(e), + } + }; + match refused { + Some(SyscallError::ResourceExhausted) => { + println!(" this instance is served {} more connections", served.len()) + } + other => red.push(format!("after {} connections, the next hello was answered {other:?}", served.len())), + } + + // Another instance, while this one holds all that. + let shell = Command::new("/system/bin/shell") + .args(["-c", &format!("/system/bin/toybox echo {SAID} > {OTHER}")]) + .output() + .expect("launch a shell"); + if !shell.status.success() { + red.push(format!("the other instance's shell failed: {shell:?}")); + } + match fs::read_to_string(OTHER) { + Ok(text) if text.trim_end() == SAID => println!(" another instance connected and streamed"), + other => red.push(format!("another instance's stream wrote {other:?}")), + } + + // Connections that never say hello. + let opened: Vec = + (0..UNANSWERED).map(|_| names.open("fs:/home").expect("connect to /home")).collect(); + let poller = Poller::new(UNANSWERED as u32); + for (i, conn) in opened.iter().enumerate() { + poller.watch(conn, READABLE, i as u64); + } + let mut ended = Vec::new(); + // Two, so the server had let the first of them go whole before the second + // was answered: it takes one connection at a time. + poller.wait(2, HANG_NS, |token| ended.push(token as usize)); + ended.sort_unstable(); + match ended.first() { + None => red.push(format!("none of {UNANSWERED} unanswered connections ended")), + Some(0) => red.push(format!("the first unanswered connection ended first, with {ended:?}")), + Some(_) => println!(" unanswered connections past the share ended first: {ended:?}"), + } + // So this hello finds the server gone, and reads what it was answered. + if let Some(&first) = ended.first() { + match hello(&opened[first], &window) { + Err(SyscallError::ResourceExhausted) => println!(" a hello on a connection let go is told why"), + other => red.push(format!("a hello on connection {first}, let go, was answered {:?}", other.map(|_| ()))), + } + // As many hellos as this process has handle slots: one that kept the + // window it could not lend would fill the table before the last. + let refused = (0..RawHandle::MAX_SLOTS) + .map(|_| hello(&opened[first], &window).map(|_| ())) + .enumerate() + .find(|(_, answer)| *answer != Err(SyscallError::Gone)); + match refused { + None => println!(" {} hellos on a connection let go kept no window", RawHandle::MAX_SLOTS), + Some((i, answer)) => red.push(format!("hello {i} on connection {first}, let go, was answered {answer:?}")), + } + } + + // A table the last arm filled has no room left to report it in. + drop((poller, opened, served, streams)); + assert!(red.is_empty(), "fs_share:\n {}", red.join("\n ")); + println!("fs_share: PASS"); +} diff --git a/tests/toyos-rust-tests/src/bin/fs_stream_offset.rs b/tests/toyos-rust-tests/src/bin/fs_stream_offset.rs index c95322ea6c5..6f6bfb5737c 100644 --- a/tests/toyos-rust-tests/src/bin/fs_stream_offset.rs +++ b/tests/toyos-rust-tests/src/bin/fs_stream_offset.rs @@ -8,7 +8,7 @@ //! program's `/apps`, `/config`, `/home` and `/state` are on. use toyos::fs::{ - window_put, Reply, Request, HELLO, MAX_FILE_BYTES, OPEN, O_CREATE, O_WRITE, REPLY, STREAM, WINDOW_BYTES, + hello, window_put, Reply, Request, MAX_FILE_BYTES, OPEN, O_CREATE, O_WRITE, REPLY, STREAM, WINDOW_BYTES, }; use toyos::ipc::Connection; use toyos::shm::SharedMemory; @@ -36,9 +36,7 @@ fn main() { let names = toyos::endow::namespace().expect("this program was endowed a namespace"); let conn = names.open("fs:/home").expect("this program holds fs:/home"); let window = SharedMemory::create(WINDOW_BYTES).expect("a window"); - conn.send_with_handles(&[window.share().expect("the window, shared")], HELLO, &Request::new()) - .expect("hello"); - assert_eq!(answer(&conn).status, 0, "fs:/home answers its hello"); + hello(&conn, &window).expect("fs:/home answers its hello"); let opened = open(&conn, &window); assert_eq!(opened.status, 0, "{} opened to write", core::str::from_utf8(NAME).unwrap_or("")); diff --git a/tests/toyos.rs b/tests/toyos.rs index ba3aac3a3c2..1efc811d348 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -127,6 +127,10 @@ const RUST_SKIP: &[&str] = &[ // A kernel primitive with no use for any one boot's devices: the // `port_badge` metal row runs it on tests/proctreecase. "port_badge", + // Needs a launcher whose row lists a program that streams into a file, so + // that another instance asks DATA's server while this one holds its + // shares: the `fs_share` metal row runs it on tests/proctreecase. + "fs_share", // It asserts nothing at all: it holds a `tests/lanleasecase` boot open for // twenty seconds. On a shared boot it would be twenty seconds of nothing. "lan_hold", @@ -686,6 +690,12 @@ const METAL: &[(&str, metal::Metal)] = &[ "port_badge", metal::Metal { arms: PROCTREECASE, judge: |b| b[0].job_passed("test_rs_port_badge") }, ), + ( + // One instance holding all a file server lets it hold leaves the + // server answering another. + "fs_share", + metal::Metal { arms: PROCTREECASE, judge: |b| b[0].job_passed("test_rs_fs_share") }, + ), // ---- one image: tests/metalcase ---- ( "metal_sim_scanout_wc", @@ -910,13 +920,20 @@ const USB_RESET_BOOTS: &[metal::Arm] = &[ const METALCASE: &[metal::Arm] = &[metal::once("metalcase", "tests/metalcase", &[], &[])]; /// A launcher and a declared `cat` and shell, which `process_tree`'s subtree -/// launches, a `toybox` row holding `roster`, which `launch_toctou` races, and -/// the rows `launch_authority` is refused and started. +/// launches, a `toybox` row holding `roster`, which `launch_toctou` races, the +/// rows `launch_authority` is refused and started, and the shell `fs_share` +/// asks DATA's server through. const PROCTREECASE: &[metal::Arm] = &[metal::once( "proctreecase", "tests/proctreecase", &[], - &["test_rs_process_tree", "test_rs_launch_toctou", "test_rs_launch_authority", "test_rs_port_badge"], + &[ + "test_rs_process_tree", + "test_rs_launch_toctou", + "test_rs_launch_authority", + "test_rs_port_badge", + "test_rs_fs_share", + ], )]; /// netstack in front of the T14's I219 with its lease probe armed: netstack's exit code diff --git a/toyos-abi/src/syscall.rs b/toyos-abi/src/syscall.rs index c5c4d71930d..fee81aed0f3 100644 --- a/toyos-abi/src/syscall.rs +++ b/toyos-abi/src/syscall.rs @@ -173,8 +173,7 @@ pub const SYS_NAMESPACE_OPEN: u64 = 102; /// /// The batch is queued on the connection, not interleaved with its bytes, so /// **handles are sent before the frame that announces them** and a receiver -/// that has the frame already has the handles. The SDK's -/// `Connection::send_with_handles` is that ordering written once. +/// that has the frame already has the handles. pub const SYS_HANDLE_SEND: u64 = 103; /// Take the oldest batch of handles the peer sent. See [`handle_recv`]. pub const SYS_HANDLE_RECV: u64 = 104; diff --git a/toyos-manifest/src/lib.rs b/toyos-manifest/src/lib.rs index ccd3af63c12..c9845e57a76 100644 --- a/toyos-manifest/src/lib.rs +++ b/toyos-manifest/src/lib.rs @@ -68,16 +68,31 @@ pub fn session_home() -> String { /// program key. pub const STATE: &str = "/state"; -/// The file-server roles, and the directories each serves: one capability per -/// directory, named `fs:` and the directory. -pub const ROLES: [(&str, &[&str]); 3] = [ - ("data", &["/apps", "/config", "/home", "/state"]), - ("log", &["/log"]), - ("boot", &["/boot"]), +/// One directory a file-server role serves. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct RoleDir { + /// What every program's namespace calls it, after `fs:`: `/home`. + pub dir: &'static str, + /// Where it is on the role's volume, which a grant on it names + /// (`toyos::fs::Grant`): DATA keeps its directories side by side, and LOG + /// and BOOT are each served whole. + pub root: &'static str, +} + +const fn dir(dir: &'static str, root: &'static str) -> RoleDir { + RoleDir { dir, root } +} + +/// The file-server roles, and the directories each serves: one port per role, +/// and one grant on it per directory. +pub const ROLES: [(&str, &[RoleDir]); 3] = [ + ("data", &[dir("/apps", "apps"), dir("/config", "config"), dir("/home", "home"), dir("/state", "state")]), + ("log", &[dir("/log", "")]), + ("boot", &[dir("/boot", "")]), ]; /// The directories `role` serves, or `None` for a name that is no role. -pub fn role_dirs(role: &str) -> Option<&'static [&'static str]> { +pub fn role_dirs(role: &str) -> Option<&'static [RoleDir]> { ROLES.iter().find(|(name, _)| *name == role).map(|(_, dirs)| *dirs) } @@ -185,7 +200,7 @@ pub struct Program { pub service: bool, /// The file-server roles this row serves, one process each: the supervisor starts /// the binary once per role, with the role as its argument and the - /// acceptors of the role's directories ([`role_dirs`]). + /// acceptor of the role's port ([`role_dirs`] are the grants on it). pub roles: Vec, /// The supervisor starts it again when it ends, on the same ports, for as long as /// it does not end faster than [`RESTARTS`] allows. @@ -634,8 +649,12 @@ mod tests { let fileserver = m.program("fileserver").unwrap(); assert_eq!(fileserver.roles, ["data", "log"]); assert!(fileserver.restart); - assert_eq!(role_dirs("data"), Some(&["/apps", "/config", "/home", "/state"][..])); - assert_eq!(role_dirs("boot"), Some(&["/boot"][..])); + let dirs = |role| role_dirs(role).map(|dirs| dirs.iter().map(|d| (d.dir, d.root)).collect::>()); + assert_eq!( + dirs("data"), + Some(vec![("/apps", "apps"), ("/config", "config"), ("/home", "home"), ("/state", "state")]) + ); + assert_eq!(dirs("boot"), Some(vec![("/boot", "")])); assert_eq!(role_dirs("tmp"), None); let mut bad = sample(); bad.programs[3].roles = vec!["tmp".into()]; diff --git a/toyos/src/fs.rs b/toyos/src/fs.rs index b31694d1f3c..b86cf109f39 100644 --- a/toyos/src/fs.rs +++ b/toyos/src/fs.rs @@ -3,8 +3,11 @@ //! //! **A directory capability is a connector in the program's namespace**, named //! [`CAPABILITY_PREFIX`] and the absolute directory it serves (`fs:/home`). -//! The supervisor builds each program's set and tells the server which directory and -//! which rights each of its ports serves; a program names a file only under a +//! Each is a connector to its role's one port, which the supervisor minted with +//! a [`Grant`]: the directory, and whose share of the server it spends. The +//! kernel stamps that on every connection made through it and answers it to the +//! port's acceptor alone, so the server reads what was granted off the +//! connection and nothing the client says. A program names a file only under a //! directory it holds, and the kernel's part is who holds which connector. //! //! **The server resolves; the client only asks.** A path on the wire is @@ -28,14 +31,14 @@ //! here keeps a write the server acknowledged and never made durable: that is //! what `Fsync` is for. -use toyos_abi::syscall::{SyscallError, MAX_SERVICE_NAME}; +use toyos_abi::syscall::{self, SyscallError, MAX_BADGE, MAX_SERVICE_NAME}; use crate::ipc::{Connection, IpcError}; use crate::ipc_payload; use crate::namespace::Namespace; use crate::shm::SharedMemory; use crate::volatile::Window; -use crate::{AsHandle, Pipe, RawHandle}; +use crate::{AsHandle, OwnedHandle, Pipe, RawHandle}; /// What a directory capability's namespace name begins with. pub const CAPABILITY_PREFIX: &str = "fs:"; @@ -141,6 +144,55 @@ impl Reply { } } +/// What a connection to a file server's role port was granted: the badge the +/// supervisor mints the connector with. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Grant<'a> { + /// The start it was minted for. The supervisor counts its starts, so a + /// process it started and every child that process spawned directly, + /// holding the same connectors, are one instance, and spend one share. + pub instance: u64, + /// The directory, as a path on the role's volume, every path on the + /// connection is resolved beneath: `home`, or the empty path for a volume + /// served whole. [`canonical`], and at most [`MAX_GRANT_ROOT`] bytes. + pub root: &'a str, +} + +/// The format [`Grant::encode`] writes. Carried because a swap replaces a file +/// server and not the supervisor, so one server reads grants another build +/// minted, and an older one is refused by name rather than read as this one. +const GRANT_VERSION: u8 = 1; + +/// The longest root a grant carries: what one badge holds past the version and +/// the instance. +pub const MAX_GRANT_ROOT: usize = MAX_BADGE - 1 - 8; + +impl<'a> Grant<'a> { + /// The version, the instance, then the root: `None` for a root no grant + /// can carry. + pub fn encode<'b>(&self, out: &'b mut [u8; MAX_BADGE]) -> Option<&'b [u8]> { + if self.root.len() > MAX_GRANT_ROOT || !canonical(self.root) { + return None; + } + out[0] = GRANT_VERSION; + out[1..9].copy_from_slice(&self.instance.to_le_bytes()); + let end = 9 + self.root.len(); + out[9..end].copy_from_slice(self.root.as_bytes()); + Some(&out[..end]) + } + + /// `None` for bytes [`Self::encode`] cannot have written. + pub fn decode(bytes: &'a [u8]) -> Option { + let (&version, rest) = bytes.split_first()?; + if version != GRANT_VERSION || rest.len() < 8 || rest.len() - 8 > MAX_GRANT_ROOT { + return None; + } + let instance = u64::from_le_bytes(rest[..8].try_into().expect("eight bytes")); + let root = core::str::from_utf8(&rest[8..]).ok()?; + canonical(root).then_some(Self { instance, root }) + } +} + /// Whether `path` is a relative path the wire may carry: components of at /// least one byte, none of them `.` or `..`, no leading or trailing `/`, and /// the empty path for the capability's own directory. @@ -271,9 +323,7 @@ impl Dir { self.conn = None; let name = core::str::from_utf8(&self.name[..self.name_len]).map_err(|_| SyscallError::InvalidArgument)?; let conn = self.names.open(name)?; - let lent = self.window.share()?; - conn.send_with_handles(&[lent], HELLO, &Request::new()).map_err(transport)?; - let reply = receive(&conn)?.result()?; + let reply = hello(&conn, &self.window)?; self.writable = reply.value & RIGHT_WRITE != 0; self.conn = Some(conn); self.generation += 1; @@ -492,6 +542,29 @@ fn stat_of(reply: &Reply) -> Stat { Stat { kind: reply.kind, size: reply.value2, mtime: reply.mtime } } +/// Lend `window` to the server at the other end of `conn` ([`HELLO`]), and +/// answer what it granted, or why it refused. +/// +/// A server that refuses a connection as it takes it answers before it lets +/// go, so a hello that finds it gone still reads why. +pub fn hello(conn: &Connection, window: &SharedMemory) -> Result { + let lent = OwnedHandle(window.share()?); + // Not `send_with_handles`, which cannot say which send was refused: a + // refused handle send leaves `lent` in this table, and it closes as it drops. + let sent = match syscall::handle_send(conn.as_handle(), &[lent.raw()]) { + Ok(()) => { + lent.into_raw(); + conn.send(HELLO, &Request::new()).map_err(transport) + } + Err(e) => Err(e), + }; + match sent { + Ok(()) | Err(SyscallError::Gone) => {} + Err(e) => return Err(e), + } + receive(conn)?.result() +} + fn receive(conn: &Connection) -> Result { let header = conn.recv_header().map_err(transport)?; if header.msg_type != REPLY { @@ -555,6 +628,50 @@ mod tests { assert!(!canonical(&"a".repeat(MAX_PATH + 1))); } + #[test] + fn a_grant_round_trips_at_every_bound() { + let longest = "r".repeat(MAX_GRANT_ROOT); + for (instance, root) in [(0, ""), (1, "home"), (u64::MAX, "home/toy/Documents"), (7, longest.as_str())] { + let grant = Grant { instance, root }; + let mut out = [0u8; MAX_BADGE]; + let bytes = grant.encode(&mut out).expect("a root a grant carries"); + assert_eq!(Grant::decode(bytes), Some(grant), "{root:?}"); + } + } + + #[test] + fn no_grant_carries_a_root_the_wire_refuses_or_one_past_the_badge() { + let mut out = [0u8; MAX_BADGE]; + let past = "r".repeat(MAX_GRANT_ROOT + 1); + for root in ["/home", "home/", "a//b", ".", "a/../b", past.as_str()] { + assert_eq!(Grant { instance: 1, root }.encode(&mut out), None, "{root:?}"); + } + } + + #[test] + fn bytes_no_grant_was_encoded_as_are_refused() { + let mut out = [0u8; MAX_BADGE]; + let good = Grant { instance: 3, root: "home" }.encode(&mut out).unwrap().to_vec(); + // Shorter than a version and an instance. + for n in 0..9 { + assert_eq!(Grant::decode(&good[..n]), None, "{n} bytes"); + } + // Another version. + let mut other = good.clone(); + other[0] = GRANT_VERSION + 1; + assert_eq!(Grant::decode(&other), None); + // A root the wire refuses, or not UTF-8. + for root in [&b"/home"[..], b"a/../b", b"a//b", b"\xff"] { + let mut bad = good[..9].to_vec(); + bad.extend_from_slice(root); + assert_eq!(Grant::decode(&bad), None, "{root:?}"); + } + // One byte past the longest root. + let mut long = good[..9].to_vec(); + long.extend(core::iter::repeat_n(b'r', MAX_GRANT_ROOT + 1)); + assert_eq!(Grant::decode(&long), None); + } + #[test] fn a_listing_entry_round_trips_and_a_short_one_is_refused() { let mut buf = [0u8; 64]; diff --git a/toyos/src/ipc.rs b/toyos/src/ipc.rs index c321336be51..edb56a461ac 100644 --- a/toyos/src/ipc.rs +++ b/toyos/src/ipc.rs @@ -227,11 +227,11 @@ impl Connection { /// Move `handles` to the peer and then send the frame that announces them. /// - /// **In that order, and this is the only place it is written.** The handles - /// travel in a queue of their own rather than interleaved with the bytes, - /// so a peer that has read the frame is guaranteed to find them — and a - /// peer that has not is guaranteed not to act on them early. Sending the - /// frame first would make the receiver's `recv_handles` a poll. + /// **In that order.** The handles travel in a queue of their own rather than + /// interleaved with the bytes, so a peer that has read the frame is + /// guaranteed to find them — and a peer that has not is guaranteed not to + /// act on them early. Sending the frame first would make the receiver's + /// `recv_handles` a poll. pub fn send_with_handles( &self, handles: &[RawHandle], diff --git a/userland/Cargo.lock b/userland/Cargo.lock index a597548e4e1..c862bf588e8 100644 --- a/userland/Cargo.lock +++ b/userland/Cargo.lock @@ -1247,6 +1247,7 @@ dependencies = [ "toyos-blockring", "toyos-fat32", "toyos-gpt", + "toyos-manifest", ] [[package]] diff --git a/userland/fileserver/Cargo.toml b/userland/fileserver/Cargo.toml index 202a913ed49..79cf61d8688 100644 --- a/userland/fileserver/Cargo.toml +++ b/userland/fileserver/Cargo.toml @@ -17,6 +17,7 @@ path = "src/main.rs" [dependencies] toyos-abi = { path = "../../toyos-abi" } toyos = { path = "../../toyos" } +toyos-manifest = { path = "../../toyos-manifest" } toyos-blockring = { path = "../../toyos-blockring" } toyos-gpt = { path = "../../toyos-gpt" } toyos-fat32 = { path = "../../toyos-fat32" } diff --git a/userland/fileserver/src/lib.rs b/userland/fileserver/src/lib.rs index ee7767b426d..1dc8bb7b990 100644 --- a/userland/fileserver/src/lib.rs +++ b/userland/fileserver/src/lib.rs @@ -9,8 +9,8 @@ //! there; a write lands there and reaches the disk at a sync, or when the //! cache has more dirty blocks than it keeps. The kernel holds none of it. //! -//! The service around it is `src/main.rs`: its port per directory, its clients -//! and their windows, and the wire (`toyos::fs`). +//! The service around it is `src/main.rs`: its role's port and the grant on +//! each connection, its clients and their windows, and the wire (`toyos::fs`). pub mod absent; pub mod cache; diff --git a/userland/fileserver/src/main.rs b/userland/fileserver/src/main.rs index e1853366a74..83d1882e0c5 100644 --- a/userland/fileserver/src/main.rs +++ b/userland/fileserver/src/main.rs @@ -1,8 +1,8 @@ //! fileserver: one role's file server — DATA, LOG or BOOT — serving the directories //! of that role as capabilities. //! -//! **What it holds**: the acceptor of each directory its role serves, endowed -//! by the supervisor under `serve:fs:`; for its volume, a claim on each partition +//! **What it holds**: the acceptor of its role's one port, endowed by the +//! supervisor under `serve:fs:`; for its volume, a claim on each partition //! of its role a disk the kernel drives carries, and diskserver's `block` //! connector in its namespace; and nothing else of the machine. DATA is one //! partition counted over both, and two are refused by name, never guessed @@ -10,9 +10,17 @@ //! unique GUID of the partition the loader named for it when no claim on it //! was minted. //! -//! **A connection is bound to the directory whose port it came in on**, and -//! every path on it is resolved there (`fileserver::resolve`); a write on a read-only -//! volume is refused before the volume sees it. +//! **A connection is what its grant says** (`toyos::fs::Grant`): the badge +//! the supervisor minted its connector with, which the kernel stamped on it and +//! answers this port's acceptor alone. Every path on it is resolved beneath +//! the grant's root (`fileserver::resolve`); a write on a read-only volume is +//! refused before the volume sees it. +//! +//! **One instance cannot take the server.** Beneath each machine-wide bound — +//! connections waiting on their hello, connections served, streams — each +//! instance a grant names has a share of its own, so one holding all it may +//! leaves the rest of each bound to the others. Four at their shares take the +//! server (`issues/a-file-servers-shares-are-per-instance-and-one-session-launches-instances.md`). //! //! **A server never blocks on a client.** Accept and the first frame are two //! events; a request is buffered until whole; every reply is one `try_send`, @@ -43,7 +51,7 @@ use toyos::port::Acceptor; use toyos::shm::SharedMemory; use toyos::volatile::Window; use toyos::Pipe; -use toyos_abi::syscall::{SyscallError, DEV_PREFIX, SERVE_PREFIX}; +use toyos_abi::syscall::{SyscallError, DEV_PREFIX, MAX_BADGE, SERVE_PREFIX}; /// Clients served at once, machine-wide: one connection per directory per /// process. The next is answered `ResourceExhausted` at its hello and let go, @@ -51,7 +59,7 @@ use toyos_abi::syscall::{SyscallError, DEV_PREFIX, SERVE_PREFIX}; const MAX_SERVED: usize = 128; /// Connections taken and not yet answered at their hello. While this many -/// wait, the next waits in its port's queue — for at most +/// wait, the next waits in the port's queue — for at most /// [`HANDSHAKE_TIMEOUT`], by which each of these is answered or let go. const MAX_HANDSHAKES: usize = 32; @@ -61,14 +69,20 @@ const MAX_FIDS: usize = 1024; /// Streams served at once, machine-wide. const MAX_STREAMS: usize = 64; +/// One instance's shares of [`MAX_SERVED`], [`MAX_HANDSHAKES`] and +/// [`MAX_STREAMS`]. Past its share of served clients a hello is answered +/// `ResourceExhausted`, and past its share of streams a `STREAM` is; past its +/// share of handshakes a connection is, as it is taken, and let go. A quarter +/// of each bound, so an instance at its share leaves the bound to three more. +const SERVED_SHARE: usize = MAX_SERVED / 4; +const HANDSHAKE_SHARE: usize = MAX_HANDSHAKES / 4; +const STREAM_SHARE: usize = MAX_STREAMS / 4; + /// What one turn of a stream appends at most. const STREAM_READ: usize = 64 * 1024; -/// Directories one server serves: DATA's four, with room. -const MAX_DIRS: usize = 32; - -// One wait watches every acceptor, connection and stream at once. -const _: () = assert!(MAX_SERVED + MAX_HANDSHAKES + MAX_STREAMS + MAX_DIRS <= Poller::MAX_HANDLES as usize); +// One wait watches the acceptor, every connection and every stream at once. +const _: () = assert!(1 + MAX_SERVED + MAX_HANDSHAKES + MAX_STREAMS <= Poller::MAX_HANDLES as usize); /// How long an accepted connection may take to lend its window. const HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(2); @@ -77,6 +91,7 @@ const HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(2); /// blocks, of which only what is written costs anything. const RAM_BLOCKS: u64 = 1 << 18; +const TOKEN_ACCEPTOR: u64 = 0; const TOKEN_CLIENT: u64 = 1 << 32; const TOKEN_STREAM: u64 = 2 << 32; @@ -96,15 +111,14 @@ impl Role { _ => None, } } -} -/// One directory this server serves. -struct Capability { - /// Its absolute name, `/home`. - dir: String, - /// Where it is on the volume. - root: String, - acceptor: Acceptor, + fn name(self) -> &'static str { + match self { + Self::Data => "data", + Self::Log => "log", + Self::Boot => "boot", + } + } } /// One file a client holds open. @@ -117,7 +131,10 @@ struct Fid { struct Client { conn: Connection, rx: ipc::FrameRx<{ core::mem::size_of::() }>, - cap: usize, + /// Its grant's root, beneath which every path it names is resolved. + root: String, + /// Its grant's instance, whose shares it spends. + instance: u64, window: Option, fids: BTreeMap, next_fid: u64, @@ -129,6 +146,8 @@ struct Stream { pipe: Pipe, node: Node, offset: u64, + /// The instance of the client that asked for it, whose share it spends. + instance: u64, } fn main() { @@ -145,18 +164,21 @@ fn main() { } } fileserver::volume::take_anchor(); - let caps = capabilities(role); - let roots: Vec = caps.iter().map(|c| c.root.clone()).collect(); - let roots: Vec<&str> = roots.iter().map(String::as_str).collect(); + let dirs = toyos_manifest::role_dirs(role.name()).expect("every role this server parses is a manifest role"); + let label = format!("{SERVE_PREFIX}{CAPABILITY_PREFIX}{}", role.name()); + let acceptor: Acceptor = Endowments::get() + .take(&label) + .unwrap_or_else(|| panic!("fileserver: started without its role's acceptor, `{label}`")); + let roots: Vec<&str> = dirs.iter().map(|d| d.root).collect(); let volume = open_volume(role, guid, &roots); println!( "fileserver: {role:?} serving {} — {}", - caps.iter().map(|c| c.dir.as_str()).collect::>().join(", "), + dirs.iter().map(|d| d.dir).collect::>().join(", "), volume.describe() ); Server { volume, - caps, + acceptor, clients: BTreeMap::new(), next_client: 0, streams: BTreeMap::new(), @@ -167,26 +189,6 @@ fn main() { .serve() } -/// Every directory the supervisor endowed this process an acceptor for. -fn capabilities(role: Role) -> Vec { - let prefix = format!("{SERVE_PREFIX}{CAPABILITY_PREFIX}"); - let labels: Vec = - Endowments::get().labels().filter(|l| l.starts_with(&prefix)).map(str::to_string).collect(); - let mut caps = Vec::new(); - for label in labels { - let dir = label[prefix.len()..].to_string(); - let acceptor: Acceptor = Endowments::get().take(&label).expect("fileserver: an acceptor its label names"); - let root = match role { - Role::Data => dir.trim_start_matches('/').to_string(), - Role::Log | Role::Boot => String::new(), - }; - caps.push(Capability { dir, root, acceptor }); - } - assert!(!caps.is_empty(), "fileserver: started serving no directory"); - assert!(caps.len() <= MAX_DIRS, "fileserver: started serving {} directories, past {MAX_DIRS}", caps.len()); - caps -} - /// The partition claims the supervisor minted: one per partition of the role on a disk /// the kernel drives. fn claims() -> Vec { @@ -309,7 +311,8 @@ fn fat_on(disk: D, writable: bool) -> Result, struct Server { volume: Box, - caps: Vec, + /// The role's port, which every grant is minted on. + acceptor: Acceptor, clients: BTreeMap, next_client: u64, streams: BTreeMap, @@ -369,9 +372,7 @@ impl Server { let mut ready = Vec::new(); loop { if self.clients.values().filter(|c| c.window.is_none()).count() < MAX_HANDSHAKES { - for (i, cap) in self.caps.iter().enumerate() { - poller.watch(&cap.acceptor, READABLE, i as u64); - } + poller.watch(&self.acceptor, READABLE, TOKEN_ACCEPTOR); } for (id, c) in &self.clients { poller.watch(&c.conn, READABLE, TOKEN_CLIENT + id); @@ -392,8 +393,8 @@ impl Server { poller.wait(1, timeout, |t| ready.push(t)); for &token in &ready { - if token < TOKEN_CLIENT { - self.accept(token as usize); + if token == TOKEN_ACCEPTOR { + self.accept(); } else if token < TOKEN_STREAM { self.pump(token - TOKEN_CLIENT); } else { @@ -418,17 +419,39 @@ impl Server { } } - fn accept(&mut self, cap: usize) { - let conn = match self.caps[cap].acceptor.accept() { + /// Take the next connection and read its grant. Only the supervisor mints + /// on this port, so a connection without one this server reads is let go + /// by name; one whose instance already has its share of handshakes is + /// answered `ResourceExhausted` and let go. + fn accept(&mut self) { + let conn = match self.acceptor.accept() { Ok(conn) => conn, Err(why) => panic!("fileserver: its own acceptor refused an accept: {why:?}"), }; + let mut badge = [0u8; MAX_BADGE]; + let grant = match self.acceptor.badge(&conn, &mut badge) { + Ok(bytes) => match Grant::decode(bytes) { + Some(grant) => grant, + None => return println!("fileserver: letting a connection go: its badge is no grant: {bytes:?}"), + }, + Err(why) => return println!("fileserver: letting a connection go: it carries no grant ({why:?})"), + }; + let mine = self.clients.values().filter(|c| c.instance == grant.instance && c.window.is_none()).count(); + if mine >= HANDSHAKE_SHARE { + // Let go whether or not the refusal went: either way it is said. + let _ = conn.try_send(REPLY, &Reply::refused(SyscallError::ResourceExhausted)); + return println!( + "fileserver: letting a connection go: instance {} has {HANDSHAKE_SHARE} waiting on their hello", + grant.instance + ); + } let id = self.next_client; self.next_client += 1; let client = Client { conn, rx: ipc::FrameRx::new(), - cap, + root: grant.root.to_string(), + instance: grant.instance, window: None, fids: BTreeMap::new(), next_fid: 1, @@ -440,7 +463,7 @@ impl Server { fn drop_client(&mut self, id: u64, why: &str) { let Some(client) = self.clients.remove(&id) else { return }; if !why.is_empty() { - println!("fileserver: dropping client {id} of {}: {why}", self.caps[client.cap].dir); + println!("fileserver: dropping client {id} of {:?}, instance {}: {why}", client.root, client.instance); } for fid in client.fids.values() { // Nobody is left to answer: a refused close is said, and its node @@ -507,7 +530,7 @@ impl Server { if !canonical(rel) { return Err(SyscallError::InvalidArgument); } - let root = self.caps[self.clients[&id].cap].root.clone(); + let root = self.clients[&id].root.clone(); let volume = &mut self.volume; let mut lookup = |p: &str| match volume.lstat(p) { Ok(meta) if meta.kind == Kind::Symlink => volume.read_link(p).map(Found::Link).map_err(drop), @@ -530,15 +553,21 @@ impl Server { fn answer(&mut self, id: u64, op: u32, r: Request) -> Answer { if op == HELLO { - let served = self.clients.values().filter(|c| c.window.is_some()).count(); + let instance = self.clients[&id].instance; + let served = self.clients.values().filter(|c| c.window.is_some()); + let (all, mine) = served.fold((0, 0), |(all, mine), c| (all + 1, mine + usize::from(c.instance == instance))); let client = self.clients.get_mut(&id).expect("pumped"); if client.window.is_some() { return Answer::Drop("it lent a second window"); } - if served >= MAX_SERVED { + if all >= MAX_SERVED { let why = format!("it is refused, since {MAX_SERVED} clients are served already"); return Answer::Refuse(SyscallError::ResourceExhausted, why); } + if mine >= SERVED_SHARE { + let why = format!("it is refused, since its instance is served {SERVED_SHARE} clients already"); + return Answer::Refuse(SyscallError::ResourceExhausted, why); + } let Some([lent]) = client.conn.recv_handles_exact::<1>() else { return Answer::Drop("its hello carried no window"); }; @@ -692,14 +721,16 @@ impl Server { return Err(SyscallError::InvalidArgument); } let node = f.node; - if self.streams.len() >= MAX_STREAMS { + let instance = self.clients[&id].instance; + let mine = self.streams.values().filter(|s| s.instance == instance).count(); + if self.streams.len() >= MAX_STREAMS || mine >= STREAM_SHARE { return Err(SyscallError::ResourceExhausted); } let (read, write) = toyos::pipe_pair()?; self.volume.hold(node); let sid = self.next_stream; self.next_stream += 1; - self.streams.insert(sid, Stream { pipe: read, node, offset: r.offset }); + self.streams.insert(sid, Stream { pipe: read, node, offset: r.offset, instance }); Ok(Answer::WithHandle(Reply::ok(), write.into_raw())) } STAT | LSTAT => { diff --git a/userland/fileserver/src/resolve.rs b/userland/fileserver/src/resolve.rs index 089a8cfd642..3f92b98178a 100644 --- a/userland/fileserver/src/resolve.rs +++ b/userland/fileserver/src/resolve.rs @@ -2,7 +2,7 @@ //! given, and nowhere else. //! //! **The root is a floor, not a starting point.** A connection is bound to one -//! directory of the volume (its capability's); a client's path is canonical +//! directory of the volume (its grant's root); a client's path is canonical //! and relative to it (`toyos::fs::canonical`), and each component is looked up //! in turn. A symlink met on the way is expanded where it stands: a relative //! target is read against the link's own directory, and a `..` that would diff --git a/userland/supervisor/src/main.rs b/userland/supervisor/src/main.rs index 2463f06c633..ed35fbd0d11 100644 --- a/userland/supervisor/src/main.rs +++ b/userland/supervisor/src/main.rs @@ -72,7 +72,7 @@ use toyos_manifest::launch::{self as authority, Authority, Session, Target}; use toyos_manifest::package::{self, Package}; use toyos_manifest::{Manifest, Program}; use toyos::endow::Endowments; -use toyos::fs::CAPABILITY_PREFIX; +use toyos::fs::{Grant, CAPABILITY_PREFIX}; use toyos::ipc::{self, Connection, RxStep}; use toyos::launch::{self, Parent, Request, LAUNCHER}; use toyos::namespace::{self, Namespace}; @@ -377,26 +377,27 @@ fn main() { connectors.insert(name, connector); } - // One port per directory a file server serves, made here for the same - // reason: a program's first open works whether or not its server runs yet. - let mut dirs: BTreeMap = BTreeMap::new(); - let mut dir_acceptors: BTreeMap = BTreeMap::new(); + // One port per file-server role, made here for the same reason: a + // program's first open works whether or not its server runs yet. Its + // connector goes at once: every connector a program holds to it is a + // grant minted for that program's start (`Grants`). + let mut role_acceptors: BTreeMap<&str, Acceptor> = BTreeMap::new(); for role in system.programs.iter().flat_map(|p| p.roles.iter()) { - for dir in toyos_manifest::role_dirs(role).expect("supervisor: the build refuses a role it does not know") { - let name = format!("{CAPABILITY_PREFIX}{dir}"); - let (acceptor, connector) = - port::create().unwrap_or_else(|e| panic!("supervisor: no port for `{name}`: {e:?}")); - dir_acceptors.insert(name.clone(), acceptor); - dirs.insert(name, connector); - } + let (acceptor, _) = port::create().unwrap_or_else(|e| panic!("supervisor: no port for the `{role}` role: {e:?}")); + role_acceptors.insert(role, acceptor); } - // The supervisor's own files are resolved through the same directories as every - // program's, and it is the one process nobody endows a namespace: std - // resolves through this one, and the stop's syncs through the second. + // The supervisor's own files are resolved through grants like every + // program's, under the instance no start is given, and it is the one + // process nobody endows a namespace: std resolves through this one, and + // the stop's syncs through the second. let files: &'static Namespace = { + let own: Vec<(String, Connector)> = role_acceptors + .iter() + .flat_map(|(role, acceptor)| mint_grants(role, acceptor, SUPERVISOR_INSTANCE)) + .collect(); let build = || { let mut builder = namespace::build(); - for (name, connector) in &dirs { + for (name, connector) in &own { builder = builder.add(name, connector); } builder.finish().expect("supervisor: no namespace for its own files") @@ -418,7 +419,7 @@ fn main() { syscap: &syscap, acceptors, connectors, - dirs, + grants: Grants { roles: Vec::new(), next: Cell::new(SUPERVISOR_INSTANCE + 1) }, files, services: Vec::new(), log, @@ -426,7 +427,7 @@ fn main() { worker: Worker::start(), restarting: false, }; - supervisor.boot(&mut dir_acceptors, wake_write.into_raw()); + supervisor.boot(&mut role_acceptors, wake_write.into_raw()); // Nothing else holds a `serves` acceptor that has not been launched yet, so // the supervisor outliving its children is what keeps those ports open. It parks @@ -521,9 +522,8 @@ struct Supervisor<'a> { /// takes by move. acceptors: BTreeMap<&'a str, Acceptor>, connectors: BTreeMap<&'a str, Connector>, - /// Every directory capability, by its namespace name: what each program's - /// view is built from. - dirs: BTreeMap, + /// What each program's view is minted from. + grants: Grants<'a>, /// The same directories as a namespace of the supervisor's own, for the stop's syncs. files: &'static Namespace, /// What `[boot] start` named, a file server once per role. @@ -613,7 +613,7 @@ impl<'a> Service<'a> { system: &Manifest, syscap: &SysCap, connectors: &BTreeMap<&str, Connector>, - dirs: &BTreeMap, + grants: &Grants<'_>, launcher: &Acceptor, log: &mut Log, ) -> Result { @@ -643,7 +643,7 @@ impl<'a> Service<'a> { syscap, served, connectors, - dirs, + grants, &[], storage, (launcher, Session::Machine), @@ -757,7 +757,7 @@ impl<'a> Supervisor<'a> { /// process it started. The storage rows go first — every other start may /// make a directory, and a directory is a file server's — and the /// session's home is made before the first row that is not one. - fn boot(&mut self, dir_acceptors: &mut BTreeMap, wake: toyos::RawHandle) { + fn boot(&mut self, role_acceptors: &mut BTreeMap<&str, Acceptor>, wake: toyos::RawHandle) { let system = self.system; let storage_first = system .start @@ -789,22 +789,20 @@ impl<'a> Supervisor<'a> { (served.clone(), acceptor) }) .collect(), - Some(role) => toyos_manifest::role_dirs(role) - .expect("known role") - .iter() - .map(|dir| { - let name = format!("{CAPABILITY_PREFIX}{dir}"); - let acceptor = dir_acceptors - .remove(&name) - .unwrap_or_else(|| panic!("supervisor: the `{role}` role is served twice")); - (name, acceptor) - }) - .collect(), + Some(role) => { + let acceptor = role_acceptors + .remove(role) + .unwrap_or_else(|| panic!("supervisor: the `{role}` role is served twice")); + vec![(format!("{CAPABILITY_PREFIX}{role}"), acceptor)] + } }; self.make_home(program); let mut service = Service::new(program, role, kept, wake); + if let Some(role) = role { + self.grants.roles.push((role, Arc::clone(&service.kept))); + } let started = - service.spawn(&program.path, &[], system, self.syscap, &self.connectors, &self.dirs, &self.launcher, &mut self.log); + service.spawn(&program.path, &[], system, self.syscap, &self.connectors, &self.grants, &self.launcher, &mut self.log); match started { Ok(_) => {} Err(StartError::Partition(why)) => { @@ -1183,7 +1181,7 @@ impl<'a> Supervisor<'a> { let _ = old.wait(); } let started = - service.spawn(&path, &owed, self.system, self.syscap, &self.connectors, &self.dirs, &self.launcher, &mut self.log); + service.spawn(&path, &owed, self.system, self.syscap, &self.connectors, &self.grants, &self.launcher, &mut self.log); match started { Ok(pid) => { swapped( @@ -1283,7 +1281,7 @@ impl<'a> Supervisor<'a> { let mut syncing = Vec::new(); for role in roles.filter(|r| *r != "boot") { let Some(dir) = toyos_manifest::role_dirs(role).and_then(|dirs| dirs.first()) else { continue }; - let name = format!("{CAPABILITY_PREFIX}{dir}"); + let name = format!("{CAPABILITY_PREFIX}{}", dir.dir); let (done_tx, done_rx) = std::sync::mpsc::channel(); let (asked, names) = (name.clone(), self.files); let spawned = std::thread::Builder::new().name(format!("sync-{role}")).spawn(move || { @@ -1347,7 +1345,7 @@ impl<'a> Supervisor<'a> { kept.ended = false; kept.acceptors.clear(); say!( - "supervisor: {label} ended {} times in {} s; its ports are closed and its clients answered Gone", + "supervisor: {label} ended {} times in {} s; its ports are closed, its clients answered Gone, and a program started from now on is given none of its directories", toyos_manifest::RESTARTS + 1, toyos_manifest::RESTART_WINDOW_SECS ); @@ -1357,7 +1355,7 @@ impl<'a> Supervisor<'a> { let (path, owed, program) = (service.path.clone(), service.devices.clone(), service.program); self.make_home(program); let service = &mut self.services[index]; - match service.spawn(&path, &owed, self.system, self.syscap, &self.connectors, &self.dirs, &self.launcher, &mut self.log) { + match service.spawn(&path, &owed, self.system, self.syscap, &self.connectors, &self.grants, &self.launcher, &mut self.log) { Ok(new) => say!("supervisor: {label} (pid {pid}) ended; started again as pid {new}"), Err(e) => { let mut kept = service.kept.lock().expect("supervisor: a service's state is poisoned"); @@ -1377,7 +1375,7 @@ impl<'a> Supervisor<'a> { self.make_home(program); let service = &mut self.services[index]; let name = service.program.name.clone(); - match service.spawn(previous, owed, self.system, self.syscap, &self.connectors, &self.dirs, &self.launcher, &mut self.log) { + match service.spawn(previous, owed, self.system, self.syscap, &self.connectors, &self.grants, &self.launcher, &mut self.log) { Ok(pid) => { service.kept.lock().expect("supervisor: a service's state is poisoned").swapping = false; swapped(&self.log, &name, Word::Restored, &format!("{previous} as pid {pid}")); @@ -1616,7 +1614,7 @@ impl Supervisor<'_> { self.syscap, Served::Move(&mut self.acceptors), &self.connectors, - &self.dirs, + &self.grants, &extras, Storage::default(), (&self.launcher, session), @@ -1889,7 +1887,7 @@ fn start<'a>( syscap: &SysCap, served: Served<'_, 'a>, connectors: &BTreeMap<&str, Connector>, - dirs: &BTreeMap, + grants: &Grants<'_>, extras: &[(&str, Connector)], storage: Storage, launcher: (&Acceptor, Session), @@ -1921,7 +1919,7 @@ fn start<'a>( command.endow(&label, raw.0); held.0.push(raw); } - if let Some(ns) = build_namespace(program, system, connectors, dirs, extras)? { + if let Some(ns) = build_namespace(program, system, connectors, grants, extras)? { let raw = ns.into_raw(); command.endow(SVC_LABEL, raw.0); held.0.push(raw); @@ -2214,13 +2212,13 @@ fn build_namespace( program: &Program, system: &Manifest, connectors: &BTreeMap<&str, Connector>, - dirs: &BTreeMap, + grants: &Grants<'_>, extras: &[(&str, Connector)], ) -> std::io::Result> { // Never the swap port: [`swap_namespace`] says why. let receives: Vec<&String> = program.receives.iter().filter(|name| *name != toyos_swap::PORT).collect(); - let view: Vec<(&String, &Connector)> = dirs.iter().filter(|(name, _)| in_view(program, name)).collect(); + let view = grants.view(program); if receives.is_empty() && extras.is_empty() && view.is_empty() { return Ok(None); } @@ -2235,7 +2233,7 @@ fn build_namespace( ), } } - for (name, connector) in view { + for (name, connector) in &view { builder = builder.add(name, connector); } // A `provides` name reaches its holder from whoever made the port, never @@ -2252,15 +2250,67 @@ fn build_namespace( } } -/// Whether the directory capability `name` is in `program`'s view. +/// The instance of the supervisor's own grants, which no start is given. +const SUPERVISOR_INSTANCE: u64 = 0; + +/// What each program's directory capabilities are minted from: each +/// file-server role's port, and the count of starts. /// -/// **Every program sees the whole tree the file servers serve**, which is the -/// kernel's old view kept whole until each row declares its own -/// (`issues/every-program-sees-only-the-files-it-was-given.md`, -/// stage 2), with one exception: a storage row sees none, since a file server -/// resolving a path of its own through itself waits for ever. -fn in_view(program: &Program, name: &str) -> bool { - !is_storage(program) && name.starts_with(CAPABILITY_PREFIX) +/// **Each start is minted grants of its own** (`toyos::fs::Grant`), one per +/// directory of every role, all naming one instance, so the servers count the +/// process and every child it spawns directly, which holds the same +/// connectors, against one share. A role whose ports closed for good is +/// minted nothing. +struct Grants<'a> { + /// Each role's service, whose kept acceptor is the role's port. + roles: Vec<(&'a str, Arc>)>, + /// The instance the next start is minted. + next: Cell, +} + +impl Grants<'_> { + /// The directory capabilities `program` is endowed for one start, by + /// namespace name. + /// + /// **Every program sees the whole tree the file servers serve**, which is + /// the kernel's old view kept whole until each row declares its own + /// (`issues/every-program-sees-only-the-files-it-was-given.md`, stage 2), + /// with one exception: a storage row sees none, since a file server + /// resolving a path of its own through itself waits for ever. Asked before + /// anything is locked, since a storage row's start holds its own kept state. + fn view(&self, program: &Program) -> Vec<(String, Connector)> { + if is_storage(program) { + return Vec::new(); + } + let instance = self.next.get(); + self.next.set(instance + 1); + let mut view = Vec::new(); + for (role, kept) in &self.roles { + let kept = kept.lock().expect("supervisor: a service's state is poisoned"); + if let Some((_, acceptor)) = kept.acceptors.first() { + view.extend(mint_grants(role, acceptor, instance)); + } + } + view + } +} + +/// A grant on `acceptor`, the `role`'s port, for each of its directories, +/// naming `instance`: each by the namespace name a program opens it under. +fn mint_grants(role: &str, acceptor: &Acceptor, instance: u64) -> Vec<(String, Connector)> { + let dirs = toyos_manifest::role_dirs(role).expect("supervisor: the build refuses a role it does not know"); + dirs.iter() + .map(|dir| { + let mut badge = [0u8; MAX_BADGE]; + let badge = Grant { instance, root: dir.root } + .encode(&mut badge) + .unwrap_or_else(|| panic!("supervisor: {}'s root {:?} is no grant's", dir.dir, dir.root)); + let connector = acceptor + .mint(badge) + .unwrap_or_else(|e| panic!("supervisor: no grant on {} for instance {instance}: {e:?}", dir.dir)); + (format!("{CAPABILITY_PREFIX}{}", dir.dir), connector) + }) + .collect() } /// [`toyos_swap::PORT`] in a namespace of its own, for a program whose row