diff --git a/Cargo.lock b/Cargo.lock index 75516233810..d33802e483a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -6089,6 +6089,7 @@ version = "0.1.0" dependencies = [ "ed25519-dalek 2.2.0", "sha2 0.10.9", + "toyos-wallclock", ] [[package]] diff --git a/issues/a-directory-rename-on-data-is-not-atomic.md b/issues/a-directory-rename-on-data-is-not-atomic.md new file mode 100644 index 00000000000..457e55f9211 --- /dev/null +++ b/issues/a-directory-rename-on-data-is-not-atomic.md @@ -0,0 +1,32 @@ +--- +status: open +kind: defect +opened: 2026-10-09 +--- + +# A directory rename on DATA is not atomic + +Fileserver's. `userland/fileserver/src/data.rs`'s `rename` of a directory +renames each entry under it one at a time, because the format keys every file +by its whole path and has no rename of a prefix; its own comment says a kill +in the middle leaves the directory in two halves. A refused write does the +same without a kill: a host test of `DataVolume` (scratch, not committed) made +`home/staged/a` (5 bytes) and `home/staged/z` (240 pages on a fragmented +volume), and renamed `home/staged` to a 300-byte name. `rename` answered +`ResourceExhausted` (the format's `EntryTooLarge { size: 4192, max: 4064 }` +for `z`), and the volume then held `/a` and `home/staged/z`: half the +directory under each name, and the call reported as failed. The format keeps +no journal either (`issues/bcachefs-crate-is-not-bcachefs.md`), so even one +entry's rename is only as whole as the sync that writes it. + +**What it blocks.** The package track's stage-then-commit +(`issues/a-package-is-a-directory-under-apps-and-the-installer-is-a-program.md`): +`/system/bin/pkg` stages `/apps/` privately and commits it in one step, +which a rename that can leave half a package under `/apps` is not. + +## Exit condition + +A directory rename on DATA leaves the directory whole under exactly one of its +names whatever write is refused and wherever the server is killed, measured by +a test that refuses every write of the rename in turn and kills the server at +every one. diff --git a/issues/a-package-is-a-directory-under-apps-and-the-installer-is-a-program.md b/issues/a-package-is-a-directory-under-apps-and-the-installer-is-a-program.md index 52b029bfcb1..fd442484691 100644 --- a/issues/a-package-is-a-directory-under-apps-and-the-installer-is-a-program.md +++ b/issues/a-package-is-a-directory-under-apps-and-the-installer-is-a-program.md @@ -52,12 +52,14 @@ the binary in a ToyOS guest is this track's harness's job, not gbae's. under `/apps` because `/apps` is writable to it, and it asks the user before it installs — at install, the moment the user typed the command, never at first run, so a refusal leaves nothing on disk and no answer has - to be stored. `pkg install `, `pkg install `, `pkg remove + to be stored. `pkg install `, `pkg install `, `pkg remove `, `pkg list`. -- **Verification is by the release's own `SHA256SUMS` first**: - the installer fetches the sums file from the same release, checks the - archive against it, and refuses on mismatch or absence. Signatures are a - later stage of the same file, not a different mechanism. +- **Verification is by a signed repository** (owner ruling: `pkg install + ` and the release's `SHA256SUMS` behind it are retired): a root pinned + in the image, then a timestamp and a targets naming each archive by length + and SHA-256 (`toyos-update/src/repo.rs`, written by `src/publish.rs`). + `pkg install ` stays for local and offline installs, checked against + the `SHA256SUMS` beside it. - **Fetching is HTTPS.** GitHub serves releases only over TLS, so `pkg` carries a TLS client; the network stack under it is netd's. A crate that does TLS is not our job to write and is widely used; it takes the fork @@ -81,16 +83,20 @@ The storage track's users and mount-protocol stages do not block this one. a device or a right. The stage-then-commit above amends it: `pkg` writes `/apps//` in place today, its `manifest.toml` last (`userland/pkg/src/main.rs`). -2. The HTTPS fetch: TLS client under `pkg`, the GitHub redirect, the sums - file from the same release. This is the internet-client track's last +2. The HTTPS fetch: TLS client under `pkg`, the GitHub redirect, the signed + repository's files and the archives its targets names. This is the + internet-client track's last stage (`issues/the-internet-clients-work-unchanged.md`). Judged in QEMU against a server the harness runs on the host in Rust; then once against GitHub itself, by hand, with the owner watching. No registered test fetches anything. 3. Updates: `pkg install` of a newer version replaces the directory whole after the new archive verified; the old one is gone only after the new one is in place. -4. Signatures over the sums file, from a key the owner publishes with the - project. +4. The signed repository. Landed: the verifier and the publisher, on the + host. Owed: `pkg install ` from a mirror list whose one kind is a + local directory, the pinned root and its floors under `/system/etc/pkg/`, + the machine's under `/state/pkg`, and the commit by rename, which waits on + `issues/a-directory-rename-on-data-is-not-atomic.md`. 5. The users track's per-user `/home` (`issues/a-user-is-a-home-tree-and-a-login-row.md`) decides where a package's own data goes. Until then nothing says where: a @@ -112,10 +118,9 @@ The storage track's users and mount-protocol stages do not block this one. 7. **The apps leave this repository.** Each app (snake first, as the pilot: it builds unchanged for every OS) moves to its own repository, built with only the published SDK crates and the released toolchain, and published as - a release archive with its `SHA256SUMS`, the shape gbae already has. The - image then carries none of them; `pkg install ` brings them. Blocked - by stage 6. - Installing by name (`pkg install snake`) needs an index and is undesigned. + a release archive, the shape gbae already has, which the signed + repository's targets names. The image then carries none of them; `pkg + install ` brings them. Blocked by stage 6. **Doom goes at this stage too** (owner ruling, 2026-09-26): the `doom` crate with the doomgeneric C it compiles, `assets/DOOM1.WAD`, and `assets/soundfont.sf2`, which doom alone opens, leave the image as one diff --git a/src/flags.rs b/src/flags.rs index d9173d79433..cf2a6b2259b 100644 --- a/src/flags.rs +++ b/src/flags.rs @@ -76,6 +76,9 @@ declare_flags!(pub CARGO_RUN = { /// Write the image `ssh update` takes to this path, signed with /// the owner's key. pub UPDATE_IMAGE = "--update-image", Next; + /// Publish the packages this `packages.toml` lists into the package + /// repository of the key this run signs with (`src/publish.rs`). + pub PUBLISH = "--publish", Next; }); /// What became of a command line, checked before anything else in `main` runs. diff --git a/src/lib.rs b/src/lib.rs index 45229856279..77e33a131b7 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -35,6 +35,7 @@ pub mod metaldevices; pub mod metalimage; pub mod metaltimings; pub mod n2; +pub mod publish; pub mod release; pub mod sdkversion; pub mod soundfont; diff --git a/src/main.rs b/src/main.rs index fb9ccd2ed8a..9498e8be192 100644 --- a/src/main.rs +++ b/src/main.rs @@ -130,6 +130,31 @@ fn main() { } } } + // Writes the key's package repository and builds nothing. + if let Some(manifest) = CARGO_RUN.value(&args, &flags::PUBLISH) { + let key = toyos_build::signing::key(); + let now = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("a clock after 1970") + .as_secs(); + let published = toyos_build::publish::repository(&root, key) + .and_then(|dir| toyos_build::publish::publish(Path::new(manifest), &dir, key, now).map(|p| (dir, p))); + match published { + Ok((dir, p)) => println!( + "Published into {}: root {}, targets {}, timestamp {}, signed by {}.", + dir.display(), + p.root, + p.targets, + p.timestamp, + key.fingerprint() + ), + Err(why) => { + eprintln!("Error: {why}"); + std::process::exit(1); + } + } + return; + } let arch = toyos_build::build::arch_for(&args); check_prerequisites(&root, arch); diff --git a/src/publish.rs b/src/publish.rs new file mode 100644 index 00000000000..dda43848151 --- /dev/null +++ b/src/publish.rs @@ -0,0 +1,451 @@ +//! The package repository's publisher: `packages.toml` in, a signed +//! repository out, in the format `toyos_update::repo` reads and nothing else +//! writes. +//! +//! **Which repository is decided by which key, as for an image** +//! (`src/signing.rs`): this checkout's throwaway key publishes into +//! [`THROWAWAY_DIR`] under its `target/`, the owner's into [`OWNER_DIR`] under +//! `$HOME`, beside his key and outside every checkout. Stage one fills every +//! role with that one key at threshold one; the root names every role and its +//! threshold, so separating the keys is a new root and not a new client. +//! +//! **A publish only moves forward.** Root `N` stays until it is within +//! [`RENEW`] of its expiry, and then root `N+1` carries the same keys; the +//! targets and the timestamp each take their next version; an item's sequence +//! never falls, and an equal one names the same archive; an archive under +//! `archives/` is never rewritten. Every file lands by rename, synced with its +//! directory, the timestamp last, so a copy of the directory taken mid-publish +//! or after a power loss names only what it holds. **What is written has first +//! been read back through the client** ([`repo::refresh`]), by a machine +//! pinning root 1 and holding the directory as it was, every archive streamed +//! through [`repo::Archive`]: the client's floors are the publisher's, and a +//! repository this writes is one such a machine accepts. +//! +//! ```toml +//! [[package]] +//! name = "gbae" +//! target = "x86_64-unknown-toyos" +//! sequence = 3 +//! version = "0.2.0" +//! archive = "gbae-v0.2.0-toyos-x86_64.tar.gz" # beside this file +//! ``` + +use std::collections::BTreeMap; +use std::fs; +use std::io::Read; +use std::path::{Path, PathBuf}; + +use serde::Deserialize; +use toyos_update::repo::{self, render, Grant, Held, Item, Mirror, Role, Root, Snapshot, Targets, Timestamp}; + +use crate::signing::{Key, Whose}; + +const DAY: u64 = 86_400; +/// How long a root is good for. +pub const ROOT_LIFE: u64 = 365 * DAY; +/// How long a targets is good for: the owner signs one at least this often. +pub const TARGETS_LIFE: u64 = 90 * DAY; +/// How long a timestamp is good for: a freeze lasts no longer. +pub const TIMESTAMP_LIFE: u64 = 7 * DAY; +/// How close to its expiry a root is renewed. +pub const RENEW: u64 = 30 * DAY; + +/// The throwaway key's repository, under the checkout. +pub const THROWAWAY_DIR: &str = "target/pkg-repository"; +/// The owner's repository, under `$HOME`. +pub const OWNER_DIR: &str = ".config/toyos/pkg-repository"; +/// Where archives sit in a repository. +const ARCHIVES: &str = "archives"; + +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct Manifest { + package: Vec, +} + +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct Row { + name: String, + target: String, + sequence: u64, + version: String, + /// Relative to the manifest's directory. + archive: PathBuf, +} + +/// The versions one publish left the repository at. +#[derive(Debug, PartialEq, Eq)] +pub struct Published { + pub root: u64, + pub targets: u64, + pub timestamp: u64, +} + +/// The repository `key` publishes into. +pub fn repository(checkout: &Path, key: &Key) -> Result { + match key.whose() { + Whose::Throwaway => Ok(checkout.join(THROWAWAY_DIR)), + Whose::Owner(_) => { + let home = std::env::var_os("HOME").ok_or("no $HOME, which the owner's repository is under")?; + Ok(PathBuf::from(home).join(OWNER_DIR)) + } + } +} + +/// Publish what `manifest` lists into `dir`, signed by `key`, at `now`. +pub fn publish(manifest: &Path, dir: &Path, key: &Key, now: u64) -> Result { + let text = fs::read_to_string(manifest).map_err(|e| format!("{}: {e}", manifest.display()))?; + let rows: Manifest = toml::from_str(&text).map_err(|e| format!("{}: {e}", manifest.display()))?; + let beside = manifest.parent().unwrap_or(Path::new(".")); + + let mut new: BTreeMap> = BTreeMap::new(); + let current = current_root(dir)?; + let root = match ¤t { + Some((r, _)) if !one_key(r, key) => { + return Err(format!( + "{}'s root {} is not {}'s alone, and a publish rotates no key", + dir.display(), + r.version, + key.fingerprint() + )); + } + Some((r, _)) if r.expires >= now + RENEW => r.version, + _ => { + let version = current.as_ref().map_or(1, |(r, _)| r.version + 1); + let bytes = signed(render::root(&minted(version, now + ROOT_LIFE, key)), Role::Root, key); + new.insert(repo::root_file(version), bytes); + version + } + }; + + let mut items = Vec::new(); + for row in &rows.package { + let path = beside.join(&row.archive); + let bytes = fs::read(&path).map_err(|e| format!("{}: {e}", path.display()))?; + let file = row.archive.file_name().and_then(|f| f.to_str()).ok_or_else(|| format!("{} names no file", path.display()))?; + let url = format!("{ARCHIVES}/{file}"); + // An archive already published, or listed by an earlier row, is never + // rewritten: the read-back's stream refuses one with other bytes. + let there = dir.join(&url).try_exists().map_err(|e| format!("{}: {e}", dir.join(&url).display()))?; + if !there && !new.contains_key(&url) { + new.insert(url.clone(), bytes.clone()); + } + items.push(Item { + name: row.name.clone(), + target: row.target.clone(), + sequence: row.sequence, + version: row.version.clone(), + url, + length: bytes.len() as u64, + sha256: toyos_update::sha256(&bytes), + }); + } + items.sort_by(|a, b| (&a.name, &a.target).cmp(&(&b.name, &b.target))); + + let held = previous(dir)?; + let (timestamp_version, targets_version) = + held.as_ref().map_or((1, 1), |p| (p.timestamp_version + 1, p.targets_version + 1)); + let targets = Targets { version: targets_version, expires: now + TARGETS_LIFE, items }; + let targets_bytes = signed(render::targets(&targets), Role::Targets, key); + let timestamp = Timestamp { + version: timestamp_version, + expires: now + TIMESTAMP_LIFE, + targets: Snapshot { + version: targets.version, + length: targets_bytes.len() as u64, + sha256: toyos_update::sha256(&targets_bytes), + }, + }; + new.insert(repo::targets_file(targets.version), targets_bytes); + new.insert(repo::TIMESTAMP_FILE.into(), signed(render::timestamp(×tamp), Role::Timestamp, key)); + + // Read back before anything is written, as a machine pinning root 1 and + // holding what the directory holds now would read it, every archive + // streamed through the client's check. + let first = match new.get(&repo::root_file(1)) { + Some(bytes) => bytes.clone(), + None => fs::read(dir.join(repo::root_file(1))).map_err(|e| format!("{}: {e}", dir.display()))?, + }; + let machine = current.as_ref().map(|(_, root)| Held { + root, + timestamp: held.as_ref().map(|p| p.timestamp.as_slice()), + targets: held.as_ref().map(|p| p.targets.as_slice()), + }); + let mut overlay = Overlay { dir, new: &new }; + let refused = |what: &str, why: repo::Refused| format!("a machine holding what {} holds now refuses {what}: {why}", dir.display()); + let fresh = repo::refresh(&mut overlay, Held { root: &first, timestamp: None, targets: None }, machine, now) + .map_err(|why| refused("what this publish would leave", why))?; + for item in &fresh.targets.items { + overlay.stream(item)?.map_err(|why| refused(&item.url, why))?; + } + assert_eq!(fresh.targets, targets, "the client reads back the targets this rendered"); + + fs::create_dir_all(dir.join(ARCHIVES)).map_err(|e| format!("{}: {e}", dir.display()))?; + let (mut last, mut rest): (Vec<_>, Vec<_>) = new.iter().partition(|(name, _)| *name == repo::TIMESTAMP_FILE); + rest.append(&mut last); + for (name, bytes) in rest { + land(&dir.join(name), bytes)?; + } + Ok(Published { root, targets: targets.version, timestamp: timestamp.version }) +} + +/// Whether `root` gives every role to `key` alone, at threshold one. +fn one_key(root: &Root, key: &Key) -> bool { + root.keys == [key.public()] && root.grants.iter().all(|g| g.threshold == 1) +} + +fn minted(version: u64, expires: u64, key: &Key) -> Root { + let grant = Grant { threshold: 1, keys: vec![repo::key_id(&key.public())] }; + Root { version, expires, keys: vec![key.public()], grants: [grant.clone(), grant.clone(), grant] } +} + +fn signed(body: String, role: Role, key: &Key) -> Vec { + let line = key.sign_document(role, body.as_bytes()); + (body + &line).into_bytes() +} + +/// The newest root `dir` holds, walked from root 1 as the client walks. +fn current_root(dir: &Path) -> Result)>, String> { + let mut found = None; + for version in 1.. { + let path = dir.join(repo::root_file(version)); + match fs::read(&path) { + Ok(bytes) => { + let root = Root::parse(&bytes).map_err(|why| format!("{}: {why}", path.display()))?; + found = Some((root, bytes)); + } + Err(e) if e.kind() == std::io::ErrorKind::NotFound => break, + Err(e) => return Err(format!("{}: {e}", path.display())), + } + } + Ok(found) +} + +/// What `dir` holds past its roots, where it holds a timestamp. +struct Previous { + timestamp: Vec, + timestamp_version: u64, + /// The targets the timestamp names. + targets: Vec, + targets_version: u64, +} + +fn previous(dir: &Path) -> Result, String> { + let path = dir.join(repo::TIMESTAMP_FILE); + let timestamp = match fs::read(&path) { + Ok(bytes) => bytes, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(e) => return Err(format!("{}: {e}", path.display())), + }; + let parsed = Timestamp::parse(×tamp).map_err(|why| format!("{}: {why}", path.display()))?; + let path = dir.join(repo::targets_file(parsed.targets.version)); + let targets = fs::read(&path).map_err(|e| format!("{}: {e}", path.display()))?; + Ok(Some(Previous { timestamp, timestamp_version: parsed.version, targets, targets_version: parsed.targets.version })) +} + +/// Write `bytes` to `path` whole or not at all, and durably before the next. +fn land(path: &Path, bytes: &[u8]) -> Result<(), String> { + let staged = path.with_extension(format!("staged.{}", std::process::id())); + let parent = path.parent().expect("a file under the repository"); + fs::write(&staged, bytes) + .and_then(|()| fs::File::open(&staged)?.sync_all()) + .and_then(|()| fs::rename(&staged, path)) + .and_then(|()| fs::File::open(parent)?.sync_all()) + .map_err(|e| format!("{}: {e}", path.display())) +} + +/// The repository as it will be: this publish's files over the directory's. +struct Overlay<'a> { + dir: &'a Path, + new: &'a BTreeMap>, +} + +impl Overlay<'_> { + /// `item`'s archive, streamed through the client's check. + fn stream(&self, item: &Item) -> Result, String> { + let mut archive = repo::Archive::of(item); + if let Some(bytes) = self.new.get(&item.url) { + return Ok(archive.take(bytes).and_then(|()| archive.finish())); + } + let path = self.dir.join(&item.url); + let mut file = fs::File::open(&path).map_err(|e| format!("{}: {e}", path.display()))?; + let mut chunk = vec![0; 1 << 16]; + loop { + match file.read(&mut chunk).map_err(|e| format!("{}: {e}", path.display()))? { + 0 => return Ok(archive.finish()), + n => { + if let Err(why) = archive.take(&chunk[..n]) { + return Ok(Err(why)); + } + } + } + } + } +} + +impl Mirror for Overlay<'_> { + fn fetch(&mut self, name: &str, cap: usize) -> Result>, String> { + if let Some(bytes) = self.new.get(name) { + return Ok(Some(bytes[..bytes.len().min(cap + 1)].to_vec())); + } + let mut out = Vec::new(); + match fs::File::open(self.dir.join(name)) { + Ok(file) => file.take(cap as u64 + 1).read_to_end(&mut out).map(|_| Some(out)).map_err(|e| e.to_string()), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None), + Err(e) => Err(e.to_string()), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use toyos_tmpdir::TempDir; + + /// 2026-10-09T00:00:00Z. + const NOW: u64 = 1_791_504_000; + + fn key() -> Key { + Key::throwaway_from([3; 32]) + } + + /// A manifest listing `rows` of `(name, sequence, archive file, bytes)`, + /// written beside its archives. + fn manifest(dir: &Path, rows: &[(&str, u64, &str, &[u8])]) -> PathBuf { + let mut text = String::new(); + for (name, sequence, file, bytes) in rows { + fs::create_dir_all(dir.join(file).parent().unwrap()).unwrap(); + fs::write(dir.join(file), bytes).unwrap(); + text += &format!( + "[[package]]\nname = {name:?}\ntarget = \"x86_64-unknown-toyos\"\nsequence = {sequence}\nversion = \"1.0\"\narchive = {file:?}\n" + ); + } + let path = dir.join("packages.toml"); + fs::write(&path, text).unwrap(); + path + } + + fn read(dir: &Path, name: &str) -> Vec { + fs::read(dir.join(name)).unwrap_or_else(|e| panic!("{name}: {e}")) + } + + fn gbae(targets: &Targets) -> &Item { + targets.items.iter().find(|i| i.name == "gbae").expect("gbae") + } + + /// What a machine holding `dir`'s files accepts next. + fn client(dir: &Path, now: u64, machine: Option>) -> Result { + let first = read(dir, "root.1.txt"); + let none = BTreeMap::new(); + repo::refresh(&mut Overlay { dir, new: &none }, Held { root: &first, timestamp: None, targets: None }, machine, now) + } + + #[test] + fn a_first_publish_mints_root_one_and_a_machine_accepts_it() { + let src = TempDir::new("publish-src"); + let out = TempDir::new("publish-repo"); + let m = manifest(src.path(), &[("snake", 1, "snake-v1.tar.gz", b"snake bytes"), ("gbae", 3, "gbae-v1.tar.gz", b"gbae bytes")]); + assert_eq!(publish(&m, out.path(), &key(), NOW), Ok(Published { root: 1, targets: 1, timestamp: 1 })); + let fresh = client(out.path(), NOW, None).expect("what was published"); + let names: Vec<&str> = fresh.targets.items.iter().map(|i| i.name.as_str()).collect(); + assert_eq!(names, ["gbae", "snake"]); + assert_eq!(read(out.path(), &gbae(&fresh.targets).url), b"gbae bytes"); + let root = Root::parse(&fresh.root).unwrap(); + assert_eq!((root.keys.as_slice(), root.expires), ([key().public()].as_slice(), NOW + ROOT_LIFE)); + assert!(!out.path().read_dir().unwrap().any(|e| e.unwrap().file_name().to_string_lossy().contains("staged"))); + } + + /// The second publish moves every version forward, keeps the root, and a + /// machine holding the first accepts it; one inside the root's last + /// [`RENEW`] adds root 2, which the machine walks to. + #[test] + fn a_publish_moves_forward_and_renews_a_root_near_its_expiry() { + let src = TempDir::new("publish-src"); + let out = TempDir::new("publish-repo"); + let m = manifest(src.path(), &[("gbae", 3, "gbae-v1.tar.gz", b"gbae bytes")]); + publish(&m, out.path(), &key(), NOW).unwrap(); + let first = client(out.path(), NOW, None).unwrap(); + let m = manifest(src.path(), &[("gbae", 4, "gbae-v2.tar.gz", b"gbae two")]); + assert_eq!(publish(&m, out.path(), &key(), NOW + DAY), Ok(Published { root: 1, targets: 2, timestamp: 2 })); + let held = Held { root: &first.root, timestamp: Some(&first.timestamp), targets: Some(&first.targets_bytes) }; + let second = client(out.path(), NOW + DAY, Some(held)).expect("the next publish, past the first's floors"); + assert_eq!(gbae(&second.targets).sequence, 4); + + let late = NOW + ROOT_LIFE - RENEW + 1; + assert_eq!(publish(&m, out.path(), &key(), late), Ok(Published { root: 2, targets: 3, timestamp: 3 })); + let renewed = client(out.path(), late, None).expect("root 2, walked from root 1"); + assert_eq!(Root::parse(&renewed.root).unwrap().version, 2); + } + + #[test] + fn a_publish_that_would_move_anything_back_is_refused_by_name() { + let src = TempDir::new("publish-src"); + let out = TempDir::new("publish-repo"); + publish(&manifest(src.path(), &[("gbae", 3, "gbae-v1.tar.gz", b"gbae bytes")]), out.path(), &key(), NOW).unwrap(); + let before = read(out.path(), "timestamp.txt"); + + // The client's floors, held as the directory holds them. + let (name, target, holder) = (String::from("gbae"), String::from("x86_64-unknown-toyos"), repo::Holder::Machine); + let lower = publish(&manifest(src.path(), &[("gbae", 2, "gbae-v0.tar.gz", b"old")]), out.path(), &key(), NOW); + let why = repo::Refused::Sequence { name: name.clone(), target: target.clone(), sequence: 2, floor: 3, holder }; + assert!(lower.as_ref().unwrap_err().ends_with(&why.to_string()), "{lower:?}"); + let same = publish(&manifest(src.path(), &[("gbae", 3, "gbae-v1b.tar.gz", b"other")]), out.path(), &key(), NOW); + let why = repo::Refused::Reissued { name, target, sequence: 3, holder }; + assert!(same.as_ref().unwrap_err().ends_with(&why.to_string()), "{same:?}"); + let rewritten = publish(&manifest(src.path(), &[("gbae", 4, "gbae-v1.tar.gz", b"rewritten")]), out.path(), &key(), NOW); + assert!(rewritten.as_ref().unwrap_err().contains("refuses archives/gbae-v1.tar.gz: "), "{rewritten:?}"); + let other = publish(&manifest(src.path(), &[("gbae", 4, "gbae-v4.tar.gz", b"new")]), out.path(), &Key::throwaway_from([4; 32]), NOW); + assert!(other.as_ref().unwrap_err().contains("a publish rotates no key"), "{other:?}"); + let unknown = src.path().join("unknown.toml"); + fs::write(&unknown, "[[package]]\nname = \"gbae\"\nowner = \"me\"\n").unwrap(); + assert!(publish(&unknown, out.path(), &key(), NOW).unwrap_err().contains("owner")); + let bad_name = publish(&manifest(src.path(), &[("Gbae", 1, "g.tar.gz", b"g")]), out.path(), &key(), NOW); + assert!(bad_name.as_ref().unwrap_err().contains("refuses what this publish would leave"), "{bad_name:?}"); + + assert_eq!(read(out.path(), "timestamp.txt"), before, "a refused publish wrote nothing"); + assert!(!out.path().join("archives/gbae-v0.tar.gz").exists(), "a refused publish wrote no archive"); + } + + /// A timestamp on disk naming targets 2^64 − 1 leaves no next version: + /// the publish refuses it by name rather than counting past it. + #[test] + fn a_timestamp_naming_the_last_targets_version_is_refused() { + let src = TempDir::new("publish-src"); + let out = TempDir::new("publish-repo"); + let m = manifest(src.path(), &[("gbae", 3, "gbae-v1.tar.gz", b"gbae bytes")]); + publish(&m, out.path(), &key(), NOW).unwrap(); + let mut held = Timestamp::parse(&read(out.path(), "timestamp.txt")).unwrap(); + held.targets.version = u64::MAX; + let last = signed(render::timestamp(&held), Role::Timestamp, &key()); + fs::write(out.path().join("timestamp.txt"), &last).unwrap(); + fs::copy(out.path().join("targets.1.txt"), out.path().join(repo::targets_file(u64::MAX))).unwrap(); + + let why = repo::Refused::Malformed { + role: Role::Timestamp, + line: 3, + why: "`targets` is not ` ` within the targets cap", + }; + assert_eq!(Timestamp::parse(&last).err(), Some(why.clone())); + let refused = publish(&m, out.path(), &key(), NOW + DAY); + assert!(refused.as_ref().unwrap_err().ends_with(&format!("timestamp.txt: {why}")), "{refused:?}"); + assert_eq!(read(out.path(), "timestamp.txt"), last, "a refused publish wrote nothing"); + } + + /// Two rows whose archives share a file name share its url: one archive, + /// or a refusal, never the second row's bytes under the first's SHA-256. + #[test] + fn two_rows_naming_one_archive_with_other_bytes_are_refused() { + let src = TempDir::new("publish-src"); + let out = TempDir::new("publish-repo"); + let rows = [("gbae", 1, "a/x.tar.gz", &b"gbae bytes"[..]), ("snake", 1, "b/x.tar.gz", b"snake bytes")]; + let clash = publish(&manifest(src.path(), &rows), out.path(), &key(), NOW); + let why = repo::Refused::ArchiveShort { length: 11, got: 10 }; + assert!(clash.as_ref().unwrap_err().ends_with(&format!("refuses archives/x.tar.gz: {why}")), "{clash:?}"); + assert!(!out.path().join("timestamp.txt").exists(), "a refused publish wrote nothing"); + + let rows = [("gbae", 1, "a/x.tar.gz", &b"one archive"[..]), ("snake", 1, "b/x.tar.gz", b"one archive")]; + publish(&manifest(src.path(), &rows), out.path(), &key(), NOW).expect("one archive, named twice"); + assert_eq!(read(out.path(), "archives/x.tar.gz"), b"one archive"); + } +} diff --git a/src/signing.rs b/src/signing.rs index 4faf307b757..ae6f35781a0 100644 --- a/src/signing.rs +++ b/src/signing.rs @@ -1,4 +1,5 @@ -//! The key an image is signed with, and the one place a private key is held. +//! The key an image and the package repository are signed with +//! (`src/publish.rs`), and the one place a private key is held. //! //! **Two keys, chosen by what the image is for.** An image for a QEMU guest //! of `cargo run` or `cargo test`, a CI run or a metal-loop stick is signed @@ -24,6 +25,7 @@ use std::path::{Path, PathBuf}; use std::sync::OnceLock; use toyos_update::image::{Header, HEADER_BYTES, SIGNATURE_BYTES, SIGNED_BYTES}; +use toyos_update::repo::{base64_encode, Role}; /// The variable the loader and `/system/bin/update` take the public key from /// at compile time: 64 lowercase hex digits. @@ -104,6 +106,12 @@ impl Key { out } + /// The `sig` line this key vouches for a package repository document's + /// `body` with, as `role` (`src/publish.rs`). + pub fn sign_document(&self, role: Role, body: &[u8]) -> String { + toyos_update::repo::render::signature(&self.seed, role, body) + } + /// A key from a seed the caller chose, for a test that needs a second, /// wrong key or a fixed one. pub fn throwaway_from(seed: [u8; 32]) -> Self { @@ -356,38 +364,10 @@ fn openssh_private(seed: &[u8; 32], public: &[u8; 32]) -> String { text } -const ALPHABET: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; - -fn base64_encode(bytes: &[u8]) -> String { - let mut out = String::new(); - for chunk in bytes.chunks(3) { - let n = chunk.iter().enumerate().fold(0u32, |acc, (i, &b)| acc | u32::from(b) << (16 - 8 * i)); - for i in 0..4 { - if i <= chunk.len() { - out.push(ALPHABET[(n >> (18 - 6 * i) & 63) as usize] as char); - } else { - out.push('='); - } - } - } - out -} - +/// An armoured body's base64, its line breaks dropped. fn base64_decode(text: &str) -> Result, String> { - let digits: Vec = text.bytes().filter(|c| !c.is_ascii_whitespace() && *c != b'=').collect(); - let mut out = Vec::new(); - for chunk in digits.chunks(4) { - if chunk.len() == 1 { - return Err("base64 that ends one digit into a group".into()); - } - let mut n = 0u32; - for (i, c) in chunk.iter().enumerate() { - let v = ALPHABET.iter().position(|a| a == c).ok_or_else(|| format!("{c:#x} is not base64"))?; - n |= (v as u32) << (18 - 6 * i); - } - out.extend_from_slice(&n.to_be_bytes()[1..chunk.len()]); - } - Ok(out) + let joined: String = text.split_ascii_whitespace().collect(); + toyos_update::repo::base64_decode(&joined).ok_or_else(|| "the key is not canonical base64".into()) } #[cfg(test)] @@ -466,8 +446,5 @@ mod tests { blob[at] ^= 1; assert!(openssh_seed(&armour(&blob)).unwrap_err().contains("does not make the public key")); assert!(key.fingerprint().starts_with("SHA256:")); - assert_eq!(base64_decode(&base64_encode(b"any carnal pleas")).unwrap(), b"any carnal pleas"); - assert_eq!(base64_encode(b"Man"), "TWFu"); - assert_eq!(base64_encode(b"Ma"), "TWE="); } } diff --git a/toyos-update/Cargo.toml b/toyos-update/Cargo.toml index 29a60acd30a..0be96e4992a 100644 --- a/toyos-update/Cargo.toml +++ b/toyos-update/Cargo.toml @@ -1,11 +1,13 @@ # A member of the host workspace (root `Cargo.toml`): the bootloader and # `/system/bin/update` depend on it by path and its tests run on the host. It is # every decision a signed image, a slot and its record make, so the loader and -# the updater cannot read one format two ways. +# the updater cannot read one format two ways; and a signed package +# repository's format and every decision its client makes, which the publisher +# (`src/publish.rs`) reads its own output back through. [package] name = "toyos-update" -description = "A signed image, the slots it installs into, and every decision the loader and update make about one, pure." +description = "A signed image, the slots it installs into, and a signed package repository, with every decision a verifier makes about them, pure." version = "0.1.0" edition = "2021" license = "MIT OR Apache-2.0" @@ -29,3 +31,5 @@ ed25519-dalek = { version = "2.2", default-features = false } # SHA-256 over the sections and SHA-512 over the signed header, as SSHSIG # names it. Already resolved in every lockfile that builds this crate. sha2 = { version = "0.10", default-features = false } +# The calendar a repository document's `expires` is written in, the tree's one. +toyos-wallclock = { path = "../toyos-wallclock" } diff --git a/toyos-update/src/lib.rs b/toyos-update/src/lib.rs index 2410b1ac513..b427220bad4 100644 --- a/toyos-update/src/lib.rs +++ b/toyos-update/src/lib.rs @@ -1,6 +1,7 @@ //! A signed image, the slots it is installed into, and every decision the -//! loader and `/system/bin/update` make about one. Pure: no firmware, no -//! device, no allocation. +//! loader and `/system/bin/update` make about one; and the package +//! [`repo`]sitory's signed metadata, and every decision `/system/bin/pkg` makes +//! about it. Pure: no firmware, no device, and no allocation but [`repo`]'s. //! //! **The contract.** An [`image`] is a header naming a monotonic version and //! the SHA-256 of each of its sections — the kernel, its boot parameter and @@ -22,10 +23,13 @@ #![cfg_attr(not(test), no_std)] #![forbid(unsafe_code)] +extern crate alloc; + pub mod floor; pub mod image; pub mod policy; pub mod record; +pub mod repo; pub mod sig; pub mod slots; diff --git a/toyos-update/src/repo.rs b/toyos-update/src/repo.rs new file mode 100644 index 00000000000..29b34322c16 --- /dev/null +++ b/toyos-update/src/repo.rs @@ -0,0 +1,1618 @@ +//! The package repository: what a machine may install, as signed statements +//! the bytes then have to match, and every decision the client makes about +//! them. +//! +//! **The format.** A document is printable ASCII in LF-ended lines of words +//! with one space between: a header `toyos-repo `, then +//! `expires `, then the role's fields in a fixed order — +//! an unknown, repeated or reordered one is refused — then `sig +//! ` lines. A signature is an SSHSIG blob in base64 over exactly the +//! bytes before the first `sig` line, in the role's namespace +//! (`toyos-`), so `ssh-keygen -Y sign -n toyos-` makes one and +//! `ssh-keygen -Y verify` checks one. A key ID is the SHA-256 of the key's +//! OpenSSH public blob. Every document has a size cap, read no further. +//! +//! - `root..txt` declares the keys, and for every [`Role`] which of them +//! sign it and how many must: **a threshold counts distinct key IDs**, so a +//! signature repeated is one signature. +//! - `timestamp.txt` names the one current targets by version, length and +//! SHA-256. +//! - `targets..txt` names each item by name and target, with a sequence +//! that never falls, and its archive by a path under the repository, a +//! length and a SHA-256. The archive is trusted by those two and nothing +//! else, so its bytes may come from anywhere. +//! +//! **The client** ([`refresh`]) is TUF's workflow over a [`Mirror`]: from the +//! newer of the root the image pins and the one the machine holds, it walks +//! `root..txt` while there is one, each signed by its predecessor's +//! threshold and its own; then the timestamp, then the targets it names, each +//! against the final root's threshold, its expiry, and the floors the image and +//! the machine hold: never a lower version, never an equal one with other +//! bytes, never a lower sequence for an item. A held floor counts only while +//! the final root gives its role the keys the root held beside it did, which is +//! how a rotation recovers from a stolen key's fast-forward. [`Archive`] checks +//! the archive's bytes as they stream. + +use alloc::format; +use alloc::string::String; +use alloc::vec::Vec; +use core::fmt; + +use sha2::Digest as _; + +use crate::sig; +use crate::{sha256, Digest}; + +/// The largest `root..txt` read. +pub const ROOT_CAP: usize = 16 << 10; +/// The largest `timestamp.txt` read. +pub const TIMESTAMP_CAP: usize = 4 << 10; +/// The largest `targets..txt` read, and the most a timestamp may name. +pub const TARGETS_CAP: usize = 1 << 20; +/// The longest archive an item may name. +pub const ARCHIVE_CAP: u64 = 1 << 30; +/// The most root versions one [`refresh`] walks. +pub const ROOT_STEPS: u64 = 32; +/// The longest item name and target. +const NAME_MAX: usize = 64; + +const MAGIC: &str = "toyos-repo"; + +/// The file the current timestamp is at. +pub const TIMESTAMP_FILE: &str = "timestamp.txt"; + +/// The file root version `version` is at. +pub fn root_file(version: u64) -> String { + format!("root.{version}.txt") +} + +/// The file targets version `version` is at. +pub fn targets_file(version: u64) -> String { + format!("targets.{version}.txt") +} + +/// Who signs a document, and in which namespace. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Role { + Root, + Timestamp, + Targets, +} + +impl Role { + /// In the order a root names them. + pub const ALL: [Role; 3] = [Role::Root, Role::Timestamp, Role::Targets]; + + pub const fn name(self) -> &'static str { + match self { + Role::Root => "root", + Role::Timestamp => "timestamp", + Role::Targets => "targets", + } + } + + pub const fn namespace(self) -> &'static str { + match self { + Role::Root => "toyos-root", + Role::Timestamp => "toyos-timestamp", + Role::Targets => "toyos-targets", + } + } + + pub const fn cap(self) -> usize { + match self { + Role::Root => ROOT_CAP, + Role::Timestamp => TIMESTAMP_CAP, + Role::Targets => TARGETS_CAP, + } + } +} + +/// A document's signed bytes, in its role's namespace. +pub struct Body<'a> { + pub role: Role, + pub bytes: &'a [u8], +} + +impl sig::Signed for Body<'_> { + fn namespace(&self) -> &'static str { + self.role.namespace() + } + fn bytes(&self) -> &[u8] { + self.bytes + } +} + +/// Whose copy a floor is. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Holder { + /// The image's, under `/system/etc/pkg`: vouched for by the image's + /// signature. + Image, + /// The machine's, under `/state/pkg`: what this client last accepted. + Machine, +} + +/// Why one signature line does not count. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum SigRefused { + /// Its key ID is not one the root gives this role. + NotTheRolesKey, + /// Not canonical base64 of an Ed25519 SSHSIG blob. + Encoding, + /// The blob names a key other than the one its ID does. + OtherKey, + /// The blob is in another role's, or another thing's, namespace. + Namespace, + /// Not the key's signature over these bytes. + Signature, +} + +impl fmt::Display for SigRefused { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(match self { + Self::NotTheRolesKey => "its key is not one the root gives this role", + Self::Encoding => "it is not an Ed25519 SSHSIG blob in canonical base64", + Self::OtherKey => "its blob names another key than its key ID does", + Self::Namespace => "it is signed in another namespace", + Self::Signature => "it is not its key's signature over these bytes", + }) + } +} + +/// Why the client installs nothing. Every variant is a refusal by name. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum Refused { + /// Past the role's size cap. + TooLarge { role: Role, cap: usize }, + /// Not this format. + Malformed { role: Role, line: usize, why: &'static str }, + /// Fewer distinct keys of the role vouch for it than root `root` requires. + Threshold { role: Role, version: u64, root: u64, valid: usize, needed: usize, first: Option }, + /// `root..txt` says another version. + RootVersion { want: u64, got: u64 }, + /// More than [`ROOT_STEPS`] roots in one walk. + RootChain, + /// Past its expiry by the wall clock. + Expired { role: Role, version: u64, expires: u64, now: u64 }, + /// Below a version the image or the machine holds. + Rollback { role: Role, version: u64, floor: u64, holder: Holder }, + /// The version a holder holds, with other bytes. + Changed { role: Role, version: u64, holder: Holder }, + /// A file the workflow requires is not in the repository. + Absent { file: String }, + /// The mirror could not be read. + Fetch { file: String, why: String }, + /// The targets is not the length the timestamp names. + TargetsLength { want: u64, got: u64 }, + /// The targets is not the SHA-256 the timestamp names. + TargetsDigest { version: u64 }, + /// The targets says another version than the timestamp names. + TargetsVersion { want: u64, got: u64 }, + /// An item's sequence below one a holder's targets names. + Sequence { name: String, target: String, sequence: u64, floor: u64, holder: Holder }, + /// An item's sequence equal to a holder's, naming another archive. + Reissued { name: String, target: String, sequence: u64, holder: Holder }, + /// An archive longer than its item says. + ArchivePast { length: u64 }, + /// An archive that ended before its item's length. + ArchiveShort { length: u64, got: u64 }, + /// An archive that is not its item's SHA-256. + ArchiveDigest, + /// A copy a holder keeps that is not a document of its role. + Held { role: Role, holder: Holder }, +} + +impl fmt::Display for Holder { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(match self { + Holder::Image => "the image", + Holder::Machine => "this machine", + }) + } +} + +impl fmt::Display for Refused { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::TooLarge { role, cap } => write!(f, "the {} document is past its cap of {cap} bytes", role.name()), + Self::Malformed { role, line, why } => write!(f, "the {} document's line {line}: {why}", role.name()), + Self::Threshold { role, version, root, valid, needed, first } => { + write!(f, "{} {version} carries {valid} valid signature(s) of the {needed} root {root} requires", role.name())?; + match first { + Some(why) => write!(f, "; the first that did not count: {why}"), + None => Ok(()), + } + } + Self::RootVersion { want, got } => write!(f, "{} calls itself root {got}", root_file(*want)), + Self::RootChain => write!(f, "the repository holds more than {ROOT_STEPS} roots past this machine's"), + Self::Expired { role, version, expires, now } => write!( + f, + "{} {version} expired at {} and the clock says {}", + role.name(), + time_text(*expires), + time_text(*now) + ), + Self::Rollback { role, version, floor, holder } => { + write!(f, "{} {version} is below the {floor} {holder} holds", role.name()) + } + Self::Changed { role, version, holder } => { + write!(f, "{} {version} is not the {} {version} {holder} holds", role.name(), role.name()) + } + Self::Absent { file } => write!(f, "the repository has no {file}"), + Self::Fetch { file, why } => write!(f, "{file}: {why}"), + Self::TargetsLength { want, got } => { + write!(f, "the targets is {got} bytes and the timestamp names {want}") + } + Self::TargetsDigest { version } => { + write!(f, "{} is not the SHA-256 the timestamp names", targets_file(*version)) + } + Self::TargetsVersion { want, got } => { + write!(f, "{} calls itself targets {got}", targets_file(*want)) + } + Self::Sequence { name, target, sequence, floor, holder } => { + write!(f, "{name} for {target} is sequence {sequence}, below the {floor} {holder} holds") + } + Self::Reissued { name, target, sequence, holder } => write!( + f, + "{name} for {target} is sequence {sequence} with another archive than {holder}'s sequence {sequence}" + ), + Self::ArchivePast { length } => write!(f, "the archive runs past its signed {length} bytes"), + Self::ArchiveShort { length, got } => { + write!(f, "the archive ended at {got} bytes, short of its signed {length}") + } + Self::ArchiveDigest => write!(f, "the archive is not the SHA-256 its item names"), + Self::Held { role, holder } => write!(f, "the {} {holder} holds is not one", role.name()), + } + } +} + +/// An Ed25519 public key. +pub type PublicKey = [u8; 32]; + +const ED25519: &[u8] = b"ssh-ed25519"; + +/// `string "ssh-ed25519" | string key`: what OpenSSH names a key by. +pub fn public_blob(key: &PublicKey) -> [u8; 51] { + let mut out = [0u8; 51]; + out[..4].copy_from_slice(&11u32.to_be_bytes()); + out[4..15].copy_from_slice(ED25519); + out[15..19].copy_from_slice(&32u32.to_be_bytes()); + out[19..].copy_from_slice(key); + out +} + +/// A key's ID: the SHA-256 of its public blob, the digest `ssh-keygen -l` +/// prints in base64. +pub fn key_id(key: &PublicKey) -> Digest { + sha256(&public_blob(key)) +} + +/// One role's signers in a root. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Grant { + pub threshold: usize, + /// Ascending, distinct, each declared by the root. + pub keys: Vec, +} + +/// `root..txt`. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Root { + pub version: u64, + pub expires: u64, + /// Every key any role names, ascending by ID. + pub keys: Vec, + /// In [`Role::ALL`]'s order. + pub grants: [Grant; 3], +} + +/// `timestamp.txt`. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Timestamp { + pub version: u64, + pub expires: u64, + pub targets: Snapshot, +} + +/// The targets a timestamp names. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Snapshot { + pub version: u64, + pub length: u64, + pub sha256: Digest, +} + +/// `targets..txt`. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Targets { + pub version: u64, + pub expires: u64, + /// Ascending by name, then target; no pair twice. + pub items: Vec, +} + +/// One installable thing. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Item { + pub name: String, + /// The triple it runs on. + pub target: String, + /// Never falls for a name and target: the package's rollback floor. + pub sequence: u64, + /// What a person reads; nothing decides by it. + pub version: String, + /// The archive's path under the repository. + pub url: String, + pub length: u64, + pub sha256: Digest, +} + +impl Root { + pub fn parse(bytes: &[u8]) -> Result { + Self::of(&document(bytes, Role::Root)?) + } + + pub fn grant(&self, role: Role) -> &Grant { + &self.grants[role as usize] + } + + fn key(&self, id: &Digest) -> Option<&PublicKey> { + self.keys.iter().find(|k| key_id(k) == *id) + } + + fn of(doc: &Doc<'_>) -> Result { + let mut c = doc.fields(); + let mut keys: Vec = Vec::new(); + let mut last: Option = None; + while c.is("key") { + let words: Vec<&str> = c.take("key")?.split(' ').collect(); + let [id, kind, blob] = words[..] else { + return Err(c.last("a key is not `key ssh-ed25519 `")); + }; + let id = hex32(id).ok_or_else(|| c.last("a key ID that is not 64 lowercase hex digits"))?; + let blob = base64_decode(blob).filter(|_| kind == "ssh-ed25519"); + let key: PublicKey = blob + .filter(|b| b.len() == 51 && b[..19] == public_blob(&[0; 32])[..19]) + .map(|b| b[19..].try_into().expect("32 bytes")) + .ok_or_else(|| c.last("a key that is not an ssh-ed25519 public key"))?; + if key_id(&key) != id { + return Err(c.last("a key ID that is not its key's SHA-256")); + } + if last.is_some_and(|l| l >= id) { + return Err(c.last("keys out of order or repeated")); + } + last = Some(id); + keys.push(key); + } + if keys.is_empty() { + return Err(c.here("a root that declares no key")); + } + let ids: Vec = keys.iter().map(key_id).collect(); + let mut grants = Vec::new(); + for role in Role::ALL { + let words: Vec<&str> = c.take("role")?.split(' ').collect(); + if words.len() < 3 || words[0] != role.name() { + return Err(c.last("roles are not `role …` for root, timestamp, targets")); + } + let named = words.len() - 2; + let threshold = number(words[1]) + .filter(|t| (1..=named as u64).contains(t)) + .ok_or_else(|| c.last("a threshold that is not between 1 and the keys its role names"))?; + let mut keys: Vec = Vec::new(); + for word in &words[2..] { + let id = hex32(word).ok_or_else(|| c.last("a key ID that is not 64 lowercase hex digits"))?; + if !ids.contains(&id) { + return Err(c.last("a role names a key the root does not declare")); + } + if keys.last().is_some_and(|l| *l >= id) { + return Err(c.last("a role's keys out of order or repeated")); + } + keys.push(id); + } + grants.push(Grant { threshold: threshold as usize, keys }); + } + c.end()?; + let grants: [Grant; 3] = grants.try_into().expect("one grant per role"); + Ok(Root { version: doc.version, expires: doc.expires, keys, grants }) + } +} + +impl Timestamp { + pub fn parse(bytes: &[u8]) -> Result { + Self::of(&document(bytes, Role::Timestamp)?) + } + + fn of(doc: &Doc<'_>) -> Result { + let mut c = doc.fields(); + let words: Vec<&str> = c.take("targets")?.split(' ').collect(); + let targets = snapshot(&words) + .ok_or_else(|| c.last("`targets` is not ` ` within the targets cap"))?; + c.end()?; + Ok(Timestamp { version: doc.version, expires: doc.expires, targets }) + } +} + +fn snapshot(words: &[&str]) -> Option { + let [version, length, digest] = words else { return None }; + Some(Snapshot { + version: number(version).filter(|v| (1..u64::MAX).contains(v))?, + length: number(length).filter(|l| (1..=TARGETS_CAP as u64).contains(l))?, + sha256: hex32(digest)?, + }) +} + +impl Targets { + pub fn parse(bytes: &[u8]) -> Result { + Self::of(&document(bytes, Role::Targets)?) + } + + fn find(&self, name: &str, target: &str) -> Option<&Item> { + self.items.iter().find(|i| i.name == name && i.target == target) + } + + fn of(doc: &Doc<'_>) -> Result { + let mut c = doc.fields(); + let mut items: Vec = Vec::new(); + while c.is("item") { + let name = c.word("item")?; + if !is_name(name) { + return Err(c.last("an item name that is not 1 to 64 of a-z, 0-9, `.`, `_`, `-`, first not `.`")); + } + let target = c.word("target")?; + if !is_name(target) { + return Err(c.last("a target that is not 1 to 64 of a-z, 0-9, `.`, `_`, `-`, first not `.`")); + } + let sequence = number(c.word("sequence")?).ok_or_else(|| c.last("a sequence that is not a number"))?; + let version = c.word("version")?; + if version.len() > NAME_MAX || version.contains('"') { + return Err(c.last("a version past 64 bytes or with a quote in it")); + } + let url = c.word("url")?; + if !is_relative(url) { + return Err(c.last("a url that is not a path under the repository")); + } + let length = number(c.word("length")?) + .filter(|l| (1..=ARCHIVE_CAP).contains(l)) + .ok_or_else(|| c.last("a length that is not between 1 and the archive cap"))?; + let sha256 = hex32(c.word("sha256")?).ok_or_else(|| c.last("a sha256 that is not 64 lowercase hex digits"))?; + if items.last().is_some_and(|l| (l.name.as_str(), l.target.as_str()) >= (name, target)) { + return Err(c.last("items out of order, or one name and target twice")); + } + items.push(Item { + name: name.into(), + target: target.into(), + sequence, + version: version.into(), + url: url.into(), + length, + sha256, + }); + } + c.end()?; + Ok(Targets { version: doc.version, expires: doc.expires, items }) + } +} + +/// A repository, as the client reads one: a local directory in this stage. +pub trait Mirror { + /// The file `name` under the repository, read to no more than `cap + 1` + /// bytes, so a file past `cap` is seen to be without being read; `None` + /// where the repository has no such file. + fn fetch(&mut self, name: &str, cap: usize) -> Result>, String>; +} + +/// The documents one holder keeps: a root always, the others once accepted. +#[derive(Clone, Copy)] +pub struct Held<'a> { + pub root: &'a [u8], + pub timestamp: Option<&'a [u8]>, + pub targets: Option<&'a [u8]>, +} + +impl<'a> Held<'a> { + fn of(&self, role: Role) -> Option<&'a [u8]> { + match role { + Role::Root => Some(self.root), + Role::Timestamp => self.timestamp, + Role::Targets => self.targets, + } + } +} + +/// What a [`refresh`] accepted: the three documents' bytes, for the machine to +/// hold, and the targets. +pub struct Fresh { + pub root: Vec, + pub timestamp: Vec, + pub targets_bytes: Vec, + pub targets: Targets, +} + +/// The repository's current targets, or the refusal naming why not. +/// +/// `image` is what the image pins, `machine` what this machine accepted last; +/// `now` is the wall clock in Unix seconds. +pub fn refresh(mirror: &mut dyn Mirror, image: Held<'_>, machine: Option>, now: u64) -> Result { + let held_root = |held: Held<'_>, holder| Root::parse(held.root).map_err(|_| Refused::Held { role: Role::Root, holder }); + let image_root = held_root(image, Holder::Image)?; + let machine_root = machine.map(|m| held_root(m, Holder::Machine)).transpose()?; + let holders = [(Holder::Image, Some(image), Some(&image_root)), (Holder::Machine, machine, machine_root.as_ref())]; + + let (mut root, mut root_bytes) = match (&machine_root, machine) { + (Some(m), Some(held)) if m.version > image_root.version => (m.clone(), held.root.to_vec()), + (Some(m), Some(held)) if m.version == image_root.version && held.root != image.root => { + return Err(Refused::Changed { role: Role::Root, version: m.version, holder: Holder::Image }); + } + _ => (image_root.clone(), image.root.to_vec()), + }; + let mut steps = 0; + loop { + let want = root.version + 1; + let Some(bytes) = fetch(mirror, &root_file(want), ROOT_CAP)? else { break }; + if steps == ROOT_STEPS { + return Err(Refused::RootChain); + } + steps += 1; + let doc = document(&bytes, Role::Root)?; + let next = Root::of(&doc)?; + if next.version != want { + return Err(Refused::RootVersion { want, got: next.version }); + } + vouched(&doc, &root)?; + vouched(&doc, &next)?; + root = next; + root_bytes = bytes; + } + unexpired(Role::Root, root.version, root.expires, now)?; + + // Each holder's copy of a role counts while the final root gives the role + // the keys the root held beside it did. + let floors = |role: Role| -> Vec<(Holder, &[u8])> { + let mut out = Vec::new(); + for (holder, held, held_root) in holders { + let (Some(held), Some(held_root)) = (held, held_root) else { continue }; + let Some(bytes) = held.of(role) else { continue }; + if held_root.grant(role).keys == root.grant(role).keys { + out.push((holder, bytes)); + } + } + out + }; + + let timestamp_floors = floors(Role::Timestamp); + let timestamp_bytes = + fetch(mirror, TIMESTAMP_FILE, TIMESTAMP_CAP)?.ok_or_else(|| Refused::Absent { file: TIMESTAMP_FILE.into() })?; + let doc = document(×tamp_bytes, Role::Timestamp)?; + let timestamp = Timestamp::of(&doc)?; + vouched(&doc, &root)?; + for &(holder, held) in ×tamp_floors { + let floor = document(held, Role::Timestamp).map_err(|_| Refused::Held { role: Role::Timestamp, holder })?; + not_back(Role::Timestamp, timestamp.version, ×tamp_bytes, floor.version, held, holder)?; + } + unexpired(Role::Timestamp, timestamp.version, timestamp.expires, now)?; + + let snapshot = timestamp.targets; + let mut targets_floors = Vec::new(); + for (holder, held) in floors(Role::Targets) { + let floor = Targets::parse(held).map_err(|_| Refused::Held { role: Role::Targets, holder })?; + if snapshot.version < floor.version { + return Err(Refused::Rollback { role: Role::Targets, version: snapshot.version, floor: floor.version, holder }); + } + targets_floors.push((holder, held, floor)); + } + let file = targets_file(snapshot.version); + let targets_bytes = fetch(mirror, &file, snapshot.length as usize)?.ok_or(Refused::Absent { file })?; + if targets_bytes.len() as u64 != snapshot.length { + return Err(Refused::TargetsLength { want: snapshot.length, got: targets_bytes.len() as u64 }); + } + if sha256(&targets_bytes) != snapshot.sha256 { + return Err(Refused::TargetsDigest { version: snapshot.version }); + } + let doc = document(&targets_bytes, Role::Targets)?; + let targets = Targets::of(&doc)?; + if targets.version != snapshot.version { + return Err(Refused::TargetsVersion { want: snapshot.version, got: targets.version }); + } + vouched(&doc, &root)?; + for (holder, held, floor) in &targets_floors { + not_back(Role::Targets, targets.version, &targets_bytes, floor.version, held, *holder)?; + for item in &targets.items { + let Some(was) = floor.find(&item.name, &item.target) else { continue }; + if item.sequence < was.sequence { + return Err(Refused::Sequence { + name: item.name.clone(), + target: item.target.clone(), + sequence: item.sequence, + floor: was.sequence, + holder: *holder, + }); + } + if item.sequence == was.sequence && (item.length, item.sha256) != (was.length, was.sha256) { + return Err(Refused::Reissued { + name: item.name.clone(), + target: item.target.clone(), + sequence: item.sequence, + holder: *holder, + }); + } + } + } + unexpired(Role::Targets, targets.version, targets.expires, now)?; + + Ok(Fresh { root: root_bytes, timestamp: timestamp_bytes, targets_bytes, targets }) +} + +/// An item's archive, checked as it streams: never past the signed length, +/// and the signed SHA-256 at its end. +pub struct Archive { + length: u64, + sha256: Digest, + seen: u64, + hash: sha2::Sha256, +} + +impl Archive { + pub fn of(item: &Item) -> Self { + Archive { length: item.length, sha256: item.sha256, seen: 0, hash: sha2::Sha256::new() } + } + + /// The next bytes, refused where they run past the signed length. + pub fn take(&mut self, chunk: &[u8]) -> Result<(), Refused> { + let seen = self.seen.saturating_add(chunk.len() as u64); + if seen > self.length { + return Err(Refused::ArchivePast { length: self.length }); + } + self.hash.update(chunk); + self.seen = seen; + Ok(()) + } + + /// Whether what streamed is the whole archive the item names. + pub fn finish(self) -> Result<(), Refused> { + if self.seen < self.length { + return Err(Refused::ArchiveShort { length: self.length, got: self.seen }); + } + if Digest::from(self.hash.finalize()) != self.sha256 { + return Err(Refused::ArchiveDigest); + } + Ok(()) + } +} + +fn fetch(mirror: &mut dyn Mirror, file: &str, cap: usize) -> Result>, Refused> { + mirror.fetch(file, cap).map_err(|why| Refused::Fetch { file: file.into(), why }) +} + +fn unexpired(role: Role, version: u64, expires: u64, now: u64) -> Result<(), Refused> { + if now >= expires { + return Err(Refused::Expired { role, version, expires, now }); + } + Ok(()) +} + +/// Never below a held version, and an equal one is the held bytes. +fn not_back(role: Role, version: u64, bytes: &[u8], floor: u64, held: &[u8], holder: Holder) -> Result<(), Refused> { + if version < floor { + return Err(Refused::Rollback { role, version, floor, holder }); + } + if version == floor && bytes != held { + return Err(Refused::Changed { role, version, holder }); + } + Ok(()) +} + +/// Whether `by`'s threshold of distinct keys for the document's role signed +/// it. +fn vouched(doc: &Doc<'_>, by: &Root) -> Result<(), Refused> { + let grant = by.grant(doc.role); + let body = Body { role: doc.role, bytes: doc.signed }; + let mut counted: Vec = Vec::new(); + let mut first = None; + for (id, line) in &doc.sigs { + let public = by.key(id).filter(|_| grant.keys.contains(id)); + let verdict = match public { + None => Err(SigRefused::NotTheRolesKey), + Some(public) => check(line, public, &body), + }; + match verdict { + Ok(()) if !counted.contains(id) => counted.push(*id), + Ok(()) => {} + Err(why) => { + first.get_or_insert(why); + } + } + } + if counted.len() < grant.threshold { + return Err(Refused::Threshold { + role: doc.role, + version: doc.version, + root: by.version, + valid: counted.len(), + needed: grant.threshold, + first, + }); + } + Ok(()) +} + +/// One `sig` line's blob, against the key its ID names. +fn check(blob: &str, public: &PublicKey, body: &Body<'_>) -> Result<(), SigRefused> { + let blob = base64_decode(blob).ok_or(SigRefused::Encoding)?; + let (key, namespace, signature) = sshsig(&blob).ok_or(SigRefused::Encoding)?; + if key != public_blob(public) { + return Err(SigRefused::OtherKey); + } + if namespace != body.role.namespace().as_bytes() { + return Err(SigRefused::Namespace); + } + sig::verify(public, body, &signature).map_err(|_| SigRefused::Signature) +} + +/// `(public key blob, namespace, signature)` out of an SSHSIG blob held to +/// `PROTOCOL.sshsig` for Ed25519, or `None`. +fn sshsig(blob: &[u8]) -> Option<(&[u8], &[u8], [u8; 64])> { + let mut rest = blob.strip_prefix(b"SSHSIG")?; + if take(&mut rest, 4)? != 1u32.to_be_bytes() { + return None; + } + let key = string(&mut rest)?; + let namespace = string(&mut rest)?; + let reserved = string(&mut rest)?; + let hash = string(&mut rest)?; + let mut signature = string(&mut rest)?; + if !rest.is_empty() || !reserved.is_empty() || hash != b"sha512" || string(&mut signature)? != ED25519 { + return None; + } + let raw: [u8; 64] = string(&mut signature)?.try_into().ok()?; + signature.is_empty().then_some((key, namespace, raw)) +} + +fn take<'a>(bytes: &mut &'a [u8], n: usize) -> Option<&'a [u8]> { + let (head, rest) = bytes.split_at_checked(n)?; + *bytes = rest; + Some(head) +} + +fn string<'a>(bytes: &mut &'a [u8]) -> Option<&'a [u8]> { + let len = u32::from_be_bytes(take(bytes, 4)?.try_into().expect("four bytes")); + take(bytes, len as usize) +} + +/// A document split into its header, fields and signatures. +struct Doc<'a> { + role: Role, + version: u64, + expires: u64, + /// The bytes the signatures are over: everything before the first `sig`. + signed: &'a [u8], + lines: Vec<&'a str>, + /// The lines between `expires` and the first `sig`. + fields: core::ops::Range, + sigs: Vec<(Digest, &'a str)>, +} + +fn document(bytes: &[u8], role: Role) -> Result, Refused> { + if bytes.len() > role.cap() { + return Err(Refused::TooLarge { role, cap: role.cap() }); + } + let bad = |line: usize, why| Refused::Malformed { role, line, why }; + if let Some(at) = bytes.iter().position(|&b| b != b'\n' && !(b' '..=b'~').contains(&b)) { + let line = bytes[..at].iter().filter(|&&b| b == b'\n').count() + 1; + return Err(bad(line, "a byte that is not printable ASCII")); + } + let text = core::str::from_utf8(bytes).expect("printable ASCII is UTF-8"); + let Some(text) = text.strip_suffix('\n') else { + return Err(bad(text.split('\n').count(), "the last line has no newline")); + }; + let lines: Vec<&str> = text.split('\n').collect(); + if let Some(at) = + lines.iter().position(|l| l.is_empty() || l.starts_with(' ') || l.ends_with(' ') || l.contains(" ")) + { + return Err(bad(at + 1, "a line that is not words with one space between")); + } + let first_sig = lines.iter().position(|l| l.starts_with("sig ")).unwrap_or(lines.len()); + let signed = &bytes[..lines[..first_sig].iter().map(|l| l.len() + 1).sum::()]; + let version = match lines[0].split(' ').collect::>()[..] { + // Below the largest, so a version always has a next one. + [MAGIC, name, version] if name == role.name() => number(version).filter(|v| (1..u64::MAX).contains(v)), + _ => None, + } + .ok_or_else(|| bad(1, "the header is not `toyos-repo ` for this role, 1 to 2^64 - 2"))?; + let mut sigs = Vec::new(); + for (at, line) in lines.iter().enumerate().skip(first_sig) { + let sig = match line.split(' ').collect::>()[..] { + ["sig", id, blob] => hex32(id).map(|id| (id, blob)), + _ => None, + }; + sigs.push(sig.ok_or_else(|| bad(at + 1, "a line among the signatures that is not `sig `"))?); + } + let mut doc = Doc { role, version, expires: 0, signed, lines, fields: 1..first_sig, sigs }; + let mut c = doc.fields(); + let expires = c.word("expires")?; + doc.expires = parse_time(expires).ok_or_else(|| c.last("`expires` is not a YYYY-MM-DDTHH:MM:SSZ that exists"))?; + doc.fields.start = 2; + Ok(doc) +} + +impl<'a> Doc<'a> { + fn fields(&self) -> Fields<'_, 'a> { + Fields { doc: self, at: self.fields.start } + } +} + +/// A walk over a document's fields, each `key value` taken in order. +struct Fields<'d, 'a> { + doc: &'d Doc<'a>, + at: usize, +} + +impl<'a> Fields<'_, 'a> { + fn is(&self, key: &str) -> bool { + self.at < self.doc.fields.end && self.doc.lines[self.at].split_once(' ').is_some_and(|(k, _)| k == key) + } + + /// The next line's value, where its key is `key`. + fn take(&mut self, key: &str) -> Result<&'a str, Refused> { + if self.at >= self.doc.fields.end { + return Err(self.here("the fields end before one the format requires")); + } + match self.doc.lines[self.at].split_once(' ') { + Some((k, value)) if k == key => { + self.at += 1; + Ok(value) + } + _ => Err(self.here("a field that is unknown, repeated, missing or out of order")), + } + } + + /// [`Self::take`], of a value that is one word. + fn word(&mut self, key: &str) -> Result<&'a str, Refused> { + let value = self.take(key)?; + if value.contains(' ') { + return Err(self.last("a value that is not one word")); + } + Ok(value) + } + + fn end(&self) -> Result<(), Refused> { + if self.at < self.doc.fields.end { + return Err(self.here("a field that is unknown, repeated or out of order")); + } + Ok(()) + } + + /// A refusal at the line not yet taken. + fn here(&self, why: &'static str) -> Refused { + Refused::Malformed { role: self.doc.role, line: self.at + 1, why } + } + + /// A refusal at the line last taken. + fn last(&self, why: &'static str) -> Refused { + Refused::Malformed { role: self.doc.role, line: self.at, why } + } +} + +/// A decimal with no sign and no leading zero. +fn number(text: &str) -> Option { + let canonical = !text.is_empty() && text.bytes().all(|b| b.is_ascii_digit()) && (text == "0" || !text.starts_with('0')); + if !canonical { + return None; + } + text.parse().ok() +} + +fn hex32(text: &str) -> Option { + let bytes = text.as_bytes(); + if bytes.len() != 64 { + return None; + } + let digit = |c: u8| match c { + b'0'..=b'9' => Some(c - b'0'), + b'a'..=b'f' => Some(c - b'a' + 10), + _ => None, + }; + let mut out = [0u8; 32]; + for (i, byte) in out.iter_mut().enumerate() { + *byte = digit(bytes[2 * i])? << 4 | digit(bytes[2 * i + 1])?; + } + Some(out) +} + +fn is_name(text: &str) -> bool { + (1..=NAME_MAX).contains(&text.len()) + && !text.starts_with('.') + && text.bytes().all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b"._-".contains(&b)) +} + +/// `a/b.tar.gz`: components of `A-Z a-z 0-9 . _ + -`, none empty, `.` or +/// `..`, so no path leaves the repository and none names a scheme. +fn is_relative(text: &str) -> bool { + text.len() <= 255 + && text.split('/').all(|part| { + !part.is_empty() + && part != "." + && part != ".." + && part.bytes().all(|b| b.is_ascii_alphanumeric() || b"._+-".contains(&b)) + }) +} + +/// `YYYY-MM-DDTHH:MM:SSZ`, an instant that exists, in Unix seconds. +pub fn parse_time(text: &str) -> Option { + let b = text.as_bytes(); + let shape = b.len() == 20 + && b.iter().enumerate().all(|(i, &c)| match i { + 4 | 7 => c == b'-', + 10 => c == b'T', + 13 | 16 => c == b':', + 19 => c == b'Z', + _ => c.is_ascii_digit(), + }); + if !shape { + return None; + } + let field = |at: core::ops::Range| text[at].parse::().expect("digits"); + let civil = toyos_wallclock::Civil { + year: field(0..4), + month: field(5..7), + day: field(8..10), + hour: field(11..13), + min: field(14..16), + sec: field(17..19), + }; + civil.is_valid().then(|| civil.to_unix_secs()) +} + +/// The form [`parse_time`] reads. +pub fn time_text(secs: u64) -> String { + let c = toyos_wallclock::Civil::from_unix_secs(secs); + format!("{:04}-{:02}-{:02}T{:02}:{:02}:{:02}Z", c.year, c.month, c.day, c.hour, c.min, c.sec) +} + +const ALPHABET: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + +/// RFC 4648 base64, padded. +pub fn base64_encode(bytes: &[u8]) -> String { + let mut out = String::new(); + for chunk in bytes.chunks(3) { + let n = chunk.iter().enumerate().fold(0u32, |acc, (i, &b)| acc | u32::from(b) << (16 - 8 * i)); + for i in 0..4 { + out.push(if i <= chunk.len() { ALPHABET[(n >> (18 - 6 * i) & 63) as usize] as char } else { '=' }); + } + } + out +} + +/// RFC 4648 base64, padded and canonical: what [`base64_encode`] writes and +/// nothing else, so one blob has one spelling. +pub fn base64_decode(text: &str) -> Option> { + let bytes = text.as_bytes(); + let pad = bytes.iter().rev().take_while(|&&c| c == b'=').count(); + if !bytes.len().is_multiple_of(4) || pad > 2 { + return None; + } + let mut out = Vec::with_capacity(bytes.len() / 4 * 3); + let (mut acc, mut bits) = (0u32, 0); + for &c in &bytes[..bytes.len() - pad] { + acc = acc << 6 | ALPHABET.iter().position(|&a| a == c)? as u32; + bits += 6; + if bits >= 8 { + bits -= 8; + out.push((acc >> bits) as u8); + acc &= (1 << bits) - 1; + } + } + (acc == 0).then_some(out) +} + +/// The renderer: the publisher's half, beside the parser it has to agree with. +#[cfg(any(test, feature = "sign"))] +pub mod render { + use super::*; + + /// `root`'s signed bytes, unsigned. + pub fn root(root: &Root) -> String { + let mut out = header(Role::Root, root.version, root.expires); + let mut keys: Vec<(Digest, &PublicKey)> = root.keys.iter().map(|k| (key_id(k), k)).collect(); + keys.sort(); + for (id, key) in keys { + out += &format!("key {} ssh-ed25519 {}\n", hex(&id), base64_encode(&public_blob(key))); + } + for role in Role::ALL { + let grant = root.grant(role); + out += &format!("role {} {}", role.name(), grant.threshold); + for id in &grant.keys { + out += &format!(" {}", hex(id)); + } + out.push('\n'); + } + out + } + + pub fn timestamp(timestamp: &Timestamp) -> String { + let t = timestamp.targets; + header(Role::Timestamp, timestamp.version, timestamp.expires) + + &format!("targets {} {} {}\n", t.version, t.length, hex(&t.sha256)) + } + + pub fn targets(targets: &Targets) -> String { + let mut out = header(Role::Targets, targets.version, targets.expires); + for i in &targets.items { + out += &format!( + "item {}\ntarget {}\nsequence {}\nversion {}\nurl {}\nlength {}\nsha256 {}\n", + i.name, + i.target, + i.sequence, + i.version, + i.url, + i.length, + hex(&i.sha256) + ); + } + out + } + + /// The `sig` line `seed` signs `body` with, as `role`. + pub fn signature(seed: &[u8; 32], role: Role, body: &[u8]) -> String { + let public = sig::public_of(seed); + let signature = sig::sign(seed, &Body { role, bytes: body }); + let mut blob = b"SSHSIG".to_vec(); + blob.extend_from_slice(&1u32.to_be_bytes()); + let mut inner = Vec::new(); + put(&mut inner, ED25519); + put(&mut inner, &signature); + for field in [&public_blob(&public)[..], role.namespace().as_bytes(), b"", b"sha512", &inner] { + put(&mut blob, field); + } + format!("sig {} {}\n", hex(&key_id(&public)), base64_encode(&blob)) + } + + fn put(out: &mut Vec, bytes: &[u8]) { + out.extend_from_slice(&(bytes.len() as u32).to_be_bytes()); + out.extend_from_slice(bytes); + } + + fn header(role: Role, version: u64, expires: u64) -> String { + format!("{MAGIC} {} {version}\nexpires {}\n", role.name(), time_text(expires)) + } + + pub fn hex(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02x}")).collect() + } +} + +#[cfg(test)] +mod tests { + use super::render; + use super::*; + use alloc::collections::BTreeMap; + + /// 2026-10-09T00:00:00Z: every test's wall clock. + const NOW: u64 = 1_791_504_000; + const DAY: u64 = 86_400; + const TRIPLE: &str = "x86_64-unknown-toyos"; + const ARCHIVE: &[u8] = b"the archive's bytes"; + + fn seed(n: u8) -> [u8; 32] { + [n; 32] + } + + fn public(n: u8) -> PublicKey { + sig::public_of(&seed(n)) + } + + fn signed(body: String, role: Role, signers: &[u8]) -> Vec { + let mut out = body.clone(); + for &n in signers { + out += &render::signature(&seed(n), role, body.as_bytes()); + } + out.into_bytes() + } + + /// A root whose roles are signed by the key numbers given, each at + /// `threshold`. + fn root(version: u64, roles: [(usize, &[u8]); 3]) -> Root { + let mut all: Vec = roles.iter().flat_map(|(_, k)| k.iter().copied()).collect(); + all.sort(); + all.dedup(); + let grants = roles.map(|(threshold, keys)| { + let mut keys: Vec = keys.iter().map(|&n| key_id(&public(n))).collect(); + keys.sort(); + Grant { threshold, keys } + }); + Root { version, expires: NOW + 365 * DAY, keys: all.iter().map(|&n| public(n)).collect(), grants } + } + + fn one_key(version: u64, n: u8) -> Root { + root(version, [(1, &[n]), (1, &[n]), (1, &[n])]) + } + + fn item(sequence: u64, archive: &[u8]) -> Item { + Item { + name: "gbae".into(), + target: TRIPLE.into(), + sequence, + version: "0.2.0".into(), + url: "archives/gbae-v0.2.0-toyos-x86_64.tar.gz".into(), + length: archive.len() as u64, + sha256: sha256(archive), + } + } + + fn targets(version: u64, items: Vec) -> Targets { + Targets { version, expires: NOW + 90 * DAY, items } + } + + fn timestamp_of(version: u64, targets: &[u8]) -> Timestamp { + let snapshot = Targets::parse(targets).expect("a targets").version; + Timestamp { + version, + expires: NOW + 7 * DAY, + targets: Snapshot { version: snapshot, length: targets.len() as u64, sha256: sha256(targets) }, + } + } + + /// A repository in memory, and the files the client asked it for. + #[derive(Default, Clone)] + struct Repo(BTreeMap>); + + impl Mirror for Repo { + fn fetch(&mut self, name: &str, cap: usize) -> Result>, String> { + Ok(self.0.get(name).map(|b| b[..b.len().min(cap + 1)].to_vec())) + } + } + + impl Repo { + fn put(&mut self, name: &str, bytes: Vec) { + self.0.insert(name.into(), bytes); + } + + /// A targets and the timestamp naming it, both signed by `n`. + fn publish(&mut self, timestamp: u64, t: &Targets, n: u8) -> Vec { + let bytes = signed(render::targets(t), Role::Targets, &[n]); + let ts = signed(render::timestamp(×tamp_of(timestamp, &bytes)), Role::Timestamp, &[n]); + self.put(&targets_file(t.version), bytes.clone()); + self.put(TIMESTAMP_FILE, ts); + bytes + } + } + + /// The image pins root 1, timestamp 1 and targets 1, all key 1's; the + /// repository serves timestamp 2 naming targets 2, with gbae at sequence 3. + struct World { + image_root: Vec, + image_timestamp: Vec, + image_targets: Vec, + repo: Repo, + } + + impl World { + fn new() -> Self { + let mut repo = Repo::default(); + let image_root = signed(render::root(&one_key(1, 1)), Role::Root, &[1]); + let image_targets = repo.publish(1, &targets(1, vec![item(3, ARCHIVE)]), 1); + let image_timestamp = repo.0[TIMESTAMP_FILE].clone(); + repo.publish(2, &targets(2, vec![item(3, ARCHIVE)]), 1); + World { image_root, image_timestamp, image_targets, repo } + } + + fn image(&self) -> Held<'_> { + Held { root: &self.image_root, timestamp: Some(&self.image_timestamp), targets: Some(&self.image_targets) } + } + + /// The repository and the image's copies, borrowed apart. + fn parts(&mut self) -> (&mut Repo, Held<'_>) { + let image = Held { root: &self.image_root, timestamp: Some(&self.image_timestamp), targets: Some(&self.image_targets) }; + (&mut self.repo, image) + } + + fn refresh(&mut self) -> Result { + let (repo, image) = self.parts(); + refresh(repo, image, None, NOW) + } + } + + /// The refusal, and never a print of what was accepted. + fn refused(result: Result) -> Refused { + match result { + Ok(_) => panic!("the client accepted what it must refuse"), + Err(why) => why, + } + } + + #[test] + fn a_repository_the_renderer_writes_is_one_the_client_accepts() { + let mut world = World::new(); + let fresh = world.refresh().expect("the repository"); + let gbae = fresh.targets.find("gbae", TRIPLE).expect("gbae"); + assert_eq!((gbae.sequence, gbae.length), (3, ARCHIVE.len() as u64)); + assert_eq!(fresh.root, world.image_root, "no root past the image's"); + assert_eq!(fresh.targets_bytes, world.repo.0["targets.2.txt"]); + let mut archive = Archive::of(gbae); + archive.take(&ARCHIVE[..5]).unwrap(); + archive.take(&ARCHIVE[5..]).unwrap(); + assert_eq!(archive.finish(), Ok(())); + + // What the machine then holds is a floor and not a refusal: the same + // repository fetched again is accepted. + let machine = Held { root: &fresh.root, timestamp: Some(&fresh.timestamp), targets: Some(&fresh.targets_bytes) }; + let (repo, image) = world.parts(); + refresh(repo, image, Some(machine), NOW).expect("the same repository again"); + } + + /// **The negative control's target.** Two keys, threshold two, and the + /// one key's signature twice: one signature, which does not meet two. + #[test] + fn a_duplicated_signature_does_not_meet_a_threshold() { + let mut world = World::new(); + let two = root(2, [(1, &[1]), (1, &[1]), (2, &[1, 2])]); + world.repo.put("root.2.txt", signed(render::root(&two), Role::Root, &[1])); + let t = targets(2, vec![item(3, ARCHIVE)]); + let body = render::targets(&t); + let line = render::signature(&seed(1), Role::Targets, body.as_bytes()); + let bytes = format!("{body}{line}{line}").into_bytes(); + let ts = signed(render::timestamp(×tamp_of(2, &bytes)), Role::Timestamp, &[1]); + world.repo.put("targets.2.txt", bytes); + world.repo.put(TIMESTAMP_FILE, ts); + assert_eq!( + refused(world.refresh()), + Refused::Threshold { role: Role::Targets, version: 2, root: 2, valid: 1, needed: 2, first: None } + ); + + // Both keys: met. + let both = signed(body, Role::Targets, &[1, 2]); + let ts = signed(render::timestamp(×tamp_of(2, &both)), Role::Timestamp, &[1]); + world.repo.put("targets.2.txt", both); + world.repo.put(TIMESTAMP_FILE, ts); + world.refresh().expect("two distinct keys meet two"); + } + + /// A signature the role's own key made over the very bytes, in another + /// namespace — a timestamp's, or an image's — vouches for nothing here. + #[test] + fn a_signature_in_another_namespace_is_refused() { + let mut world = World::new(); + let body = render::targets(&targets(2, vec![item(3, ARCHIVE)])); + let as_timestamp = render::signature(&seed(1), Role::Timestamp, body.as_bytes()); + let bytes = format!("{body}{as_timestamp}").into_bytes(); + let ts = signed(render::timestamp(×tamp_of(2, &bytes)), Role::Timestamp, &[1]); + world.repo.put("targets.2.txt", bytes); + world.repo.put(TIMESTAMP_FILE, ts); + assert_eq!( + refused(world.refresh()), + Refused::Threshold { role: Role::Targets, version: 2, root: 1, valid: 0, needed: 1, first: Some(SigRefused::Namespace) } + ); + + // The blob relabelled into the right namespace still carries the + // signature made in the other: the bytes signed differ. + let blob = base64_decode(as_timestamp.trim_end().rsplit(' ').next().unwrap()).unwrap(); + let at =blob.windows(15).position(|w| w == b"toyos-timestamp").unwrap(); + let mut moved = blob[..at - 4].to_vec(); + moved.extend_from_slice(&13u32.to_be_bytes()); + moved.extend_from_slice(b"toyos-targets"); + moved.extend_from_slice(&blob[at + 15..]); + let line = format!("sig {} {}\n", render::hex(&key_id(&public(1))), base64_encode(&moved)); + let bytes = format!("{body}{line}").into_bytes(); + let ts = signed(render::timestamp(×tamp_of(2, &bytes)), Role::Timestamp, &[1]); + world.repo.put("targets.2.txt", bytes); + world.repo.put(TIMESTAMP_FILE, ts); + assert!(matches!( + refused(world.refresh()), + Refused::Threshold { first: Some(SigRefused::Signature), valid: 0, .. } + )); + + // And the image's verifier refuses a repository signature: the + // namespaces are the things' own. + let header = [0u8; crate::image::HEADER_BYTES]; + let as_targets = sig::sign(&seed(1), &Body { role: Role::Targets, bytes: &header }); + assert_eq!(sig::verify(&public(1), &header, &as_targets), Err(sig::Refused::Signature)); + } + + #[test] + fn a_role_version_below_the_held_one_is_refused() { + let mut world = World::new(); + let fresh = world.refresh().unwrap(); + let machine_ts = fresh.timestamp.clone(); + let machine_targets = fresh.targets_bytes.clone(); + // The repository rolls back to timestamp 1. + world.repo.put(TIMESTAMP_FILE, world.image_timestamp.clone()); + let machine = Held { root: &fresh.root, timestamp: Some(&machine_ts), targets: Some(&machine_targets) }; + assert_eq!( + refused(refresh(&mut world.repo.clone(), world.image(), Some(machine), NOW)), + Refused::Rollback { role: Role::Timestamp, version: 1, floor: 2, holder: Holder::Machine } + ); + + // A newer timestamp naming the targets below the image's own. + let mut repo = World::new().repo; + let old = repo.0["targets.2.txt"].clone(); + let ts = signed(render::timestamp(×tamp_of(3, &old)), Role::Timestamp, &[1]); + repo.put(TIMESTAMP_FILE, ts); + let image_targets = signed(render::targets(&targets(3, vec![item(3, ARCHIVE)])), Role::Targets, &[1]); + let image = Held { targets: Some(&image_targets), ..world.image() }; + assert_eq!( + refused(refresh(&mut repo, image, None, NOW)), + Refused::Rollback { role: Role::Targets, version: 2, floor: 3, holder: Holder::Image } + ); + } + + #[test] + fn the_same_version_with_other_bytes_is_refused() { + let mut world = World::new(); + // Timestamp 1 again, re-signed with another expiry: the version the + // image holds, and not its bytes. + let image_targets = world.image_targets.clone(); + let mut again = timestamp_of(1, &image_targets); + again.expires += 1; + world.repo.put(TIMESTAMP_FILE, signed(render::timestamp(&again), Role::Timestamp, &[1])); + assert_eq!( + refused(world.refresh()), + Refused::Changed { role: Role::Timestamp, version: 1, holder: Holder::Image } + ); + + // Targets 1 again, another body under the version the image holds. + let mut world = World::new(); + world.repo.publish(2, &targets(1, vec![item(4, ARCHIVE)]), 1); + assert_eq!(refused(world.refresh()), Refused::Changed { role: Role::Targets, version: 1, holder: Holder::Image }); + } + + #[test] + fn expired_metadata_is_refused_with_both_times() { + let mut world = World::new(); + let mut late = World::new(); + let then = Timestamp::parse(&world.repo.0[TIMESTAMP_FILE]).unwrap().expires; + let (repo, image) = world.parts(); + let why = refused(refresh(repo, image, None, then)); + assert_eq!(why, Refused::Expired { role: Role::Timestamp, version: 2, expires: then, now: then }); + assert!(why.to_string().contains(&time_text(then)), "{why}"); + late.repo.publish(3, &Targets { expires: NOW - 1, ..targets(2, vec![item(3, ARCHIVE)]) }, 1); + assert!(matches!(refused(late.refresh()), Refused::Expired { role: Role::Targets, version: 2, .. })); + let root_expiry = Root::parse(&late.image_root).unwrap().expires; + assert!(matches!( + refused(refresh(&mut late.repo, World::new().image(), None, root_expiry)), + Refused::Expired { role: Role::Root, version: 1, .. } + )); + } + + #[test] + fn targets_not_matching_the_timestamp_are_refused() { + let mut world = World::new(); + let mut bytes = world.repo.0["targets.2.txt"].clone(); + bytes.push(b'\n'); + world.repo.put("targets.2.txt", bytes.clone()); + let length = bytes.len() as u64 - 1; + assert_eq!(refused(world.refresh()), Refused::TargetsLength { want: length, got: length + 1 }); + bytes.pop(); + let last = bytes.len() - 2; + bytes[last] ^= 1; + world.repo.put("targets.2.txt", bytes); + assert_eq!(refused(world.refresh()), Refused::TargetsDigest { version: 2 }); + } + + #[test] + fn an_archive_past_its_length_or_not_its_hash_is_refused() { + let gbae = item(3, ARCHIVE); + let mut past = Archive::of(&gbae); + past.take(ARCHIVE).unwrap(); + assert_eq!(past.take(b"x"), Err(Refused::ArchivePast { length: ARCHIVE.len() as u64 })); + let mut long = Archive::of(&gbae); + assert_eq!(long.take(&[ARCHIVE, b"x"].concat()), Err(Refused::ArchivePast { length: ARCHIVE.len() as u64 })); + + let mut bent = ARCHIVE.to_vec(); + bent[0] ^= 1; + let mut wrong = Archive::of(&gbae); + wrong.take(&bent).unwrap(); + assert_eq!(wrong.finish(), Err(Refused::ArchiveDigest)); + let mut short = Archive::of(&gbae); + short.take(&ARCHIVE[..3]).unwrap(); + assert_eq!(short.finish(), Err(Refused::ArchiveShort { length: ARCHIVE.len() as u64, got: 3 })); + } + + #[test] + fn a_lower_sequence_or_a_reissued_one_is_refused() { + let mut world = World::new(); + world.repo.publish(2, &targets(2, vec![item(2, ARCHIVE)]), 1); + assert_eq!( + refused(world.refresh()), + Refused::Sequence { name: "gbae".into(), target: TRIPLE.into(), sequence: 2, floor: 3, holder: Holder::Image } + ); + let mut world = World::new(); + world.repo.publish(2, &targets(2, vec![item(3, b"other bytes")]), 1); + assert_eq!( + refused(world.refresh()), + Refused::Reissued { name: "gbae".into(), target: TRIPLE.into(), sequence: 3, holder: Holder::Image } + ); + let mut world = World::new(); + world.repo.publish(2, &targets(2, vec![item(4, b"other bytes")]), 1); + world.refresh().expect("a higher sequence with other bytes"); + } + + #[test] + fn a_root_not_signed_by_the_previous_threshold_is_refused() { + // Root 2 hands every role to key 2, signed by key 2 alone. + let mut world = World::new(); + let two = one_key(2, 2); + world.repo.put("root.2.txt", signed(render::root(&two), Role::Root, &[2])); + assert_eq!( + refused(world.refresh()), + Refused::Threshold { role: Role::Root, version: 2, root: 1, valid: 0, needed: 1, first: Some(SigRefused::NotTheRolesKey) } + ); + // Signed by key 1 alone: not by its own threshold. + world.repo.put("root.2.txt", signed(render::root(&two), Role::Root, &[1])); + assert_eq!( + refused(world.refresh()), + Refused::Threshold { role: Role::Root, version: 2, root: 2, valid: 0, needed: 1, first: Some(SigRefused::NotTheRolesKey) } + ); + // A root that calls itself another version. + world.repo.put("root.2.txt", signed(render::root(&one_key(3, 1)), Role::Root, &[1])); + assert_eq!(refused(world.refresh()), Refused::RootVersion { want: 2, got: 3 }); + } + + /// Signed by both: the rotation is taken, the image's floors were key 1's + /// and count no more, so a repository the stolen key fast-forwarded is + /// left behind; and the new key's own documents are accepted. + #[test] + fn a_rotated_root_is_walked_and_drops_the_old_keys_floors() { + let mut world = World::new(); + let two = one_key(2, 2); + world.repo.put("root.2.txt", signed(render::root(&two), Role::Root, &[1, 2])); + assert!(matches!( + refused(world.refresh()), + Refused::Threshold { role: Role::Timestamp, root: 2, first: Some(SigRefused::NotTheRolesKey), .. } + )); + world.repo.publish(1, &targets(1, vec![item(1, ARCHIVE)]), 2); + let fresh = world.refresh().expect("the rotated repository from version 1"); + assert_eq!(Root::parse(&fresh.root).unwrap(), two); + } + + /// The machine holds root 2, which took every role from key 1 for key 2; + /// a mirror that withholds `root.2.txt` and serves key 1's timestamp is + /// still judged by root 2, never by the image's older root 1. + #[test] + fn a_machines_newer_root_holds_though_the_mirror_withholds_it() { + let mut world = World::new(); + let two = signed(render::root(&one_key(2, 2)), Role::Root, &[1, 2]); + let machine = Held { root: &two, timestamp: None, targets: None }; + let (repo, image) = world.parts(); + assert!(matches!( + refused(refresh(repo, image, Some(machine), NOW)), + Refused::Threshold { role: Role::Timestamp, root: 2, first: Some(SigRefused::NotTheRolesKey), .. } + )); + } + + #[test] + fn a_machines_root_at_the_images_version_with_other_bytes_is_refused() { + let mut world = World::new(); + let other = signed(render::root(&Root { expires: NOW + DAY, ..one_key(1, 1) }), Role::Root, &[1]); + assert_ne!(other, world.image_root); + let machine = Held { root: &other, timestamp: None, targets: None }; + let (repo, image) = world.parts(); + assert_eq!( + refused(refresh(repo, image, Some(machine), NOW)), + Refused::Changed { role: Role::Root, version: 1, holder: Holder::Image } + ); + } + + /// No document is at the last version, so a walk always has a next one to + /// ask for: a held root there is not a root, and a mirror's is refused by + /// its header. + #[test] + fn a_root_at_the_last_version_is_refused() { + let mut world = World::new(); + let last = signed(render::root(&one_key(u64::MAX, 1)), Role::Root, &[1]); + let machine = Held { root: &last, timestamp: None, targets: None }; + let (repo, image) = world.parts(); + assert_eq!( + refused(refresh(repo, image, Some(machine), NOW)), + Refused::Held { role: Role::Root, holder: Holder::Machine } + ); + + let mut world = World::new(); + let before = signed(render::root(&one_key(u64::MAX - 1, 1)), Role::Root, &[1]); + world.repo.put(&root_file(u64::MAX), last.clone()); + let machine = Held { root: &before, timestamp: None, targets: None }; + let (repo, image) = world.parts(); + assert!(matches!( + refused(refresh(repo, image, Some(machine), NOW)), + Refused::Malformed { role: Role::Root, line: 1, .. } + )); + } + + #[test] + fn a_chain_past_the_walk_bound_is_refused() { + let mut world = World::new(); + for v in 2..=ROOT_STEPS + 2 { + world.repo.put(&root_file(v), signed(render::root(&one_key(v, 1)), Role::Root, &[1])); + } + assert_eq!(refused(world.refresh()), Refused::RootChain); + world.repo.0.remove(&root_file(ROOT_STEPS + 2)); + assert_eq!(Root::parse(&world.refresh().unwrap().root).unwrap().version, ROOT_STEPS + 1); + } + + #[test] + fn a_document_past_its_cap_is_refused_unread() { + let mut world = World::new(); + let body = render::timestamp(&Timestamp::parse(&world.repo.0[TIMESTAMP_FILE]).unwrap()); + let mut bytes = signed(body, Role::Timestamp, &[1]); + bytes.resize(TIMESTAMP_CAP + 4096, b'x'); + world.repo.put(TIMESTAMP_FILE, bytes); + assert_eq!(refused(world.refresh()), Refused::TooLarge { role: Role::Timestamp, cap: TIMESTAMP_CAP }); + let mut world = World::new(); + world.repo.put("root.2.txt", vec![b'k'; ROOT_CAP + 1]); + assert_eq!(refused(world.refresh()), Refused::TooLarge { role: Role::Root, cap: ROOT_CAP }); + } + + /// Every bend of a valid document a parser could be lenient about. + #[test] + fn a_document_that_is_not_the_format_is_refused_by_line() { + let body = render::targets(&targets(2, vec![item(3, ARCHIVE)])); + let good = signed(body.clone(), Role::Targets, &[1]); + Targets::parse(&good).expect("the unbent document"); + let bend = |from: &str, to: &str| { + assert!(body.contains(from), "{from:?}"); + let text = String::from_utf8(good.clone()).unwrap().replacen(from, to, 1); + Targets::parse(text.as_bytes()).expect_err(&format!("{from:?} -> {to:?}")) + }; + let line = |r: Refused| match r { + Refused::Malformed { line, .. } => line, + other => panic!("not a format refusal: {other}"), + }; + assert_eq!(line(bend("toyos-repo targets 2", "toyos-repo timestamp 2")), 1); + assert_eq!(line(bend("toyos-repo targets 2", "toyos-repo targets 02")), 1); + assert_eq!(line(bend("-01-07T", "-02-30T")), 2); + assert_eq!(line(bend("T00:00:00Z", "T24:00:00Z")), 2); + assert_eq!(line(bend("sequence 3\n", "sequence 3\nsequence 3\n")), 6); + assert_eq!(line(bend("version 0.2.0\n", "")), 6); + assert_eq!(line(bend("version 0.2.0\n", "version 0.2.0\nowner me\n")), 7); + assert_eq!(line(bend("item gbae", "item Gbae")), 3); + assert_eq!(line(bend("url archives/", "url ../")), 7); + assert_eq!(line(bend("url archives/", "url https://example.org/")), 7); + assert_eq!(line(bend("length ", "length 0")), 8); + assert_eq!(line(bend("\n", "\r\n")), 1); + assert_eq!(line(bend("version 0.2.0", "version 0.2.0 beta")), 6); + assert_eq!(line(bend("version 0.2.0", "version 0.2.0")), 6); + let mut unended = good.clone(); + unended.pop(); + assert!(matches!(Targets::parse(&unended), Err(Refused::Malformed { .. }))); + let after = [good.clone(), b"item late\n".to_vec()].concat(); + assert!(matches!(Targets::parse(&after), Err(Refused::Malformed { .. }))); + + // Items in order, once each. + let twice = render::targets(&targets(2, vec![item(3, ARCHIVE), item(3, ARCHIVE)])); + assert!(matches!(Targets::parse(twice.as_bytes()), Err(Refused::Malformed { line: 16, .. }))); + + // A root whose key line is not its key's ID, a threshold past its + // keys, roles out of order. + let r = render::root(&root(1, [(1, &[1]), (1, &[1]), (1, &[1, 2])])); + let id1 = render::hex(&key_id(&public(1))); + let id2 = render::hex(&key_id(&public(2))); + let rbend = |from: &str, to: &str| { + assert!(r.contains(from), "{from:?}"); + Root::parse(r.replacen(from, to, 1).as_bytes()).expect_err(&format!("{from:?} -> {to:?}")) + }; + Root::parse(r.as_bytes()).expect("the unbent root"); + assert!(matches!(rbend(&format!("key {id1}"), &format!("key {id2}")), Refused::Malformed { .. })); + assert!(matches!(rbend("role targets 1", "role targets 3"), Refused::Malformed { .. })); + assert!(matches!(rbend("role root", "role timestamp"), Refused::Malformed { line: 5, .. })); + assert!(matches!(rbend(&format!("role timestamp 1 {id1}"), &format!("role timestamp 1 {}", "0".repeat(64))), Refused::Malformed { .. })); + } + + #[test] + fn base64_has_one_spelling() { + for bytes in [&b""[..], b"M", b"Ma", b"Man", b"any carnal pleas"] { + assert_eq!(base64_decode(&base64_encode(bytes)).as_deref(), Some(bytes)); + } + assert_eq!(base64_encode(b"Ma"), "TWE="); + for bent in ["TWF", "TWE", "TWF=", "TW==x", "TQ=", "TR==", "T===", "TWE=\n", "TW E="] { + assert_eq!(base64_decode(bent), None, "{bent:?}"); + } + } + + #[test] + fn a_time_is_one_that_exists_and_reads_back() { + assert_eq!(parse_time("2026-10-09T00:00:00Z"), Some(NOW)); + assert_eq!(time_text(NOW), "2026-10-09T00:00:00Z"); + for bad in ["2026-10-09 00:00:00Z", "2026-10-09T00:00:00", "2026-13-01T00:00:00Z", "2027-02-29T00:00:00Z", "+026-10-09T00:00:00Z"] { + assert_eq!(parse_time(bad), None, "{bad}"); + } + } +} diff --git a/toyos-update/src/sig.rs b/toyos-update/src/sig.rs index 6f19663be15..d9adb982af1 100644 --- a/toyos-update/src/sig.rs +++ b/toyos-update/src/sig.rs @@ -1,19 +1,20 @@ -//! The signature over an image's header: Ed25519, over the message OpenSSH's -//! SSHSIG format builds (`PROTOCOL.sshsig`), in the [`NAMESPACE`] this tree -//! owns. +//! The signature over anything the owner signs: Ed25519, over the message +//! OpenSSH's SSHSIG format builds (`PROTOCOL.sshsig`), in the namespace of +//! what is signed. //! //! ```text -//! signed data "SSHSIG" | string NAMESPACE | string "" | string "sha512" -//! | string SHA-512(header) +//! signed data "SSHSIG" | string namespace | string "" | string "sha512" +//! | string SHA-512(bytes) //! ``` //! //! where `string` is a big-endian `u32` length and the bytes. **Why SSHSIG -//! and not the header raw**: it is exactly what `ssh-keygen -Y sign -n -//! toyos-image` signs, so an image signed by an implementation this tree did -//! not write verifies here, and an owner may hold the key in any agent or -//! token OpenSSH can sign with. The namespace is what stops a signature the -//! owner's key made for anything else — a git commit, a file — from verifying -//! as an image. +//! and not the bytes raw**: it is exactly what `ssh-keygen -Y sign -n +//! ` signs, so a signature an implementation this tree did not +//! write made verifies here, and an owner may hold the key in any agent or +//! token OpenSSH can sign with. **The namespace is the [`Signed`] thing's own, +//! never an argument**: it is what stops a signature the owner's key made for +//! anything else — a git commit, a package repository's targets, an image — +//! from verifying as this. use ed25519_dalek::{Signature, VerifyingKey}; use sha2::{Digest as _, Sha512}; @@ -26,53 +27,83 @@ pub const NAMESPACE: &str = "toyos-image"; const PREAMBLE: &[u8; 6] = b"SSHSIG"; const HASH: &str = "sha512"; -/// The signed data's length: the preamble, four strings, and a SHA-512. -pub const MESSAGE_BYTES: usize = 6 + (4 + NAMESPACE.len()) + 4 + (4 + HASH.len()) + (4 + 64); +/// The longest namespace anything here is signed in: `toyos-timestamp`. +const MAX_NAMESPACE: usize = 15; -/// The bytes Ed25519 signs for `header`. -pub fn message(header: &[u8; HEADER_BYTES]) -> [u8; MESSAGE_BYTES] { - let mut out = [0u8; MESSAGE_BYTES]; - let mut at = 0; +/// What a signature is over: bytes, and the namespace they are signed in. +pub trait Signed { + fn namespace(&self) -> &'static str; + fn bytes(&self) -> &[u8]; +} + +/// An image's header, in [`NAMESPACE`]. +impl Signed for [u8; HEADER_BYTES] { + fn namespace(&self) -> &'static str { + NAMESPACE + } + fn bytes(&self) -> &[u8] { + self + } +} + +/// The bytes Ed25519 signs for one [`Signed`]: the preamble, four strings and +/// a SHA-512, held without allocating. +pub struct Message { + bytes: [u8; 6 + (4 + MAX_NAMESPACE) + 4 + (4 + HASH.len()) + (4 + 64)], + len: usize, +} + +impl Message { + pub fn as_bytes(&self) -> &[u8] { + &self.bytes[..self.len] + } +} + +/// The bytes Ed25519 signs for `signed`. +pub fn message(signed: &S) -> Message { + let namespace = signed.namespace().as_bytes(); + assert!(namespace.len() <= MAX_NAMESPACE, "a namespace this crate names is at most {MAX_NAMESPACE} bytes"); + let mut out = Message { bytes: [0; 6 + (4 + MAX_NAMESPACE) + 4 + (4 + HASH.len()) + (4 + 64)], len: 0 }; let mut put = |bytes: &[u8]| { - out[at..at + bytes.len()].copy_from_slice(bytes); - at += bytes.len(); + out.bytes[out.len..out.len + bytes.len()].copy_from_slice(bytes); + out.len += bytes.len(); }; put(PREAMBLE); - for field in [NAMESPACE.as_bytes(), b"", HASH.as_bytes()] { + for field in [namespace, b"", HASH.as_bytes()] { put(&(field.len() as u32).to_be_bytes()); put(field); } put(&64u32.to_be_bytes()); - put(&Sha512::digest(header)); + put(&Sha512::digest(signed.bytes())); out } -/// Why a signature does not vouch for a header. +/// Why a signature does not vouch for what it is over. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum Refused { - /// The embedded key is not a point on the curve: the binary was built wrong. + /// The key is not a point on the curve. Key, - /// The signature is not the key's over this header. + /// The signature is not the key's over these bytes in this namespace. Signature, } impl core::fmt::Display for Refused { fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { match self { - Self::Key => write!(f, "the embedded image key is not an Ed25519 public key"), - Self::Signature => write!(f, "the signature is not this machine's key's over the header"), + Self::Key => write!(f, "the key is not an Ed25519 public key"), + Self::Signature => write!(f, "the signature is not this key's over these bytes"), } } } -/// Whether `signature` is `key`'s over `header`. +/// Whether `signature` is `key`'s over `signed`, in its namespace. /// /// `verify_strict`: a signature with a non-canonical `S`, or a key of small /// order, is refused rather than accepted by one implementation and not /// another. -pub fn verify(key: &[u8; 32], header: &[u8; HEADER_BYTES], signature: &[u8; SIGNATURE_BYTES]) -> Result<(), Refused> { +pub fn verify(key: &[u8; 32], signed: &S, signature: &[u8; SIGNATURE_BYTES]) -> Result<(), Refused> { let key = VerifyingKey::from_bytes(key).map_err(|_| Refused::Key)?; - key.verify_strict(&message(header), &Signature::from_bytes(signature)) + key.verify_strict(message(signed).as_bytes(), &Signature::from_bytes(signature)) .map_err(|_| Refused::Signature) } @@ -81,12 +112,12 @@ pub fn public_of(seed: &[u8; 32]) -> [u8; 32] { ed25519_dalek::SigningKey::from_bytes(seed).verifying_key().to_bytes() } -/// `seed`'s signature over `header`: the host's half, and no binary on the +/// `seed`'s signature over `signed`: the host's half, and no binary on the /// machine is built with it. -#[cfg(feature = "sign")] -pub fn sign(seed: &[u8; 32], header: &[u8; HEADER_BYTES]) -> [u8; SIGNATURE_BYTES] { +#[cfg(any(test, feature = "sign"))] +pub fn sign(seed: &[u8; 32], signed: &S) -> [u8; SIGNATURE_BYTES] { use ed25519_dalek::Signer as _; - ed25519_dalek::SigningKey::from_bytes(seed).sign(&message(header)).to_bytes() + ed25519_dalek::SigningKey::from_bytes(seed).sign(message(signed).as_bytes()).to_bytes() } /// A 32-byte key from 64 hex digits, at compile time: how the loader and the @@ -187,6 +218,8 @@ mod tests { fn the_message_is_sshsigs_signed_data() { let header = [0xA5u8; HEADER_BYTES]; let m = message(&header); + let m = m.as_bytes(); + assert_eq!(m.len(), 6 + (4 + 11) + 4 + (4 + 6) + (4 + 64)); assert_eq!(&m[..6], b"SSHSIG"); assert_eq!(&m[6..10], &11u32.to_be_bytes()); assert_eq!(&m[10..21], b"toyos-image"); diff --git a/toyos-update/tests/fixtures/repo/root.1.txt b/toyos-update/tests/fixtures/repo/root.1.txt new file mode 100644 index 00000000000..c06c5783695 --- /dev/null +++ b/toyos-update/tests/fixtures/repo/root.1.txt @@ -0,0 +1,7 @@ +toyos-repo root 1 +expires 2027-10-09T00:00:00Z +key 6dd5a817e2e964819ac2f7879f38ff29de14a1f2a64538876c1f4393829a0317 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFCVwn18/3w2fx2VodItUuPdoj1RMleeWnQ0rFkS4qFf +role root 1 6dd5a817e2e964819ac2f7879f38ff29de14a1f2a64538876c1f4393829a0317 +role timestamp 1 6dd5a817e2e964819ac2f7879f38ff29de14a1f2a64538876c1f4393829a0317 +role targets 1 6dd5a817e2e964819ac2f7879f38ff29de14a1f2a64538876c1f4393829a0317 +sig 6dd5a817e2e964819ac2f7879f38ff29de14a1f2a64538876c1f4393829a0317 U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgUJXCfXz/fDZ/HZWh0i1S492iPVEyV55adDSsWRLioV8AAAAKdG95b3Mtcm9vdAAAAAAAAAAGc2hhNTEyAAAAUwAAAAtzc2gtZWQyNTUxOQAAAED6Qc3YmljZZcrs8OtttQoDFbRJ5aBBNWo5ViAdui4cm+hW86pm74rITIx9yOgSN46lvm+DBKMdCzsjLx3zankP diff --git a/toyos-update/tests/fixtures/repo/targets.1.txt b/toyos-update/tests/fixtures/repo/targets.1.txt new file mode 100644 index 00000000000..698beb8d8ad --- /dev/null +++ b/toyos-update/tests/fixtures/repo/targets.1.txt @@ -0,0 +1,10 @@ +toyos-repo targets 1 +expires 2027-01-07T00:00:00Z +item hello +target x86_64-unknown-toyos +sequence 1 +version 1.0.0 +url archives/hello.tar.gz +length 35 +sha256 b27d019bf57b3d45a7449ec9c35679649a1c7aa3dd9bc415448787285fd52986 +sig 6dd5a817e2e964819ac2f7879f38ff29de14a1f2a64538876c1f4393829a0317 U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgUJXCfXz/fDZ/HZWh0i1S492iPVEyV55adDSsWRLioV8AAAANdG95b3MtdGFyZ2V0cwAAAAAAAAAGc2hhNTEyAAAAUwAAAAtzc2gtZWQyNTUxOQAAAEDTOMXDSXILeY/97pbQCDrRhOx0OFUivrnwjjGCWQJ8yHipxJ9fLinah6Dsz4q8XQOjWv6hDR9mk8cX/l9XRnkI diff --git a/toyos-update/tests/fixtures/repo/timestamp.txt b/toyos-update/tests/fixtures/repo/timestamp.txt new file mode 100644 index 00000000000..e79a72ced27 --- /dev/null +++ b/toyos-update/tests/fixtures/repo/timestamp.txt @@ -0,0 +1,4 @@ +toyos-repo timestamp 1 +expires 2026-10-16T00:00:00Z +targets 1 536 824360abf6cb0d9ffe9d48018ff4c44f4a1786f67ca180de52dadadff15816dd +sig 6dd5a817e2e964819ac2f7879f38ff29de14a1f2a64538876c1f4393829a0317 U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgUJXCfXz/fDZ/HZWh0i1S492iPVEyV55adDSsWRLioV8AAAAPdG95b3MtdGltZXN0YW1wAAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5AAAAQBtgTzbD3jtf8PWqig3r8YeE/eTSAsob3twz8PrLgnWnKGkGMWz3f82lc34eJ3WFgCnxKxEM2e1YTLnTditzlAk= diff --git a/toyos-update/tests/repo_oracle.rs b/toyos-update/tests/repo_oracle.rs new file mode 100644 index 00000000000..d6f6f0de95d --- /dev/null +++ b/toyos-update/tests/repo_oracle.rs @@ -0,0 +1,99 @@ +//! **The independent oracle for the repository's signatures**: a repository +//! whose every signature OpenSSH made is one the client accepts, and one byte +//! bent anywhere in it is refused. +//! +//! The fixture is OpenSSH's own: `ssh-keygen -t ed25519` minted a throwaway +//! key, the three bodies were written by hand to the format, `ssh-keygen -Y +//! sign -n toyos-` signed each, its armour's base64 became the `sig` +//! line, and the private key was deleted. The key ID in each was taken from +//! `ssh-keygen -l`'s SHA-256 fingerprint, so the client agreeing with it is +//! the key ID's definition checked too. Nothing runs `ssh-keygen` at test time. + +use toyos_update::repo::{self, Archive, Held, Mirror, Refused, Role, SigRefused}; + +const ROOT: &[u8] = include_bytes!("fixtures/repo/root.1.txt"); +const TIMESTAMP: &[u8] = include_bytes!("fixtures/repo/timestamp.txt"); +const TARGETS: &[u8] = include_bytes!("fixtures/repo/targets.1.txt"); +/// The archive the targets names, by length and SHA-256. +const ARCHIVE: &[u8] = b"the archive ssh-keygen vouched for\n"; +/// 2026-10-09T00:00:00Z, inside every expiry the fixture carries. +const NOW: u64 = 1_791_504_000; + +struct Fixture { + timestamp: Vec, + targets: Vec, +} + +impl Mirror for Fixture { + fn fetch(&mut self, name: &str, cap: usize) -> Result>, String> { + let bytes = match name { + "timestamp.txt" => &self.timestamp, + "targets.1.txt" => &self.targets, + _ => return Ok(None), + }; + Ok(Some(bytes[..bytes.len().min(cap + 1)].to_vec())) + } +} + +fn refresh(root: &[u8], timestamp: &[u8], targets: &[u8]) -> Result { + let mut mirror = Fixture { timestamp: timestamp.to_vec(), targets: targets.to_vec() }; + repo::refresh(&mut mirror, Held { root, timestamp: None, targets: None }, None, NOW) +} + +/// The refusal, and never a print of what was accepted. +fn refused(result: Result) -> Refused { + match result { + Ok(_) => panic!("the client accepted a bent repository"), + Err(why) => why, + } +} + +#[test] +fn a_repository_ssh_keygen_signed_is_accepted_and_a_bent_one_is_not() { + let fresh = refresh(ROOT, TIMESTAMP, TARGETS).expect("the repository OpenSSH signed"); + let hello = fresh.targets.items.iter().find(|i| (i.name.as_str(), i.target.as_str()) == ("hello", "x86_64-unknown-toyos")).expect("the item"); + let mut archive = Archive::of(hello); + archive.take(ARCHIVE).expect("within its length"); + archive.finish().expect("its SHA-256"); + + // The low bit of one character flipped, so the document stays the format + // and only what it says changes: a day of the timestamp's expiry, a + // character of its signature, the targets' sequence. The root is pinned, + // so its own bend is refused as the walk's next root. + let bend = |doc: &[u8], at: usize| { + let mut bent = doc.to_vec(); + bent[at] ^= 1; + bent + }; + let at = |doc: &[u8], text: &str| doc.windows(text.len()).position(|w| w == text.as_bytes()).unwrap() + text.len() - 1; + + let timestamp = refused(refresh(ROOT, &bend(TIMESTAMP, at(TIMESTAMP, "expires 2026-10-16")), TARGETS)); + assert!( + matches!(timestamp, Refused::Threshold { role: Role::Timestamp, first: Some(SigRefused::Signature), .. }), + "{timestamp}" + ); + let signature = refused(refresh(ROOT, &bend(TIMESTAMP, TIMESTAMP.len() - 10), TARGETS)); + assert!(matches!(signature, Refused::Threshold { role: Role::Timestamp, .. }), "{signature}"); + + // A targets bent is no longer the SHA-256 the timestamp names. + let targets = refused(refresh(ROOT, TIMESTAMP, &bend(TARGETS, at(TARGETS, "sequence 1")))); + assert_eq!(targets, Refused::TargetsDigest { version: 1 }); + + struct Next(Vec); + impl Mirror for Next { + fn fetch(&mut self, name: &str, cap: usize) -> Result>, String> { + Ok((name == "root.2.txt").then(|| self.0[..self.0.len().min(cap + 1)].to_vec())) + } + } + let as_two = String::from_utf8(ROOT.to_vec()).unwrap().replacen("toyos-repo root 1", "toyos-repo root 2", 1); + let walked = refused(repo::refresh( + &mut Next(as_two.into_bytes()), + Held { root: ROOT, timestamp: None, targets: None }, + None, + NOW, + )); + assert!( + matches!(walked, Refused::Threshold { role: Role::Root, version: 2, root: 1, first: Some(SigRefused::Signature), .. }), + "{walked}" + ); +}