toyos-update verifies a signed package repository and src/publish.rs writes one through it; pkg install <name> waits on an atomic directory rename on DATA

issues/a-directory-rename-on-data-is-not-atomic.md

@@ -0,0 +1,32 @@
+---
+status: open
+kind: defect
+opened: 2026-10-09
+---
+
+# A directory rename on DATA is not atomic
+
+Fileserver's. `userland/fileserver/src/data.rs`'s `rename` of a directory
+renames each entry under it one at a time, because the format keys every file
+by its whole path and has no rename of a prefix; its own comment says a kill
+in the middle leaves the directory in two halves. A refused write does the
+same without a kill: a host test of `DataVolume` (scratch, not committed) made
+`home/staged/a` (5 bytes) and `home/staged/z` (240 pages on a fragmented
+volume), and renamed `home/staged` to a 300-byte name. `rename` answered
+`ResourceExhausted` (the format's `EntryTooLarge { size: 4192, max: 4064 }`
+for `z`), and the volume then held `<to>/a` and `home/staged/z`: half the
+directory under each name, and the call reported as failed. The format keeps
+no journal either (`issues/bcachefs-crate-is-not-bcachefs.md`), so even one
+entry's rename is only as whole as the sync that writes it.
+
+**What it blocks.** The package track's stage-then-commit
+(`issues/a-package-is-a-directory-under-apps-and-the-installer-is-a-program.md`):
+`/system/bin/pkg` stages `/apps/<name>` privately and commits it in one step,
+which a rename that can leave half a package under `/apps` is not.
+
+## Exit condition
+
+A directory rename on DATA leaves the directory whole under exactly one of its
+names whatever write is refused and wherever the server is killed, measured by
+a test that refuses every write of the rename in turn and kills the server at
+every one.

issues/a-package-is-a-directory-under-apps-and-the-installer-is-a-program.md

@@ -52,12 +52,14 @@ the binary in a ToyOS guest is this track's harness's job, not gbae's.
   under `/apps` because `/apps` is writable to it, and it asks the user
   before it installs — at install, the moment the user typed the command,
   never at first run, so a refusal leaves nothing on disk and no answer has
-  to be stored. `pkg install <url>`, `pkg install <file>`, `pkg remove
+  to be stored. `pkg install <name>`, `pkg install <file>`, `pkg remove
   <name>`, `pkg list`.
-- **Verification is by the release's own `SHA256SUMS` first**:
-  the installer fetches the sums file from the same release, checks the
-  archive against it, and refuses on mismatch or absence. Signatures are a
-  later stage of the same file, not a different mechanism.
+- **Verification is by a signed repository** (owner ruling: `pkg install
+  <url>` and the release's `SHA256SUMS` behind it are retired): a root pinned
+  in the image, then a timestamp and a targets naming each archive by length
+  and SHA-256 (`toyos-update/src/repo.rs`, written by `src/publish.rs`).
+  `pkg install <file>` stays for local and offline installs, checked against
+  the `SHA256SUMS` beside it.
 - **Fetching is HTTPS.** GitHub serves releases only over TLS, so `pkg`
   carries a TLS client; the network stack under it is netd's. A crate that
   does TLS is not our job to write and is widely used; it takes the fork
@@ -81,16 +83,20 @@ The storage track's users and mount-protocol stages do not block this one.
    a device or a right. The stage-then-commit above amends it: `pkg` writes
    `/apps/<name>/` in place today, its `manifest.toml` last
    (`userland/pkg/src/main.rs`).
-2. The HTTPS fetch: TLS client under `pkg`, the GitHub redirect, the sums
-   file from the same release. This is the internet-client track's last
+2. The HTTPS fetch: TLS client under `pkg`, the GitHub redirect, the signed
+   repository's files and the archives its targets names. This is the
+   internet-client track's last
    stage (`issues/the-internet-clients-work-unchanged.md`). Judged in QEMU against a server the harness
    runs on the host in Rust; then once against GitHub itself, by hand, with the owner
    watching. No registered test fetches anything.
 3. Updates: `pkg install` of a newer version replaces the directory whole
    after the new archive verified; the old one is gone only after the new
    one is in place.
-4. Signatures over the sums file, from a key the owner publishes with the
-   project.
+4. The signed repository. Landed: the verifier and the publisher, on the
+   host. Owed: `pkg install <name>` from a mirror list whose one kind is a
+   local directory, the pinned root and its floors under `/system/etc/pkg/`,
+   the machine's under `/state/pkg`, and the commit by rename, which waits on
+   `issues/a-directory-rename-on-data-is-not-atomic.md`.
 5. The users track's per-user `/home`
    (`issues/a-user-is-a-home-tree-and-a-login-row.md`) decides
    where a package's own data goes. Until then nothing says where: a
@@ -112,10 +118,9 @@ The storage track's users and mount-protocol stages do not block this one.
 7. **The apps leave this repository.** Each app (snake first, as the pilot:
    it builds unchanged for every OS) moves to its own repository, built with
    only the published SDK crates and the released toolchain, and published as
-   a release archive with its `SHA256SUMS`, the shape gbae already has. The
-   image then carries none of them; `pkg install <url>` brings them. Blocked
-   by stage 6.
-   Installing by name (`pkg install snake`) needs an index and is undesigned.
+   a release archive, the shape gbae already has, which the signed
+   repository's targets names. The image then carries none of them; `pkg
+   install <name>` brings them. Blocked by stage 6.
    **Doom goes at this stage too** (owner ruling, 2026-09-26): the `doom`
    crate with the doomgeneric C it compiles, `assets/DOOM1.WAD`, and
    `assets/soundfont.sf2`, which doom alone opens, leave the image as one

src/flags.rs

@@ -76,6 +76,9 @@ declare_flags!(pub CARGO_RUN = {
     /// Write the image `ssh <machine> update` takes to this path, signed with
     /// the owner's key.
     pub UPDATE_IMAGE = "--update-image", Next;
+    /// Publish the packages this `packages.toml` lists into the package
+    /// repository of the key this run signs with (`src/publish.rs`).
+    pub PUBLISH = "--publish", Next;
 });
 
 /// What became of a command line, checked before anything else in `main` runs.

src/lib.rs

@@ -35,6 +35,7 @@ pub mod metaldevices;
 pub mod metalimage;
 pub mod metaltimings;
 pub mod n2;
+pub mod publish;
 pub mod release;
 pub mod sdkversion;
 pub mod soundfont;

src/main.rs

@@ -130,6 +130,31 @@ fn main() {
             }
         }
     }
+    // Writes the key's package repository and builds nothing.
+    if let Some(manifest) = CARGO_RUN.value(&args, &flags::PUBLISH) {
+        let key = toyos_build::signing::key();
+        let now = std::time::SystemTime::now()
+            .duration_since(std::time::UNIX_EPOCH)
+            .expect("a clock after 1970")
+            .as_secs();
+        let published = toyos_build::publish::repository(&root, key)
+            .and_then(|dir| toyos_build::publish::publish(Path::new(manifest), &dir, key, now).map(|p| (dir, p)));
+        match published {
+            Ok((dir, p)) => println!(
+                "Published into {}: root {}, targets {}, timestamp {}, signed by {}.",
+                dir.display(),
+                p.root,
+                p.targets,
+                p.timestamp,
+                key.fingerprint()
+            ),
+            Err(why) => {
+                eprintln!("Error: {why}");
+                std::process::exit(1);
+            }
+        }
+        return;
+    }
 
     let arch = toyos_build::build::arch_for(&args);
     check_prerequisites(&root, arch);