diff --git a/.github/audit/application-security.md b/.github/audit/application-security.md index 57710a805..c63853406 100644 --- a/.github/audit/application-security.md +++ b/.github/audit/application-security.md @@ -9,19 +9,37 @@ **Output file:** `audit-application.md` This is a code-and-specs audit of the product's own boundaries — the remote -control stack, and the local application. You need no GitHub API access and no -PAT — do not use one. +control stack, and the local application. You need no PAT — do not use one. +The two pgstencil provenance checks below do read the GitHub API, but only a +public repository, which the workflow's default `GITHUB_TOKEN` and the +operator's own `gh` login both reach; if that API is unreachable, report those +two checks as `UNVERIFIABLE`. For Hosted accounts, read `docs/specs/hosted.md`, `hosted/server/`, -`hosted/src/`, `hosted/scripts/`, `hosted/wrangler.jsonc`, the packed core/auth -modules in `vendor/`, and `.github/workflows/hosted-preview.yml` and +`hosted/src/`, `hosted/scripts/`, `hosted/wrangler.jsonc`, and +`.github/workflows/hosted-preview.yml` and `.github/workflows/hosted-production.yml` — `docs/specs/security-hosted.md`'s Deployment boundary quantifies over the preview and production paths, which live in those scripts and workflows rather than in the Worker. Verify the -archive hashes against `vendor/build.json`. Distinguish tested code from -pending production configuration; do not treat local provider simulations as -live OAuth acceptance, and treat a checked-in placeholder as no evidence about -an external control. +vendored packages by their provenance rather than by reading them: hash each +archive in `vendor/` against `vendor/build.json`; read each archive's own claim +with `tar -xOf vendor/.tgz package/dist/provenance.json` and check +that it names `build.json`'s commit and does not record `dirty`; then check +that commit against pgstencil `main` and its audit: + +```sh +gh api repos/diffplug/pgstencil/compare/...main --jq .status +gh api repos/diffplug/pgstencil/commits//check-runs \ + --jq '.check_runs[] | select(.name=="security-audit") | .conclusion' +``` + +The first must be `ahead` or `identical`, the second `success`. The packed code +itself is audited in `diffplug/pgstencil` by that repository's own +`security-audit` workflow against its `SECURITY.md`; do not audit the tarballs' +contents here — audit how `hosted/` configures the adapter. Distinguish tested +code from pending production configuration; do not treat local provider +simulations as live OAuth acceptance, and treat a checked-in placeholder as no +evidence about an external control. Read, at minimum: `docs/specs/remote-security-model.md` **and its paired `docs/specs/remote-security-model.rationale.md`**, `docs/specs/relay.md`, diff --git a/docs/specs/hosted.md b/docs/specs/hosted.md index df00f3c26..6e7825e81 100644 --- a/docs/specs/hosted.md +++ b/docs/specs/hosted.md @@ -9,7 +9,7 @@ **Must run committed Better Auth migrations before deploying code that needs them, never during a Worker request.** Postgres is reached through an uncached Hyperdrive binding. The runtime creates and closes its database pool within each request. -**Must pin locally packed core/auth packages through root pnpm overrides and commit archives, provenance, and lockfile together.** `vendor/build.json` records the source commit, archive hashes, and `dirty` — true for every `--working-tree` build, which production preflight refuses. No runtime import depends on a sibling checkout. The auth migrations remain owned by the package. +**Must pin locally packed core/auth packages through root pnpm overrides and commit archives, provenance, and lockfile together.** `vendor/build.json` records the source commit, archive hashes, and `dirty` — true for every `--working-tree` build, which production preflight refuses; each archive's `package/dist/provenance.json` names its source commit, and a pack pgstencil marked dirty cannot be vendored. No runtime import depends on a sibling checkout. The auth migrations remain owned by the package. **Must declare every peer dependency of the pinned archives in `hosted/package.json`**, so they share Hosted's copy and Renovate updates them. @@ -65,7 +65,7 @@ Source of truth: `touchesHosted` in `hosted/scripts/changed.mjs`; `.github/workf ## Production releases -**Must deploy only manually selected main revisions after Hosted tests/build and accepted clean package provenance.** `verifyPackages` checks the archive hashes; preflight checks uncached Hyperdrive, matching migration/runtime database identity with distinct roles, and required Worker secret names. Back up, encrypt, decrypt, and restore-test before applying migrations; upload only the encrypted archive. Production has no public candidate URL. +**Must deploy only manually selected main revisions after Hosted tests/build and accepted clean package provenance.** `verifyPackages` checks the archive hashes and each archive's own packed provenance; preflight checks uncached Hyperdrive, matching migration/runtime database identity with distinct roles, and required Worker secret names. Back up, encrypt, decrypt, and restore-test before applying migrations; upload only the encrypted archive. Production has no public candidate URL. **Must record an immutable annotated hosted/YYYY-MM-DD tag only after live verification.** Tags identify the deployed commit and verification run/attempt; retries are idempotent and redeployments get new tags. Dating and repeat-deployment suffixes: `recordDeployment`. Code rollback never reverses migrations. diff --git a/docs/specs/security-hosted.md b/docs/specs/security-hosted.md index 375d7edc2..99f106ffb 100644 --- a/docs/specs/security-hosted.md +++ b/docs/specs/security-hosted.md @@ -23,8 +23,11 @@ Pinned by `hosted/server/tests/workers.test.ts` and `hosted/server/tests/policy. ## Deployment boundary +**Must vendor pgstencil from a commit on its `main`.** A Dormouse branch may vendor a pgstencil branch while a cross-repo change is in flight; `main` must not merge it until pgstencil has. + - **FAIL IF** a production Worker exposes the captured-email inbox or deterministic clock controls, or imports the testing injection module; inspect `hosted/server/worker.ts`, the build configuration, and `hosted/server/tests/worker-entry.ts`. -- **FAIL IF** an archive's SHA-256 differs from `vendor/build.json`, the core/auth pnpm overrides cease resolving to those archives, or a runtime import depends on a sibling source checkout. +- **FAIL IF** an archive's SHA-256 differs from `vendor/build.json`, `build.json` records `dirty`, either archive's `package/dist/provenance.json` is missing, records `dirty`, or names a commit other than `build.json`'s, the core/auth pnpm overrides cease resolving to those archives, or a runtime import depends on a sibling source checkout. +- **FAIL IF** `vendor/build.json`'s commit is not on pgstencil `main` (`gh api repos/diffplug/pgstencil/compare/...main`, status `ahead` or `identical`), or that commit's `security-audit` check run (`gh api repos/diffplug/pgstencil/commits//check-runs`) is missing or not `success`. pgstencil's own audit is the evidence for the packed code; Dormouse audits only how Hosted configures it. - **FAIL IF** the local email inbox accepts a foreign Host or Origin or cross-site Fetch Metadata; inspect `allowedDevRequest` in `hosted/server/dev-host-guard.ts`, including the upgrade guard in `hosted/server/dev.ts`. - **FAIL IF** preview mail or OAuth calls reach external providers, preview configuration copies production routes/bindings, or a preview exposes deterministic time controls; inspect `hosted/server/preview-worker.ts`, `hosted/scripts/preview.mjs`, and `hosted/server/tests/workers.test.ts`. diff --git a/docs/specs/security.md b/docs/specs/security.md index a880ed6a0..8346fb8e2 100644 --- a/docs/specs/security.md +++ b/docs/specs/security.md @@ -142,13 +142,14 @@ lints can only read. `.github/workflows/security-audit.yaml` audits the repository against these specs. Three subagents, each owning the specs below, run every `FAIL IF` as a mechanical check with evidence, then read their domain adversarially for what -no check names. A failure, or a run that reaches no verdict, files a public +no check names. A failure, or a run reaching no verdict, files a public issue labeled [`security-audit-failure`](https://github.com/diffplug/dormouse/issues?q=is%3Aissue+label%3Asecurity-audit-failure) and holds the release; a later pass closes it. Open issues are live; closed -ones are the record of what tripped and what changed. +ones record what tripped and changed. `scripts/security-audit-local.sh` runs the same prompts locally. -[security-audit.md](./security-audit.md) is the contract. +[security-audit.md](./security-audit.md) is the contract. pgstencil audits the +packages Hosted consumes in its own repository; these checks verify that link. | Domain | Specs | Covers | | --- | --- | --- | diff --git a/hosted/README.md b/hosted/README.md index 8f70818c4..0c9d049cd 100644 --- a/hosted/README.md +++ b/hosted/README.md @@ -43,13 +43,19 @@ node scripts/sync-pgstencil.mjs /path/to/pgstencil [revision] This checks out the pgstencil revision (default `HEAD`) in a temporary clean worktree, runs `pnpm packages:pack` there, vendors core/auth, records the commit, -`dirty: false` and SHA-256 hashes in `vendor/build.json`, and installs. To try -uncommitted pgstencil changes, pass `--working-tree` instead of a revision; it -packs the checkout as it stands against its local install and always records -`dirty: true`, which production preflight rejects. The direct Node command also works before the -archives exist (pnpm may otherwise auto-install first). See -`docs/specs/hosted.md` -> "Application boundary" for what has to be committed -together. +`dirty: false` and SHA-256 hashes in `vendor/build.json`, and installs. Each +archive carries its own `package/dist/provenance.json`, and the sync refuses one +that is missing it, that reports a dirty pack, or that names a commit other than +the one packed here. To try uncommitted pgstencil changes, pass `--working-tree` +instead of a revision; it packs the checkout as it stands against its local +install and always records `dirty: true`, which production preflight rejects. +The sync also warns when the vendored commit is not on pgstencil's +`origin/main`: a Dormouse branch may vendor a pgstencil branch while a +cross-repo change is in flight, but Dormouse `main` must vendor a pgstencil +`main` commit, and the nightly audit fails until it does. The direct Node +command also works before the archives exist (pnpm may otherwise auto-install +first). See `docs/specs/hosted.md` -> "Application boundary" for what has to be +committed together. Re-run integration tests after every refresh. Vendor an accepted pgstencil revision before a production release. diff --git a/hosted/scripts/production.mjs b/hosted/scripts/production.mjs index ca549553e..732d7e06a 100644 --- a/hosted/scripts/production.mjs +++ b/hosted/scripts/production.mjs @@ -34,12 +34,12 @@ export async function verifyPackages() { const manifest = JSON.parse( await readFile(new URL("../../vendor/build.json", import.meta.url), "utf8"), ); + assert.match(manifest.commit, /^[a-f0-9]{40}$/); assert.equal( manifest.dirty, false, - "Production requires accepted, clean pgstencil provenance; refresh the vendored packages first", + "Production requires a sync of a committed pgstencil revision, not --working-tree; refresh the vendored packages first", ); - assert.match(manifest.commit, /^[a-f0-9]{40}$/); assert.deepEqual( manifest.files.map((entry) => entry.filename).sort(), ["pgstencil-0.1.0.tgz", "pgstencil-auth-0.1.0.tgz"], @@ -47,14 +47,37 @@ export async function verifyPackages() { ); for (const entry of manifest.files) { assert.match(entry.filename, /^pgstencil(?:-auth)?-[\w.-]+\.tgz$/); - const bytes = await readFile( - new URL(`../../vendor/${entry.filename}`, import.meta.url), - ); + const archive = new URL(`../../vendor/${entry.filename}`, import.meta.url); assert.equal( - createHash("sha256").update(bytes).digest("hex"), + createHash("sha256") + .update(await readFile(archive)) + .digest("hex"), entry.sha256, "Vendored archive checksum mismatch", ); + // The archive's own provenance is what pgstencil's audit is keyed to — + // docs/specs/security-hosted.md -> "Deployment boundary". + const packed = spawnSync( + "tar", + ["-xOf", fileURLToPath(archive), "package/dist/provenance.json"], + { encoding: "utf8" }, + ); + assert.equal( + packed.status, + 0, + `${entry.filename} carries no package/dist/provenance.json; refresh the vendored packages first`, + ); + const provenance = JSON.parse(packed.stdout); + assert.notEqual( + provenance.dirty, + true, + "Production requires accepted, clean pgstencil provenance; refresh the vendored packages first", + ); + assert.equal( + provenance.commit, + manifest.commit, + `${entry.filename} was packed from a commit other than the one vendor/build.json records`, + ); } } export async function preflight(env, config, api = cloudflare(env)) { diff --git a/hosted/server/tests/artifacts.test.ts b/hosted/server/tests/artifacts.test.ts index 440063272..1a814f7c7 100644 --- a/hosted/server/tests/artifacts.test.ts +++ b/hosted/server/tests/artifacts.test.ts @@ -5,23 +5,36 @@ import { execFileSync } from "node:child_process"; import { fileURLToPath } from "node:url"; test("consumed package bytes match the recorded source snapshot", () => { - const provenance = JSON.parse( - readFileSync("../vendor/build.json", "utf8"), - ) as { + const build = JSON.parse(readFileSync("../vendor/build.json", "utf8")) as { commit: string; + dirty: boolean; files: { filename: string; sha256: string }[]; }; - expect(provenance.commit).toMatch(/^[a-f0-9]{40}$/); - expect(provenance.files.map((file) => file.filename).sort()).toEqual([ + expect(build.commit).toMatch(/^[a-f0-9]{40}$/); + // A --working-tree sync records dirty: true; production preflight refuses it. + expect(build.dirty).toBe(false); + expect(build.files.map((file) => file.filename).sort()).toEqual([ "pgstencil-0.1.0.tgz", "pgstencil-auth-0.1.0.tgz", ]); - for (const file of provenance.files) + for (const file of build.files) { expect( createHash("sha256") .update(readFileSync("../vendor/" + file.filename)) .digest("hex"), ).toBe(file.sha256); + // Each archive names the pgstencil commit it was packed from, so the bytes + // themselves — not just build.json — say what pgstencil's own audit covers. + const provenance = JSON.parse( + execFileSync("tar", [ + "-xOf", + "../vendor/" + file.filename, + "package/dist/provenance.json", + ]).toString(), + ) as { commit: string; dirty?: boolean }; + expect(provenance.commit).toBe(build.commit); + expect(provenance.dirty).not.toBe(true); + } // Same-version tarball refreshes must update installed code as well as metadata. for (const [archive, entry] of [ ["pgstencil-0.1.0.tgz", "pgstencil"], diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 1d9fd670d..36b8a96e9 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -2386,7 +2386,7 @@ packages: os: [win32] '@pgstencil/auth@file:vendor/pgstencil-auth-0.1.0.tgz': - resolution: {integrity: sha512-gfnfb5HOlYS7anaPCD3890fLvlloczS7gvb3+Fh6mXldrYNoK0HjtDVit68pQb9kbPivyUSduuoKS/2jAoeHgw==, tarball: file:vendor/pgstencil-auth-0.1.0.tgz} + resolution: {integrity: sha512-LjVC3COsHxwUaR9CEx06vqSSZk5+YPXzoho1whqm/IWcyHBmLBu+LW0p7eh2Rb1n3OomUTOQhP0g/FvAMbxoMA==, tarball: file:vendor/pgstencil-auth-0.1.0.tgz} version: 0.1.0 engines: {node: '>=24'} peerDependencies: @@ -5204,7 +5204,7 @@ packages: resolution: {integrity: sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==} pgstencil@file:vendor/pgstencil-0.1.0.tgz: - resolution: {integrity: sha512-tJncl/ELHO09guw0Mm1kBDC9de8gNOMQLv96MLCoWqDFW7oyHyfNxO/2A8Jy7EeVh/K1PELawu507Ohs3WSXMQ==, tarball: file:vendor/pgstencil-0.1.0.tgz} + resolution: {integrity: sha512-CbfIuJbdeFbpUeSa0sFAVHEHoNQoh0ACqaorTRtf/fkXrLJK7elR5C7DAZHsZtU0WcXa6EDnRpECFUOF3vCClg==, tarball: file:vendor/pgstencil-0.1.0.tgz} version: 0.1.0 engines: {node: '>=24'} peerDependencies: diff --git a/scripts/spec-word-budgets.json b/scripts/spec-word-budgets.json index ff885e533..85c550cbb 100644 --- a/scripts/spec-word-budgets.json +++ b/scripts/spec-word-budgets.json @@ -20,7 +20,7 @@ "docs/specs/remote-security-model.md": 4750, "docs/specs/security-audit.md": 1950, "docs/specs/security-ci.md": 2700, - "docs/specs/security-hosted.md": 450, + "docs/specs/security-hosted.md": 550, "docs/specs/security-local.md": 3150, "docs/specs/security-remote.md": 5750, "docs/specs/security-supply-chain.md": 1150, diff --git a/scripts/sync-pgstencil.mjs b/scripts/sync-pgstencil.mjs index c99642d2d..2a56ca1da 100644 --- a/scripts/sync-pgstencil.mjs +++ b/scripts/sync-pgstencil.mjs @@ -33,6 +33,40 @@ const run = (command, args, cwd, env = process.env) => execFileSync(command, args, { cwd, stdio: "inherit", env }); const git = (...args) => execFileSync("git", args, { cwd: repository, encoding: "utf8" }).trim(); +const succeeds = (attempt) => { + try { + attempt(); + return true; + } catch { + return false; + } +}; +// Every pgstencil pack writes package/dist/provenance.json: the archive's own +// claim of the commit it came from, plus `dirty` when it was packed +// --allow-dirty. Dormouse verifies that claim instead of auditing the packed +// code, which pgstencil audits in its own repository — +// docs/specs/security-hosted.md -> "Deployment boundary". +const provenanceOf = (archive) => { + let raw; + try { + raw = execFileSync( + "tar", + ["-xOf", archive, "package/dist/provenance.json"], + { encoding: "utf8" }, + ); + } catch { + throw new Error(`${archive} carries no package/dist/provenance.json`); + } + let claim; + try { + claim = JSON.parse(raw); + } catch { + throw new Error(`${archive}: package/dist/provenance.json is not JSON`); + } + if (typeof claim?.commit !== "string" || !/^[0-9a-f]{40}$/.test(claim.commit)) + throw new Error(`${archive}: provenance.json names no commit`); + return claim; +}; const manifest = JSON.parse( readFileSync(resolve(root, "hosted/package.json"), "utf8"), ); @@ -56,6 +90,9 @@ const overrides = new Map( // --working-tree packs the checkout as it stands, for trying unfinished // pgstencil changes. That result depends on local state even when git status is // clean, so it is always recorded dirty and production preflight refuses it. +// Either way `commit` is the HEAD of the checkout that was packed, so it is +// what each archive's own provenance must name; a pack that reports itself +// dirty (pgstencil's --allow-dirty) is refused below rather than vendored. const commit = git( "rev-parse", "--verify", @@ -92,11 +129,22 @@ try { run("pnpm", ["install", "--frozen-lockfile"], checkout, pgstencil); run("pnpm", ["packages:pack"], checkout, pgstencil); mkdirSync(resolve(root, "vendor"), { recursive: true }); - for (const filename of archives) - copyFileSync( - resolve(checkout, "dist/packages", filename), - resolve(root, "vendor", filename), - ); + for (const filename of archives) { + const target = resolve(root, "vendor", filename); + copyFileSync(resolve(checkout, "dist/packages", filename), target); + // Both archives are held to the commit packed here, so a stale pack — + // or two archives disagreeing with each other — stops the sync before + // build.json can record a commit the bytes do not carry. + const claim = provenanceOf(target); + if (claim.dirty === true) + throw new Error( + `${filename} was packed --allow-dirty; a dirty pack cannot be vendored`, + ); + if (claim.commit !== commit) + throw new Error( + `${filename} was packed from ${claim.commit}, not the ${commit} packed here`, + ); + } } finally { if (!workingTree) git("worktree", "remove", "--force", checkout); } @@ -117,3 +165,17 @@ run( ["--filter", "dormouse-hosted", "update", "pgstencil", "@pgstencil/auth"], root, ); +// Dormouse `main` must vendor a pgstencil `main` commit, because the nightly +// audit reads that commit's pgstencil `security-audit` check run — but a +// Dormouse branch may vendor a pgstencil branch while a cross-repo change is +// in flight, so this warns and does not fail. +const fetched = succeeds(() => git("fetch", "origin", "main")); +const onMain = + fetched && + succeeds(() => git("merge-base", "--is-ancestor", commit, "origin/main")); +if (!onMain) + console.warn( + fetched + ? `warning: pgstencil ${commit} is not on origin/main — Dormouse main must vendor a pgstencil main commit, and the nightly audit fails until it does` + : `warning: could not fetch pgstencil origin/main to check ${commit} — Dormouse main must vendor a pgstencil main commit`, + ); diff --git a/vendor/build.json b/vendor/build.json index 60423f2fa..8bbf64271 100644 --- a/vendor/build.json +++ b/vendor/build.json @@ -1,14 +1,14 @@ { - "commit": "297edf6590e61200857b199d69160f0567bbb3c8", + "commit": "ddbf96c828c3e6a1129ff7601afeee2e47f80172", "dirty": false, "files": [ { "filename": "pgstencil-0.1.0.tgz", - "sha256": "5ed674d0c62619a2eaa09c40ea447b730670b92ba3e32ae784755b70ed8f2c7e" + "sha256": "f9210dfabe12719c0907e797e3bf5e3cbbcd5b9825f5a53343d493bb1aae895f" }, { "filename": "pgstencil-auth-0.1.0.tgz", - "sha256": "7ada0c3d2031b3d06bec00c623c4f339d4045fd63b1f7a7fe371bd40ca7b7baa" + "sha256": "4671664cb76ec8cbfcd9d19e1d40774971aef58a8b05aaa9af48c3e741805b93" } ] } diff --git a/vendor/pgstencil-0.1.0.tgz b/vendor/pgstencil-0.1.0.tgz index 6bd1f73e5..43e403e03 100644 Binary files a/vendor/pgstencil-0.1.0.tgz and b/vendor/pgstencil-0.1.0.tgz differ diff --git a/vendor/pgstencil-auth-0.1.0.tgz b/vendor/pgstencil-auth-0.1.0.tgz index 0ce135362..93a91f97d 100644 Binary files a/vendor/pgstencil-auth-0.1.0.tgz and b/vendor/pgstencil-auth-0.1.0.tgz differ