Skip to content

publish, publish_attestation and GitHubPagesHost: one Git Data API commit to a publish-mode host (typedstandards#141 P2) - #8

Merged
npstorey merged 10 commits into
mainfrom
ts141/p2-publish
Oct 9, 2026
Merged

npstorey merged 10 commits into
mainfrom
ts141/p2-publish

Conversation

@npstorey

@npstorey npstorey commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

P2 of the notebook-publish sprint (npstorey/typedstandards#141; G0 record: its comment 6027279902).

A notebook author publishes a signed record to a GitHub Pages host made from the host template's publish mode, with one call. ts.GitHubPagesHost("owner/repo"), ts.publish(signed, host=…, title=…, notebook=… or name=…, role=…, revises=…) and ts.publish_attestation(node, host=…, name=…) each write one Git Data API commit: blobs, a tree on base_tree, a commit, and a fast-forward of the branch. Every refusal happens before any write. The token reaches no output. The README documents all of it, including the seed's custody in a hosted notebook. There is no release in this PR: the version stays 0.1.1, and the changes are under ## Unreleased.

Live smoke done (G3, 2026-10-08 at 576c637; again 2026-10-09 at 9e24876): see "Live smoke" and "Fix after the seat's review" below.

Branch and blast zone

ts141/p2-publish, head 9e2487694d2c775cac363ebc4fd583b102fa2363, on 6c0c20c. There are ten commits, each SSH-signed with a sign-off. The last two, 3e10022 (red) and 9e24876 (fix), answer the seat's review of 576c637: see "Fix after the seat's review" below. The numstat that follows is at 576c637.

15	0	CHANGELOG.md
160	3	README.md
192	0	scripts/github_stub.py
100	0	scripts/smoke_publish.py
32	3	scripts/smoke_wheel.py
9	0	src/typedstandards/__init__.py
749	0	src/typedstandards/_publish.py
13	0	src/typedstandards/errors.py
7	0	tests/conftest.py
8	1	tests/fixtures/README.md
23	0	tests/fixtures/template-host-policy.json
20	0	tests/fixtures/template-host.json
116	0	tests/publish_support.py
575	0	tests/test_publish.py
338	0	tests/test_publish_token.py
64	0	tests/test_readme.py
69	0	tests/test_smoke_publish.py

These are unchanged, as git diff origin/main...HEAD over them is empty: tests/guards.py, tests/test_guards.py, .gitleaks.toml, package.json, package-lock.json, pyproject.toml, .github/, the CLI pin, the Node floor and the version. Nothing under src/ imports hashlib, names the seed variable, passes env= or changes os.environ.

Acceptance (#141 P2, with G0-3 and G0-4)

Red at 130585a, over a typed stub: 89 failed, 249 passed. Each new test fails at its call (NotImplementedError) or at the missing README section. Green at head: 346 passed.

  1. httpx.MockTransport paths (tests/test_publish.py, tests/test_publish_token.py):
    • A new record: two blobs, one tree, one commit and one fast-forward. The test asserts the exact (method, path) sequence.
    • A listed hash: no write, written: False.
    • A listed name with another hash:
      • refused without revises=;
      • written with it (G0-4): under <name>-<first 8 hex of its envelopeHash>, with the revises node on the target's entry, in one commit.
    • Names: a records or evidence segment is refused (4 cases); a name failing host-core's rule is refused (10 cases).
    • Other refusals: a BlobRef output; a signer other than the policy's; a role no active rule admits (G0-3); an empty title.
    • Non-fast-forward: the call re-reads the head (a write that errored may have landed), retries once, and then errors.
    • Token shape: a token that does not start github_pat_, or holds whitespace, quotes or op://, is refused, and the message does not contain the value (10 cases).
    • No write before a refusal: every refusal test asserts that no POST, PATCH, PUT or DELETE reached the transport.
    • A host in its starting state: "records": [] and no records/ directory (the template's setup, step 5); the first entry is appended.
  2. The token scanner.
    • Across 16 publish paths, the token's value is in no captured stdout, stderr, DEBUG log record, warning, return value repr, exception str or repr, or traceback, including a traceback with frame locals.
    • test_the_scanner_fails_on_an_offender drives it over a path that leaks: red at 130585a, green with 6 cases.
  3. Git Data API only.
    • The write tests assert only GET, POST and PATCH, no write to /contents/, and one commit per ref update.
    • _Api.write refuses anything but POST or PATCH under /git/.
    • The two /contents/ lines in src/ are GETs.
  4. The token's source.
    • The token comes from token=, else TYPEDSTANDARDS_GITHUB_TOKEN, read when the call runs.
    • No file is opened or written during a publish.
    • repr(GitHubPagesHost(...)) shows the repository and branch only, and the host neither pickles nor exposes vars().
  5. README and CHANGELOG. tests/test_readme.py checks each of these:
    • the one line that sets TYPEDSTANDARDS_SIGNING_SEED_B64 from a hosted platform's secret store;
    • the unsafe forms;
    • the custody sentence, verbatim: "In a hosted notebook, the hosting service's runtime holds the seed for as long as the kernel runs.";
    • the Codespaces line;
    • the calls and the receipt's keys;
    • the default name (<stem>/<date>-<8 hex>, plan §7), which comes before the derived-name rule, per the seat's note on G0-4;
    • reading the author's did:key from the first signed record, before the first publish.
      The Codespaces line rests on GitHub's documentation, read 2026-10-06 (the account-specific Codespaces secrets page; the Actions secrets and variables concept pages). That a Jupyter kernel started in a codespace inherits the variable was not measured.
  6. CI's checks, run locally.
    • Re-run by ORCH at 576c637: 346 passed on Python 3.11, 3.12 and 3.14 with Node v24.21.0; ruff check: All checks passed!; ruff format --check: 46 files already formatted.
    • The implementing session also ran Python 3.12 with Node v22.23.1 (346 passed), and the wheel job from a fresh clone: sdist then wheel, installed into a fresh environment, scripts/smoke_wheel.py exit 0. Its new line reads: publish: … in one commit, then a no-op, then a withdrawal, over MockTransport.
    • All runs were on macOS arm64. This PR's checks cover Ubuntu.
  7. scripts/smoke_publish.py (G3). It signs one record with the installed wheel and publishes it to the scratch copy. It prints only the receipt's fields. It exits 2 on a refusal, 3 on an API error, 4 on a CLI error and 5 on an origin mismatch. Three tests run it over the stub API. It has not been run live.

gitleaks over origin/main..HEAD: no leaks found. An offline emulation of the push guard's keyword scan (digests masked, pushguard.allow applied) gave 0 hits over the range.

Fixture provenance

Fixture Source Read at SHA-256 Assertion
tests/fixtures/template-host.json the template's host.json 70bfd18 8a80c9ea344196ac2b27b3a91eb3439f9217e72c5491b18540d253925b28a6fe byte-equal (test_template_fixture_is_the_verbatim_copy)
tests/fixtures/template-host-policy.json the template's host-policy.json 70bfd18 9150db40fee1f2e17bf09d128c080f0bee9a3ac0c8bda9dbd0467e11a144c0f8 byte-equal (same test)

Signed documents and nodes are made in each test session by the vendored CLI 0.2.0, under fresh seeds, and are not committed.

Deviations and premises that did not hold as written

  • The default name's stem is an argument, notebook=. The signed document carries no file name. Only the stem is used, and the file is not read. The date is createdAt's, in UTC.
  • trust_env=False on publish's own client. httpx's default reads the whole environment, which holds the seed, when a client is built (measured). A caller who needs a proxy or a certificate bundle passes client=. pin still uses the default; that is out of scope here.
  • Traceback frame locals. A verbose notebook traceback (%xmode Verbose) prints frame locals. The refusal path no longer holds the token as a local; tested red, then green.
  • A repeat is a no-op under any name. Corrected at 9e24876; see "Fix after the seat's review".
  • Added beside G0-3: a record whose type the policy's top-level type does not name is refused, for the same reason as the role check: it would fail every later build.

Flags, not fixed

  • httpx reads SSLKEYLOGFILE even with trust_env=False. pin has the same exposure.
  • The README links the template README's "Publishing from a notebook" section, which resolves once the template's P1 merges.

Live smoke (G3), 2026-10-08

The wheel was built at this PR's head (576c637) from a clean detached worktree, and installed into a fresh Python 3.12 environment. The owner ran the smoke from that environment, against the scratch copy npstorey/typedstandards-publish-check. The token and the copy's kept seed reached the process only through op run.

typedstandards 0.1.1; publishing one record to npstorey/typedstandards-publish-check (main)
signed c18084e65a8b5c2a20687f819d438fd42eb40a656b74a8d073cf05865a53b299 by did:key:z6MkfvR4js3VSKzmGGw5Nk19FVymUGvG5eyBqvn4xWsQS3su
name: publish-check/2026-10-08-c18084e6
written: True
commit: b9c0c2beeaaaacc0f6fc250a99768ee5d3cf696a
bundle_url: https://publish-check.typedstandards.org/bundles/publish-check/2026-10-08-c18084e6.bundle.json
verify_url: https://typedstandards.org/verify?url=https%3A%2F%2Fpublish-check.typedstandards.org%2Fbundles%2Fpublish-check%2F2026-10-08-c18084e6.bundle.json
registry_url: https://publish-check.typedstandards.org/.well-known/typed-publisher.json
run: None
publish check passed: the host's workflow now builds and deploys the commit
exit 0

Read back by ORCH:

  • The commit. GET commits/b9c0c2b: one commit, parent f166c95 (the copy's previous head, so a fast-forward), message Publish publish-check/2026-10-08-c18084e6. Files: host.json modified, records/publish-check/2026-10-08-c18084e6.signed.json added. verification: unsigned, as #139 F7 found for API commits. Committed at 12:24:45Z. The rules on main are deletion and non_fast_forward, and they let the commit through.
  • The token. It was made at G1 with Contents read and write and nothing else, as the owner set it; its permissions cannot be read from here. #139 M4 had measured a write only with a token that also held Actions read.
  • The deploy. The publish run (37776666814) went from 12:24:48Z to 12:25:15Z: success. check was skipped.
  • What is served, first hop, at 12:25:21Z, 36 s after the commit:
    /bundles/publish-check/2026-10-08-c18084e6.bundle.json first-hop: 200 location= type=application/json; charset=utf-8 acao=* etag="6ac78ba4-f25" age=0
    /records.json first-hop: 200 location= type=application/json; charset=utf-8 acao=* etag="6ac78ba4-2dc" age=0
    /.well-known/typed-publisher.json first-hop: 200 location= type=application/json; charset=utf-8 acao=* etag="6ac78ba4-469" age=0
    
  • The index. records.json: {"name":"publish-check/2026-10-08-c18084e6","status":"active","extensions":{"role":"notebook"}}, its only record.
  • The default name. It is <stem>/<date>-<8 hex>: the stem from notebook=, the date from createdAt (UTC), and the first eight hex of the envelopeHash.

Fix after the seat's review (9e24876)

The defect at 576c637. "The same signed document under two explicit names becomes two entries" is refused by host-core 0.1.1, the version the template pins. At typedstandards 116882a, packages/host-core/src/build.ts:65-66 throws … is the record <other> already serves (envelopeHash …). So a publish under another name fast-forwarded main to a manifest that every later build refuses. That broke D8 A and the acceptance line "a listed hash: no write, written: False".

The fix. Each call reads every listed record's signed file, once (one GET each). A record whose envelopeHash any entry carries is not written: written: False, and the receipt names that entry. This holds under name=, under notebook=, and for the revises= derived name. README:273 is corrected, and the README states the per-call cost.

Red at 3e10022, green at 9e24876. ORCH reproduced the three reds in a detached worktree:

E       AssertionError: assert [('POST', '/r.../heads/main')] == []
FAILED tests/test_publish.py::test_a_hash_listed_under_another_name_writes_nothing
FAILED tests/test_publish.py::test_a_hash_listed_under_another_default_name_writes_nothing
FAILED tests/test_publish.py::test_a_revision_whose_hash_is_listed_writes_nothing
3 failed, 80 deselected

At 9e24876, ORCH ran 368 passed on Python 3.11, 3.12 and 3.14 with Node 24. ruff is clean, gitleaks finds nothing in origin/main..HEAD, and the guard files are unchanged. The implementing session also ran Python 3.12 with Node 22 (368 passed), and the wheel job from a fresh clone (smoke exit 0).

Earlier tests changed in the fix commit.

  • Three published docs.first again under a new name and expected a write, which is the defect. They now publish another record:
    • test_every_write_is_a_git_data_call (docs.third);
    • test_the_token_comes_from_the_argument_else_the_environment (docs.second and docs.third for "b" and "c");
    • test_revises_whose_fields_do_not_match_is_refused[another target] listed a record by another key, which the new scan refuses first. It now lists a same-signer record.
  • The request-sequence helper paths() now includes the scan's reads.

Every host-core refusal a publish could trigger (at 116882a): each HostError that build reaches was mapped to a refusal before any write, with its test, or argued unreachable. Beyond the duplicate hash, the fix also refuses:

  • a record whose signer, binding tier, display name or signature kid disagrees with the records already listed (build.ts:94, :97);
  • a signed or attestation path another entry already names;
  • a manifest host-core's parseManifest would refuse (the manifest is checked as it would be written);
  • a withdrawal or supersession whose resulting status no policy rule displays (display.ts:142);
  • a signed file whose JSON host-core would not parse.

The full table is in the implementing session's report, held by ORCH.

Flags, not fixed.

  • The template's policy has no superseded rule, so publish_attestation of a supersedes node is refused on a stock copy. P3 publishes none. Whether the template should carry that rule is a later ruling.
  • A withdrawal added after a later node has already set a record's status may be over-refused.
  • A record whose signature does not verify is caught only by the workflow's verify. publish does not run the CLI (plan §9), so only a hand-edited document reaches that.

G3 again, at 9e24876 (2026-10-09). The wheel was rebuilt at this head into a fresh environment, and the owner ran the same command:

signed ef245b02830826c6e5194625bc72049f8343bded04437129614aeb99dcc7d59c by did:key:z6MkfvR4js3VSKzmGGw5Nk19FVymUGvG5eyBqvn4xWsQS3su
name: publish-check/2026-10-09-ef245b02
written: True
commit: 75183d7bab1719d0727bc5c2742ae4e1d3a8c185
publish check passed: the host's workflow now builds and deploys the commit
exit 0

Read back by ORCH:

  • The commit: 75183d7, one commit, parent b9c0c2b (a fast-forward), unsigned. Files: host.json modified, records/publish-check/2026-10-09-ef245b02.signed.json added.
  • The deploy: publish ran 01:03:35Z to 01:04:00Z: success.
  • What is served: the bundle answered 200 application/json; charset=utf-8 with acao=* at 01:04:16Z. records.json lists both records, active, role notebook.
  • The new path: this run read the first record's signed file from the live API before writing, which the scan added.

A correction to the record. The copy had two failed publish runs, not one: 827b628 (the deliberate failing record) and f166c95 (the re-key, an empty manifest refused at build). Both are expected and documented.

Model

Implemented by IMPL P2 on Opus 5.5 at high effort, as the impl agent file pins. Re-verified by ORCH NOTEBOOK-PUBLISH on Opus 5.5.

🤖 Generated with Claude Code

npstorey and others added 8 commits October 6, 2026 19:40
…ped stub

The tests for typedstandards#141 P2's acceptance 1 to 5, against a stub module whose names
raise NotImplementedError, so the suite collects and each new test fails at its call:

- tests/test_publish.py: a new record, a listed hash, a listed name with and without revises=,
  the name rules, a BlobRef output, the policy's signer, role and type, an empty title, the
  non-fast-forward retry, publish_attestation, and the Git Data API as the only writer;
- tests/test_publish_token.py: the token's source and shape, a scanner of every captured output
  driven over each publish path and over offenders, no file opened, and the host's repr;
- tests/test_readme.py: the README's publishing section;
- scripts/github_stub.py: an in-memory GitHub API for httpx.MockTransport;
- tests/fixtures/template-host*.json: the host template's host.json and host-policy.json at
  70bfd18, verbatim and pinned by SHA-256.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
ts.publish(signed, host=, title=, name= or notebook=, role="notebook", revises=None) reads
host.json and host-policy.json at the branch head and writes the signed file and the edited
host.json in one commit: a blob each, a tree on the head's tree, a commit whose parent is the head,
and a fast-forward of the branch. A ref update that does not answer 200 re-reads the head first;
if the commit did not land and the branch moved, the call plans again from the new head once.
ts.publish_attestation(node, host=, name=) adds what withdraw or attest printed to a listed
record's entry the same way.

Refused before any write: a token that is not a fine-grained one or holds whitespace, a quote or
op://, a name failing host-core's rule or with a records or evidence segment, an empty title, a
BlobRef output, a signer or type the policy does not name, a role no active rule admits
(typedstandards#141 G0-3), and a listed name with another record unless revises= matches by type,
successorNodeId and targetNodeId (G0-4). A listed hash is a no-op. The receipt is
{name, commit, bundle_url, verify_url, registry_url, written, run: None}.

The module computes no hash (blob ids come back from the API), runs no CLI and reads no seed. The
token comes from token=, else TYPEDSTANDARDS_GITHUB_TOKEN when a publish runs, and is held only in
the client's Authorization header. scripts/smoke_publish.py is the live check for gate G3;
scripts/smoke_wheel.py now also publishes over MockTransport.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
httpx's default trust_env=True iterates os.environ for proxy variables when a client is built,
and the environment of a signing notebook holds the seed. The client publish builds now passes
trust_env=False, so it reads no variable but the token's (and httpx's own SSLKEYLOGFILE) and no
.netrc. A caller who needs a proxy or a certificate bundle passes client=. The new test drives the
live-client path under the guard tests' RecordingEnviron; with trust_env=True it fails on
read_all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
The README's new section documents GitHubPagesHost, publish and publish_attestation, the default
name first and then the derived name a listed name takes with revises=, the refusals, the
receipt's keys (run is None: publish does not wait for the deploy), the token (fine-grained, one
repository, Contents read and write, never a literal in a cell), and the seed in a hosted
notebook: the one line that sets it from the hosting service's secret store, the unsafe forms,
the custody sentence, and GitHub Codespaces. publish and publish_attestation join the calls that
run without Node; a test pins it. CHANGELOG: under Unreleased; the version stays 0.1.1.

The README test's ordering assertion compares the default name with the derived-name rule
rather than with the first "revises=", which the call's signature line carries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
A notebook's verbose traceback (IPython's %xmode Verbose) prints the locals of every frame it
shows. The scanner now also renders each exception with capture_locals=True, and an offender that
raises from a frame holding the header proves it fails there. Against the current module, every
token-shape refusal fails: the value is a local of the frame that raises.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
The token's shape is checked by a helper that returns the refusal's words, and the value is
deleted before the refusal is raised. The request headers are built inside the calls that use
them, so neither the API's constructor nor its request method holds them as a local.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
…did:key read

A publish-mode copy starts with "records": [] and no records/ directory (the template's setup,
step 5). The new test publishes into that state: the first entry is appended to the empty list and
the tree on base_tree creates records/<name>.signed.json. It passes as written: publish already
handles both. The README test fails: the README does not yet show the author how to read their
did:key, which the template's setup puts in host-policy.json, before the first publish.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
…s setup

The template's setup puts the author's did:key in host-policy.json's signer, and publish refuses
any other signer. The README shows reading it from the first signed record
(signed["package"]["signer"]["identifier"]), since the CLI prints a did:key only in what it signs,
and links the template README's "Publishing from a notebook" section rather than restating it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
npstorey and others added 2 commits October 8, 2026 20:25
…eaches

host-core 0.1.1's build refuses one envelopeHash listed twice (build.ts:65-66 at typedstandards
116882a), so publish must not write a record any listed entry already carries, under any name.
Three tests: a hash listed as "a" published as "b"; a hash listed under one default name
published with notebook= of another stem; a revision whose hash is already listed. Each expects
no write and a receipt naming the listed entry; at 576c637 each writes a commit.

The read of every listed record's signed file adds a GET per entry to the expected request
sequences, and a test pins one read per file per call.

From a read of every HostError the build reaches, the throws publish did not yet refuse, a test
each: a signer, display name or key unlike the listed records' (build.ts:93-94); a signature kid
other than the signer (96-97); a signature host-core's checkSignature refuses (records.ts:44-48);
an empty createdAt (build.ts:61-62); with registry null, a signer that is not self-certifying
(147-148, produce-core's view); a signed or node path another entry already names (65-66, 70-71);
a host.json parseManifest refuses (manifest.ts); a listed record whose file cannot be read
(build.ts:49-50); a file host-core's strict UTF-8 JSON.parse refuses (json.ts:31-41); and, for the
workflow's display step, a withdrawal or supersession no policy rule displays. The test host's
first-note file is now a record signed by the test key, as a copy's would be.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
…build would

host-core 0.1.1's build refuses one envelopeHash listed twice (build.ts:65-66), so a record
published under a second name, or under the default name of another stem, fast-forwarded main to
a manifest every later build refuses. publish now reads every listed record's signed file, once
per call, and a record whose hash any entry carries is not written: written is False and the
receipt names that entry, with its bundle_url and verify_url. The revises= target is found in
the same reads.

From a read of every HostError the build reaches at typedstandards 116882a, publish also refuses
before any write: a file host-core's strict UTF-8 JSON.parse refuses; a signature checkSignature
refuses or a kid other than the signer; an empty createdAt or signer.identifier; under a
registry, a signer, display name or key unlike the first record's; with registry null, a signer
that is not a pseudonymous did:key; a path another entry names; a host.json parseManifest
refuses (ported); and a host whose build already fails on a listed record. publish_attestation
refuses a withdrawal or supersession that would leave its record in a status no policy rule
displays, which the template's display step would refuse.

Two earlier tests published one document under two names and expected a second write, the
defect itself; they now publish distinct records. One revises= case lists a third record by the
test key instead of one signed by another key, which the scan now refuses first.
README.md and CHANGELOG.md state the behaviour and its cost: one read per listed record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LJBCtCpvKX2vdg53xqw8ji
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
@npstorey
npstorey merged commit 656f025 into main Oct 9, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant