[spec-review] Update Safe Outputs conformance checker for recent spec changes

[github-actions]

Summary

Updates scripts/check-safe-outputs-conformance.sh to match the Safe Outputs specification v1.29.13. The script previously matched v1.29.8. This PR adds 6 checks for new normative requirements and fixes 1 false positive caused by a spec wording change.

Specification Changes Reviewed (last 7 days)

• 020a1c6 Recognize merged base ancestors when the base tip advances (Recognize merged base ancestors when the base tip advances #69998) — v1.29.13, BGO-001–008, allow-full-history-fetch
• 0389684 Bind safe-output patch and bundle application to validated bytes (Bind safe-output patch and bundle application to validated bytes #68732) — PTI-001–007, dash-only branch rejection
• 5a79aa3 Require explicit multi: true for multi-repository checkout (Require explicit multi: true for multi-repository checkout #68816) — v1.29.12, RCR8 scoped to literal multi: true
• 7f88694 Add allowlisted dynamic repository checkout lists (Add allowlisted dynamic repository checkout lists #68593) — v1.29.10/11, RCR8–RCR15
• 53db138 Make add-comment link limits configurable and exclude footer links (Make add-comment link limits configurable and exclude footer links #68723) — max-links
• 845b2e8 Keep signed-delivery rebase attached for git push fallback (Keep signed-delivery rebase attached for git push fallback #68052) — v1.29.9, SCD-001–003
• ff43e6b Add in-place replacement for add-comment status updates (Add in-place replacement for add-comment status updates #68013) — replace-older-comments

📋 Script Updates & Testing Details

Script Updates

New Checks Added

• TYPE-016: add-comment.max-links. Checks that the limit is a positive integer in the schema, that it is enforced both when the tool is called (MCP server) and when the comment is posted (handler), that only agent-written content is counted (footer and metadata links are excluded), and that E008 reports the configured value.
• TYPE-017: replace-older-comments. Checks that it cannot be combined with hide-older-comments (handler and schema), that only comments by the authenticated identity are matched (bot-only when the identity is unknown), that only issue/PR comments with a caller or workflow marker are eligible, and that staged mode returns before any lookup or update.
• SCD-001 (SCD-001–003): Checks that the stale-commit rebase does not pass an explicit HEAD argument, which would detach HEAD. Also checks that allowGitPushFallback: false refuses an unsigned push, and that the real-git test evidence exists.
• PTI-001 (PTI-001–007): Checks that both PR handlers use transport_integrity.cjs and resolve_transport_paths.cjs. Also checks for: files opened with O_NOFOLLOW/O_NONBLOCK, regular-file checks, a sha256 digest plus a dev/inode identity check, private read-only (0o400) snapshots, the stable ERR_TRANSPORT_INTEGRITY code, retries rejected before custom classifiers run, and snapshot disposal.
• RCR-001 (RCR8, RCR9, RCR12): Checks that checkout.multi must be a literal boolean, that single mode allows only one repository, that max-repos defaults to 10 and is enforced, that private/internal targets are denied by default behind a literal allow-private-repos, that a public host can never check out a private target, and that runtime lists hard-code lfs/submodules to false and use --no-recurse-submodules.
• BGO-001 (BGO-001, 002, 004, 005): Checks that bundle inspection runs offline (GIT_ALLOW_PROTOCOL=\"\" and GIT_NO_LAZY_FETCH=1). Checks that allow-full-history-fetch is a literal opt-in with a boolean, default-false schema for both handlers, scoped to each handler, and warns that downloads have no byte limit. Checks that allow-merged-base-baseline is a literal opt-in, and that discovery is bounded: 5s timeout, 1 MiB output, 100 records with a 101st record to detect overflow, at most 4 REST comparisons, retries disabled, and an abort signal.

Checks Modified

• MCE-001: The add_comment tool description now says the link limit is the configured max-links (default: 50) instead of a fixed "50 links". The check now accepts either wording, which fixes a MEDIUM false positive.
• Header: the spec and script versions were bumped to 1.29.13.

Checks Removed

• None. No requirements were removed.

Not automated (left to the runtime and TLA+ evidence the spec cites)

• RCR10/11/13–15: job-local manifests, credential precedence, finalization, failure reporting, staged isolation.
• BGO-003/006–008: credential roles, conservative fallback, protection provenance, safe diagnostics.
• PTI-003/004: cost bound and trusted transformations.

Grep checks cannot reliably verify these behavioral properties.

Testing

$ bash -n scripts/check-safe-outputs-conformance.sh   # OK
$ bash scripts/check-safe-outputs-conformance.sh
[PASS] MCE-001: Tool descriptions properly disclose enforcement constraints
[PASS] TYPE-016: add_comment max-links is configurable, excludes generated links, and is enforced at invocation and posting time
[PASS] TYPE-017: replace-older-comments is exclusive with hide-older-comments, identity/marker scoped, and skipped in staged mode
[PASS] SCD-001: Signed delivery rebases keep HEAD attached and honor allowGitPushFallback: false
[PASS] PTI-001: Patch/bundle transport binds captured bytes, uses private snapshots, and fails permanently on integrity errors
[PASS] RCR-001: Runtime repository-list checkouts enforce explicit multi mode, bounded preflight, private opt-in, and no secondary fetches
[PASS] BGO-001: Bundle Git operations are offline by default, recovery is literal opt-in, and baseline discovery is bounded
Critical: 0  High: 0  Medium: 0 (was 1)  Low: 3 (unchanged; pre-existing USE-001/USE-003)
PASS: All checks passed (exit 0)

Negative test: temporarily setting GIT_NO_LAZY_FETCH: \"0\", loosening the allowFullHistoryFetch !== true gate, and hard-coding the E008 maximum produced the expected BGO-001/BGO-002 CRITICAL and TYPE-016 MEDIUM failures. The files were restored afterwards.

Related Files

• Specification: docs/src/content/docs/specs/safe-outputs-specification.md
• Conformance Script: scripts/check-safe-outputs-conformance.sh

Generated by 🔒 Weekly Safe Outputs Specification Review · pi · opus55 · 102.4 AIC · ⌖ 40.8 AIC · ⊞ 1.4K · ◷

• expires on Oct 25, 2026, 8:41 PM UTC-08:00