Skip to content

**Spec**: creative asset host allowlist - #1248

Closed
dhruv8sh wants to merge 5 commits into
mainfrom
spec/1231-creative-asset-host-allowlist
Closed

dhruv8sh wants to merge 5 commits into
mainfrom
spec/1231-creative-asset-host-allowlist

Conversation

@dhruv8sh

@dhruv8sh dhruv8sh commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds the design spec and implementation plan for Add a host allowlist for creative asset rewriting #1231. The issue asks for the spec to be approved in its own PR before any implementation PR is opened.
  • The spec adds a [rewrite] include_domains allowlist for creative asset rewriting. Today every absolute asset URL in a winning creative is proxied, so every third-party asset host receives the EC ID. The allowlist lets operators limit that to hosts they choose. Click-through links are not affected.
  • It settles the open decisions in the issue:
    • Setting: [rewrite] include_domains.
    • to_abs split: to_abs becomes a pure normalizer, normalize_creative_url, and host policy moves to Rewrite::should_proxy_asset and Rewrite::should_wrap_click.
    • /first-party/sign: an off-list host gets a 502, the same as an excluded host, so the browser falls back to a direct load.
    • exclude_domains: matching becomes case-insensitive. This is a behavior change and gets a CHANGELOG note.
    • Coordination with Make creative click rewriting a separate feature with its own switch #1234: both plans contain the same shared normalizer and policy step, so whichever implementation lands first introduces it and the other skips it.
  • The spec also fixes an existing bug: on the inline creative path, <link imagesrcset> was proxied twice.

Changes

File Change
docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md Design spec: problem, goals, design, resolved open decisions, test plan, rollout and rollback
docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md Step-by-step TDD implementation plan, including the shared step with #1234

Closes

Part of #1231. This PR adds only the spec and plan. The implementation PR will link the approved spec and close the issue.

Test plan

  • cargo test-fastly && cargo test-axum
  • cargo clippy-fastly && cargo clippy-axum
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run
  • JS format: cd crates/trusted-server-js/lib && npm run format
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • Other: prettier --config docs/.prettierrc --check on both files. This PR changes no code, so no Rust or JS gates were run. The plan was checked by implementing it in a local branch, where fmt, all clippy aliases, test-fastly, test-fastly-reuse, test-axum, test-cloudflare and test-spin passed. That code is not included here.

Checklist

  • Changes follow AGENTS.md conventions
  • No unwrap() in production code — use expect("should ...")
  • Uses tracing macros (not println!)
  • New code has tests
  • No secrets or credentials committed

Spec and implementation plan for #1231: a [rewrite] include_domains allowlist for creative asset rewriting, case-insensitive exclude_domains matching, and a single rewrite of link imagesrcset on the inline path.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
@dhruv8sh dhruv8sh self-assigned this Oct 7, 2026
@dhruv8sh dhruv8sh changed the title Add design spec for creative asset host allowlist **Spec**: creative asset host allowlist Oct 7, 2026
Describe the review fixes: strict include_domains validation with a separate non-ASCII error, binary-first upgrade and rollback docs, the distinct link imagesrcset candidate, the host-in-both-lists test across every handler, and the shared-step CHANGELOG entry with the sign status change.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
@dhruv8sh dhruv8sh linked an issue Oct 7, 2026 that may be closed by this pull request
9 tasks

@prk-Jr prk-Jr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Spec and plan for #1231's [rewrite] include_domains asset-host allowlist. The design holds up: the code claims check out against 7a0ecb4c, the <link imagesrcset> double rewrite is real (reproduced against lol_html 2.9.0), the shared step is byte-identical with #1249's, and the plan's snippets compile and pass clippy. One blocking issue: the include-list validator doesn't meet its own stated guarantee for some IPv6 and xn-- entries.

7 of the inline comments below carry a one-click GitHub suggestion — use Commit suggestion (or Add suggestion to batch) to apply them. The remaining comments describe the fix in prose because it spans several places or lines outside a single range.

Blocking

🔧 wrench

  • Validator's IPv6 check uses Rust's formatter, not host_str()'s — see inline at docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md:967

Non-blocking

⛏ nitpick

  • Step 5 grep can never pass — see inline at docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md:671
  • Shared step: is_host_permitted visibility contradicts the plan — see inline at docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md:312
  • Only test-fastly runs core tests — see inline at docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md:321
  • rewrite_clicks doesn't exist on main — see inline at docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md:1720
  • Leftover sentence in Task 3 Step 2 — see inline at docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md:876
  • Module-doc anchor that doesn't exist — see inline at docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md:1479
  • Output-parity and blob-hash wording — see inline at docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md:233
  • GTM citation points at test code — see inline at docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md:94
  • Test-plan cases the plan doesn't include — see inline at docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md:325

♻️ refactor

  • link[href] snippet relies on comment placement to pass clippy — see inline at docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md:1593-1613
  • Task 2 red step can't be observed — see inline at docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md:657

🤔 thinking

  • Inert exclude_domains entries stay silent — see inline at docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md:132

Cross-cutting / body-level findings

  • 📌 Trailing-dot hosts bypass exclude_domains (pre-existing) — https://blocked.example.com./x has host_str() blocked.example.com., which matches neither blocked.example.com nor *.example.com in is_host_allowed (proxy.rs:1242-1255), so it is proxied, signed and gets ts-ec despite the exclusion. include_domains fails safe here (an off-list host stays raw). Not for this PR; worth a follow-up issue (e.g. strip one trailing . before matching).
  • 👍 Solid design choices — the sign handler's ordering keeps 403 reserved for proxy.allowed_domains; the normalizer/policy split is clean and parse-once; the shared step with #1234 is reproducible verbatim; and the plan's red/green expectations line up with the code paths.

CI Status

  • format-docs: PASS
  • format-typescript: PASS
  • CLAUDE.md symlink guard: PASS
  • CodeQL: SKIPPED
  • cargo fmt, cargo test, cargo test (axum native), cargo test (cross-adapter parity), cargo check/build/test (spin native + wasm32-wasip1), cargo check (cloudflare native + wasm32-unknown-unknown), cargo test (ts CLI, native) (ubuntu-latest / macos-latest), vitest, prepare integration artifacts, Analyze (rust / javascript-typescript / actions / python): PENDING at review time (docs-only change)

Comment thread docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md Outdated
Comment thread docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md Outdated
Comment thread docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md Outdated
Comment thread docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md Outdated
Comment thread docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md Outdated
Comment thread docs/superpowers/plans/2026-10-07-1231-creative-asset-host-allowlist.md Outdated
Comment thread docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md Outdated
Comment thread docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md Outdated
Comment thread docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md Outdated
Comment thread docs/superpowers/specs/2026-10-07-1231-creative-asset-host-allowlist-design.md Outdated
@aram356
aram356 marked this pull request as draft October 9, 2026 05:27
@aram356

aram356 commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

@dhruv8sh Please merge into implementation PR #1253

aram356 and others added 2 commits October 8, 2026 22:30
Round-trip include_domains entries through url::Host::parse so IPv4-mapped IPv6 and invalid xn-- entries are rejected, warn on exclude_domains entries that never match, and fix the grep, red-step, citation and wording issues raised in review.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
@dhruv8sh

Copy link
Copy Markdown
Collaborator Author

Closing in favor of #1253

@dhruv8sh dhruv8sh closed this Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a host allowlist for creative asset rewriting

3 participants