Add a host allowlist for creative asset rewriting #1231

Summary

We need a way to limit creative rewriting to specific asset hosts. Today the creative rewrite pass sends every absolute http(s) URL in a winning creative through /first-party/proxy unless its host matches [rewrite] exclude_domains. There is no way to say "proxy only these hosts". An allowlist also narrows which third-party hosts receive the EC ID, because proxied fetches append ts-ec to the upstream URL (append_ec_id, proxy.rs:1203).

Click wrapping (<a href> to /first-party/click) runs in the same pass and goes through the same per-URL check, but it is out of scope here. #1234 makes it a separate feature. This allowlist must not apply to anchors. Both issues touch to_abs and its call sites in creative.rs, so coordinate if they're worked in parallel.

Spec first

Write a design spec and get it approved before opening an implementation PR.

• Put it at docs/superpowers/specs/YYYY-MM-DD-1231-<slug>-design.md and follow the existing specs there. 2026-08-26-first-party-sign-allowlist-enforcement-design.md covers the neighboring sign endpoint and is a good model. Include the issue link, date, status, problem, goals, non-goals, design, test plan and rollout.
• Land the spec in its own PR. The implementation PR links the approved spec and follows it. If the design changes during implementation, update the spec in the same PR.
• The spec must settle these open decisions:
  • The setting's name and table. [rewrite] include_domains is a suggestion.
  • How to_abs splits into URL normalization and policy, and the resulting signatures at the asset call sites, the anchor handler and /first-party/sign.
  • The /first-party/sign response for an off-list host. This issue proposes the same non-403 error that excluded hosts get.
  • Whether lowercasing exclude_domains entries counts as a behavior change. Entries written in uppercase never match today and would start matching after the fix, so it needs at least a CHANGELOG note.
  • How the change lands alongside Make creative click rewriting a separate feature with its own switch #1234 if both are in flight.

Current behavior

Line numbers are at 8048301.

• One lol_html pass, rewrite_creative_html_impl (crates/trusted-server-core/src/creative.rs:1125), rewrites assets to /first-party/proxy. Assets means img, srcset, script, link, media, object/embed, input[type=image], SVG href, iframe, inline style and <style> URLs. The same pass rewrites anchors to /first-party/click (creative.rs:1283-1291).
• Every per-URL decision goes through to_abs (creative.rs:54-75). It returns None, which leaves the URL untouched, for relative URLs, non-http schemes, and hosts that settings.rewrite.is_excluded matches (line 70).
• Three entry points share the pass:
  • POST /auction adm, via process_auction_creative (auction/formats.rs:398).
  • SSAT and page-bids inline creatives, via process_inline_auction_creative (publisher.rs:5835).
  • HTML and CSS fetched through /first-party/proxy (CreativeHtmlProcessor and CreativeCssProcessor, proxy.rs:645-680).
• /first-party/sign (proxy.rs:1625) signs URLs at runtime for the client render guard. It calls to_abs, then calls is_excluded again at line 1683 because protocol-relative input skips to_abs. An excluded host gets a 502 and the browser loads the raw URL directly. A host outside proxy.allowed_domains gets a 403 and the browser blocks the load.

proxy.allowed_domains won't do this job. It is the fetch-time SSRF control, and the Prebid external bundle fetch uses it too. With it set, the rewriter still turns off-list URLs into /first-party/proxy links and the fetch then fails with a 403. The asset breaks instead of loading directly.

Proposed change

Add a list next to exclude_domains, for example [rewrite] include_domains. The name is open, but avoid allowed_domains so it can't be confused with proxy.allowed_domains.

• Empty, the default, keeps today's behavior. Every eligible asset URL is proxied.
• When non-empty, only asset URLs whose host matches the list are rewritten. Everything else keeps its original absolute URL.
• exclude_domains wins when a host matches both lists.
• Same pattern syntax as exclude_domains. An entry is an exact host, or *.example.com, which matches the apex and any subdomain. Normalize entries at load by trimming and lowercasing, and reject empty entries and a bare *.
• Applies to every asset handler in the pass on all three entry points, including srcset candidates and CSS url() and @import. Nested documents fetched through /first-party/proxy go through the same pass and pick it up with no extra work.
• Does not apply to anchors.

Suggested shape: split to_abs into URL normalization and policy. The asset call sites (proxy_if_abs, proxied_attr_value, rewrite_srcset, CssUrlRewriter::rewrite) then check the exclude list and the allowlist, and the anchor handler checks exclude_domains only.

/first-party/sign must apply the same allowlist to both absolute and protocol-relative input. An off-list host should get the same non-403 error that excluded hosts get today. The render guard then falls back to a direct load instead of blocking the resource.

Constraints

• Config structs use deny_unknown_fields, so older binaries reject blobs that carry the new key. Skip serializing the default value the way rewrite_creatives does (auction_config_types.rs:46-50, tested in config_payload.rs:715). The upgrade order stays binary first, config second.
• EdgeZero env overlays only replace scalar leaves that already exist in the TOML, so the list can only be set in trusted-server.toml and published with ts config push.
• Rewrite::is_excluded (settings.rs:646-671) compares raw config entries against the parsed host, which the url crate has already lowercased, so an entry written in uppercase never matches. proxy::is_host_allowed (proxy.rs:1242) lowercases both sides. Use one case-insensitive matcher for both rewrite lists and fix exclude_domains in the same change. Drop the stale #[allow(dead_code)] on is_excluded too.

Out of scope

• Click rewriting (Make creative click rewriting a separate feature with its own switch #1234).
• Domain checks at /first-party/proxy, /first-party/click and /first-party/proxy-rebuild. Fetch-time SSRF stays with proxy.allowed_domains, and the click endpoints redirect any validly signed target by design (see the comment at proxy.rs:1896-1906).
• Client-side changes. The render guard asks /first-party/sign, so it follows the server's decision without new config.

Done when

• The design spec is approved and merged under docs/superpowers/specs/, and the implementation PR links it.
• With the allowlist empty, rewritten output for every existing creative fixture is unchanged.
• With the allowlist set, listed hosts are rewritten to /first-party/proxy and other hosts keep their raw URL. Tests cover img, srcset, script, link, iframe, SVG href, inline style and <style> including @import.
• A host in both lists is left alone.
• Matching is case-insensitive, and *.example.com matches example.com.
• /first-party/sign rejects off-list hosts with a non-403 error for absolute and // input.
• Anchors are wrapped regardless of the allowlist.
• A config round-trip test shows the default is left out of the serialized blob.
• docs/guide/configuration.md, docs/guide/creative-processing.md, docs/guide/first-party-proxy.md, trusted-server.example.toml and CHANGELOG.md are updated. Also fix creative-processing.md:654-661, which says *.cdn.example.com does not match cdn.example.com. The code does match it.

Affected area

HTML processing / JS injection