feat: add deterministic visual-change detection for PRs.

[BillLeoutsakosvl346]

Summary

Add a deterministic, artifact-only visual-change analyzer for staging PRs across product, landing pages, emails, docs, desktop, shared components, themes and assets. It reads immutable Git objects and compares the merge-base with the PR head. Decisions remain flag or exempt; unresolved effects flag only when changed source has presentation evidence or a traceable rendering input. Completed analyses exit 0; operational failures exit nonzero with flagged: null.

Schema 3.0.0, engine 0.3.0, policy 3.0.0 replace the earlier draft report. Findings stay grouped by changed source with locations, direct changes, a representative consumer and partial usage counts. Full semantic hashes determine differences before 4 KiB previews and deterministic list sampling; serialized reports are capped at 5 MiB with truncation metadata.

This revision removes module-wide dependency taint, narrows each helper hop and known environment keys, avoids repeated consumer expansion after changed-source evidence is established, distinguishes SQL/telemetry bindings from actual DOM/canvas rendering, traces selected imports/properties/helper returns and supported equivalent refactors, traces rendered React state setter inputs, follows configured Fumadocs OpenAPI file inputs, and compares rendering dependency closures instead of every lockfile edit. The same conservative movement policy remains in place.

The advisory production workflow runs trusted default-branch engine/configuration/dependencies with read-only permissions. It retains only a JSON artifact identified by PR/head for seven days. It becomes operational after reaching main. Its absence here and the earlier schema-2 smoke run do not validate this revised engine. No comments, labels, annotations, required-check registration, Slack, screenshots or AI stages are added.

See scripts/design-diff/README.md and the frozen manifest at scripts/design-diff/benchmark/comparisons.json for supported conventions, limits and reproduction commands. Schema-2 readers must handle schema-3 value summaries and omitted-detail counts.

Type of Change

• Bug fix
• New feature
• Breaking change (draft JSON schema 3 replaces schema 2)
• Documentation
• Other

Testing

• Bun 1.4.1: 478 script tests / 33 suites pass. Coverage includes audited false positives, selected bindings, conditions, mutable collections, positive visual categories, OpenAPI inputs, Git edge cases, byte-identical repeated/truncated reports, changes beyond previews and source/plugin non-execution sentinels.
• Engine TypeScript, targeted Biome, script-test discovery, mandatory API-validation audit, Actionlint and git diff --check pass locally. Trusted dependencies were installed from the frozen lockfile with lifecycle scripts disabled.
• Historical benchmark: the fresh 180-comparison replay is running against immutable engine 8697761f84762a2550e2b4318a84a805bda773bd: https://github.com/simstudioai/sim/actions/runs/34577479566. Baseline remains 108/120 flagged, including 39/49 manually nonvisual cases. Final measured results pending; incomplete prior replays are excluded. The research replay uses a recorded 1,800-second deadline; production remains 15 minutes, and comparisons exceeding that budget will be reported separately.
• Latest-head PR CI: pending; final statuses will be checked against 8697761f84762a2550e2b4318a84a805bda773bd.

Remaining limitations: source labels are not rendered pixel ground truth. Arbitrary runtime behavior, broad authentication/data factories, unresolved conditions and unsupported drawing/plugin mechanisms can still overflag or escape precise tracing. Arbitrary event reachability, reducer/effect scheduling, async rejection effects and dynamic/generated inputs outside configured conventions are not fully modeled. Usage counts are partial references, not a claim that every instance changes visually.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing locally
• No new warnings introduced across all application CI checks
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

Source-analysis automation; screenshot capture is a later stage.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 11, 2026 8:07am UTC