Add source map symbolication and source view support#6018
Draft
canova wants to merge 11 commits into
Draft
Conversation
…SOURCES Version 7 of the WebChannel renames the GET_JS_SOURCES request field from `sourceUuids` to `sourceIds`. The sender picks the field name based on the negotiated WebChannel version so both Firefox 145 (v6) and newer (v7+) builds are supported.
Lets the browser fetch source maps from URLs the frontend cannot reach (file://, chrome://, localhost, ...). The frontend identifies the bundle source via its sourceId, and the browser returns the parsed source map. Currently it's plumbing only. No caller dispatches GET_SOURCE_MAP yet. Adds source-map and url as direct dependencies; `source-map` provides the RawSourceMap type and runtime parser, and `url` is required by source-map's URL resolution in browser bundles.
Extends the processed profile shape with a per-thread-shared SourceMapInfoTable and a content column on the SourceTable. Each frame and func gains a nullable SourceMapInfoTable index, set to null by all existing initializers and upgraders. Source content stays null until JS symbolication populates it. The new tables are plumbed through data-structures, profile-compacting, merge-compare, and every importer. The derived Thread carries sourceMapInfo so later consumers can read it. Adds a v63 upgrader and a CHANGELOG entry. No call site reads the new columns yet, behavior is unchanged.
The nonymous algorithm (johnjbarton.github.io/nonymous) is how SpiderMonkey labels anonymous JS functions inside its NameFunctions.cpp pass. Adds a parser and serializer for the `/`-separated, `<`-marked name format Gecko emits, so later symbolication can produce names that match what the browser already shows. Pure utility, no callers yet.
Parses compiled JS with @lezer/javascript and walks the CST to build a tree of function scopes with name-mapping locations - the character offsets later probed against the source map to recover original function names. `@lezer/javascript` is already a transitive dep via `@codemirror/lang-javascript`. Pure utility, no callers yet. Tested via the new source-map-scope-tree.test.ts.
Implements the off-main-thread part of JS source map symbolication. SourceMapStore wraps the source-map library's WASM-backed SourceMapConsumer, source-map-symbolication.ts walks every func and frame and maps its compiled position back to the original source via exact-match source-map lookups (with scope-tree probes to recover function names), and source-map.worker.ts wires those pieces up as a Web Worker entry point. The action thunk doSourceMapSymbolication spawns the worker on demand and dispatches SOURCE_MAP_SYMBOLICATION_APPLIED on success. (The action and the SOURCE_MAP_WORKER_PATH global declaration land now so later wire-up patches can call into them.) No caller dispatches the thunk yet, so this patch is dormant. `src/types/globals/global.d.ts` declares the SOURCE_MAP_WORKER_PATH global; esbuild defines its value in a later patch.
The Web Worker added in the previous patch needs its own bundle so the @lezer/javascript and source-map dependencies (plus the source-map WASM mappings) ship to the browser. Adds: - sourceMapWorkerConfig in esbuild-configs.mjs: standalone IIFE so the worker loads without ES module support; hashed filename in production, fixed name in dev. - mappings.wasm copy plugin so the source-map library can find its WASM backing at runtime. - build.mjs: build the worker first, then inject its hashed path into the main bundle via SOURCE_MAP_WORKER_PATH. - dev-server / run-dev-server: watch the worker config alongside the main bundle. - build-profiler-cli: define a dummy SOURCE_MAP_WORKER_PATH so shared code can reference the constant without a real browser worker. - jest + test/fixtures/node-worker: let the worker entry point load under jsdom for tests. Still no caller invokes the worker, so this patch only changes how `yarn build` lays out the dist directory.
Activates JS source map symbolication end-to-end. After native symbolication settles, the receive-profile flow now fetches source maps and compiled JS for the bundle sources visible in any track, hands them to doSourceMapSymbolication, and lets the worker rewrite funcTable / frameTable / sourceMapInfo / sources / stringArray in-place via SOURCE_MAP_SYMBOLICATION_APPLIED. Adds: - doResolveSourceMaps in receive-profile.ts: fetches source maps and compiled sources from the browser in parallel, skipping sources without a sourceId. - collectSourceIndicesFromThreads in profile-data.ts: walks samples and allocation stacks to find the set of sources actually referenced. - The profile-view reducer case that swaps the new tables in. After this patch, sourceMapInfo is populated but UI code still reads the compiled positions. The next patch makes it consult sourceMapInfo.
The sourceMapInfo table is populated after the previous patch's wire-up, but no UI code reads it yet. This patch teaches the source view, call tree, tooltip, line timings, and getOriginAnnotationForFunc to prefer source-mapped positions when present and fall back to the compiled position otherwise. Frame-level entries override func-level entries (so inlined code lands in the right original file). Adds the inline-content fallback in selectors/code.tsx: when the profile already carries sources.content (from a shared profile or from JS symbolication), the source view uses it directly instead of hitting the browser. This is what makes the source view work offline.
I noticed this bug while testing the source map PR locally. I was doing this: - Capture a profile with the JS sources feature enabled that includes source maps. - After the symbolication and source fetching, double click on a JS function that is source mapped. - Refresh the page. I was expecting it to work, but it was throwing errors because the source that I already opened wasn't in the profile and string table anymore (since it's added after the source map symbolication). So this fixes this issue by checking if the string table index is undefined
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6018 +/- ##
==========================================
- Coverage 83.82% 83.77% -0.05%
==========================================
Files 328 332 +4
Lines 34255 34788 +533
Branches 9574 9719 +145
==========================================
+ Hits 28713 29144 +431
- Misses 5114 5215 +101
- Partials 428 429 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bugzilla part of this PR: https://bugzilla.mozilla.org/show_bug.cgi?id=2035493
This requires a Firefox that has patches applied in bug 2035493. And also it requires the "JavaScript Sources" features to be on.
It adds a pipeline that maps compiled/bundled JS frame positions back to their original source files using source maps fetched from the browser after profile load, similarly to our native symbolication step. And it adds a way to see the source mapped source contents.
After this PR, the call tree, flame graph tooltip, source view, and line timings show original TS/JS positions and function names for frames recorded in minified bundles.
I split the work in multiple patches so it's easier to review, but admittedly it's a lot of changes. The initial commits in the PR don't change the behavior until the commit that wires everything up and updates the visualization.
Example STR:
You should then see the symbolicated functions on the frames that are coming from Firefox Profiler source code. (Note that the react code will still be minified as it doesn't have source maps). You should also be able to double click on a JS function function to be able to see the source mapped JS source.