Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ speedscope is designed to ingest profiles from a variety of different profilers
](https://github.com/jlfwong/speedscope/wiki/Importing-from-async%E2%80%90profiler-(Java))
- [Importing from Java Flight Recorder](https://github.com/jlfwong/speedscope/wiki/Importing-from-Java-Flight-Recorder)
- [Importing Java thread dumps (`jstack` / `Thread.print`)](docs/JAVA_THREAD_DUMPS.md)
- [Importing ZIP archives of Java thread-dump snapshots](docs/JAVA_THREAD_DUMP_ZIP.md)
- Android
- [Classic Android method traces (versions 1–3)](docs/IMPORT_FORMATS.md#classic-android-method-traces)
- Erlang/Elixir
- [eflambe](https://github.com/Stratus3D/eflambe)
- Native code
Expand All @@ -63,8 +66,9 @@ speedscope is designed to ingest profiles from a variety of different profilers
- [Importing from custom sources](https://github.com/jlfwong/speedscope/wiki/Importing-from-custom-sources)

Contributions to add support for additional formats are welcome! Open an issue in
[this fork](https://github.com/node-projects/speedscope/issues). The importer guides
above are maintained upstream and remain useful for the supported file formats.
[this fork](https://github.com/node-projects/speedscope/issues). Upstream importer
guides remain useful for their formats; [fork-specific import behavior](docs/IMPORT_FORMATS.md)
documents the Android, archive, pprof and background-import policies.
# Usage

Visit [the hosted app](https://node-projects.github.io/speedscope/), then browse for a
Expand All @@ -74,8 +78,10 @@ local recording. See [offline use and update behavior](docs/OFFLINE.md).
Repository builds can also produce a self-contained HTML document with
`npm run build:single-file`; see [single-file builds and CSP](docs/SINGLE_FILE.md).

Large JSON and trace captures import in the background with progress and a
**Cancel import** button. This improves responsiveness; very large captures can
Recordings of at least 1 MiB use a background importer when workers are available,
including perf, folded stacks, pprof, JFR, Android traces and renamed captures.
Progress and **Cancel import** keep the app responsive; generic formats still
materialize their input and profile model. Very large captures can
still exceed browser memory limits. Search offers Contains, Whole word and Exact
matching for function names and source paths.
Graph-search navigation retains at most 10,000 results and clearly marks larger
Expand Down
99 changes: 99 additions & 0 deletions docs/IMPORT_FORMATS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Import formats and background processing

The [README](../README.md#supported-file-formats) lists supported sources.
This guide describes additional fork-specific behavior and its limits.

## Classic Android method traces

Open the classic concatenated key/data format emitted by Android method tracing:
a text key section followed by a little-endian `SLOW` binary section. Versions
1, 2 and 3 are supported, including version-3 record padding. The file can be
named `.trace` or renamed; detection uses the format signature.

The importer preserves method signatures, thread identities and available Java
source locations. Clock metadata is required. `thread-cpu` and `wall` clocks
remain distinct; version-3 `dual` input creates separate labeled CPU and wall
profiles for each thread. Version-1 `global` means wall time. All values use
microseconds. CPU clocks are local to their thread; wall offsets remain relative
to the trace origin, so leading unobserved time can appear as idle weight.
The absolute date in the binary header is not added to those offsets.

Initial method exits reconstruct calls already active at trace start. Calls
still open at EOF close at that thread's last observed event. These are clipped
observation windows and do not invent work before or after the recording.

Fixed records are scanned without retaining a second parsed-record array.
Temporary state includes key dictionaries and per-thread stacks, alongside the
output builders. Key text is decoded line by line with a 16 MiB section limit
and 1 MiB line limit. The input buffer and final profiles still occupy memory.

Raw binary-only/streaming traces, newer ART packet formats, Perfetto protobuf,
Android systrace HTML and wrapped timestamp recovery are unsupported. Unknown
clocks, partial records, invalid method actions and decreasing per-thread clocks
produce explicit errors. See the
[synthetic fixture, generation script and AOSP references](../sample/profiles/android-method-trace/README.md)
for the exact supported layouts and clock policy.

## Java thread-dump archives

Traditional HotSpot `jstack` / `Thread.print` text can be opened directly or as
an archive of captures from one JVM. Observed stacks contribute raw samples;
they do not measure elapsed time. The ZIP importer checks declared sizes and
actual expansion, feeds bounded UTF-8 lines into the parser and supports
cancellation without returning partial results.

See [text snapshot semantics](JAVA_THREAD_DUMPS.md) and the
[ZIP format and resource policy](JAVA_THREAD_DUMP_ZIP.md). Archive support is
limited to classic ZIP32 STORE/DEFLATE; it is not a generic archive extractor.

## pprof metric and source identity

pprof's `default_sample_type` selects a sample metric by its string-table
identity. Protobuf integer objects are compared by value. If the preferred
metric is absent or cannot be found, the existing last-metric fallback applies.
For example, a recording can prefer allocation counts even when a later sample
column measures bytes; the displayed unit follows the chosen metric.

Function IDs and location IDs retain their complete unsigned 64-bit identity.
They are not truncated to their low 32 bits or rounded through a JavaScript
number, so IDs that share a low word remain distinct. String-table positions
must fit safe nonnegative integers. Source lines must fit safe positive integers.

Each location retains its own source line, falling back to the function's
definition line when needed. One location no longer overwrites the shared
function metadata used by another location. The current importer retains the
outermost entry of an inline-location chain; expanding all inlined functions
is outside this change.

This does not add signed/differential flamegraphs, exact arithmetic for weights
larger than JavaScript's safe-integer range, or a metric chooser. Negative Go
heap delta weights remain a separate unsupported case. See the
[pprof schema used by the importer](../src/import/profile.proto).

## Background import for large recordings

Inputs of at least **1 MiB** use a Web Worker when available, regardless of
filename extension. This includes perf, folded stacks, pprof, JFR, Android
method traces, supported ZIP archives and renamed captures. Format detection,
generic parsing, demangling and profile snapshot preparation happen away from
the main UI thread. Cancelling the task terminates its worker.

The worker first attempts the existing duration-trace streaming path, then
uses the generic importer when that format does not apply. Generic formats
still read/materialize their input and retain their final model; routing them
through a worker does not make them streaming or out-of-core. Snapshot transfer,
geometry and final view construction still have memory costs proportional to
the recording. [Large-trace guidance](LARGE_TRACE_PERFORMANCE.md) describes the
duration-trace limits and measurements separately.

If workers are unavailable or blocked by a host's CSP, the app falls back to
the main-thread importer. DOM-dependent formats can also fall back after worker
detection. A transferred input buffer is returned before this fallback, so the
main thread does not attempt to parse an empty detached buffer. Synchronous
generic fallback parsing can block the UI until it returns; the asynchronous
ZIP importer also yields and checks cancellation on the main thread.

Oversized supported ZIP files receive a four-byte signature preflight before
their complete body is read, both in the main importer and worker path. Other
formats retain their existing input/resource policies; the ZIP compressed-size
limit is not a blanket cap on every supported recording.
41 changes: 41 additions & 0 deletions docs/IMPORT_VALIDATION_2026-11-21.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Import-format validation, 2026-11-21

This batch adds classic Android method traces, bounded Java thread-dump ZIP
archives, broader large-file worker routing, and pprof metadata corrections.
See [format coverage and limitations](IMPORT_FORMATS.md) and
[ZIP limits](JAVA_THREAD_DUMP_ZIP.md). It does not make generic formats streaming
or the final profile model bounded in size.

## PLC regression

The private PLC gzip recording imported in 3,004 ms in one Node run on this
Windows ARM64 machine. This is a regression observation, not a controlled speedup
comparison or a browser timing guarantee. The export retained four profiles,
17,510 frames and 1,039,656 call events. Its baseline hash remains
`522de77afbfc4dd94f1f30c3a9551979d12db506305ffef999db9c910ae8eddb`.
The capture and benchmark artifacts remain uncommitted.

Browser checks used the built single-HTML app served over loopback HTTP with
external connections blocked by CSP. The PLC imported successfully, exposed
four profiles and rendered its call tree without console errors. The synthetic
Android dual-clock capture exposed CPU/wall totals of 20/40 microseconds for
the main thread and 7/20 microseconds for its worker. The two-snapshot Java ZIP
produced two raw samples and correctly disabled chronological Time Order.
These checks do not validate file:// loading or a real Android device capture.

## Automated verification

The final clean-cache Jest run passed 458 tests across 75 suites with 307 snapshots
using the official local Node 22.23.5 ARM64 runtime. System Node 24.19.0 ARM64
passed all five CLI/release-helper Node tests. Previous native crashes of full
Jest on this system's Node 24 ARM64 remain a local runtime limitation; no tests
were weakened to hide them. The CI matrix covers Node 22 and 24 on Windows and
Linux x64.

Type checking, linting, schema/backlog checks and Pages/standalone/single-HTML builds
passed. npm dry-pack includes the new format and ZIP resource-policy guides.
Targeted regressions cover ZIP expansion ceilings, cancellation, corrupt archives,
300 snapshots, Android record/clock validation, generic worker ownership and pprof
source identities. Pprof's existing fixture totals were reviewed after separating
source positions; frame cardinality can increase, while aggregate weights remain
unchanged. Signed negative pprof weights still require separate model support.
Loading
Loading