Skip to content

feat: Adds stability domain - /memory-leak, /race-condition-repro skills - #81

Open
MajorLift wants to merge 8 commits into
mainfrom
jongsun/add/memory-leak-hunt-skill-v2
Open

feat: Adds stability domain - /memory-leak, /race-condition-repro skills#81
MajorLift wants to merge 8 commits into
mainfrom
jongsun/add/memory-leak-hunt-skill-v2

Conversation

@MajorLift

@MajorLift MajorLift commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Overview

Adds a new stability domain — a leak kills the tab: availability, not throughput — with two skills.

memory-leak — a two-phase retention review. Phase 1 is static, from the diff: name each retention primitive the change introduces — listener, timer, pending-map entry, subscription, module singleton, growing collection — as a holder → held set → outlived boundary, and pair every acquire against its release. Findings are scoped to what the diff adds. Phase 2 — heap snapshots, retainer graph, a falsifying lifecycle test — is reached only where Phase 1 cannot pair a primitive.

race-condition-repro — forces the interleaving instead of waiting for it: fake timers advanced into the pending window. The proof obligation is that the interleaving happened, not that the assertion passed.

Motivation

A vacuous green that looks exactly like a real one is the shared failure: a missing release is an absence, so a passing suite is what the leak looks like, and a test whose operations ran in sequence exercised no race.

Showcase

Two merged metamask-extension PRs, opposite answers:

PR Phase 1 Phase 2
#40684 every acquire pairs — no unreleased retention in this diff not needed
#44352 nothing to pair — the leak is a native object's lifecycle, not the diff's +~105 MB retained per popup cycle, linear, surviving a forced GC

The slope, not one reading, separates a leak from a working set.

Trial runs on PRs nobody flagged:

PR Finding
#42823 Gap — listener registered, never removed; charged as latent
#45035 Clean — 2 primitives introduced, both paired

race-condition-repro has no trial run yet, so nothing here speaks to it.

The evidence showcase carries the concurrency example: #44003 (deterministic interleaving) — 38/38 at head, 13/38 failing with the implementation reverted, all inside the retry blocks and 0 elsewhere, on an identical test file.

Two-phase retention review for JavaScript/TypeScript.

Phase 1 is a static read of a diff: enumerate the retention primitives the change
introduces — listeners, timers, pending-request registries, subscriptions, module
singletons, growing collections — and pair every acquire with its release site.
A primitive with a teardown is safe; one without is the finding.

Phase 2 escalates to DevTools/CDP heap snapshots only for a primitive the read
cannot pair. Leading with the read rather than the instrument settles most leak
claims without ever taking a snapshot.
@MajorLift MajorLift changed the title feat(coding): add memory-leak-hunt skill feat(coding): add memory-leak-hunt skill Jul 30, 2026
@MajorLift
MajorLift marked this pull request as draft July 30, 2026 14:03
The skill covers runtime retention behaviour, not code authoring, and `coding`
reads as language- and style-level guidance. Registers `/domains/stability/` in
CODEOWNERS alongside the other platform-owned domains.
@MajorLift MajorLift changed the title feat(coding): add memory-leak-hunt skill feat(stability): add memory-leak-hunt skill Jul 30, 2026
@MajorLift
MajorLift marked this pull request as ready for review July 30, 2026 18:15
@MajorLift

MajorLift commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Context budget

What this PR costs an agent, measured from an install rather than read from the diff. Three tiers, and only the first is unavoidable.

Skill Frontmatter Selected + refs & knowledge
memory-leak 992 chars ~2,701 tok ~5,225 tok

Frontmatter is the only tier paid unconditionally — every agent loads it on every run once the skill is installed, used or not, because it is what the agent reads to decide relevance. The 28 skills across the eleven open skill PRs sit at a median of ~1,716 tokens selected and ~1,860 with references followed. All are within the 1,536-character description budget.

Selected is paid only when the agent picks the skill. + refs & knowledge is the ceiling if every bundled reference is then read; it is a worst case, not an expectation.

Method

tools/install --repo metamask-extension --maturity experimental against this branch at affac8f1f, measured per installed skill directory. Repo overlays are merged into the emitted SKILL.md, so they land in the selected tier rather than being missed by a source-byte count. Token figures are bytes/4 — a proxy for scale, not accounting.

These figures are pinned to the commit above and drift on every push; #96 tracks automating them.

`hunt` disambiguated nothing. No sibling skill targets memory leaks, and none
would: this one already covers both halves — the static retention read from the
diff and the heap investigation when the read cannot settle it — so there is no
detection/diagnosis split for the suffix to mark.

Installed as `mms-memory-leak`.
@MajorLift MajorLift changed the title feat(stability): add memory-leak-hunt skill feat(stability): add memory-leak skill Jul 31, 2026
Missed when the rename swept the other branches: the description, the section
heading, and two prose references all still named `pr-validate`. The evidence
category is now linked to the catalog rather than named bare.
The listener pass matched only `.on('event', handler)` and
`.addListener('event', handler)` — a method named exactly `on` or `addListener`
with a quoted event name. Any form carrying the event in the method name was
invisible.

Running it on extension#42823 returned "no retention path INTRODUCED" for a file
containing `background.onNotification(routeMessengerEventNotification)` with zero
`removeOnNotification` call sites anywhere in `ui/`. A clean verdict over a real
unpaired listener is the worst output this script can produce, because it reports
what the pattern can see as though it were what is there.

Adds `onXxx(handler)`, `subscribe(handler)`, `addEventListener`, and
`addXxxListener` forms, each paired against its corresponding release
(`removeOnXxx`/`offXxx`, `unsubscribe`, `removeEventListener`, `removeXxx`).
The same file now reports the primitive as NEW and OPEN.

Verified no regression: `client.on('connected', connected)` and its siblings in
qr-sync-controller.ts are still detected by the quoted-event pass.

Also drops a hardcoded `PR #40684` from the header, which printed on every run
whatever was scanned, and a re-run hint naming a script that does not exist.
The installer emits `mms-memory-leak`; the description advertised `/memory-leak`.
@MajorLift MajorLift changed the title feat(stability): add memory-leak skill feat: Adds stability domain - memory leak skill Aug 31, 2026
@MajorLift MajorLift changed the title feat: Adds stability domain - memory leak skill feat: Adds stability domain - /memory-leak skill Aug 31, 2026
## Summary

- Adds `race-condition-repro`, the engine behind `evidence`'s B7
deterministic-interleaving evidence category and the sibling
`red-on-base` names for ordering bugs.
- Covers claims where correctness *is* the interleaving rather than a
value: cancellation, supersession, retry ordering, debounce/throttle,
locks, queues, async state machines.
- The falsifier is a test that never interleaved — operations that run
to completion in sequence exercise no race and produce a vacuous green
indistinguishable from a real pass. The proof obligation is therefore to
show the interleaving occurred, not that the assertion passed.
- Lands in `domains/stability/` beside `memory-leak`, the other
defect-class engine `evidence` delegates to.

## Test plan

- [ ] `node .github/scripts/lint-skill-entry.mjs
domains/stability/skills/race-condition-repro/skill.md` — 0 errors
- [ ] Frontmatter: `name` matches directory, `maturity: experimental`,
description within the 1536-char budget
- [ ] `domains/stability/` is also created by #81; domains are
discovered via `readdirSync`, not a registry, so both can add it
independently





---

## Validation runs

Trial runs of this PR's skills against merged `metamask-extension` PRs
nobody flagged. Every claim was re-verified against the real diff before
posting. Clean results are included on purpose — a skill that only ever
reports problems cannot be calibrated.

| PR | Skill | Verdict | Finding |
|---|---|---|---|
|
[#41917](MetaMask/metamask-extension#41917 (comment))
| `race-condition-repro` | Mixed | one guarantee forced, the sibling
untested |
|
[#44194](MetaMask/metamask-extension#44194 (comment))
| `race-condition-repro` | Gap | abort path: 0 refs across 181 test
lines |

Each comment carries a trial-run disclaimer and links back here for
feedback.

---------

Co-authored-by: abretonc7s <107169956+abretonc7s@users.noreply.github.com>
Co-authored-by: behroozreview <behrooz.aghakhanian@gmail.com>
Co-authored-by: Ramon AC <36987446+racitores@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@MajorLift MajorLift changed the title feat: Adds stability domain - /memory-leak skill feat: Adds stability domain - /memory-leak, /race-condition-repro skills Aug 31, 2026
The 25 conflicts were all in files this branch picked up when #97 merged in,
never in its own `domains/stability/` content. `main` has since carried five
more commits over those same files — #109, #115, #54, #120, #132 — against
this branch's single snapshot of them, so `main` is authoritative for every
one and its version is taken throughout.

That includes six deletions: #120 dropped Detox E2E, and the four
`references/detox/` files plus `detox-to-appium.md` and
`detox/legacy-playbook.md` go with it. Nothing references them.

The merged tree now differs from `main` only in the `CODEOWNERS` line this PR
adds and its five `domains/stability/` files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant