Skip to content

Report which packages and modules would run in a Node island - #1691

Open
cramforce wants to merge 5 commits into
mainfrom
ni-2-node-island-report
Open

cramforce wants to merge 5 commits into
mainfrom
ni-2-node-island-report

Conversation

@cramforce

@cramforce cramforce commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This is milestone NI-2 of the scope-B plan: the Node island policy, the project configuration and the detector, all in report-only mode. scriptc coverage now lists the npm packages that a build would run in a Node island and explains why. Code generation does not change, and no build output changes. Builds and coverage gain one new failure, by design: an unknown value in the project's scriptc configuration is now an error (SC8004).

  • Project configuration (project-config.ts). One reader for the scriptc key of the entry's nearest package.json, with a schema. An unknown key or value in scriptc.packages or scriptc.nodeIslands is SC8004, located at the key. Until now configurePackageShims ignored anything except "unloaded", so a typo such as "unloded" silently did nothing. Package shims and the WebAssembly scanner now read through this module, and scriptc.wasm keeps its lenient reading.
    • Packages accept "unloaded" | "node" | "native" | "auto" or { run, evaluate, share, marshal, schedule }.
    • nodeIslands accepts auto, modules, node, nodeVersion, packaging and speculativeStart.
  • Detector (node-islands/):
    • origin.ts: whether a file is user, npm, shim or builtin code, and which package instance (identified by its directory) an npm file belongs to.
    • rules.ts: the rule families R1.1–R2.5 and the classes runtime, object-model, gap and packaging. A fifth value, marked, covers islands that have no reason of their own.
    • detect.ts: a syntactic pre-scan for R1 and R2 markers.
    • scan.ts: module and package-edge collection on the final program, and on the program that still held each package the drop-and-reload loop removed.
    • classify.ts: maps a refusal, a runtime fence or a package refusal to a class. Some refusal sites carry a reason id, and the rest are matched by their messages. The sites with a reason id are computed import(), computed require(), createRequire, native addons, require.cache, WeakRef/eval and sloppy arguments.
    • policy.ts: auto | explicit | off, packages forced to node or native, and user modules marked by glob or path.
    • report.ts: decisions and the island closure. A package imported only by islands runs inside the island. A package reached from both native code and an island is reported as shared.
  • Coverage:
    • A "Node islands" section lists gap islands first.
    • --explain-island=<pkg> shows the first reason per module, the native import sites (the boundary) and the packages inside the island's closure.
    • The diagnostics envelope gains a nodeIslands field.
    • --fail-on gains node-islands, eager-node-islands (the same as node-islands until deferred evaluation lands) and island-gaps.
  • CLI:
    • --node-islands=auto|explicit|off, --node-island=<pkg-or-path>, --native=<pkg> and --node=<path>: accepted by build, run and coverage, validated, and report-only.
    • --strict-user-js (coverage): reports R2 constructs in the user's own .js/.cjs/.mjs files as SC8101–SC8105 blockers. These are the SC81xx codes, kept behind a flag.
    • The bootstrap fast paths hand any island flag to the full CLI.

Zero cost

  • scripts/zero-cost-gate.mjs against origin/main 0237d70, Linux sandbox:
    • --set=bench,self: 34 identical, PASS.
    • --set=corpus --corpus=all, in two shards:
      • shard 2/2: 1754 identical, PASS;
      • shard 1/2: 1753 identical and one program reported as different, corpus/fs-extra-package. It differs only in IR source locations: the path of the compiler's own fs-extra shim, which lives in the compiler checkout, so baseline and candidate necessarily disagree. LLVM and the executable are byte-identical, and the IR is byte-identical once that path is normalized. This is an artifact of the gate, not of this change.
  • npm fixture programs (tests/fixtures/npm/cases, npm-static, commander-calc), LLVM from both compilers: 112 byte-identical. The other 28 fail to build under both compilers with identical diagnostics. None differ.
  • The time coverage takes is unchanged. On the same VM, the corpus coverage sweep (shard 1/4) took 212.8 s against 215.8 s on main. analyze() on commander-calc, jose and effect is within noise of main.
  • The self-hosting build (native/cli.ts --emit=ir) produces no diagnostics.

Classification

npm fixture corpus (tests/fixtures/npm/cases, npm-static, commander-calc): 140 programs, of which 79 have at least one would-be island. Across 122 distinct packages:

Outcome Packages Detail
Island: gap 37 first reasons by code: SC1090 15, SC2020 10, SC1010 4, SC1016 4, SC1013 2, SC1030 1, SC1031 1
Island: object-model 19 R2.1 7, R2.5 5, R2.3 4, R2.2 3
Island: packaging 6 2 without .d.ts, 1 typed by a third-party @types package, 3 bundler-interop refusals
Island: runtime 5 R1.2 2 (createRequire escapes in bundles), R1.1 1, R1.4 1, R1.5 1
Inside an island's closure 3
Native 52

Many fixture packages exist to exercise edge cases, so gaps are over-represented. Some notable packages:

  • commander is a gap island because of a fence in an untaken path: SC1030, a reference to a var above its declaration.
  • chalk (R2.2) and hot-shots (R2.1) are object-model islands, consistent with the registry shims planned for NI-9.

Real-world fixtures:

  • effect 3.22, in four programs from effect and effect4: object-model, from R2.2 Object.setPrototypeOf.
  • jose 6.2.12, in corpus programs 6331 and 6295:
    • 6331: object-model, from R2.3 Object.defineProperty on function objects, plus 30 gap fences in untaken WebCrypto and deflate paths;
    • 6295: gap.
  • Corpus directory programs: 9 of 278 have would-be islands: callback-hash (gap), corked-writer (R2.1), fs.realpath (R2.5), both jose programs, metrics-client (R2.1), promise-mixin (R2.3), retry-lite (gap) and run-node (R2.3).
  • The three.js fixtures stop at preflight without the options their tests pass, so they produce no report.
  • opentui, workspace-source and benchmarks/builds have no npm islands.

Design corrections found

  1. The jose canary contradicts the reachability rule. §6.4 and §5.5.4 expect jose to have zero islands. But §1.4 says a refused statement in a module that loads is a reason, and D6 removes the npm fences. Under those two rules, typed jose has about 30 fenced statements in untaken paths plus R2.3 defineProperty-built function objects, so it would become an island. One of these has to change:
    • per-function reachability for typed packages;
    • keeping deferred fences for typed ESM;
    • treating defineProperty on a fresh function literal whose .d.ts declares the member as native.
  2. Islands in the typed corpus. §5.5.4 says the typed corpus has no islands, but the 9 corpus directory programs above have committed node_modules that would become islands. Either the policy test is limited to programs without npm code, or those programs become island tests.
  3. The nodeIslands JSON shape. It is an object, { mode: "would-be", policy, islands, userModules, npmPackages, violations, unknownPackages }, rather than a bare array, so the report-only state is machine-readable. npmPackages lives inside it.
  4. SC8002 and SC8003 in report-only mode. They are reported as would-be violations that fail the island --fail-on gates. They are not build errors.
  5. The class of a package with several reasons. The decisive order is runtime > object-model > packaging > gap. A package is a gap island only when nothing else keeps it out of the native subset, which is what makes gap islands actionable compiler work.

Tests

  • Unit tests:
    • node-islands/detect.test.ts: every marker of R1.1–R2.5, plus the exceptions (the global-object idiom, the probe in try, the transpiled-Error idiom, null-prototype literals, interop helpers, strict code);
    • node-islands/report.test.ts: decisions, closure, forced native and node, off and explicit, marked user modules, SC81xx, glob matching;
    • project-config.test.ts: the schema and SC8004 messages and locations.
  • Coverage snapshots:
    • tests/coverage-fixtures/node-islands: one package per class, a package inside an island's closure and a shared package. Covers the explain output, the envelope, the gates and the command-line policy.
    • node-islands-user-js.cjs: SC8101–SC8105.
  • Diagnostics fixture: tests/diagnostics/node-islands-config (SC8004 with code frames in package.json). Its order-parity baseline is recorded, and pnpm test:ts7 --baselines-only passes.
  • Focused suites pass in the sandbox: compiler unit tests, coverage (the full corpus sweep in 4 shards), diagnostics, package shims, project config, npm. The 4 failures in project-config and npm (node24-url-authority, WebCrypto HMAC, net.BlockList, legacy url) also fail on main in the same sandbox, because its Node is 24.15.
  • pnpm lint and pnpm format:check pass.

Not in this milestone

  • SC8001 build-summary notes and statementsInNodeIslands. Both depend on islands being real (NI-4).
  • Docs (NI-11).
  • The build-container and pi trees of §6.1 and §6.2. They are private and were not available to this run.

🤖 Generated with Claude Code

Adds the report-only Node island policy, configuration and detector of
the scope-B plan (milestone NI-2). Coverage now lists the npm packages a
build would run in a Node island and why, gap islands first, without
changing code generation.

- project-config.ts reads the `scriptc` key of the entry's package.json
  with a schema; an unknown value in `scriptc.packages` or
  `scriptc.nodeIslands` is SC8004 instead of being ignored. Package shims
  and the WebAssembly scanner read through it.
- node-islands/: origin (user, npm, shim, builtin; package instances),
  rules R1.1-R2.5 and the four classes, a syntactic pre-scan for R1/R2
  markers, classification of refusals and runtime fences (reason ids at
  the refusal sites that need them), the policy (auto, explicit, off;
  forced node and native packages; marked user modules) and the report
  with island closure.
- coverage: a "Node islands" section, `--explain-island=<pkg>`, a
  `nodeIslands` field in the diagnostics envelope, and `--fail-on`
  `node-islands`, `eager-node-islands` and `island-gaps`.
- CLI: `--node-islands`, `--node-island`, `--native`, `--node` (validated,
  report-only) and `--strict-user-js` (SC8101-SC8105 for R2 constructs in
  user JavaScript).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
scriptc Ready Ready Preview, v0 Oct 22, 2026 3:19am UTC

Comment on lines +121 to +123
s.importers.add("");
s.nativeSites.push(e.loc);
} else if (e.fromDir !== s.key) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Packages imported by a marked user-module island are always treated as having native importers, so coverage incorrectly labels island-only dependencies as native and misses shared dependencies.

Fix on Vercel

Comment on lines +121 to +126
s.importers.add("");
s.nativeSites.push(e.loc);
} else if (e.fromDir !== s.key) {
s.importers.add(e.fromDir);
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a native npm package imports an island, its import site is omitted from both --explain-island and the diagnostics envelope because only user-code edges populate nativeSites.

Fix on Vercel

Comment on lines +295 to +298
if (marked !== null) {
userModules.push({ file: m.file, source: marked });
continue;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With --node-islands=off, marked user modules are still reported as islands and bypass --strict-user-js diagnostics instead of producing a policy violation.

Fix on Vercel

Comment on lines +260 to +269
const reachingIslands: string[] = [];
for (const from of s.importers) {
const importer = states.get(from);
if (importer === undefined) continue;
const owner = importer.island ? importer.key : importer.carriedBy;
if (owner !== null) {
const name = states.get(owner)!.name;
if (!reachingIslands.includes(name)) reachingIslands.push(name);
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A dependency of a shared package is reported as native-only because reachingIslands considers only direct island importers and packages carried exclusively by an island.

Fix on Vercel

cramforce and others added 3 commits October 21, 2026 20:07
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — d7e4c91d Deployed Oct 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant