test_runner: change-aware test selection #66006
Description
Activity
Wouldn't this require Node.js to maintain an implementation of git? That seems a bit out of scope for core, and instead should be a userland package.
Every such tool has to reimplement module resolution, and gets TypeScript, subpath imports and node_modules boundaries subtly wrong. The resolver is already in core.
Surely there's a simpler way, I imagine a tool just needs to maintain a mapping of
testFilePath: [...watchFilePaths], and pass the list like that, without a complete resolver, etc.Reacted by JihwanWouldn't this require Node.js to maintain an implementation of git?
No git in core, that's what the "Paths rather than a
--changed=<rev>flag" paragraph is about.--relatedtakes paths, and where they come from is the caller's problem:$ git diff --name-only | node --test --related=-I imagine a tool just needs to maintain a mapping of
testFilePath: [...watchFilePaths]Building that mapping is the part that needs the resolver. You cannot know a test reaches
src/util.tswithout resolving its imports transitively, through TypeScript extensions, subpath imports andnode_modulesboundaries. Passing the list is the easy half.
What is the problem this feature will solve?
node --testalways runs every test file. There is no way to run only the tests affected by a change.--watchdoes this, but only while it stays running. CI, a pre-commit hook, or a branch diff all start cold and run everything.Jest (
--onlyChanged) and Vitest (--changed) both do this.What is the feature you are proposing to solve the problem?
Run only the test files whose module graph reaches a given set of files.
Graph-aware, not path-based: a test that imports a module that imports the changed file is selected. Conservative by default, so anything the graph cannot see has to run, and a change to
package.jsonor a lockfile disables filtering entirely.Paths rather than a
--changed=<rev>flag, because that would mean core shelling out togit. There is no VCS dependency anywhere inlib/today and I don't think this justifies introducing one.--changedcan be layered on later if the team wants it.Things to figure out
require(), since the bundled lexer reports exports rather than requires. CJS files would be opaque: any test reaching one always runs. Correct, but a CJS-heavy project gets little out of this.fs, or uses dynamicimport(), cannot be selected statically. Those have to resolve to "run it".Under-selection is the failure mode that matters. Silently skipping a test the change broke is worse than not having the feature.
What alternatives have you considered?
Userland: a wrapper computing the list and passing it to
run({ files }). Every such tool has to reimplement module resolution, and gets TypeScript, subpath imports andnode_modulesboundaries subtly wrong. The resolver is already in core.--test-rerun-failurescovers rerunning what failed, not running what a change could break.cc @nodejs/test_runner