Skip to content
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
250 changes: 250 additions & 0 deletions proposals/proposal - unbreak test cli arg.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,250 @@
# Proposal: un/break `--test`

Currently, this flag's behaviour is very unexpected, often resulting in silent-failure footguns, and precludes command nesting, as demonstrated in a very simple and typical CI setup:

```yaml title="unit & e2e tests"
- name: tests with coverage
run: node --run test -- --test-reporter lcov
```

That results in

```sh
node --test --test-reporter lcov
```

In most cases, `--test-reporter` gets lost (the `lcov` reporter is not enabled); in a worse case, this unexpectedly causes tests within a directory named `lcov` to be run.

## More info

Currently (since always):

`--test` receives everything after it (space-delimited).

`--test` currently does 2 things:

* enables the test runner
* accepts paths for the runner to consume

This is similar to another existing feature with which we want to improve interop: `watch` mode.

`--watch` currently does 2 things:

* enables `watch` mode
* optionally accepts 1 value to override the entrypoint
Comment thread
Ethan-Arrowood marked this conversation as resolved.

```sh
node
--watch
--watch-path ./src/**/*.ts
--watch-path ./test/**/*.ts
```

## Proposed options
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The minutes from the last meeting says regarding this proposal: "Proceeding with the broadly accepted current proposal and can potentially pursue the subcommand in future."

But without rereading all the minutes from multiple meeting it's hard to figure out which of these options was broadly accepted. I suggest that all proposals that offer multiple options include a consensus or conclusions section before being accepted.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if the proposal spans multiple meetings and discussion threads, the consensus summary is updated in this section as it evolves.


### Option: Break all the things

Currently `watch` has 2 flags (which are redundant):

```sh title='reads "main" etc from package.json'
node --watch
```
```sh
node --watch ./src/not-pjson-main.js
```
```sh
node
--watch ./src/not-pjson-main.js
--watch-path ./src/**/*.js
```

This could be simplified into just 1 `--watch` flag (where the default value is a single-element array of the derived entrypoint).

```sh title='reads "main" etc from package.json'
node --watch
```
```sh title='override derived entrypoint'
node --watch ./src/not-pjson-main.js
```
```sh title='package.json "main" + an additional path'
node --watch ./src/**/*.js
```
```sh title='explicit entrypoint + additional paths'
node
--watch ./assets/**
--watch ./src/**/*.js
./src/not-pjson-main.js
```

An explicit entrypoint, like all node commands, must come last and must be a relative or absolute path (not a glob/pattern).
Comment thread
JakobJingleheimer marked this conversation as resolved.
Outdated

`test` would then work the same way:

```sh title='reads "main" etc from package.json'
node --test
```
```sh title='package.json "main" + an additional path'
node --test ./src/**/*.test.js
```
```sh title='explicit entrypoint + an additional path'
node
--test ./src/foo/*.test.js
--test ./src/bar/*.test.js
./src/not-pjson-main.js
Comment thread
JakobJingleheimer marked this conversation as resolved.
Outdated
Comment thread
JakobJingleheimer marked this conversation as resolved.
Outdated
```

#### All together

`--watch` receives a clone of `--test`'s resolved value(s) **plus** `--watch`'s own values—with 1 exception: when both `test` and `watch` modes are enabled, `--watch` does not include a default entrypoint (it's irrelevant and it would likely result in perf waste at best, and unexpected behaviour at worst).

All examples are sequence-independent (all within the same heading behave the same).

##### Watch + test paths & main entrypoint:

```sh
node
--test ./src/foo/*.test.js
--test ./src/bar/*.test.js
--watch
./src/not-pjson-main.js
Comment thread
JakobJingleheimer marked this conversation as resolved.
Outdated
```
```sh
node
--watch
--test ./src/foo/*.test.js
--test ./src/bar/*.test.js
./src/not-pjson-main.js
```

##### Without an entrypoint (just paths):

```sh
node
--test ./src/foo/*.test.js
--test ./src/bar/*.test.js
--watch
```
```sh
node
--watch
--test ./src/foo/*.test.js
--test ./src/bar/*.test.js
```

##### _Additional_ watch paths

```sh
node
--test ./src/foo/*.test.js
--test ./src/bar/*.test.js
--watch ./src/**/*.js
```
```sh
node
--watch ./src/**/*.js
--test ./src/foo/*.test.js
--test ./src/bar/*.test.js
```

### Option: Apply `--watch`'s current design to `--test`

`--test` is optional and optionally accepts 1 value, whose value defaults to node's built-in glob defaults (`./src/**/*.test.(c|m)?(j|t)s`, etc). Additional paths are supplied via a new `--test-path` flag:

<table>
<thead>
<tr>
<th>Description</th>
<th>Code sample</th>
<th>Resulting <code>--test</code> value</th>
</tr>
</thead>

<tbody>

<tr>
<td>
Override default
</td>
<td>

```sh
node --test ./src/foo/*.test.js
```

</td>
<td>

`./src/foo/*.test.js`

</td>
</tr>

<tr>
<td>
Override default & Set additional path
</td>
<td>

```sh
node
--test ./src/foo/*.test.js
--test-path ./src/bar/*.test.js
```

</td>
<td>

`./src/foo/*.test.js`
`./src/bar/*.test.js`

</td>
</tr>

<tr>
<td>
Defaults + Additional paths
</td>
<td>

```sh
node
--test
--test-path ./src/foo/*.test.js
--test-path ./src/bar/*.test.js
```

</td>
<td>

`./src/**/*.test.(c|m)?(j|t)s`
~`./src/foo/*.test.js`~
~`./src/bar/*.test.js`~

which reduces to only the default: `./src/**/*.test.(c|m)?(j|t)s`

</td>
</tr>

<tr>
<td>
No defaults & Additional paths
</td>
<td>

```sh
node
--test-path ./src/foo/*.test.js
--test-path ./src/bar/*.test.js
```

</td>
<td>

`./src/foo/*.test.js`
`./src/bar/*.test.js`

</td>
</tr>

</tbody>
</table>