Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Very Good CLI is a Dart command-line tool by Very Good Ventures for generating s

```bash
# Install dependencies
dart pub get && cd bricks/test_optimizer && dart pub get && cd ../../
dart pub get && cd bricks/test_optimizer/hooks && dart pub get && cd ../../../

# Run unit tests (excludes expensive pull-request-only and e2e tests)
flutter test -x pull-request-only -x e2e
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ To develop for Very Good CLI you will need to become familiar with Very Good Ven
very_good packages get -r

# Or get project dependencies manually
dart pub get && cd bricks/test_optimizer && dart pub get && cd ../../
dart pub get && cd bricks/test_optimizer/hooks && dart pub get && cd ../../../
```

3. Run all Very Good CLI tests:
Expand Down Expand Up @@ -82,7 +82,7 @@ Where `<branch-name>` is an appropriate name describing your change.
very_good packages get -r

# Or get project dependencies manually
dart pub get && cd bricks/test_optimizer && dart pub get && cd ../../
dart pub get && cd bricks/test_optimizer/hooks && dart pub get && cd ../../../
```

3. Ensure you have a meaningful [semantic][conventional_commits_link] commit message.
Expand Down
4 changes: 1 addition & 3 deletions bricks/test_optimizer/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,7 @@ dart pub get
dart test

# 💻 Run `very_good test` end to end tests (from e2e/)
dart test test/src/commands/test/async_main_test.dart &&
dart test test/src/commands/test/no_project_test.dart &&
dart test test/src/commands/test/spaced_golden_file_name.dart
dart test test/commands/test --run-skipped
```

If not all test passed out of the box please submit an [issue](https://github.com/VeryGoodOpenSource/very_good_cli/issues/new/choose) so it can get fixed.
Expand Down
31 changes: 28 additions & 3 deletions bricks/test_optimizer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,40 @@ The above command will generate a `.test_optimizer.dart` in the `test` directory
// GENERATED CODE - DO NOT MODIFY BY HAND
// Consider adding this file to your .gitignore.

import 'package:test/test.dart';

import 'app/view/app_test.dart' as _a;
import 'counter/cubit/counter_cubit_test.dart' as _b;
import 'counter/view/counter_page_test.dart' as _c;

void main() {
group('app_view_app_test_dart', () { _a.main(); });
group('counter_cubit_counter_cubit_test_dart', () { _b.main(); });
group('counter_view_counter_page_test_dart', () { _c.main(); });
group('app/view/app_test.dart', () { _a.main(); });
group('counter/cubit/counter_cubit_test.dart', () { _b.main(); });
group('counter/view/counter_page_test.dart', () { _c.main(); }, tags: ['slow']);
}
```

## Annotations

`package:test` reads `@Skip`, `@Tags`, `@Timeout`, `@TestOn`, `@OnPlatform` and
`@Retry` only from the metadata of a file's first directive, which a bundled
file no longer has. The pre-gen hook parses each file and forwards those
annotations to its `group` call, as `tags` is forwarded above, so a bundled file
behaves the way it does under plain `dart test`.

Arguments are forwarded as source text into a file that imports only
`package:test` and `dart:core`, so an annotation naming anything else, such as
`@Timeout(kSlowSuite)`, is left out with a warning instead of breaking the
bundle. `@TestOn` cannot save a file that imports something platform-specific,
since every optimized file is imported regardless of platform.

Tag a file with `skip_very_good_optimization` to keep it out of the bundle. The
tag needs a directive such as `library;` beneath it, the only place
`package:test` reads metadata from:

```dart
@Tags(['skip_very_good_optimization'])
library;
```

[1]: https://github.com/felangel/mason
2 changes: 1 addition & 1 deletion lib/src/cli/templates/test_optimizer_bundle.dart

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 23 additions & 2 deletions site/docs/commands/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,12 +100,33 @@ Golden tests are tests that compare the output of a test to a "golden" file. If
:::info For an example on specifying a custom [`GoldenFileComparator`](https://api.flutter.dev/flutter/flutter_test/GoldenFileComparator-class.html) that accepts a certain amount of difference (toleration threshold), refer to the [`goldenFileComparator` Flutter documentation](https://api.flutter.dev/flutter/flutter_test/goldenFileComparator.html).
:::

### File-level annotations

`package:test` reads `@Skip`, `@Tags`, `@Timeout`, `@TestOn`, `@OnPlatform` and `@Retry` only from the metadata of a file's first directive. The optimizer forwards them to the group that wraps each bundled file, so an optimized run behaves the way the same suite does under plain `dart test` or `flutter test`.

```dart
@Tags(['integration'])
@Timeout(Duration(minutes: 5))
library;

import 'package:test/test.dart';
```

The file above keeps its five minute timeout inside the bundle, is skipped when `dart_test.yaml` marks `integration` as `skip:`, and is left out by `--exclude-tags integration`.

Arguments are forwarded as source text into a file that imports only `package:test` and `dart:core`, so an annotation naming anything else, such as `@Timeout(kSlowSuite)`, is left out with a warning instead of breaking the bundle. Inline the value to keep the annotation.

`@TestOn` cannot save a file that imports something platform-specific, such as `dart:html`, since every optimized file is imported regardless of platform. Keep those files out with the `skip_very_good_optimization` tag or `--exclude-optimization`.

### Skip optimization for specific tests

By default, all tests run with optimizations enabled; use the `--no-optimization` flag to disable globally, or add the `skip_very_good_optimization` tag to specific test files to disable them individually.

```js
The tag needs a directive such as `library;` beneath it, the only place `package:test` reads metadata from.

```dart
@Tags(['skip_very_good_optimization'])
library;

import 'package:test/test.dart';
```
Expand All @@ -126,7 +147,7 @@ test:

Globs are matched against each test file's path relative to the package root, and they match everything nested underneath what they name, so `test/integration` excludes every test in that directory. See [`optimization`](../configuration.md#optimization) for the glob syntax details.

Excluded files are left out of the optimized bundle but still run, each as its own test suite. Because they run as their own suites, their file-level `@Tags` are honored again, so combining `--exclude-optimization test/integration` with `--exclude-tags integration` skips a suite annotated with `@Tags(['integration'])`.
Excluded files are left out of the optimized bundle but still run, each as its own test suite, in their own isolate. Exclude a file when it cannot share an isolate with the rest of the suite, for example because it mutates global state or installs its own bindings. File-level annotations do not need an exclusion, as the bundle forwards them.

Prefer narrow globs. Every excluded file becomes its own suite with its own VM startup, so excluding hundreds of files gives up most of what the optimization buys you.

Expand Down
3 changes: 1 addition & 2 deletions site/docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,8 +135,7 @@ so `test/**/serial_*_test.dart` matches both `test/serial_a_test.dart` and
`test/nested/serial_a_test.dart`. Matching is case-sensitive on every
platform.

Excluded files still run, each as its own test suite, which also means their
file-level `@Tags` are honored again. See
Excluded files still run, each as its own test suite, in their own isolate. See
[Skip optimization for specific tests](commands/test.md#skip-optimization-for-specific-tests).

With `--recursive`, the closest `very_good.yaml` applies to every package in the
Expand Down
Loading