Repository navigation
fix: test optimizer drops library-level annotations (@Skip, @Tags, @Timeout, ...) #1723
Description
Description
The test optimizer imports every test file and wraps its main() in a plain
group('<path>', () { ... }). Annotations that package:test only reads at the
library level (@Skip, @Tags, @Timeout, @TestOn, @OnPlatform) are
never carried over, so under very_good test / very_good dart test:
- a file marked
@Skip(...)runs; - a file marked
@Tags([...])runs even whendart_test.yamlmarks that tag as
skip:, and-x <tag>/--exclude-tagshas no effect on it; - a file-level
@Timeout(...)is ignored and the default 30 seconds applies.
very_good test is the default, optimized path, so a project that relies on
these annotations behaves differently from plain dart test / flutter test
without any warning. The only documented escape hatch is opting the whole file
out of the optimizer with skip_very_good_optimization.
This repository is itself affected: test/ensure_build_test.dart is tagged
pull-request-only (skipped by dart_test.yaml) and carries a 5 minute
@Timeout, yet very_good test -x pull-request-only still runs it with the
default timeout.
Steps To Reproduce
Five files in an empty directory:
# pubspec.yaml
name: optimizer_annotations_repro
environment:
sdk: ^3.13.0
dev_dependencies:
test: ^1.31.2# dart_test.yaml
tags:
slow:
skip: "Only run in CI"// test/fast_test.dart
import 'package:test/test.dart';
void main() {
test('fast', () => expect(1, 1));
}// test/slow_test.dart
@Tags(['slow'])
import 'package:test/test.dart';
void main() {
test('slow', () => expect(1, 1));
}// test/skipped_test.dart
@Skip('not ready')
import 'package:test/test.dart';
void main() {
test('skipped', () => fail('must never run'));
}// test/timeout_test.dart
@Timeout(Duration(milliseconds: 100))
import 'package:test/test.dart';
void main() {
test('slow to finish', () => Future<void>.delayed(const Duration(seconds: 1)));
}Then:
dart pub get
dart test # 1 passed, 2 skipped, 1 failed (the timeout)
dart test -x slow # same, minus the `slow` suite
very_good dart test # 3 passed, 1 failed: `slow` and the timed-out test pass, the @Skip file failsObserved with very_good_cli 1.5.0 on Dart 3.13.2 (macOS):
$ dart test -r expanded
00:00 +0: test/slow_test.dart: (suite)
Skip: Only run in CI
00:00 +0 ~1: test/skipped_test.dart: (suite)
Skip: not ready
00:00 +1 ~2 -1: test/timeout_test.dart: slow to finish [E]
TimeoutException after 0:00:00.100000: Test timed out after 0.1 seconds.
00:00 +1 ~2 -1: Some tests failed.
Failing tests:
test/timeout_test.dart: slow to finish
$ very_good dart test
00:01 +1: slow to finish # @Timeout dropped: 1s test passes under the default 30s
00:01 +2: slow # @Tags dropped: runs despite dart_test.yaml (and despite -x slow)
skipped test/skipped_test.dart (FAILED) # @Skip dropped: file ran and hit fail()
00:01 +3 -1: fast
00:01 +3 -1: Some tests failed.
- test/skipped_test.dart
- [FAILED] skipped
Expected Behavior
very_good dart test matches dart test: the @Skip file is skipped, the
slow tag is honoured (skipped by dart_test.yaml, excluded by -x slow),
and the file-level @Timeout fails the 1 second test. Failing that, the
optimizer should at least surface a warning when it drops an annotation, and
the docs should state the limitation next to skip_very_good_optimization.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
marcossevilla commentedon Sep 16, 2026
hey @peter-trost thanks for opening this issue!
I opened #1735 to address this, you mind taking a look and testing locally to see if this solves the issue? thanks again 👍
peter-trost commentedon Sep 16, 2026
@marcossevilla #1735 fixes the issue for me ✅ Just tested it.