Skip to content

fix(mcp): test passes timeout_seconds and check_ignore to runners that reject them #1779

Open

Description

@deremer

Description

The MCP test tool builds a command line from its arguments without checking which runner will receive it. Two arguments are accepted by the tool schema but rejected by one of the two runners, so the call fails with exit 64 before any test runs.

Tool arguments Command built Result
dart: true, timeout_seconds: 180 very_good dart test --timeout 180 -- … Could not find an option named "--timeout".
check_ignore: true on a Flutter package very_good test --coverage --min-coverage 100 --check-ignore … Could not find an option named "--check-ignore".

The cause is visible on main (8c772f3):

  • _parseTest in lib/src/mcp/mcp_server.dart appends --check-ignore and --timeout unconditionally, whichever runner dart selects.
  • --timeout is registered only on very_good test (lib/src/commands/test/test.dart).
  • --check-ignore is registered only on very_good dart test (lib/src/commands/dart/commands/dart_test_command.dart).
  • Neither the timeout_seconds nor the check_ignore schema description says it is runner-specific.

This bites agents twice: an agent told to "always set timeout_seconds" cannot test a pure Dart package, and an agent told to pass check_ignore cannot test a Flutter one. The plugin's green-gate skill currently instructs both (VeryGoodOpenSource/vgv-ai-flutter-plugin#178).

Steps To Reproduce

  1. Call the MCP test tool on a pure Dart package with {"dart": true, "timeout_seconds": 60}.
  2. Observe partial_failure / validation: Could not find an option named "--timeout".
  3. Call the MCP test tool on a Flutter package with {"check_ignore": true}.
  4. Observe partial_failure / validation: Could not find an option named "--check-ignore".

Expected Behavior

One of the following, in order of preference:

  1. Each argument works on both runners: very_good dart test gains --timeout (ideally the wall-clock kill from fix(mcp): test timeout_seconds does not kill a hung test run, which blocks every later tool call #1765 / fix(mcp): enforce wall-clock timeout for test runs #1766) and very_good test gains --check-ignore.
  2. Otherwise, the handler validates combinations before running and returns a validation failure that names the argument and the runner it applies to, and the schema descriptions say which runner each argument supports.

Additional Context

Activity

  1. added
    bugSomething isn't working as expected
    on Oct 19, 2026
  2. ananya-kaul commented on Oct 30, 2026

    @ananya-kaul

    # fix(mcp): validate check_ignore before running Flutter tests

    ## Problem
    The MCP test tool forwards check_ignore: true to the Flutter test command, which does not support --check-ignore. The command exits with code 64 before running any tests.

    ## Change
    Reject this combination before invoking the command runner, with a structured validation error explaining that check_ignore requires dart: true. Flutter callers can omit the option or set it to false. Valid Dart calls still forward the flag.

    Addresses the remaining check_ignore portion of #1779. The timeout portion is already handled upstream; this does not implement coverage-ignore support in Flutter.

    ## Validation

    • Two regression cases fail before the change: omitted dart and dart: false.
    • Four added cases cover invalid Flutter requests and disabled flags for both runners. Existing all-arguments test covers enabled Dart behavior.
    • MCP tests: 85 passed.
    • Strict static analysis and formatting checks passed.
    • Unit suite: 678 passed, 1 skipped (pull-request-only and e2e excluded).
    • MCP server executable line coverage: 169/169, including the new validation branch.
    • Tested on macOS arm64 with Dart 3.13.7.
  3. unicoderbot commented on Oct 30, 2026

    @unicoderbot
    Contributor

    Great, I will work on a PR and push it soon.

    • Preparing environment
      • Cloning repository
      • Setting up dev environment
    • Working on changes
    • Reviewing changes
    • Opening pull request

    I've opened a pull request: #1791

  4. moved this from Needs Triage to In Progress in VGV Open Source 🦄 🧙🌟on Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working as expected

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions