Repository navigation
Description
Written by Claude (Claude Code) at the request of ocots.
Part of #310. Follows #327 (changed problems), #341 (what a run tested), #360 (convergence off by default). Replaces the precompilation of #373 (closed, not planned) as the way to shorten the CI.
Why
On a pull request, the CI should test what the pull request changes, and labels should only force more. Today (tools/ci/scope.py):
- the problems are narrowed automatically only when every changed file is a per-problem file; any other file runs every problem;
- the environments are never chosen automatically: without a scope label every environment runs. Changing
ext/JuMPModels/schemes/build.jlortest/suite/JuMP/runs OptimalControl, OptimalControlScalarized and comparison too; changingdocs/runs everything; - the scope labels restrict (
test JuMP backendruns JuMP only): without a label, everything; with a label, only what it names. That is the opposite of "automatic unless overridden".
Scope
1. Automatic scope: environments × problems, from the changed files
Each changed file gives a set of environments and a set of problems; the run is the union over the files. A file that is not understood, a removed or renamed file, a truncated or unreadable file list, the release label or an event other than a pull request run everything, as today: narrowing happens only on positive evidence.
The rules are computed from the repository, not written as a list of paths: the environments are the directories of test/suite/ with a Project.toml; the modeling of ext/<M>Models/ is the environment M; the files that a test file includes (include(...)) are read from the sources.
| Changed file | Environments | Problems |
|---|---|---|
ext/<M>Models/<p>.jl (a problem of a modeling) |
M, comparison |
p |
ext/<M>Models/** other (e.g. schemes/) |
M, comparison |
all |
ext/<M>Models.jl (entry point of an extension) |
every environment of the modelings it loads, comparison |
all |
ext/MetaData/<p>.jl, ext/Descriptions/<p>.md |
all | p |
test/suite/<env>/** |
<env> |
all |
test/common/<f>.jl |
the environments whose tests include <f>, directly or not |
all |
test/cron/**, test/robustness/** |
core only (not run on a pull request) |
— |
docs/**, *.md outside ext/, tools/cron/** |
core only |
— |
tools/ci/** |
core only (its unit tests run in the scope job) |
— |
src/**, Project.toml, test/runtests.jl, test/Project.toml, .github/workflows/CI.yml, anything else |
all | all |
The arguments of test/runtests.jl are one product, environments × problems: two files ext/JuMPModels/a.jl and ext/OptimalControlModels/b.jl give JuMP OptimalControl comparison problem=a,b, slightly more than needed, accepted.
core always runs (it does today).
2. Labels add, they never restrict
test <M> backend,test comparison: add the environment to the automatic scope, with all its problems.test all problems: every problem, in the environments of the automatic scope and of the labels.test everything(new label): every environment and every problem.test convergence: unchanged (adds the convergence report).run ci github-runner: unchanged (the CI runs only with it).
test/README.md, the comments of CI.yml and the job summary say this.
Labels or a comment command?
Kept: labels. They are already wired (adding or removing one reruns the CI), only maintainers can set them (protects the runners), and they stay visible on the pull request. A comment command (/test JuMP all) needs an issue_comment workflow, a permission check of its author, the head commit of the pull request, and its state is lost in the thread; checkboxes in the description rerun the CI on every edit. With an automatic scope, labels become rare.
Out of scope: the GPU environment (self-hosted runners, its own labels), the crons (they test everything of their scope), #344 (the full CI matrix).
Done when
-
tools/ci/scope.pycomputes environments × problems from the changed files with the rules above; unit tests intools/ci/test/test_scope.pyfor every row of the table and for the fallbacks, each shown to fail on a broken rule. - Labels add to the automatic scope;
test everythingexists and runs everything. - The job summary shows, for each file, the environments and problems it adds.
- Shown on this pull request: its own scope (a change of
tools/ci/and docs givescore), then withtest JuMP backendand withtest everything. -
test/README.mdand the comments ofCI.ymlup to date.
Activity
ocots commentedon Nov 7, 2026
Taken by an agent; plan to follow, then a draft PR (ct-conventions v0.10.0, ct-agents v0.4.0).
ocots commentedon Nov 7, 2026
Written by Claude (Claude Code) at the request of ocots.
Two corrections to the description, found while writing the plan:
test/runtests.jlare one product, environments × problems, so a label cannot add an environment "with all its problems" while another environment keeps a narrowed list. A scope label therefore adds the environment with the problems of the automatic scope;test all problems(ortest everything) widens the problems..github/workflows/CI.yml(its comments), which runs everything by the rules. The rules are shown instead by the unit tests, and by runningscope.pyon the file lists of a few recent pull requests (test: NLP equivalence of the scalarized OptimalControl models (#399) #400, test: compare the JuMP schemes on problems chosen by their features (#401) #402, feat: one initial guess per problem, in its metadata, shared by every modeling (#377) #395), with their output pasted in the PR.ocots commentedon Nov 7, 2026
Done in #406, ready for review. The label
test everythingis created.Merge pull request #406 from control-toolbox/feat/405-ci-scope