Repository navigation
Description
Agents: read the guide for agents in #310 first (order, difficulty, how to claim an issue, working method). The specification is in this description and in the comments below.
Updated on 2026-11-06 after #360 (convergence is reported, not tested) and #375 (reference objectives); results now come from the convergence cron. Where an older comment contradicts this description, the description wins; what changed is listed in the update comment.
Goal
Separate the computations shown in the documentation (solving every problem, plots), which are long and may fail, from the documentation build, which must be fast and never fail because of a problem. Jobs compute results and store them; the documentation only reads what exists; each problem page shows the results if they exist and are up to date, says so otherwise. Problem pages are reached through a gallery instead of a list in the sidebar.
One source of results: the convergence cron (decision of 2026-11-06). Since #360 and #375, the weekly cron of each scope already solves every problem, in every modeling and configuration, on its default grid, and publishes the results on the results branch (*-results.tsv: solver status, objective, iterations, time, gap to the reference). The documentation results come from these runs, not from a second computation: the cron jobs also write result.json, the plots and the environment of the run (below). No separate computation workflow to keep in sync with the tests.
Part of #307. Related: #305 (crons write their reports to the same branch), #306 (features for the gallery filters), #134 (visualize), #311 (VitePress).
Storage: a results branch
- The orphan branch
results(exists since [Tests] Crons per scope for the slow tests, with reports #305:cron/with the reports andstate.json; protected against deletion by the rulesetprotect-results), rewritten with a single commit on each update so that it does not grow. Not a tracked directory inmain(regenerated SVGs would bloat the history). - Written only by the publish job of the crons (
.github/workflows/cron-reusable.yml, concurrency groupcron-results). - Locally: a git-ignored directory (e.g.
docs/results/) filled by a script, so the documentation also builds offline and in previews.
results/ # branch `results`
├── manifest.json # date, commit, list of runs
├── env/<run id>/ # environment of each computation run
│ ├── Project.toml, Manifest.toml
│ ├── versioninfo.txt # output of versioninfo()
│ └── status.txt, status-manifest.txt # Pkg.status(), Pkg.status(mode=PKGMODE_MANIFEST)
├── problems/<problem>/
│ ├── result.json # per modeling and configuration: solver status, objective, iterations, time, gap to the reference; fingerprint; run id
│ ├── state.svg, control.svg, costate.svg
│ ├── visualize.svg # if the problem has a visualisation (#134)
│ └── thumbnail.svg # for the gallery
└── cron/ # reports of the crons (#305)
Robustness
-
One writer. The test jobs of the crons run in parallel (per scope and platform) and only upload workflow artifacts; the publish job collects them and makes the single commit (as today for the reports). No concurrent writes; a failing problem does not prevent the others from being published. Plots need
Plotsin the cron environments (or a step after the solves): to decide when taking the issue. -
Freshness. Each
result.jsonrecords a fingerprint: a hash of the problem files (metadata, model files of every modeling, description) and the versions of the packages used. At build time the documentation recomputes the fingerprint and shows, per problem:- up to date: the results;
- stale: the results, marked "computed for commit X";
- missing: "results not computed yet".
The build never fails on missing or stale results.
-
Triggers of the computations: those of the crons (weekly per scope,
workflow_dispatch); to add if needed: a push tomainthat changes a problem (only problems whose fingerprint changed), through the same cron workflows withproblem=…. The documentation workflow never triggers or waits for them. -
VitePress: copy the results into
docs/src/public/results/beforemakedocs(VitePress silently drops files it does not recognise outsidepublic/, see the notes on the VitePress backend in ct-conventionsrules/site.md).
Problem pages
Generated, never hand-written (docs/problems.jl today):
- description (
ext/Descriptions/<p>.md), parameters, features ([Metadata] Computed features, reference objective, shared initial guess #306), status per modeling ([Dev] Per-modeling lists of problems, discovered automatically, with a status per modeling #303), references; - how to get and solve it in each modeling where it exists (short code, not executed beyond trivial calls);
- results from
result.json, with the freshness badge: solver status and gap to the reference per configuration, no ✓ or ✗ on convergence ([Tests] Report convergence instead of testing it; keep hard checks for breakage and model equivalence #360, [Doc] Rewrite the documentation: getting started, user guide, problems, developers #307); plots as SVG (ruleSITE12),visualize.svgwhen it exists; - Reproducibility block, same presentation as the OptimalControl.jl documentation (
docs/src/index.md, section "Reproducibility"): a table to downloadProject.tomlandManifest.tomlof the run that produced the results, and a collapsible::: detailsblock withversioninfo,Pkg.status()and the full dependency tree, read fromenv/<run id>/(they cannot be executed at build time, since the results come from another environment).
No heavy computation at documentation build time: the only code executed by the build is short (tutorials, trivial calls).
Gallery
Replaces the list of problems in the sidebar, since the collection will grow quickly:
- one card per problem: thumbnail (
thumbnail.svg, or the visualisation), name, modelings and statuses, tags; - filters by modeling, status, tag and features (dimensions, free final time, constraints);
- a Vue component (see
docs/src/components/in CTBase.jl), with the existing interactive table (docs/browser.jl) as a second view of the same page, or replaced by it.
Done when
- The cron workflows write
result.json, the plots andenv/<run id>/for each run, through the existing publish job; branchresultsrewritten with one commit. - Fingerprint computed identically by the jobs and by the documentation build; up to date / stale / missing shown on the pages, shown on purpose for each case.
- Local script filling
docs/results/; documentation builds with no results at all. - Problem pages generated with results, plots, visualisation and Reproducibility block.
- Gallery page with filters; problems removed from the sidebar list.
- Documentation build time reported before and after.
Activity
ocots commentedon Nov 4, 2026
Reference for the Reproducibility block: the dev documentation of OptimalControl.jl, https://control-toolbox.org/OptimalControl.jl/dev/ (home page, section "Reproducibility"; source
docs/src/index.mdanddocs/make.jlonmain). The stable site is not up to date.ocots commentedon Nov 4, 2026
Every modeling handled generically (OptimalControl, OptimalControlScalarized, JuMP, and Corleone later, #308):
docs/): display name; the short code to get and solve a problem (the::: code-grouptab of the problem page); how its result is computed and written toresult.json([Doc] Results pipeline for the documentation, and a gallery of problems #314); how its solution is plotted (aCTModels.Solutionwhen there is one, see [Paper] Port to OptimalControl v2 and new problem interface #283; otherwise what is available).ocots commentedon Nov 4, 2026
Correction: where this issue says "Corleone" as a modeling, read ModelingToolkit (
ModelingToolkitBackend, #308): the problem is a ModelingToolkit system, and the documentation adapter shows how to get it and solve it with Corleone. Installation page: ModelingToolkit for the modeling, Corleone to solve it.ocots commentedon Nov 4, 2026
Documentation (decision in #307): this issue also writes the README of the results pipeline (how results are computed, the
result.jsonformat, theresultsbranch, the documentation adapter of a modeling), placed next to the code of the pipeline. Procedures only. Add it to the 'Done when' checklist.ocots commentedon Nov 6, 2026
Update: results come from the convergence cron (decision of the maintainer, 2026-11-06)
Principle since #360: three kinds of tests. Does not break (every CI run, short) and model equivalence (every CI run, no solve) can fail; the convergence report (cron, or label
test convergence) gives the solver status and the relative gap of the objective to the reference objective (#375), never fails, no tolerance.Since #360 and #375 the weekly cron of each scope solves every problem in every modeling and configuration on its default grid, and publishes the results on the
resultsbranch. A separate computation workflow for the documentation would duplicate it. The description is updated:result.json, the plots andenv/<run id>/; the existing publish job makes the single commit.result.jsonholds the solver status and the gap to the reference; the pages show it without ✓/✗ on convergence ([Doc] Rewrite the documentation: getting started, user guide, problems, developers #307).resultsbranch exists (cron/,state.json) and is protected against deletion (rulesetprotect-results).problem=….Plotsin the cron environments, or a step after the solves.🤖 Generated with Claude Code
ocots commentedon Nov 7, 2026
Written by Claude (Claude Code) at the request of ocots.
Note from #307 (draft PR #408), for whoever takes this issue: the problem pages are now generated by
docs/problems.jlwith one tab per modeling and a Compare tab, and already show the results per configuration fromext/MetaData/references/<problem>.toml. What remains here: plots (SITE12is set indocs/make.jl), the freshness badge, the Reproducibility block per problem, and the gallery, which replaces the sidebar list of the problems. A modeling tab is a<ProblemPanel id="<modeling>">in the generated page: the results and plots of #314 go there.🤖 Generated with Claude Code
ocots commentedon Nov 7, 2026
Taken by an agent. Written by Claude (Claude Code) at the request of ocots.
Plan: results pipeline and gallery (#314)
Conventions: ct-conventions v0.11.0; ct-agents v0.4.0. From
main0f6b1ec.What and why
The problem pages (#408) already show the objective of every configuration, read from
ext/MetaData/references/<problem>.toml. What #314 still adds: plots of the solutions, thefreshness of the results, the Reproducibility block of the run, and a gallery instead of the
sidebar list. The source stays the convergence crons, which already publish to the
resultsbranch and, since #412 (#409 part 1), write
<report>-results.tsv(how each solve wasconfigured) and
<report>-environment.toml(versions, platform, commit) for every run.Decisions (agreed with the maintainer, 2026-11-07)
(
problems/<problem>/<modeling>/<configuration>.json: time grid, state, control, costate,on the solve's grid, downsampled to at most ~500 points). The pages draw them in the
browser (Vue component), interactively: the reader chooses the curves, the
configurations and the modelings to overlay. Julia computes, it does not draw.
Plotsin the cron environments, no Julia in the publish job.result.jsonper problem, merged per scope. Each cron scope publishes on its own, sothe publish job reads the existing
problems/<problem>/result.json, replaces the entries ofits scope, and keeps the others; same single-commit rewrite as today.
package (
src/), used by the tests (written with each result) and by the documentation:SHA-256 of the files of the problem (metadata, model files, description). Versions are
recorded and shown, not compared: the docs environment is not the crons'. The publication
extends
tools/cron/report.py, which already publishes the reports; no second publisher.Phases
A — The crons record trajectories and the environment of the run
test/common/status.jl:result!takes the solution;write_reportwrites the trajectories(
test/reports/trajectories/…json) next to the.tsv.test/common/run_environment.jl: also copiesProject.toml,Manifest.toml,versioninfo(),Pkg.status()andPkg.status(mode=PKGMODE_MANIFEST)of the test environment intotest/reports/env/(Reproducibility block).src/:_fingerprint(problem), written into each result entry.problem=beam,convergence) produces the files; a test of thefingerprint shows it changes when a model file changes.
B — The publish job writes
problems/andenv/on theresultsbranchtools/cron/report.py, extended: from the downloaded artifacts, also writesproblems/<problem>/result.json(entries of the scope replaced),the trajectories,
env/<run id>/, andmanifest.json(date, commit, runs)..github/workflows/cron-reusable.yml: one more step inpublish, before the commit.tools/cron/test/.problem=beam), theresultstree inspected.A dispatched cron updates the cron issues (test: record how each solve was configured and the environment of the run (#409, part 1) #412 notes it): run it on one problem only.
C — The documentation reads the results
docs/results.jl: fills the git-ignoreddocs/results/from theresultsbranch, copiedto
docs/src/public/results/; the build works without it ("results not computed yet").docs/problems.jl, in each modeling tab and in Compare: an interactive plot component(
ProblemPlot.vue) reading the trajectories, the freshness badge (up to date / stale for commit X / missing), and the Reproducibilityblock of the run (
::: details, downloads served fromdocs/src/public/).D — Gallery
docs/src/components/ProblemsGallery.vue: one card per problem (thumbnail drawn from thetrajectories, name, modelings and statuses, features); filters by modeling, status and
features; the browser table as a second view of the same page.
tools/cron/report.py(procedures only).Out of scope
Typed references and reference solutions (#409 part 2, in progress elsewhere:
tools/referencesand
ext/MetaData/referencesare not touched);visualize(#134); JuMP costate signs (#326).Human checkpoints
🤖 Generated with Claude Code
Merge pull request #417 from control-toolbox/feat/314-record-trajecto…
Merge pull request #419 from control-toolbox/feat/314-publish-results
Merge pull request #420 from control-toolbox/feat/314-docs-results
Merge pull request #422 from control-toolbox/feat/314-gallery
Merge branch 'main' into chore/413-src-submodules
ocots commentedon Nov 8, 2026
Done — merged in four PRs: #417 (record trajectories), #419 (publish results on
results), #420 (problem pages read the results), #422 (gallery).Done when
result.json, the trajectories andenv/<run id>/for each run, through the existing publish job; branchresultsrewritten with one commit — verified on the manual runs of the three scopes (JuMP, OptimalControl, OptimalControlScalarized).up to date/stale/missingshown on purpose —missingeverywhere until the first runs,currentonce published,staleon every page now since refactor: split src into submodules, keeping the public API (#413) #421 changed the model files (the next weekly runs refresh tocurrent).docs/results/(docs/results.jl); documentation builds with no results at all.0f6b1ec, local build) vs 111 s after (docs: a gallery of the problems (#314, phase D) #422).Deviations from the description, as decided along the way:
visualize.svgis left to [General] Visualisation #134 (per the plan).tools/cron/report.py: the pipeline is documented in.github/workflows/README.md, next to the crons, to avoid a second place to keep up to date.state.svg/control.svg/costate.svg/thumbnail.svgare replaced by the trajectory JSONs andresults/thumbnails.json.