Skip to content

[Tests] Corleone solve helper and ModelingToolkit test suite (smoke, reference solution) (#308) #438

Closed

Description

@ocots

Agents: read the guide for agents in #310 first (order, difficulty, how to claim an issue, working method). Read AGENTS.md and clone .ct-conventions (v0.11.0) and .ct-agents (v0.4.0). Specification: this description and #308.

Sub-issue of #308, part 2 of 3. Depends on #437 (ModelingToolkitBackend and continuous bryson_denham model).


Goal

Implement the Corleone-based solve helper in test/common/solve_modelingtoolkit.jl (keeping Corleone strictly in the test environment, outside the package extension), complete the test/suite/ModelingToolkit/ test suite (test_methods.jl, test_problem.jl, test_smoke.jl, test_reference_solution.jl), and promote bryson_denham from :draft to :valid for ModelingToolkit.

At the end of this issue, bryson_denham(ModelingToolkitBackend()) is verified as a valid ModelingToolkit.System model (dynamics, costs, constraints, boundary conditions), solved with Corleone + Ipopt using the shared :init from metadata, verified against the analytic reference solution (:reference_solution), marked :valid, and all suite/ModelingToolkit and suite/core tests pass.


Technical Findings (from #308 preliminary checks)

  1. Model verification vs. Solver algorithmics:
    Unlike JuMPBackend where we write our own transcription in this repository and must verify that our discrete NLP matches CTDirect (suite/comparison/test_equivalence.jl), ModelingToolkitBackend produces a continuous symbolic model (ModelingToolkit.System), and Corleone is an external solver using direct shooting with ODE integration (OrdinaryDiffEq).
    Because the transcriptions are completely different (shooting vs. collocation) and the algorithmics are internal to Corleone, there is no algorithmic or NLP equivalence test with Corleone.
    What is essential is verifying the model itself:

    • Structure and equations (test_problem.jl): state and control variables, independent variable $t$, dynamics $\dot{x} = f(x, u, t)$, boundary conditions, path constraints, and cost functional.
    • Solvability (test_smoke.jl): verifying that the model can be fed to Corleone without translation or dimensional mismatch and advance at least 1 iteration.
    • Solution accuracy (test_reference_solution.jl): checking that solving with Corleone converges to the analytic reference solution ($J^* = 4.5$ for Bryson-Denham) within discretization tolerance.
  2. Discretizing continuous path constraints for Corleone:
    Corleone.CorleoneDynamicOptProblem(sys, ...) (Corleone/ext/MTKExtension/utils.jl:75) requires every signal call x(τ) in ModelingToolkit.get_constraints(sys) to have a concrete finite real number τ (time isa Real && isfinite(time)), throwing ArgumentError("Signal times must be finite real numbers: x₁(t).") on symbolic t = ModelingToolkit.get_iv(sys).
    In test/common/solve_modelingtoolkit.jl, before calling CorleoneDynamicOptProblem, inspect ModelingToolkit.get_constraints(sys):

    • Keep point constraints (neither lhs nor rhs contains iv = ModelingToolkit.get_iv(sys) as a free variable, e.g. x₁(tf) ~ x1_tf) as-is.
    • Expand any path constraint (whose lhs or rhs contains iv, e.g. x₁(t) ≲ x1_max) by Symbolics.substitute(c, Dict(iv => τ)) for each τ in constraint_grid (filtering out t0 if initial state is fixed), and rebuild a ModelingToolkit.System with the discretized constraints.
  3. Extracting tspan = (t0, tf) from sys:
    Extract (t0, tf) from the Symbolics.Integral domain in ModelingToolkit.get_costs(sys) (DomainSets.infimum / supremum or interval.left / interval.right).

  4. Injecting shared :init (#306) into model.ps:
    In model = Corleone.CorleoneDynamicOptProblem(sys_disc, tspan, ...; switches=control_grid[2:(end - 1)], shooting=shooting_grid, alg=OrdinaryDiffEqTsit5.Tsit5()):

    • model.ps has fields (initialcondition = ..., parameter = ...).
    • Controls (model.ps.parameter.parameters): for N = length(control_grid) - 1 piecewise-constant intervals [t_k, t_{k+1}], evaluate init.control at the interval midpoints (or left endpoints) and write into each control's .ps vector of length N.
    • States (model.ps.initialcondition): in single shooting with fixed initial state (tunable = false), model.ps.initialcondition is empty; in multiple shooting, evaluate init.state at the shooting nodes and write into model.ps.initialcondition (or via Corleone.custom_initialization).
    • Pass the updated ps to Optimization.OptimizationProblem(model, Optimization.AutoForwardDiff(); ps=ps) so opt.u0 uses our shared initial guess.

Mandatory Rules & Conventions (ct-conventions v0.11.0, #310)

  • Worktree per issue (#310 rule 3a): Never edit or switch branches in the main checkout. Use a dedicated worktree (git worktree add ../OptimalControlProblems-<issue> -b feat/<issue>-mtk-corleone-tests main).
  • Separation of modeling and solving: Corleone must not be added to the root Project.toml or ext/ModelingToolkitModels.jl; it belongs only in test/suite/ModelingToolkit/Project.toml and test/common/solve_modelingtoolkit.jl.
  • Qualified imports only (MOD3, MOD4): Always use using Pkg: Pkg, sym. Never use bare using Pkg or import.
  • Fake types at module top-level (TST3): Any helper structs in tests must be defined at module top-level, never inside @testset or functions.
  • No hand-written problem lists (test_no_handwritten_list in test/suite/core/test_consistency.jl): Never write :bryson_denham or "bryson_denham" literals in test/! Always loop with Main.TestStatus.for_each_problem(..., OptimalControlProblems.ModelingToolkitBackend()).

Detailed Implementation Plan

Files to Create or Modify

File Action Details
test/suite/ModelingToolkit/Project.toml Modify Add Corleone (compat = "2"), OrdinaryDiffEqTsit5 ("2"), Optimization ("5"), OptimizationMOI ("1"), Ipopt ("1"), ComponentArrays ("0.15"), ForwardDiff ("1"), OptimalControl ("2.2"), Symbolics, SciMLBase.
test/common/configs.jl Modify Add ModelingToolkit = [(shooting=:single, solver=:ipopt)] to CONFIGS.
test/common/status.jl Modify Add "ModelingToolkit", "Corleone", "OrdinaryDiffEqTsit5" to KEY_DEPENDENCIES.
test/common/solve_modelingtoolkit.jl Create Module SolveModelingToolkit:
- _extract_tspan(sys)
- _build_grids(tspan; grid_size, shooting, control_grid, constraint_grid, shooting_grid)
- _discretize_path_constraints(sys, constraint_grid)
- _apply_initial_guess!(model, sys, init, control_grid, shooting_grid)
- prepare(pb; shooting=:single, grid_size=nothing, ...)
- solve(pb; shooting=:single, solver=:ipopt, grid_size=nothing, max_iter=1000, tol=1e-6, print_level=0, ...)
- outcome(sol, model, reconstruct), settings(pb, cfg; grid_size=nothing), trajectory(sol, model, reconstruct, sys)
test/suite/ModelingToolkit/test_methods.jl Create Unit-test SolveModelingToolkit: _extract_tspan, _build_grids (single & multiple shooting), _discretize_path_constraints, _apply_initial_guess!, settings, outcome, trajectory.
test/suite/ModelingToolkit/test_problem.jl Modify / Complete Test the ModelingToolkit.System model directly: components, dimensions, symbolic dynamics evaluation against expected equations, cost expression, and constraints.
test/suite/ModelingToolkit/test_smoke.jl Create Smoke test every problem in ModelingToolkitBackend() across CONFIGS.ModelingToolkit with grid_size = TestConfigs.SMALL_GRID (20) and max_iter = TestConfigs.SMOKE_MAX_ITER (10): verify that Corleone runs without error and returns finite values.
test/suite/ModelingToolkit/test_reference_solution.jl Create For each problem in ModelingToolkitBackend() with has_reference_solution(problem), solve with Corleone + Ipopt (grid_size=50, tol=1e-6) and verify that the optimal objective matches reference_solution_data(problem).objective within discretization tolerance.
ext/MetaData/bryson_denham.jl Modify Switch ModelingToolkit status from :draft to :valid.

Verification Commands

julia --project=. -e 'using Pkg; Pkg.test(test_args=["suite/ModelingToolkit"])'
julia --project=. -e 'using Pkg; Pkg.test(test_args=["suite/core"])'

Done when

  • test/common/solve_modelingtoolkit.jl discretizes continuous path constraints onto constraint_grid, injects shared :init into model.ps, and solves with Corleone + Ipopt.
  • test_methods.jl, test_problem.jl, test_smoke.jl, and test_reference_solution.jl all pass in suite/ModelingToolkit.
  • bryson_denham status for ModelingToolkit is :valid in ext/MetaData/bryson_denham.jl.
  • suite/core passes (including test_consistency.jl and test_no_handwritten_list).

Refs #308, #310

Activity

  1. changed the title [-][Tests] Corleone solve helper and ModelingToolkit test suite (smoke, equivalence, reference solution) (#308)[/-] [+][Tests] Corleone solve helper and ModelingToolkit test suite (smoke, reference solution) (#308)[/+] on Nov 8, 2026
  2. added a commit that references this issue on Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

featureNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions