Gradient fit aborts on incidental non-differentiable actions (unscored parameter_scan / non-ODE simulate)

[wshlavacek]

Summary

On a gradient-based fit (fit_type = trf / lbfgs), PyBNF enables a sensitivity request on the whole model, and any action that cannot carry forward sensitivities aborts the entire fit — even when that action's output is not a scored gradient target.

Triggers (both raised during model.execute())

• Carried-state scan (simulate() → parameter_scan() with no reset between them) → PybnfError at pybnf/bngsim_model/net_model.py:1178 (_scan_carried_state).
• Non-ODE-method action (ssa / psa / nfsim) → PybnfError at pybnf/bngsim_model/net_model.py:309-316 (_sensitivity_request_kwargs).

Both guards fire whenever self._sensitivity_request is not None — i.e. on every gradient fit — regardless of whether the offending action's output is actually scored against data.

Why this is a bug

A model whose fit target is a differentiable ODE time course, but which also carries an incidental action (a stochastic diagnostic simulate, or a pre-equilibrated simulate→parameter_scan block) that is never scored, cannot be gradient-fit at all. The unscored action needs no sensitivities, yet it aborts the run.

This also contradicts the module's own documented intent in pybnf/algorithms/optimizers/gradient_base.py:320-328:

a non-ODE simulation method (SSA / NFsim) is likewise non-differentiable, but the method is an action-level property (a model can mix actions) rather than a model-structure one, so it is not hoisted here; it keeps its existing clean per-evaluation refusal

The intent is that only sensitivity-bearing (scored) actions should require differentiability. The implementation over-reaches by forcing a sensitivity request onto every action in the model.

Proposed fix

Gate sensitivity-enabling per action on whether that action's output is a scored gradient target. The scoring↔action mapping is already available — gradient_base.py:447-452 intersects res.simdata with exp_data during gradient assembly. Run incidental / unscored non-differentiable actions on the ordinary path with sensitivities off, so they neither compute wasted sensitivity tensors nor abort the fit.

Reproduction sketch

Edition-2 gradient fit (fit_type = trf or lbfgs). Model has:

• one scored ODE simulate action mapped to time-course data, plus
• one unscored action that is either non-ODE (nfsim / ssa) or a carried-state simulate→parameter_scan block.

The fit aborts at the first evaluation with the PybnfError from one of the guards above, even though the scored objective is fully differentiable.

[wshlavacek]

Fixed in eab02fca405d (pushed to main).

What was wrong

A gradient fit enables a forward-sensitivity request on the whole model in _setup_gradient_path, and both guards fired on every action whenever self._sensitivity_request is not None — regardless of whether that action's output was a scored gradient target:

• _sensitivity_request_kwargs (net_model.py) raised for any non-ODE method (ssa/psa/nfsim);
• _scan_carried_state (net_model.py) raised for a carried-state pre-equilibration parameter_scan (Edition-2 experiment: can't express a pre-equilibration → intervention → dose-scan protocol (parameter_scan+preequil refused; no species setConcentration) — a published fit (Erickson 2019 IGF1R) is legacy-only #474 shape).

So a model whose fit target is a differentiable ODE time course, but which also carried an incidental unscored action, aborted at the first evaluation.

How it was cleared

Both guards now gate on whether the action's output is a scored gradient target, rather than on the mere presence of a model-wide request:

• BngsimModel.set_scored_suffixes() records the model's scored exp_data suffixes. _action_bears_sensitivities() is true only on the gradient path and when the current action's suffix — folded with this instance's mutant/condition offset — is in that set. It returns False on the scalar path, so scalar carried-state scans run unguarded exactly as before.
• A scored non-ODE / carried-state action still refuses cleanly (its gradient genuinely cannot be supplied); an unscored one runs on the ordinary sensitivity-free path — no wasted sensitivity tensor, no abort. This matches the module's documented intent quoted in the issue.
• ODE actions stay always-bearing, so the persistent-simulator sensitivity continuity across carried states ([#385 layer J] Gradient through pre-equilibration / steady-state (blocked: bngsim sensitivities) #457) is untouched; an unscored ODE action's unused tensor is left harmless rather than risk that continuity.
• GradientOptimizer._setup_gradient_path declares each model's scored suffixes (rides the scatter alongside the request); the current-action suffix is threaded in _prepare_simulate_run / _run_parameter_scan, and the mutant suffix offset is set in _get_mutant_model_bngsim so condition: experiments key correctly.

Verification

6 new tests in tests/test_bngsim_output_sensitivities.py reproduce the issue's two triggers at the model.execute() layer where they were raised: an unscored SSA diagnostic beside a scored ODE course (both orderings), unscored and scored carried-state scans, the mutant-offset keying, the kwargs helper's three cases, and scalar-path safety. Green across the gradient/routing/assembly/pre-equilibration suites plus the real-bngsim trf/lbfgs decay recovery and the scalar carried-state dose-scan recovery.