feat: add exact and heuristic probability SVGs (steps 8–11)

.gitignore

@@ -55,7 +55,7 @@ doxygen-doc
 doxygen-doc/*
 conf3108*
 test-driver
-
+tmp/*
 
 # temp dir
 tmp

ChangeLog

@@ -14,6 +14,28 @@
 
 ## Interface and handling
 
+- orient probability SVGs from the upper-left corner and highlight every MFE
+  pair, splitting seed/MFE cell borders into orange and green (PR #259 review)
+
+- enable approximate actual-pair CSV/SVG probabilities for unseeded and seeded
+  heuristic ensembles, labelled explicitly as approximate (PR #259, step 11)
+
+- reject nonfinite partition totals and finite-energy boundary underflow before
+  publishing exact unseeded or heuristic pair probabilities (PR #259 review)
+
+- enable base-pair CSV/SVG probabilities for exact unseeded model P/mode M
+  predictions (PR #259, step 10)
+
+- revise SVG axes, sequence labels and rounded tooltips; show intramolecular
+  pairing (1-Pu) frames and the best interaction region (PR #259 review)
+
+- BUGFIX : use each strand's accessibility energy scale for SVG frames and
+  unpaired-probability exports, including mixed computed/imported inputs
+
+- --out=bpsvg:FILE renders actual-pair probabilities with nucleotide/accessibility
+  frames, a shared CSS palette, signed-coordinate guides, hover values and admitted
+  seed outlines (PR #258, step 8)
+
 - --out=bpProb:FILE writes actual-pair conditional probabilities for exact
   model P/mode M stack-seed ensembles, with disjoint-region merging, original
   coordinates and success-only publication (issue #257, implementation step 6)
@@ -46,6 +68,15 @@
 
 ## Technical changes and Optimizations
 
+- document independent release/debug, installed-API and performance validation
+  for the reviewed unseeded and heuristic probability outputs (PR #259)
+
+- transfer completed probability-output buffers without copying; independently
+  audit performance, accessibility conversion and recurrence depictions (PR #259)
+
+- depict ensemble predictor recurrences and the exact base-pair inside/outside
+  computation in internal SVG documentation (PR #258, step 9)
+
 - validate exact pair probabilities on independent mathematical/native oracles,
   release/debug suites and installed consumers; document measured runtime and
   storage, and supply the required logging link dependency (issue #257, step 7)
@@ -156,6 +187,99 @@ energies, restricted partition sums, and trackers.
 ################################################################################  
 ################################################################################
 
+261012 Alexander Mitrofanov
+ * BasePairProbabilitySvg, BasePairProbabilityWriter, CommandLineParsing
+   * restore top-to-bottom target indexing, aligned frames/guides and MFE region
+   + reconstruct the MFE structure inside the best ensemble site for SVG only
+   + highlight each MFE pair; shared seed pairs use orange vertical and green
+     horizontal edges, preserving both annotations (PR #259 review)
+ * testBasePairProbabilitySvg.py, README.md, seeded-probabilities-validation.md
+   + verify complete internal MFE pairs, shared borders, signed guides and
+     output-off parity; document the fixed-boundary traceback and orientation
+
+261009 Alexander Mitrofanov
+ * doc/analysis/seeded-probabilities-validation.md, doc/recursions :
+   * record final review validation and controlled runtime/memory measurements;
+     update the seeded-heuristic depiction for approximate pair output (PR #259)
+
+261009 Alexander Mitrofanov
+ * PredictorMfeEns2dHeuristic, PredictorMfeEns2dHeuristicSeedExtension,
+   PredictorMfeEns2dSeedExtension, BasePairProbabilities, probability writers :
+   + reverse retained candidate chains and signed seed-extension recurrences to
+     obtain marginals of the existing heuristic ensemble; preserve denominator
+     and seed-overlap corrections, validate all masses before publication
+   + expose approximation metadata in CSV/SVG and retain seed provenance
+   * bound cropped noLP continuations by the active region and correct default
+     region offsets; reject unsupported singleton heuristic seeds explicitly
+ * CommandLineParsing, tests, README.md, doc/recursions :
+   + enable model P/mode H probability output; add independent tiny-chain and
+     signed-extension checks plus CLI parity/metadata coverage (PR #259 step 11)
+ * PredictorMfeEns2d, probability validation tests and report :
+   * reject finite-energy boundary underflow and nonfinite heuristic denominators
+     before publication; record independent review, full suites and performance
+
+261009 Alexander Mitrofanov
+ * PredictorMfeEns2d, CommandLineParsing :
+   + compute actual-pair masses by reversing the existing unseeded recurrence;
+     preserve site filters, noLP and numerical cutoffs, and merge disjoint regions
+   * enable CSV/SVG with --noSeed; fix default-ended region offset truncation
+ * tests/PredictorUnseededProbability_test.cpp, tests/Makefile.am, README.md :
+   + compare against independent chain enumeration, filters, regions and failures;
+     document the public optional sink (PR #259 step 10)
+
+261009 Alexander Mitrofanov
+ * BasePairProbabilitySvg, BasePairProbabilityWriter, CommandLineParsing, IntaRNA :
+   * place target origin at bottom left and query name below the matrix; include
+     nondefault sequence names in a subtitle and names in rounded hover values
+   * show 1-Pu intramolecular pairing probabilities on the nucleotide frame
+   + outline the best reported interaction region in green, including when
+     ranked output is disabled; preserve ordinary output counts (PR #259 review)
+ * README.md, tests/testBasePairProbabilitySvg.py :
+   * document and cover coordinates, frame probabilities, tooltips and MFE bounds
+
+261009 Alexander Mitrofanov
+ * BasePairProbabilityWriter, BasePairProbabilitySvg, CommandLineParsing :
+   * pass and validate separate target/query accessibility RT values for SVG
+     frames; imported probabilities no longer inherit the interaction model RT
+   * share the source-specific conversion with tPu/qPu, correcting their earlier
+     physical-temperature conversion of computed base-pair accessibility (RT=1)
+   * move completed SVG/CSV stream buffers instead of copying large documents
+ * tests, README.md, doc/analysis/seeded-probabilities-validation.md :
+   + cover imported Pu/ED, mixed sources, nondefault temperatures and invalid RT;
+     compare against input probabilities and independent ED conversion
+   + document independent review, controlled runtime/RSS comparisons and final
+     release/debug, installation and public-header checks (PR #259)
+ * doc/recursions/IntaRNA.PredictorMfeEns2dHeuristic.svg,
+   doc/recursions/IntaRNA.PredictorMfeEnsSeedOnly.svg,
+   doc/recursions/IntaRNA.SeededPartitionFunction.*.svg :
+   * show the heuristic initial right-end GU gate and complete-site filters;
+     place nonstack arrivals before inside aggregation and clarify outside notation
+
+261009 Alexander Mitrofanov
+ * doc/recursions/IntaRNA.PredictorMfeEns*.svg,
+   doc/recursions/IntaRNA.SeededPartitionFunction.*.svg :
+   + add eight standalone internal SVG depictions for unseeded exact/heuristic,
+     seed-only, exact stack-seeded, legacy and heuristic seed-extension paths
+   + show noLP transitions, seed admission and energy ownership, the production
+     suffix inside/outside recurrences, actual-pair marking and regional merging
+   + include implementation anchors and retain editable vector text; validate
+     XML, glyph coverage and rendered layouts (PR #258 step 9, PR #259)
+
+261009 Alexander Mitrofanov
+ * BasePairProbabilityWriter, BasePairProbabilitySvg, CommandLineParsing :
+   + add success-only SVG dot plots on files, gzip and synchronized standard
+     streams; share CSV normalization and use original sequence coordinates
+   + show sequence labels, 1-nt accessibility, seven CSS probability classes,
+     legends, signed 10/50-nt guides, missing-zero boundary and hover values
+ * BasePairProbabilities, PredictorMfeEns2dSeedExtension :
+   + optionally retain admitted seed-cell masks with completed regional masses;
+     annotate seed cells without changing probability or ranked-output semantics
+ * tests, README.md, src/IntaRNA/Makefile.am, tests/Makefile.am :
+   + document the SVG contract and register the writer and regression coverage
+     for coordinates, seed admission, accessibility, empty/failed results and I/O
+   + investigate the referenced dev-frank profile tracker files: their current
+     versions contain profile output but no SVG implementation (PR #258 step 8)
+
 261008 Alexander Mitrofanov
  * tests, README.md, doc/analysis, doc/seeded-probabilities-example.cpp :
    + finish probability regression and native performance gates, document the

README.md

@@ -1876,14 +1876,15 @@ targeted file/stream name:
 
 - `qSpotProb:`/`tSpotProb:` [query/target's spot probability profile](#profileSpotProb) (CSV format), respectively
 - `spotProb:` [all spot probabilities](#spotProb) (CSV format)
+- `bpsvg:` [SVG base-pair probability dot plot](#bpsvg), with pairing-probability frames and seed/MFE outlines
 - `bpProb:` [actual base-pair probabilities](#bpProb) for exact stack-seeded ensembles (CSV matrix)
 - `qMinE:`/`tMinE:` [the query/target's minimal interaction energy profile](profileMinE) (CSV format), respectively
 - `pMinE:` [minimal interaction energy for all query-target index pairs](pairMinE) (CSV format)
 - `qAcc:`/`tAcc:` the [query/target's ED accessibility values](#accessibility) (RNAplfold-like format), respectively
 - `qPu:`/`tPu:` the [query/target's unpaired probabilities](#accessibility) (RNAplfold format; rounded!!), respectively
 
 Note, for *multiple sequences* in FASTA input, the provided file names are suffixed by
-- `-t#q#` : for `bpProb`, `*spotProb` and `*minE` output, and
+- `-t#q#` : for `bpProb`, `bpsvg`, `*spotProb` and `*minE` output, and
 - `-s#` : for `*Acc` and `*Pu` output,
 where `#` denotes the according target/query sequence number
 within the input where numbering starts with 1.
@@ -1892,6 +1893,9 @@ The column separator within tabular CSV output (defaulting to `;`) can be change
 using `--outSep`, e.g. to produce tab-separated `.tsv` output.
 
 Note further, `qPu:`|`tPu:` will report unpaired probability values based on rounded accessibility (ED) values.
+The ED-to-probability conversion uses each strand's accessibility energy scale:
+computed base-pair accessibility uses RT=1, while ViennaRNA and imported
+accessibility use the configured temperature, independently of the interaction model.
 Thus, these values will most likely differ from values eg. produced by the program RNAplfold.
 We therefore strongly recommend to store `qAcc:`|`tAcc:` values when you want to use them
 as input for subsequent IntaRNA calls!
@@ -2012,8 +2016,9 @@ or `STDERR` instead of a file name.
 ## Actual base-pair probabilities
 
 `--out=bpProb:FILE` writes the probability that each target/query nucleotide pair
-actually pairs, conditional on an allowed seeded interaction. It requires
-`--model=P --mode=M` and a stack-only seed handler. For example:
+actually pairs, conditional on an allowed interaction. It requires
+`--model=P --mode=M` with `--noSeed` or a stack-only seed handler, or
+`--model=P --mode=H` for an approximate heuristic ensemble. For example:
 
 ```sh
 IntaRNA -t GGAGGG -q CCCC --energy=B --acc=N --seedBP=2 \
@@ -2026,12 +2031,19 @@ is also used without `bpProb`, so requesting the matrix does not change `Zall`.
 Computed seeds preserve their handler-specific admission thresholds. Explicit
 stacked seeds may have different lengths, including singletons; competing
 explicit patterns at the same start still retain only the selected minimum-energy
-pattern. Other predictors retain their existing behavior. Heuristic, kinetic,
-evaluation, seed-only, unseeded and bulged-seed `bpProb` requests are rejected.
+pattern. With `--mode=H`, probabilities describe the predictor's retained
+candidate ensemble, not all possible interactions. Unseeded H retains one
+continuation per left pair; seeded H retains its seed-extension candidates and
+selected right extension, including the existing seed-overlap corrections.
+Seeded H supports computed or explicit seeds with at least two pairs, including
+bulged seeds. These heuristic ensembles can retain path multiplicities and
+must not be interpreted as the exact unique-chain ensemble of mode M.
+Kinetic, evaluation, seed-only and exact bulged-seed probability requests remain
+unsupported.
 
 Rows follow the original target and columns the original query, both 5' to 3'.
-The header starts with `bpProb`; labels contain each nucleotide and its display
-index (including `tIdxPos0`/`qIdxPos0` shifts). Entries use sufficient significant
+The header starts with `bpProb` (exact) or `bpProbApproximate` (heuristic); labels
+contain each nucleotide and its display index (including `tIdxPos0`/`qIdxPos0` shifts). Entries use sufficient significant
 digits for the partition type, with scientific notation for small probabilities.
 A successful empty ensemble contains `NA`; pairs absent from a nonempty ensemble
 have probability zero. The output uses `outSep`, the usual multi-FASTA filename
@@ -2050,20 +2062,98 @@ bulge can have positive site coverage and zero actual-pair probability. There
 is no added unbound-state weight. Individual pair events can coexist, so
 `1 - sum(pair probabilities)` is not a probability of no interaction.
 
-The backend checks nonnegative arithmetic and reports numerical range failures
+The unseeded exact predictor is enabled by `--noSeed` with the same CSV/SVG
+outputs. Its outside pass follows the existing fixed-right-boundary recurrence,
+including its numerical zero cutoffs, without changing ordinary prediction.
+
+The seeded backend checks nonnegative arithmetic and reports numerical range failures
 as errors, including underflow, overflow and NaN. Positive subnormals are rejected;
 there is currently no scaling fallback. Failed or cancelled regional work never
-produces a successful-looking probability matrix. Ordinary floating-point
-rounding remains; "exact" describes the enumerated ensemble, not real arithmetic.
+produces a successful-looking probability matrix. Heuristic reverse passes likewise
+validate finite arithmetic and reject invalid masses before publication. Ordinary
+floating-point rounding remains; "exact" describes the enumerated ensemble, not
+real arithmetic.
 
 For library use, keep a `BasePairProbabilities` owner alive across predictions,
-pass its optional non-owning pointer to `PredictorMfeEns2dSeedExtension`, and call
+pass its optional non-owning pointer to `PredictorMfeEns2dSeedExtension` (stack
+seeds), `PredictorMfeEns2d` (unseeded), or their heuristic subclasses, and call
 `finalize()` only after all requested disjoint regions succeed. A writer is called
 explicitly after finalization; its destructor publishes nothing. The output
 constraint must enable `needZall`. The [standalone API example](doc/seeded-probabilities-example.cpp)
 and [implementation validation report](doc/analysis/seeded-probabilities-validation.md)
 show the public API and measured limits.
 
+<a name="bpsvg" />
+
+### SVG base-pair probability dot plots
+
+`--out=bpsvg:FILE` renders the same actual-pair probabilities as `bpProb` in a
+standalone SVG. Both outputs can be requested together. The same supported
+model/mode combinations, regional ensemble, normalization and success-only
+publication rules apply. Heuristic SVGs are explicitly labelled approximate.
+
+```sh
+IntaRNA -t GGAGGG -q CCCC --energy=B --acc=N --seedBP=2 \
+  --model=P --mode=M --out=bpsvg:pairs.svg --out=bpProb:pairs.csv
+```
+
+The query runs left to right and the target top to bottom, both in original
+5' to 3' order, starting at the upper-left corner. Sequence names label the
+axes, with the query name below the matrix. A subtitle names both sequences
+when either name differs from the defaults `query` and `target`. Nucleotides
+appear on all four sides. Their colored backgrounds show **intramolecular
+pairing probability**, `1-Pu`, computed from the accessibility model's opening
+energies. These are distinct from the interaction-conditional pair probabilities
+in the matrix. Disabled accessibility gives zero at unconstrained bases.
+Each probability rectangle has a hover title with sequence names, indices,
+probability type and three decimal places (scientific notation with two decimal
+places below 0.001). Full-precision values remain in `data-probability` attributes.
+An empty interaction ensemble uses gray `NA` cells while its frame remains defined.
+
+A shared seven-step, white-to-dark-blue palette encodes both probability types:
+`[0,0.01)`, `[0.01,0.1)`, `[0.1,0.25)`, `[0.25,0.5)`, `[0.5,0.75)`,
+`[0.75,0.9)`, and `[0.9,1]`. The legend appears below the plot. Edit the `.p0`
+through `.p6` fill rules in the SVG's `<style>` element to recolor the matrix,
+frames and legend together. No external stylesheet or JavaScript is needed.
+
+Gray guides use the final, shifted sequence indices: every tenth nucleotide,
+with stronger lines at multiples of 50. For a negative multiple, the line is
+before the labelled nucleotide (for example `-11 | -10`); for a positive one,
+it is after it (`10 | 11`). A dashed guide marks `-1 | +1` when both occur.
+Guide labels sit beside the associated nucleotide, on both sides of the plot;
+`+1` is also labelled when it starts a sequence.
+
+Dark orange outlines mark the union of pairs belonging to **admitted seeds**
+contained in searched regions, including overlapping, singleton and mixed-length
+explicit seeds. This annotation uses the predictor's retained seed family and
+span limits; it is independent of ranked interaction reporting and can include
+zero-probability cells. A seed legend is included whenever an outline is present.
+A green rectangle encloses the best reported ensemble interaction's site, and
+green cell borders mark a minimum-energy structure reconstructed within that
+site using the seed, loop and noLP constraints. Ensemble predictors ordinarily
+report only site boundaries; the SVG requests this additional MFE traceback.
+It does not change site ranking or the probability matrix. Where an MFE pair
+also belongs to an admitted seed, its left/right edges are orange and its
+top/bottom edges are green. Both annotations have legends. The best site is retained
+for SVG even with `--outNumber=0`, without adding ranked text output. If no site
+passes the reporting filters, the rectangle and its legend are omitted.
+
+Filenames support `.gz` compression and the usual `-t#q#` multi-FASTA suffixes.
+`STDOUT` and `STDERR` emit each complete SVG as one synchronized block. To obtain
+a standalone SVG on standard output, redirect ordinary output, for example
+`--out=/dev/null --out=bpsvg:STDOUT`. Multiple sequence pairs on a shared stream
+produce consecutive SVG documents; use filenames for separate viewable files.
+
+Library callers can use `BasePairProbabilityWriter::writeSvg()` or
+`writeSvgFile()` with a finalized result, the original energy model, and separate
+target/query accessibility RT values. These must match the ED conversion scales
+of the accessibility sources, which can differ from the interaction model's RT
+(for example, imported Pu with a base-pair interaction model). Construct
+`BasePairProbabilities(targetLength, queryLength, true)` to retain seed annotations;
+the default CSV-only result does not allocate that mask. SVG creation buffers
+the document and uses one rectangle per pair plus the frames, so its size grows
+with the full target-by-query matrix.
+
 <a name="spotProb" />
 
 ### Interaction probabilities for interaction spots of interest