Skip to content

README.md

Latest commit

 

History

History
132 lines (127 loc) · 7.86 KB

README.md

File metadata and controls

132 lines (127 loc) · 7.86 KB
Raw

Baselines

Recorded baseline reports — the durable before/after history of the budgets.json budgets (convention: ../methodology.md §4). One file per recorded measurement:

<milestone>-<workload>.md        e.g. m0-synthetic.md (M0-EXIT-01)

Each baseline file contains the full AGENTS §12 metadata (hardware, OS, compiler + version, build type, flags, workload, warm-up, sample count, summary statistics), the stable 4-line budgetCheck report verbatim, and the exact command + commit that produced it. Baseline files are immutable: superseding a baseline adds a new file and updates measured in budgets.json — it never edits an existing baseline.

Recorded so far:

  • m0-synthetic.md (M0-EXIT-01, 2026-09-13) — the synthetic harness workload; proves the measurement pipeline end to end but does not measure any real budget.
  • m1-ecs-stress.md (M1-ECS-07, 2026-09-14) — the M1 ECS stress workload (10k entities × 6 component types × 10k frames of add/remove churn): no leaks (ASan), pool high-water stable, iteration within the documented cost, accounted ECS storage bytes. Not a budgets.json workload — no measured field updated.
  • m1-profiler-cost.md (M1-PROF-01, 2026-09-21) — the enabled-profiler cost gate (ON vs OFF on 10k-entity ticks, bounded at 1%): measured +0.28% on the canonical Debug tree. Not a budgets.json workload — the FR-11.1 counters are diagnostics, not a budgeted subsystem.
  • m1-sim-tick.md (M1-BENCH-01, 2026-09-25) — the PRD §8.1 simulation-tick workload (10k entities, 2k dynamic bodies, 60 Hz, movement + per-tick state hash, both SimMath backends): 0.59 ms mean / 0.62 ms p99 on the canonical Debug tree, both backends PASS. The first baseline to update budgets.json measured (sim_tick_avg, sim_tick_p99 — the worse of the two backends).
  • m2-iso-depth-table.md (M2-ISO-02, 2026-09-30) — the PRD §8.1 isometric depth-key-rebuild workload (10k dirty cells after a terrain edit, 128×128 grid, 100×100 block, both SimMath backends): 0.087 ms mean on the canonical Debug tree, both backends PASS (2.3× inside the 0.2 ms gate); the gate also passes on the reference-class clang -O0 tree (0.149 ms). The first M2 baseline — updates budgets.json measured for iso_depthkey_rebuild (the worse of the two backends).
  • m2-iso-depth-table-workload-fix.md (M2-ISO-02 workload fix, 2026-10-01) — the same workload with the measured window's applyEdit loop made division-free (the original loop's three per-iteration div/idiv instructions at -O0 — a harness artifact — pushed the CI clang-18 reference lane to 0.209 ms, over the 0.2 ms bar): 0.0814 ms mean on the canonical Debug tree, both backends PASS (2.5× inside the gate); 0.139 ms on the local clang -O0 tree. Supersedes m2-iso-depth-table.md as the latest recorded value — updates budgets.json measured for iso_depthkey_rebuild to 0.0814067. Every other budgets.json entry still has measured: 0 (its subsystem lands in a later milestone).
  • m2-iso-picking.md (M2-ISO-03, 2026-10-02) — the PRD §8.1 isometric-picking workload (one screen-to-grid pick, O(1)): 0.000136 ms mean on the canonical Debug tree (73× inside the 0.01 ms gate); 0.000189 ms on the clang -O0 cross-run. The sixth baseline — updates budgets.json measured for iso_picking to 0.000136262 (index entry added retroactively 2026-10-03 by the M2-SORT-01 PR — the M2-ISO-03 step landed the baseline file without updating this list).
  • m2-depth-sort.md (M2-SORT-01, 2026-10-03) — the PRD §8.1 10k-sort workload (10 000 32-bit depth keys, one stable radix sort, the 3 000-frame stress): 0.225852 ms mean on the canonical Debug tree (4.4× inside the 1.0 ms gate); 0.161389 ms on the clang -O0 cross-run. The seventh baseline — updates budgets.json measured for depth_sort_10k to 0.225852.
  • m2-iso-depth-table-budget-rebaseline.md (M2-ISO-02 budget revision, 2026-10-04) — re-baselines the iso_depthkey_rebuild budget after the CI reference lane (Clang 18.1.3 -O0, ubuntu-24.04) measured 0.194–0.267 ms on unchanged engine code against the 0.2 ms bar (repeated zero-margin gate failures, no regression): PRD §8.1 v0.4 target 0.2 → 0.3 ms, budgets.json measured 0.0814067 → 0.202204 (latest recorded reference-lane value, worse backend). Workload and engine code unchanged. The eighth baseline — supersedes m2-iso-depth-table-workload-fix.md as the latest recorded value of iso_depthkey_rebuild.
  • m2-particle-render.md (M2-PART-02, 2026-10-07) — the FR-2.7 particle → sprite conversion workload (10 000-particle declareParticles pass, both SimMath backends, the declare pass ONLY — the batcher's sort is the separate depth_sort_10k budget): 1.07798 ms mean on the canonical Debug tree (worse backend fpx16_16; 1.86× inside the 2.0 ms step-level gate). The tenth baseline — updates budgets.json measured for particle_render_10k to 1.07798 (the budgets.json entry count moves 16 → 17 — the budget_harness_tests.cpp table pin).
  • m2-particle-render-budget-rebaseline.md (M2-PART-02 budget revision, 2026-10-08) — re-baselines the particle_render_10k budget after the CI reference lane (Clang 18.1.3 -O0, ubuntu-24.04) measured 1.91–2.68 ms on unchanged engine code against the 2.0 ms bar (three consecutive master merge-lane failures since M2-PART-02 landed, no regression — the gate was calibrated from local-machine runs and never verified with margin on the CI reference toolchain): budgets.json target 2.0 → 3.0 ms, measured 1.07798 → 2.07786 (latest recorded reference-lane value, worse backend). Step-level budget — no PRD revision (the depth_sort_10k precedent); workload and engine code unchanged. The eleventh baseline — supersedes m2-particle-render.md as the latest recorded value of particle_render_10k.
  • m2-render-50k.md (M2-PERF-01, 2026-10-12) — the PRD §8.1 "50k visible sprites" workload (the FULL M2-SCENE-01 reference scene, offscreen 1280×720, software GL, both SimMath backends, 3000 measured frames of the batch stage): 60.4450 ms mean on the CI reference lane (worse backend fp32_pinned; the g++ lane measured 69.4684 ms) against the 2.0 ms bar that predates the concrete workload, and 14 draw calls/frame (2.1× inside the 30 budget). The twelfth baseline — first recording of sprites_50k_cpu and sprites_50k_draw_calls; supersedes nothing (measured was 0); calibrates sprites_50k_cpu target 2.0 → 100.0 ms in the same PR (PRD §8.1 v0.5), measured 0 → 60.4450 and 14.
  • m2-ac4.md (M2-AC-01, 2026-10-14) — the PRD §4 AC-4.x acceptance record: the ac4 / render-sort-10k workload (10 000 overlapping sprites, 100×10 grid, 10 stacked per cell, four (atlas, material, blend) groups, 4 draw calls/frame, whole 60 FPS frame at 1280×720 offscreen software GL, both backends, 3000 measured frames): 12.1453 ms mean on the CI reference lane (worse backend fp32_pinned; worst across lanes 12.6959 ms) against the 16.7 ms frame budget — 37.5% margin, zero per-frame allocation, exactly 4 draw calls. The thirteenth baseline — first recording of sprites_10k_frame; supersedes nothing (measured was 0); carries the AC-4.1…AC-4.4 checklist and the root-cause record of the ac4 exit-time segfault (llvmpipe worker pool vs the context teardown/dlclose — fixed by the glFinish drain in GlContext::Impl::destroy + the end of the EGL driver dlclose).