Skip to content

Automate local source preparation and bounded near-match repairs - #135

Merged
RetroGameDeveloper merged 8 commits into
mainfrom
feat/source-assisted-permuter
Oct 28, 2026
Merged

RetroGameDeveloper merged 8 commits into
mainfrom
feat/source-assisted-permuter

Conversation

@RetroGameDeveloper

@RetroGameDeveloper RetroGameDeveloper commented Oct 28, 2026 •

Copy link
Copy Markdown

Summary

Add a free preparation → deterministic repair → random permutation pipeline for unmatched functions, using the existing exact judges throughout.

Local source preparation

  • New Settings → Local source preparation panel with opt-in reference folder, explicit function/file mappings, optional declarations header and include roots, symbol aliases and numeric retail macro overrides.
  • Preserve reference function line breaks and original line numbers; make an isolated static entry point visible without moving its body.
  • Read selected folders without modifying them or executing their build scripts. Bound source/include reads, reject path escapes and ambiguous definitions, and keep local paths/mappings outside portable project config.
  • Run a bounded preparation job with progress/logs, or automatically prepare seeds before normal permuter / Run again repairs. Cache unchanged compiler trials, including failures.
  • Compile/link ROM candidates before changing anything. Only a strictly better candidate gets a private per-function header and saved source, atomically, with current claim, target, symbols, header and config checks. Existing matched functions and shared headers are untouched.
  • Imported projects use their existing build/objdiff judge and headers. Automatic injection of reference headers into imported builds remains an explicit documented gap.

Better bounded free search

  • Add scalar/pointer stack storage, unused local and declaration-order hypotheses, keeping earlier returning-branch repairs reachable.
  • Rank deterministic repairs using fresh mismatches; focus on stack/register issues, call signatures, constants or control flow. Unranked families remain available.
  • Try up to eight cached local-ordering probes inside the random turn deadline before waiting on random drift: adjacent equal-constant stores and shift/add operand order. Real compiler checks cover both.
  • Alternate diff-focused decomp-permuter batches with upstream default weights, retaining an exploration path. Fallback rewrites use the same advisory focus.
  • Count baseline compiles inside existing repair allowances and cache their evidence per exact input.
  • Renew bounded random-turn allowances after an improved external seed or changed header/compiler/target/symbol/build input. Adopt existing counters on first observation; claimed functions wait. The permuter's own hill climbing cannot reset its cap forever.
  • Rotate cached optimization/debug probes on close ROM misses, respecting manual compiler approval and existing scoped-change checks.
  • Expose authenticated /api/permuter start/stop and /api/source-preparation configuration/job endpoints.

Verification

  • uv run ruff check src tests projects and git diff --check.
  • After integrating the latest main, regression suite: 298 passed, 3 skipped, covering preparation, safe promotion, stale claims/inputs, trial caching, retry limits, repair ordering, GUI progress lifecycle, API jobs and compiler evidence.
  • Full suite exercised: 1,930 passed, 157 skipped initially. All four failing tests were corrected and re-run successfully (uv run pytest --lf -q: 4 passed). The fixes preserve branch-repair priority, include baseline compiles in the trial budget, and retain default-weight random batches.
  • Final real compiler checks: 3 passed in 6.09 seconds (private-header preparation/export plus both near-ordering cases).
  • Real compiler integration on the homemade N64 demo: prepared source earns an exact linked match, uses its private header, and the exported image rebuilds byte for byte.
  • On an isolated copy of an owned retail N64 project, the new preparation path recovered one previously unmatched 272-byte function from an explicitly mapped local reference body using installed IDO 5.3. Repeating the pass did no further work. It also passes export definition/constant-placement preflight. This is an isolated function result, not a newly verified full retail-image checkpoint.
  • Physical host verification: macOS. No user app restart, live project edit, proprietary source, toolchain download or game data is included in this PR.

Usage and limits

See Local source preparation. Users still need correct reference mappings and retail declarations/macros; source from a different version does not guarantee a match. Accepted source/declarations become normal project work and therefore appear in saved/exported projects, while reference paths and configuration stay local. Whole-image verification remains a separate gate.

Try unused scalar locals, local and parameter storage, and adjacent declaration order through shared ROM/imported deterministic repairs. Preserve source lines, opaque comments and shared headers. Prioritize storage families for stack diffs. Rotate two cached debug/optimization flag probes per pass on close ROM misses, require exact evidence and respect manual compiler approval. Existing compile/time budgets and exact judges remain in place.
…permuter

# Conflicts:
#	src/decompnet/app/repair_ranking.py
@github-actions github-actions Bot added the enhancement New feature or request label Oct 28, 2026
@RetroGameDeveloper
RetroGameDeveloper merged commit 0cde5ad into main Oct 28, 2026
8 checks passed
@RetroGameDeveloper
RetroGameDeveloper deleted the feat/source-assisted-permuter branch October 28, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant