Support engine 1.0: in-process runner reads C ABI 4, default runtime pineforge-release 1.0.0

[luisleo526]

Summary

pineforge-data now works with PineForge engine v1.0.0 and codegen 1.0.0, shipped together as pineforge-release 1.0.0.

• In-process runner reads C ABI 4. Every struct PineForgeBacktestRunner mirrors was checked against engine v1.0.0's include/pineforge/pineforge.h, field by field. The Docker suite now repeats that check with offsetof against the header inside the pinned image.
  • pf_trade_t gains open_at_end (appended in ABI 3). A row is now 104 bytes, and the copied trades carry open_at_end as a boolean.
  • pf_report_t gains broker_state_hash and broker_state_hash_len (ABI 4). The report is now 960 bytes. It is caller-allocated, so without these two fields the engine would write 16 bytes past the Python buffer. report_free releases the array; BacktestReport does not copy it.
  • sharpe_tv and sortino_tv keep offsets 48 and 56 and their report keys. The header now spells them sharpe_monthly and sortino_monthly.
  • Re-verified with no change needed: pf_bar_t, pf_trade_tick_t, pf_trade_stats_t, pf_metrics_t, pf_equity_point_t, pf_security_diag_t, pf_trace_entry_t, and the run_backtest_full, setter and stream signatures.
• What happens to ABI 2 and 3. EXPECTED_PF_ABI is 4, and the check stays an exact match. pineforge.h asks every caller for an exact match, and the image's own run_json.py enforces one. The runner refuses ABI 2 libraries (engine v0.10.2 to v0.12.3) and ABI 3 libraries (engine v0.13.0 to v0.13.1) before any strategy call, and the EngineBacktestError says what to do. For example: PineForge ABI mismatch: strategy reports 3, expected 4; engine v0.13.0 to v0.13.1 built it: rebuild the strategy with engine 1.x. The ABI 2 message also says that pineforge-data 0.2.0 reads ABI 2.
  • ABI 2 rows are 8 bytes shorter, so every trade after the first would be misread.
  • ABI 3 reports have no broker-state hash.
• Trade excursion unit. max_runup and max_drawdown are now documented as whole-trade excursions in account currency, net of entry fees. The pre-1.0 header called them "price travel ($/unit qty)", but the engine's values have not changed. Since v0.10.2 (ABI 2), every engine has computed a money amount, not a per-unit price move: runup * pv - entry_commission, multiplied by the account FX rate from late 0.10.x on. On the README quick start, engine 0.11.0 and engine 1.0.0 report identical excursions for all 88 trades both produce.
• Default runtime. release_contract.py and docker/server.Dockerfile now pin pineforge-release:1.0.0@sha256:b271873d0e91a9eeeb4595daaded9bdfbdbcd3925d3ec1300a833395e2004c86. That digest is the multi-arch index (amd64 and arm64) with engine 1.0.0 and codegen 1.0.0. The trace refusal message now names 1.0.0, which still has no trace option.
• twine. The release extra now requires twine>=7,<8. hatchling writes Metadata-Version 2.5, and twine 6.2.0 fails twine check --strict with '2.5' is not a valid metadata version. twine 7.0.0 passes on both the wheel and the sdist.
• Codegen 1.0 strategy() defaults. The runtime image policy (and the README note that points to it) now states what runs on both images show. A script that omits initial_capital, default_qty_type or default_qty_value gets TradingView's Pine v6 defaults on 1.0.0: 100000, strategy.percent_of_equity and 100. Release 0.1.12 used 1000000, strategy.fixed and 1. At 100% of equity, the default margin refuses entries that fill above their signal price, so such scripts can trade less.
• Other refusals. A library without pf_abi_version is attributed to engine v0.10.1 or earlier and gets a rebuild hint; an ABI above 4 gets a hint to upgrade pineforge-data.
• Docs now describe 1.0 support as it is, not as pending: the README, the backtesting runtime image policy, the server runtime channel, and the API reference. 0.1.12 stays documented as the digest-pinned way to reproduce earlier results.
• Test fixture. tests/fixtures/sma_cross.pine now declares default_qty_type=strategy.fixed, default_qty_value=1. Codegen 1.0 fills an omitted order size with TradingView's Pine v6 default of 100% of equity (see the codegen 1.0.0 CHANGELOG). Under Pine v6's default 100% margin, every entry in the fixture's short series is refused for lack of funds at that size, which leaves 0 trades to compare. The same script trades at 90% of equity, or at 100% with margin_long=50.

Tests against engine 1.0.0

Host: Linux arm64 (Ubuntu 24.04) with Docker, Python 3.13.15. The steps are those of CI. "Before" is 9322f81: the 1.0.0 pins and the new tests, with the old runner and the old twine pin. "After" is this head, d983d9b.

Check	Before (9322f81)	After (d983d9b)
ruff check .	pass	pass
mypy src	pass	pass
pytest	6 failed, 101 passed, 5 skipped	107 passed, 5 skipped
python -m build	pass	pass
python -m twine check --strict dist/*	fail on twine 6.2.0: InvalidDistribution: Invalid distribution metadata: '2.5' is not a valid metadata version	wheel and sdist PASSED on twine 7.0.0
mkdocs build --strict	pass	pass
PINEFORGE_DOCKER_TEST=1 pytest tests/test_docker_integration.py	2 failed, 2 passed	4 passed
./scripts/run_database_e2e.sh	2 passed	2 passed

These tests failed before the fix:

• Unit tests:
  • the three runner tests, because the fake library now reports ABI 4 and the old runner refuses it;
  • test_runner_refuses_pre_1_0_libraries_with_a_rebuild_hint[2-…] and [3-…];
  • test_report_structures_match_pineforge_abi_4_layout, where sizeof(_PfTrade) was 96, not 104.
• Docker tests:
  • test_report_structures_match_the_release_header, where pf_trade_t was 96 bytes in ctypes and 104 in the header;
  • test_in_process_runner_matches_the_release_harness, which failed with PineForge ABI mismatch: strategy reports 4, expected 2.

README quick start and the in-process runner

Steps 1 to 3 were taken verbatim from the README of each revision. Step 1 installs the checkout instead of the published wheel.

Run	Result
main (589e678), default 0.1.12	steps exit 0; engine 0.11.0 and codegen 0.9.0; 88 trades
this head, default 1.0.0	steps exit 0; step 2 prints the bitstamp bar timestamp_ms=1751413500000 … close=105709.0; engine 1.0.0 and codegen 1.0.0 at the pinned digest; 89 trades: the same 88, plus 1 open_at_end close of the position left open after the last bar; net P&L 23.807288969668754
this head, README bars run in-process: the strategy is compiled in the pinned image and read by PineForgeBacktestRunner	89 trades, 1 open_at_end; trades, metrics and equity curve identical to the image's own run_json.py; header and library report ABI 4; ctypes layout equals the C layout
macOS arm64, this head, in-process, built by hand from the engine v1.0.0 macos-universal release tarball, pineforge-codegen 1.0.0 from PyPI and Apple clang	89 trades, 1 open_at_end, net profit 23.807288969668754, bit-identical to the Linux run

On the CI runners, the Docker job now also compiles one strategy inside the release image. CI for this head is green, including the amd64 Docker job (4 passed), so the layout probe and the in-process comparison also pass on x86_64. The layout probe compares each member's name, offset, size and scalar kind (a C11 _Generic tag against the ctypes type).

Compatibility

• In-process users with ABI 2 libraries (engine v0.10.2 to v0.12.3) must rebuild them with engine 1.x, or stay on pineforge-data 0.2.0. This is a breaking change for them, so a minor release (0.3.0) fits this PR. The repository has no breaking-change label yet, so none is set here.
• The default runtime changes results. On the README quick start the 88 shared trades are unchanged and one open_at_end row is added. Scripts that omit their capital or order size are sized differently (see above).

An independent review found no critical issues. Its major finding (the undocumented codegen 1.0 sizing defaults) and its minor findings are addressed in 4820127 and d983d9b.

🤖 Generated with Claude Code