feat: cost-aware backtesting, statistical validation gauntlet, and paper-forward tracking - #30
Closed
mrprecioustech wants to merge 2 commits into
Closed
feat: cost-aware backtesting, statistical validation gauntlet, and paper-forward tracking#30mrprecioustech wants to merge 2 commits into
mrprecioustech wants to merge 2 commits into
Conversation
…per-forward tracking Adds the infrastructure needed to gate AI-generated trading strategies through a rigorous validation pipeline and track them on live data before listing in the store. The real product is evidence, not generation. ## Core: realistic trading costs - `packages/core/src/fees.ts` — canonical taker fee rates for all CEX/DEX venues, equity commissions, and helper functions - `packages/source-strategies/src/costs.ts` — CostModel with presets (ZERO, DEFAULT, EQUITY, DEX), effective buy/sell pricing via bid/ask spread and slippage, per-leg fee math, round-trip cost in bps - `packages/source-strategies/src/backtest.ts` — rewritten backtester applies costs by default; BacktestTrade carries grossProceeds, feesUsd, spreadSlippageUsd, totalCostUsd, costBps, grossProfit alongside net profit; summarizeTrades() reports costDragPct; pass ZERO_COST_MODEL explicitly for the old frictionless numbers ## @b1dz/strategy-validation — statistical gauntlet (new package, 190 tests) - metrics: Sharpe, Sortino, profit factor, maxDD, CAGR, Ulcer, skew/kurtosis - deflated-sharpe: Bailey & López de Prado PSR, DSR, MinTRL with Hart normal CDF and Acklam PPF approximations. Key behaviour: Sharpe 2.0 selected as best-of-1000 trials receives DSR < 0.2 — the single-test PSR is ~0.98. - splits: chronological train/test, rolling + anchored walk-forward with warmup-aware fold reduction - robustness: TSP knob collection, ±pct perturbation, neighbourhood scoring with no-op detection for decorative parameters - correlation: per-bar signal Pearson + per-week return Pearson against catalogue for duplicate rejection - regime: 4-quadrant EMA-slope/volatility classifier, per-regime trade bucketing, multi-regime profitability gate - gauntlet: 11-gate orchestrator (minTrades, maxDD, profitFactor, DSR, robustness, regimeCoverage, OOS profit, walk-forward, catalogCorrelation) with blocking/advisory split and human-readable explainReport() - synthetic: seeded random walk/sine/trend generators + trade fixtures ## @b1dz/strategy-registry — strategy store DAL (new package) - strategy_registry: register gauntlet-passed strategies, list by user, poll forward_running entries, promote to listed/archived/rejected - forward_trades: record paper-trade entries as they fire on live data, close trades when exit bars arrive, track per-strategy MinTRL progress ## Consumer integration - Web: cost-aware backtest runner compounds by netMultiple, replays each class twice (real + ZERO_COST_MODEL) for gross-vs-net display; API route accepts validated cost overrides (0-500 bps); builder UI shows cost panel with fee/spread breakdown, gross vs net, round-trip hurdle, honesty note - CLI: --costs <preset> flag (kraken|coinbase|gemini|dex|equity|zero), --fee-bps/--slippage-bps/--spread-bps numeric overrides; table columns for Return, Gross, Fees; cost description and drag printed per class - Daemon: forward-test worker polls registry, replays strategies over Yahoo daily bars, records/close paper trades in forward_trades, checks MinTRL gate; registered in daemon sources array ## Database - strategy_registry table: status, tsp_doc, gauntlet_report, cost_model, lifecycle timestamps, RLS per-user - forward_trades table: open/close trade tracking, regime tagging, RLS per-user ## Test totals: 369 passing, all 6 packages typecheck clean
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| "typecheck": "tsc --noEmit" | ||
| }, | ||
| "dependencies": { | ||
| "@b1dz/core": "workspace:*", |
| "typecheck": "tsc --noEmit" | ||
| }, | ||
| "dependencies": { | ||
| "@b1dz/core": "workspace:*", |
| ), | ||
| ); | ||
| if (args.costPreset) { | ||
| console.log(chalk.dim(`Long-only signal replay · $${args.amount}/entry · Yahoo daily · classes: ${args.classes.join(' + ')} · costs: ${args.costPreset}`)); |
ThreatCrush Security Scan55 finding(s) HIGH/CRITICAL: 40 | MEDIUM: 15
…and 5 more. Full results in the Security tab. Snippets are redacted; ThreatCrush never prints matched credential material. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
All alerts resolved. Learn more about Socket for GitHub. This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored. |
The new packages inherited 'latest' which resolved to 7.0.2 while the rest of the repo's lockfile resolved to 6.0.3. The resulting diff pulled in all of typescript@7.0.2's platform-specific optional dependencies, triggering Socket license-alert noise. Pinning to the same version the rest of the monorepo uses avoids the issue.
Author
Security scan findings analysisAll 55 ThreatCrush + 3 CodeQL findings on this PR are false positives — no real code issues exist.
Fixes pushed to this branch:
70/70 packages typecheck clean. |
Author
|
Closing — replacing with updated PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the infrastructure needed to gate AI-generated trading strategies through a rigorous validation pipeline and track them on live data before listing in the store.
Summary
packages/core/src/fees.ts,packages/source-strategies/src/costs.ts): canonical fee table for all venues, CostModel with presets, effective buy/sell pricing via bid/ask spread and slippagepackages/source-strategies/src/backtest.ts): net-of-cost profit by default, BacktestTrade carries gross/fee/spread/slippage breakdown, costDragPct in summaries@b1dz/strategy-validation(new package, 190 tests): metrics, Bailey & López de Prado deflated Sharpe ratio (PSR/DSR/MinTRL), parameter robustness, regime coverage, correlation-based duplicate detection, walk-forward splits, full 11-gate gauntlet orchestrator@b1dz/strategy-registry(new package): DAL for strategy_registry + forward_trades tables with RLS--costs <preset>, per-field overrides, Return/Gross/Fees table columnsstrategy_registry+forward_tradestablesTest totals