Add PPO agent to trading pipeline using stable-baselines3 for adaptive position sizing and prediction trust

[Copilot]

Description

Adds a stable-baselines3-backed PPO (Proximal Policy Optimisation) agent to the simulation pipeline. The agent observes ML model predictions alongside market state and learns online whether to trust those predictions before executing trades — addressing position sizing and stock selection via reinforcement learning rather than fixed heuristics.

The implementation uses stable_baselines3.PPO for all neural-network and optimisation logic, with a minimal gymnasium.Env wrapper for SB3 model initialisation. Online learning is preserved by manually populating SB3's RolloutBuffer step-by-step and calling model.train() after each rollout window.

Changes in this pull request

• src/simulation/strategy/ppo.py — New module containing:

  • _TradingEnv: Minimal gymnasium.Env with a 5-dim continuous observation space and 3-action discrete action space, used to initialise the SB3 PPO model
  • PPOAgent: Wraps stable_baselines3.PPO; collects transitions via store(), populates SB3's RolloutBuffer, and triggers model.train() updates; exposes select_action, value, and buffer_size for use by the strategy
  • PPOStrategy: Wraps PPOAgent in the Strategy interface, registered as "ppo" in StrategyRegistry; state vector includes raw ML pred_return so the agent learns signal reliability over time

• src/simulation/strategy/__init__.py — Import both ppo and regime_aware modules (conflict with main resolved; both imports are now present)

• pyproject.toml / uv.lock — Added stable-baselines3==2.8.0 dependency (transitively adds torch==2.11.0 and gymnasium==1.2.3)

• tests/simulation/test_strategies.py — 15 new tests (TestPPOAgent, TestPPOStrategy) covering action sampling, buffer lifecycle, gradient update stability, state construction, registry integration, and an end-to-end TradingSimulator smoke test; TestStrategyRegistry updated to include "ppo" in expected strategy set

Usage via existing registry:

strategy = StrategyRegistry.create("ppo", simulator=sim, capital=10_000, seed=42)
results = sim.simulate(predictions=preds, actual_returns=actuals, prices=prices, dates=dates, strategy=strategy)

The agent's observation vector [pred_return, position_flag, portfolio_ratio, days_held_norm, abs_pred_return] means it learns not just when to trade, but when the upstream ML signal is worth acting on.

[ch55secake]

@copilot Resolve the conflicts

[Copilot]

@copilot Resolve the conflicts

Resolved in 63f8b6f. The only conflict was in src/simulation/strategy/__init__.py — our branch added from . import ppo while main added from . import regime_aware. Both imports are now included.