Skip to content

Refresh production PR Review leaderboards - #858

Merged
dayland merged 1 commit into
mainfrom
feature/refresh-pr-review-leaderboards
Sep 7, 2026
Merged

Refresh production PR Review leaderboards#858
dayland merged 1 commit into
mainfrom
feature/refresh-pr-review-leaderboards

Conversation

@dayland

@dayland dayland commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Publishes the complete BC PR Review v0.10.0 baseline refresh from immutable BC-Bench commit 06a8bf298db822e27c2be3970f5e9c7a66869f4d.

  • Adds three production runs for each of the eight curated models
  • Replaces the earlier partial Luna and Gemini baseline
  • Preserves all 80 direct-agent historical runs and 25 experiment runs
  • Retains the exact Copilot CLI, BC-ALAgents, and BCQuality identity on every refreshed run and aggregate
  • Leaves advanced diagnostics unavailable where the original run artifacts did not preserve them

This PR is independent of #851 and can merge directly to main. The retained harness identity fields will become visible in the dashboards when #851 merges.

Replace the partial Luna and Gemini baseline with three production runs for all eight curated models while preserving direct-agent history and experiment results.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 57308020-6803-4e62-b08e-57fa875bff89

@haoranpb Sun Haoran (haoranpb) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is not 100% independent of #851 , specifically the added fields

"benchmark_commit": null,
"copilot_cli_version": null,
"bc_alagents_repository": null,
...

but should be harmless to have them

@dayland

dayland commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator Author

This PR is not 100% independent of #851 , specifically the added fields

"benchmark_commit": null,
"copilot_cli_version": null,
"bc_alagents_repository": null,
...

but should be harmless to have them

Agreed, I left the metrics but they are benign until/if we enable the added UX

@dayland
dayland merged commit 2315389 into main Sep 7, 2026
16 checks passed
@dayland
dayland deleted the feature/refresh-pr-review-leaderboards branch September 7, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants