dataset(code-review): add synthetic reaction gold answers from BCApps PR 10466

[Wenjie Fan (gggdttt)]

Summary

Adds minimal synthetic code-review gold answers from explicit BCApps PR feedback.

Full source PR patches are intentionally excluded. Positive reactions become
isolated expected findings; thumbs-down findings become precision guards by
omission. Existing equivalent gold answers are not duplicated.

Accepted findings

Rejected findings

• Rejected precision guard: Resolve purchase lines to items on exact description match BCApps#10466 (comment)

Knowledge dependency

• Related BCQuality improvement: knowledge: improve review precision from BCApps PR 10466 feedback BCQuality#169

Validation

See the gate-owned offline evaluation section below.

Generated by the BC-ALAgentsInternal self-improvement workflow.

Offline evaluation: regression

Candidate correctness failed: unexpected or missed findings remain. Existing ignored gold comments retain their neutral scoring semantics.

Preparation attempt 1: validated.

Selection: new. New: synthetic__breaking-fallback-description-match-01. Reused (complete payloads): ``.

• synthetic__breaking-fallback-description-match-01 / calibration_context / Resolve purchase lines to items on exact description match BCApps#10466 (comment): Negative (THUMBS_DOWN) reaction with a developer reply ('That's new behavior this PR is doing') that confirms rather than refutes the described behavior change, so this is a recommendation disagreement (opt-in gating) over a genuine finding, not a clean negative. Keeps an expected finding on the fallback resolution call site so the boundary isn't miscoded as a false-positive guard. Limitation: does not validate the reviewer's specific 'add an opt-in setting' remediation, only the underlying silent-default-change premise; severity/remediation choice remains a human-reviewed judgment call.
  Coverage references and outcome shapes are validated mechanically. Semantic equivalence, severity calibration and recommendation quality are NOT proved by these checks or a matching F1.

Common engine code: ecf8e31759d6ddd6d78e3a0b7836b40134368009; dataset SHA256: AB3371B46C6015755BEC8562BDEDEF179EB6385D8D51C5C9CDAD8BA6D19A4AF7.
Model: gpt-5.6-luna; judge: gpt-5.3-codex. Exact D: synthetic__breaking-fallback-description-match-01.
Dataset base: 26365d6a41a96fac41df9dcc8339cb71241fcb1f; candidate: 273d0e0a458c7d59771272e6a54036340fd6b122.

Arm	Evaluation commit	Engine pin	Knowledge pin
baseline	f400d204bf717089cab86e3a9e38c9cb7110d2e4	4407f20b44fe92e3693d0444d25cc3430f1d5262	8584217c7506eea7eef27a9db353d78c0cd6a9a1
candidate	d38b7b8cddf87e6389d82b111d73b02e6c90e4bd	b03ac0db52874297405e05b70324bcc74da6ddcd	c08f36de18453cffe1830e0d28fa68ce101f8b77

Baseline

Run: https://github.com/microsoft/BC-Bench/actions/runs/34346037868; conclusion: success; wall clock: 6.6 minutes.
Raw findings and runtime pin proofs are retained in the self-improvement-feedback artifact and the linked evaluation run.

Entry	Expected	Generated	Missed	Unexpected	F1	AI credits	Agent seconds
synthetic__breaking-fallback-description-match-01	1	0	1	0	0	unavailable	172.605483866

Observed AI credits: unavailable; coverage 0/1 entries. Missing entries are not extrapolated into a total.

Aggregate metric	Value
total	1
expected_comment_count	1
generated_comment_count	0
matched_comment_count	0
missed_comment_count	1
incorrect_comment_count	0
precision	1
recall	0
f1	0

Candidate

Run: https://github.com/microsoft/BC-Bench/actions/runs/34346655887; conclusion: success; wall clock: 24.5 minutes.
Raw findings and runtime pin proofs are retained in the self-improvement-feedback artifact and the linked evaluation run.

Entry	Expected	Generated	Missed	Unexpected	F1	AI credits	Agent seconds
synthetic__breaking-fallback-description-match-01	1	2	1	2	0	unavailable	1276.604446395

Observed AI credits: unavailable; coverage 0/1 entries. Missing entries are not extrapolated into a total.

Aggregate metric	Value
total	1
expected_comment_count	1
generated_comment_count	2
matched_comment_count	0
missed_comment_count	1
incorrect_comment_count	2
precision	0
recall	0
f1	0

Missing telemetry is unavailable, not zero. Evaluation-only metrics exclude candidate generation and are not the full-cycle cost.
Human review must verify source-patch fidelity, gold correctness, and target attribution. No automatic merge or branch-protection claim is made.