dataset(code-review): add synthetic reaction gold answers from BCApps PR 10255 - #869
Draft
Wenjie Fan (gggdttt) wants to merge 1 commit into
Draft
dataset(code-review): add synthetic reaction gold answers from BCApps PR 10255#869Wenjie Fan (gggdttt) wants to merge 1 commit into
Wenjie Fan (gggdttt) wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds minimal synthetic code-review gold answers from explicit BCApps PR feedback.
Full source PR patches are intentionally excluded. Positive reactions become
isolated expected findings; thumbs-down findings become precision guards by
omission. Existing equivalent gold answers are not duplicated.
Accepted findings
Rejected findings
Knowledge dependency
Validation
See the gate-owned offline evaluation section below.
Generated by the BC-ALAgentsInternal self-improvement workflow.
Offline evaluation: regression
Candidate correctness failed: unexpected or missed findings remain. Existing ignored gold comments retain their neutral scoring semantics.
Preparation attempt 1: validated.
Selection: new. New:
synthetic__privacy-activity-log-lifecycle-01. Reused (complete payloads): ``.synthetic__privacy-activity-log-lifecycle-01/false_positive_guard/ Add expense activity history foundation BCApps#10255 (comment): Reproduces the rejected finding's exact shape: an OnDelete trigger on a posted document header cascades deletion of activity-log rows matched by source table number and the document's SystemId, mirroring PostedExpenseReportHeader.OnDelete calling ExpenseActivityLogMgt.DeleteEntriesForSource(Database::"Posted Expense Report Header", Rec.SystemId). The developer reply confirms this cascade is by-design lifecycle ownership to avoid orphaned actor/comment/amount personal data, not an audit-trail defect, so expected_comments is empty by omission. Limitation: this guard only covers the cascade-delete-on-OnDelete boundary for activity/audit-style child records; it does not generalize to cases where the child table holds compliance-mandated retention data with no privacy offset, which could still warrant a finding.Coverage references and outcome shapes are validated mechanically. Semantic equivalence, severity calibration and recommendation quality are NOT proved by these checks or a matching F1.
Common engine code:
ecf8e31759d6ddd6d78e3a0b7836b40134368009; dataset SHA256:D3F92967ABDE17CF834EFFE7ED69523E42DDDCA8A135B9D8A24A7A0313034A05.Model:
gpt-5.6-luna; judge:gpt-5.3-codex. Exact D:synthetic__privacy-activity-log-lifecycle-01.Dataset base:
26365d6a41a96fac41df9dcc8339cb71241fcb1f; candidate:c555f9294b31cfe6dc807e43f276a1f5bb9f53af.ac33eb8e634f538ce5b377078ee6f263a548207d4407f20b44fe92e3693d0444d25cc3430f1d52628584217c7506eea7eef27a9db353d78c0cd6a9a1c83280f7651a03d27e3ddd09b8c676d8eaa001e98558750e58fac1624187201ae462383f4d697661ff91a136a29cc8c49a78c6e3739bcd976dec4f91Baseline
Run: https://github.com/microsoft/BC-Bench/actions/runs/34349857118; conclusion: success; wall clock: 7.9 minutes.
Raw findings and runtime pin proofs are retained in the self-improvement-feedback artifact and the linked evaluation run.
synthetic__privacy-activity-log-lifecycle-01Observed AI credits: unavailable; coverage 0/1 entries. Missing entries are not extrapolated into a total.
Candidate
Run: https://github.com/microsoft/BC-Bench/actions/runs/34350651192; conclusion: success; wall clock: 7.5 minutes.
Raw findings and runtime pin proofs are retained in the self-improvement-feedback artifact and the linked evaluation run.
synthetic__privacy-activity-log-lifecycle-01Observed AI credits: unavailable; coverage 0/1 entries. Missing entries are not extrapolated into a total.
Missing telemetry is unavailable, not zero. Evaluation-only metrics exclude candidate generation and are not the full-cycle cost.
Human review must verify source-patch fidelity, gold correctness, and target attribution. No automatic merge or branch-protection claim is made.