Preserve Klaud baseline preflight and report failure reasons / 保留 Klaud 基线预检并报告失败原因 - #3443
Conversation
在候选任务中复用规划阶段的基线预检，并记录脱敏失败原因。
让 SRT 测试夹具识别 git -c 参数后的 clone 子命令。
|
Claude finished @adibarra's task in 48s —— View job LGTM - no blocking issues found Traced the preflight hand-off (plan artifact LCA → |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline finding, I checked two other things and ruled them out: prepare_baseline() in reporting.py validates every binding field (candidate id, base SHA, model, source identity, family, date, image) before trusting a preflight artifact, so a candidate can't forge the baseline used to judge itself; and the diagnostics path in __main__.py that runs when finish is never reached only affects report availability, not the reason-code enforcement that finish itself performs.
Extended reasoning...
This PR changes Klaud's baseline-preflight handoff (reporting.py, main.py) so the candidate job reuses a planner-signed baseline roster instead of refetching evidence, and adds a mandatory sanitized reason-code for baseline failures surfaced through outcome/diagnostics/report; it touches trust boundaries between planner and candidate jobs (evidence forgery, CI workflow permissions) which is security-relevant. A confirmed inline finding shows the new reason-code check in the finish command can raise VerificationError on a legitimate klaud-handoff-labeled PR whose outcome file predates or omits the reason-code, turning what used to be a clean handoff into a hard failure, so this needs a human decision rather than approval.
|
|
||
| outcome = CandidateOutcome.model_validate_json(args.outcome_file.read_text()) | ||
| if ( | ||
| outcome.outcome == "failed" | ||
| and outcome.phase == "baseline" | ||
| and not outcome.reason_code | ||
| ): | ||
| raise VerificationError("Baseline failure requires a fixed reason-code") | ||
| outcome = current_session().finish(outcome) |
There was a problem hiding this comment.
🟡 (optional) If a maintainer labels a candidate PR klaud-handoff while its self-reported outcome is failed/baseline without a reason-code, finish now errors instead of returning the clean handoff the base branch gave. The new check at main.py:863-870 raises VerificationError before calling current_session().finish(outcome) at :871, so Session.finish's handed_off check (lifecycle.py:273) never runs for this input, and finish exits 1 with "Baseline failure requires a fixed reason-code" instead of writing a handoff outcome.json. recover-current's reconcile() (lifecycle.py:455) still detects handed_off later and releases the family, so this is not a stuck state, but finish's own contract to handle handoff regardless of outcome shape is now violated for this input class. …
Why this was flagged
…Fix: check handed_off (or otherwise defer the reason-code requirement) before enforcing it, so any handed-off PR still short-circuits to a handoff outcome as before.
Trigger: a maintainer applies the klaud-handoff label to a candidate PR while requested-outcome.json (written by the agent per .github/klaud-candidate-prompt.md) has outcome="failed", phase="baseline" and no reason-code (agent bug, or an older outcome file). Entry: the finish CLI command, run by the agent via klaud finish --outcome-file ... (main.py:862-871). The added gate at :863-870 raises VerificationError before current_session().finish(outcome) executes, so Session.finish (lifecycle.py:269-280), which checks self.handed_off(pull) first and would return a clean CandidateOutcome(outcome="handoff",...), never runs. Before this change, finish() always ran and unconditionally recognized handoff regardless of the outcome's shape. Now the CLI exits 1 with a misleading baseline-reason error and skips writing outcome.json for that invocation; only the later always-run recover-current step (calling…
Verification: Severity: nit. The candidate's mechanism is correctly identified and reachable. The new gate at infx/klaud/main.py:865-870 raises VerificationError when outcome=="failed", phase=="baseline", and reason_code is falsy — and it runs BEFORE current_session().finish(outcome) at line 871. Session.finish at infx/klaud/lifecycle.py:273-280 short-circuits `if pull and self.handed_off(pull):…
1 participant
When Klaud's planner passed baseline preflight, the candidate later fetched and reconstructed the same evidence again. A baseline stop also produced only
failed/baseline, leaving the blocker unclassified. The planner now hands its candidate-bound baseline roster to the candidate job, which validates the binding and reuses that roster. Baseline failures must include one fixed, sanitized reason code in the outcome, diagnostics, and completion report.This does not establish the cause of earlier baseline failures; it makes subsequent failures diagnosable without exposing raw agent output or private telemetry.
Validation: 14 Klaud tests and 26 SRT launcher cases passed locally; Ruff check and format check passed for the Klaud package; workflow security audit found no new findings. The SRT test fixture now recognizes
git -c … clone, fixing 14 pre-existing CI failures also observed onmain. No GPU benchmark was dispatched for this tooling change.AI model disclosure: GPT-6 (the exact runtime model identifier was not exposed) prepared the code, tests, and documentation. No delegated agents contributed.
中文
此前 Klaud 的规划器通过基线预检后，候选任务仍会再次获取并重建相同证据。基线阶段停止时也只记录
failed/baseline，无法区分具体阻碍。现在规划器将绑定候选任务的基线结果交给候选任务；候选任务验证绑定关系后复用该结果。基线失败必须在结果、诊断信息和完成报告中包含一个固定的脱敏原因代码。此变更不能确定此前基线失败的原因；它使后续失败可以在不暴露原始代理输出或私有遥测的情况下得到分类。
验证：14 项 Klaud 测试和 26 项 SRT 启动器测试在本地通过；Klaud 包的 Ruff 检查及格式检查通过；工作流安全审计没有发现新问题。SRT 测试夹具现在可识别
git -c … clone，修复了在main上也出现的 14 项既有 CI 失败。本次工具修改没有调度 GPU 基准测试。AI 模型披露： GPT-6（运行环境未提供精确的模型标识符）编写了代码、测试和文档。没有委派其他代理参与。