Repository navigation
Keep Evolver out of the model context: report at turn end, distill at session end - #51
Merged
Merged
Conversation
… session end The plugin no longer registers evolver_* tools or the capability-evolver skill, and the injected strategy ends with "Apply them where they fit." instead of asking the agent to search, report or distill. Reuse is reported by the turn-end hook (now also as aborted when a session goes away or is killed mid-turn); a verified turn leaves a candidate on disk, distilled once at session end from a digest of the turn through the host model. /evolver-* commands run in the plugin and answer in the UI. Measured on 60 tasks x 2 attempts with real Hub recall (dsh-local-bench, layer 4): the form without tools or skill used the fewest context tokens per step, and the stock plugin loaded the skill in 62/120 trials and reported by hand in 53. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Sentinel 正在审核此变更，尚未形成最终结论。 大 PR 拆片审核中
开发 agent 查询详情： 调用 MCP 工具 由 Sentinel 审核 · 目标提交 15f3772b6e9d |
… default) The skill is rewritten for the hook-only design: it explains what runs on its own and points to the /evolver-* commands, names no tool, and no longer tells the agent to load it before every substantive task. Off by default, because its catalog line rides every turn and its body every later step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bugbot is paused — on-demand spend limit reachedBugbot uses usage-based billing for this team and has hit its on-demand spend limit. A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue. |
|
Sentinel 已审核此变更，并发现 1 个已核验问题。 另有 17 个需要人工确认的风险；黄色 advisory 不阻塞合并，建议开发 agent review advisory 评论并逐条确认。 审核覆盖提示： 开发 agent 查询详情： 调用 MCP 工具 由 Sentinel 审核 · 目标提交 883c43a8068c |
|
Sentinel 保留了 17 个尚未确认的审核候选。这些黄色条目不阻塞合并；红色 P0/P1 与其他独立硬门禁不受影响。 单一猜测、缺少变更行证据或纯 checklist 的候选会统一降为 P3。
需要完整证据时，可用行末 由 Sentinel 审核 · 目标提交 883c43a8068c |
![evomap-aiqa-bot[bot]](https://avatars.githubusercontent.com/u/278697308?s=60&v=4){kind=small}
| const pending = readState(pendingPath(), {}); | ||
| const due = Object.entries(pending) | ||
| .filter(([id, candidate]) => id === sessionId || now - Number(candidate?.at) > staleMs); | ||
| if (due.length === 0) return []; |
There was a problem hiding this comment.
蒸馏任务在异步执行前即被删除，失败后无法重试
🟠 严重程度：中（P2）
takeCandidates() 在模型调用开始前便从 distill-pending.json 删除候选并写盘；之后 distillCandidate() 遇到无模型、超时、流式调用异常、无法解析 JSON 或代理请求失败时只返回失败状态。distillSession() 和 distillLeftovers() 均未把候选写回，因此临时失败被错误地当作生命周期完成，重试与下次启动补偿路径都无法再发现该任务。
Finding：finding_2c0451dc04801473cb308a36
…sions Addresses the Sentinel review of #51: - A candidate is no longer removed before the model call: it is settled only once distilled, and a failed try (no model, timeout, unparsed answer, Proxy error) keeps it for the next flush or session start, up to three tries. One distillation per session runs at a time. - A session counts as killed only when the dsh process that wrote its ledger or candidate is gone (signal-0 probe on the recorded pid), not when it has been idle for a while; entries without a pid are left alone. - Keys and tokens are stripped from the prompt, commands, checks and answer before a candidate is written. - Command arguments keep quoted words together; a missing `evolver` executable falls back to `npx -y @evomap/evolver` again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Sentinel review of 883c43a addressed in e5baf2b (172 tests pass):
🤖 Generated with Claude Code |
|
Sentinel 已审核此变更，并发现 3 个已核验问题。 另有 16 个需要人工确认的风险；黄色 advisory 不阻塞合并，建议开发 agent review advisory 评论并逐条确认。 审核覆盖提示： 开发 agent 查询详情： 调用 MCP 工具 由 Sentinel 审核 · 目标提交 e5baf2b9d86d |
|
Sentinel 保留了 16 个尚未确认的审核候选。这些黄色条目不阻塞合并；红色 P0/P1 与其他独立硬门禁不受影响。 单一猜测、缺少变更行证据或纯 checklist 的候选会统一降为 P3。
需要完整证据时，可用行末 由 Sentinel 审核 · 目标提交 e5baf2b9d86d |
| const pending = readState(pendingPath(), {}); | ||
| const current = pending[sessionId]; | ||
| if (!current || current.at !== candidate.at) return; | ||
| const attempts = (current.attempts ?? 0) + 1; |
There was a problem hiding this comment.
失败或被新候选替换的蒸馏任务没有后续调度
🟠 严重程度：中（P2）
失败时只增加 attempts 并保留候选，没有定时或即时重试。下一个 session 启动时，distillLeftovers() 只处理 owner PID 已退出的候选；如果宿主进程继续运行，已 disposed 的旧 session 候选不会再被领取。类似地，旧候选蒸馏期间产生新候选时，at 校验会保留新候选，但共享旧 inFlight 的最终 flush 不会为新候选再调度一次。
Finding：finding_fc2929d3b984283e18c36354
| * process still runs keeps its candidate however long it waits. | ||
| */ | ||
| export function dueCandidates({ sessionId, isGone = ownerIsGone }) { | ||
| return Object.entries(readState(pendingPath(), {})) |
There was a problem hiding this comment.
孤儿候选没有跨进程原子领取
🟠 严重程度：中（P2）
dueCandidates() 只读取并筛选候选，网络提交完成后才由 settleCandidate() 删除；没有文件锁、claim 状态或 CAS。inFlight 只是单进程 Map，因此多个宿主可同时领取并持久化同一候选。各写入函数还采用独立的读改写，跨进程 settle/remember/drop 时可覆盖其他 session 的更新。
Finding：finding_fe6fa8af74728b2f5a0b86bc
| checks: record.checks.map(({ command, failed }) => ({ command: withoutSecrets(command), failed })), | ||
| answer: withoutSecrets(record.answer).slice(0, ANSWER_MAX_CHARS), | ||
| model, | ||
| turn, |
There was a problem hiding this comment.
文件路径和手动 focus 绕过秘密脱敏
🟠 严重程度：中（P2）
prompt、命令、检查和答案经过 withoutSecrets()，但 record.paths 原样写盘并加入模型提示；命令入口传入的 focus 也被原样插入提示。模型生成的 args 在提交代理前同样未再次脱敏，因此模型若复述路径或 focus，内容还会继续进入持久化代理请求。
Finding：finding_7700e577b672de3073fb1b6e
Addresses the second Sentinel review of #51: - `updateState()` reads, changes and writes a state file under an exclusive lock whose dead owner is taken over, so two dsh processes sharing `~/.evolver/state` no longer overwrite each other. The candidate file and the injection ledgers go through it; four parallel writers keep all 60 entries, against 17 unlocked. - A session that ends marks its candidate closed, so a distillation that failed at its end is tried again at the next session start instead of waiting for the host process to die. Leftovers are claimed under the lock by one process at a time, and a failed try releases the claim. - Touched paths and a `/evolver-distill` focus go through `withoutSecrets()` like the rest of the digest. - Every ledger entry is stamped with the live session and pid on each injection, so entries written before this version, or by the process a resumed session ran in, are neither orphaned nor unclaimable. - A cancelled or timed-out CLI command takes down the process group it leads, and the npx fallback is held to `@evomap/evolver@^2.0.39`, the range this plugin is built against. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bugbot is paused — on-demand spend limit reachedBugbot uses usage-based billing for this team and has hit its on-demand spend limit. A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue. |
Contributor
Author
|
Sentinel review of e5baf2b addressed in 7dc84d0 (178 tests pass). Verified findings
Advisories
🤖 Generated with Claude Code |
|
Sentinel 已审核此变更，并发现 1 个已核验问题。 另有 18 个需要人工确认的风险；黄色 advisory 不阻塞合并，建议开发 agent review advisory 评论并逐条确认。 审核覆盖提示： 开发 agent 查询详情： 调用 MCP 工具 由 Sentinel 审核 · 目标提交 7dc84d077405 |
|
Sentinel 保留了 18 个尚未确认的审核候选。这些黄色条目不阻塞合并；红色 P0/P1 与其他独立硬门禁不受影响。 单一猜测、缺少变更行证据或纯 checklist 的候选会统一降为 P3。
需要完整证据时，可用行末 由 Sentinel 审核 · 目标提交 7dc84d077405 |
| distillLeftovers() { | ||
| for (const due of claimLeftoverCandidates()) once(due).catch(() => {}); | ||
| }, | ||
| }; |
There was a problem hiding this comment.
纠正提示无法取消已经开始的陈旧蒸馏
🟠 严重程度：中（P2）
纠正路径只通过 dropCandidate(sessionId)删除磁盘候选，没有取消对应的 inFlight 调用，也没有在代理 POST 前重新验证候选是否仍是当前权威版本。因此，删除成功后，已启动的模型请求仍可完成并将这个“已判定不值得蒸馏”的旧候选持久化；之后 settleCandidate()因候选已不存在而仅成为空操作。
Finding：finding_1f29b180026f775562b7cf5d
A correction now also aborts a distillation already running for the session, and every distillation asks once more whether its candidate is still the one on disk before it writes to the Proxy, so a turn judged not worth keeping is never persisted. A lock carries a token unique to its acquisition. A dead owner's lock is moved aside before it is deleted and put back when what was moved is not the lock that was inspected, and a process releases a lock only while it still holds its own token, so reclaiming a stale lock can no longer delete one another process just took. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Sentinel review of 7dc84d0: the two items below are fixed in 20f20d9 (181 tests pass).
The remaining yellow advisories are unchanged by this commit. |
|
Sentinel 正在审核此变更，尚未形成最终结论。 大 PR 拆片审核中
开发 agent 查询详情： 调用 MCP 工具 由 Sentinel 审核 · 目标提交 20f20d9d4f8c |
The candidate leaves the disk once the Proxy answers, a moment after the request arrives; asserting right after the request raced the settle on slower CI runners. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Sentinel 已审核此变更，并发现 2 个已核验问题。 另有 20 个需要人工确认的风险；黄色 advisory 不阻塞合并，建议开发 agent review advisory 评论并逐条确认。 审核覆盖提示： 开发 agent 查询详情： 调用 MCP 工具 由 Sentinel 审核 · 目标提交 0a824f3b1f20 |
|
Sentinel 保留了 20 个尚未确认的审核候选。这些黄色条目不阻塞合并；红色 P0/P1 与其他独立硬门禁不受影响。 单一猜测、缺少变更行证据或纯 checklist 的候选会统一降为 P3。
需要完整证据时，可用行末 由 Sentinel 审核 · 目标提交 0a824f3b1f20 |
| fs.renameSync(lockPath, aside); | ||
| } catch { | ||
| return; | ||
| } |
There was a problem hiding this comment.
三方陈旧锁竞争仍可让两个 owner 同时进入临界区
🔴 严重程度：高（P1）
A 在读取旧 token 后执行 renameSync(lockPath, aside)，可能实际移走 B 刚创建的新锁。A 检测到 token 不同后尝试用硬链接恢复，但如果 C 已占用 lockPath，linkSync() 会失败且异常被吞掉；随后 A 删除 aside。结果 B 的有效锁被永久移除，而 C 持有路径上的新锁，B 与 C 可同时执行受保护的读改写。
Finding：finding_99a609880648b92290cd6088
| const lockPath = `${target}.lock`; | ||
| const token = acquire(lockPath, Date.now() + LOCK_WAIT_MS); | ||
| try { | ||
| const next = mutate(readState(target, fallback)); |
There was a problem hiding this comment.
锁等待超时后仍执行无锁读改写
🟠 严重程度：中（P2）
acquire() 在超时或非 EEXIST 错误时返回 null，但 updateState() 随后仍执行完整的 readState → mutate → writeState。此时持锁进程也可能继续写入，两个完整快照会互相覆盖。代码注释明确将该行为作为“不阻塞会话”的回退，但它绕过了本次新增锁要提供的互斥硬边界。
Finding：finding_90ff6295456d59694728cd5f
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Nothing of Evolver but the recalled steps enters the model's context. Reporting and distillation move to session events; commands run in the plugin.
Evidence
After: the same plugin without tools, skill or nudges ("clean"): first-step context 6.2k, zero skill loads, manual reports or distill calls; on the pairs both solved, −32% total tokens on procedural tasks, −14% (median −28%) on logic tasks. This PR is that form plus hook-only reporting and session-end distillation.
npm test: 167 pass, 0 fail. Newtest/session-distill.test.js(9): candidate rules, digest prompt, Proxy body withdistill_usage, a verified turn distilled exactly once at session end, a corrected turn not distilled, an open strategy reported failed/aborted onsession/disposed, a killed session's reports and candidate taken over at the next session start.applytest now asserts zero tools and zero skills.Merge Danger
Door: two-way
Revert restores the tools, skill and old commands; state files added (
~/.evolver/state/distill-pending.json,sessionIdin injected ledgers) are ignored by older versions.Blast Radius: agent-surface
evolver_search_assets/evolver_fetch_asset/evolver_publish_asset/evolver_poll/evolver_ackthemselves; people use/evolver-search,/evolver-syncetc. Anything that relied on the model publishing assets or acking mailbox messages stops.mismatched) disappear; misfit now shows only asapplied: unusedfrom the hook. In the benchmark the model reported misleading assets assuccessin most trials, so this loses little signal.session/flushwhen a session closes; if it also flushes mid-session the candidate is distilled early (a later verified turn makes a new one). Worth confirming in a real dsh session.🤖 Generated with Claude Code