thread-briefs: ask the summarizer for status instead of deriving it - #10
Merged
trotterdylan merged 1 commit intoOct 5, 2026
Conversation
Done was a side effect of the model leaving nextStep and blockedOn blank, while the same prompt asked for "the single most concrete next action". On the bb-dylan server 11 of 21 Done threads were Done only by a hand pin. Read against their transcripts, they failed because an offer or optional check was recorded as the step, because nextStep/blockedOn were fed back from the previous brief and outlived the turns that retired them, or because the step left was the user's own. - The prompt asks for status directly, as the stored enum, with one definition and one example each, framed as: assume the user does what the thread asks; is the task then finished? nextStep becomes descriptive. - The four done-related rules and the stage/nextStep agreement rule are gone (prompt 1100 -> ~850 words), and so is reconcileStage. The parser keeps one hard guard: a non-empty blockedOn is never done. An unreadable status falls back to waiting-on-me. - The previous brief fed back carries only title, goal, currentState and constraints. - modelStatus is stored on the row; briefs written before it keep the old derivation (legacyStatus) until their thread is next summarized. - eval/ exports real threads from a server and scores any checkout's prompt against the live model. Fixtures stay out of the repo. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trotterdylan
deleted the
bb/thread-briefs-finished-threads-almost-never-read-thr_8txf3h94jw
branch
October 5, 2026 14:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Finished threads almost never read Done. On the bb-dylan server, 11 of 21 Done threads were Done only because someone had pinned them by hand. Each of those 11 was checked against its transcript:
nextStep("want me to file an issue?", "if you want the hash confirmed, run…")nextStepis descriptive onlynextStep/blockedOncarried over from the previous brief after the thread moved onStatus used to be derived:
nextStepandblockedOnboth empty meant Done, while the prompt asked for "the single most concrete next action". This replaces the derivation instead of adding a fifth rule to it. #9 took the other route and should be closed in favour of this.What changed
nextStep.reconcileStageis removed. The parser keeps only one hard guard, which holds whatever the model says: a non-emptyblockedOnis neverdone. A status it can't read falls back towaiting-on-me.title,goal,currentStateandconstraints.modelStatusis stored on the row. Rows written before this change have nomodelStatusand use the old derivation (legacyStatus) until their thread is next summarized.eval/:export.tsfreezes real threads from a server, andrun.tsscores any checkout's prompt against the live model. Point--srcat a worktree ofmainto compare. The fixtures are real transcripts, and this repo is public, so they are not committed.Evaluation
Model:
glm-5p3-flashon Fireworks (the production setting), temperature 0. Labels follow the definition above and were agreed with the user.27 current threads (every briefed thread on bb-dylan except the one doing this work; 22 done, 5 waiting), previous brief fed back, 2 runs each:
main56 mid-thread snapshots. These are the same threads cut at points where the agent stopped and the user replied: 46 waiting, 10 done, labelled by hand, with 27 ambiguous cuts skipped. They exist to catch false Dones, which the set above has too few negatives to show:
mainWhat still misses. The model still reads a step that is the user's alone ("merge the guidance branch", "run
kploy image add") as Waiting on you about half the time. That is the safe direction, and the manual pin covers it. Pushing the prompt further toward Done on this small set would be tuning to these exact threads. The two false Dones in the snapshots are the same as onmain: an agent offering options and stopping.Archiving
A Done thread can now carry a step that is the user's, such as "approve PR #12". It is archived on the usual 48-hour clock. That is a deliberate choice, documented in both docs:
Rollout
Tests:
npm test(392 passing) andnpm run typecheck, both clean.🤖 Generated with Claude Code