Skip to content

feat(insights): real token usage from the member's computer; one row per agent role - #268

Open
hoangsnowy wants to merge 1 commit into
mainfrom
feat/real-cli-usage
Open

hoangsnowy wants to merge 1 commit into
mainfrom
feat/real-cli-usage

Conversation

@hoangsnowy

Copy link
Copy Markdown
Owner

Summary

Seen on the lab after #265: Insights showed ~6,000 tokens for three coding runs (an estimate from prompt length) and two "Coding" rows.

  • Real usage: the runner forwards the CLI's own usage (closing result event — the whole agentic run) as RemoteExecResult.InputTokens/OutputTokens/CacheRead/CacheWrite, normalised so cache ⊂ input (Claude's input_tokens excludes cache). RemoteAgentLlmClient uses it; older runners / no usage → the estimate. Server cost stays $0 (subscription); the ≈ API figure comes from real tokens.
  • One row per role: Insights merges CodingAgent (pipeline) and Coding (Board); latency call-weighted.

Test plan

  • CliProgressTests: normalised usage; none reported → null.
  • RemoteAgentTests: real usage used (incl. cache), $0 cost; older runner → estimate.
  • InsightsByRoleTests: merge + weighted latency. Full suite green.
  • Lab: after the next run, Insights shows real (much larger) token counts and cache hit rate.

🤖 Generated with Claude Code

https://claude.ai/code/session_018WAdQ7Rm24xss5QDSvXtJB

…per agent role

On the lab Insights showed ~6,000 tokens for three coding runs and two separate "Coding" rows.
- Usage: the runner sends the CLI's own usage from its closing event (the whole agentic run — every tool turn),
  normalised so cache reads/writes are a subset of the input (Claude reports input_tokens without them).
  RemoteAgentLlmClient uses it; only an older runner or a CLI that reports none falls back to the estimate. Cost on
  the server stays $0 (subscription, not billed); Insights derives its ≈ API figure from the real tokens.
- Insights merges buckets by role ("CodingAgent" from the 5-agent pipeline and "Coding" from the Board are one row;
  latency is call-weighted).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018WAdQ7Rm24xss5QDSvXtJB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant