diff --git a/.github/pr-screenshots/overview-usage-partial/after-dark.png b/.github/pr-screenshots/overview-usage-partial/after-dark.png new file mode 100644 index 000000000..0fc21a39a Binary files /dev/null and b/.github/pr-screenshots/overview-usage-partial/after-dark.png differ diff --git a/.github/pr-screenshots/overview-usage-partial/after-light.png b/.github/pr-screenshots/overview-usage-partial/after-light.png new file mode 100644 index 000000000..535ae56ba Binary files /dev/null and b/.github/pr-screenshots/overview-usage-partial/after-light.png differ diff --git a/.github/pr-screenshots/overview-usage-partial/before-dark.png b/.github/pr-screenshots/overview-usage-partial/before-dark.png new file mode 100644 index 000000000..06a1b8e0d Binary files /dev/null and b/.github/pr-screenshots/overview-usage-partial/before-dark.png differ diff --git a/.github/pr-screenshots/overview-usage-partial/before-light.png b/.github/pr-screenshots/overview-usage-partial/before-light.png new file mode 100644 index 000000000..df158a8c3 Binary files /dev/null and b/.github/pr-screenshots/overview-usage-partial/before-light.png differ diff --git a/changes/1908-usage-partial-total.md b/changes/1908-usage-partial-total.md new file mode 100644 index 000000000..e40bcfde1 --- /dev/null +++ b/changes/1908-usage-partial-total.md @@ -0,0 +1,11 @@ +--- +section: Fixed +--- + +## en + +- The Overview's token usage no longer shows "Unknown" for the whole week when some usage went unreported, for example rows kept from before an upgrade. The total now counts what was reported and is marked with **+**, and each day's bar shows its reported usage ([#1908](https://github.com/BotHarness/DeepSeekBot/pull/1908)). + +## zh + +- 总览的 token 用量不再因为部分用量未上报（例如升级前保留的旧记录）而整周显示“未知”。总量现在只计入已上报的部分并标记 **+**，每天的柱子也显示已上报的用量（[#1908](https://github.com/BotHarness/DeepSeekBot/pull/1908)）。 diff --git a/docs/architecture/botharness-architecture.md b/docs/architecture/botharness-architecture.md index ebdaa28a1..0905ae428 100644 --- a/docs/architecture/botharness-architecture.md +++ b/docs/architecture/botharness-architecture.md @@ -801,7 +801,7 @@ application-defined `botharness/channelActivityToday` 查询由 Channel owner 总览优先排列需 Human 行动的 Bot，并先展示 canonical 行动表单，再显示执行中的 Session。显式全部已读命令先固定每个 Human 可见 Channel 的消息位置，再推进既有已读游标；之后到达的消息仍未读，不改变请求解决、Inbox 移除或 Bot attention。统计复用锁定版本的 TanStack 图表与主题，以前端偏好记忆折叠状态，并保留可访问的发送者明细。 ([#705](https://github.com/BotHarness/BotHarness/issues/705)). -总览 token 统计（#709）通过应用定义的 `overviewUsage(period, after?)` Typert 查询消费既有保留 Usage 权威。今日或含今日的七个 Host 本地日期返回完整整体总量／每日分项，与有界 Bot 明细分页独立；缺失报告保持可空未知，核对和历史基线状态明确。当前 Registry 名称标记行；不在当前 Bot 列表中的保留统计仍可见，但不虚构身份或 Profile 入口。Client 复用 Profile UsageChart／主题，以可清理的轮询读取已结算事实，并通过既有 DM 导航打开当前 Bot Profile。不增加账本、Channel token 归因或模型／wake 策略。 +总览 token 统计（#709）通过应用定义的 `overviewUsage(period, after?)` Typert 查询消费既有保留 Usage 权威。今日或含今日的七个 Host 本地日期返回完整整体总量／每日分项，与有界 Bot 明细分页独立；缺失报告不再让整段合计变成未知：每个桶返回已上报部分的合计，只有完全没有上报值的桶才是空；总量缺失的行用其已上报分项作下限，并以 `partial` 标记该合计不完整，Web 与手机显示为“N+”；核对和历史基线状态明确。当前 Registry 名称标记行；不在当前 Bot 列表中的保留统计仍可见，但不虚构身份或 Profile 入口。Client 复用 Profile UsageChart／主题，以可清理的轮询读取已结算事实，并通过既有 DM 导航打开当前 Bot Profile。不增加账本、Channel token 归因或模型／wake 策略。 总览 Memory 统计（#716）通过应用定义的 `overviewMemory(after?)` Typert 查询读取现有 Memory Service 与 Registry。每页至多十个当前 Bot；每个仓库的所有非恢复／stash 引用可达的普通 Git 提交按 committer 时间归入含今日的七个 Host 本地日期，共享提交仅计一次。当前 staged／unstaged／untracked 状态单独显示，不推断文件作者或审批需求；缺失、无效、超时或读取失败的仓库显示不可用，不伪装为零提交或 clean。仓库 Git 查询有超时并异步执行，避免阻塞共享 Host；查询关闭可选 Git 锁与 fsmonitor，不 stage、commit、reconcile 或建立 checkpoint；沿用 ADR-0068 的仓库权威，不增持久统计账本。Client 复用 Profile 紧凑卡片及 TanStack 主题；可展开每日值，保留已加载分页刷新，折叠 Statistics／退出视图时清理查询资源。 diff --git a/docs/bot-cli.md b/docs/bot-cli.md index fc5beeb64..67e192405 100644 --- a/docs/bot-cli.md +++ b/docs/bot-cli.md @@ -498,7 +498,7 @@ deepseekbot group-activity `model-catalog` lists the models the Host can run and the Profile default, as the model pickers do. `model-plan-set` replaces a Bot's whole plan with stdin `{"orchestrator":{...},"assignmentDefault":{...},"assignmentModels":[...]}`; `model-plan-assignments-set` takes the same object without `orchestrator`. A route is `{"provider","model","reasoningEffort"?}` and an Assignment model is `{"provider","model","allowedEfforts":[...],"defaultEffort","note"?}`. `model-plan-customize` changes only the Orchestrator of a Bot that already has a plan. `model-plan-inherit` drops the Bot's own model so it follows the Profile default. `model-preset-update` changes a preset's name or routes; flags you leave out keep the current values, and an optional `--plan-stdin` `{"assignmentModels":[...]}` replaces its Assignment models. The Host validates every route against its catalog. Plan and preset writes check a revision: `--expected-revision` makes that check explicit; without it the command reads the current revision first, like `memory-save` does with `--expected-head`. A plan changed meanwhile fails with the Host's `invalid-input`. -`overview-usage` (`--period today|week`, default `today`) and `overview-memory` return the Overview page's per-Bot rows; pass `nextCursor` back as `--after` for more. `activity-overview` shows what every Bot is doing and `channel-activity-today` today's messages per Channel. The `bot-*` readers address a Bot by id and read the Profile panels through its DM: `bot-usage` needs a non-future `--start`/`--end` range shorter than 182 days, `bot-session-usage` groups usage by Session and Assignment, and `bot-activity-history` returns the activity calendar (`--weeks` 1–104). `group-activity` takes a group Channel id. +`overview-usage` (`--period today|week`, default `today`) and `overview-memory` return the Overview page's per-Bot rows; pass `nextCursor` back as `--after` for more. In `overview-usage`, each bucket is the sum of what providers reported and is `null` only when nothing in it was reported; `partial: true` on the totals, a day or a Bot means some usage in it went unreported, so its `totalTokens` is a lower bound. `activity-overview` shows what every Bot is doing and `channel-activity-today` today's messages per Channel. The `bot-*` readers address a Bot by id and read the Profile panels through its DM: `bot-usage` needs a non-future `--start`/`--end` range shorter than 182 days, `bot-session-usage` groups usage by Session and Assignment, and `bot-activity-history` returns the activity calendar (`--weeks` 1–104). `group-activity` takes a group Channel id. ### Marketplace, releases, usage statistics, Git and pairing review diff --git a/packages/client/src/client/bridge.ts b/packages/client/src/client/bridge.ts index ca02ae09a..b6d6bc425 100644 --- a/packages/client/src/client/bridge.ts +++ b/packages/client/src/client/bridge.ts @@ -5296,7 +5296,8 @@ export async function loadOverviewUsage( row !== undefined && ['inputTokens', 'outputTokens', 'cacheReadTokens', 'cacheWriteTokens', 'totalTokens'].every( (key) => row[key] === null || (Number.isSafeInteger(row[key]) && Number(row[key]) >= 0), - ) + ) && + (row['partial'] === undefined || typeof row['partial'] === 'boolean') ); }; if ( diff --git a/packages/client/src/client/locale.ts b/packages/client/src/client/locale.ts index 972cd899a..c6c9add68 100644 --- a/packages/client/src/client/locale.ts +++ b/packages/client/src/client/locale.ts @@ -593,7 +593,7 @@ export const zh = { 'overviewUsage.reconciling': '正在核对用量，数据尚未完整。', 'overviewUsage.degraded': '用量核对失败，当前统计可能不完整。', 'overviewUsage.legacy': '历史基线尚未完全归类。', - 'overviewUsage.partial': '部分 token 桶或总量未知；图表仅绘制已确认的用量。', + 'overviewUsage.partial': '部分用量未上报；带 + 的总量和图表只计入已上报的用量。', 'overviewUsage.trend': '每日 token 用量趋势', 'overviewUsage.daily': '每日精确值', 'overviewUsage.bots': '按 Bot', @@ -4173,7 +4173,7 @@ export const en = { 'overviewUsage.degraded': 'Usage reconciliation failed; statistics may be incomplete.', 'overviewUsage.legacy': 'Historical baseline is not fully classified.', 'overviewUsage.partial': - 'Some token buckets or totals are unknown; the chart shows only confirmed usage.', + 'Some usage was not reported; totals marked + and the chart count only reported usage.', 'overviewUsage.trend': 'Daily token usage trend', 'overviewUsage.daily': 'Exact daily values', 'overviewUsage.bots': 'By Bot', diff --git a/packages/client/src/client/overview-usage-view.tsx b/packages/client/src/client/overview-usage-view.tsx index 9ce10773f..d96f3e1d5 100644 --- a/packages/client/src/client/overview-usage-view.tsx +++ b/packages/client/src/client/overview-usage-view.tsx @@ -173,7 +173,11 @@ export function OverviewUsageView({ {t('overviewUsage.total')}:{' '} - {count(value.totals.totalTokens)} tokens + + {count(value.totals.totalTokens)} + {value.totals.partial && value.totals.totalTokens !== null ? '+' : ''} + {' '} + tokens {value.freshness !== 'ready' ? (

@@ -181,7 +185,8 @@ export function OverviewUsageView({

) : null} {value.legacyBaseline ?

{t('overviewUsage.legacy')}

: null} - {value.days.some((row) => Object.values(row).some((amount) => amount === null)) ? ( + {value.totals.partial || + value.days.some((row) => Object.values(row).some((amount) => amount === null)) ? (

{t('overviewUsage.partial')}

) : null} {rows.some((row) => row.totalTokens !== null && row.totalTokens > 0) ? ( diff --git a/packages/client/test/overview-usage-view.test.ts b/packages/client/test/overview-usage-view.test.ts index 2e1c006ae..b444f1645 100644 --- a/packages/client/test/overview-usage-view.test.ts +++ b/packages/client/test/overview-usage-view.test.ts @@ -45,6 +45,7 @@ function result(period: 'today' | 'week', total = 155): OverviewUsage { cacheReadTokens: 0, cacheWriteTokens: 0, totalTokens: total, + partial: false, }; return { period, @@ -181,6 +182,37 @@ it('drops a delayed Today refresh after changing range and surfaces unavailable container.remove(); } }); +it('marks a total that leaves out unreported usage as a lower bound', async () => { + Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + const value = result('today'); + value.totals.partial = true; + const call: BridgeCall = async () => ({ ok: true, value }); + const container = document.createElement('div'); + document.body.append(container); + const root = createRoot(container); + try { + await act(async () => + root.render( + createElement(OverviewUsageView, { + actions: createActions(call, createStore()), + t: zhTranslate, + }), + ), + ); + expect(container.querySelector('[data-usage-total]')?.textContent).toBe('155+'); + expect(container.textContent).toContain(zhTranslate('overviewUsage.partial')); + } finally { + await act(async () => root.unmount()); + container.remove(); + } + await expect( + loadOverviewUsage( + async () => ({ ok: true, value: { ...value, totals: { ...value.totals, partial: 'yes' } } }), + 'today', + ), + ).rejects.toThrow('invalid Overview usage'); +}); + it('preserves unknown totals and rejects malformed usage instead of reporting zero', async () => { const value = result('today'); value.totals.totalTokens = null; diff --git a/packages/core/src/usage/overview.ts b/packages/core/src/usage/overview.ts index ecb20bd36..8b33eab8f 100644 --- a/packages/core/src/usage/overview.ts +++ b/packages/core/src/usage/overview.ts @@ -7,6 +7,7 @@ export interface UsageOverviewBuckets { cacheReadTokens: number | null; cacheWriteTokens: number | null; totalTokens: number | null; + partial: boolean; } export interface UsageOverviewResult { period: UsageOverviewPeriod; @@ -27,23 +28,33 @@ function dayOf(date: Date): string { return `${date.getFullYear()}-${String(date.getMonth() + 1).padStart(2, '0')}-${String(date.getDate()).padStart(2, '0')}`; } const columns = [ - ['input_tokens', 'unknown_input', 'inputTokens'], - ['output_tokens', 'unknown_output', 'outputTokens'], - ['cache_read_tokens', 'unknown_cache_read', 'cacheReadTokens'], - ['cache_write_tokens', 'unknown_cache_write', 'cacheWriteTokens'], - ['total_tokens', 'unknown_total', 'totalTokens'], -] - .map( + ...[ + ['input_tokens', 'unknown_input', 'inputTokens'], + ['output_tokens', 'unknown_output', 'outputTokens'], + ['cache_read_tokens', 'unknown_cache_read', 'cacheReadTokens'], + ['cache_write_tokens', 'unknown_cache_write', 'cacheWriteTokens'], + [ + 'CASE WHEN unknown_total > 0 THEN MAX(total_tokens, input_tokens + output_tokens + cache_read_tokens + cache_write_tokens) ELSE total_tokens END', + 'unknown_total', + 'totalTokens', + ], + ].map( ([value, unknown, alias]) => - `CASE WHEN SUM(${unknown}) > 0 THEN NULL ELSE COALESCE(SUM(${value}), 0) END AS ${alias}`, - ) - .join(', '); + `CASE WHEN COALESCE(SUM(${value}), 0) = 0 AND SUM(${unknown}) > 0 THEN NULL ELSE COALESCE(SUM(${value}), 0) END AS ${alias}`, + ), + 'COALESCE(SUM(unknown_total), 0) > 0 AS partial', +].join(', '); +type Row = Omit & { partial: number }; +function bucketsOf(row: T): Omit & { partial: boolean } { + return { ...row, partial: row.partial > 0 }; +} const empty: UsageOverviewBuckets = { inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0, totalTokens: 0, + partial: false, }; export function queryOverviewUsage( db: DatabaseSync, @@ -61,19 +72,23 @@ export function queryOverviewUsage( date.setDate(date.getDate() + 1); } while (days.at(-1) !== end); const start = days[0]!; - const totals = db - .prepare(`SELECT ${columns} FROM usage_daily WHERE day >= ? AND day <= ?`) - .get(start, end) as unknown as UsageOverviewBuckets; + const totals = bucketsOf( + db + .prepare(`SELECT ${columns} FROM usage_daily WHERE day >= ? AND day <= ?`) + .get(start, end) as unknown as Row, + ); const daily = db .prepare( `SELECT day, ${columns} FROM usage_daily WHERE day >= ? AND day <= ? GROUP BY day ORDER BY day`, ) - .all(start, end) as unknown as UsageOverviewResult['days']; + .all(start, end) + .map((row) => bucketsOf(row as unknown as Row & { day: string })); const bots = db .prepare( `SELECT bot_slug AS slug, ${columns} FROM usage_daily WHERE day >= ? AND day <= ? AND bot_slug > ? GROUP BY bot_slug ORDER BY bot_slug LIMIT 21`, ) - .all(start, end, after ?? '') as unknown as UsageOverviewResult['bots']; + .all(start, end, after ?? '') + .map((row) => bucketsOf(row as unknown as Row & { slug: string })); return { period, start, diff --git a/packages/core/test/overview-usage.test.ts b/packages/core/test/overview-usage.test.ts index ab62d4ca4..25f5bdc6d 100644 --- a/packages/core/test/overview-usage.test.ts +++ b/packages/core/test/overview-usage.test.ts @@ -7,7 +7,7 @@ import { expect, it, vi } from 'vitest'; import { createBridgeMethods } from '../src/bridge/methods.js'; import { createChannelStore } from '../src/channels/store.js'; -import { mountOperationalDatabase } from '../src/database/owner.js'; +import { attachOperationalModule, mountOperationalDatabase } from '../src/database/owner.js'; import { BOT_HARNESS_SCHEMA_PLAN } from '../src/database/schema-plan.js'; import { createBotStateTracker } from '../src/state/bot-state.js'; import { createUsageProjection, usageLocalDay } from '../src/usage/usage.js'; @@ -281,6 +281,35 @@ it('distinguishes a ready empty interval from missing provider usage and an unav expect(unknown.ok && unknown.value.days[0]?.inputTokens).toBeNull(); }); +it('counts reported usage beside an unreported total and marks the sum as partial', () => { + const { methods, fold, owner, time } = setup(); + fold({ inputTokens: 100, outputTokens: 40, totalTokens: 155 }); + attachOperationalModule(owner, 'usage').transaction((db) => + db + .prepare( + `INSERT INTO usage_daily + (bot_slug, day, provider, model, purpose, input_tokens, output_tokens, cache_read_tokens, + cache_write_tokens, total_tokens, unknown_cache_read, unknown_cache_write, unknown_total) + VALUES ('legacy', ?, 'actual', 'old', 'orchestrator', 300, 60, 0, 0, 0, 1, 1, 1)`, + ) + .run(usageLocalDay(time)), + ); + const result = methods.overviewUsage({ period: 'week' }); + if (!result.ok) throw new Error(result.error.message); + expect(result.value.totals).toMatchObject({ + inputTokens: 400, + outputTokens: 100, + cacheReadTokens: null, + totalTokens: 515, + partial: true, + }); + expect(result.value.days.at(-1)).toMatchObject({ totalTokens: 515, partial: true }); + expect(result.value.days[0]).toMatchObject({ totalTokens: 0, partial: false }); + expect( + Object.fromEntries(result.value.bots.map((bot) => [bot.slug, [bot.totalTokens, bot.partial]])), + ).toEqual({ ada: [155, false], legacy: [360, true] }); +}); + it('uses seven local calendar dates across a daylight-saving change', () => { vi.stubEnv('TZ', 'America/New_York'); try {