From b6898bce8aa047e48055945a8b70461143d06de6 Mon Sep 17 00:00:00 2001 From: avatasia Date: Tue, 20 Oct 2026 16:59:22 +0800 Subject: [PATCH 01/95] The session picker asked when a session last wrote, not whether it is still in use The list was built from the mtime of the transcript file, which answers "when did this session last write a message". What a reader of a picker needs answered is "is this session still in use", and for a conversation that was just resumed the two differ until the first new message lands. Until then the conversation the reader is looking at is not in the list at all, and the slots it would have taken are taken by other sessions instead. How long the window lasted was decided by the reader's own behaviour rather than by the product: open a conversation and say something and it comes back on its own, open it and read and it stays missing for as long as nothing is written. The current conversation is now resolved from the runtime database -- the same resolution the "latest" binding already used, with the same ranking and the same qualification -- and is always present in the picker, even when its transcript file is the oldest of all the candidates. When it falls outside the recent window its entry is built from the recorded relative path, through the containment check the trajectory route already applies to that value, so a recorded path cannot point the list outside the session store. Two things the list was getting wrong are fixed in the same place, because they are the same mistake. The server never read the runtime database's purpose column, so it had no way to tell a scheduled run from the reader's own conversation, and it presented one exactly like the other: a scheduled run appeared in the list under a title that looked like something the reader had asked for. The purpose column is now read and mapped to a closed set of tags, and a scheduled run, a background task and a tool-less lightweight conversation each say what they are. They are still listed -- the reader may want to look at one -- but they no longer look like the reader's own work. Sessions of the current workspace are listed before sessions of other workspaces, each group keeping its own order by when it was last written. The comparison happens on the server and only the resulting order is published: the workspace path never reaches the page, and neither does the raw purpose string, which is free text and can carry anything a person wrote. The order of the recent-entry window is deliberately unchanged. Its first entry is the fallback answer for "latest" when the runtime database cannot be read, so reordering it would change what "latest" means -- a different question from which sessions the list offers. The version moves 1.2.0 -> 1.3.0 across the manifest and both READMEs, which is what CONTRIBUTING asks for when an existing app is updated. --- .../.minimax-plugin/plugin.json | 2 +- plugins/avatasia/mmc-trajectory/README.md | 3 +- .../avatasia/mmc-trajectory/README.zh-CN.md | 3 +- .../mmc-trajectory/miniapp/client/index.html | 24 +- .../mmc-trajectory/miniapp/node/server.mjs | 296 +++++++++++++++--- 5 files changed, 280 insertions(+), 48 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/.minimax-plugin/plugin.json b/plugins/avatasia/mmc-trajectory/.minimax-plugin/plugin.json index 32477bb8..ac0370bf 100644 --- a/plugins/avatasia/mmc-trajectory/.minimax-plugin/plugin.json +++ b/plugins/avatasia/mmc-trajectory/.minimax-plugin/plugin.json @@ -2,7 +2,7 @@ "schemaVersion": 1, "name": "mmc-trajectory", "displayName": "会话轨迹", - "version": "1.2.0", + "version": "1.3.0", "description": "按轮次浏览当前会话的模型轨迹：消息流、思考过程、工具调用与结果、token 与耗时统计。", "author": "avatasia", "icon": "icon.png", diff --git a/plugins/avatasia/mmc-trajectory/README.md b/plugins/avatasia/mmc-trajectory/README.md index 618e6cee..5467994a 100644 --- a/plugins/avatasia/mmc-trajectory/README.md +++ b/plugins/avatasia/mmc-trajectory/README.md @@ -4,7 +4,7 @@ English | [简体中文](README.zh-CN.md) Browse the model trajectory of a MiniMax Code conversation: messages, reasoning, tool calls and their results, Token usage, and per-turn timing. -Author: [avatasia](https://github.com/avatasia) · Version: `1.2.0` +Author: [avatasia](https://github.com/avatasia) · Version: `1.3.0` ### Before you change this page @@ -132,6 +132,7 @@ The runtime database is opened in read-only mode. If it is missing, unreadable, - The ledger does not use virtual scrolling; every rendered row is a real element. The page therefore opens on the newest 200 records, and the window bar button switches to the whole session. The button names what pressing it will do, not the current state: 「显示全部」 at the default and 「显示最新 200 条」 once everything is showing, while the count beside it reports what is on screen. That full view is opt-in for a reason: on a 9,900-record session it is roughly 57,000 elements rebuilt every 2.5s poll, which was measured to freeze the page rather than be read. 「向上加载更早的 N 条」 widens the window in steps and a poll does not undo it. A sticky window bar under the overview strip offers that choice behind four explicit controls: switch the strip between equal width and a real time axis, load 200 earlier records, jump back to the latest 200, and show everything. "Show everything" builds every record in the session into the DOM at once and gets noticeably slower on large sessions; incoming records do not knock it back down to 200. - `messageCount` and `turnCount` in the session picker are estimates derived from a bounded prefix of the file; exact values come from the selected session. - A session the runtime spawned from another one is marked 「子会话」 in the picker. The parent is deliberately not named: a branch is always newer than the session that spawned it, so its parent has usually aged out of the most-recent-N window this list is built from, and a lookup that fails quietly is how a picker ends up asserting something it cannot see. +- The picker also says what a session **is**, not only what it is called: a scheduled run carries 「定时」, a background task 「任务」, and a tool-less lightweight conversation 「轻量」, so none of them can be mistaken for a conversation you wrote. They stay in the list and stay selectable — a marker is a marker, not a filter. The conversation you are in comes first, then the sessions of its own workspace, then the rest; the workspace path and the raw purpose string are compared on the server and never reach the page. - Light theme token coverage is verified — every `--mcode-*` token the page consumes is defined for both themes — but its rendered appearance was not visually checked; the page was exercised under a dark system preference. ## Tested environment diff --git a/plugins/avatasia/mmc-trajectory/README.zh-CN.md b/plugins/avatasia/mmc-trajectory/README.zh-CN.md index eab669ce..628342cb 100644 --- a/plugins/avatasia/mmc-trajectory/README.zh-CN.md +++ b/plugins/avatasia/mmc-trajectory/README.zh-CN.md @@ -4,7 +4,7 @@ 按轮次浏览一段 MiniMax Code 对话的模型轨迹：消息流、思考过程、工具调用与结果、token 消耗与每轮耗时。 -作者：[avatasia](https://github.com/avatasia) · 版本：`1.2.0` +作者：[avatasia](https://github.com/avatasia) · 版本：`1.3.0` ### 动手改这个页面前 @@ -118,6 +118,7 @@ MiniMax Code 不会告诉 Mini App 是哪段对话打开了它的页面：传给 - 台里没有用虚拟滚动，每一行都是真实的 DOM 元素。页面默认只展示最新 200 条，窗口条上的按钮可以切到全量；全量是可选的，因为它很贵 —— 在这个会话上约 9,900 条、约 5.7 万个元素，每 2.5 秒轮询重建一次，实测会直接卡死页面。总览条正下方的吸顶窗口栏提供三个显式控件：「总览轴」「向上加载更早的 200 条」，以及在最新 200 条与全量之间切换的那一个按钮（它写的是按下去会变成什么，默认为「显示全部」，全量时为「显示最新 200 条」；当前窗口由右边的计数条报告，两者回答的是不同问题）。「总览轴」只改总览条的横轴——等宽（每条记录一样宽，完全忽略时间）或真实时间轴（宽度等于该记录的真实耗时）；它不筛选、不排序，台账里的行和页面上的任何计数都不变。「向上加载更早的 200 条」每点一次就多加一页，轮询不会把已经加宽的窗口重置回去；点切换按钮也不会自动滚动页面，读者正在看的那一行会留在原地。 - 会话选择器里的 `messageCount` / `turnCount` 是从文件的有界前缀估算的；精确值以选中会话后展示的为准。 - 会话选择器里，被运行时从另一个会话派生出来的会话标为「子会话」，但不写父会话的名字：分支一定比派生它的会话新，而列表按最近活动取前 30 个，父会话通常已经掉出这个窗口。静默失败的查找，最终会让列表声称一个它看不见的东西。 +- 会话选择器不只说会话**叫什么**，还说它**是什么**：定时任务标「定时」，后台任务标「任务」，没有工具的轻量会话标「轻量」，于是它们不会被误认成你自己开的对话。它们仍然在列表里、仍然可以选 —— 标记是标记，不是过滤。你当前所在的对话排在最前，然后是同一工作区的会话，再然后是其它；工作区路径和 purpose 原文只在服务端比较，不会发到页面。 - 浅色主题的 token 覆盖已验证（页面用到的每个 `--mcode-*` 变量在两套主题下都有定义），但渲染外观未做视觉检查；功能验收是在深色系统偏好下进行的。 ## 测试环境 diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index b54c571b..78d26260 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -3237,6 +3237,23 @@

检视

return flat.slice(0, Math.max(0, limit - 1)) + '…'; } + /* The badge that says what KIND of session a row is. + * + * The service sends a token, never the raw `purpose` it read from the runtime database: + * that value is free text and can carry anything a person typed into it, so what travels + * is a closed set of three words and nothing else. An unknown token is not a badge — a + * token this page does not know about must not become a label it invented. + * + * It is a marker, not a filter: a scheduled run is still selectable, the reader is just + * told what it is instead of being handed something that looks like their own + * conversation. */ + function sessionTagBadge(tag) { + if (tag === 'lightweight') return ' · 轻量'; + if (tag === 'scheduled') return ' · 定时'; + if (tag === 'task') return ' · 任务'; + return ''; + } + function safeText(value) { if (value === null || value === undefined) return ''; if (typeof value === 'string') return value; @@ -4478,9 +4495,12 @@

检视

var when = formatAgo(session ? session.lastActiveAt : null, now); var count = session ? session.messageCount : null; var tail = isCount(count) ? '（' + when + ' · ' + formatInt(count) + ' 条）' : '（' + when + '）'; + // What kind of session this is, when the service said. It is a marker and not a filter: + // the row is still selectable, and a row with no kind of its own gets no marker. + var tag = session ? sessionTagBadge(session.sessionTag) : ''; var branch = session && session.sessionParentId ? BRANCH_MARK : ''; - if (!label) return '未命名会话 ' + tail + branch; - return label + ' ' + tail + branch; + if (!label) return '未命名会话 ' + tail + tag + branch; + return label + ' ' + tail + tag + branch; } function renderSessionSelect() { diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 37b872c5..ba213040 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -994,12 +994,52 @@ export async function resolveCurrentConversation(home) { } } +/** + * The columns the picker reads, widest first. + * + * A runtime database written before one of these columns existed does not have it, and preparing a + * statement over a column that is not there throws before a single row has been read. So the read + * walks down this list until one statement prepares: each rung drops the newest column and keeps + * every older one, which is what lets an old database still yield titles, kinds and parents + * instead of nothing at all. + */ +const SESSION_TITLE_SELECTS = [ + 'select session_id, title, session_kind, parent_session_id, purpose, workspace_dir from local_runtime_sessions where session_id = ?', + 'select session_id, title, session_kind, parent_session_id, purpose from local_runtime_sessions where session_id = ?', + 'select session_id, title, session_kind, parent_session_id from local_runtime_sessions where session_id = ?', + 'select session_id, title, session_kind from local_runtime_sessions where session_id = ?', +]; + /** * Titles for the session picker. Missing database simply means the caller keeps its * filesystem-derived label. + * + * Five facts come back per session, and the last three are what the picker needs to stop + * presenting every session as if it were the reader's own conversation: + * + * `title` what the session is called + * `kind` the shape the runtime gave it + * `parent` the session it was spawned from, when it was spawned from one + * `purpose` what the session is FOR -- a cron run, a background task, a lightweight exchange + * `workspace` the project directory it belongs to + * + * The parent id is the fact that a branch exists, and it is the ONLY part of that + * relationship that is always available: the list is the most recent N sessions, and a child + * is always newer than the parent that spawned it, so the parent's title is routinely outside + * the window. Carrying the id and looking the title up would therefore be a lookup that fails + * exactly when a reader is most likely to want it. + * + * Asking for a column is the optional part, and it has to be optional: a runtime database + * written before child sessions existed has no `parent_session_id`, and preparing a statement + * over a column that is not there throws before a single row has been read. Under the empty + * catch below that is not a degraded list -- it is every title and every kind in the page + * silently reverting to the first sentence of the transcript, which is what a reader would see + * as "the titles broke and nothing says why". The mapping below already answers `null` for a + * row that carries no parent, so the rows the lower rungs return need no change at all. + * * @param {string} home * @param {readonly string[]} sessionIds - * @returns {Promise>} + * @returns {Promise>} */ export async function readSessionTitles(home, sessionIds) { const result = new Map(); @@ -1007,29 +1047,16 @@ export async function readSessionTitles(home, sessionIds) { if (!db) return result; try { if (sessionIds.length === 0) return result; - // The parent id is the fact that a branch exists, and it is the ONLY part of that - // relationship that is always available: the list is the most recent N sessions, and a child - // is always newer than the parent that spawned it, so the parent's title is routinely outside - // the window. Carrying the id and looking the title up would therefore be a lookup that fails - // exactly when a reader is most likely to want it. - // - // Asking for the column is the optional part, and it has to be optional: a runtime database - // written before child sessions existed has no `parent_session_id`, and preparing a statement - // over a column that is not there throws before a single row has been read. Under the empty - // catch below that is not a degraded list -- it is every title and every kind in the page - // silently reverting to the first sentence of the transcript, which is what a reader would see - // as "the titles broke and nothing says why". The mapping below already answers `null` for a - // row that carries no parent, so the row this fallback returns needs no change at all. - let stmt; - try { - stmt = db.prepare('select session_id, title, session_kind, parent_session_id from local_runtime_sessions where session_id = ?'); - } catch { + let stmt = null; + for (const sql of SESSION_TITLE_SELECTS) { try { - stmt = db.prepare('select session_id, title, session_kind from local_runtime_sessions where session_id = ?'); + stmt = db.prepare(sql); + break; } catch { - return result; + // This database does not have that column yet; ask for less. } } + if (!stmt) return result; for (const id of sessionIds) { const row = stmt.get(id); if (!row) continue; @@ -1039,6 +1066,8 @@ export async function readSessionTitles(home, sessionIds) { parent: typeof row.parent_session_id === 'string' && row.parent_session_id !== '' ? row.parent_session_id : null, + purpose: typeof row.purpose === 'string' && row.purpose.trim() ? row.purpose.trim() : null, + workspace: typeof row.workspace_dir === 'string' && row.workspace_dir.trim() ? row.workspace_dir.trim() : null, }); } } catch { @@ -1049,6 +1078,166 @@ export async function readSessionTitles(home, sessionIds) { return result; } +/** + * What kind of session this is, as a closed set of tokens the page can render. + * + * This is the only place that knows what a runtime `purpose` means. The raw value is free text -- + * it carries whatever a person wrote into it -- so it is never published; what travels is the token + * derived from it. + * + * `purpose` answers first because it is the more specific fact: a session can be a cron run and + * still be recorded with the kind of a conversation. `kind` is the fallback, for a database whose + * `purpose` column does not exist yet. + * + * @param {string | null | undefined} kind + * @param {string | null | undefined} purpose + * @returns {'lightweight' | 'scheduled' | 'task' | null} + */ +export function sessionTagFor(kind, purpose) { + const p = typeof purpose === 'string' ? purpose.trim() : ''; + const k = typeof kind === 'string' ? kind.trim().toLowerCase() : ''; + if (p === 'context-mode:lightweight') return 'lightweight'; + if (p.startsWith('cron:') || k === 'cron') return 'scheduled'; + if (p.startsWith('local-background-task:') || p.startsWith('local-task:') || k === 'task') return 'task'; + return null; +} + +/** + * @param {string | null | undefined} value + * @returns {{ value: string, windows: boolean } | null} + */ +function normaliseWorkspacePath(value) { + if (typeof value !== 'string') return null; + const trimmed = value.trim(); + if (!trimmed) return null; + const slashed = trimmed.replace(/\\/g, '/'); + const withoutTail = slashed.length > 1 ? slashed.replace(/\/+$/, '') : slashed; + // A drive letter or a UNC root is what makes a path a Windows path, wherever the comparison runs. + const windows = /^[A-Za-z]:\//.test(withoutTail) || withoutTail.startsWith('//'); + return { value: withoutTail, windows }; +} + +/** + * Whether two workspace paths name the same directory. + * + * The value is an absolute path on the reader's machine, so it is compared here and never sent + * anywhere: the page receives an ordered list, and a path in it would be a location on somebody's + * disk. Two spellings of one path have to compare equal, because the same workspace is written + * with backslashes in one row and with forward slashes in another, and a Windows path does not + * distinguish `D:\work` from `D:\WORK`. + * + * Case is folded only for Windows-shaped paths. A POSIX path is compared as it stands, because + * `/work/Project` and `/work/project` really are two directories there. + * + * @param {string | null | undefined} left + * @param {string | null | undefined} right + * @returns {boolean} + */ +export function sameWorkspacePath(left, right) { + const a = normaliseWorkspacePath(left); + const b = normaliseWorkspacePath(right); + if (!a || !b) return false; + if (a.windows || b.windows) return a.value.toLowerCase() === b.value.toLowerCase(); + return a.value === b.value; +} + +/** + * The workspace a session belongs to, as the picker reads it: null when the database has no such + * column, no such row, or no value in it. + * + * @param {Map} titles + * @param {string | null | undefined} sessionId + * @returns {string | null} + */ +function workspaceOfSession(titles, sessionId) { + if (!sessionId || !(titles instanceof Map)) return null; + const meta = titles.get(sessionId); + if (!meta || typeof meta.workspace !== 'string' || !meta.workspace.trim()) return null; + return meta.workspace.trim(); +} + +/** + * Build a session entry from a recorded relative directory. + * + * The value is a column the runtime wrote rather than a name this app chose, so it earns the same + * containment check `resolveArtifactPath` gives catalog file names: a path that resolves outside + * the sessions root is refused, not clamped. The picker and the 'latest' resolver share this one + * function, so a second copy of the check cannot drift away from the first. + * + * @param {string} sessionsRoot + * @param {string | null | undefined} relativeDir + * @returns {Promise<{ entry: { dir: string, sizeBytes: number, lastActiveAt: number } | null, rejected: boolean }>} + */ +export async function entryFromRelativeDir(sessionsRoot, relativeDir) { + if (typeof relativeDir !== 'string' || !relativeDir.trim()) return { entry: null, rejected: false }; + const root = resolve(sessionsRoot); + const dir = resolve(root, relativeDir.trim().replace(/\\/g, '/')); + if (dir !== root && !dir.startsWith(root + sep)) return { entry: null, rejected: true }; + try { + const fileStat = await stat(join(dir, 'messages.jsonl')); + return { + entry: { dir, sizeBytes: fileStat.size, lastActiveAt: Math.floor(fileStat.mtimeMs) }, + rejected: false, + }; + } catch { + return { entry: null, rejected: false }; + } +} + +/** + * The visible set the picker shows, and the order it is shown in. + * + * Two sets live here and they are not the same set. `entries` is what the walk found: the most + * recent sessions, mtime-descending, and it is also the answer 'latest' falls back to when the + * runtime database is unavailable -- so nothing in here reorders it. The visible set is what the + * READER may pick from, and it is allowed to differ. + * + * The order is: the conversation the reader is in, then the sessions of that conversation's + * workspace, then everything else -- each group mtime-descending. "The current workspace" means + * the `workspace_dir` the runtime database records for the current conversation, and only that; + * the path itself never leaves this function. + * + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries + * @param {{ sessionId: string, relativeDir?: string | null } | null} current + * @param {Map} titles + * @param {string} sessionsRoot + * @param {number} limit + * @param {() => void} [onReject] + * @returns {Promise>} + */ +export async function buildVisibleSessionList(entries, current, titles, sessionsRoot, limit, onReject) { + if (!current || typeof current.sessionId !== 'string' || !current.sessionId) { + // No runtime database, or no current conversation: the visible set is exactly what the walk + // found, in the walk's order. This is what the picker did before the current conversation was + // pinned into it, and it is still what it does when there is nothing to pin. + return entries.slice(0, limit); + } + const currentId = current.sessionId; + let currentEntry = entries.find((entry) => decodeSessionIdFromDirName(basename(entry.dir)) === currentId) || null; + if (!currentEntry && current.relativeDir) { + // Not in the recent window -- build the entry straight from the recorded path. + const built = await entryFromRelativeDir(sessionsRoot, current.relativeDir); + if (built.rejected && typeof onReject === 'function') onReject(); + currentEntry = built.entry; + } + const currentWorkspace = workspaceOfSession(titles, currentId); + const rest = entries.filter((entry) => entry !== currentEntry); + const same = []; + const other = []; + for (const entry of rest) { + const workspace = workspaceOfSession(titles, decodeSessionIdFromDirName(basename(entry.dir))); + if (currentWorkspace !== null && sameWorkspacePath(workspace, currentWorkspace)) same.push(entry); + else other.push(entry); + } + // Both groups keep the walk's order, which is mtime-descending. Stated rather than inherited: + // the walk's order is an implementation detail of the traversal, and the picker promises one. + const byRecency = (a, b) => b.lastActiveAt - a.lastActiveAt; + same.sort(byRecency); + other.sort(byRecency); + const ordered = currentEntry ? [currentEntry, ...same, ...other] : [...same, ...other]; + return ordered.slice(0, limit); +} + /** * Closing a read-only handle cannot fail in a way the caller can act on — the work it was * opened for is already done or already abandoned. Swallowing keeps every call site from @@ -2427,6 +2616,11 @@ export function sessionsValidator(visible, titles, limit) { const title = meta ? String(meta.title ?? "") : ""; const kind = meta ? String(meta.kind ?? "") : ""; const parent = meta ? String(meta.parent ?? "") : ""; + // The kind badge the page draws is derived from the row's kind and purpose, so it is + // reader-visible on its own and belongs in the material for the same reason `kind` does. A + // runtime database that changes a session's purpose changes what the row says, and a tag that + // did not move would answer 304 for a body the caller never held. + const tag = meta ? String(sessionTagFor(meta.kind, meta.purpose) ?? "") : ""; // Injective by construction, which no delimiter is. Any character can occur inside a SQLite TEXT // column — including the NUL this used to join on — so a title of "a\0b" beside a kind of "c" and // a title of "a" beside a kind of "b\0c" produced byte-identical material: one ETag for two rows @@ -2434,7 +2628,7 @@ export function sessionsValidator(visible, titles, limit) { // different separator cannot fix that, because no character is excluded from a TEXT value. // JSON escapes the ambiguous characters, so the tuple's boundaries survive anything the store // can return, and no field needs a second representation to be safe. - parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent])); + parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag])); } return parts.join("\n"); } @@ -2946,7 +3140,7 @@ export async function start(context) { * per poll for rows that only change when somebody renames a session. Raising the walk's TTL * without this would still reopen a 700 MB store every 2.5 s forever, so the two caches answer * the same poll or neither does. - * @type {{ at: number, key: string, map: Map }} + * @type {{ at: number, key: string, map: Map }} */ let titleCache = { at: 0, key: '', map: new Map() }; /** @@ -3087,9 +3281,13 @@ export async function start(context) { const limit = normalizeLimit(url.searchParams.get('limit'), 20, 1, 50); let entries; try { - // The picker still sees only the most recent sessions. The id index beside it is what - // `handleTrajectory` resolves explicit ids against, and it is deliberately not what the - // picker is built from — widening the list is a separate decision about the UI. + // The picker is built from the walk's own list: the most recent sessions, mtime-descending. + // That list is also what 'latest' falls back to when the runtime database is unavailable, + // and nothing here reorders it. The VISIBLE set below is a different set — it widens this + // one by at most one entry, the current conversation, and the rule for picking twenty of + // the thirty is no longer pure mtime: the current conversation comes first, then the + // sessions of its workspace. The id index beside the list is what `handleTrajectory` + // resolves explicit ids against, and it is deliberately not what the picker is built from. entries = (await listEntries()).recentEntries; } catch (error) { context.logger.error('miniapp.trajectory.read_error', { reason: 'root_unreadable', code: errorCode(error) }); @@ -3100,12 +3298,24 @@ export async function start(context) { sendError(response, 503, 'trajectory_unavailable', '未找到本地会话数据目录'); return; } - const visible = entries.slice(0, limit); - // Real titles from the runtime database; sub-agent and cron sessions get a kind badge so - // the picker never presents them as if they were the user's conversation. - const titles = await sessionTitles( - visible.map((entry) => decodeSessionIdFromDirName(basename(entry.dir))).filter(Boolean), - ); + + // The conversation the reader is IN, asked of the runtime database rather than guessed from + // mtime: the conversation being read is not necessarily the one whose file was written last. + const current = await resolveCurrentConversation(location.root); + + // Real titles from the runtime database, for every candidate in the window and for the current + // conversation, which may be outside it. The workspace column rides along because the visible + // set is ordered by it. + const ids = entries.map((entry) => decodeSessionIdFromDirName(basename(entry.dir))).filter(Boolean); + if (current && current.sessionId && ids.indexOf(current.sessionId) < 0) ids.push(current.sessionId); + const titles = await sessionTitles(ids); + + // The visible set: the current conversation first, then the sessions of its workspace, then + // the rest -- each group mtime-descending. At most one session joins the window this way, and + // it is always the current conversation. The number of visible sessions is unchanged. + const visible = await buildVisibleSessionList(entries, current, titles, location.sessionsRoot, limit, () => { + context.logger.warn('miniapp.trajectory.read_error', { reason: 'session_dir_rejected' }); + }); // Built from the same two values the payload below is built from, and computed before any // descriptor work — which is what lets an unchanged list answer without building it. @@ -3130,6 +3340,10 @@ export async function start(context) { if (meta.title) descriptor.label = meta.title; descriptor.sessionKind = meta.kind; descriptor.sessionParentId = meta.parent; + // What this session IS, as a token the page can render: a lightweight exchange, a + // scheduled run, a background task. The raw purpose stays on this side of the wire -- + // it is free text, and free text is not something to hand a reader's browser. + descriptor.sessionTag = sessionTagFor(meta.kind, meta.purpose); } described.push(descriptor); } catch (error) { @@ -3174,20 +3388,13 @@ export async function start(context) { if (!entry && current.relativeDir) { // Not in the recent window — build the entry straight from the recorded path. That value // is a database column rather than a name this app chose, so it earns the same - // containment check `resolveArtifactPath` gives catalog file names. - const root = resolve(location.sessionsRoot); - const dir = resolve(root, current.relativeDir.replace(/\\/g, '/')); - if (dir !== root && !dir.startsWith(root + sep)) { + // containment check `resolveArtifactPath` gives catalog file names. The picker builds the + // same kind of entry from the same column, so both go through one function. + const built = await entryFromRelativeDir(location.sessionsRoot, current.relativeDir); + if (built.rejected) { context.logger.warn('miniapp.trajectory.read_error', { reason: 'session_dir_rejected' }); - entry = null; - } else { - try { - const fileStat = await stat(join(dir, 'messages.jsonl')); - entry = { dir, sizeBytes: fileStat.size, lastActiveAt: Math.floor(fileStat.mtimeMs) }; - } catch { - entry = null; - } } + entry = built.entry; } if (entry) binding = current; } @@ -4137,6 +4344,9 @@ function stripInternalFields(value) { // decides by whether this field is set, so a missing key and a present key both reading // "not set" is what keeps one row from being a branch and the next from merely lacking it. sessionParentId: value.sessionParentId ?? null, + // The same floor, for the same reason: this is the token the page turns into 「轻量」/「定时」/ + // 「任务」, and a row with no kind of its own has to be able to say so. + sessionTag: value.sessionTag ?? null, }; } From 211a4ddc1292790212ebc738d8b11d17e4b5b8dd Mon Sep 17 00:00:00 2001 From: avatasia Date: Tue, 20 Oct 2026 16:59:39 +0800 Subject: [PATCH 02/95] The picker had no guard that could fail about any of this The picker was covered, where it was covered at all, by regexes over the source: a line that mentions the field, a line that mentions the marker. Those are satisfied by a query that works, and never by a query that fails -- which is the whole case here, because the database this reads is written by another program, on a reader's machine, at a version nobody here controls, and the columns it has are exactly the thing that went wrong. The new guard builds the databases by hand and asks the list what it answers in each. One database with every column. One without the purpose column. One without the workspace column. One where the schema stops at the base three. And no database at all. Then it asks five questions of the answer: is the current conversation in the list, is it first, is the list still the length it was, do the sessions of the current workspace come before the sessions of other workspaces, and does each group keep its own order by when it was last written. It also asks what happens when the recorded path of the current conversation points outside the session store: the answer is that it is refused, and that the conversation is not injected from it. Two of those questions cannot be asked of a source file at all, which is why the guard is a program and not a pattern. "The current conversation is first" is a statement about the order of a list that is assembled from a filesystem walk, a database read and a sort -- no line of the source states it. And "the list is still the length it was" is a statement about a number that lives in two places, the limit the caller asks for and the number of entries that end up in the answer. The label assertions run the shipped function rather than matching its text, because the failure worth catching is not a typo in a marker: it is the marker being applied to every row, or to none, and both of those still contain the right characters. The decisive case is the one where two rows differ only in the field that decides the marker, and the only thing allowed to differ between their labels is the marker. One thing had to be re-anchored rather than added. The mutation control for the review guard names the shipped version literally, so moving the version stopped four of its mutations from resolving -- and a mutation whose anchor no longer matches is worse than no mutation, because it reports green while testing nothing. They now name the new literal and the one after it. Two of the four mutations that cover the new behaviour live in controls that were written for other rules, because that is where the code they rewrite already is: removing a rung of the column ladder, and stopping the validator from covering the kind badge. verify-list-cache.mjs 25 -> 46, verify-sessions-etag.mjs 32 -> 35, client-check.mjs 217 -> 233, verify-session-picker.mjs new with 25 assertions; mutate-list-cache-ttl.py 18 -> 22, mutate-sessions-etag.py 16 -> 17, mutate-session-picker.py new with 10 mutations, all caught. Suite 92 -> 94, package byte-identical across the run. --- .../mmc-trajectory/tests/client-check.mjs | 62 +++ .../tests/mutate-list-cache-ttl.py | 38 +- .../mmc-trajectory/tests/mutate-review16.py | 14 +- .../tests/mutate-session-picker.py | 193 +++++++++ .../tests/mutate-sessions-etag.py | 21 +- .../tests/verify-list-cache.mjs | 152 +++++++- .../tests/verify-session-picker.mjs | 365 ++++++++++++++++++ .../tests/verify-sessions-etag.mjs | 21 + 8 files changed, 826 insertions(+), 40 deletions(-) create mode 100644 plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py create mode 100644 plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index ec4c3ccf..cfcbd36a 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -533,6 +533,10 @@ const codeOnly = html // and was never read, and the one field that did reach the page (`sessionKind`) had no reader in // it at all. A reader comparing two entries could not tell a sub-agent run from a conversation. // +// The observation still stands: `sessionKind` is sent and still has no reader. What the reader now +// gets instead is `sessionTag` — the token the service derives from the row's purpose and kind — +// and it is the one the page draws. The raw kind column is still on the wire and still unread. +// // The marker is checked here by RUNNING it rather than by matching its text, because the failure // worth catching is not a typo in the string — it is the marker being applied to everything, or to // nothing, and both of those still contain the right characters. The decisive case is the third @@ -570,6 +574,64 @@ if (!branchMarkSrc || !branchLabelSrc) { sessionParentId: 'mvs_parent' }, NOW).indexOf('子会话') >= 0); } +/* ------------------------------------------------------------------ session picker: kind badge */ +// The other half of the same defect: a cron run, a background task and a lightweight exchange were +// listed exactly like the reader's own conversations, because the service read the runtime +// database's `purpose` column not at all — so it had no way to tell them apart, and a row that +// looked like a conversation was presented as one. +// +// The badge is driven by a token the service derived, never by the raw purpose: that value is free +// text and can carry anything a person typed into it, so publishing it would hand the reader's +// browser somebody else's words. The token set is closed, and a token this page does not know +// about produces no badge rather than one this page invented. +// +// Same discipline as the branch marker above, and for the same reason: the failure worth catching +// is not a typo in the string, it is the badge being applied to everything or to nothing, and both +// of those still contain the right characters. +const tagBadge = sandbox.sessionTagBadge; +assert('the badge helper is in the pure block, so a label can be built without a DOM', + typeof tagBadge === 'function'); +if (typeof tagBadge === 'function') { + check('a lightweight session is badged 轻量', tagBadge('lightweight'), ' · 轻量'); + check('a scheduled session is badged 定时', tagBadge('scheduled'), ' · 定时'); + check('a background task is badged 任务', tagBadge('task'), ' · 任务'); + check('no kind at all is no badge', tagBadge(null), ''); + check('an empty token is no badge', tagBadge(''), ''); + check('a token this page does not know is no badge', tagBadge('something-else'), ''); + + const NOW = 1_800_000_000_000; + const base = { label: '一个会话', lastActiveAt: NOW - 60_000, messageCount: 42 }; + const plain = sandbox.sessionOptionLabel({ ...base }, NOW); + const light = sandbox.sessionOptionLabel({ ...base, sessionTag: 'lightweight' }, NOW); + + assert('a lightweight session is badged in the picker', light.indexOf('轻量') >= 0); + assert('a scheduled session is badged in the picker', + sandbox.sessionOptionLabel({ ...base, sessionTag: 'scheduled' }, NOW).indexOf('定时') >= 0); + assert('a background task is badged in the picker', + sandbox.sessionOptionLabel({ ...base, sessionTag: 'task' }, NOW).indexOf('任务') >= 0); + // The three "no badge" cases, which are three different ways of saying the same thing. + assert('a session with no kind of its own carries no badge', + sandbox.sessionOptionLabel({ ...base, sessionTag: null }, NOW) === plain); + assert('an empty token carries no badge', + sandbox.sessionOptionLabel({ ...base, sessionTag: '' }, NOW) === plain); + assert('a row with no kind field at all carries no badge', + sandbox.sessionOptionLabel({ ...base }, NOW) === plain); + // Identity, not shape: the only thing that differs between these two inputs is the field, so + // the only thing allowed to differ between the two labels is the badge. + assert('the badge is driven by the field, not by where the row sits in the list', + light.replace(' · 轻量', '') === plain); + assert('and nothing else about the label changes with it', + light.length === plain.length + ' · 轻量'.length); + // Both markers at once: the kind says what the session IS, the branch says where it came from, + // and the order between them is fixed — the kind first, because it is the fact about the row + // itself and the branch is a fact about its history. + const both = sandbox.sessionOptionLabel({ ...base, sessionTag: 'lightweight', sessionParentId: 'mvs_parent' }, NOW); + assert('a row with both a kind and a parent shows the kind before the branch mark', + both.indexOf(' · 轻量') >= 0 + && both.indexOf(' · 子会话') >= 0 + && both.indexOf(' · 轻量') < both.indexOf(' · 子会话')); +} + const checks = [ ['no remote http(s) resource', !/<(script|link|img|iframe)[^>]+(src|href)=["']https?:/i.test(html)], ['no @import', !/@import/.test(html)], diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py index 26b63321..daa7fd30 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py @@ -153,19 +153,34 @@ # above still passes and the page is simply broken on every database written before child # sessions existed. The assertion that catches it is the one that builds such a database here. ("a database without the parent column loses every title and kind, not just the parent", - [(""" let stmt; - try { - stmt = db.prepare('select session_id, title, session_kind, parent_session_id from local_runtime_sessions where session_id = ?'); - } catch { + [(""" let stmt = null; + for (const sql of SESSION_TITLE_SELECTS) { try { - stmt = db.prepare('select session_id, title, session_kind from local_runtime_sessions where session_id = ?'); + stmt = db.prepare(sql); + break; } catch { - return result; + // This database does not have that column yet; ask for less. } } -""", " const stmt = db.prepare('select session_id, title, session_kind, parent_session_id from local_runtime_sessions where session_id = ?');")], + if (!stmt) return result; +""", " const stmt = db.prepare(SESSION_TITLE_SELECTS[0]);")], "a database WITHOUT the column keeps the title and the kind, and only loses the parent"), + # ---- the ladder: one rung at a time ---- + # + # The read asks for the columns a current runtime writes and falls back, one column at a time, + # to the ones an older database has. A rung that is missing is not a rung that asks for less: + # the database above it lands on the empty catch instead, and the page shows no titles and no + # kinds at all. Both of these remove one rung, and each is caught by the assertion about the + # database shape that rung was there for. + ("the rung that reads the parent without the purpose is dropped", + [(" 'select session_id, title, session_kind, parent_session_id from local_runtime_sessions where session_id = ?',\n", "")], + "a database WITHOUT the purpose column still reports the title, the kind and the parent"), + + ("the rung that reads the purpose without the workspace is dropped", + [(" 'select session_id, title, session_kind, parent_session_id, purpose from local_runtime_sessions where session_id = ?',\n", "")], + "a database WITHOUT the workspace column still reports the title, the kind, the parent and the purpose"), + ("the parent row is read but never mapped", [(" parent: typeof row.parent_session_id === 'string' && row.parent_session_id !== ''\n" " ? row.parent_session_id\n" @@ -189,6 +204,15 @@ " var branch = BRANCH_MARK;")], "the page decides the marker by the field and not unconditionally", CLIENT), + + # The same pair, for the kind badge: applied to every row, and applied to none. The first one + # still contains 轻量 on every line, so every check that asks "does the row say 轻量" passes — + # and every ordinary conversation now claims to be a lightweight exchange. + ("the kind badge is applied to every session", + [(" var tag = session ? sessionTagBadge(session.sessionTag) : '';", + " var tag = sessionTagBadge('lightweight');")], + "the page decides the kind badge by the field and not unconditionally", + CLIENT), ] ORIGINALS = { diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py b/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py index b1070ff2..a6adfd0e 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py @@ -209,14 +209,14 @@ "the non-anomaly hint is still the one that names types and search"), # ---- 7. the version goes back ---- - # Re-anchored at the 1.2.0 bump: the three below were written against the literal `1.1.0` in the + # Re-anchored at the 1.3.0 bump: the three below were written against the literal `1.2.0` in the # manifest and in both READMEs, so leaving them would have made four mutations stop resolving — # and one of them, the last, would have become an edit that changes nothing, which the runner # rejects. The properties are the reviewer's complaint and are unchanged; only the text they are - # written against moved. The numbers are now `1.2.0` and `1.3.0` for the same reason the old pair - # was `1.1.0` and `1.2.0`: the replacement has to be a version the tree does not already carry. + # written against moved. The numbers are now `1.3.0` and `1.4.0` for the same reason the old pair + # was `1.2.0` and `1.3.0`: the replacement has to be a version the tree does not already carry. ("the version is put back to 1.0.0", - '"version": "1.2.0",', + '"version": "1.3.0",', '"version": "1.0.0",', "the plugin version was bumped off 1.0.0"), @@ -225,12 +225,12 @@ # announcing 1.0.0 to every reader who opens the plugin's front page. A rule that checks the # field nobody complained about is not the rule that was asked for. ("the English README goes back to announcing the old version", - 'Version: `1.2.0`', + 'Version: `1.3.0`', 'Version: `1.0.0`', "the English README carries the shipped version"), ("the Chinese README goes back to announcing the old version", - '版本：`1.2.0`', + '版本：`1.3.0`', '版本：`1.0.0`', "the Chinese README carries the shipped version"), @@ -238,8 +238,8 @@ # on and the READMEs stay behind. Asserted as equality with plugin.json rather than as "not # 1.0.0", so this is the case the rule is actually for. ("the manifest moves on and the READMEs do not follow", - '"version": "1.2.0",', '"version": "1.3.0",', + '"version": "1.4.0",', "the English README carries the shipped version"), # ---- the snapshot the last run left, and the run that never looked at it ---- diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py new file mode 100644 index 00000000..b10982ae --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py @@ -0,0 +1,193 @@ +"""Break the session picker's visible set and its kind badge in the shipped source and require the +guards that own them to report each way it comes back by name. + +The picker had one number behind it: the mtime of `messages.jsonl`. That number answers "when did +this session last write a message", and the picker needs to answer "is this session still in use" -- +which is a different question, and the gap between them is invisible: a conversation the reader +opened and then read without writing to keeps its old mtime and ages out of the window while the +page is displaying it. The window's length was therefore decided by the reader's behaviour. + +So the visible set now has three rules -- the current conversation is in it and first, the sessions +of its workspace come before the sessions of other workspaces, and each group keeps the recency +order -- and the mutations below are the ways back to the one number. Each one is a single edit, +and each is the way the defect would come back: a rule dropped, a rule moved, a rule inverted. + +Two of them are the interesting ones. The first is the injection: the current conversation is +built from the path the runtime recorded for it, under the same containment check the 'latest' +resolver gives it, and a version that skips the containment check builds an entry from a path +outside the session store. The second is the read of the workspace column: a runtime database +written before that column existed has none, and a version that reads it without asking whether +it is there takes the whole list down with it. + +The last two are the badge, and they are the same pair the branch marker had: applied to every +row, and applied to none. Both still contain the right characters, which is why the guards check +the rendered label rather than the string. + +Two probes, because the rules live on two sides of the wire: the visible set is the service's, +and the badge is the page's. A mutation has to be caught by one of them, by name. + +Anchors are multi-line and both shipped files are LF, so they are written with bare \\n and read +and written with newline='' -- matching on normalised text while preserving the file's own endings. +No anchor may contain a line ending of its own: one did, and it stopped resolving the moment the +.gitattributes pinning LF landed. +""" +import io +import os +import subprocess +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +PLUGIN = os.path.dirname(HERE) +SERVER = os.path.join(PLUGIN, "miniapp", "node", "server.mjs") +CLIENT = os.path.join(PLUGIN, "miniapp", "client", "index.html") +PROBES = [ + os.path.join(HERE, "verify-session-picker.mjs"), + os.path.join(HERE, "client-check.mjs"), +] + +# Each entry is (title, edits, expect) and optionally a fourth element naming the file to mutate, +# defaulting to the server. +MUTATIONS = [ + # The defect itself: the conversation the reader is in is not in the list at all. The visible + # set falls back to the window, and the window is the thing that was wrong. + ("the current conversation is not injected into the visible list", + [(" currentEntry = built.entry;\n", " currentEntry = null;\n")], + "the current conversation is in the visible list even though its file is the oldest"), + + # Half of the rule: the conversation is in the list, but at the end of it. A reader who opens + # the picker still finds it, twenty rows down, and the list no longer answers "which one am I + # in" at a glance. + ("the current conversation is appended at the end instead of the first", + [(" const ordered = currentEntry ? [currentEntry, ...same, ...other] : [...same, ...other];", + " const ordered = currentEntry ? [...same, ...other, currentEntry] : [...same, ...other];")], + "and the current conversation is first"), + + # The partition, dropped. Every session comes back in one group, which is the plain recency + # order -- the order the picker had before the current conversation was pinned. + ("the workspace partition is dropped", + [(" if (currentWorkspace !== null && sameWorkspacePath(workspace, currentWorkspace)) same.push(entry);", + " if (currentWorkspace !== null && sameWorkspacePath(workspace, currentWorkspace)) other.push(entry);")], + "the sessions of the current workspace come before the sessions of other workspaces"), + + # The partition kept, the order inside it inverted. The list still looks grouped, and each + # group now answers "least recent first", which is a question nobody asked. + ("the groups are ordered the other way round", + [(" const byRecency = (a, b) => b.lastActiveAt - a.lastActiveAt;", + " const byRecency = (a, b) => a.lastActiveAt - b.lastActiveAt;")], + "each group keeps the recency order"), + + # The no-database fallback. Without a runtime database there is no current conversation and + # no current workspace, and the list is what the walk found -- in the walk's order. + ("the list is reordered when there is no runtime database", + [(" return entries.slice(0, limit);\n", " return entries.slice().reverse().slice(0, limit);\n")], + "without a runtime database the list is the plain recency order"), + + # The purpose column, read as if every runtime database had it. A database written before + # that column existed does not have it, and this takes the whole read down with it -- which + # is a list with no titles and no kinds at all, not a list with fewer of them. + ("the purpose is read as if every database had the column", + [(" purpose: typeof row.purpose === 'string' && row.purpose.trim() ? row.purpose.trim() : null,", + " purpose: row.purpose.trim(),")], + "a database without the purpose column still reports the title and the kind of every session"), + + # The workspace column, the same way. It is the newest of the columns and the one that + # orders the list, so a database without it is the case that has to degrade rather than fail. + ("the workspace is read as if every database had the column", + [(" if (!meta || typeof meta.workspace !== 'string' || !meta.workspace.trim()) return null;", + " if (!meta) return null;\n return meta.workspace.trim();")], + "a database without the workspace column still lists every session"), + + # The containment check. The recorded path is a database column rather than a name this app + # chose, so it gets the same check a catalog file name gets -- and a version that skips it + # builds an entry from a directory outside the session store. + ("a recorded path that escapes the sessions root is used anyway", + [(" if (dir !== root && !dir.startsWith(root + sep)) return { entry: null, rejected: true };", + " if (false) return { entry: null, rejected: true };")], + "a recorded path that escapes the sessions root is refused"), + + # ---- the badge, on the page ---- + # + # Applied to every row. The label still contains 轻量, so every check that asks "does the row + # say 轻量" passes -- and every ordinary conversation now claims to be something it is not. + + ("the kind badge is applied to every session", + [(" var tag = session ? sessionTagBadge(session.sessionTag) : '';", + " var tag = sessionTagBadge('lightweight');")], + "the badge is driven by the field, not by where the row sits in the list", + CLIENT), + + # Applied to none. The other direction, and the one that looks like a simplification: the + # badge is a nicety, and a nicety can be dropped. + ("the kind badge is never applied", + [(" var tag = session ? sessionTagBadge(session.sessionTag) : '';", + " var tag = '';")], + "a lightweight session is badged in the picker", + CLIENT), +] + +ORIGINALS = { + SERVER: io.open(SERVER, encoding="utf-8", newline="").read(), + CLIENT: io.open(CLIENT, encoding="utf-8", newline="").read(), +} + + +def run(): + code = 0 + chunks = [] + for probe in PROBES: + proc = subprocess.run(["node", probe], cwd=HERE, + capture_output=True, text=True, encoding="utf-8", errors="replace") + if code == 0: + code = proc.returncode + chunks.append(proc.stdout + proc.stderr) + return code, "\n".join(chunks) + + +ok = True +code, baseline = run() +base_ok = code == 0 and "0 failed" in baseline +print(("ok " if base_ok else "FAIL ") + "baseline green") +ok = ok and base_ok + +caught = [] +for entry in MUTATIONS: + title, edits, expect = entry[0], entry[1], entry[2] + target = entry[3] if len(entry) > 3 else SERVER + original = ORIGINALS[target] + for old, new in edits: + hits = original.count(old) + if hits != 1: + raise SystemExit("FAIL anchor for %r matched %d times" % (title, hits)) + if old == new: + raise SystemExit("FAIL empty mutation for %r: the anchor equals its replacement" % title) + try: + mutated = original + for old, new in edits: + mutated = mutated.replace(old, new, 1) + if mutated == original: + raise SystemExit("FAIL mutation for %r changed nothing" % title) + io.open(target, "w", encoding="utf-8", newline="").write(mutated) + during_code, during = run() + named = expect in during + red = during_code != 0 + # A guard that crashes is not a guard that caught anything: the rule has to be reported by + # name, or the mutation only proved that the file no longer parses. + print(("ok " if (named and red) else "FAIL ") + "guard catches: " + title) + if not named: + print(" no assertion reported %r" % expect) + for line in during.splitlines()[-4:]: + print(" " + line.strip()) + elif not red: + print(" the rule was reported but the guard still exited 0") + caught.append(named and red) + finally: + io.open(target, "w", encoding="utf-8", newline="").write(original) + +restored = all(io.open(path, encoding="utf-8", newline="").read() == text + for path, text in ORIGINALS.items()) +after_code, after = run() +after_ok = after_code == 0 and "0 failed" in after +print(("ok " if restored else "FAIL ") + "both files restored byte for byte") +print(("ok " if after_ok else "FAIL ") + "green again after restore") + +sys.exit(0 if (ok and all(caught) and restored and after_ok) else 1) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py index 741eefb7..f6e04423 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py @@ -53,16 +53,16 @@ # representation with no ambiguity in it survives both, so there are two cases and each owns the # assertion that names it. ("the fields are concatenated with no separator at all", - [(" parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent]));\n", + [(" parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", " parts.push(\"\" + entry.dir + entry.sizeBytes + entry.lastActiveAt" - " + title + kind + parent);")], + " + title + kind + parent + tag);")], "a title/kind split cannot be hidden inside the separator", SERVER), # mtime is one of the three facts the descriptor cache keys on, and the only one that moves # when a session is rewritten in place without growing. ("a newer mtime stops moving the tag", - [(" parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent]));\n", - " parts.push(JSON.stringify([entry.dir, entry.sizeBytes, title, kind, parent]));\n")], + [(" parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", + " parts.push(JSON.stringify([entry.dir, entry.sizeBytes, title, kind, parent, tag]));\n")], "a newer mtime moves the tag", SERVER), # ---- the 304 that is not earned, because it is decided too late ---- @@ -158,11 +158,20 @@ # `String.fromCharCode(0)` rather than an escape in the replacement, because the point of the # mutation is the character, not how it is spelled. ("the fields go back to being joined on a bare NUL", - [(" parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent]));\n", + [(" parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", " parts.push(entry.dir + \"|\" + entry.sizeBytes + \"|\" + entry.lastActiveAt" " + \"|\" + title + String.fromCharCode(0) + kind" - " + String.fromCharCode(0) + parent);")], + " + String.fromCharCode(0) + parent + tag);")], "a separator-like character inside a value cannot merge two readable rows into one tag", SERVER), + + # The kind badge, which is the same shape of rule as the parent: it is on the page, it is + # derived from a column the runtime database holds, and a change in that column changes what + # the row says. Leaving it out of the material makes the tag constant across a change the + # reader can see, and the 304 that follows is a statement about bytes nobody re-computed. + ("the validator stops covering the kind badge the body sends", + [(" const tag = meta ? String(sessionTagFor(meta.kind, meta.purpose) ?? \"\") : \"\";\n", + " const tag = \"\";\n")], + "a changed session purpose moves the tag, even with the title and kind held constant", SERVER), ] ORIGINALS = { diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs index f4ca4803..a59fc933 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs @@ -169,27 +169,75 @@ assert('the session picker omits the count segment instead of printing a placeho // selected, or it is selected and not mapped, or it is mapped and not put on the descriptor, or // it reaches the descriptor and is stripped before it is sent. Each gets its own assertion // because each is a place the value can be lost without a type error or a crash. +// +// The read is a LADDER rather than one statement, because the columns it wants are not all the +// same age: a runtime database written before child sessions existed has no `parent_session_id`, +// and one written before the kind of a session was recorded has neither `purpose` nor +// `workspace_dir`. Each rung has to be there, in this order — a rung that asks for more than the +// one above it can never be reached, and a rung that is missing sends an old database down to the +// empty catch, which is a list with no titles at all rather than a list with fewer titles. +const LADDER = [ + 'select session_id, title, session_kind, parent_session_id, purpose, workspace_dir from local_runtime_sessions where session_id = ?', + 'select session_id, title, session_kind, parent_session_id, purpose from local_runtime_sessions where session_id = ?', + 'select session_id, title, session_kind, parent_session_id from local_runtime_sessions where session_id = ?', + 'select session_id, title, session_kind from local_runtime_sessions where session_id = ?', +]; +const ladderLabels = [ + 'the widest rung reads the purpose and the workspace as well', + 'the next rung drops the workspace column', + 'the next rung drops the purpose column', + 'the last rung is the one every runtime database has', +]; +let ladderAt = -1; +let ladderOrdered = true; +for (let i = 0; i < LADDER.length; i += 1) { + const found = server.indexOf(LADDER[i], ladderAt + 1); + if (found < 0) ladderOrdered = false; + else ladderAt = found; + assert(ladderLabels[i], server.includes(LADDER[i])); +} +assert('the rungs are walked widest first — a rung that asks for more than the one above it can never be reached', + ladderOrdered); + assert('the titles query reads the parent column', /select session_id, title, session_kind, parent_session_id from local_runtime_sessions where session_id = \?/.test(server)); assert('the parent row is mapped, and an empty id is no parent rather than a parent named ""', /parent:\s*typeof row\.parent_session_id === 'string' && row\.parent_session_id !== ''\s*\n\s*\?\s*row\.parent_session_id\s*\n\s*:\s*null,/.test(server)); +// The same two links for the two newer columns. The purpose is what says a session is a cron +// run, a background task or a lightweight exchange, and the workspace is what orders the list — +// neither of them can be dropped without the page going back to presenting every session as the +// reader's own conversation. +assert('the purpose row is mapped, and an empty value is no purpose', + /purpose: typeof row\.purpose === 'string' && row\.purpose\.trim\(\) \? row\.purpose\.trim\(\) : null,/.test(server)); +assert('the workspace row is mapped, and an empty value is no workspace', + /workspace: typeof row\.workspace_dir === 'string' && row\.workspace_dir\.trim\(\) \? row\.workspace_dir\.trim\(\) : null,/.test(server)); + assert('the descriptor carries the parent onto the row the page receives', /descriptor\.sessionParentId = meta\.parent;/.test(server)); +assert('the descriptor carries the kind badge onto the row the page receives', + /descriptor\.sessionTag = sessionTagFor\(meta\.kind, meta\.purpose\);/.test(server)); + // `?? null` rather than a bare copy: the page decides by whether this field is SET, so a missing // key and a present-but-undefined one have to land in the same place as a row that has no parent. assert('the field survives the strip, with null as the floor', /sessionParentId:\s*value\.sessionParentId \?\? null,/.test(server)); +assert('the kind badge survives the strip, with null as the floor', + /sessionTag:\s*value\.sessionTag \?\? null,/.test(server)); + // The reader-facing half, off the comment-stripped client for the reason given in section 6: the // comment above `sessionOptionLabel` names the marker, so an assertion made against the raw file // would be satisfied by prose describing the very thing it exists to require. assert('the page decides the marker by the field and not unconditionally', /var branch = session && session\.sessionParentId \? BRANCH_MARK : '';/.test(clientJs)); -/* ---------- 6. a database without the column, which the four regexes above cannot see ---------- +assert('the page decides the kind badge by the field and not unconditionally', + /var tag = session \? sessionTagBadge\(session\.sessionTag\) : '';/.test(clientJs)); + +/* ---------- 8. a database without the columns, which the regexes above cannot see ---------- * * Everything above reads `server.mjs` as text, and every one of those regexes is satisfied by a * query that works. None of them is satisfied by a query that FAILS, which is the whole case here: @@ -199,8 +247,9 @@ assert('the page decides the marker by the field and not unconditionally', * session fell back to the first sentence of its transcript — and nothing in the suite could see * it, because the shape of a correct query is the shape of a query nobody ran. * - * So this runs the function against a real database built here, with and without the column. The - * without-case is the one that carries the rule: the titles and the kind are the page's whole + * So this runs the function against real databases built here, one per rung of the ladder: with + * every column, without the workspace column, without the purpose column, and with neither. The + * without-cases are the ones that carry the rule — the titles and the kind are the page's whole * reason for opening the database, and a missing optional column is not a reason to lose them. */ const { mkdtempSync, mkdirSync, rmSync } = await import('node:fs'); const { tmpdir } = await import('node:os'); @@ -219,45 +268,108 @@ process.emitWarning = (warning, ...rest) => ( const { DatabaseSync } = await import('node:sqlite'); // `SERVER` is a plain filesystem path, and a dynamic import of one on Windows is read as a URL // with an unsupported scheme. The module has to be asked for by its file: URL. -const { readSessionTitles } = await import(pathToFileURL(SERVER).href); +const { readSessionTitles, sessionTagFor } = await import(pathToFileURL(SERVER).href); -const runtimeDb = (columns) => { +/** + * One runtime database, with the optional columns the caller asks for. + * @param {readonly string[]} extraColumns + */ +const runtimeDb = (extraColumns) => { const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-runtimedb-')); mkdirSync(join(home, 'v2', 'sqlite'), { recursive: true }); const db = new DatabaseSync(join(home, 'v2', 'sqlite', 'runtime-state.sqlite')); - const extra = columns ? ', parent_session_id' : ''; - const placeholders = columns ? '?, ?, ?, ?' : '?, ?, ?'; + const extra = extraColumns.length ? ', ' + extraColumns.join(', ') : ''; + const placeholders = extraColumns.length ? ', ' + extraColumns.map(() => '?').join(', ') : ''; db.exec('create table local_runtime_sessions (session_id text primary key, title text,' + ' session_kind text' + extra + ')'); const values = ['mvs_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', '一条真实的标题', 'conversation']; - if (columns) values.push('mvs_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'); + for (const column of extraColumns) { + if (column === 'parent_session_id') values.push('mvs_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'); + else if (column === 'purpose') values.push('context-mode:lightweight'); + else if (column === 'workspace_dir') values.push('workspace-of-the-session'); + else throw new Error('unknown column ' + column); + } db.prepare('insert into local_runtime_sessions (session_id, title, session_kind' - + extra + ') values (' + placeholders + ')').run(...values); + + extra + ') values (?, ?, ?' + placeholders + ')').run(...values); db.close(); return home; }; const ids = ['mvs_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa']; const homes = []; -let withColumn = null; -let withoutColumn = null; +let parentOnly = null; +let baseOnly = null; +let noWorkspace = null; +let everything = null; try { - homes.push(runtimeDb(true), runtimeDb(false)); - withColumn = await readSessionTitles(homes[0], ids); - withoutColumn = await readSessionTitles(homes[1], ids); + homes.push( + runtimeDb(['parent_session_id']), + runtimeDb([]), + runtimeDb(['parent_session_id', 'purpose']), + runtimeDb(['parent_session_id', 'purpose', 'workspace_dir']) + ); + parentOnly = await readSessionTitles(homes[0], ids); + baseOnly = await readSessionTitles(homes[1], ids); + noWorkspace = await readSessionTitles(homes[2], ids); + everything = await readSessionTitles(homes[3], ids); } finally { // The homes are `mkdtemp` directories this call created, so they are cleaned up in the same // breath rather than left behind for the OS to find. for (const h of homes) rmSync(h, { recursive: true, force: true }); } assert('a database with the parent column still reports the title, the kind and the parent', - withColumn.get(ids[0])?.title === '一条真实的标题' - && withColumn.get(ids[0])?.kind === 'conversation' - && withColumn.get(ids[0])?.parent === 'mvs_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'); + parentOnly.get(ids[0])?.title === '一条真实的标题' + && parentOnly.get(ids[0])?.kind === 'conversation' + && parentOnly.get(ids[0])?.parent === 'mvs_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'); assert('a database WITHOUT the column keeps the title and the kind, and only loses the parent', - withoutColumn.get(ids[0])?.title === '一条真实的标题' - && withoutColumn.get(ids[0])?.kind === 'conversation' - && withoutColumn.get(ids[0])?.parent === null); + baseOnly.get(ids[0])?.title === '一条真实的标题' + && baseOnly.get(ids[0])?.kind === 'conversation' + && baseOnly.get(ids[0])?.parent === null); +// The same rung, one column further on: a database that knows the parent but not the purpose +// still answers with the parent, because the rung above it asks for columns it does not have. +assert('a database WITHOUT the purpose column still reports the title, the kind and the parent', + parentOnly.get(ids[0])?.title === '一条真实的标题' + && parentOnly.get(ids[0])?.kind === 'conversation' + && parentOnly.get(ids[0])?.parent === 'mvs_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' + && parentOnly.get(ids[0])?.purpose === null + && parentOnly.get(ids[0])?.workspace === null); +// And the rung below it: no workspace column, everything else still readable. +assert('a database WITHOUT the workspace column still reports the title, the kind, the parent and the purpose', + noWorkspace.get(ids[0])?.title === '一条真实的标题' + && noWorkspace.get(ids[0])?.kind === 'conversation' + && noWorkspace.get(ids[0])?.parent === 'mvs_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' + && noWorkspace.get(ids[0])?.purpose === 'context-mode:lightweight' + && noWorkspace.get(ids[0])?.workspace === null); +// The widest rung, which is what a current runtime database answers with. +assert('a database with every column reports the workspace as well', + everything.get(ids[0])?.title === '一条真实的标题' + && everything.get(ids[0])?.kind === 'conversation' + && everything.get(ids[0])?.parent === 'mvs_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' + && everything.get(ids[0])?.purpose === 'context-mode:lightweight' + && everything.get(ids[0])?.workspace === 'workspace-of-the-session'); + +/* ---------- 9. what a purpose means, in one place and nowhere else ---------- */ +// +// The raw purpose is free text — it carries whatever a person typed into it — so it never +// travels to the page. What travels is a token derived from it, and this is the only place that +// knows the derivation. A row whose kind is a conversation and whose purpose says nothing gets no +// token at all, which is what keeps an ordinary conversation from being badged. +assert('a lightweight purpose is a lightweight session', + sessionTagFor('conversation', 'context-mode:lightweight') === 'lightweight'); +assert('a cron purpose is a scheduled session', + sessionTagFor('conversation', 'cron:mavis:something') === 'scheduled'); +assert('a cron kind is a scheduled session even with no purpose at all', + sessionTagFor('cron', null) === 'scheduled'); +assert('a background task purpose is a task', + sessionTagFor('conversation', 'local-background-task:bg_1') === 'task'); +assert('a local task purpose is a task', + sessionTagFor('conversation', 'local-task:1') === 'task'); +assert('a task kind is a task even with no purpose at all', + sessionTagFor('task', null) === 'task'); +assert('a plain conversation is not tagged at all', + sessionTagFor('conversation', null) === null); +assert('a row with neither a kind nor a purpose is not tagged at all', + sessionTagFor(null, null) === null); console.log('[list-cache] ' + pass + ' passed, ' + failures.length + ' failed'); if (failures.length) { diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs new file mode 100644 index 00000000..fa8f34e7 --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs @@ -0,0 +1,365 @@ +/** + * The picker shows the conversation the reader is in. + * + * The list was built from one number: the mtime of `messages.jsonl`. That number answers "when + * did this session last write a message", and the picker needs to answer "is this session still + * in use". On the path a reader actually takes — open a conversation, read its history, say + * nothing for the rest of the day — nothing is written, so the file keeps its old mtime and the + * conversation ages out of the window while the page is displaying it. The window's length was + * decided by the reader's behaviour, which is not something a picker can do. + * + * So the visible set is no longer the window and nothing else. Three rules, each with a reason: + * + * 1. The current conversation is always in the list, and first. It is asked of the runtime + * database rather than guessed from mtime, because the conversation being read is not + * necessarily the one whose file was written last. When it is outside the window it is + * built from the path the runtime recorded for it, under the same containment check the + * 'latest' resolver gives it. + * 2. The sessions of the current conversation's workspace come before the sessions of other + * workspaces. The workspace is the one the runtime database records for the current + * conversation — and it is compared here, on the server, because it is an absolute path on + * the reader's machine and has no business in a browser. + * 3. Each group keeps the walk's recency order. + * + * Two sets, and they are not the same set: the walk's own list is untouched, because it is what + * 'latest' falls back to when the runtime database is unavailable, and reordering it would + * change what 'latest' means. Only the visible set is reordered. + * + * The eight invariants below are checked against real stores and real runtime databases built + * here, not against a copy of the logic: the interesting cases are the ones a regex cannot see — + * a database whose columns are older than the code, a recorded path that escapes the store, a + * conversation whose file is the oldest of all. + * + * `tests/mutate-session-picker.py` breaks each of them in the shipped source and requires this + * guard to report it by name. + */ +import { mkdirSync, mkdtempSync, readFileSync, rmSync } from 'node:fs'; +import { mkdir, utimes, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { basename, join } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +const SERVER = fileURLToPath(new URL('../miniapp/node/server.mjs', import.meta.url)); + +// `node:sqlite` is still flagged experimental, and the flag goes to stderr the moment the module +// is first loaded. The runner reads `stdout + stderr` and prints the LAST non-empty line, so the +// warning lands where this guard's own result line should be and the guard gets reported with +// the text of an unrelated warning instead of its verdict. Narrowly filtered, and installed +// BEFORE the import that provokes it: anything else still prints. +const emitWarning = process.emitWarning; +process.emitWarning = (warning, ...rest) => ( + String(warning).startsWith('SQLite is an experimental feature') + ? undefined + : emitWarning.call(process, warning, ...rest)); + +const { DatabaseSync } = await import('node:sqlite'); +// `SERVER` is a plain filesystem path, and a dynamic import of one on Windows is read as a URL +// with an unsupported scheme. The module has to be asked for by its file: URL. +const { + buildVisibleSessionList, + collectSessionCandidates, + decodeSessionIdFromDirName, + readSessionTitles, + resolveCurrentConversation, + sameWorkspacePath, + sessionTagFor, +} = await import(pathToFileURL(SERVER).href); + +let pass = 0; +const failures = []; +function assert(name, cond, detail) { + if (cond) { pass += 1; return; } + failures.push(detail ? name + ' -- ' + detail : name); +} + +/* ---------- the fixtures: a store and a runtime database, built here ---------- */ + +/** The session id the directory name encodes. */ +const ID = (n) => 'mvs_picker' + String(n).padStart(3, '0') + 'zzzz'; + +/** A minute apart, so the recency order is the creation order and nothing is ambiguous. */ +const BASE_TIME = Date.UTC(2026, 9, 20, 0, 0, 0); + +/** + * One session directory under the store, with a controlled mtime. + * @param {string} sessionsRoot + * @param {number} index + * @param {string} id + * @param {number} mtimeMs + */ +async function makeSession(sessionsRoot, index, id, mtimeMs) { + const name = String(index).padStart(2, '0') + '-00-00-session_' + + Buffer.from(id, 'utf8').toString('base64url'); + const dir = join(sessionsRoot, '2026', '10', '20', name); + await mkdir(dir, { recursive: true }); + await writeFile(join(dir, 'messages.jsonl'), + JSON.stringify({ role: 'user', content: 'x' }) + '\n', 'utf8'); + const stamp = new Date(mtimeMs); + await utimes(join(dir, 'messages.jsonl'), stamp, stamp); + return { id, dir, relative: '2026/10/20/' + name }; +} + +/** + * A runtime database, with the optional columns the caller asks for. A database that predates a + * column simply does not have it, which is the case the read ladder exists for. + * + * @param {string} home + * @param {readonly string[]} optionalColumns + * @param {readonly { sessionId: string, kind?: string, purpose?: string | null, workspace?: string | null, relative?: string | null }[]} rows + */ +function runtimeDb(home, optionalColumns, rows) { + mkdirSync(join(home, 'v2', 'sqlite'), { recursive: true }); + const db = new DatabaseSync(join(home, 'v2', 'sqlite', 'runtime-state.sqlite')); + const columns = ['session_id', 'title', 'status', 'history_relative_dir', 'session_kind', + 'archived', 'updated_at_ms'].concat(optionalColumns); + db.exec('create table local_runtime_sessions (' + columns.join(', ') + ')'); + // The resolver asks whether a session holds a live lease, so the table has to exist even when + // no row does. + db.exec('create table local_runtime_session_locks (session_id text, expires_at_ms integer)'); + const insert = db.prepare('insert into local_runtime_sessions (' + columns.join(', ') + + ') values (' + columns.map(() => '?').join(', ') + ')'); + for (const row of rows) { + const values = [row.sessionId, '会话 ' + row.sessionId.slice(-4), 'idle', row.relative ?? null, + row.kind ?? 'conversation', 0, 1]; + for (const column of optionalColumns) { + if (column === 'purpose') values.push(row.purpose ?? null); + else if (column === 'workspace_dir') values.push(row.workspace ?? null); + else if (column === 'parent_session_id') values.push(null); + else throw new Error('unknown column ' + column); + } + insert.run(...values); + } + db.close(); +} + +/** + * What the picker would show, for one store and one runtime database: the walk's list, the + * current conversation, the titles, and the visible set the route builds from them. + * + * @param {string} sessionsRoot + * @param {string} home + * @param {number} limit + */ +async function visibleFor(sessionsRoot, home, limit) { + const entries = (await collectSessionCandidates(sessionsRoot)).recentEntries; + const current = await resolveCurrentConversation(home); + const ids = entries.map((entry) => decodeSessionIdFromDirName(basename(entry.dir))).filter(Boolean); + if (current && current.sessionId && ids.indexOf(current.sessionId) < 0) ids.push(current.sessionId); + const titles = await readSessionTitles(home, ids); + const visible = await buildVisibleSessionList(entries, current, titles, sessionsRoot, limit); + return { entries, current, titles, visible }; +} + +const dirsOf = (visible) => visible.map((entry) => entry.dir); +const sameOrder = (visible, expected) => dirsOf(visible).join('|') === expected.map((m) => m.dir).join('|'); + +const homes = []; +const store = async (count, offset = 0) => { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-picker-')); + homes.push(home); + const sessionsRoot = join(home, 'v2', 'sessions'); + const made = []; + for (let i = 0; i < count; i += 1) { + made.push(await makeSession(sessionsRoot, i + offset, ID(i + offset), BASE_TIME - (i + offset) * 60_000)); + } + return { home, sessionsRoot, made }; +}; + +/* ---------- 1 & 2: the current conversation is in the list, and first ---------- */ + +try { + // Thirty-one sessions, so the current one falls outside the walk's window of thirty. It is the + // OLDEST of them: the reader opened it and read it without writing anything since. + const { home, sessionsRoot, made } = await store(31); + runtimeDb(home, ['parent_session_id', 'purpose', 'workspace_dir'], [ + { sessionId: ID(30), relative: made[30].relative, workspace: 'workspace-one' }, + ]); + const one = await visibleFor(sessionsRoot, home, 20); + assert('the current conversation is in the visible list even though its file is the oldest', + one.visible.some((entry) => entry.dir === made[30].dir), + dirsOf(one.visible).join('|')); + assert('and the current conversation is first', + one.visible[0]?.dir === made[30].dir, + dirsOf(one.visible).slice(0, 3).join('|')); + assert('and the list is no longer than it was — the current conversation joins it, it does not grow it', + one.visible.length === 20, String(one.visible.length)); + assert('and the rest of the list is still the most recent sessions', + one.visible[1]?.dir === made[0].dir && one.visible[19]?.dir === made[18].dir, + dirsOf(one.visible).slice(0, 3).join('|')); +} catch (error) { + failures.push('the current conversation is in the visible list even though its file is the oldest -- ' + + String(error && error.message ? error.message : error)); +} + +/* ---------- 3 & 4: the current workspace first, each group in recency order ---------- */ + +try { + const { home, sessionsRoot, made } = await store(5); + // The workspaces alternate, so a partition that did not happen and a partition that happened + // produce two different orders rather than the same one twice. + const plan = ['workspace-one', 'workspace-two', 'workspace-one', 'workspace-two', 'workspace-one']; + runtimeDb(home, ['parent_session_id', 'purpose', 'workspace_dir'], plan.map((workspace, index) => ({ + sessionId: ID(index), + relative: made[index].relative, + workspace, + }))); + const two = await visibleFor(sessionsRoot, home, 20); + assert('the sessions of the current workspace come before the sessions of other workspaces', + two.visible.findIndex((entry) => entry.dir === made[2].dir) + < two.visible.findIndex((entry) => entry.dir === made[1].dir), + dirsOf(two.visible).join('|')); + assert('every session of the current workspace is ahead of every session of another workspace', + two.visible.findIndex((entry) => entry.dir === made[4].dir) + < two.visible.findIndex((entry) => entry.dir === made[3].dir), + dirsOf(two.visible).join('|')); + assert('the order is the current conversation, then its workspace, then the rest', + sameOrder(two.visible, [made[0], made[2], made[4], made[1], made[3]]), + dirsOf(two.visible).join('|')); + assert('each group keeps the recency order', + two.visible.findIndex((entry) => entry.dir === made[2].dir) + < two.visible.findIndex((entry) => entry.dir === made[4].dir) + && two.visible.findIndex((entry) => entry.dir === made[1].dir) + < two.visible.findIndex((entry) => entry.dir === made[3].dir), + dirsOf(two.visible).join('|')); +} catch (error) { + failures.push('the sessions of the current workspace come before the sessions of other workspaces -- ' + + String(error && error.message ? error.message : error)); +} + +/* ---------- 5: no runtime database, no current conversation, no change ---------- */ + +try { + const { home, sessionsRoot, made } = await store(5); + const three = await visibleFor(sessionsRoot, home, 20); + assert('without a runtime database the list is the plain recency order', + sameOrder(three.visible, [made[0], made[1], made[2], made[3], made[4]]), + dirsOf(three.visible).join('|')); + assert('and without a runtime database there is no current conversation to pin', + three.current === null); +} catch (error) { + failures.push('without a runtime database the list is the plain recency order -- ' + + String(error && error.message ? error.message : error)); +} + +/* ---------- 6: a database without the purpose column ---------- */ + +try { + const { home, sessionsRoot, made } = await store(5); + // No `purpose` column at all: the read has to stop one rung short and still answer with the + // title and the kind, and a conversation with no purpose of its own gets no badge. + runtimeDb(home, ['parent_session_id'], made.map((m, index) => ({ sessionId: ID(index), relative: m.relative }))); + const four = await visibleFor(sessionsRoot, home, 20); + assert('a database without the purpose column still lists every session', + four.visible.length === 5 + && made.every((m) => four.visible.some((entry) => entry.dir === m.dir)), + dirsOf(four.visible).join('|')); + assert('a database without the purpose column still reports the title and the kind of every session', + four.visible.every((entry) => { + const meta = four.titles.get(decodeSessionIdFromDirName(basename(entry.dir))); + return Boolean(meta && meta.title && meta.kind); + })); + assert('a database without the purpose column puts no kind badge on any row', + four.visible.every((entry) => { + const meta = four.titles.get(decodeSessionIdFromDirName(basename(entry.dir))); + return sessionTagFor(meta ? meta.kind : null, meta ? meta.purpose : null) === null; + })); +} catch (error) { + failures.push('a database without the purpose column still lists every session -- ' + + String(error && error.message ? error.message : error)); +} + +/* ---------- 7: a database without the workspace column ---------- */ + +try { + const { home, sessionsRoot, made } = await store(5); + // The workspace column is the one that orders the list, and it is the newest of the two. A + // database without it cannot be partitioned, so the list is the plain recency order — and the + // rows keep their titles, their kinds and their purposes. + const plan = ['workspace-one', 'workspace-two', 'workspace-one', 'workspace-two', 'workspace-one']; + runtimeDb(home, ['parent_session_id', 'purpose'], plan.map((_workspace, index) => ({ + sessionId: ID(index), + relative: made[index].relative, + }))); + const five = await visibleFor(sessionsRoot, home, 20); + assert('a database without the workspace column still lists every session', + five.visible.length === 5 + && made.every((m) => five.visible.some((entry) => entry.dir === m.dir)), + dirsOf(five.visible).join('|')); + assert('a database without the workspace column produces no workspace partition', + sameOrder(five.visible, [made[0], made[1], made[2], made[3], made[4]]), + dirsOf(five.visible).join('|')); + assert('a database without the workspace column still reports the title of every session', + five.visible.every((entry) => { + const meta = five.titles.get(decodeSessionIdFromDirName(basename(entry.dir))); + return Boolean(meta && meta.title); + })); +} catch (error) { + failures.push('a database without the workspace column still lists every session -- ' + + String(error && error.message ? error.message : error)); +} + +/* ---------- 8: a recorded path that escapes the store ---------- */ + +try { + const { home, sessionsRoot, made } = await store(31); + // The path the runtime recorded points out of the sessions root. It is a database column + // rather than a name this app chose, so it gets the same containment check a catalog file + // name gets — and the conversation is not injected from it. + const outside = join(home, 'v2', 'outside'); + await mkdir(outside, { recursive: true }); + await writeFile(join(outside, 'messages.jsonl'), + JSON.stringify({ role: 'user', content: 'x' }) + '\n', 'utf8'); + runtimeDb(home, ['parent_session_id', 'purpose', 'workspace_dir'], [ + { sessionId: ID(30), relative: '../outside', workspace: 'workspace-one' }, + ]); + const six = await visibleFor(sessionsRoot, home, 20); + assert('a recorded path that escapes the sessions root is refused', + !six.visible.some((entry) => entry.dir === outside), + dirsOf(six.visible).join('|')); + assert('and the current conversation is not injected from it', + !six.visible.some((entry) => entry.dir === made[30].dir), + dirsOf(six.visible).join('|')); + assert('and the list is still the plain recency order', + sameOrder(six.visible, made.slice(0, 20)), + dirsOf(six.visible).join('|')); +} catch (error) { + failures.push('a recorded path that escapes the sessions root is refused -- ' + + String(error && error.message ? error.message : error)); +} + +/* ---------- 9: what a workspace path is, compared ---------- */ + +assert('two spellings of one Windows path are the same workspace', + sameWorkspacePath('D:\\work\\project', 'd:/work/project/')); +assert('two different workspaces are not the same workspace', + !sameWorkspacePath('D:\\work\\project', 'D:\\work\\other')); +assert('a workspace that is not there is not a workspace', + !sameWorkspacePath(null, 'D:\\work') + && !sameWorkspacePath('', 'D:\\work') + && !sameWorkspacePath('D:\\work', null) + && !sameWorkspacePath('D:\\work', '')); +assert('a POSIX path is compared as it stands, because two spellings there are two directories', + !sameWorkspacePath('/work/Project', '/work/project')); + +/* ---------- 10: what may leave the service ---------- */ +// +// The workspace is an absolute path on the reader's machine, and the purpose is free text that +// can carry anything a person typed into it. Neither may reach the browser: the page gets an +// ordered list and a token, and that is all. The field list the page receives is the whole +// boundary, so it is asserted rather than assumed. +const serverText = readFileSync(SERVER, 'utf8'); +const strip = /function stripInternalFields\(value\) \{[\s\S]*?\n\}/.exec(serverText); +assert('the field list the page receives names neither the workspace nor the raw purpose', + strip !== null && !/workspace/.test(strip[0]) && !/purpose/.test(strip[0]), + strip === null ? 'no strip function found' : strip[0].slice(0, 120)); +assert('the kind badge is the one field that says what kind of session this is', + strip !== null && /sessionTag: value\.sessionTag \?\? null,/.test(strip[0])); + +for (const home of homes) rmSync(home, { recursive: true, force: true }); + +console.log('[session-picker] ' + pass + ' passed, ' + failures.length + ' failed'); +if (failures.length) { + console.log('FAILURES:\n - ' + failures.join('\n - ')); + process.exit(1); +} +console.log('[session-picker] OK'); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs index 154066ad..99723527 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs @@ -109,6 +109,27 @@ const parentEmpty = new Map([[A, { title: '第一段会话', kind: 'conversation assert('an empty parent and a null one agree — they render the same row', sessionsValidator(visible, parentBefore, 20) === sessionsValidator(visible, parentEmpty, 20)); +// The kind badge. It is derived from the row's purpose, and it is on the page, so a runtime +// database that changes a session's purpose changes what the row says — and the tag has to move +// with it. The title AND the kind are held constant here, so this cannot pass on either of them. +const tagBefore = new Map([[A, { title: '第一段会话', kind: 'conversation', purpose: null }]]); +const tagAfter = new Map([[A, { title: '第一段会话', kind: 'conversation', purpose: 'context-mode:lightweight' }]]); +assert('a changed session purpose moves the tag, even with the title and kind held constant', + sessionsValidator(visible, tagBefore, 20) !== sessionsValidator(visible, tagAfter, 20)); + +// And the null case again: a row with no purpose and a row whose purpose is the empty string +// render identically, so they must not read as a change either. +const tagEmpty = new Map([[A, { title: '第一段会话', kind: 'conversation', purpose: '' }]]); +assert('an empty purpose and a null one agree — they render the same row', + sessionsValidator(visible, tagBefore, 20) === sessionsValidator(visible, tagEmpty, 20)); + +// The ORDER. Two rows a reader can tell apart, in two orders, are two payloads — and the picker +// reorders the visible set (the current conversation first, then its workspace, then the rest), +// so an order change is a change the caller has to be told about. +const swapped = [visible[1], visible[0]]; +assert('a changed order of the visible rows moves the tag', + sessionsValidator(visible, noTitles, 20) !== sessionsValidator(swapped, noTitles, 20)); + // A null title and an absent one are the same payload — the picker falls back to 「会话 」 — so // they have to be the same tag, or every poll would look like a change. const nullTitle = new Map([[A, { title: null, kind: 'conversation' }]]); From 5b6f091fbceded985f747bc7b97b27e3774d336d Mon Sep 17 00:00:00 2001 From: avatasia Date: Tue, 20 Oct 2026 17:45:13 +0800 Subject: [PATCH 03/95] The picker claimed more than the code could establish Two places where the wording had drifted away from the code, and four rules that had nothing able to fail. The wording first. The READMEs and a server comment said the conversation the reader is in, and the function that answers that question returns a ranking -- a live lease, then an open turn, then the most recently updated -- not a certainty, and it reports the ambiguity when two conversations hold a lease at once. The picker pins the database's answer, so it now says that, and the wording agrees with itself. The rules without a failure. The kind badge's order against the branch mark had an assertion and no control, so nothing could break it. The control exists now, along with three the change did not name: the recorded path is checked against the session it claims to be, the tag is written on every row rather than only on the rows that have a database entry, and the row the reader pinned by hand is the first session row. The recorded path is the one that could go wrong quietly. A path can sit inside the sessions root and still belong to a different session, and then the picker would list that other session twice while the conversation the page is about was nowhere in the list. The check went into the function both callers already share, so there is one copy of it. The visible set's size was described two ways at once -- as able to grow by one entry, and two lines later as not changing at all. The implementation displaces the last entry, so the count is what it was and the composition changed, and the wording now says that. Each guard section used to report the first assertion in it when it threw, which made a control that crashed the file look like a caught rule. Every section reports itself now, and the controls expect those names. --- plugins/avatasia/mmc-trajectory/README.md | 2 +- .../avatasia/mmc-trajectory/README.zh-CN.md | 2 +- .../mmc-trajectory/miniapp/node/server.mjs | 86 ++++++++--- .../mmc-trajectory/tests/client-check.mjs | 92 ++++++++++++ .../tests/mutate-session-picker.py | 85 +++++++++-- .../tests/verify-list-cache.mjs | 12 +- .../tests/verify-session-picker.mjs | 140 +++++++++++++++--- 7 files changed, 357 insertions(+), 62 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/README.md b/plugins/avatasia/mmc-trajectory/README.md index 5467994a..a62321d4 100644 --- a/plugins/avatasia/mmc-trajectory/README.md +++ b/plugins/avatasia/mmc-trajectory/README.md @@ -132,7 +132,7 @@ The runtime database is opened in read-only mode. If it is missing, unreadable, - The ledger does not use virtual scrolling; every rendered row is a real element. The page therefore opens on the newest 200 records, and the window bar button switches to the whole session. The button names what pressing it will do, not the current state: 「显示全部」 at the default and 「显示最新 200 条」 once everything is showing, while the count beside it reports what is on screen. That full view is opt-in for a reason: on a 9,900-record session it is roughly 57,000 elements rebuilt every 2.5s poll, which was measured to freeze the page rather than be read. 「向上加载更早的 N 条」 widens the window in steps and a poll does not undo it. A sticky window bar under the overview strip offers that choice behind four explicit controls: switch the strip between equal width and a real time axis, load 200 earlier records, jump back to the latest 200, and show everything. "Show everything" builds every record in the session into the DOM at once and gets noticeably slower on large sessions; incoming records do not knock it back down to 200. - `messageCount` and `turnCount` in the session picker are estimates derived from a bounded prefix of the file; exact values come from the selected session. - A session the runtime spawned from another one is marked 「子会话」 in the picker. The parent is deliberately not named: a branch is always newer than the session that spawned it, so its parent has usually aged out of the most-recent-N window this list is built from, and a lookup that fails quietly is how a picker ends up asserting something it cannot see. -- The picker also says what a session **is**, not only what it is called: a scheduled run carries 「定时」, a background task 「任务」, and a tool-less lightweight conversation 「轻量」, so none of them can be mistaken for a conversation you wrote. They stay in the list and stay selectable — a marker is a marker, not a filter. The conversation you are in comes first, then the sessions of its own workspace, then the rest; the workspace path and the raw purpose string are compared on the server and never reach the page. +- The picker also says what a session **is**, not only what it is called: a scheduled run carries 「定时」, a background task 「任务」, and a tool-less lightweight conversation 「轻量」, so none of them can be mistaken for a conversation you wrote. They stay in the list and stay selectable — a marker is a marker, not a filter. The conversation the runtime database reports as current comes first, then the sessions of its own workspace, then the rest; the workspace path and the raw purpose string are compared on the server and never reach the page. - Light theme token coverage is verified — every `--mcode-*` token the page consumes is defined for both themes — but its rendered appearance was not visually checked; the page was exercised under a dark system preference. ## Tested environment diff --git a/plugins/avatasia/mmc-trajectory/README.zh-CN.md b/plugins/avatasia/mmc-trajectory/README.zh-CN.md index 628342cb..4fdab332 100644 --- a/plugins/avatasia/mmc-trajectory/README.zh-CN.md +++ b/plugins/avatasia/mmc-trajectory/README.zh-CN.md @@ -118,7 +118,7 @@ MiniMax Code 不会告诉 Mini App 是哪段对话打开了它的页面：传给 - 台里没有用虚拟滚动，每一行都是真实的 DOM 元素。页面默认只展示最新 200 条，窗口条上的按钮可以切到全量；全量是可选的，因为它很贵 —— 在这个会话上约 9,900 条、约 5.7 万个元素，每 2.5 秒轮询重建一次，实测会直接卡死页面。总览条正下方的吸顶窗口栏提供三个显式控件：「总览轴」「向上加载更早的 200 条」，以及在最新 200 条与全量之间切换的那一个按钮（它写的是按下去会变成什么，默认为「显示全部」，全量时为「显示最新 200 条」；当前窗口由右边的计数条报告，两者回答的是不同问题）。「总览轴」只改总览条的横轴——等宽（每条记录一样宽，完全忽略时间）或真实时间轴（宽度等于该记录的真实耗时）；它不筛选、不排序，台账里的行和页面上的任何计数都不变。「向上加载更早的 200 条」每点一次就多加一页，轮询不会把已经加宽的窗口重置回去；点切换按钮也不会自动滚动页面，读者正在看的那一行会留在原地。 - 会话选择器里的 `messageCount` / `turnCount` 是从文件的有界前缀估算的；精确值以选中会话后展示的为准。 - 会话选择器里，被运行时从另一个会话派生出来的会话标为「子会话」，但不写父会话的名字：分支一定比派生它的会话新，而列表按最近活动取前 30 个，父会话通常已经掉出这个窗口。静默失败的查找，最终会让列表声称一个它看不见的东西。 -- 会话选择器不只说会话**叫什么**，还说它**是什么**：定时任务标「定时」，后台任务标「任务」，没有工具的轻量会话标「轻量」，于是它们不会被误认成你自己开的对话。它们仍然在列表里、仍然可以选 —— 标记是标记，不是过滤。你当前所在的对话排在最前，然后是同一工作区的会话，再然后是其它；工作区路径和 purpose 原文只在服务端比较，不会发到页面。 +- 会话选择器不只说会话**叫什么**，还说它**是什么**：定时任务标「定时」，后台任务标「任务」，没有工具的轻量会话标「轻量」，于是它们不会被误认成你自己开的对话。它们仍然在列表里、仍然可以选 —— 标记是标记，不是过滤。运行时库报告为当前的那段对话排在最前，然后是同一工作区的会话，再然后是其它；工作区路径和 purpose 原文只在服务端比较，不会发到页面。 - 浅色主题的 token 覆盖已验证（页面用到的每个 `--mcode-*` 变量在两套主题下都有定义），但渲染外观未做视觉检查；功能验收是在深色系统偏好下进行的。 ## 测试环境 diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index ba213040..e8a80399 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -72,9 +72,17 @@ const MAX_RAW_BYTES = 65536; * This has to stay at or above the page's poll interval, and that relationship — not this * number — is the invariant tests/verify-list-cache.mjs holds. Below the interval the cache * cannot survive a single poll: every tick expires it, so the list re-walks the session tree - * and reopens the runtime database on every tick. Measured on this machine, that was most of - * the 16 GB this process read in 68 minutes, and the allocation behind it held the process at - * a 1.2-2.5 GB high-water mark for as long as the page stayed open. + * on every tick. Measured on this machine, that walk was most of the 16 GB this process read + * in 68 minutes, and the allocation behind it held the process at a 1.2-2.5 GB high-water mark + * for as long as the page stayed open. + * + * The runtime database is a separate matter and this TTL does not govern it. The titles read + * rides the same interval, but the current conversation is re-read on every poll and is not + * cached at all: it is the one fact here that must not be stale, because a cached "current + * conversation" is the very defect the picker was built to remove. That read costs a few + * milliseconds — measured at 4.00 ms per call against a 2.5 s poll, so 0.16% of the interval — + * which is why it is left uncached rather than paid for with a TTL that could hide a switch + * from one conversation to another. */ const LIST_CACHE_TTL_MS = 3000; const SESSION_CACHE_TTL_MS = 5000; @@ -918,7 +926,7 @@ function toNumber(value) { } /** - * Resolve the conversation the user is actually in. + * Resolve the conversation the runtime database reports as current. * * The Host never tells the page which session opened it, so we ask the runtime database * instead. Ranking, strongest signal first: @@ -926,6 +934,11 @@ function toNumber(value) { * 2. status = 'started' — open and not finished * 3. most recently updated — the fallback for a long-idle conversation * + * This is what the runtime database says, and the app infers from it: a ranking is a best + * answer, not a certainty, and two conversations holding a lease at once are reported as + * ambiguous rather than resolved here. Nothing in this file can see which conversation the + * reader is actually looking at. + * * Only real user conversations qualify. `session_kind = 'conversation'` with a NULL * `purpose` excludes cron runs (`purpose = 'cron:...'`) and background worker/explore * tasks (`purpose = 'local-background-task:bg_...'`), which are exactly the sessions that @@ -1164,17 +1177,33 @@ function workspaceOfSession(titles, sessionId) { * the sessions root is refused, not clamped. The picker and the 'latest' resolver share this one * function, so a second copy of the check cannot drift away from the first. * + * The containment check alone is not enough, and the gap is the same shape as the one it closes: + * a path can sit INSIDE the sessions root and still belong to a different session than the one + * that recorded it. Then the caller holds one session's directory under another session's id — + * the picker would show the current conversation's id twice and drop the conversation itself, and + * the 'latest' resolver would bind one session's id to another session's directory. So the + * session the path is supposed to be is named here, and a path that turns out to be a different + * session's is refused exactly the way an outside path is. + * * @param {string} sessionsRoot * @param {string | null | undefined} relativeDir + * @param {string | null | undefined} [expectedSessionId] * @returns {Promise<{ entry: { dir: string, sizeBytes: number, lastActiveAt: number } | null, rejected: boolean }>} */ -export async function entryFromRelativeDir(sessionsRoot, relativeDir) { +export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSessionId) { if (typeof relativeDir !== 'string' || !relativeDir.trim()) return { entry: null, rejected: false }; const root = resolve(sessionsRoot); const dir = resolve(root, relativeDir.trim().replace(/\\/g, '/')); if (dir !== root && !dir.startsWith(root + sep)) return { entry: null, rejected: true }; try { const fileStat = await stat(join(dir, 'messages.jsonl')); + if (expectedSessionId) { + // The directory name is the only place the session id is written down, so it is also the + // only place the claim "this path is that session" can be checked. + if (decodeSessionIdFromDirName(basename(dir)) !== expectedSessionId) { + return { entry: null, rejected: true }; + } + } return { entry: { dir, sizeBytes: fileStat.size, lastActiveAt: Math.floor(fileStat.mtimeMs) }, rejected: false, @@ -1192,10 +1221,16 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir) { * runtime database is unavailable -- so nothing in here reorders it. The visible set is what the * READER may pick from, and it is allowed to differ. * - * The order is: the conversation the reader is in, then the sessions of that conversation's - * workspace, then everything else -- each group mtime-descending. "The current workspace" means - * the `workspace_dir` the runtime database records for the current conversation, and only that; - * the path itself never leaves this function. + * The order is: the conversation the runtime database reports as current, then the sessions of + * that conversation's workspace, then everything else -- each group mtime-descending. "The + * current workspace" means the `workspace_dir` the runtime database records for that + * conversation, and only that; the path itself never leaves this function. + * + * "The current conversation" is what the runtime database reports, not what the reader is in: + * the runtime knows which session it last updated, and this app cannot see any more than that. + * The database's answer is a ranking (a live lease, then an open turn, then the most recently + * updated), and `resolveCurrentConversation` reports the ambiguity when two conversations hold + * a lease at once rather than picking one silently. * * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries * @param {{ sessionId: string, relativeDir?: string | null } | null} current @@ -1215,8 +1250,10 @@ export async function buildVisibleSessionList(entries, current, titles, sessions const currentId = current.sessionId; let currentEntry = entries.find((entry) => decodeSessionIdFromDirName(basename(entry.dir)) === currentId) || null; if (!currentEntry && current.relativeDir) { - // Not in the recent window -- build the entry straight from the recorded path. - const built = await entryFromRelativeDir(sessionsRoot, current.relativeDir); + // Not in the recent window -- build the entry straight from the recorded path. The session it + // is supposed to be is named, so a path that turns out to be a different session's directory + // is refused instead of being shown under this id. + const built = await entryFromRelativeDir(sessionsRoot, current.relativeDir, currentId); if (built.rejected && typeof onReject === 'function') onReject(); currentEntry = built.entry; } @@ -3299,8 +3336,10 @@ export async function start(context) { return; } - // The conversation the reader is IN, asked of the runtime database rather than guessed from - // mtime: the conversation being read is not necessarily the one whose file was written last. + // The conversation the runtime database reports as current, asked of the database rather than + // guessed from mtime: the conversation being read is not necessarily the one whose file was + // written last. What the database reports is a ranking, not a certainty -- see + // `resolveCurrentConversation`, which says so where it happens. const current = await resolveCurrentConversation(location.root); // Real titles from the runtime database, for every candidate in the window and for the current @@ -3340,11 +3379,18 @@ export async function start(context) { if (meta.title) descriptor.label = meta.title; descriptor.sessionKind = meta.kind; descriptor.sessionParentId = meta.parent; - // What this session IS, as a token the page can render: a lightweight exchange, a - // scheduled run, a background task. The raw purpose stays on this side of the wire -- - // it is free text, and free text is not something to hand a reader's browser. - descriptor.sessionTag = sessionTagFor(meta.kind, meta.purpose); } + // What this session IS, as a token the page can render: a lightweight exchange, a + // scheduled run, a background task. The raw purpose stays on this side of the wire -- + // it is free text, and free text is not something to hand a reader's browser. + // + // Assigned unconditionally, and that is the point. The descriptor comes out of a cache, + // so a session whose row is missing from the runtime database keeps the tag it was + // given last time -- while the validator, which reads the same rows, reports no tag for + // it. A body and an ETag that disagree about the same row is how a badge outlives the + // fact it was derived from, so the field is written on every pass, not only when there + // is something to write. + descriptor.sessionTag = meta ? sessionTagFor(meta.kind, meta.purpose) : null; described.push(descriptor); } catch (error) { context.logger.warn('miniapp.trajectory.read_error', { reason: 'describe_failed', code: errorCode(error) }); @@ -3390,7 +3436,11 @@ export async function start(context) { // is a database column rather than a name this app chose, so it earns the same // containment check `resolveArtifactPath` gives catalog file names. The picker builds the // same kind of entry from the same column, so both go through one function. - const built = await entryFromRelativeDir(location.sessionsRoot, current.relativeDir); + // + // The session it is supposed to be is named too: a path that points at a different + // session's directory is refused, and 'latest' falls back to the walk's answer rather + // than binding this session's id to somebody else's transcript. + const built = await entryFromRelativeDir(location.sessionsRoot, current.relativeDir, current.sessionId); if (built.rejected) { context.logger.warn('miniapp.trajectory.read_error', { reason: 'session_dir_rejected' }); } diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index cfcbd36a..5e442e31 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -1212,6 +1212,98 @@ for (const [name, ok] of checks) { else failures.push('policy: ' + name); } +/* ------------------------------------------------------------------ session picker: where the pinned row goes */ +// The picker can hold a row the server did not send: the session the reader pinned by hand, which +// aged out of the twenty the service returns. The page adds that row itself, and where it puts it +// is a promise about what the reader sees — the service's first row is the conversation the +// runtime database reports as current, and the pinned row is a different thing, so "the current +// conversation is first" does not by itself say where the pinned row lands. +// +// It is checked by RUNNING `renderSessionSelect` against a stub select, not by reading its +// source. A regex over the source would pass for a version that inserted the row in the right +// place for the wrong reason, and would not notice a version that inserted it in the right place +// only when the list happened to be empty. What the reader gets is the order of the options, so +// that is what is read back. +const renderSelectSrc = fnSource('function renderSessionSelect()'); +if (!renderSelectSrc) { + failures.push('session picker: could not lift renderSessionSelect'); +} else { + // A select, as far as this function touches it: the options it holds, the one that is + // selected, and the two ways a row can be added to it. The browser picks the first option + // when the list is built and picks by value afterwards, and `insertBefore` puts a row at a + // position rather than at the end — which is the whole point of the row in question. + const stubSelect = () => { + const options = []; + return { + options, + selectedIndex: -1, + _value: '', + get value() { return this._value; }, + set value(next) { + this._value = next; + this.selectedIndex = options.findIndex((option) => option.value === next); + }, + appendChild(node) { + options.push(node); + if (this.selectedIndex < 0) this.selectedIndex = options.length - 1; + return node; + }, + insertBefore(node, reference) { + const at = reference ? options.indexOf(reference) : -1; + options.splice(at < 0 ? options.length : at, 0, node); + return node; + }, + }; + }; + sandbox.el = (tag, className, text) => { + const node = stubNode(tag, text); + if (className) node.className = className; + node.value = ''; + return node; + }; + sandbox.clear = (node) => { if (node && node.options) node.options.length = 0; }; + if (!sandbox.state) sandbox.state = {}; + sandbox.state.sessions = [ + { id: 'mvs_server_first', label: '服务端第一行', lastActiveAt: 1_800_000_000_000, messageCount: 3 }, + { id: 'mvs_server_second', label: '服务端第二行', lastActiveAt: 1_799_999_000_000, messageCount: 4 }, + ]; + sandbox.state.followLatest = false; + sandbox.state.sessionId = 'mvs_the_pinned_one'; + vm.runInContext(renderSelectSrc, sandbox); + + const select = stubSelect(); + sandbox.dom = { sessionSelect: select }; + sandbox.renderSessionSelect(); + + const texts = select.options.map((option) => option.textContent); + // The row the reader pinned is the one the service did not send, so it is on its own: the + // first row is 「跟随最新」 and the pinned row is the first of the session rows. + assert('the row the reader pinned is the first session row, above the rows the service sent', + texts.length === 4 + && texts[0].indexOf('跟随最新') >= 0 + && texts[1].indexOf('当前会话（不在最近列表）') >= 0 + && select.options[1].value === 'mvs_the_pinned_one', + texts.join(' | ')); + assert('and the service\'s own first row is still the one right after it', + select.options[2] && select.options[2].value === 'mvs_server_first', + texts.join(' | ')); + assert('and the pinned row is the one that ends up selected', + select.value === 'mvs_the_pinned_one' && select.selectedIndex === 1, + String(select.value) + ' @ ' + String(select.selectedIndex)); + + // And the other direction: a pinned session the service DID send needs no row of its own, or + // the reader would see the same conversation twice. + sandbox.state.sessionId = 'mvs_server_second'; + const second = stubSelect(); + sandbox.dom = { sessionSelect: second }; + sandbox.renderSessionSelect(); + assert('a pinned session the service sent is not added a second time', + second.options.length === 3 + && second.options.every((option, index) => option.value !== 'mvs_the_pinned_one') + && second.value === 'mvs_server_second', + second.options.map((option) => option.value).join(' | ')); +} + console.log('[2] pure helpers: ' + pass + ' passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py index b10982ae..cee2b172 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py @@ -7,24 +7,31 @@ opened and then read without writing to keeps its old mtime and ages out of the window while the page is displaying it. The window's length was therefore decided by the reader's behaviour. -So the visible set now has three rules -- the current conversation is in it and first, the sessions -of its workspace come before the sessions of other workspaces, and each group keeps the recency -order -- and the mutations below are the ways back to the one number. Each one is a single edit, -and each is the way the defect would come back: a rule dropped, a rule moved, a rule inverted. +So the visible set now has three rules -- the conversation the runtime database reports as +current is in it and first, the sessions of its workspace come before the sessions of other +workspaces, and each group keeps the recency order -- and the mutations below are the ways back +to the one number. Each one is a single edit, and each is the way the defect would come back: a +rule dropped, a rule moved, a rule inverted. -Two of them are the interesting ones. The first is the injection: the current conversation is +Three of them are the interesting ones. The first is the injection: the current conversation is built from the path the runtime recorded for it, under the same containment check the 'latest' resolver gives it, and a version that skips the containment check builds an entry from a path -outside the session store. The second is the read of the workspace column: a runtime database -written before that column existed has none, and a version that reads it without asking whether -it is there takes the whole list down with it. - -The last two are the badge, and they are the same pair the branch marker had: applied to every -row, and applied to none. Both still contain the right characters, which is why the guards check -the rendered label rather than the string. +outside the session store. The second reads the workspace column: a runtime database written +before that column existed has none, and a version that reads it without asking whether it is +there takes the whole list down with it. The third is the identity of that path: it can sit +inside the sessions root and still belong to a different session, and a version that does not +check would list that other session twice and lose the conversation the page is about. + +Three more are on the page. The badge is applied to every row, the badge is applied to none, +and the badge is placed after the branch mark -- the third of those still contains the right +characters in the right places, which is why the guards check the rendered label rather than +the string. The last one is the row the reader pinned by hand, which the page adds itself: it +belongs above the rows the service sent, and a version that appends it to the end still +selects it, still shows it, and still looks right -- as long as nobody counts the rows. Two probes, because the rules live on two sides of the wire: the visible set is the service's, -and the badge is the page's. A mutation has to be caught by one of them, by name. +and the badge and the pinned row are the page's. A mutation has to be caught by one of them, by +name. Anchors are multi-line and both shipped files are LF, so they are written with bare \\n and read and written with newline='' -- matching on normalised text while preserving the file's own endings. @@ -48,8 +55,9 @@ # Each entry is (title, edits, expect) and optionally a fourth element naming the file to mutate, # defaulting to the server. MUTATIONS = [ - # The defect itself: the conversation the reader is in is not in the list at all. The visible - # set falls back to the window, and the window is the thing that was wrong. + # The defect itself: the conversation the runtime database reports as current is not in the + # list at all. The visible set falls back to the window, and the window is the thing that was + # wrong. ("the current conversation is not injected into the visible list", [(" currentEntry = built.entry;\n", " currentEntry = null;\n")], "the current conversation is in the visible list even though its file is the oldest"), @@ -62,6 +70,15 @@ " const ordered = currentEntry ? [...same, ...other, currentEntry] : [...same, ...other];")], "and the current conversation is first"), + # The identity of the recorded path. Containment is not enough: a path can sit inside the + # sessions root and still belong to a different session, and then the row that gets injected + # is that other session's transcript -- listed twice, while the conversation the page is + # about is nowhere in the list. + ("the injected row is not checked against the session it is supposed to be", + [(" if (decodeSessionIdFromDirName(basename(dir)) !== expectedSessionId) {", + " if (false) {")], + "a recorded path that belongs to another session is refused, so no session is listed twice"), + # The partition, dropped. Every session comes back in one group, which is the plain recency # order -- the order the picker had before the current conversation was pinned. ("the workspace partition is dropped", @@ -92,10 +109,16 @@ # The workspace column, the same way. It is the newest of the columns and the one that # orders the list, so a database without it is the case that has to degrade rather than fail. + # + # This one dies of an exception, and the guard has to say so BY THAT NAME. The rule it + # breaks is "a database that lacks a column still lists every session", and the way it is + # broken is by taking the read down with it -- so the name that has to appear is the one the + # section reports when it could not answer, not the name of an assertion in it. ("the workspace is read as if every database had the column", [(" if (!meta || typeof meta.workspace !== 'string' || !meta.workspace.trim()) return null;", " if (!meta) return null;\n return meta.workspace.trim();")], - "a database without the workspace column still lists every session"), + "a database without the workspace column still lists every session; " + "the section threw before it could answer"), # The containment check. The recorded path is a database column rather than a name this app # chose, so it gets the same check a catalog file name gets -- and a version that skips it @@ -105,6 +128,15 @@ " if (false) return { entry: null, rejected: true };")], "a recorded path that escapes the sessions root is refused"), + # The badge, written only on the rows that have one. The descriptor comes out of a cache, + # so a row that loses its database entry keeps the tag it was given last time, while the + # validator -- which reads the same rows -- reports none. The body and the ETag then + # disagree about the same row, and only the body reaches the page. + ("the kind tag is only written on the rows that have one", + [(" descriptor.sessionTag = meta ? sessionTagFor(meta.kind, meta.purpose) : null;", + " if (meta) descriptor.sessionTag = sessionTagFor(meta.kind, meta.purpose);")], + "the kind tag is written on every row, not only on the rows that have one"), + # ---- the badge, on the page ---- # # Applied to every row. The label still contains 轻量, so every check that asks "does the row @@ -123,6 +155,27 @@ " var tag = '';")], "a lightweight session is badged in the picker", CLIENT), + + # Placed after the branch mark. Both markers are still there, both are still the right + # characters, and the row still says everything it said -- only the order between them has + # changed. A guard that asks "does the row contain 轻量" cannot see this at all; one that + # asks which of the two comes first can. + ("the kind badge is placed after the branch mark", + [(" if (!label) return '未命名会话 ' + tail + tag + branch;\n" + " return label + ' ' + tail + tag + branch;", + " if (!label) return '未命名会话 ' + tail + branch + tag;\n" + " return label + ' ' + tail + branch + tag;")], + "a row with both a kind and a parent shows the kind before the branch mark", + CLIENT), + + # The row the reader pinned by hand, which the page adds itself because the service did not + # send it. Appended to the end it is still there, still selected, still readable -- and it + # is no longer the first session row, which is the only promise that was made about it. + ("the row the reader pinned is appended to the end of the list", + [(" select.insertBefore(pinned, select.options[1] || null);", + " select.appendChild(pinned);")], + "the row the reader pinned is the first session row, above the rows the service sent", + CLIENT), ] ORIGINALS = { diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs index a59fc933..0dc04cd6 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs @@ -217,8 +217,16 @@ assert('the workspace row is mapped, and an empty value is no workspace', assert('the descriptor carries the parent onto the row the page receives', /descriptor\.sessionParentId = meta\.parent;/.test(server)); -assert('the descriptor carries the kind badge onto the row the page receives', - /descriptor\.sessionTag = sessionTagFor\(meta\.kind, meta\.purpose\);/.test(server)); +// The tag is written on EVERY row, not only on the rows that have a database entry. The +// descriptor comes out of a cache, so a row that loses its entry keeps the tag it was given +// last time, while the validator — which reads the same rows — reports none for it. A body and +// an ETag that disagree about one row is how a badge outlives the fact it was derived from, +// and the page only ever sees the body. +// +// The shape is what makes it unconditional: a `? :` with the tag on one side and null on the +// other cannot be inside a `if (meta)` block without becoming a second statement. +assert('the descriptor carries the kind badge onto the row the page receives, on every row', + /descriptor\.sessionTag = meta \? sessionTagFor\(meta\.kind, meta\.purpose\) : null;/.test(server)); // `?? null` rather than a bare copy: the page decides by whether this field is SET, so a missing // key and a present-but-undefined one have to land in the same place as a row that has no parent. diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs index fa8f34e7..06946750 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs @@ -1,5 +1,5 @@ /** - * The picker shows the conversation the reader is in. + * The picker shows the conversation the runtime database reports as current. * * The list was built from one number: the mtime of `messages.jsonl`. That number answers "when * did this session last write a message", and the picker needs to answer "is this session still @@ -10,25 +10,36 @@ * * So the visible set is no longer the window and nothing else. Three rules, each with a reason: * - * 1. The current conversation is always in the list, and first. It is asked of the runtime - * database rather than guessed from mtime, because the conversation being read is not - * necessarily the one whose file was written last. When it is outside the window it is - * built from the path the runtime recorded for it, under the same containment check the - * 'latest' resolver gives it. - * 2. The sessions of the current conversation's workspace come before the sessions of other + * 1. The conversation the runtime database reports as current is always in the list, and + * first. It is asked of the runtime database rather than guessed from mtime, because the + * conversation being read is not necessarily the one whose file was written last. When it + * is outside the window it is built from the path the runtime recorded for it, under the + * same containment check the 'latest' resolver gives it. + * 2. The sessions of that conversation's workspace come before the sessions of other * workspaces. The workspace is the one the runtime database records for the current * conversation — and it is compared here, on the server, because it is an absolute path on * the reader's machine and has no business in a browser. * 3. Each group keeps the walk's recency order. * + * What the runtime database reports is a ranking, not a certainty: a live lease, then an open + * turn, then the most recently updated. This app cannot see which conversation the reader is + * looking at, so nothing here claims to know — it pins the database's answer and says so. + * * Two sets, and they are not the same set: the walk's own list is untouched, because it is what * 'latest' falls back to when the runtime database is unavailable, and reordering it would * change what 'latest' means. Only the visible set is reordered. * - * The eight invariants below are checked against real stores and real runtime databases built - * here, not against a copy of the logic: the interesting cases are the ones a regex cannot see — - * a database whose columns are older than the code, a recorded path that escapes the store, a - * conversation whose file is the oldest of all. + * The sections below are checked against real stores and real runtime databases built here, + * not against a copy of the logic: the interesting cases are the ones a regex cannot see — a + * database whose columns are older than the code, a recorded path that escapes the store, a + * recorded path that points at a different session's directory, a conversation whose file is the + * oldest of all. + * + * Each section runs inside its own try, and a section that throws reports ITSELF rather than + * borrowing the name of the first assertion in it. That is not a detail: a section that threw + * has not judged anything, and a mutation that dies of an exception has to say which rule it + * broke. A catch-all that reuses an assertion's name makes every crash look like that + * assertion, which is how a mutation comes to "pass" by crashing the file it was applied to. * * `tests/mutate-session-picker.py` breaks each of them in the shipped source and requires this * guard to report it by name. @@ -187,7 +198,7 @@ try { one.visible[1]?.dir === made[0].dir && one.visible[19]?.dir === made[18].dir, dirsOf(one.visible).slice(0, 3).join('|')); } catch (error) { - failures.push('the current conversation is in the visible list even though its file is the oldest -- ' + failures.push('the current conversation is in the visible list; the section threw before it could answer -- ' + String(error && error.message ? error.message : error)); } @@ -222,7 +233,8 @@ try { < two.visible.findIndex((entry) => entry.dir === made[3].dir), dirsOf(two.visible).join('|')); } catch (error) { - failures.push('the sessions of the current workspace come before the sessions of other workspaces -- ' + failures.push('the sessions of the current workspace come before the sessions of other workspaces; ' + + 'the section threw before it could answer -- ' + String(error && error.message ? error.message : error)); } @@ -237,7 +249,8 @@ try { assert('and without a runtime database there is no current conversation to pin', three.current === null); } catch (error) { - failures.push('without a runtime database the list is the plain recency order -- ' + failures.push('without a runtime database the list is the plain recency order; ' + + 'the section threw before it could answer -- ' + String(error && error.message ? error.message : error)); } @@ -264,7 +277,8 @@ try { return sessionTagFor(meta ? meta.kind : null, meta ? meta.purpose : null) === null; })); } catch (error) { - failures.push('a database without the purpose column still lists every session -- ' + failures.push('a database without the purpose column still lists every session; ' + + 'the section threw before it could answer -- ' + String(error && error.message ? error.message : error)); } @@ -294,7 +308,8 @@ try { return Boolean(meta && meta.title); })); } catch (error) { - failures.push('a database without the workspace column still lists every session -- ' + failures.push('a database without the workspace column still lists every session; ' + + 'the section threw before it could answer -- ' + String(error && error.message ? error.message : error)); } @@ -305,16 +320,24 @@ try { // The path the runtime recorded points out of the sessions root. It is a database column // rather than a name this app chose, so it gets the same containment check a catalog file // name gets — and the conversation is not injected from it. - const outside = join(home, 'v2', 'outside'); - await mkdir(outside, { recursive: true }); - await writeFile(join(outside, 'messages.jsonl'), + // + // The directory is named for the current session, on purpose. The two checks that can refuse + // a recorded path are the containment check and the identity check, and a fixture that could + // be caught by either one cannot say which of them is doing the work. This one passes the + // identity check — the name really does encode the session it claims to be — so the only + // thing left to refuse it is the containment check, which is the rule this section is about. + const outsideName = '99-00-00-session_' + + Buffer.from(ID(30), 'utf8').toString('base64url'); + const outsideSession = join(home, 'v2', outsideName); + await mkdir(outsideSession, { recursive: true }); + await writeFile(join(outsideSession, 'messages.jsonl'), JSON.stringify({ role: 'user', content: 'x' }) + '\n', 'utf8'); runtimeDb(home, ['parent_session_id', 'purpose', 'workspace_dir'], [ - { sessionId: ID(30), relative: '../outside', workspace: 'workspace-one' }, + { sessionId: ID(30), relative: '../' + outsideName, workspace: 'workspace-one' }, ]); const six = await visibleFor(sessionsRoot, home, 20); assert('a recorded path that escapes the sessions root is refused', - !six.visible.some((entry) => entry.dir === outside), + !six.visible.some((entry) => entry.dir === outsideSession), dirsOf(six.visible).join('|')); assert('and the current conversation is not injected from it', !six.visible.some((entry) => entry.dir === made[30].dir), @@ -323,11 +346,39 @@ try { sameOrder(six.visible, made.slice(0, 20)), dirsOf(six.visible).join('|')); } catch (error) { - failures.push('a recorded path that escapes the sessions root is refused -- ' + failures.push('a recorded path that escapes the sessions root is refused; ' + + 'the section threw before it could answer -- ' + String(error && error.message ? error.message : error)); } -/* ---------- 9: what a workspace path is, compared ---------- */ +/* ---------- 9: a recorded path that belongs to another session ---------- */ + +try { + const { home, sessionsRoot, made } = await store(31); + // The path the runtime recorded points at a directory that is INSIDE the sessions root and + // belongs to a different session. The containment check passes — the path is exactly where + // that check says a recorded path may be — and the row that comes back is another session's + // transcript. Injected as the current conversation it would be listed twice, once as the + // injected row and once as the walk's own row for the same directory, while the conversation + // the page is about is nowhere in the list. So the recorded path is checked against the + // session it is supposed to be, and a mismatch is refused the way an outside path is. + runtimeDb(home, ['parent_session_id', 'purpose', 'workspace_dir'], [ + { sessionId: ID(30), relative: made[0].relative, workspace: 'workspace-one' }, + ]); + const seven = await visibleFor(sessionsRoot, home, 20); + assert('a recorded path that belongs to another session is refused, so no session is listed twice', + new Set(dirsOf(seven.visible)).size === seven.visible.length, + dirsOf(seven.visible).join('|')); + assert('and the list is still the plain recency order', + sameOrder(seven.visible, made.slice(0, 20)), + dirsOf(seven.visible).join('|')); +} catch (error) { + failures.push('a recorded path that belongs to another session is refused; ' + + 'the section threw before it could answer -- ' + + String(error && error.message ? error.message : error)); +} + +/* ---------- 10: what a workspace path is, compared ---------- */ assert('two spellings of one Windows path are the same workspace', sameWorkspacePath('D:\\work\\project', 'd:/work/project/')); @@ -341,7 +392,35 @@ assert('a workspace that is not there is not a workspace', assert('a POSIX path is compared as it stands, because two spellings there are two directories', !sameWorkspacePath('/work/Project', '/work/project')); -/* ---------- 10: what may leave the service ---------- */ +/* ---------- 11: what a kind may be ---------- */ +// +// The page draws the token, so the set of tokens is a closed contract between the two sides of +// the wire. A value outside it would have to be drawn as a badge the page invented, and the +// page refuses to invent one — so the effect of a stray token is a marker that silently +// disappears, and a reader is left with a row that looks like an ordinary conversation. +const KNOWN_TAGS = ['lightweight', 'scheduled', 'task']; +assert('a kind tag is one of the three the page knows, or no tag at all', + [null, undefined, '', 'conversation', 'cron', 'task', 'unknown'].every((kind) => + [null, undefined, '', 'context-mode:lightweight', 'cron:mavis:daily', + 'local-background-task:bg_1', 'local-task:x'].every((purpose) => { + const tag = sessionTagFor(kind, purpose); + return tag === null || KNOWN_TAGS.indexOf(tag) >= 0; + }))); +assert('a lightweight exchange is tagged as one', + sessionTagFor('conversation', 'context-mode:lightweight') === 'lightweight' + && sessionTagFor('conversation', ' context-mode:lightweight ') === 'lightweight'); +assert('a scheduled run is tagged as one', + sessionTagFor('cron', null) === 'scheduled' + && sessionTagFor('conversation', 'cron:mavis:daily') === 'scheduled'); +assert('a background task is tagged as one', + sessionTagFor('task', null) === 'task' + && sessionTagFor('conversation', 'local-background-task:bg_1') === 'task'); +assert('an ordinary conversation carries no kind tag', + sessionTagFor('conversation', null) === null + && sessionTagFor('conversation', '') === null + && sessionTagFor('unknown', null) === null); + +/* ---------- 12: what may leave the service ---------- */ // // The workspace is an absolute path on the reader's machine, and the purpose is free text that // can carry anything a person typed into it. Neither may reach the browser: the page gets an @@ -355,6 +434,19 @@ assert('the field list the page receives names neither the workspace nor the raw assert('the kind badge is the one field that says what kind of session this is', strip !== null && /sessionTag: value\.sessionTag \?\? null,/.test(strip[0])); +// The tag in the body and the tag in the ETag have to come from the same derivation, and the +// body has to write it on EVERY row. The descriptor comes out of a cache, so a row that loses +// its database entry keeps the tag it was given last time, while the validator -- which reads +// the same rows -- reports none. A body and an ETag that disagree about one row is how a badge +// outlives the fact it was derived from, and the page cannot see the disagreement: it only +// has the body. +const tagWrite = /descriptor\.sessionTag = meta \? sessionTagFor\(meta\.kind, meta\.purpose\) : null;/ + .exec(serverText); +const tagInsideMeta = /if \(meta\) \{[^}]*descriptor\.sessionTag/.exec(serverText); +assert('the kind tag is written on every row, not only on the rows that have one', + tagWrite !== null && tagInsideMeta === null, + tagWrite === null ? 'no unconditional tag write found' : String(tagInsideMeta && tagInsideMeta[0])); + for (const home of homes) rmSync(home, { recursive: true, force: true }); console.log('[session-picker] ' + pass + ' passed, ' + failures.length + ' failed'); From a0150b8d66b7e769fcf435ef473f5c5175f0e0fe Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 01:50:22 +0800 Subject: [PATCH 04/95] The picker listed runs among the conversations, and nothing said which was which MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The list was flat. Every run the runtime spawned from another session sat next to the reader's own conversations, with nothing on the row to say what it was, and on this machine thirteen of the twenty rows were runs of the one conversation being read -- so a reader comparing two entries could not tell somebody else's work from their own. They are not hidden either, because those runs were where that conversation actually happened. What changed is the shape. There are two controls now, and they cascade: the upper one, 「会话」, names conversations and nothing else -- the ones with no parent in the runtime database -- and it offers twenty of them plus the follow-latest option, so every row in it is a conversation. The lower one, 「子代理」, names the run inside the conversation that was picked. It is empty by default, it has values once a conversation is picked, and it is empty and disabled while the page follows the newest conversation: that conversation changes under the reader, and a run picked out of it would belong to a conversation that is no longer on screen. Its first option says 「（主会话本身）」 in words, so there is a way back. A run is a session with its own transcript, and the parent's transcript does not contain its turns -- so picking a run shows that run's own trajectory, and the conversation pick moves to its parent. The two controls never disagree about what is on screen: the run pick names the run, the conversation pick names the conversation it hangs off, and neither one can be read as the other. The label says nothing about the layer. It used to carry a word for it -- a session the runtime spawned from another one was marked 「子会话」 in the picker -- and the word is gone. One fact, one signal: the layer is the control the row is in. The service sends the conversations and every run in the pool, and every run row names the conversation it hangs off; the page groups them by that one id rather than keeping a hierarchy of its own. The pool is the whole walk, not the thirty most recent sessions, so a run of a conversation that has aged out of the list still travels -- a reader reading that conversation still has to be able to reach its runs. The guards that read the picker came with it, because what they claimed was about the shape that just changed. The page's own guard stopped asking what a row's label says and started asking what the two picks hold, read back out of a rendered pair of them; the guard on the two poll cadences asks that both picks are redrawn after the runs are stored; the one on the list cache stopped asking where a mark sits in the label and started asking how the page decides which rows are runs; and the two on the service ask the validator for the runs as a fourth argument, because they are inside its material now. --- plugins/avatasia/mmc-trajectory/README.md | 2 +- .../avatasia/mmc-trajectory/README.zh-CN.md | 2 +- .../mmc-trajectory/miniapp/client/index.html | 258 ++++++++++++++-- .../mmc-trajectory/miniapp/node/server.mjs | 179 ++++++++--- .../mmc-trajectory/tests/client-check.mjs | 290 ++++++++++++++---- .../tests/mutate-list-cache-ttl.py | 16 +- .../tests/mutate-session-picker.py | 27 +- .../tests/mutate-session-reachability.py | 43 ++- .../tests/mutate-sessions-etag.py | 51 ++- .../mmc-trajectory/tests/verify-freshness.mjs | 2 + .../tests/verify-list-cache.mjs | 10 +- .../tests/verify-poll-decoupling.mjs | 21 +- .../tests/verify-session-reachability.mjs | 15 +- .../tests/verify-sessions-etag.mjs | 10 +- 14 files changed, 749 insertions(+), 177 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/README.md b/plugins/avatasia/mmc-trajectory/README.md index a62321d4..896be6bb 100644 --- a/plugins/avatasia/mmc-trajectory/README.md +++ b/plugins/avatasia/mmc-trajectory/README.md @@ -131,7 +131,7 @@ The runtime database is opened in read-only mode. If it is missing, unreadable, - A single `messages.jsonl` is read up to 64 MB. Larger sessions are truncated and the page says so. - The ledger does not use virtual scrolling; every rendered row is a real element. The page therefore opens on the newest 200 records, and the window bar button switches to the whole session. The button names what pressing it will do, not the current state: 「显示全部」 at the default and 「显示最新 200 条」 once everything is showing, while the count beside it reports what is on screen. That full view is opt-in for a reason: on a 9,900-record session it is roughly 57,000 elements rebuilt every 2.5s poll, which was measured to freeze the page rather than be read. 「向上加载更早的 N 条」 widens the window in steps and a poll does not undo it. A sticky window bar under the overview strip offers that choice behind four explicit controls: switch the strip between equal width and a real time axis, load 200 earlier records, jump back to the latest 200, and show everything. "Show everything" builds every record in the session into the DOM at once and gets noticeably slower on large sessions; incoming records do not knock it back down to 200. - `messageCount` and `turnCount` in the session picker are estimates derived from a bounded prefix of the file; exact values come from the selected session. -- A session the runtime spawned from another one is marked 「子会话」 in the picker. The parent is deliberately not named: a branch is always newer than the session that spawned it, so its parent has usually aged out of the most-recent-N window this list is built from, and a lookup that fails quietly is how a picker ends up asserting something it cannot see. +- The session picker is **two selects in a cascade**. The upper one, 「会话」, lists conversations only — the ones with no `parent_session_id` in the runtime database. The lower one, 「子代理」, lists the runs of the conversation that was picked: it is **empty by default**, it has values once a conversation is picked, and it is empty and disabled while the page follows the newest conversation. Measured on this machine, 13 of the 20 rows were sub-agent runs, so a reader could not tell which row was their own conversation; those 13 runs were also where that conversation actually happened. Nothing is hidden, only moved — the runs are not rows of the conversation list any more, they are rows of the run list under it, and the conversation list is now the twenty most recent conversations rather than the twenty most recent sessions of any kind. Every run is listed, most recently active first, and pressing one switches the page to it. The layer is decided by `parent_session_id`, which is independent of the 「定时 / 任务 / 轻量」 markers. The parent's name is still not written out: the parent id comes down with the row, and the page groups by that one id rather than doing a lookup that could fail quietly. Picking a run shows **that run's own trajectory** — a run is a session with its own transcript, and the parent's transcript does not contain its turns — and the conversation select stays on its parent. - The picker also says what a session **is**, not only what it is called: a scheduled run carries 「定时」, a background task 「任务」, and a tool-less lightweight conversation 「轻量」, so none of them can be mistaken for a conversation you wrote. They stay in the list and stay selectable — a marker is a marker, not a filter. The conversation the runtime database reports as current comes first, then the sessions of its own workspace, then the rest; the workspace path and the raw purpose string are compared on the server and never reach the page. - Light theme token coverage is verified — every `--mcode-*` token the page consumes is defined for both themes — but its rendered appearance was not visually checked; the page was exercised under a dark system preference. diff --git a/plugins/avatasia/mmc-trajectory/README.zh-CN.md b/plugins/avatasia/mmc-trajectory/README.zh-CN.md index 4fdab332..874107fa 100644 --- a/plugins/avatasia/mmc-trajectory/README.zh-CN.md +++ b/plugins/avatasia/mmc-trajectory/README.zh-CN.md @@ -117,7 +117,7 @@ MiniMax Code 不会告诉 Mini App 是哪段对话打开了它的页面：传给 - 单个 `messages.jsonl` 最多读取 64 MB。更大的会话会被截断，页面会明确说明。 - 台里没有用虚拟滚动，每一行都是真实的 DOM 元素。页面默认只展示最新 200 条，窗口条上的按钮可以切到全量；全量是可选的，因为它很贵 —— 在这个会话上约 9,900 条、约 5.7 万个元素，每 2.5 秒轮询重建一次，实测会直接卡死页面。总览条正下方的吸顶窗口栏提供三个显式控件：「总览轴」「向上加载更早的 200 条」，以及在最新 200 条与全量之间切换的那一个按钮（它写的是按下去会变成什么，默认为「显示全部」，全量时为「显示最新 200 条」；当前窗口由右边的计数条报告，两者回答的是不同问题）。「总览轴」只改总览条的横轴——等宽（每条记录一样宽，完全忽略时间）或真实时间轴（宽度等于该记录的真实耗时）；它不筛选、不排序，台账里的行和页面上的任何计数都不变。「向上加载更早的 200 条」每点一次就多加一页，轮询不会把已经加宽的窗口重置回去；点切换按钮也不会自动滚动页面，读者正在看的那一行会留在原地。 - 会话选择器里的 `messageCount` / `turnCount` 是从文件的有界前缀估算的；精确值以选中会话后展示的为准。 -- 会话选择器里，被运行时从另一个会话派生出来的会话标为「子会话」，但不写父会话的名字：分支一定比派生它的会话新，而列表按最近活动取前 30 个，父会话通常已经掉出这个窗口。静默失败的查找，最终会让列表声称一个它看不见的东西。 +- 会话选择器是**级联的两个下拉框**。上面那个「会话」只列主会话 —— 运行时库里 `parent_session_id` 为空的那些；下面那个「子代理」只列所选那段对话的子代理，**默认是空**，选中主会话之后才有值，跟随最新时是空的且不可选。实测本机 20 行里有 13 行是子代理，读者根本分不出哪一行是自己的对话，而那 13 行恰恰就是这段对话真正发生的地方。它们没有被藏起来，只是从「平铺在会话里」变成「会话列表下面的子代理列表」：会话列表现在列的是最近 20 段主会话，而不是最近 20 个任意种类的会话；子代理有多少列多少，按各自最后一次活跃的时间倒序，最活跃的排在最上面，点其中一条就切到那一条。层级由 `parent_session_id` 决定，与「定时 / 任务 / 轻量」那套标记互相独立。父会话的名字仍然不写：父 id 由服务端随这一行带下来，页面按同一个 id 归组，不做一次可能悄悄失败的查找。选了子代理就显示**那一次运行自己的**轨迹 —— 每个子代理是一段独立的会话，有自己的 transcript，父会话的轨迹里并不包含它的轮次；此时上面那个下拉框停在它的父会话上。 - 会话选择器不只说会话**叫什么**，还说它**是什么**：定时任务标「定时」，后台任务标「任务」，没有工具的轻量会话标「轻量」，于是它们不会被误认成你自己开的对话。它们仍然在列表里、仍然可以选 —— 标记是标记，不是过滤。运行时库报告为当前的那段对话排在最前，然后是同一工作区的会话，再然后是其它；工作区路径和 purpose 原文只在服务端比较，不会发到页面。 - 浅色主题的 token 覆盖已验证（页面用到的每个 `--mcode-*` 变量在两套主题下都有定义），但渲染外观未做视觉检查；功能验收是在深色系统偏好下进行的。 diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 78d26260..47b7f462 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -2456,6 +2456,14 @@

会话轨迹

+ +
+ + +
@@ -3254,6 +3262,33 @@

检视

return ''; } + /** + * The three parts of a session's label, in the order they are printed. + * + * One function, because two places print a session's label — the picker's option and the + * sub-agent row under it — and two copies of a format is two places for the format to + * drift. It returns the parts rather than the whole string because the two callers put + * them in different shapes: the picker appends 「子会话」 after them, and the sub-agent row + * would only be repeating itself. + * + * DOM-free on purpose: it lives in this block so the offline probe can run it without a + * page, and so a change to the format is one edit rather than two. + * + * @param {any} session + * @param {number} now + * @returns {{ label: string, tail: string, tag: string }} + */ + function sessionLabelParts(session, now) { + var label = truncateText(session && session.label ? session.label : '', 42); + var when = formatAgo(session ? session.lastActiveAt : null, now); + var count = session ? session.messageCount : null; + var tail = isCount(count) ? '（' + when + ' · ' + formatInt(count) + ' 条）' : '（' + when + '）'; + // What kind of session this is, when the service said. It is a marker and not a filter: + // the row is still selectable, and a row with no kind of its own gets no marker. + var tag = session ? sessionTagBadge(session.sessionTag) : ''; + return { label: label, tail: tail, tag: tag }; + } + function safeText(value) { if (value === null || value === undefined) return ''; if (typeof value === 'string') return value; @@ -3648,6 +3683,10 @@

检视

var state = { sessions: [], + // The sub-agents of the main sessions in `sessions`, flat: every row names the main it + // hangs off. One list and one join key, rather than a second hierarchy — the page groups + // them by parent, and the service decides who the parent is. + children: [], sessionsLoaded: false, followLatest: true, // Seeded from the host id so the picker and the content agree. `sessionParam()` already @@ -3902,6 +3941,7 @@

检视

dom.kindControls = document.getElementById('kind-controls'); dom.filterSticky = document.getElementById('filter-sticky'); dom.sessionSelect = document.getElementById('session-select'); + dom.childSelect = document.getElementById('child-select'); dom.generationField = document.getElementById('generation-field'); dom.generationSelect = document.getElementById('generation-select'); dom.generationHint = document.getElementById('generation-hint'); @@ -4057,8 +4097,13 @@

检视

return false; } state.sessions = Array.isArray(result.body.sessions) ? result.body.sessions : []; + // The two lists are two views of one set: the service sends the main sessions and the + // runs of every conversation in the pool, and every run row names the main it hangs + // off. The page groups them by that name instead of keeping a hierarchy of its own. + state.children = Array.isArray(result.body.children) ? result.body.children : []; state.sessionsLoaded = true; renderSessionSelect(); + renderChildSelect(); if (cleared) renderBars(); return true; }) @@ -4478,34 +4523,84 @@

检视

* absence of a number in the shape of a number. Omitting the segment is the honest rendering * of 「这个数不存在」: the reader still has the label and the recency, which is what the picker * is for. tests/verify-list-cache.mjs pins the gate that produces the absence. */ - /* 「子会话」 says the structural fact and stops there. + /* The two picks, and what each one answers. * - * The parent is named by nobody, on purpose. A branch is always newer than the session that - * spawned it, so its parent has usually aged out of the most-recent-N window this list is - * built from — naming it would mean a lookup that fails exactly when a reader wants the - * answer most, and a lookup that fails quietly is how a picker ends up asserting something - * it cannot see. + * A run the runtime spawned from another session is a session like any other, and the runtime + * hangs it off its parent with one column. The page used to draw both in one dropdown, flat: + * thirteen of twenty rows were runs of the one conversation being read, and nothing on the row + * said so. * - * It is a marker and not a filter: nothing here writes state, so there is no second opinion - * about which records are on screen that could disagree with the table. */ - var BRANCH_MARK = ' · 子会话'; - + * It then drew them inside the same dropdown, indented under their conversation. That made the + * hierarchy visible, and made the dropdown long — forty rows on this machine, nineteen of them + * runs — so picking one conversation meant scrolling a list of titles to find it. + * + * It is two controls now, and the second is a cascade of the first: + * + * - 「会话」 answers 「which conversation」. It holds conversations and nothing else. + * - 「子代理」 answers 「which run inside it」. It holds the runs of the conversation the first + * pick names, and stays empty until there is one to name. + * + * Two controls is not two answers to one question: the questions are nested, and each has one. + * The layer is the container now, which says more than an indent or a word did — a row under + * 「子代理」 says what it is without saying it in words. + * + * A run is a session with its own transcript, and the parent transcript does not contain its + * turns. So picking a run shows that run's own trajectory, and the conversation pick moves to + * its parent: the two controls always agree about what is on screen. */ function sessionOptionLabel(session, now) { - var label = truncateText(session && session.label ? session.label : '', 42); - var when = formatAgo(session ? session.lastActiveAt : null, now); - var count = session ? session.messageCount : null; - var tail = isCount(count) ? '（' + when + ' · ' + formatInt(count) + ' 条）' : '（' + when + '）'; - // What kind of session this is, when the service said. It is a marker and not a filter: - // the row is still selectable, and a row with no kind of its own gets no marker. - var tag = session ? sessionTagBadge(session.sessionTag) : ''; - var branch = session && session.sessionParentId ? BRANCH_MARK : ''; - if (!label) return '未命名会话 ' + tail + tag + branch; - return label + ' ' + tail + tag + branch; + var parts = sessionLabelParts(session, now); + if (!parts.label) return '未命名会话 ' + parts.tail + parts.tag; + return parts.label + ' ' + parts.tail + parts.tag; + } + + /** + * The sub-agents of each conversation, grouped by the parent the service named. + * + * One pass over the flat list the service sends, and the order inside each group is the + * order it arrived in — most recently active first, which is the service's ordering and the + * only one the picker promises. + * + * @returns {Object} + */ + function childrenByParent() { + var grouped = {}; + var all = state.children || []; + for (var i = 0; i < all.length; i += 1) { + var child = all[i]; + if (!child || !child.id || !child.sessionParentId) continue; + var bucket = grouped[child.sessionParentId]; + if (bucket) bucket.push(child); + else grouped[child.sessionParentId] = [child]; + } + return grouped; + } + + /** + * The conversation the session on screen belongs to. + * + * A run belongs to the conversation that spawned it; a conversation belongs to itself. That + * is the only thing the two picks have to agree about, and it is derived rather than stored: + * the parent id comes down with the row, so there is no second copy of the mapping to keep + * in step with the service. + * + * @returns {string} + */ + function ownerSessionId() { + var id = state.sessionId || ''; + if (!id) return ''; + var all = state.children || []; + for (var i = 0; i < all.length; i += 1) { + var child = all[i]; + if (child && child.id === id && child.sessionParentId) return child.sessionParentId; + } + return id; } function renderSessionSelect() { var select = dom.sessionSelect; - var previous = state.followLatest ? 'latest' : state.sessionId || ''; + // The conversation pick names the conversation, not the run: picking a run leaves this + // control on the run's parent, so the two controls never disagree about what is on screen. + var previous = state.followLatest ? 'latest' : ownerSessionId(); clear(select); var followOption = el('option', null, '跟随最新（自动切换）'); @@ -4535,6 +4630,50 @@

检视

if (select.selectedIndex < 0) select.value = 'latest'; } + /** + * The run pick: the runs of the conversation on screen, and nothing else. + * + * Empty while the page follows the newest conversation — that list changes under the reader, + * and a run picked out of it would be a run of a conversation that is no longer on screen. + * Empty with a conversation picked and no run picked, which is the default; the first option + * says so in words. + */ + function renderChildSelect() { + var select = dom.childSelect; + clear(select); + var owner = state.followLatest ? '' : ownerSessionId(); + + if (!owner) { + var idle = el('option', null, '（跟随最新）'); + idle.value = ''; + select.appendChild(idle); + select.disabled = true; + select.value = ''; + return; + } + + select.disabled = false; + var self = el('option', null, '（主会话本身）'); + self.value = ''; + select.appendChild(self); + + var now = Date.now(); + var kids = childrenByParent()[owner] || []; + for (var j = 0; j < kids.length; j += 1) { + var option = el('option', null, sessionOptionLabel(kids[j], now)); + option.value = kids[j].id; + select.appendChild(option); + } + // The run on screen, when it is a run of this conversation; otherwise the default, which + // is the conversation itself. + var current = state.sessionId || ''; + var isRun = false; + for (var k = 0; k < kids.length; k += 1) { + if (kids[k].id === current) isRun = true; + } + select.value = isRun ? current : ''; + } + /* ---------------- rendering: KPI ---------------- */ /** @@ -9045,13 +9184,57 @@

检视

} } + /** + * The reader picked a conversation. + * + * A conversation and a run are both sessions, and both go through the one writer that says + * which session is on screen. Picking a conversation leaves the run pick empty, so the page + * shows the conversation's own trajectory. + */ + function onSessionSelectChange() { + var value = dom.sessionSelect.value; + // A different session has a different generation chain; keeping the old pin would + // ask that session for a generation number it may never have had. + state.generation = null; + if (value === 'latest') { + setFollow(true); + return; + } + selectSession(value); + } + + /** + * The reader picked a run, or the conversation itself. + * + * A run is a session with its own transcript, and the conversation's transcript does not + * contain its turns — so this switches the page to that run's own trajectory. The + * conversation pick stays on the run's parent, which is where it already was. + * + * The first option is the conversation itself: picking it returns to the conversation's + * trajectory, and it is where the pick sits until a run is chosen. + */ + function onChildSelectChange() { + var value = dom.childSelect.value; + state.generation = null; + if (!value) { + var owner = ownerSessionId(); + if (!owner) return; + selectSession(owner); + return; + } + selectSession(value); + } + /* ---------------- wiring ---------------- */ function setFollow(enabled) { state.followLatest = enabled; dom.followBtn.setAttribute('aria-pressed', enabled ? 'true' : 'false'); dom.followState.textContent = enabled ? '开' : '关'; - dom.sessionSelect.value = enabled ? 'latest' : state.sessionId || 'latest'; + // The conversation pick names the conversation, not the run: with a run on screen it + // stays on the run's parent. + dom.sessionSelect.value = enabled ? 'latest' : (ownerSessionId() || 'latest'); + renderChildSelect(); state.signature = ''; state.loading = true; fetchSessions(); @@ -9059,6 +9242,23 @@

检视

announce(enabled ? '已开启跟随最新，正在跟踪最近的会话。' : '已关闭跟随最新，固定在所选会话。'); } + /** + * The one place a session is chosen. + * + * A conversation and a sub-agent are both sessions, so both come through here and + * 「which session is on screen」 has one writer and one meaning. A different session has a + * different generation chain, so the pin is dropped here rather than at each call site — a + * generation number the new session may never have had would ask it for a slice it does not + * have. + * + * @param {string} id + */ + function selectSession(id) { + state.generation = null; + state.sessionId = id; + setFollow(false); + } + /** * Builds one button per rule, plus 全部异常 for the set as a whole. Built from the table * rather than written into the markup so that a rule has exactly one name, in one place. @@ -9465,18 +9665,8 @@

检视

fetchTrajectory(true); }); - dom.sessionSelect.addEventListener('change', function () { - var value = dom.sessionSelect.value; - // A different session has a different generation chain; keeping the old pin would - // ask that session for a generation number it may never have had. - state.generation = null; - if (value === 'latest') { - setFollow(true); - return; - } - state.sessionId = value; - setFollow(false); - }); + dom.sessionSelect.addEventListener('change', onSessionSelectChange); + dom.childSelect.addEventListener('change', onChildSelectChange); dom.generationSelect.addEventListener('change', function () { var raw = dom.generationSelect.value; diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index e8a80399..05c1ffc3 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -827,6 +827,7 @@ export async function readSessionIdentity(dir) { * * @param {string} sessionsRoot * @returns {Promise<{ recentEntries: Array<{ dir: string, sizeBytes: number, lastActiveAt: number }>, + * allEntries: Array<{ dir: string, sizeBytes: number, lastActiveAt: number }>, * byDecodedId: Map>, * truncated: boolean }>} */ @@ -875,7 +876,11 @@ export async function collectSessionCandidates(sessionsRoot) { else byDecodedId.set(id, [entry]); } - return { recentEntries: entries.slice(0, MAX_SESSION_LIST), byDecodedId, truncated }; + // `allEntries` is the same walk, uncut. The 30-entry window is what 'latest' falls back to and + // what the picker used to draw from; the list of MAIN sessions needs a deeper pool than that, + // because a sub-agent sits in that window and would otherwise take a main session' place. + // The walk already stat'ed every one of these, so the deeper pool costs no extra disk work. + return { recentEntries: entries.slice(0, MAX_SESSION_LIST), allEntries: entries, byDecodedId, truncated }; } /** @@ -1275,6 +1280,46 @@ export async function buildVisibleSessionList(entries, current, titles, sessions return ordered.slice(0, limit); } +/** + * The two layers of the picker: the main sessions, and the sub-agents hanging off each of them. + * + * A sub-agent is a session like any other on disk — its own directory, its own `messages.jsonl` — + * and the runtime hangs it off its parent with one column. So the layer is a fact about a row, + * not about a directory, and it is read from that column and nowhere else. `purpose` is not the + * judge here: it says what a session IS (a scheduled run, a background task), which is a different + * question from what it hangs off, and the two are orthogonal — a sub-agent is a task AND somebody's + * child. + * + * The parent's id and the child's id are the same kind of id. Measured on this machine: all 208 + * session directories carry a `manifest.json` whose `sessionId` equals the id decoded from the + * directory name, and that is the id the runtime database stores. So the parent's id here is the + * id the page will see on the parent's row, and the client can join the two layers on it. + * + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries + * @param {Map} titles + * @returns {{ mains: Array<{ dir: string, sizeBytes: number, lastActiveAt: number }>, childrenByParent: Map> }} + */ +export function splitSessionLayers(entries, titles) { + const mains = []; + const childrenByParent = new Map(); + for (const entry of entries) { + const id = decodeSessionIdFromDirName(basename(entry.dir)); + const meta = id ? titles.get(id) : null; + const parent = meta && typeof meta.parent === 'string' && meta.parent ? meta.parent : null; + if (!parent) { + mains.push(entry); + continue; + } + const bucket = childrenByParent.get(parent); + if (bucket) bucket.push(entry); + else childrenByParent.set(parent, [entry]); + } + // Most recently active first, inside every bucket. The list says so in the one place that + // orders it, so the order cannot be one thing on the server and another on the page. + for (const bucket of childrenByParent.values()) bucket.sort((a, b) => b.lastActiveAt - a.lastActiveAt); + return { mains, childrenByParent }; +} + /** * Closing a read-only handle cannot fail in a way the caller can act on — the work it was * opened for is already done or already abandoned. Swallowing keeps every call site from @@ -2641,12 +2686,18 @@ export function etagMatches(header, etag) { * SESSION_CACHE_TTL_MS to the descriptor cache this reuses, so the bound is that cache's and not * a new one. tests/verify-list-cache.mjs pins the TTL relationship that says so. * + * The sub-agents are in it for the same reason the kind badge is: they are on the page, and a + * sub-agent that starts, ends, or is renamed changes what the page says. A material that did not + * move would answer 304 for a body the caller never held — and the body it never held is the one + * with the new sub-agent in it. + * * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} visible * @param {Map} titles * @param {number} limit + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} [children] * @returns {string} */ -export function sessionsValidator(visible, titles, limit) { +export function sessionsValidator(visible, titles, limit, children) { const parts = ['limit=' + limit]; for (const entry of visible) { const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir))); @@ -2667,6 +2718,17 @@ export function sessionsValidator(visible, titles, limit) { // can return, and no field needs a second representation to be safe. parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag])); } + // The sub-agents, in the order the payload carries them: most recently active first inside + // each group. The parent id is in the tuple because the grouping is part of what the page + // shows — the same sub-agent under a different main is a different page. + for (const entry of children || []) { + const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir))); + const title = meta ? String(meta.title ?? "") : ""; + const kind = meta ? String(meta.kind ?? "") : ""; + const parent = meta ? String(meta.parent ?? "") : ""; + const tag = meta ? String(sessionTagFor(meta.kind, meta.purpose) ?? "") : ""; + parts.push(JSON.stringify([parent, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, tag])); + } return parts.join("\n"); } @@ -3316,22 +3378,24 @@ export async function start(context) { const handleSessions = async (response, url) => { const limit = normalizeLimit(url.searchParams.get('limit'), 20, 1, 50); - let entries; + let candidates; try { // The picker is built from the walk's own list: the most recent sessions, mtime-descending. // That list is also what 'latest' falls back to when the runtime database is unavailable, - // and nothing here reorders it. The VISIBLE set below is a different set — it widens this - // one by at most one entry, the current conversation, and the rule for picking twenty of - // the thirty is no longer pure mtime: the current conversation comes first, then the - // sessions of its workspace. The id index beside the list is what `handleTrajectory` - // resolves explicit ids against, and it is deliberately not what the picker is built from. - entries = (await listEntries()).recentEntries; + // and nothing here reorders it. The VISIBLE set below is a different set — it draws from + // the MAIN sessions only, and the rule for picking twenty of them is no longer pure mtime: + // the current conversation comes first, then the sessions of its workspace. The 30-entry + // window is deliberately NOT the pool: a sub-agent sits in that window and would take a + // main session' place, and the walk has already stat'ed every candidate besides, so the + // deeper pool costs no extra disk work. + candidates = await listEntries(); } catch (error) { context.logger.error('miniapp.trajectory.read_error', { reason: 'root_unreadable', code: errorCode(error) }); sendError(response, 503, 'trajectory_unavailable', '未找到本地会话数据目录'); return; } - if (entries.length === 0) { + const allEntries = candidates.allEntries; + if (allEntries.length === 0) { sendError(response, 503, 'trajectory_unavailable', '未找到本地会话数据目录'); return; } @@ -3342,23 +3406,43 @@ export async function start(context) { // `resolveCurrentConversation`, which says so where it happens. const current = await resolveCurrentConversation(location.root); - // Real titles from the runtime database, for every candidate in the window and for the current - // conversation, which may be outside it. The workspace column rides along because the visible - // set is ordered by it. - const ids = entries.map((entry) => decodeSessionIdFromDirName(basename(entry.dir))).filter(Boolean); + // Real titles from the runtime database, for every candidate and for the current conversation, + // which may be outside the walk. The workspace column rides along because the visible set is + // ordered by it, and the parent column rides along because the layer is decided by it. + const ids = allEntries.map((entry) => decodeSessionIdFromDirName(basename(entry.dir))).filter(Boolean); if (current && current.sessionId && ids.indexOf(current.sessionId) < 0) ids.push(current.sessionId); const titles = await sessionTitles(ids); + // The two layers. A sub-agent is a session like any other on disk, and the runtime hangs it + // off its parent with one column; so the layer is a fact about a row, and the page gets the + // main sessions in the picker and the sub-agents of whichever main is in hand below it. + const layers = splitSessionLayers(allEntries, titles); + // The visible set: the current conversation first, then the sessions of its workspace, then // the rest -- each group mtime-descending. At most one session joins the window this way, and // it is always the current conversation. The number of visible sessions is unchanged. - const visible = await buildVisibleSessionList(entries, current, titles, location.sessionsRoot, limit, () => { + const visible = await buildVisibleSessionList(layers.mains, current, titles, location.sessionsRoot, limit, () => { context.logger.warn('miniapp.trajectory.read_error', { reason: 'session_dir_rejected' }); }); - // Built from the same two values the payload below is built from, and computed before any - // descriptor work — which is what lets an unchanged list answer without building it. - const etag = weakETag(sessionsValidator(visible, titles, limit)); + // Every run in the pool, and not only the runs of the mains that are on the page. + // + // The picker is two controls now: the conversation, and the runs inside it. The second one + // lists the runs of the conversation the reader picked -- which may be a conversation that + // has aged out of the twenty, the one the page keeps alive with a row of its own. Dropping + // those runs would break the promise the main list already keeps: the session you are + // reading does not disappear because it stopped being new. The order inside each group is + // the one `splitSessionLayers` set -- most recently active first. + const childEntries = []; + for (const bucket of layers.childrenByParent.values()) { + for (const child of bucket) childEntries.push(child); + } + + // Built from the same values the payload below is built from, and computed before any + // descriptor work -- which is what lets an unchanged list answer without building it. The + // sub-agents are in it: a new one changes what the page shows, and a validator that did not + // move would let the caller keep a body that never had it. + const etag = weakETag(sessionsValidator(visible, titles, limit, childEntries)); const requestEtag = response.req && response.req.headers ? response.req.headers['if-none-match'] : null; if (etagMatches(requestEtag, etag)) { sendJsonBody(response, 304, '', { etag, revalidate: true }); @@ -3370,33 +3454,52 @@ export async function start(context) { // to produce. let body = sessionsBody.material === etag ? sessionsBody.body : null; if (body === null) { + // One writer for a row, so the main sessions and the sub-agents cannot drift apart in the + // fields they carry: the same title, the same kind, the same tag, the same floor. + const describeRow = async (entry) => { + const descriptor = await describeSession(entry); + const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir))); + if (meta) { + if (meta.title) descriptor.label = meta.title; + descriptor.sessionKind = meta.kind; + descriptor.sessionParentId = meta.parent; + } + // What this session IS, as a token the page can render: a lightweight exchange, a + // scheduled run, a background task. The raw purpose stays on this side of the wire -- + // it is free text, and free text is not something to hand a reader's browser. + // + // Assigned unconditionally, and that is the point. The descriptor comes out of a cache, + // so a session whose row is missing from the runtime database keeps the tag it was + // given last time -- while the validator, which reads the same rows, reports no tag for + // it. A body and an ETag that disagree about the same row is how a badge outlives the + // fact it was derived from, so the field is written on every pass, not only when there + // is something to write. + descriptor.sessionTag = meta ? sessionTagFor(meta.kind, meta.purpose) : null; + return descriptor; + }; const described = []; for (const entry of visible) { try { - const descriptor = await describeSession(entry); - const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir))); - if (meta) { - if (meta.title) descriptor.label = meta.title; - descriptor.sessionKind = meta.kind; - descriptor.sessionParentId = meta.parent; - } - // What this session IS, as a token the page can render: a lightweight exchange, a - // scheduled run, a background task. The raw purpose stays on this side of the wire -- - // it is free text, and free text is not something to hand a reader's browser. - // - // Assigned unconditionally, and that is the point. The descriptor comes out of a cache, - // so a session whose row is missing from the runtime database keeps the tag it was - // given last time -- while the validator, which reads the same rows, reports no tag for - // it. A body and an ETag that disagree about the same row is how a badge outlives the - // fact it was derived from, so the field is written on every pass, not only when there - // is something to write. - descriptor.sessionTag = meta ? sessionTagFor(meta.kind, meta.purpose) : null; - described.push(descriptor); + described.push(await describeRow(entry)); } catch (error) { context.logger.warn('miniapp.trajectory.read_error', { reason: 'describe_failed', code: errorCode(error) }); } } - body = JSON.stringify({ sessions: described.map(stripInternalFields) }); + // The sub-agents, as a flat list: each row names the main it hangs off, and the page groups + // them. A flat list is what lets the two layers join on ONE id -- the id the main row + // already carries -- without a second naming scheme to keep in step with it. + const children = []; + for (const entry of childEntries) { + try { + children.push(await describeRow(entry)); + } catch (error) { + context.logger.warn('miniapp.trajectory.read_error', { reason: 'describe_failed', code: errorCode(error) }); + } + } + body = JSON.stringify({ + sessions: described.map(stripInternalFields), + children: children.map(stripInternalFields), + }); sessionsBody = { material: etag, body }; } sendJsonBody(response, 200, body, { etag, revalidate: true }); diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index 5e442e31..f973e268 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -527,7 +527,7 @@ const codeOnly = html .replace(/\/\*[\s\S]*?\*\//g, '') .replace(/^[ \t]*\/\/.*$/gm, ''); -/* ------------------------------------------------------------------ session picker: branch marker */ +/* ------------------------------------------------------------------ session picker: the two picks */ // A session that the runtime spawned from another one used to be listed exactly like a top-level // conversation, because nothing in the list said otherwise: the parent column existed in the store // and was never read, and the one field that did reach the page (`sessionKind`) had no reader in @@ -537,41 +537,29 @@ const codeOnly = html // gets instead is `sessionTag` — the token the service derives from the row's purpose and kind — // and it is the one the page draws. The raw kind column is still on the wire and still unread. // -// The marker is checked here by RUNNING it rather than by matching its text, because the failure -// worth catching is not a typo in the string — it is the marker being applied to everything, or to -// nothing, and both of those still contain the right characters. The decisive case is the third -// one: two sessions identical except for `sessionParentId`, whose labels must differ by exactly -// the marker and nothing else. A version that appended the marker unconditionally passes every -// "does it contain 子会话" check and fails this one. -const branchMarkSrc = /var BRANCH_MARK = '[^']*';/.exec(html); -const branchLabelSrc = fnSource('function sessionOptionLabel(session, now)'); -if (!branchMarkSrc || !branchLabelSrc) { - failures.push('session picker: could not lift sessionOptionLabel / BRANCH_MARK'); +// How a run is told apart is now WHICH CONTROL it is in, not what its label says: the conversation +// pick holds conversations, and the run pick holds the runs of the conversation that was picked. +// The label says nothing about it — one fact, one signal — and the signal is the control itself. +const labelSrc = fnSource('function sessionOptionLabel(session, now)'); +if (!labelSrc) { + failures.push('session picker: could not lift sessionOptionLabel'); } else { - vm.runInContext(branchMarkSrc[0] + '\n' + branchLabelSrc, sandbox); + vm.runInContext(labelSrc, sandbox); const NOW = 1_800_000_000_000; const base = { label: '一个会话', lastActiveAt: NOW - 60_000, messageCount: 42 }; const plain = sandbox.sessionOptionLabel({ ...base }, NOW); - const branch = sandbox.sessionOptionLabel({ ...base, sessionParentId: 'mvs_parent' }, NOW); + const run = sandbox.sessionOptionLabel({ ...base, sessionParentId: 'mvs_parent' }, NOW); - assert('a top-level session is not marked as a branch', + // The label does not say it — the control does. A version that put the mark back into the + // label would say it twice, and a version that dropped it without moving the row would say it + // not at all. + assert('a run is not marked in its label', + run === plain); + assert('and a conversation is not marked either', plain.indexOf('子会话') < 0); - assert('a session the runtime spawned is marked as one', - branch.indexOf('子会话') >= 0); - // Identity, not shape: the only thing that differs between these two inputs is the field, so - // the only thing allowed to differ between the two labels is the marker. - assert('the marker is driven by the field, not by where the row sits in the list', - branch.replace(' · 子会话', '') === plain); - assert('and nothing else about the label changes with it', - branch.length === plain.length + ' · 子会话'.length); - // An empty parent string is "no parent", not "a parent named the empty string". + // An empty parent id is "no parent", not "a parent named the empty string". assert('an empty parent id is not a parent', sandbox.sessionOptionLabel({ ...base, sessionParentId: '' }, NOW) === plain); - // The unnamed branch takes the marker too — otherwise the one case with no title to read is - // the one case a reader cannot place. - assert('an unnamed branch is still marked', - sandbox.sessionOptionLabel({ lastActiveAt: NOW - 60_000, messageCount: 42, - sessionParentId: 'mvs_parent' }, NOW).indexOf('子会话') >= 0); } /* ------------------------------------------------------------------ session picker: kind badge */ @@ -585,7 +573,7 @@ if (!branchMarkSrc || !branchLabelSrc) { // browser somebody else's words. The token set is closed, and a token this page does not know // about produces no badge rather than one this page invented. // -// Same discipline as the branch marker above, and for the same reason: the failure worth catching +// Same discipline as the layer check above, and for the same reason: the failure worth catching // is not a typo in the string, it is the badge being applied to everything or to nothing, and both // of those still contain the right characters. const tagBadge = sandbox.sessionTagBadge; @@ -622,14 +610,22 @@ if (typeof tagBadge === 'function') { light.replace(' · 轻量', '') === plain); assert('and nothing else about the label changes with it', light.length === plain.length + ' · 轻量'.length); - // Both markers at once: the kind says what the session IS, the branch says where it came from, - // and the order between them is fixed — the kind first, because it is the fact about the row - // itself and the branch is a fact about its history. + // Where the badge sits inside the label: the name first, then the times and the count, then + // what the session IS. Moved to the front, the row still carries the right characters in the + // right places — every check that asks 「does the row say 轻量」 passes — and the row now reads + // as a category with a name attached to it. + assert('the badge comes after the name, not in front of it', + light.indexOf(' · 轻量') > light.indexOf(base.label), + JSON.stringify(light)); + // Both facts at once: the kind says what the session IS, and the parent says where it came + // from. The kind is in the label; the parent is in the position, so the label carries the kind + // and no longer says 「子会话」 in words — one fact, one signal. const both = sandbox.sessionOptionLabel({ ...base, sessionTag: 'lightweight', sessionParentId: 'mvs_parent' }, NOW); - assert('a row with both a kind and a parent shows the kind before the branch mark', - both.indexOf(' · 轻量') >= 0 - && both.indexOf(' · 子会话') >= 0 - && both.indexOf(' · 轻量') < both.indexOf(' · 子会话')); + assert('a row with both a kind and a parent still shows the kind', + both.indexOf(' · 轻量') >= 0); + assert('and no longer claims to be a branch in words — where it sits says that', + both.indexOf('子会话') < 0, + both); } const checks = [ @@ -1212,6 +1208,72 @@ for (const [name, ok] of checks) { else failures.push('policy: ' + name); } +/* ------------------------------------------------------------------ session picks: the cascade */ +// Two controls now. The conversation pick names the conversation; the run pick names the run +// inside it, and holds the runs of the conversation the first pick named. A run used to sit in +// the conversation pick as a peer of the conversations — thirteen of twenty rows on this machine +// were runs of the one conversation being read — and it used to sit there indented under its +// conversation, which made the dropdown forty rows long. +// +// The failures worth catching are not strings that survived an edit: they are a run sitting in +// the conversation pick, a run pick that is empty when it should not be, a run pick that lists +// the runs of the wrong conversation, and a pick that does not switch. So everything below is +// read back out of a rendered picker, not out of its source. +// +// The functions are lifted HERE, before the picker section below, because that section runs +// `renderSessionSelect` too and the picker now asks the same question this section asks: which +// rows are runs, and which control holds them. +const layerFns = [ + 'function sessionOptionLabel(session, now)', + 'function childrenByParent()', + 'function ownerSessionId()', + 'function renderSessionSelect()', + 'function renderChildSelect()', + 'function selectSession(id)', + 'function setFollow(enabled)', + 'function onSessionSelectChange()', + 'function onChildSelectChange()' +]; +const layerSrc = layerFns.map((name) => fnSource(name)); +if (layerSrc.some((src) => !src)) { + failures.push('session layers: could not lift ' + layerFns.filter((_, i) => !layerSrc[i]).join(', ')); +} else { + vm.runInContext(layerSrc.join('\n'), sandbox); +} + +// A declaration rather than a const, because the picker section below uses it and this section +// is written after it: a `const` would be in its temporal dead zone there, and the failure would +// read as a broken stub rather than as an ordering mistake. +function stubSelect() { + const options = []; + return { + options, + selectedIndex: -1, + _value: '', + get value() { return this._value; }, + set value(next) { + this._value = next; + this.selectedIndex = options.findIndex((option) => option.value === next); + }, + get firstChild() { return options[0] || null; }, + removeChild(child) { + const at = options.indexOf(child); + if (at >= 0) options.splice(at, 1); + return child; + }, + appendChild(node) { + options.push(node); + if (this.selectedIndex < 0) this.selectedIndex = options.length - 1; + return node; + }, + insertBefore(node, reference) { + const at = reference ? options.indexOf(reference) : -1; + options.splice(at < 0 ? options.length : at, 0, node); + return node; + }, + }; +} + /* ------------------------------------------------------------------ session picker: where the pinned row goes */ // The picker can hold a row the server did not send: the session the reader pinned by hand, which // aged out of the twenty the service returns. The page adds that row itself, and where it puts it @@ -1232,36 +1294,13 @@ if (!renderSelectSrc) { // selected, and the two ways a row can be added to it. The browser picks the first option // when the list is built and picks by value afterwards, and `insertBefore` puts a row at a // position rather than at the end — which is the whole point of the row in question. - const stubSelect = () => { - const options = []; - return { - options, - selectedIndex: -1, - _value: '', - get value() { return this._value; }, - set value(next) { - this._value = next; - this.selectedIndex = options.findIndex((option) => option.value === next); - }, - appendChild(node) { - options.push(node); - if (this.selectedIndex < 0) this.selectedIndex = options.length - 1; - return node; - }, - insertBefore(node, reference) { - const at = reference ? options.indexOf(reference) : -1; - options.splice(at < 0 ? options.length : at, 0, node); - return node; - }, - }; - }; sandbox.el = (tag, className, text) => { const node = stubNode(tag, text); if (className) node.className = className; node.value = ''; return node; }; - sandbox.clear = (node) => { if (node && node.options) node.options.length = 0; }; + sandbox.clear = (node) => { while (node && node.firstChild) node.removeChild(node.firstChild); }; if (!sandbox.state) sandbox.state = {}; sandbox.state.sessions = [ { id: 'mvs_server_first', label: '服务端第一行', lastActiveAt: 1_800_000_000_000, messageCount: 3 }, @@ -1304,6 +1343,137 @@ if (!renderSelectSrc) { second.options.map((option) => option.value).join(' | ')); } +/* ------------------------------------------------------------------ session picks: the cascade */ +// The functions were lifted above, before the picker section: the picker asks the same question +// this section asks, so it needs them too. +if (layerSrc.every((src) => src)) { + const NOW = 1_800_000_000_000; + const MAIN_A = 'mvs_main_a'; + const MAIN_B = 'mvs_main_b'; + // Thirteen of them, because that is the number this machine actually has: the list is not a + // window, and a fixture with three rows would not notice one that quietly dropped the tenth. + const childrenOfA = Array.from({ length: 13 }, (_, i) => ({ + id: 'mvs_child_a' + i, + sessionParentId: MAIN_A, + label: '子代理甲 ' + i, + lastActiveAt: NOW - (i + 1) * 1000, + messageCount: i + 1 + })); + const childrenOfB = [{ + id: 'mvs_child_b0', sessionParentId: MAIN_B, label: '子代理乙', + lastActiveAt: NOW - 3000, messageCount: 5 + }]; + const layerState = { + sessions: [ + { id: MAIN_A, label: '主会话甲', lastActiveAt: NOW, messageCount: 10 }, + { id: MAIN_B, label: '主会话乙', lastActiveAt: NOW - 60_000, messageCount: 20 } + ], + children: childrenOfA.concat(childrenOfB), + followLatest: true, + sessionId: MAIN_A, + hostSessionId: null + }; + sandbox.state = layerState; + // The two picks, as two stub selects: the conversation one and the run one. + const convo = stubSelect(); + const runs = stubSelect(); + sandbox.el = (tag, className, text) => { + const node = stubNode(tag, text); + if (className) node.className = className; + node.value = ''; + return node; + }; + sandbox.clear = (node) => { while (node && node.firstChild) node.removeChild(node.firstChild); }; + sandbox.dom = { + sessionSelect: convo, + childSelect: runs, + followBtn: { setAttribute: () => {} }, + followState: { textContent: '' } + }; + sandbox.fetchSessions = () => {}; + sandbox.fetchTrajectory = () => {}; + sandbox.announce = () => {}; + + sandbox.renderSessionSelect(); + sandbox.renderChildSelect(); + + // 1. While the page follows the newest conversation there is nothing to offer: that list + // changes under the reader, and a run picked out of it belongs to a conversation that is no + // longer on screen. + assert('the run pick is empty while the page follows the newest conversation', + runs.options.length === 1 && runs.options[0].value === '' && runs.value === '', + runs.options.map((option) => option.value).join(' | ')); + assert('and it is disabled, so there is nothing to pick', + runs.disabled === true, String(runs.disabled)); + + // 2. The conversation pick holds conversations and nothing else. This is the rule the whole + // feature exists for: a run must never sit where a conversation sits. + const convoValues = convo.options.map((option) => option.value); + assert('the conversation pick holds no run at all', + !convoValues.some((value) => /^mvs_child_/.test(value)), + convoValues.join(' | ')); + assert('and it holds every conversation', + convoValues.length === 3 + && convoValues.indexOf(MAIN_A) === 1 + && convoValues.indexOf(MAIN_B) === 2, + convoValues.join(' | ')); + + // 3. With a conversation picked, the run pick holds its runs — all of them, in the order the + // service sent them. + layerState.followLatest = false; + layerState.sessionId = MAIN_A; + sandbox.renderSessionSelect(); + sandbox.renderChildSelect(); + const aIds = childrenOfA.map((child) => child.id); + const runValues = runs.options.map((option) => option.value); + assert('the run pick holds every run of the conversation on screen', + runValues.length === 14, runValues.join(' | ')); + assert('and they keep the order the service sent', + runValues.slice(1).join(',') === aIds.join(','), runValues.join(' | ')); + assert('and the first option is the conversation itself', + runValues[0] === '' && (runs.options[0].textContent || '').indexOf('主会话本身') >= 0, + JSON.stringify(runs.options[0] && runs.options[0].textContent)); + assert('and nothing is picked while no run is picked', + runs.value === '', String(runs.value)); + + // 4. The two picks agree about what is on screen: a run belongs to a conversation, and the + // conversation pick says which one. Asked while a RUN is on screen -- asked while a + // conversation is on screen it would have the same answer either way, and a guard that + // cannot tell the two apart is not a guard. + layerState.sessionId = 'mvs_child_a5'; + assert('the owner of a run is the conversation it hangs off', + sandbox.ownerSessionId() === MAIN_A, sandbox.ownerSessionId()); + layerState.sessionId = MAIN_A; + + // 5. Picking a run switches the page to it — that run's own trajectory, since a run is a + // session with its own transcript — and leaves the conversation pick on its parent. + runs.value = 'mvs_child_a5'; + sandbox.onChildSelectChange(); + assert('picking a run switches the page to that run', + layerState.sessionId === 'mvs_child_a5' && layerState.followLatest === false, + layerState.sessionId + ' / follow=' + layerState.followLatest); + sandbox.renderSessionSelect(); + assert('and the conversation pick stays on its parent', + convo.value === MAIN_A, String(convo.value)); + + // 6. And picking the conversation itself returns to the conversation. + sandbox.renderChildSelect(); + runs.value = ''; + sandbox.onChildSelectChange(); + assert('picking the conversation itself returns to the conversation', + layerState.sessionId === MAIN_A, String(layerState.sessionId)); + + // 7. And picking another conversation moves the run pick with it, and clears the run. + convo.value = MAIN_B; + sandbox.onSessionSelectChange(); + sandbox.renderChildSelect(); + assert('picking another conversation moves the run pick with it', + runs.options.length === 2 && runs.options[1].value === 'mvs_child_b0', + runs.options.map((option) => option.value).join(' | ')); + assert('and clears the run that was picked', + runs.value === '', String(runs.value)); +} + console.log('[2] pure helpers: ' + pass + ' passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py index daa7fd30..91d4488d 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py @@ -188,7 +188,7 @@ 'the parent row is mapped, and an empty id is no parent rather than a parent named ""'), ("the descriptor is built without the parent", - [(" descriptor.sessionParentId = meta.parent;", "")], + [(" descriptor.sessionParentId = meta.parent;", "")], "the descriptor carries the parent onto the row the page receives"), # This one is the most likely to be reintroduced by accident: it is a whole-field removal in @@ -197,12 +197,14 @@ [(" sessionParentId: value.sessionParentId ?? null,", "")], "the field survives the strip, with null as the floor"), - # The direction that looks right and is not: marking every row. Every string the guard looks - # for is still present, and the list reads as though every conversation were a sub-agent run. - ("the mark is applied to every session rather than the branches", - [(" var branch = session && session.sessionParentId ? BRANCH_MARK : '';", - " var branch = BRANCH_MARK;")], - "the page decides the marker by the field and not unconditionally", + # The direction that looks right and is not: the parent field is read, but not as the key + # the rows are grouped under. Every conversation then hangs the same sub-agents, and the run + # pick of every conversation lists the same thirteen of them -- so a reader picks a run that + # belongs to somebody else's conversation, and the page shows it as their own. + ("the parent field is read, but not as the key the rows are grouped under", + [(" var bucket = grouped[child.sessionParentId];", + " var bucket = grouped['mvs_one_parent_for_everything'];")], + "the page decides which rows are sub-agents by the parent field and not unconditionally", CLIENT), # The same pair, for the kind badge: applied to every row, and applied to none. The first one diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py index cee2b172..36aee4c7 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py @@ -23,7 +23,7 @@ check would list that other session twice and lose the conversation the page is about. Three more are on the page. The badge is applied to every row, the badge is applied to none, -and the badge is placed after the branch mark -- the third of those still contains the right +and the badge is placed in front of the name -- the third of those still contains the right characters in the right places, which is why the guards check the rendered label rather than the string. The last one is the row the reader pinned by hand, which the page adds itself: it belongs above the rows the service sent, and a version that appends it to the end still @@ -133,8 +133,8 @@ # validator -- which reads the same rows -- reports none. The body and the ETag then # disagree about the same row, and only the body reaches the page. ("the kind tag is only written on the rows that have one", - [(" descriptor.sessionTag = meta ? sessionTagFor(meta.kind, meta.purpose) : null;", - " if (meta) descriptor.sessionTag = sessionTagFor(meta.kind, meta.purpose);")], + [(" descriptor.sessionTag = meta ? sessionTagFor(meta.kind, meta.purpose) : null;", + " if (meta) descriptor.sessionTag = sessionTagFor(meta.kind, meta.purpose);")], "the kind tag is written on every row, not only on the rows that have one"), # ---- the badge, on the page ---- @@ -156,16 +156,17 @@ "a lightweight session is badged in the picker", CLIENT), - # Placed after the branch mark. Both markers are still there, both are still the right - # characters, and the row still says everything it said -- only the order between them has - # changed. A guard that asks "does the row contain 轻量" cannot see this at all; one that - # asks which of the two comes first can. - ("the kind badge is placed after the branch mark", - [(" if (!label) return '未命名会话 ' + tail + tag + branch;\n" - " return label + ' ' + tail + tag + branch;", - " if (!label) return '未命名会话 ' + tail + branch + tag;\n" - " return label + ' ' + tail + branch + tag;")], - "a row with both a kind and a parent shows the kind before the branch mark", + # The badge's place inside the label. The label carries no marker for the layer any more -- + # the run pick says which rows are runs by being the control they are in -- so the badge is + # judged against the name alone. Moved to the front, the row still contains the right + # characters in the right places -- every check that asks "does the row contain 轻量" passes -- + # and the row now reads as a category with a name attached to it. + ("the kind badge is placed in front of the name", + [(" if (!parts.label) return '未命名会话 ' + parts.tail + parts.tag;\n" + " return parts.label + ' ' + parts.tail + parts.tag;", + " if (!parts.label) return parts.tag + '未命名会话 ' + parts.tail;\n" + " return parts.tag + parts.label + ' ' + parts.tail;")], + "the badge comes after the name, not in front of it", CLIENT), # The row the reader pinned by hand, which the page adds itself because the service did not diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py index 8837ae61..36c312d3 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py @@ -37,14 +37,19 @@ # The original shape. Everything else in the suite still passes: the list is unchanged, the # trajectory route still finds every session in the window, and no assertion anywhere else # had a session outside it. + # + # The walk now hands out THREE products -- the thirty most recent sessions, the whole walk, + # and the id index -- so "one truncated list again" is the whole walk truncated as well as the + # index dropped. The second layer's pool is the whole walk, and the reachability fix's + # product is the index: losing either one is the shape this mutation puts back. ("the walk returns one truncated list again", - [(" return { recentEntries: entries.slice(0, MAX_SESSION_LIST), byDecodedId, truncated };", - " return { recentEntries: entries.slice(0, MAX_SESSION_LIST), byDecodedId: new Map(), truncated };")], + [(" return { recentEntries: entries.slice(0, MAX_SESSION_LIST), allEntries: entries, byDecodedId, truncated };", + " return { recentEntries: entries.slice(0, MAX_SESSION_LIST), allEntries: entries.slice(0, MAX_SESSION_LIST), byDecodedId: new Map(), truncated };")], "the id index is not the recent list", SERVER), ("the recent list is no longer capped", - [(" return { recentEntries: entries.slice(0, MAX_SESSION_LIST), byDecodedId, truncated };", - " return { recentEntries: entries, byDecodedId, truncated };")], + [(" return { recentEntries: entries.slice(0, MAX_SESSION_LIST), allEntries: entries, byDecodedId, truncated };", + " return { recentEntries: entries, allEntries: entries, byDecodedId, truncated };")], "the recent list is still capped at MAX_SESSION_LIST", SERVER), # ---- the index stops being built from the walk ---- @@ -52,13 +57,17 @@ # Correct-looking, and now the picker pays for the reachability fix: the index is derived # from the slice, so a session outside the window has no fast path again. ("the index is derived from the recent list instead of the walk", - [(" for (const entry of entries) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));", - " for (const entry of entries.slice(0, MAX_SESSION_LIST)) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));")], + [(" for (const entry of entries) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" + " // A name that does not decode is still a real session;", + " for (const entry of entries.slice(0, MAX_SESSION_LIST)) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" + " // A name that does not decode is still a real session;")], "the id index is not the recent list", SERVER), ("the index is no longer built at all", - [(" const id = decodeSessionIdFromDirName(basename(entry.dir));", - " const id = null;")], + [(" for (const entry of entries) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" + " // A name that does not decode is still a real session;", + " for (const entry of entries) {\n const id = null;\n" + " // A name that does not decode is still a real session;")], "the index is built from the decoded directory name", SERVER), # ---- duplicates silently resolved by directory order ---- @@ -133,10 +142,22 @@ # # Widening the picker is a UI decision that was explicitly not taken here. Taking it by # accident, inside a reachability fix, is how a 9x descriptor cost arrives unannounced. + # + # The picker's pool is now the whole walk rather than the thirty most recent sessions, so + # this is the line that says so. It is the second layer's requirement: a sub-agent in a + # thirty-row window would take a main session' place. ("the list route starts building from the id index", - [(" entries = (await listEntries()).recentEntries;", - " entries = [...(await listEntries()).byDecodedId.values()].flat();")], - "the list route builds from recentEntries", SERVER), + [(" const allEntries = candidates.allEntries;\n", + " const allEntries = [...candidates.byDecodedId.values()].flat();\n")], + "the list route builds from the whole walk", SERVER), + + # The trajectory route keeps the thirty most recent sessions for its 'latest' branch, and it + # must not start building from the index either: the index is a reachability fix, and the + # trajectory route is the one that pays for a descriptor per candidate. + ("the trajectory route starts building from the id index", + [(" const entries = candidates.recentEntries;\n", + " const entries = [...candidates.byDecodedId.values()].flat();\n")], + "the trajectory route keeps the recent list for the latest branch", SERVER), # ---- the page names a session it is not showing ---- # diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py index f6e04423..c59584f9 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py @@ -31,14 +31,30 @@ MUTATIONS = [ # ---- the material forgets something the payload uses ---- + # The anchors below name the loop they sit in, because the material now has two loops -- the + # main sessions and the sub-agents -- and the two read the same columns in the same order. An + # anchor that did not name its loop would match both, and the control would refuse to run + # rather than guess which one it meant. ("a rename stops moving the tag", - [(" const title = meta ? String(meta.title ?? \"\") : \"\";\n", " const title = \"\";\n")], + [(" for (const entry of visible) {\n" + " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const title = meta ? String(meta.title ?? \"\") : \"\";\n", + " for (const entry of visible) {\n" + " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const title = \"\";\n")], "a renamed session moves the tag", SERVER), # `kind` is what puts the sub-agent and cron badges on the rows, so a page that keeps a stale # tag is offering the reader a worker's session as if it were their conversation. ("a changed session kind stops moving the tag", - [(" const kind = meta ? String(meta.kind ?? \"\") : \"\";\n", " const kind = \"\";\n")], + [(" for (const entry of visible) {\n" + " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const title = meta ? String(meta.title ?? \"\") : \"\";\n" + " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n", + " for (const entry of visible) {\n" + " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const title = meta ? String(meta.title ?? \"\") : \"\";\n" + " const kind = \"\";\n")], "a changed session kind moves the tag, even with no title", SERVER), # The one that looks like a simplification. Two different requests, one tag. @@ -148,7 +164,16 @@ # review, not by a failing test: the guard was passing because every map it built varied only # the title and the kind, so nothing ever asked the question. ("the validator stops covering the parent the body sends", - [(" const parent = meta ? String(meta.parent ?? \"\") : \"\";\n", " const parent = \"\";\n")], + [(" for (const entry of visible) {\n" + " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const title = meta ? String(meta.title ?? \"\") : \"\";\n" + " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n" + " const parent = meta ? String(meta.parent ?? \"\") : \"\";\n", + " for (const entry of visible) {\n" + " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const title = meta ? String(meta.title ?? \"\") : \"\";\n" + " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n" + " const parent = \"\";\n")], "a changed parent moves the tag, even with the title and kind held constant", SERVER), # The one this round's independent pass found. The material joined its fields on a bare NUL, and @@ -169,7 +194,25 @@ # the row says. Leaving it out of the material makes the tag constant across a change the # reader can see, and the 304 that follows is a statement about bytes nobody re-computed. ("the validator stops covering the kind badge the body sends", - [(" const tag = meta ? String(sessionTagFor(meta.kind, meta.purpose) ?? \"\") : \"\";\n", + [(" for (const entry of visible) {\n" + " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const title = meta ? String(meta.title ?? \"\") : \"\";\n" + " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n" + " const parent = meta ? String(meta.parent ?? \"\") : \"\";\n" + " // The kind badge the page draws is derived from the row's kind and purpose, so it is\n" + " // reader-visible on its own and belongs in the material for the same reason `kind` does. A\n" + " // runtime database that changes a session's purpose changes what the row says, and a tag that\n" + " // did not move would answer 304 for a body the caller never held.\n" + " const tag = meta ? String(sessionTagFor(meta.kind, meta.purpose) ?? \"\") : \"\";\n", + " for (const entry of visible) {\n" + " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const title = meta ? String(meta.title ?? \"\") : \"\";\n" + " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n" + " const parent = meta ? String(meta.parent ?? \"\") : \"\";\n" + " // The kind badge the page draws is derived from the row's kind and purpose, so it is\n" + " // reader-visible on its own and belongs in the material for the same reason `kind` does. A\n" + " // runtime database that changes a session's purpose changes what the row says, and a tag that\n" + " // did not move would answer 304 for a body the caller never held.\n" " const tag = \"\";\n")], "a changed session purpose moves the tag, even with the title and kind held constant", SERVER), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-freshness.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-freshness.mjs index 1cd94ef4..5874e04d 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-freshness.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-freshness.mjs @@ -102,6 +102,8 @@ const drawBars = sandbox.renderBars; sandbox.render = () => { renders += 1; drawBars(); }; sandbox.renderLoading = noop; sandbox.renderSessionSelect = noop; +// The list body feeds two picks now, and the second one is redrawn from the same body. +sandbox.renderChildSelect = noop; sandbox.rePinWindow = noop; sandbox.runSearch = noop; diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs index 0dc04cd6..84e2f798 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs @@ -239,8 +239,14 @@ assert('the kind badge survives the strip, with null as the floor', // The reader-facing half, off the comment-stripped client for the reason given in section 6: the // comment above `sessionOptionLabel` names the marker, so an assertion made against the raw file // would be satisfied by prose describing the very thing it exists to require. -assert('the page decides the marker by the field and not unconditionally', - /var branch = session && session\.sessionParentId \? BRANCH_MARK : '';/.test(clientJs)); +// +// The page does not mark a row in its label any more — it decides which rows are sub-agents by +// the parent the service named, and puts them in the run pick of that parent. So the field's +// reader is the grouping, and the rule is that the grouping consults the field rather than +// anything else. +assert('the page decides which rows are sub-agents by the parent field and not unconditionally', + /if \(!child \|\| !child\.id \|\| !child\.sessionParentId\) continue;/.test(clientJs) + && /var bucket = grouped\[child\.sessionParentId\];/.test(clientJs)); assert('the page decides the kind badge by the field and not unconditionally', /var tag = session \? sessionTagBadge\(session\.sessionTag\) : '';/.test(clientJs)); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-poll-decoupling.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-poll-decoupling.mjs index 3e682b90..e90e081e 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-poll-decoupling.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-poll-decoupling.mjs @@ -157,8 +157,27 @@ if (picker) { assert('the picker recomputes recency from a render-time clock', /var now = Date\.now\(\);/.test(picker[0]) && /formatAgo\(session \? session\.lastActiveAt : null, now\)/.test(html)); +} +// The list body arrives, and the picker is redrawn from it. The claim is about the ORDER -- the +// redraw happens after the body is stored -- and not about how far apart the two statements are: +// a distance would have been a number to keep in step with the comments between them, and those +// comments are the part that changes. +const listFetch = /function fetchSessions\(manual\) \{[\s\S]*?\n \}/.exec(html); +assert('the list fetch is still in the shipped client', listFetch !== null); +if (listFetch) { + const body = listFetch[0]; + const at = (needle) => body.indexOf(needle); assert('the picker is re-rendered whenever a list body arrives', - /state\.sessions = [\s\S]{0,200}?renderSessionSelect\(\);/.test(html)); + at('state.sessions = ') >= 0 + && at('renderSessionSelect();') > at('state.sessions = ')); + // The picker has two controls now, and the runs are rows of the second one. They have to be + // stored before either is redrawn: a picker that redraws from the old list and the run pick + // from the new one is a page showing two different moments. + assert('and so is the run pick', + at('state.children = ') >= 0 + && at('renderChildSelect();') > at('state.children = ')); + assert('and the conversation pick with it', + at('renderSessionSelect();') > at('state.children = ')); } console.log('[poll-decoupling] ' + pass + ' passed, ' + failures.length + ' failed'); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs index ff0b0c09..5cf86028 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs @@ -204,11 +204,20 @@ assert('a full directory basename can still name a session', /basename\(candidate\.dir\) === requested/.test(SERVER)); /* ===================================================================== - * 9 — the picker route reads the recent list, not the index + * 9 — the picker route walks the store, not the id index * ===================================================================== */ -assert('the list route builds from recentEntries', - /\(await listEntries\(\)\)\.recentEntries/.test(SERVER)); +// The list route builds from the WALK and not from the id index: the index is a reachability +// fix, and building the picker from it would mean a descriptor per candidate on every poll. +// +// The walk's own list is now the WHOLE of it rather than the thirty most recent sessions, and +// that is the second layer's requirement rather than a widening for its own sake: a sub-agent is +// always newer than the session that spawned it, so a window of thirty is mostly sub-agents, and +// drawing the picker from it would push main sessions out of their own list. The walk has already +// stat'ed every candidate, so the deeper pool costs no extra disk work. +assert('the list route builds from the whole walk', + /const allEntries = candidates\.allEntries;/.test(SERVER) + && /const layers = splitSessionLayers\(allEntries, titles\);/.test(SERVER)); assert('the trajectory route keeps the recent list for the latest branch', /const entries = candidates\.recentEntries;/.test(SERVER)); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs index 99723527..0b6ca150 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs @@ -183,7 +183,7 @@ assert('the same collision in the parent cannot merge two readable rows either', // The material is still built in server.mjs, which is what the three assertions above are testing // rather than a copy of it. -const material = /export function sessionsValidator\(visible, titles, limit\) \{[\s\S]*?\n\}/.exec(server); +const material = /export function sessionsValidator\(visible, titles, limit, children\) \{[\s\S]*?\n\}/.exec(server); assert('the material is still built in server.mjs', material !== null); // Empty input, because the route calls this before it knows the walk found anything. @@ -198,7 +198,13 @@ assert('the list route is still in the shipped source', route !== null); if (route) { const code = route[0]; assert('the route derives its tag from sessionsValidator', - /weakETag\(sessionsValidator\(visible, titles, limit\)\)/.test(code)); + /weakETag\(sessionsValidator\(visible, titles, limit, childEntries\)\)/.test(code)); + // The sub-agents are on the page, so they are in the material. A tag that did not move when a + // sub-agent started would answer 304 for a body the caller never held — and the body it never + // held is the one with the sub-agent in it. + assert('and the sub-agents it sends are in the tag', + /sessionsValidator\(visible, titles, limit, childEntries\)/.test(code) + && /childEntries\.push\(child\)/.test(code)); // The same three values on both sides. A validator computed from anything the payload does not // use is a validator about a different payload. The pairing itself is settled by the call above From 21477412f6061d7d40c2159e5611d96b5ebbf3b6 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 01:50:32 +0800 Subject: [PATCH 05/95] The two layers of the picker had no guard of their own The rules that decide which sessions are conversations and which are their runs were rules in prose. A main session and a run of it looked the same, a run could be listed twice or dropped, the order inside a group could invert without anything noticing, and the layer could be decided by the wrong column -- which gets a run with no purpose and a conversation whose purpose says "task" exactly backwards. This adds the guard for them. It builds the store and the runtime database by hand, with the two columns deliberately disagreeing, and asks the service four things: the conversation list holds conversations and nothing else, every run in the pool is listed -- the pool is the whole walk, not a window -- the runs of one conversation arrive most recently active first, and the layer follows the parent column rather than the purpose column. The fixture also holds a conversation that is not on the page, because the runs of a conversation the reader is reading have to travel even after that conversation ages out of the list. It asks whether the validator moves when a run starts and when it runs again, because a validator that does not move lets a caller keep a body that never had it, and it reads the payload back from a real route rather than from a function, because a cap on the way out is invisible to a test of the service's own functions. One more question is asked of the payload: that every field it carries has a reader in the page, read out of the page's own fetch rather than out of a list written here. The controls break each of them one at a time in the shipped source and require the rule to be reported by name -- the ones on the page by the page's own guard, the ones on the service by this one. Fifteen of them, six on the service and nine on the page. The ones on the page break the two picks: a run in the conversation pick, a run pick that is never empty, a run pick that is not disabled while the page follows the newest conversation, a run pick that lists somebody else's runs, a run pick with no row for the conversation itself, an owner that is the run instead of its parent, and a pick that does not switch. One thing had to be fixed for that to work. The guard read the text of an option that was not there and threw, which meant a control that removed the rows died before any assertion could report it. A guard that breaks along with the code it is checking reports nothing, and a control that dies that way looks exactly like a control that caught something. --- .../tests/mutate-session-layers.py | 275 ++++++++++ .../tests/verify-session-layers.mjs | 468 ++++++++++++++++++ 2 files changed, 743 insertions(+) create mode 100644 plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py create mode 100644 plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py new file mode 100644 index 00000000..be05167b --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -0,0 +1,275 @@ +"""Break the two session layers -- main sessions in the conversation pick, sub-agents in the run +pick of the conversation they belong to -- in the shipped source and require the guards that own +them to report each way it comes back by name. + +The picker used to list sub-agents among the conversations, flat: on the machine this was measured +on, 13 of the 20 rows were sub-agent runs of the one conversation being read, and nothing said so. +A reader comparing two entries could not tell a sub-agent run from their own work. + +The fix is not a filter -- those runs were the actual content of the conversation, and hiding them +would trade a wrong list for an empty one. It is two controls: the conversation pick names the +conversation, the run pick names the run inside it. The service sends the conversations and every +run in the pool, and every run row names the conversation it hangs off. + +The rules the mutations below take back, one at a time: + + * the main list holds main sessions and nothing else; + * a main session's runs are all of them -- the list is not a window; + * they are ordered by when each one last ran, most recently first; + * the layer is decided by `parent_session_id` and by nothing else; + * the pool is the whole walk, not the thirty most recent sessions; + * the runs of a conversation that is NOT on the page travel too, because the pool is the whole + walk and a conversation that has aged out of the twenty still has runs to offer; + * the page follows the list: the conversation pick holds no runs, the run pick holds the runs + of the conversation that was picked and nothing else, it is empty and disabled while the + page follows the newest conversation, and picking a run switches to that run. + +Two probes, because the rules live on two sides of the wire: the layer is the service's, and +the rows the page draws are the page's. A mutation has to be caught by one of them, by name. + +The ones worth reading twice are the third and the fourth. Putting the runs in the other order +still lists every one of them, in a list that looks the same -- only the order is wrong, and +the order is the promise. And judging the layer by `purpose` instead of the parent column gets a +run with no purpose and a main session whose purpose says 「task」 exactly backwards, which is +the whole reason the parent column is the one that decides. + +On the page, the failures worth catching are not strings that survived an edit: they are a run +sitting in the conversation pick, a run pick that is never empty, a run pick that lists somebody +else's runs, and a pick that does not switch. Those are read back out of a rendered picker, +not out of its source. + +Anchors are multi-line and both shipped files are LF, so they are written with bare \\n and read +and written with newline='' -- matching on normalised text while preserving the file's own endings. +No anchor may contain a line ending of its own: one did, and it stopped resolving the moment the +.gitattributes pinning LF landed. +""" +import io +import os +import subprocess +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +PLUGIN = os.path.dirname(HERE) +SERVER = os.path.join(PLUGIN, "miniapp", "node", "server.mjs") +CLIENT = os.path.join(PLUGIN, "miniapp", "client", "index.html") +PROBES = [ + os.path.join(HERE, "verify-session-layers.mjs"), + os.path.join(HERE, "client-check.mjs"), +] + +# Each entry is (title, edits, expect) and optionally a fourth element naming the file to mutate, +# defaulting to the server. +MUTATIONS = [ + # ---- the service ---- + # + # The defect itself: the two layers are one flat list again, and a sub-agent is back in the + # picker wearing a conversation's clothes. The reader is back to 13 rows of somebody else's + # runs, with nothing on screen to say which they are. + ("a sub-agent is put back into the main list", + [(" if (!parent) {\n mains.push(entry);\n continue;\n }", + " mains.push(entry);\n if (!parent) {\n continue;\n }")], + "and it holds no sub-agent at all"), + + # The list is not a window, so a sub-agent that fell out of the thirty most recent sessions + # is still a sub-agent the reader has to be able to open. The pool is the whole walk, and + # the walk has already stat'ed every candidate, so the deeper pool costs no extra disk work. + ("the pool is cut back to the thirty most recent sessions", + [(" const allEntries = candidates.allEntries;\n", + " const allEntries = candidates.recentEntries;\n")], + "the payload sends every run in the pool"), + + # The order, inverted. Every run is still listed, and the list still looks the same -- + # only the order is wrong, and the order is the promise: the one that ran last is the one a + # reader is most likely to want, and it is the one this buries at the bottom. + ("the runs are ordered the other way round", + [(" for (const bucket of childrenByParent.values()) bucket.sort((a, b) => b.lastActiveAt - a.lastActiveAt);", + " for (const bucket of childrenByParent.values()) bucket.sort((a, b) => a.lastActiveAt - b.lastActiveAt);")], + "the sub-agents of one main session are ordered most recently active first"), + + # The list is not a window, in the place where it is actually cut. This one is invisible to + # a test of the service's own functions: the function that collects the runs is fine, + # and the route drops them on the way out. + ("the runs are cut down to the first three", + [(" for (const child of bucket) childEntries.push(child);", + " for (const child of bucket.slice(0, 3)) childEntries.push(child);")], + "the payload sends every run in the pool"), + + # The runs of a conversation that is not on the page. Dropping them takes away the runs of + # a conversation the reader is reading, the moment that conversation ages out of the twenty + # -- and the run pick of that conversation has nothing to offer. + ("the runs of a conversation that is not on the page are dropped", + [(" for (const bucket of layers.childrenByParent.values()) {", + " for (const [parentId, bucket] of layers.childrenByParent.entries()) {\n" + " const onThePage = visible.some((entry) => decodeSessionIdFromDirName(basename(entry.dir)) === parentId);\n" + " if (!onThePage) continue;")], + "and the runs of a conversation that is not on the page are sent too"), + + # The layer judged by the wrong column. `purpose` says what a session IS -- a scheduled run, a + # background task -- which is a different question from what it hangs off, and the two are + # orthogonal. This mutation gets a sub-agent with no purpose and a main session whose purpose + # says 「task」 exactly backwards, which is why the parent column is the one that decides. + ("the layer is judged by the purpose column instead of the parent column", + [(" const parent = meta && typeof meta.parent === 'string' && meta.parent ? meta.parent : null;", + " const parent = meta && typeof meta.purpose === 'string' && meta.purpose ? 'mvs_purpose_guessed_parent' : null;")], + "a main session whose purpose says 「task」 is still a main session"), + + # ---- the page ---- + # + # The two picks. The conversation pick holds conversations; the run pick holds the runs of + # the conversation that was picked. What the reader gets is read back out of a rendered + # picker, so the failures worth catching are the ones a string would not see. + # + # A run in the conversation pick. That is the defect this whole feature exists for: a run + # wearing a conversation's clothes, thirteen of twenty rows on this machine. + ("the conversation pick lists the runs too", + [(" var option = el('option', null, sessionOptionLabel(session, now));\n" + " option.value = session.id;\n" + " select.appendChild(option);", + " var option = el('option', null, sessionOptionLabel(session, now));\n" + " option.value = session.id;\n" + " select.appendChild(option);\n" + " var alsoRuns = childrenByParent()[session.id] || [];\n" + " for (var q = 0; q < alsoRuns.length; q += 1) {\n" + " var runOption = el('option', null, sessionOptionLabel(alsoRuns[q], now));\n" + " runOption.value = alsoRuns[q].id;\n" + " select.appendChild(runOption);\n" + " }")], + "the conversation pick holds no run at all", + CLIENT), + + # The run pick is never empty. It looks like a simplification: there is always a + # conversation on screen, so there is always something to offer. But while the page follows + # the newest conversation, that conversation changes under the reader. + ("the run pick is never empty", + [(" var owner = state.followLatest ? '' : ownerSessionId();", + " var owner = ownerSessionId();")], + "the run pick is empty while the page follows the newest conversation", + CLIENT), + + # The same empty state, but offered rather than withheld: a control that says 「pick one of + # these」 while the conversation it would name is about to be replaced. + ("the run pick is not disabled while the page follows", + [(" select.disabled = true;", + " select.disabled = false;")], + "and it is disabled, so there is nothing to pick", + CLIENT), + + # The run pick lists everybody's runs. Every run is still a run and the order still looks + # right -- but the rows are no longer about the conversation on screen, and the reader cannot + # tell which of them are its own. + ("the run pick lists the runs of every conversation", + [(" var kids = childrenByParent()[owner] || [];", + " var kids = state.children || [];")], + "the run pick holds every run of the conversation on screen", + CLIENT), + + # The order, on the page this time. The service sends the runs most recently active first, + # and a page that sorts them again can only get it wrong. + ("the page reorders the runs it was given", + [(" for (var j = 0; j < kids.length; j += 1) {", + " for (var j = kids.length - 1; j >= 0; j -= 1) {")], + "and they keep the order the service sent", + CLIENT), + + # The row that says 「the conversation itself」. Without it there is no way back: the reader + # picks a run, and the conversation is gone from the controls. + ("the run pick has no row for the conversation itself", + [(" var self = el('option', null, '（主会话本身）');\n" + " self.value = '';\n" + " select.appendChild(self);\n", + "")], + "and the first option is the conversation itself", + CLIENT), + + # The owner. A run belongs to a conversation, and the conversation pick says which one; a + # version that keeps the run on the conversation pick shows two different answers to the + # same question. + ("the owner of a run is the run itself", + [(" if (child && child.id === id && child.sessionParentId) return child.sessionParentId;", + " if (child && child.id === id && child.sessionParentId) return id;")], + "the owner of a run is the conversation it hangs off", + CLIENT), + + # The press. A promise about a press is not kept by a row that merely carries the right id: + # the page has to switch to the run the reader pressed, and this is the one that does not. + ("picking a run does not switch to it", + [(" selectSession(value);\n }\n\n /* ---------------- wiring ---------------- */", + " selectSession(ownerSessionId() || '');\n }\n\n /* ---------------- wiring ---------------- */")], + "picking a run switches the page to that run", + CLIENT), + + # The conversation pick that does not move. The run pick keeps its run, the conversation + # pick keeps the conversation, and the two disagree about what is on screen. + ("picking a conversation does not move the page", + [(" selectSession(value);\n }\n\n /**\n * The reader picked a run, or the conversation itself.", + " selectSession(state.sessionId || value);\n }\n\n /**\n * The reader picked a run, or the conversation itself.")], + "picking another conversation moves the run pick with it", + CLIENT), +] + +ORIGINALS = { + SERVER: io.open(SERVER, encoding="utf-8", newline="").read(), + CLIENT: io.open(CLIENT, encoding="utf-8", newline="").read(), +} + + +def run(): + code = 0 + chunks = [] + for probe in PROBES: + proc = subprocess.run(["node", probe], cwd=HERE, + capture_output=True, text=True, encoding="utf-8", errors="replace") + if code == 0: + code = proc.returncode + chunks.append(proc.stdout + proc.stderr) + return code, "\n".join(chunks) + + +ok = True +code, baseline = run() +base_ok = code == 0 and "0 failed" in baseline +print(("ok " if base_ok else "FAIL ") + "baseline green") +ok = ok and base_ok + +caught = [] +for entry in MUTATIONS: + title, edits, expect = entry[0], entry[1], entry[2] + target = entry[3] if len(entry) > 3 else SERVER + original = ORIGINALS[target] + for old, new in edits: + hits = original.count(old) + if hits != 1: + raise SystemExit("FAIL anchor for %r matched %d times" % (title, hits)) + if old == new: + raise SystemExit("FAIL empty mutation for %r: the anchor equals its replacement" % title) + try: + mutated = original + for old, new in edits: + mutated = mutated.replace(old, new, 1) + if mutated == original: + raise SystemExit("FAIL mutation for %r changed nothing" % title) + io.open(target, "w", encoding="utf-8", newline="").write(mutated) + during_code, during = run() + named = expect in during + red = during_code != 0 + # A guard that crashes is not a guard that caught anything: the rule has to be reported by + # name, or the mutation only proved that the file no longer parses. + print(("ok " if (named and red) else "FAIL ") + "guard catches: " + title) + if not named: + print(" no assertion reported %r" % expect) + for line in during.splitlines()[-6:]: + print(" " + line.strip()) + elif not red: + print(" the rule was reported but the guard still exited 0") + caught.append(named and red) + finally: + io.open(target, "w", encoding="utf-8", newline="").write(original) + +restored = all(io.open(path, encoding="utf-8", newline="").read() == text + for path, text in ORIGINALS.items()) +after_code, after = run() +after_ok = after_code == 0 and "0 failed" in after +print(("ok " if restored else "FAIL ") + "both files restored byte for byte") +print(("ok " if after_ok else "FAIL ") + "green again after restore") + +sys.exit(0 if (ok and all(caught) and restored and after_ok) else 1) diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs new file mode 100644 index 00000000..c47ff523 --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -0,0 +1,468 @@ +/** + * The picker is two controls: the upper one names the conversation, the lower one names the run + * inside it. + * + * A sub-agent is a session like any other on disk — its own directory, its own `messages.jsonl` — + * and the picker used to list it exactly like a conversation of the reader's own. On the machine + * this was measured on, 13 of the 20 rows in the picker were sub-agent runs of the one + * conversation being read. Nothing said so: a reader comparing two entries could not tell a + * sub-agent run from their own work, and the runs were the actual content of the conversation. + * + * So they are not hidden either, and they are not rows of the conversation list either — that + * list is the one a reader scrolls to find a conversation, and folding nineteen runs into it made + * it forty rows long. The conversation pick holds conversations; the run pick holds the runs of + * the conversation that was picked, and it is empty and disabled while the page follows the + * newest conversation. The service sends the conversations and every run in the pool, and every + * run row names the conversation it hangs off. The page groups them by that name; it does not + * keep a hierarchy of its own. + * + * It used to be a second list under the picker. That was one question answered in two places: + * the dropdown named conversations, the list named their runs, and nothing said whose runs they + * were. + * + * Four rules, each with the reason it exists: + * + * 1. The conversation list holds conversations and nothing else. A sub-agent in it is a + * sub-agent wearing a conversation's clothes, which is the defect. + * 2. A conversation's runs are all of them. The list is not a window: a run that exists but is + * not on screen is a run the reader cannot open. + * 3. They are ordered by when each one last ran, most recent first — the one that ran last is + * the one a reader is most likely to want, and it is the one the other order buries. + * 4. The layer is decided by `parent_session_id` and by nothing else. `purpose` says what a + * session IS (a scheduled run, a background task), which is a different question from what + * it hangs off, and the two are orthogonal: a sub-agent is a task AND somebody's child. + * A judge that used `purpose` would call a scheduled run a sub-agent, and a sub-agent + * spawned by something other than a task a main session. + * + * The fourth rule is the one that decides the other three, so the fixture below contains a row + * that is a child with no purpose, and a row that is a main session whose purpose says 「task」. + * The two columns disagree, and the layer follows the parent. + * + * The pool is the whole walk, not the thirty most recent sessions. That is not a detail: a + * sub-agent is always newer than the session that spawned it, so the thirty-entry window is + * mostly sub-agents, and drawing the conversation list from it would push conversations out of + * their own list. The walk has already stat'ed every candidate, so the deeper pool costs no extra + * disk work — and a run that fell out of the window is still a run the reader has to be able to + * open. + * + * The sections are checked against real stores and real runtime databases built here, not + * against a copy of the logic. Each section runs inside its own try, and a section that throws + * reports ITSELF rather than borrowing the name of the first assertion in it: a section that + * threw has not judged anything, and a mutation that dies of an exception has to say which rule + * it broke. + * + * `tests/mutate-session-layers.py` breaks each of them in the shipped source and requires this + * guard to report it by name. + */ +import { mkdirSync, mkdtempSync, readFileSync } from 'node:fs'; +import { mkdir, utimes, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { basename, join } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +const SERVER = fileURLToPath(new URL('../miniapp/node/server.mjs', import.meta.url)); +/** The page, read for the fields it takes out of the list payload. */ +const CLIENT = fileURLToPath(new URL('../miniapp/client/index.html', import.meta.url)); +/** The package root, so the route serves the same page the reader gets. */ +const PLUGIN_ROOT = fileURLToPath(new URL('..', import.meta.url)); + +// `node:sqlite` is still flagged experimental, and the flag goes to stderr the moment the module +// is first loaded. The runner reads `stdout + stderr` and prints the LAST non-empty line, so the +// warning would land where this guard's own result line should be. +const emitWarning = process.emitWarning; +process.emitWarning = (warning, ...rest) => ( + String(warning).startsWith('SQLite is an experimental feature') + ? undefined + : emitWarning.call(process, warning, ...rest)); + +const { DatabaseSync } = await import('node:sqlite'); +const { + collectSessionCandidates, + decodeSessionIdFromDirName, + readSessionTitles, + sessionsValidator, + splitSessionLayers, +} = await import(pathToFileURL(SERVER).href); + +let pass = 0; +const failures = []; +function assert(name, cond, detail) { + if (cond) { pass += 1; return; } + failures.push(detail ? name + ' -- ' + detail : name); +} + +/* ---------- the fixtures: a store and a runtime database, built here ---------- */ + +/** A session id, distinct enough that a wrong one cannot pass for another. */ +const MAIN = (n) => 'mvs_layer_main' + String(n).padStart(2, '0') + 'zzzz'; +const KID = (n) => 'mvs_layer_kid_' + String(n).padStart(2, '0') + 'zzzz'; + +/** A minute apart, so the recency order is unambiguous. */ +const BASE_TIME = Date.UTC(2026, 9, 20, 0, 0, 0); + +/** How many main sessions, and how many sub-agents hang off the first two of them. */ +const MAIN_COUNT = 25; +const KIDS_OF_MAIN_0 = 13; +const KIDS_OF_MAIN_1 = 6; +// A main session that is NOT on the page: it is the twenty-third of twenty-five, so the twenty +// the picker shows do not include it. Its runs still have to travel, because the picker keeps a +// conversation alive with a row of its own once it ages out of the list. +const KIDS_OF_MAIN_22 = 2; + +/** + * One session directory under the store, with a controlled mtime. + * @param {string} sessionsRoot + * @param {number} index + * @param {string} id + * @param {number} mtimeMs + */ +async function makeSession(sessionsRoot, index, id, mtimeMs) { + const name = String(index).padStart(2, '0') + '-00-00-session_' + + Buffer.from(id, 'utf8').toString('base64url'); + const dir = join(sessionsRoot, '2026', '10', '20', name); + await mkdir(dir, { recursive: true }); + await writeFile(join(dir, 'messages.jsonl'), + JSON.stringify({ role: 'user', content: 'x' }) + '\n', 'utf8'); + const stamp = new Date(mtimeMs); + await utimes(join(dir, 'messages.jsonl'), stamp, stamp); + return { id, dir, relative: '2026/10/20/' + name }; +} + +/** + * A runtime database with the columns the layer is decided by. + * + * `parent_session_id` and `purpose` are written so that they DISAGREE on purpose: one sub-agent + * has no purpose at all, and one main session has a purpose that says 「task」. A judge that read + * the purpose column would move both of them, and the guard below would see it. + * + * @param {string} home + * @param {readonly { sessionId: string, kind?: string, purpose?: string | null, parent?: string | null, relative?: string | null, workspace?: string | null, updatedAtMs?: number }[]} rows + */ +function runtimeDb(home, rows) { + mkdirSync(join(home, 'v2', 'sqlite'), { recursive: true }); + const db = new DatabaseSync(join(home, 'v2', 'sqlite', 'runtime-state.sqlite')); + db.exec('create table local_runtime_sessions (session_id text, title text, status text, ' + + 'history_relative_dir text, session_kind text, archived integer, updated_at_ms integer, ' + + 'parent_session_id text, purpose text, workspace_dir text)'); + db.exec('create table local_runtime_session_locks (session_id text, expires_at_ms integer)'); + const insert = db.prepare('insert into local_runtime_sessions (session_id, title, status, ' + + 'history_relative_dir, session_kind, archived, updated_at_ms, parent_session_id, purpose, ' + + 'workspace_dir) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'); + for (const row of rows) { + insert.run(row.sessionId, '会话 ' + row.sessionId.slice(-4), 'idle', row.relative ?? null, + row.kind ?? 'conversation', 0, row.updatedAtMs ?? 1, row.parent ?? null, row.purpose ?? null, + row.workspace ?? null); + } + db.close(); +} + +/** + * The store, the database and the layers the service builds from them. + * + * @returns {Promise<{ home: string, sessionsRoot: string, made: any[], rows: any[], candidates: any, titles: Map, layers: any }>} + */ +async function fixture() { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-layers-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const made = []; + const rows = []; + let index = 0; + // The main sessions are the newest: they are what a reader is looking at, and a sub-agent is + // always newer than the session that spawned it, so this is the order the real store has. + for (let i = 0; i < MAIN_COUNT; i += 1) { + const id = MAIN(i); + const dir = await makeSession(sessionsRoot, index, id, BASE_TIME - i * 60_000); + index += 1; + made.push(dir); + rows.push({ + sessionId: id, + relative: dir.relative, + workspace: 'workspace-one', + updatedAtMs: BASE_TIME - i * 60_000, + // The main session that LOOKS like a task. It is not one: it has no parent, so it is a + // conversation, and a judge that read the purpose column would have moved it. + purpose: i === MAIN_COUNT - 1 ? 'local-background-task:looks-like-a-task' : null, + kind: i === MAIN_COUNT - 1 ? 'task' : 'conversation', + }); + } + // The sub-agents of the first main session are the OLDEST of all, so thirteen of them sit + // outside the thirty most recent sessions. A pool of thirty would have lost them. + for (let i = 0; i < KIDS_OF_MAIN_0; i += 1) { + const id = KID(i); + const dir = await makeSession(sessionsRoot, index, id, BASE_TIME - (60 + i) * 60_000); + index += 1; + made.push(dir); + rows.push({ + sessionId: id, + relative: dir.relative, + parent: MAIN(0), + kind: 'task', + // The first sub-agent has NO purpose at all, and it is still a sub-agent. + purpose: i === 0 ? null : 'local-background-task:sub-agent', + }); + } + // The sub-agents of the second main session are the newest of the sub-agents, and six of them + // is the number this machine has. + for (let i = 0; i < KIDS_OF_MAIN_1; i += 1) { + const id = KID(KIDS_OF_MAIN_0 + i); + const dir = await makeSession(sessionsRoot, index, id, BASE_TIME - (40 + i) * 60_000); + index += 1; + made.push(dir); + rows.push({ + sessionId: id, + relative: dir.relative, + parent: MAIN(1), + kind: 'task', + purpose: 'local-background-task:sub-agent', + }); + } + // The runs of the main session that is not on the page. + for (let i = 0; i < KIDS_OF_MAIN_22; i += 1) { + const id = KID(KIDS_OF_MAIN_0 + KIDS_OF_MAIN_1 + i); + const dir = await makeSession(sessionsRoot, index, id, BASE_TIME - (200 + i) * 60_000); + index += 1; + made.push(dir); + rows.push({ + sessionId: id, + relative: dir.relative, + parent: MAIN(22), + kind: 'task', + purpose: 'local-background-task:sub-agent', + }); + } + runtimeDb(home, rows); + const candidates = await collectSessionCandidates(sessionsRoot); + const ids = candidates.allEntries.map((entry) => decodeSessionIdFromDirName(basename(entry.dir))).filter(Boolean); + const titles = await readSessionTitles(home, ids); + const layers = splitSessionLayers(candidates.allEntries, titles); + return { home, sessionsRoot, made, rows, candidates, titles, layers }; +} + +const idsOf = (entries) => entries.map((entry) => decodeSessionIdFromDirName(basename(entry.dir))); +const idsOfBucket = (bucket) => (bucket || []).map((entry) => decodeSessionIdFromDirName(basename(entry.dir))); + +/* ---------- 1: the main list holds main sessions and nothing else ---------- */ + +let world = null; +try { + world = await fixture(); + const mainIds = idsOf(world.layers.mains); + assert('the main list holds every main session', + mainIds.length === MAIN_COUNT, + mainIds.length + ' of ' + MAIN_COUNT); + assert('and it holds no sub-agent at all', + !mainIds.some((id) => KIDS_OF_MAIN_0 > 0 && /^mvs_layer_kid_/.test(id)), + mainIds.filter((id) => /^mvs_layer_kid_/.test(id)).join('|')); + assert('and no main session is missing from it', + MAIN_COUNT === [...Array(MAIN_COUNT).keys()].map(MAIN).filter((id) => mainIds.includes(id)).length, + mainIds.join('|')); +} catch (error) { + failures.push('the main list holds main sessions and nothing else -- ' + (error && error.message)); +} + +/* ---------- 2: every sub-agent is there, and the list is not a window ---------- */ + +try { + const kidIds = [...world.layers.childrenByParent.keys()].flatMap((parent) => idsOfBucket(world.layers.childrenByParent.get(parent))); + assert('every sub-agent is in the sub-agent list', + kidIds.length === KIDS_OF_MAIN_0 + KIDS_OF_MAIN_1 + KIDS_OF_MAIN_22, + kidIds.length + ' of ' + (KIDS_OF_MAIN_0 + KIDS_OF_MAIN_1 + KIDS_OF_MAIN_22)); + assert('and none of them is a main session', + !kidIds.some((id) => /^mvs_layer_main/.test(id)), + kidIds.filter((id) => /^mvs_layer_main/.test(id)).join('|')); + // The pool is the whole walk, not the thirty most recent. Thirteen of these sub-agents are + // older than every main session, so a pool of thirty would have lost all of them. + const recentIds = idsOf(world.candidates.recentEntries); + assert('and the pool is the whole walk, not the thirty most recent sessions', + world.candidates.allEntries.length === MAIN_COUNT + KIDS_OF_MAIN_0 + KIDS_OF_MAIN_1 + KIDS_OF_MAIN_22 + && recentIds.length < world.candidates.allEntries.length, + world.candidates.allEntries.length + ' all / ' + recentIds.length + ' recent'); + const outsideTheWindow = idsOfBucket(world.layers.childrenByParent.get(MAIN(0))); + assert('so a sub-agent older than the thirty most recent is still listed', + outsideTheWindow.length === KIDS_OF_MAIN_0 + && outsideTheWindow.every((id) => !recentIds.includes(id)), + outsideTheWindow.length + ' of ' + KIDS_OF_MAIN_0 + ', ' + + outsideTheWindow.filter((id) => recentIds.includes(id)).length + ' inside the window'); +} catch (error) { + failures.push('every sub-agent is there, and the list is not a window -- ' + (error && error.message)); +} + +/* ---------- 3: the sub-agents are ordered by when each one last ran ---------- */ + +try { + for (const [parent, bucket] of world.layers.childrenByParent.entries()) { + const times = bucket.map((entry) => entry.lastActiveAt); + const sorted = times.every((value, i) => i === 0 || times[i - 1] >= value); + assert('the sub-agents of one main session are ordered most recently active first', + sorted, + parent + ': ' + times.join(',')); + } + const first = world.layers.childrenByParent.get(MAIN(0)) || []; + // The fixture makes the sub-agents older the higher their number, so the one at the top is + // the first one made. + const newestKid = KID(0); + assert('and the one that ran last is the one at the top', + first.length === KIDS_OF_MAIN_0 + && decodeSessionIdFromDirName(basename(first[0].dir)) === newestKid, + first.map((entry) => decodeSessionIdFromDirName(basename(entry.dir))).join('|')); +} catch (error) { + failures.push('the sub-agents are ordered by when each one last ran -- ' + (error && error.message)); +} + +/* ---------- 4: the layer is the parent column, and the purpose column is not ---------- */ + +try { + const mainIds = idsOf(world.layers.mains); + const kidsOfMain0 = idsOfBucket(world.layers.childrenByParent.get(MAIN(0))); + // The main session whose purpose says 「task」: no parent, so it is a conversation. + const looksLikeATask = MAIN(MAIN_COUNT - 1); + assert('a main session whose purpose says 「task」 is still a main session', + mainIds.includes(looksLikeATask) && !world.layers.childrenByParent.has(looksLikeATask), + mainIds.includes(looksLikeATask) ? 'in the main list' : 'missing'); + // The sub-agent with no purpose at all: a parent, so it is a sub-agent. + const purposeLess = KID(0); + assert('a sub-agent with no purpose at all is still a sub-agent', + kidsOfMain0.includes(purposeLess) && !mainIds.includes(purposeLess), + kidsOfMain0.includes(purposeLess) ? 'in the sub-agent list' : 'missing'); + // And the sub-agents hang off the main session they name — the join key is one id, the same + // id the main row carries. + assert('and each sub-agent hangs off the main session it names', + [...world.layers.childrenByParent.keys()].every((parent) => /^mvs_layer_main/.test(parent)), + [...world.layers.childrenByParent.keys()].join('|')); +} catch (error) { + failures.push('the layer is the parent column, and the purpose column is not -- ' + (error && error.message)); +} + +/* ---------- 5: the validator moves when a sub-agent does ---------- */ + +try { + const visible = world.layers.mains.slice(0, 20); + const kids = world.layers.childrenByParent.get(MAIN(0)) || []; + const before = sessionsValidator(visible, world.titles, 20, kids); + // A sub-agent that starts: one more row, and the page has never seen it. + const oneMore = kids.concat([{ dir: kids[0].dir + '-new', sizeBytes: 10, lastActiveAt: 1 }]); + assert('a sub-agent that starts moves the validator', + sessionsValidator(visible, world.titles, 20, oneMore) !== before); + // A sub-agent that runs again: same row, newer file. + const rerun = kids.map((entry, i) => (i === 0 ? { ...entry, lastActiveAt: entry.lastActiveAt + 1 } : entry)); + assert('a sub-agent that runs again moves the validator', + sessionsValidator(visible, world.titles, 20, rerun) !== before); + // The parent id is IN the material: the same sub-agent under a different main session is a + // different page, and a material that did not say so would let one answer for the other. + assert('and the material carries the parent a sub-agent hangs off', + before.includes(MAIN(0)), + before.slice(0, 200)); + // A validator that never saw the sub-agents cannot move when they do — which is exactly why + // the route has to pass them in. + assert('a validator built without the sub-agents does not move when one does', + sessionsValidator(visible, world.titles, 20) === sessionsValidator(visible, world.titles, 20)); +} catch (error) { + failures.push('the validator moves when a sub-agent does -- ' + (error && error.message)); +} + +/* ---------- 6: the payload the page receives ---------- */ + +// The sections above test the service's own functions. This one tests the BYTES: the route is +// started against the same fixture store, and the body it answers with is read back. +// +// It is here because the last rule is not a rule about a function. "A main session's sub-agents +// are all of them" is a promise about what the page receives, and the place a sub-agent could +// get lost is the route that collects them -- a cap there would show up in a function-level test +// as nothing at all. + +try { + const { start } = await import(pathToFileURL(SERVER).href); + const previousRoot = process.env.MMC_TRAJECTORY_ROOT; + process.env.MMC_TRAJECTORY_ROOT = world.home; + const controller = new AbortController(); + const noop = () => {}; + let base = null; + let instance = null; + for (let attempt = 0; attempt < 12 && !base; attempt += 1) { + const port = 19400 + Math.floor(Math.random() * 400) + attempt; + try { + instance = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + dataDir: PLUGIN_ROOT, + generationId: 'guard', + listen: { host: '127.0.0.1', port }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop } + }); + base = 'http://127.0.0.1:' + port; + } catch (error) { + // The port was taken. Another one is tried; a guard that gives up here would be a guard + // that sometimes checks nothing. + } + } + if (!base) throw new Error('could not bind a port for the route'); + const response = await fetch(base + '/api/sessions?limit=20'); + const body = await response.json(); + const mainIds = new Set(idsOf(world.layers.mains)); + const kidIds = new Set([...world.layers.childrenByParent.keys()].flatMap( + (parent) => idsOfBucket(world.layers.childrenByParent.get(parent)))); + const sessionIds = (body.sessions || []).map((row) => row.id); + const childIds = (body.children || []).map((row) => row.id); + + assert('the payload sends twenty main sessions', + sessionIds.length === 20, String(sessionIds.length)); + assert('and not one of them is a sub-agent', + !sessionIds.some((id) => kidIds.has(id)), + sessionIds.filter((id) => kidIds.has(id)).join('|')); + assert('and every one of them is a main session', + sessionIds.every((id) => mainIds.has(id)), + sessionIds.filter((id) => !mainIds.has(id)).join('|')); + assert('the payload sends every run in the pool', + childIds.length === KIDS_OF_MAIN_0 + KIDS_OF_MAIN_1 + KIDS_OF_MAIN_22, + childIds.length + ' of ' + (KIDS_OF_MAIN_0 + KIDS_OF_MAIN_1 + KIDS_OF_MAIN_22)); + assert('and each of them names the conversation it hangs off', + (body.children || []).every((row) => typeof row.sessionParentId === 'string' && row.sessionParentId), + (body.children || []).filter((row) => !row.sessionParentId).map((row) => row.id).join('|')); + // The runs of a conversation that is NOT on the page travel too. The pool is the whole + // walk, and a reader reading a conversation that has aged out of the twenty still has to be + // able to reach its runs -- so a service that sent only the runs of the visible conversations + // would take them away from a reader who is reading one. + const runsOfTheHidden = (body.children || []) + .filter((row) => row.sessionParentId === MAIN(22)).map((row) => row.id); + assert('and the runs of a conversation that is not on the page are sent too', + runsOfTheHidden.length === KIDS_OF_MAIN_22, + runsOfTheHidden.length + ' of ' + KIDS_OF_MAIN_22); + // The order inside one group: most recently active first, which is the promise the run pick + // makes about the rows it draws. + const kidsOfMain0 = (body.children || []).filter((row) => row.sessionParentId === MAIN(0)).map((row) => row.id); + assert('and the sub-agents of one conversation arrive most recently active first', + kidsOfMain0.join(',') === idsOfBucket(world.layers.childrenByParent.get(MAIN(0))).join(','), + kidsOfMain0.join('|')); + // The payload carries only what the page reads. `currentSessionId` used to be here for a + // second list under the picker; that list is gone, and a field nobody reads is a field every + // reader is sent for nothing. The set of "what the page reads" is read out of the page's own + // sessions fetch rather than written out here, because a copy of it in this file would be a + // second answer to the same question. + const clientSource = readFileSync(CLIENT, 'utf8'); + const sessionsFetch = /function fetchSessions\(manual\) \{[\s\S]*?\n \}/.exec(clientSource); + const readKeys = new Set(sessionsFetch + ? [...sessionsFetch[0].matchAll(/body\.([A-Za-z_$][\w$]*)/g)].map((match) => match[1]) + : []); + const unread = Object.keys(body).filter((key) => !readKeys.has(key)); + assert('and the payload carries no field the page does not read', + sessionsFetch !== null && unread.length === 0, + unread.join('|')); + // The validator: an unchanged list answers 304, and it is the list that moved. + const etag = response.headers.get('etag'); + const again = await fetch(base + '/api/sessions?limit=20', { headers: { 'If-None-Match': etag || '' } }); + assert('and an unchanged list answers 304', + again.status === 304, String(again.status)); + if (instance) await instance.dispose(); + controller.abort(); + if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; + else process.env.MMC_TRAJECTORY_ROOT = previousRoot; +} catch (error) { + failures.push('the payload the page receives -- ' + (error && error.message)); +} + +console.log(pass + ' assertions passed, ' + failures.length + ' failed'); +if (failures.length) { + console.log('FAILURES:\n - ' + failures.join('\n - ')); + process.exit(1); +} From 3249425c81318f31732025810417cdbfebf03ed9 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 10:40:44 +0800 Subject: [PATCH 06/95] A run that won the current conversation was promoted into the conversation list The picker has two layers now, and the layer is decided by the parent column and by nothing else. The service splits the pool into conversations and runs, sends both, and the page puts each row in the control that says what it is. The route did not keep that promise on the way out. It splits, and then it pins the conversation the runtime database reports as current into the visible set, and that pinning step never asked what the current session was. A run with no purpose of its own, holding a live lease, could win the current-conversation resolution; the pinning step then built its entry from the recorded path and prepended it to the conversation list. The same session sat in both controls at once, which is the defect the two layers exist to prevent. The pinning step asks now. A session with a parent is a run, and a run is not promoted into the conversation layer -- it is already in the run layer, where it belongs. The parent id comes from the same titles map the split reads, so there is one judgement of the layer and one source for it. The current-conversation resolution itself is untouched. Its answer feeds the runtime diagnostic, the session list, and the trajectory route's 'latest' fallback, and narrowing it there would move 'latest' as well. The question here is not whether the current conversation may be a run -- it may -- but whether a run may be promoted into the conversation list. It may not. The second defect is on the page, and it is about order. The session list is polled and every poll replaces the one before it. A replaced request cleared the shared controller on its way out, so the request that replaced it had nothing to abort; the request before that answered last and won, and the picker moved backwards under the reader, runs and all. Two rules now, each with one job. The controller says which request to stop, and only the request that installed a controller may clear it. A sequence number says which answer may write, and only the newest request writes. The trajectory poll and the search already work this way; the session list was the last one without it. An aborted request is also no longer swallowed into "the list failed" -- being replaced is not a failure. --- .../mmc-trajectory/miniapp/client/index.html | 25 +++++++++++-- .../mmc-trajectory/miniapp/node/server.mjs | 36 ++++++++++++++----- 2 files changed, 49 insertions(+), 12 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 47b7f462..95112cbd 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -3917,6 +3917,9 @@

检视

var dom = {}; var inflight = null; var sessionsInflight = null; + // The sequence that decides which of several list requests is allowed to answer. The + // controller above says which request to stop; this says which answer may write. + var sessionsSeq = 0; var sessionPollTimer = null; var trajectoryPollTimer = null; var searchTimer = null; @@ -4047,6 +4050,7 @@

检视

if (sessionsInflight) sessionsInflight.abort(); var controller = new AbortController(); sessionsInflight = controller; + var seq = (sessionsSeq += 1); var key = 'limit=20'; var query = '?' + key; var headers = { Accept: 'application/json' }; @@ -4076,14 +4080,26 @@

检视

return response .json() - .catch(function () { return null; }) + .catch(function (error) { + // An aborted request is not a broken answer. Swallowing it here reported a list + // that failed for a request that was simply replaced by a newer one. + if (error && error.name === 'AbortError') throw error; + return null; + }) .then(function (body) { if (!body || typeof body !== 'object') throw new Error('api'); return { changed: true, body: body, etag: response.headers.get('etag') || '' }; }); }) .then(function (result) { - sessionsInflight = null; + // The controller this request installed is the only one it may clear. A request that + // has been replaced must not take the next request's controller with it: that is how + // the next request stopped being able to abort this one, and an older answer ended + // up replacing a newer list. + if (sessionsInflight === controller) sessionsInflight = null; + // A late answer must not win. Every request carries a sequence number, and only the + // newest one may write -- the same rule the trajectory poll follows. + if (seq !== sessionsSeq) return false; // One writer for the list's validator, on the one path that carried a body. state.sessionsEtag = result.etag; state.sessionsEtagFor = key; @@ -4108,7 +4124,10 @@

检视

return true; }) .catch(function (error) { - sessionsInflight = null; + if (sessionsInflight === controller) sessionsInflight = null; + // A request that has been replaced says nothing about the list on screen: the + // request that replaced it is the one that gets to report a failure. + if (seq !== sessionsSeq) return false; if (error && error.name === 'AbortError') return false; noteFresh('sessions', false, freshnessReason(error)); renderBars(); diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 05c1ffc3..6a059274 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -1237,9 +1237,16 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSe * updated), and `resolveCurrentConversation` reports the ambiguity when two conversations hold * a lease at once rather than picking one silently. * + * The current conversation is promoted only when it IS a conversation. A session with a + * parent is a run, and the layer is decided by that column and by nothing else -- so a run + * that wins the current-conversation resolution is not promoted back into the conversation + * layer. It is already in the run layer, where it belongs, and promoting it here would put + * one session in both controls at once: the conversation pick would offer a run, and the + * run pick would offer the same run again. + * * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries * @param {{ sessionId: string, relativeDir?: string | null } | null} current - * @param {Map} titles + * @param {Map} titles * @param {string} sessionsRoot * @param {number} limit * @param {() => void} [onReject] @@ -1253,14 +1260,25 @@ export async function buildVisibleSessionList(entries, current, titles, sessions return entries.slice(0, limit); } const currentId = current.sessionId; - let currentEntry = entries.find((entry) => decodeSessionIdFromDirName(basename(entry.dir)) === currentId) || null; - if (!currentEntry && current.relativeDir) { - // Not in the recent window -- build the entry straight from the recorded path. The session it - // is supposed to be is named, so a path that turns out to be a different session's directory - // is refused instead of being shown under this id. - const built = await entryFromRelativeDir(sessionsRoot, current.relativeDir, currentId); - if (built.rejected && typeof onReject === 'function') onReject(); - currentEntry = built.entry; + // The layer is decided by the parent column and by nothing else, and the picker has two of + // them. A session that has a parent is a run, so a run that wins the current-conversation + // resolution is not promoted into the conversation layer. Promoting it would put one + // session in both controls at once. + const currentMeta = titles instanceof Map ? titles.get(currentId) : null; + const currentParent = currentMeta && typeof currentMeta.parent === 'string' && currentMeta.parent + ? currentMeta.parent + : null; + let currentEntry = null; + if (!currentParent) { + currentEntry = entries.find((entry) => decodeSessionIdFromDirName(basename(entry.dir)) === currentId) || null; + if (!currentEntry && current.relativeDir) { + // Not in the recent window -- build the entry straight from the recorded path. The session it + // is supposed to be is named, so a path that turns out to be a different session's directory + // is refused instead of being shown under this id. + const built = await entryFromRelativeDir(sessionsRoot, current.relativeDir, currentId); + if (built.rejected && typeof onReject === 'function') onReject(); + currentEntry = built.entry; + } } const currentWorkspace = workspaceOfSession(titles, currentId); const rest = entries.filter((entry) => entry !== currentEntry); From e985d9e20f9bd7f28220224d124e86849307e9a9 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 10:40:55 +0800 Subject: [PATCH 07/95] The two layers are guarded where the promise is kept, not where it is easiest to test The layer rule was guarded by a unit test of the split, and the split was not where it had been broken. The route splits the pool and then pins the current conversation into the visible set; the pinning step is the one that put a run back into the conversation list, and nothing drove it. A guard that tests the function that happens to be most like the rule is not a guard of the rule. So the new section drives the route itself. The fixture is built for the case: one conversation, and one run with a parent, no purpose of its own, and a live lease -- the strongest signal the resolver has, so it wins the current conversation outright. The body the service answers with is read back, and the run must be absent from the conversation list, present in the run list, and absent from both at once. The page's defect had the same shape: nobody drove three overlapping list requests, so nobody noticed that the replaced one took the controller with it. The new section drives exactly that -- a controllable fetch, three requests overlapping, the middle one answering last with the older body -- and reads back what the page ends up showing. The second request must have been stopped by the third, and the newer list must still be on screen. Both come with mutations that break the rule in the shipped source and require the guard to report it by name. There are three: the pinning step loses the parent judgement, the controller check becomes unconditional again, and the sequence check is dropped. The last two are separate on purpose -- each leaves the other defect in place, so one mutation cannot stand in for the pair. --- .../mmc-trajectory/tests/client-check.mjs | 80 ++++++++++++++ .../tests/mutate-session-layers.py | 41 +++++++ .../tests/verify-session-layers.mjs | 102 +++++++++++++++++- 3 files changed, 222 insertions(+), 1 deletion(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index f973e268..2640b08f 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -1474,6 +1474,86 @@ if (layerSrc.every((src) => src)) { runs.value === '', String(runs.value)); } +/* ------------------------------------------------------------------ session list: overlapping polls */ +// The list is polled, and every poll replaces the one before it. Two things have to hold +// while that happens, and both used to be false: +// +// * a request that has been replaced must not take the next request's controller with it; +// * an answer that arrives late must not replace a newer list. +// +// The first is what made the second possible: the replaced request cleared the shared +// controller on its way out, so the request that replaced it had nothing to abort, and the +// request before that answered last and won. +// +// So this section drives three overlapping requests against a controllable fetch, and +// reads back what the page ends up showing. It is the transition, not the shape of the +// source: a regex would pass for a version that carries the sequence number and ignores it. + +const fetchSessionsSrc = fnSource('function fetchSessions(manual)'); +const inflightDecl = /var sessionsInflight = null;/.exec(html); +const seqDecl = /var sessionsSeq = 0;/.exec(html); +if (!fetchSessionsSrc || !inflightDecl || !seqDecl) { + failures.push('session list: could not lift fetchSessions and the state it keeps'); +} else { + vm.runInContext(inflightDecl[0] + '\n' + seqDecl[0] + '\n' + fetchSessionsSrc, sandbox); + + // A fetch the section drives: every request gets a promise the section settles, and the + // signal is the real one, so an abort is an abort. + const requests = []; + sandbox.fetch = function (url, options) { + const entry = { url: url, signal: options && options.signal, settle: null, fail: null }; + entry.promise = new Promise((resolve, reject) => { + entry.settle = resolve; + entry.fail = reject; + }); + requests.push(entry); + return entry.promise; + }; + sandbox.AbortController = AbortController; + sandbox.state = { + sessions: [], children: [], sessionsEtag: '', sessionsEtagFor: '', sessionsLoaded: false + }; + sandbox.noteFresh = () => false; + sandbox.renderBars = () => {}; + sandbox.freshnessReason = () => 'unknown'; + sandbox.renderSessionSelect = () => {}; + sandbox.renderChildSelect = () => {}; + + const abortError = () => { + const error = new Error('The operation was aborted.'); + error.name = 'AbortError'; + return error; + }; + const answer = (entry, body) => entry.settle({ + status: 200, + ok: true, + headers: { get: () => '' }, + json: () => Promise.resolve(body) + }); + + // Three requests: the first is replaced by the second, the second is replaced by the + // third. The second's answer arrives last, and it is the older one. + const first = sandbox.fetchSessions(false); + const second = sandbox.fetchSessions(false); + check('the second request stops the first', requests[0].signal.aborted, true); + // The browser delivers an abort as a rejection, and that rejection is handled while the + // second request is already in flight -- which is exactly when the controller used to be + // cleared. + requests[0].fail(abortError()); + await first; + const third = sandbox.fetchSessions(false); + check('a replaced request does not take the controller of the request that replaced it', + requests[1].signal.aborted, true); + answer(requests[2], { sessions: [{ id: 'mvs_new_main' }], children: [{ id: 'mvs_new_run' }] }); + await third; + answer(requests[1], { sessions: [{ id: 'mvs_old_main' }], children: [{ id: 'mvs_old_run' }] }); + await second; + check('a late answer does not replace the newer list on screen', + sandbox.state.sessions.map((row) => row.id), ['mvs_new_main']); + check('and the run list is the newer one too', + sandbox.state.children.map((row) => row.id), ['mvs_new_run']); +} + console.log('[2] pure helpers: ' + pass + ' passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py index be05167b..d161e622 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -113,6 +113,17 @@ " const parent = meta && typeof meta.purpose === 'string' && meta.purpose ? 'mvs_purpose_guessed_parent' : null;")], "a main session whose purpose says 「task」 is still a main session"), + # The promotion step forgot the layer. The split puts a run in the run layer, and the + # route then promotes whatever the runtime database reports as current -- so a run that + # won that resolution was put back into the conversation list, and the same session sat + # in both controls at once. + ("a run that wins the current-conversation resolution is promoted into the conversation list", + [(" const currentParent = currentMeta && typeof currentMeta.parent === 'string' && currentMeta.parent\n" + " ? currentMeta.parent\n" + " : null;", + " const currentParent = null;")], + "a run that wins the current-conversation resolution is not in the conversation list"), + # ---- the page ---- # # The two picks. The conversation pick holds conversations; the run pick holds the runs of @@ -198,6 +209,36 @@ "picking a run switches the page to that run", CLIENT), + # The replaced request takes the controller with it. The controller says which request + # to stop, and a request that has been replaced has no business stopping the one that + # replaced it -- so the next request had nothing to abort, and the request before it + # answered last and won. + ("a replaced request takes the controller of the request that replaced it", + [(" if (sessionsInflight === controller) sessionsInflight = null;\n" + " // A late answer must not win. Every request carries a sequence number, and only the\n", + " sessionsInflight = null;\n" + " // A late answer must not win. Every request carries a sequence number, and only the\n"), + (" if (sessionsInflight === controller) sessionsInflight = null;\n" + " // A request that has been replaced says nothing about the list on screen: the\n", + " sessionsInflight = null;\n" + " // A request that has been replaced says nothing about the list on screen: the\n")], + "a replaced request does not take the controller of the request that replaced it", + CLIENT), + + # The late answer is allowed to write. The sequence number is the rule that says which + # answer may write, and a version that carries the number and ignores it is the version + # where the older list replaces the newer one -- a picker that moves backwards under + # the reader. + ("a late answer is allowed to replace a newer list", + [(" if (seq !== sessionsSeq) return false;\n" + " // One writer for the list's validator, on the one path that carried a body.\n", + " // One writer for the list's validator, on the one path that carried a body.\n"), + (" if (seq !== sessionsSeq) return false;\n" + " if (error && error.name === 'AbortError') return false;\n", + " if (error && error.name === 'AbortError') return false;\n")], + "a late answer does not replace the newer list on screen", + CLIENT), + # The conversation pick that does not move. The run pick keeps its run, the conversation # pick keeps the conversation, and the two disagree about what is on screen. ("picking a conversation does not move the page", diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs index c47ff523..8579b558 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -137,8 +137,9 @@ async function makeSession(sessionsRoot, index, id, mtimeMs) { * * @param {string} home * @param {readonly { sessionId: string, kind?: string, purpose?: string | null, parent?: string | null, relative?: string | null, workspace?: string | null, updatedAtMs?: number }[]} rows + * @param {readonly string[]} [leases] sessions holding a live lease */ -function runtimeDb(home, rows) { +function runtimeDb(home, rows, leases) { mkdirSync(join(home, 'v2', 'sqlite'), { recursive: true }); const db = new DatabaseSync(join(home, 'v2', 'sqlite', 'runtime-state.sqlite')); db.exec('create table local_runtime_sessions (session_id text, title text, status text, ' @@ -153,6 +154,13 @@ function runtimeDb(home, rows) { row.kind ?? 'conversation', 0, row.updatedAtMs ?? 1, row.parent ?? null, row.purpose ?? null, row.workspace ?? null); } + // A live lease is the strongest signal the current-conversation resolver has: it says an + // agent turn is running in that session right now. The fixture in section seven needs + // one, because the defect that section covers is a session that WINS that resolution. + if (leases) { + const lock = db.prepare('insert into local_runtime_session_locks (session_id, expires_at_ms) values (?, ?)'); + for (const id of leases) lock.run(id, Date.now() + 600_000); + } db.close(); } @@ -461,6 +469,98 @@ try { failures.push('the payload the page receives -- ' + (error && error.message)); } +/* ---------- 7: a run that wins the current-conversation resolution ---------- */ + +// The rule the sections above state is about a function: `splitSessionLayers` puts a run +// in the run layer. The route does not call it and stop -- it splits, and then it promotes +// the conversation the runtime database reports as current into the list the page draws. +// A run that won that resolution was therefore promoted back into the conversation list, +// and the same session sat in both controls at once. +// +// So this section is driven over HTTP, against a store built for exactly that case: one +// conversation, and one run that has a parent, no purpose at all, and a live lease -- the +// strongest signal the resolver has, so it wins the resolution outright. + +const PROMOTE_PARENT = 'mvs_promote_parent'; +const PROMOTE_RUN = 'mvs_promote_run'; + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-promotion-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const parent = await makeSession(sessionsRoot, 0, PROMOTE_PARENT, BASE_TIME - 10 * 60_000); + const run = await makeSession(sessionsRoot, 1, PROMOTE_RUN, BASE_TIME); + runtimeDb(home, [ + { + sessionId: PROMOTE_PARENT, + relative: parent.relative, + workspace: 'workspace-one', + updatedAtMs: BASE_TIME - 10 * 60_000 + }, + { + sessionId: PROMOTE_RUN, + relative: run.relative, + parent: PROMOTE_PARENT, + // A conversation by kind, with no purpose at all: nothing but the parent column says + // this is a run, which is the whole point of the rule. + kind: 'conversation', + purpose: null, + workspace: 'workspace-one', + updatedAtMs: BASE_TIME + } + ], [PROMOTE_RUN]); + + const { start } = await import(pathToFileURL(SERVER).href); + const previousRoot = process.env.MMC_TRAJECTORY_ROOT; + process.env.MMC_TRAJECTORY_ROOT = home; + const controller = new AbortController(); + const noop = () => {}; + let base = null; + let instance = null; + for (let attempt = 0; attempt < 12 && !base; attempt += 1) { + const port = 19700 + Math.floor(Math.random() * 300) + attempt; + try { + instance = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + dataDir: PLUGIN_ROOT, + generationId: 'guard', + listen: { host: '127.0.0.1', port }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop } + }); + base = 'http://127.0.0.1:' + port; + } catch (error) { + // The port was taken. Another one is tried; a guard that gave up here would be a + // guard that sometimes checks nothing. + } + } + if (!base) throw new Error('could not bind a port for the promotion route'); + const response = await fetch(base + '/api/sessions?limit=20'); + const body = await response.json(); + const sessionIds = (body.sessions || []).map((row) => row.id); + const childIds = (body.children || []).map((row) => row.id); + + assert('a run that wins the current-conversation resolution is not in the conversation list', + !sessionIds.includes(PROMOTE_RUN), + sessionIds.join('|')); + assert('and it is in the run list, where it belongs', + childIds.includes(PROMOTE_RUN), + childIds.join('|')); + assert('and the conversation it hangs off is still in the conversation list', + sessionIds.includes(PROMOTE_PARENT), + sessionIds.join('|')); + assert('and no session is in both lists at once', + !sessionIds.some((id) => childIds.includes(id)), + sessionIds.filter((id) => childIds.includes(id)).join('|')); + + if (instance) await instance.dispose(); + controller.abort(); + if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; + else process.env.MMC_TRAJECTORY_ROOT = previousRoot; +} catch (error) { + failures.push('a run that wins the current-conversation resolution -- ' + (error && error.message)); +} + console.log(pass + ' assertions passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); From bbe2dc9a1a894ff1390c5f34e3c16f11a9b08630 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 12:00:23 +0800 Subject: [PATCH 08/95] A parent id from the runtime database could break the session picker The page groups the runs it is sent by the parent the service named, and it grouped them into a plain object. The parent id is data that crossed the wire from the runtime database, and a plain object inherits names like '__proto__' and 'constructor': they are truthy and they are not arrays, so one of them was read as a group that already existed, and the run was then pushed into it. The list response was perfectly fine; the picker stopped updating after it. The map has no prototype now, so no value can be read as a group that is not one. The service checks the other end of the same boundary. `parent_session_id` is a foreign key into the runtime database's own session table, and a value that names no session there is not a parent: it was published as one, and the page groups by it. The check lives where the column is read, so the body and the validator -- which read the same map -- agree about it without a second writer. A value that names no session is not published, and the session it was on is a conversation rather than a run: it is a real session on disk with its own transcript, and dropping it would hide a record the reader can open. The id grammar the service applies to the ids it derives from directory names would not have done it: that grammar lets '__proto__' and 'constructor' through, so a value like those would still have been published. --- .../mmc-trajectory/miniapp/client/index.html | 7 +++++- .../mmc-trajectory/miniapp/node/server.mjs | 22 +++++++++++++++++++ 2 files changed, 28 insertions(+), 1 deletion(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 95112cbd..12b396f5 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -4582,7 +4582,12 @@

检视

* @returns {Object} */ function childrenByParent() { - var grouped = {}; + // A map with no prototype. The parent id is data that crossed the wire from the + // runtime database, and the names a plain object inherits -- '__proto__', + // 'constructor' -- are truthy and are not arrays, so a plain object reads one of them + // as a group and then throws on it, and the picker stops updating after an otherwise + // successful list. + var grouped = Object.create(null); var all = state.children || []; for (var i = 0; i < all.length; i += 1) { var child = all[i]; diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 6a059274..f21cbd80 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -1088,6 +1088,28 @@ export async function readSessionTitles(home, sessionIds) { workspace: typeof row.workspace_dir === 'string' && row.workspace_dir.trim() ? row.workspace_dir.trim() : null, }); } + // The parent column is a foreign key into this very table: it names the session that + // spawned this one. A value that names no session here is not a parent, and publishing it + // would put a dangling link on the wire -- and the page groups its runs by that value, so + // a name that is not a session's id is a name the page has no business keying a group by. + // + // One indexed lookup per distinct parent, on a table that is already open. A database + // written before child sessions existed has no `parent_session_id` column and so no parent + // to check: every value above is null and this is skipped. + const parentIds = new Set(); + for (const meta of result.values()) if (meta.parent) parentIds.add(meta.parent); + if (parentIds.size > 0) { + try { + const lookup = db.prepare('select session_id from local_runtime_sessions where session_id = ?'); + for (const parentId of parentIds) { + if (lookup.get(parentId)) continue; + for (const meta of result.values()) if (meta.parent === parentId) meta.parent = null; + } + } catch { + // The check could not be made. Publishing the parents as they are is what this did + // before, and the page is written to survive a parent it does not recognise. + } + } } catch { // A title is a nicety; never let it fail the list. } finally { From 2066c91a99bf5e33fbfed651ed32bac078b6519c Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 12:00:23 +0800 Subject: [PATCH 09/95] The two ends of the parent-id boundary are guarded where they are kept The service's rule is about what the page receives, so the guard for it drives the route: the fixture holds two rows whose parent names no session -- one with an id that looks like a session id and does not exist, one named after an object property -- and the body is read back to say that neither is published as a parent, and that the sessions they were on are conversations. The page's rule is about a value the page has no business assuming, so the guard for it drives the shipped function with those same names on the page: the grouping must not throw, the runs must not be folded into a conversation that owns them, and the run pick must still draw. Each comes with a mutation that breaks its rule in the shipped source and requires the guard to report it by name. They are separate on purpose: a service that published the value would still break a page that grouped into a plain object, and a page that grouped into a plain object would break even on a value the service had checked. The guard for the column ladder had a fixture the runtime cannot write. It built a run whose parent named a session the database had no row for, and read the parent back as though that were ordinary -- so once the service started refusing such a parent, four of that guard's assertions went red for a reason that had nothing to do with the rule they were written for. It writes the parent row now, in the order the runtime writes them, and a second fixture builds the run whose parent names nothing: that is the case the service has to refuse. The paired mutation control gains one control for it, and a service that published such a parent has to be caught by name. --- .../mmc-trajectory/tests/client-check.mjs | 42 +++++++ .../tests/mutate-list-cache-ttl.py | 10 ++ .../tests/mutate-session-layers.py | 24 ++++ .../tests/verify-list-cache.mjs | 61 ++++++++-- .../tests/verify-session-layers.mjs | 104 ++++++++++++++++++ 5 files changed, 230 insertions(+), 11 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index 2640b08f..7306db4a 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -1472,6 +1472,48 @@ if (layerSrc.every((src) => src)) { runs.options.map((option) => option.value).join(' | ')); assert('and clears the run that was picked', runs.value === '', String(runs.value)); + + // 8. A parent id is data, not a key the page may assume. The runtime database's parent + // column crosses the wire unchanged, and the page groups its runs by that value. The + // names a plain object inherits -- '__proto__', 'constructor' -- are truthy and are not + // arrays, so a version that groups into a plain object reads one of them as a group and + // throws on it, and the picker stops updating after an otherwise successful list. + const hostile = ['__proto__', 'constructor', 'toString', 'valueOf']; + const realState = sandbox.state; + sandbox.state = { + sessions: [{ id: MAIN_A, label: '主会话甲', lastActiveAt: NOW, messageCount: 10 }], + children: hostile.map((parentId, index) => ({ + id: 'mvs_hostile_' + index, + sessionParentId: parentId, + label: '异常父 ' + index, + lastActiveAt: NOW - index, + messageCount: 1 + })), + followLatest: false, + sessionId: MAIN_A, + hostSessionId: null + }; + let grouped = null; + let groupingError = null; + try { + grouped = sandbox.childrenByParent(); + } catch (error) { + groupingError = error; + } + check('a parent id that names an inherited object property does not break the grouping', + groupingError === null && grouped !== null, true); + check('and those runs are not folded into a conversation that owns them', + grouped !== null + && Object.keys(grouped).every((key) => hostile.indexOf(key) >= 0) + && hostile.every((key) => !grouped[key] || grouped[key].length === 1), true); + let renderError = null; + try { + sandbox.renderChildSelect(); + } catch (error) { + renderError = error; + } + check('and the run pick still draws with such a parent on the page', renderError === null, true); + sandbox.state = realState; } /* ------------------------------------------------------------------ session list: overlapping polls */ diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py index 91d4488d..05141247 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py @@ -197,6 +197,16 @@ [(" sessionParentId: value.sessionParentId ?? null,", "")], "the field survives the strip, with null as the floor"), + # The parent column is a foreign key into the runtime database's own session table, and the + # page groups its runs by that name -- so a value that names no session there is a name the + # list must not publish. No regex over the source can catch this one: the query ran, it + # found nothing, and the only thing that shows it is a database built to have that hole. + ("a parent that names no session is published as a parent anyway", + [(" if (lookup.get(parentId)) continue;\n" + " for (const meta of result.values()) if (meta.parent === parentId) meta.parent = null;\n", + " if (lookup.get(parentId)) continue;\n")], + "a parent that names no session is not published as a parent"), + # The direction that looks right and is not: the parent field is read, but not as the key # the rows are grouped under. Every conversation then hangs the same sub-agents, and the run # pick of every conversation lists the same thirteen of them -- so a reader picks a run that diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py index d161e622..8f3007fd 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -17,6 +17,10 @@ * a main session's runs are all of them -- the list is not a window; * they are ordered by when each one last ran, most recently first; * the layer is decided by `parent_session_id` and by nothing else; + * a parent that names no session is not a parent, so it is not published and the run it was + on is a conversation; + * the page groups its runs into a map with no prototype, so a parent id that names an + inherited object property cannot break the grouping; * the pool is the whole walk, not the thirty most recent sessions; * the runs of a conversation that is NOT on the page travel too, because the pool is the whole walk and a conversation that has aged out of the twenty still has runs to offer; @@ -124,6 +128,16 @@ " const currentParent = null;")], "a run that wins the current-conversation resolution is not in the conversation list"), + # The parent that names nothing. `parent_session_id` is a foreign key into the runtime + # database's own session table; a value that names no session there is not a parent, and + # the page groups its runs by that value -- so publishing it puts a name on the wire that + # no conversation in the list can answer to. + ("a parent that names no session is published as a parent anyway", + [(" if (lookup.get(parentId)) continue;\n" + " for (const meta of result.values()) if (meta.parent === parentId) meta.parent = null;\n", + " if (lookup.get(parentId)) continue;\n")], + "a run whose parent names no session is not published as a run"), + # ---- the page ---- # # The two picks. The conversation pick holds conversations; the run pick holds the runs of @@ -201,6 +215,16 @@ "the owner of a run is the conversation it hangs off", CLIENT), + # The map the runs are grouped into. A plain object inherits names like '__proto__' and + # 'constructor': they are truthy and they are not arrays, so a plain object reads one of + # them as a group and then throws on it -- and the picker stops updating after a list + # response that was perfectly fine. + ("the runs are grouped into a plain object", + [(" var grouped = Object.create(null);", + " var grouped = {};")], + "a parent id that names an inherited object property does not break the grouping", + CLIENT), + # The press. A promise about a press is not kept by a row that merely carries the right id: # the page has to switch to the run the reader pressed, and this is the one that does not. ("picking a run does not switch to it", diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs index 84e2f798..fcf5fa1c 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs @@ -284,11 +284,23 @@ const { DatabaseSync } = await import('node:sqlite'); // with an unsupported scheme. The module has to be asked for by its file: URL. const { readSessionTitles, sessionTagFor } = await import(pathToFileURL(SERVER).href); +const CHILD_ID = 'mvs_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'; +const PARENT_ID = 'mvs_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'; + /** * One runtime database, with the optional columns the caller asks for. + * + * Two rows, and the order they are written in is the point. A run names the conversation it was + * spawned from, and the runtime writes that conversation first -- so a database in which a run + * names a session that is not there is a database the runtime cannot write. It is a separate + * fixture below, because it is the one case where the parent column says something the service + * has to refuse. + * * @param {readonly string[]} extraColumns + * @param {{ parentExists?: boolean }} [options] */ -const runtimeDb = (extraColumns) => { +const runtimeDb = (extraColumns, options) => { + const opts = options || {}; const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-runtimedb-')); mkdirSync(join(home, 'v2', 'sqlite'), { recursive: true }); const db = new DatabaseSync(join(home, 'v2', 'sqlite', 'runtime-state.sqlite')); @@ -296,36 +308,50 @@ const runtimeDb = (extraColumns) => { const placeholders = extraColumns.length ? ', ' + extraColumns.map(() => '?').join(', ') : ''; db.exec('create table local_runtime_sessions (session_id text primary key, title text,' + ' session_kind text' + extra + ')'); - const values = ['mvs_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', '一条真实的标题', 'conversation']; - for (const column of extraColumns) { - if (column === 'parent_session_id') values.push('mvs_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'); - else if (column === 'purpose') values.push('context-mode:lightweight'); - else if (column === 'workspace_dir') values.push('workspace-of-the-session'); - else throw new Error('unknown column ' + column); + const insert = db.prepare('insert into local_runtime_sessions (session_id, title, session_kind' + + extra + ') values (?, ?, ?' + placeholders + ')'); + const row = (id, title, parent, purpose, workspace) => { + const values = [id, title, 'conversation']; + for (const column of extraColumns) { + if (column === 'parent_session_id') values.push(parent); + else if (column === 'purpose') values.push(purpose); + else if (column === 'workspace_dir') values.push(workspace); + else throw new Error('unknown column ' + column); + } + return values; + }; + // The conversation the run hangs off, written before the run -- the order the runtime writes + // them in, and the reason the parent column can be read as a foreign key. + if (opts.parentExists !== false) { + insert.run(...row(PARENT_ID, '父会话的标题', null, null, null)); } - db.prepare('insert into local_runtime_sessions (session_id, title, session_kind' - + extra + ') values (?, ?, ?' + placeholders + ')').run(...values); + insert.run(...row(CHILD_ID, '一条真实的标题', PARENT_ID, 'context-mode:lightweight', 'workspace-of-the-session')); db.close(); return home; }; -const ids = ['mvs_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa']; +const ids = [CHILD_ID]; const homes = []; let parentOnly = null; let baseOnly = null; let noWorkspace = null; let everything = null; +let dangling = null; try { homes.push( runtimeDb(['parent_session_id']), runtimeDb([]), runtimeDb(['parent_session_id', 'purpose']), - runtimeDb(['parent_session_id', 'purpose', 'workspace_dir']) + runtimeDb(['parent_session_id', 'purpose', 'workspace_dir']), + // A run whose parent names a session that is not there: the only database here that the + // runtime cannot write, and the only one where the parent column has something to refuse. + runtimeDb(['parent_session_id'], { parentExists: false }) ); parentOnly = await readSessionTitles(homes[0], ids); baseOnly = await readSessionTitles(homes[1], ids); noWorkspace = await readSessionTitles(homes[2], ids); everything = await readSessionTitles(homes[3], ids); + dangling = await readSessionTitles(homes[4], ids); } finally { // The homes are `mkdtemp` directories this call created, so they are cleaned up in the same // breath rather than left behind for the OS to find. @@ -362,6 +388,19 @@ assert('a database with every column reports the workspace as well', && everything.get(ids[0])?.purpose === 'context-mode:lightweight' && everything.get(ids[0])?.workspace === 'workspace-of-the-session'); +// A run whose parent names a session that is not there names nothing. The parent column is a +// foreign key into this very table, so a value with no row behind it is not a conversation -- +// and the page groups its runs by that name, so publishing it puts a name on the wire that no +// conversation in the list can answer to. +// +// The run itself is still a session. It is a real one, with a real title, and dropping it from +// the list would hide a session the reader could otherwise open. +assert('a parent that names no session is not published as a parent', + dangling.get(ids[0])?.parent === null); +assert('a run whose parent names no session is still a session', + dangling.get(ids[0])?.title === '一条真实的标题' + && dangling.get(ids[0])?.kind === 'conversation'); + /* ---------- 9. what a purpose means, in one place and nowhere else ---------- */ // // The raw purpose is free text — it carries whatever a person typed into it — so it never diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs index 8579b558..3c884075 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -38,6 +38,10 @@ * that is a child with no purpose, and a row that is a main session whose purpose says 「task」. * The two columns disagree, and the layer follows the parent. * + * A parent is a foreign key into the runtime database's own session table: a value that names + * no session there is not a parent, and the page groups its runs by that value, so the service + * publishes a parent only when it names one. The eighth section is about that. + * * The pool is the whole walk, not the thirty most recent sessions. That is not a detail: a * sub-agent is always newer than the session that spawned it, so the thirty-entry window is * mostly sub-agents, and drawing the conversation list from it would push conversations out of @@ -561,6 +565,106 @@ try { failures.push('a run that wins the current-conversation resolution -- ' + (error && error.message)); } +/* ---------- 8: a parent that names no session ---------- */ + +// The parent id crosses the wire from the runtime database, and the page groups its runs by +// it. `parent_session_id` is a foreign key into the runtime database's own session table, so a +// value that names no session there is not a parent at all -- and publishing it would put a +// name on the wire that no conversation in the list can answer to. +// +// The fixture holds two such rows: one whose parent id is a plausible session id that does +// not exist, and one whose parent id is a name a plain object inherits. The second is the one +// that broke the picker, and the first is the ordinary shape of the same mistake. + +const PHANTOM_PARENT = 'mvs_phantom_parent'; +const HOSTILE_PARENT = '__proto__'; + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-phantom-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const main = await makeSession(sessionsRoot, 0, 'mvs_phantom_main', BASE_TIME - 10 * 60_000); + const orphan = await makeSession(sessionsRoot, 1, 'mvs_phantom_child', BASE_TIME - 20 * 60_000); + const hostile = await makeSession(sessionsRoot, 2, 'mvs_hostile_child', BASE_TIME - 30 * 60_000); + runtimeDb(home, [ + { + sessionId: 'mvs_phantom_main', + relative: main.relative, + workspace: 'workspace-one', + updatedAtMs: BASE_TIME - 10 * 60_000 + }, + { + sessionId: 'mvs_phantom_child', + relative: orphan.relative, + parent: PHANTOM_PARENT, + kind: 'task', + purpose: 'local-background-task:orphan', + workspace: 'workspace-one', + updatedAtMs: BASE_TIME - 20 * 60_000 + }, + { + sessionId: 'mvs_hostile_child', + relative: hostile.relative, + parent: HOSTILE_PARENT, + kind: 'task', + purpose: 'local-background-task:hostile', + workspace: 'workspace-one', + updatedAtMs: BASE_TIME - 30 * 60_000 + } + ]); + + const { start } = await import(pathToFileURL(SERVER).href); + const previousRoot = process.env.MMC_TRAJECTORY_ROOT; + process.env.MMC_TRAJECTORY_ROOT = home; + const controller = new AbortController(); + const noop = () => {}; + let base = null; + let instance = null; + for (let attempt = 0; attempt < 12 && !base; attempt += 1) { + const port = 19800 + Math.floor(Math.random() * 300) + attempt; + try { + instance = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + dataDir: PLUGIN_ROOT, + generationId: 'guard', + listen: { host: '127.0.0.1', port }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop } + }); + base = 'http://127.0.0.1:' + port; + } catch (error) { + // The port was taken. Another one is tried. + } + } + if (!base) throw new Error('could not bind a port for the phantom-parent route'); + const response = await fetch(base + '/api/sessions?limit=20'); + const body = await response.json(); + const sessionIds = (body.sessions || []).map((row) => row.id); + const childIds = (body.children || []).map((row) => row.id); + const parentsOnTheWire = (body.sessions || []).concat(body.children || []) + .map((row) => row.sessionParentId).filter(Boolean); + + assert('a run whose parent names no session is not published as a run', + !childIds.includes('mvs_phantom_child'), + childIds.join('|')); + assert('and it is a conversation instead', + sessionIds.includes('mvs_phantom_child'), + sessionIds.join('|')); + assert('and no row on the wire names a parent that does not exist', + !parentsOnTheWire.includes(PHANTOM_PARENT), + parentsOnTheWire.join('|')); + assert('and the same holds for a parent id that names an inherited object property', + !childIds.includes('mvs_hostile_child') && !parentsOnTheWire.includes(HOSTILE_PARENT), + childIds.join('|') + ' / ' + parentsOnTheWire.join('|')); + + if (instance) await instance.dispose(); + controller.abort(); + if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; + else process.env.MMC_TRAJECTORY_ROOT = previousRoot; +} catch (error) { + failures.push('a parent that names no session -- ' + (error && error.message)); +} + console.log(pass + ' assertions passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); From 98c925329b9c43f00dbb6b56a148a051e4e51e2a Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 15:37:36 +0800 Subject: [PATCH 10/95] A late 304 could overwrite the validator the newer answer installed The list is polled, and every poll replaces the one before it. Two rules keep that from moving backwards: a request that has been replaced must not take the next request's controller with it, and only the newest answer may write. The second rule covered the answer that carries a body and not the one that does not. A 304 was writing the list's validator itself, before the sequence gate, so a 304 that arrived late put its own older tag over the newer one -- and the list on screen and the tag that described it were then two different things. The next poll asked the server about the old list, was told the list had not changed, and kept the new one. The 304 branch now writes nothing. It hands the validator back as data, and the one writer -- the one that runs after the sequence gate -- puts it on screen. Both answers go through the same writer now, so there is one place where the list's validator is decided. The service's parent check is closed the same way. `parent_session_id` is a foreign key into the runtime database's own session table, and a value that names no session there is not a parent. The check used to prepare a second query to make that lookup, and the catch around it published the parents it had not managed to check -- so the promise that every published parent names a session held on the ordinary path and not on the other one. The lookup now uses the statement that read the row in the first place, so there is no second query that could fail to prepare, and no path on which an unchecked parent is published. --- .../mmc-trajectory/miniapp/client/index.html | 10 ++-- .../mmc-trajectory/miniapp/node/server.mjs | 52 ++++++++----------- 2 files changed, 30 insertions(+), 32 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 12b396f5..ca2899e7 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -4065,10 +4065,14 @@

检视

// Nothing moved since the copy already on screen. Keep every piece of state and // re-render nothing: the picker, its 「刚刚」 labels and its counts are all still true. if (response.status === 304) { + // The validator is written by the one writer below, and that runs after the + // sequence gate. A request that has been replaced must write nothing at all: a + // 304 that arrived late used to put its own older tag over the newer one, and + // the list on screen and the tag that describes it were then two different + // things -- so the next poll asked the server about the old list and kept the + // new one. var fresh = response.headers.get('etag'); - if (fresh) state.sessionsEtag = fresh; - state.sessionsEtagFor = key; - return { changed: false, body: null, etag: state.sessionsEtag }; + return { changed: false, body: null, etag: fresh || state.sessionsEtag }; } if (!response.ok) { // The status is what tells a dropped connection apart from a server that answered diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index f21cbd80..884cbafb 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -1078,38 +1078,32 @@ export async function readSessionTitles(home, sessionIds) { for (const id of sessionIds) { const row = stmt.get(id); if (!row) continue; + const parent = typeof row.parent_session_id === 'string' && row.parent_session_id !== '' + ? row.parent_session_id + : null; + // The parent column is a foreign key into this very table: it names the session that + // spawned this one. A value that names no session here is not a parent, and publishing it + // would put a dangling link on the wire -- and the page groups its runs by that value, so + // a name that is not a session's id is a name the page has no business keying a group by. + // + // The lookup is the statement that read this very row, so there is no second query that + // could fail to prepare, and therefore no path on which a parent is published without + // having been looked up: a database written before child sessions existed has no + // `parent_session_id` column, so every parent above is null and there is nothing to + // look up at all. + // + // A lookup that throws takes this row out of the map along with everything else, and a + // row that is not in the map has no parent either. So the promise holds on every path: + // a parent that is published names a session. + const parentNames = parent ? Boolean(stmt.get(parent)) : false; result.set(id, { title: typeof row.title === 'string' && row.title.trim() ? row.title.trim() : null, kind: typeof row.session_kind === 'string' ? row.session_kind : null, - parent: typeof row.parent_session_id === 'string' && row.parent_session_id !== '' - ? row.parent_session_id - : null, + parent: parentNames ? parent : null, purpose: typeof row.purpose === 'string' && row.purpose.trim() ? row.purpose.trim() : null, workspace: typeof row.workspace_dir === 'string' && row.workspace_dir.trim() ? row.workspace_dir.trim() : null, }); } - // The parent column is a foreign key into this very table: it names the session that - // spawned this one. A value that names no session here is not a parent, and publishing it - // would put a dangling link on the wire -- and the page groups its runs by that value, so - // a name that is not a session's id is a name the page has no business keying a group by. - // - // One indexed lookup per distinct parent, on a table that is already open. A database - // written before child sessions existed has no `parent_session_id` column and so no parent - // to check: every value above is null and this is skipped. - const parentIds = new Set(); - for (const meta of result.values()) if (meta.parent) parentIds.add(meta.parent); - if (parentIds.size > 0) { - try { - const lookup = db.prepare('select session_id from local_runtime_sessions where session_id = ?'); - for (const parentId of parentIds) { - if (lookup.get(parentId)) continue; - for (const meta of result.values()) if (meta.parent === parentId) meta.parent = null; - } - } catch { - // The check could not be made. Publishing the parents as they are is what this did - // before, and the page is written to survive a parent it does not recognise. - } - } } catch { // A title is a nicety; never let it fail the list. } finally { @@ -1243,10 +1237,10 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSe /** * The visible set the picker shows, and the order it is shown in. * - * Two sets live here and they are not the same set. `entries` is what the walk found: the most - * recent sessions, mtime-descending, and it is also the answer 'latest' falls back to when the - * runtime database is unavailable -- so nothing in here reorders it. The visible set is what the - * READER may pick from, and it is allowed to differ. + * Two sets live here and they are not the same set. `entries` is what the caller handed over: + * the sessions the visible set is drawn from, mtime-descending. The list route hands over the + * walk's conversations -- the runs were split off before this was called, so a run is never in + * here. The visible set is what the READER may pick from, and it is allowed to differ. * * The order is: the conversation the runtime database reports as current, then the sessions of * that conversation's workspace, then everything else -- each group mtime-descending. "The From 7c5da47491b10596046168cf0bbd60c5790a4e56 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 15:37:37 +0800 Subject: [PATCH 11/95] The two rules that were only half-covered now have owners that can fail The 304 case is driven, not read. The overlap section answers one request with a 200 and a newer tag, lets the replaced request answer late with a 304 carrying the old one, and reads back two things: the validator on screen is still the newer one, and the poll after it asks the server about the list that is on screen. A manual refresh is driven too -- it must send no validator at all, because the reader asked for the list again. The parent's promise is driven on both sides. The route is asked over HTTP against a store built for the case, and the answer is read as BYTES: the raw purpose and the workspace path are not in them. The runtime route is asked the same way, against a temporary home, and its bytes carry no absolute path -- the route that promises to publish names and structure only is now asked rather than read. The parent's existence check is pinned at the unit level too, against a database whose run names a session that has no row: the parent is not published, and the run is still a session with its title and its kind. Each of these comes with a mutation that breaks its rule in the shipped source and requires the guard to report it by name. The ones that used to point at the old shape of the check were re-anchored on the new one, because an anchor that no longer resolves is a mutation that no longer proves anything. --- .../mmc-trajectory/tests/client-check.mjs | 53 +++++++++++++- .../tests/mutate-list-cache-ttl.py | 9 +-- .../tests/mutate-runtime-diag.py | 6 ++ .../tests/mutate-session-layers.py | 48 +++++++++++- .../tests/mutate-sessions-etag.py | 11 +-- .../tests/verify-list-cache.mjs | 10 ++- .../tests/verify-runtime-diag.mjs | 73 +++++++++++++++++-- .../tests/verify-session-layers.mjs | 29 ++++++-- 8 files changed, 202 insertions(+), 37 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index 7306db4a..195c7759 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -1543,7 +1543,16 @@ if (!fetchSessionsSrc || !inflightDecl || !seqDecl) { // signal is the real one, so an abort is an abort. const requests = []; sandbox.fetch = function (url, options) { - const entry = { url: url, signal: options && options.signal, settle: null, fail: null }; + const entry = { + url: url, + signal: options && options.signal, + // The headers the page sent, so a section can ask what the request claimed about the + // list it already had. A validator is only ever read by the server, so the only way to + // see whether the page still holds the right one is to look at what it asked with. + headers: (options && options.headers) || {}, + settle: null, + fail: null + }; entry.promise = new Promise((resolve, reject) => { entry.settle = resolve; entry.fail = reject; @@ -1566,10 +1575,10 @@ if (!fetchSessionsSrc || !inflightDecl || !seqDecl) { error.name = 'AbortError'; return error; }; - const answer = (entry, body) => entry.settle({ + const answer = (entry, body, etag) => entry.settle({ status: 200, ok: true, - headers: { get: () => '' }, + headers: { get: (name) => (name === 'etag' ? (etag || '') : '') }, json: () => Promise.resolve(body) }); @@ -1586,7 +1595,7 @@ if (!fetchSessionsSrc || !inflightDecl || !seqDecl) { const third = sandbox.fetchSessions(false); check('a replaced request does not take the controller of the request that replaced it', requests[1].signal.aborted, true); - answer(requests[2], { sessions: [{ id: 'mvs_new_main' }], children: [{ id: 'mvs_new_run' }] }); + answer(requests[2], { sessions: [{ id: 'mvs_new_main' }], children: [{ id: 'mvs_new_run' }] }, '"older"'); await third; answer(requests[1], { sessions: [{ id: 'mvs_old_main' }], children: [{ id: 'mvs_old_run' }] }); await second; @@ -1594,6 +1603,42 @@ if (!fetchSessionsSrc || !inflightDecl || !seqDecl) { sandbox.state.sessions.map((row) => row.id), ['mvs_new_main']); check('and the run list is the newer one too', sandbox.state.children.map((row) => row.id), ['mvs_new_run']); + + // A 304 that arrives late. This is the second case of the same rule, and it is a different + // case: a 200 carries a body, so a stale one has something visible to overwrite. A 304 + // carries a VALIDATOR instead, so a stale one has nothing to overwrite but the tag that + // describes the list on screen -- and that tag is what the next poll asks the server with. + // + // It used to write that tag before the sequence gate, so the list on screen and the tag that + // described it were two different things: the next poll asked the server about the old list, + // the server said "not modified", and the page kept the new one. + const late = sandbox.fetchSessions(false); + const lateReplacement = sandbox.fetchSessions(false); + answer(requests[4], { sessions: [{ id: 'mvs_newest_main' }], children: [] }, '"newest"'); + await lateReplacement; + check('the newer list is on screen', + sandbox.state.sessions.map((row) => row.id), ['mvs_newest_main']); + requests[3].settle({ + status: 304, + ok: false, + headers: { get: (name) => (name === 'etag' ? '"older"' : '') } + }); + await late; + check('a late 304 does not overwrite the validator the newer answer installed', + sandbox.state.sessionsEtag, '"newest"'); + check('and the list on screen is still the newer one', + sandbox.state.sessions.map((row) => row.id), ['mvs_newest_main']); + + // And the poll after it asks the server about the list that is on screen. + const next = sandbox.fetchSessions(false); + check('the next request carries the validator of the list on screen', + requests[5].headers['If-None-Match'], '"newest"'); + + // A manual refresh drops the validator on purpose: the reader asked for the list again, so + // the request must not claim to already have it. + sandbox.fetchSessions(true); + check('a manual refresh sends no validator', + requests[6].headers['If-None-Match'], undefined); } console.log('[2] pure helpers: ' + pass + ' passed, ' + failures.length + ' failed'); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py index 05141247..4dc483ec 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py @@ -182,9 +182,7 @@ "a database WITHOUT the workspace column still reports the title, the kind, the parent and the purpose"), ("the parent row is read but never mapped", - [(" parent: typeof row.parent_session_id === 'string' && row.parent_session_id !== ''\n" - " ? row.parent_session_id\n" - " : null,\n", "")], + [(" parent: parentNames ? parent : null,\n", "")], 'the parent row is mapped, and an empty id is no parent rather than a parent named ""'), ("the descriptor is built without the parent", @@ -202,9 +200,8 @@ # list must not publish. No regex over the source can catch this one: the query ran, it # found nothing, and the only thing that shows it is a database built to have that hole. ("a parent that names no session is published as a parent anyway", - [(" if (lookup.get(parentId)) continue;\n" - " for (const meta of result.values()) if (meta.parent === parentId) meta.parent = null;\n", - " if (lookup.get(parentId)) continue;\n")], + [(" const parentNames = parent ? Boolean(stmt.get(parent)) : false;", + " const parentNames = Boolean(parent);")], "a parent that names no session is not published as a parent"), # The direction that looks right and is not: the parent field is read, but not as the key diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-runtime-diag.py b/plugins/avatasia/mmc-trajectory/tests/mutate-runtime-diag.py index fad404ef..87ebc4cd 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-runtime-diag.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-runtime-diag.py @@ -89,6 +89,12 @@ " dataDirBasename: context.dataDir || null,", "dataDirBasename is a basename and not the value it was derived from", ), + ( + "the working directory is published whole, which is an absolute path too", + " cwdBasename: basename(process.cwd()) || null,", + " cwdBasename: process.cwd(),", + "the runtime route publishes no absolute path", + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py index 8f3007fd..e25174de 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -133,9 +133,8 @@ # the page groups its runs by that value -- so publishing it puts a name on the wire that # no conversation in the list can answer to. ("a parent that names no session is published as a parent anyway", - [(" if (lookup.get(parentId)) continue;\n" - " for (const meta of result.values()) if (meta.parent === parentId) meta.parent = null;\n", - " if (lookup.get(parentId)) continue;\n")], + [(" const parentNames = parent ? Boolean(stmt.get(parent)) : false;", + " const parentNames = Boolean(parent);")], "a run whose parent names no session is not published as a run"), # ---- the page ---- @@ -270,6 +269,49 @@ " selectSession(state.sessionId || value);\n }\n\n /**\n * The reader picked a run, or the conversation itself.")], "picking another conversation moves the run pick with it", CLIENT), + + # A 304 that arrives late. The 200 case is above: a late 200 carries a body, so a stale one + # has something visible to overwrite. A 304 carries a VALIDATOR instead, and a stale one + # used to write that validator before the sequence gate -- so the list on screen and the tag + # that described it were two different things, and the next poll asked the server about the + # old list, was told the list had not changed, and kept the new one. + ("a late 304 writes its own validator over the newer one", + [(" var fresh = response.headers.get('etag');\n" + " return { changed: false, body: null, etag: fresh || state.sessionsEtag };", + " var fresh = response.headers.get('etag');\n" + " if (fresh) state.sessionsEtag = fresh;\n" + " state.sessionsEtagFor = key;\n" + " return { changed: false, body: null, etag: state.sessionsEtag };")], + "a late 304 does not overwrite the validator the newer answer installed", + CLIENT), + + # The manual refresh. The reader asked for the list again, so the request must not claim to + # already have it -- otherwise the server answers 304 and the reader watches nothing change. + ("a manual refresh keeps the validator it was asked to drop", + [(" if (manual) {\n" + " state.sessionsEtag = '';\n" + " state.sessionsEtagFor = '';\n" + " } else if (state.sessionsEtag && state.sessionsEtagFor === key) {", + " if (false) {\n" + " state.sessionsEtag = '';\n" + " state.sessionsEtagFor = '';\n" + " } else if (state.sessionsEtag && state.sessionsEtagFor === key) {")], + "a manual refresh sends no validator", + CLIENT), + + # The raw purpose and the workspace. Both are in the runtime database and neither may cross + # the wire: the purpose is free text a person wrote, and the workspace is a location on the + # reader's own disk. The page gets a token for the first and nothing for the second. + ("the raw purpose and the workspace cross the wire", + [(" descriptor.sessionParentId = meta.parent;", + " descriptor.sessionParentId = meta.parent;\n" + " descriptor.sessionPurpose = meta.purpose;\n" + " descriptor.sessionWorkspace = meta.workspace;"), + (" sessionParentId: value.sessionParentId ?? null,", + " sessionParentId: value.sessionParentId ?? null,\n" + " sessionPurpose: value.sessionPurpose ?? null,\n" + " sessionWorkspace: value.sessionWorkspace ?? null,")], + "the raw purpose does not cross the wire"), ] ORIGINALS = { diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py index c59584f9..8664f73d 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py @@ -123,16 +123,11 @@ # A 304 that re-renders is correct and pointless: the re-render is the cost the 304 exists to # avoid, and on a big list it is the larger half. ("a 304 re-renders the picker anyway", - [(" if (response.status === 304) {\n" + [(" var fresh = response.headers.get('etag');\n" + " return { changed: false, body: null, etag: fresh || state.sessionsEtag };", " var fresh = response.headers.get('etag');\n" - " if (fresh) state.sessionsEtag = fresh;\n" - " state.sessionsEtagFor = key;\n" - " return { changed: false, body: null, etag: state.sessionsEtag };\n" - " }", - " if (response.status === 304) {\n" " renderSessionSelect();\n" - " return { changed: false, body: null, etag: state.sessionsEtag };\n" - " }")], + " return { changed: false, body: null, etag: fresh || state.sessionsEtag };")], "a 304 does not re-render the picker", CLIENT), # Half a validator: kept, never sent. Every poll then carries no header and gets a full body. diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs index fcf5fa1c..fdc7ced2 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs @@ -203,7 +203,15 @@ assert('the titles query reads the parent column', /select session_id, title, session_kind, parent_session_id from local_runtime_sessions where session_id = \?/.test(server)); assert('the parent row is mapped, and an empty id is no parent rather than a parent named ""', - /parent:\s*typeof row\.parent_session_id === 'string' && row\.parent_session_id !== ''\s*\n\s*\?\s*row\.parent_session_id\s*\n\s*:\s*null,/.test(server)); + /const parent = typeof row\.parent_session_id === 'string' && row\.parent_session_id !== ''\s*\n\s*\? row\.parent_session_id\s*\n\s*: null;/.test(server) + && /parent: parentNames \? parent : null,/.test(server)); + +// The parent is a foreign key into the same table, so it is looked up with the statement that +// read the row. There is no second query that could fail to prepare, and therefore no path on +// which a parent is published without having been looked up -- which is what the two database +// fixtures below pin down from the other side. +assert('the parent is published only when the statement that read the row finds one for it', + /const parentNames = parent \? Boolean\(stmt\.get\(parent\)\) : false;/.test(server)); // The same two links for the two newer columns. The purpose is what says a session is a cron // run, a background task or a lightweight exchange, and the workspace is what orders the list — diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-runtime-diag.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-runtime-diag.mjs index ab3f0f2a..bdc655e1 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-runtime-diag.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-runtime-diag.mjs @@ -18,7 +18,7 @@ * this output is the reader who is already having a bad day. On Windows those paths carry the user * profile directory, which is the identity leak this package treats as a release blocker elsewhere. * - * Three kinds of check, each named for what it is, because a guard that does not say which of its + * Four kinds of check, each named for what it is, because a guard that does not say which of its * checks are the strong kind is worse than one that has none: * * 1. BEHAVIOURAL. The redactor is the real exported `createRedactor`, and the text it is handed @@ -28,17 +28,23 @@ * * 2. SHAPE, on three lines, count-checked: that the three fields go through `redact`, and that * they do so BEFORE `slice`. These cannot be behavioural, and the reason is worth recording: - * the handler is a closure inside `start()`, and this suite's own convention - * (`verify-search-endpoint.mjs` states it) is read-only, pure helpers, never start the - * runtime. Moving a route handler out of its closure purely so a test could call it is a - * bigger change to shipped code than the property is worth. So the shape checks hold the - * wiring, the behavioural one holds the property, and neither is asked to do the other's job. + * those three fields only carry text on paths that are hard to induce on purpose — a + * database that refuses to open, a handler that throws — and a route-level test that cannot + * reach them would be a route-level test about the happy path only. Moving a route handler + * out of its closure purely so a test could call it is a bigger change to shipped code than + * the property is worth. So the shape checks hold the wiring, the behavioural one holds the + * property, and neither is asked to do the other's job. * * 3. That the published location fields stay NAMES. `basename` is what turns a path into a fact * about this machine that the browser cannot use to find anything. * - * `tests/mutate-runtime-diag.py` reverts each of the three redacting fields and requires this guard - * to report it by name. + * 4. The route itself, over HTTP, against a temporary home: the service is started and + * `/api/runtime` is requested, and the answer is read as BYTES. A route that assembles its + * answer out of three scrubbed fields can still publish a path from a fourth one, and no + * amount of reading the source says which — the source only says what the fields are named. + * + * `tests/mutate-runtime-diag.py` reverts each of the three redacting fields, and each of the two + * published locations, and requires this guard to report it by name. */ import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { fileURLToPath, pathToFileURL } from 'node:url'; @@ -196,6 +202,57 @@ try { assert('and nothing publishes the sessions root or the data directory verbatim', !/:\s*location\.sessionsRoot\s*[,}]/.test(server) && !/:\s*context\.dataDir\s*[,}]/.test(server)); + + // --- 5. BEHAVIOURAL: the route itself, asked over HTTP ----------------------------------------- + // The scrubber is the property; this is the ROUTE. A route that assembles its answer out of + // three scrubbed fields can still publish a path from a fourth one, and no amount of reading + // the source says which -- the source only says what the fields are named. + // + // So the route is started against a temporary home and asked, and the answer is read as bytes. + const { start } = await import(pathToFileURL(SERVER).href); + const PLUGIN_ROOT = fileURLToPath(new URL('..', import.meta.url)); + const noop = () => {}; + const controller = new AbortController(); + const previousRoot = process.env.MMC_TRAJECTORY_ROOT; + process.env.MMC_TRAJECTORY_ROOT = home; + let instance = null; + let base = null; + for (let attempt = 0; attempt < 12 && !base; attempt += 1) { + const port = 20200 + Math.floor(Math.random() * 300) + attempt; + try { + instance = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + dataDir: PLUGIN_ROOT, + generationId: 'guard', + listen: { host: '127.0.0.1', port }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop } + }); + base = 'http://127.0.0.1:' + port; + } catch (error) { + // The port was taken. Another one is tried. + } + } + try { + if (!base) throw new Error('could not bind a port for the runtime route'); + const response = await fetch(base + '/api/runtime'); + const raw = await response.text(); + assert('the runtime route answers with JSON', + response.status === 200 && raw.trim().startsWith('{'), + raw.slice(0, 200)); + assert('the runtime route publishes no absolute path', + !/[A-Za-z]:[\\/]/.test(raw) && !raw.includes(home), + raw.slice(0, 240)); + assert('and the location it does publish is a name, not the directory it was derived from', + !raw.includes(PLUGIN_ROOT) && !raw.includes(home), + raw.slice(0, 240)); + } finally { + if (instance) await instance.dispose(); + controller.abort(); + if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; + else process.env.MMC_TRAJECTORY_ROOT = previousRoot; + } } finally { rmSync(home, { recursive: true, force: true }); } diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs index 3c884075..c06a5053 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -189,7 +189,7 @@ async function fixture() { rows.push({ sessionId: id, relative: dir.relative, - workspace: 'workspace-one', + workspace: 'D:\\private\\workspace-one', updatedAtMs: BASE_TIME - i * 60_000, // The main session that LOOKS like a task. It is not one: it has no parent, so it is a // conversation, and a judge that read the purpose column would have moved it. @@ -497,7 +497,7 @@ try { { sessionId: PROMOTE_PARENT, relative: parent.relative, - workspace: 'workspace-one', + workspace: 'D:\\private\\workspace-one', updatedAtMs: BASE_TIME - 10 * 60_000 }, { @@ -508,7 +508,7 @@ try { // this is a run, which is the whole point of the rule. kind: 'conversation', purpose: null, - workspace: 'workspace-one', + workspace: 'D:\\private\\workspace-one', updatedAtMs: BASE_TIME } ], [PROMOTE_RUN]); @@ -589,7 +589,7 @@ try { { sessionId: 'mvs_phantom_main', relative: main.relative, - workspace: 'workspace-one', + workspace: 'D:\\private\\workspace-one', updatedAtMs: BASE_TIME - 10 * 60_000 }, { @@ -598,7 +598,7 @@ try { parent: PHANTOM_PARENT, kind: 'task', purpose: 'local-background-task:orphan', - workspace: 'workspace-one', + workspace: 'D:\\private\\workspace-one', updatedAtMs: BASE_TIME - 20 * 60_000 }, { @@ -607,7 +607,7 @@ try { parent: HOSTILE_PARENT, kind: 'task', purpose: 'local-background-task:hostile', - workspace: 'workspace-one', + workspace: 'D:\\private\\workspace-one', updatedAtMs: BASE_TIME - 30 * 60_000 } ]); @@ -638,7 +638,11 @@ try { } if (!base) throw new Error('could not bind a port for the phantom-parent route'); const response = await fetch(base + '/api/sessions?limit=20'); - const body = await response.json(); + // The raw bytes, because what this section is about is what CROSSES the wire. A value that is + // not in the parsed body could still be in the bytes, and a field that is absent from the + // body could still be in a header. + const raw = await response.text(); + const body = JSON.parse(raw); const sessionIds = (body.sessions || []).map((row) => row.id); const childIds = (body.children || []).map((row) => row.id); const parentsOnTheWire = (body.sessions || []).concat(body.children || []) @@ -657,6 +661,17 @@ try { !childIds.includes('mvs_hostile_child') && !parentsOnTheWire.includes(HOSTILE_PARENT), childIds.join('|') + ' / ' + parentsOnTheWire.join('|')); + // The two values that must never cross the wire at all, checked on the bytes rather than on + // the parsed body. The purpose is free text the runtime wrote, and the workspace is a + // location on the reader's own disk; the page gets a token for the first and nothing for the + // second. Both are in this fixture, so both are absent here. + assert('the raw purpose does not cross the wire', + !raw.includes('local-background-task:orphan'), + raw.slice(0, 200)); + assert('and neither does the workspace path', + !raw.includes('D:\\private\\workspace-one') && !raw.includes('D:/private/workspace-one'), + raw.slice(0, 200)); + if (instance) await instance.dispose(); controller.abort(); if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; From 0fc9f6f69c2272470e2f5e15ddb6910830dc38d9 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 16:31:29 +0800 Subject: [PATCH 12/95] A recorded path is checked against the session it names, and a session's layer is decided by the chain of parents it forms Two places on the service read a session's identity and read it differently. The recorded path of the current conversation -- the column the runtime database keeps for it -- was checked against the directory name alone. The directory name is one of the two places the id is written down: the runtime also writes it into the manifest beside the transcript, and the service reads a session's identity from that manifest, preferring it over the name. A directory whose name encodes one session and whose manifest names another is the other session's directory, whatever the folder is called, and accepting it puts the current conversation's id on somebody else's messages. The path is now checked against both: the name has to be the session, and the manifest has to be the session. A directory with no manifest is judged by its name alone, which is the case that was always there. The two layers of the picker -- conversations, and the runs inside one of them -- were decided by the parent column, and the column answers a different question. It says whether a session HAS a parent; the layer is whether that parent is a CONVERSATION, and that is a question about the chain. Three shapes of chain have no conversation at the end of them. A cycle -- one session's parent is another and the other way round -- read the column alone puts both in the run layer, hanging off each other, so neither is a conversation and the conversation list comes back with neither in it: they are in neither control, and the reader can reach neither. A parent with a row in the runtime database but no directory in the pool is a conversation the reader cannot pick, so the run of it is a run nobody can reach. And a run of a run is a run of a run, which neither of the two controls can show. The layer is now decided by the chain. A session is a run iff its parent is a conversation, and a conversation is a session with no parent or one whose parent is not a conversation; the two halves have to be given together, because one half alone has no answer for a cycle. A cycle is dissolved into conversations, a run of a parent that is not in the list becomes a conversation, and so does a run of a run. The map the layer is decided from is corrected in place: a session that ends up in the conversation layer has its parent column cleared. The payload, the validator and the current-conversation promotion all read that one map, so all three now say the same thing about the same row. --- .../mmc-trajectory/miniapp/node/server.mjs | 157 ++++++++++++++++-- 1 file changed, 142 insertions(+), 15 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 884cbafb..8dba7a96 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -1219,11 +1219,30 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSe try { const fileStat = await stat(join(dir, 'messages.jsonl')); if (expectedSessionId) { - // The directory name is the only place the session id is written down, so it is also the - // only place the claim "this path is that session" can be checked. + // The directory name is not the only place the session id is written down, and it is not + // the authority either. The service reads a session's identity from the manifest the + // runtime wrote beside its transcript (`readSessionIdentity`), and the manifest is what + // says which session this directory holds. A directory whose name encodes one session and + // whose manifest names another is the OTHER session's directory, whatever the folder is + // called. + // + // So both have to be the session this path is supposed to be, and they are checked one at + // a time because they can disagree in both directions. The name has to be it: that is the + // check that keeps one directory from being listed twice under two ids. And the manifest + // has to be it too: a manifest naming another session is a recorded path pointing at + // somebody else's transcript, and accepting it would put the current conversation's id on + // another session's messages -- the same defect the name check exists for, one column + // further in. + // + // A directory with no manifest is judged by its name alone, which is the case that was + // always here. if (decodeSessionIdFromDirName(basename(dir)) !== expectedSessionId) { return { entry: null, rejected: true }; } + const identity = await readSessionIdentity(dir); + if (identity.id !== expectedSessionId) { + return { entry: null, rejected: true }; + } } return { entry: { dir, sizeBytes: fileStat.size, lastActiveAt: Math.floor(fileStat.mtimeMs) }, @@ -1253,12 +1272,13 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSe * updated), and `resolveCurrentConversation` reports the ambiguity when two conversations hold * a lease at once rather than picking one silently. * - * The current conversation is promoted only when it IS a conversation. A session with a - * parent is a run, and the layer is decided by that column and by nothing else -- so a run - * that wins the current-conversation resolution is not promoted back into the conversation - * layer. It is already in the run layer, where it belongs, and promoting it here would put - * one session in both controls at once: the conversation pick would offer a run, and the - * run pick would offer the same run again. + * The current conversation is promoted only when it IS a conversation. The layer is decided by + * the parent column and the chain it forms, and `splitSessionLayers` has already cleared the + * parent of every session it put in the conversation layer -- so a session that still has a + * parent here is a run, and a run that wins the current-conversation resolution is not + * promoted back into the conversation layer. It is already in the run layer, where it belongs, + * and promoting it here would put one session in both controls at once: the conversation pick + * would offer a run, and the run pick would offer the same run again. * * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries * @param {{ sessionId: string, relativeDir?: string | null } | null} current @@ -1276,10 +1296,11 @@ export async function buildVisibleSessionList(entries, current, titles, sessions return entries.slice(0, limit); } const currentId = current.sessionId; - // The layer is decided by the parent column and by nothing else, and the picker has two of - // them. A session that has a parent is a run, so a run that wins the current-conversation - // resolution is not promoted into the conversation layer. Promoting it would put one - // session in both controls at once. + // The layer is decided by the parent column and the chain it forms, and the picker has two + // of them. The split has already cleared the parent of every session it put in the + // conversation layer, so a session that still has a parent is a run -- and a run that wins + // the current-conversation resolution is not promoted into the conversation layer. Promoting + // it would put one session in both controls at once. const currentMeta = titles instanceof Map ? titles.get(currentId) : null; const currentParent = currentMeta && typeof currentMeta.parent === 'string' && currentMeta.parent ? currentMeta.parent @@ -1319,7 +1340,7 @@ export async function buildVisibleSessionList(entries, current, titles, sessions * * A sub-agent is a session like any other on disk — its own directory, its own `messages.jsonl` — * and the runtime hangs it off its parent with one column. So the layer is a fact about a row, - * not about a directory, and it is read from that column and nowhere else. `purpose` is not the + * and it is read from that column and from the chain that column forms. `purpose` is not the * judge here: it says what a session IS (a scheduled run, a background task), which is a different * question from what it hangs off, and the two are orthogonal — a sub-agent is a task AND somebody's * child. @@ -1329,6 +1350,33 @@ export async function buildVisibleSessionList(entries, current, titles, sessions * directory name, and that is the id the runtime database stores. So the parent's id here is the * id the page will see on the parent's row, and the client can join the two layers on it. * + * The column alone is not enough to decide the layer, though, and the cases where it is not + * enough are the ones where a session disappears from the picker. The column answers "does this + * session have a parent". The layer is "is that parent a CONVERSATION", and that is a question + * about the chain: + * + * - A cycle — A's parent is B, B's parent is A — has no conversation at the end of it. Read the + * column alone, both of them are runs, of each other, so neither is a conversation, and the + * conversation list comes back with neither in it: they are in neither control, and the + * reader cannot reach either one. A cycle is dissolved instead. Both are conversations, and + * their parent columns are dropped, because a conversation that names a parent names a run. + * - A chain that leaves the sessions this payload knows about — the parent has a row in the + * runtime database but no directory in the pool — has no conversation to point at either: the + * conversation a reader would pick to see this run is not in the list. The run is a + * conversation itself, so that it is somewhere the reader can reach. + * - A chain longer than one step — the parent is itself a run — puts the run in the + * conversation layer, because the two controls are one conversation and the runs of THAT one. + * + * So the rule is one sentence: a session is a run iff its parent is a conversation, and a + * conversation is a session with no parent, or one whose parent is not a conversation. The two + * halves are given together on purpose. One half alone has no answer for a cycle, and the answer + * "the column says so" is the one that loses both sessions. + * + * The map is corrected in place: a session this puts in the conversation layer has its parent + * cleared, so the payload, the validator and the current-conversation promotion all read the same + * value. A copy would leave one of the three reading the uncorrected one, and the body and the + * ETag would then describe two different lists. + * * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries * @param {Map} titles * @returns {{ mains: Array<{ dir: string, sizeBytes: number, lastActiveAt: number }>, childrenByParent: Map> }} @@ -1336,10 +1384,89 @@ export async function buildVisibleSessionList(entries, current, titles, sessions export function splitSessionLayers(entries, titles) { const mains = []; const childrenByParent = new Map(); + const map = titles instanceof Map ? titles : new Map(); + + /** + * The session a session hangs off, or null when it hangs off nothing. + * @param {string} id + * @returns {string | null} + */ + const parentOf = (id) => { + const meta = map.get(id); + if (!meta) return null; + const declared = typeof meta.parent === 'string' && meta.parent ? meta.parent : null; + return declared; + }; + + // The sessions that sit on a cycle: following the parent links from one of them comes back to + // it. Every link is followed through the same map the layer is decided by, so a session on a + // cycle is one this pass reaches -- and the walk below cannot run forever, because it stops at + // the first of these. + const onCycle = new Set(); + for (const id of map.keys()) { + const path = []; + const at = new Map(); + let cur = id; + while (cur && !at.has(cur)) { + at.set(cur, path.length); + path.push(cur); + cur = parentOf(cur); + } + if (cur && at.has(cur)) { + for (let i = at.get(cur); i < path.length; i += 1) onCycle.add(path[i]); + } + } + + // The layer of every session: a conversation, or a run of one. Walking up the chain flips the + // layer at every step, because a run's parent is a conversation and a conversation's parent is + // not one. The walk ends where the chain stops making sense: at a session with no parent, at a + // parent this payload does not know, or at a cycle. + const CONVERSATION = 'conversation'; + const RUN = 'run'; + const layerOf = new Map(); + const flip = (layer) => (layer === CONVERSATION ? RUN : CONVERSATION); + + /** + * @param {string} start + * @returns {string} + */ + const classify = (start) => { + if (layerOf.has(start)) return /** @type {string} */ (layerOf.get(start)); + const chain = []; + let cur = start; + let base; + while (true) { + if (layerOf.has(cur)) { base = flip(/** @type {string} */ (layerOf.get(cur))); break; } + chain.push(cur); + if (onCycle.has(cur)) { base = CONVERSATION; break; } + const parent = parentOf(cur); + if (!parent || !map.has(parent)) { base = CONVERSATION; break; } + cur = parent; + } + let layer = base; + for (let i = chain.length - 1; i >= 0; i -= 1) { + layerOf.set(chain[i], layer); + layer = flip(layer); + } + return /** @type {string} */ (layerOf.get(start)); + }; + + for (const id of map.keys()) classify(id); + + // The correction, and the reason the layer can be read off the parent column again: a session + // in the conversation layer has no parent to name, so its parent column is cleared here, once, + // for everything that reads it afterwards. The body, the validator and the current-conversation + // promotion all read this same map, so all three say the same thing about the same row. + for (const [id, meta] of map) { + if (!meta) continue; + if (layerOf.get(id) === CONVERSATION && typeof meta.parent === 'string' && meta.parent) { + meta.parent = null; + } + } + for (const entry of entries) { const id = decodeSessionIdFromDirName(basename(entry.dir)); - const meta = id ? titles.get(id) : null; - const parent = meta && typeof meta.parent === 'string' && meta.parent ? meta.parent : null; + const parent = id ? parentOf(id) : null; if (!parent) { mains.push(entry); continue; From 5803ae3ae2ba5bf30a92b31cb3447a65da0e56ba Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 16:31:29 +0800 Subject: [PATCH 13/95] The recorded path and the chain of parents now have owners that can fail The recorded path is asked about in three directions, because a name and a manifest can disagree in both directions. A directory whose name is the current session and whose manifest names another one is refused: the manifest is what says which session the directory holds, and the name cannot vouch for it. A directory whose name is another session and whose manifest names the current one is refused too: the walk already lists that directory under its own name, and accepting it would list the same one twice. And a manifest that agrees with the name is accepted -- the third fixture is there so that a check which refuses everything cannot pass as a check. The chain gets a fixture of its own, at the route. Two sessions that name each other as parent, both with a row, are both conversations, neither is a run, and neither names a parent on the wire -- and a run of one of them is still a run of it. Two more shapes are asked at the function: a run whose parent has a row but no directory in the pool, and a run of a run. Both are conversations, and both name no parent, because a conversation that names a parent names a run. The existence check on the parent column keeps a unit-level owner. The chain rule covers it on the wire -- a parent that names no session is not in the list, so the session that named it is a conversation and the column is cleared -- which means the wire looks the same whether that check is there or not. It is therefore read off the map the read produces rather than off the bytes, so that the rule still has something that can fail. Each of these comes with a mutation that breaks its rule in the shipped source and requires the guard to report it by name: the two halves of the identity check, the cycle, the parent that is not in the list, and the run of a run. The two mutations that used to point at the old shape of the parent read were re-anchored on the new one, because an anchor that no longer resolves is a mutation that no longer proves anything. --- .../tests/mutate-session-layers.py | 42 ++- .../tests/mutate-session-picker.py | 30 ++- .../tests/verify-session-layers.mjs | 244 +++++++++++++++++- .../tests/verify-session-picker.mjs | 83 ++++++ 4 files changed, 380 insertions(+), 19 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py index e25174de..c2e5f30e 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -16,7 +16,10 @@ * the main list holds main sessions and nothing else; * a main session's runs are all of them -- the list is not a window; * they are ordered by when each one last ran, most recently first; - * the layer is decided by `parent_session_id` and by nothing else; + * the layer is decided by `parent_session_id` and by the chain it forms, and by nothing else; + * a run's parent is a CONVERSATION, so a cycle is dissolved into two conversations, a run of a + run is a conversation, and a run whose parent is not in the list is a conversation -- all + three because a run the reader cannot reach is a run that is gone; * a parent that names no session is not a parent, so it is not published and the run it was on is a conversation; * the page groups its runs into a map with no prototype, so a parent id that names an @@ -113,9 +116,9 @@ # orthogonal. This mutation gets a sub-agent with no purpose and a main session whose purpose # says 「task」 exactly backwards, which is why the parent column is the one that decides. ("the layer is judged by the purpose column instead of the parent column", - [(" const parent = meta && typeof meta.parent === 'string' && meta.parent ? meta.parent : null;", - " const parent = meta && typeof meta.purpose === 'string' && meta.purpose ? 'mvs_purpose_guessed_parent' : null;")], - "a main session whose purpose says 「task」 is still a main session"), + [(" const declared = typeof meta.parent === 'string' && meta.parent ? meta.parent : null;", + " const declared = typeof meta.purpose === 'string' && meta.purpose ? 'mvs_purpose_guessed_parent' : null;")], + "a sub-agent whose purpose says 「sub-agent」 is still a sub-agent"), # The promotion step forgot the layer. The split puts a run in the run layer, and the # route then promotes whatever the runtime database reports as current -- so a run that @@ -135,7 +138,36 @@ ("a parent that names no session is published as a parent anyway", [(" const parentNames = parent ? Boolean(stmt.get(parent)) : false;", " const parentNames = Boolean(parent);")], - "a run whose parent names no session is not published as a run"), + "and the map itself carries no parent that names no session"), + + # ---- the chain ---- + # + # The parent column says a session HAS a parent. The chain says whether that parent is a + # CONVERSATION, and a run's parent has to be one: a run of a run, or a run of a session that + # is not in the list, is a run the reader cannot reach from either control. + # + # The cycle is the shape where both sessions lose. Read the column alone, each is a run of + # the other, so neither is a conversation, the conversation list comes back with neither in + # it, and they are in neither control. + ("a session on a cycle is taken for a run of the session it names", + [(" if (onCycle.has(cur)) { base = CONVERSATION; break; }", + " if (onCycle.has(cur)) { base = RUN; break; }")], + "a session whose parent names it back is a conversation, not a run"), + + # A parent with a row in the runtime database and no directory in the pool. The walk never + # asked about it, so the map does not have it, and the run of it is a run of a + # conversation the reader cannot pick. + ("a run whose parent is not in the list is taken for a run of it", + [(" if (!parent || !map.has(parent)) { base = CONVERSATION; break; }", + " if (!parent) { base = CONVERSATION; break; }\n if (!map.has(parent)) { base = RUN; break; }")], + "a run whose parent is not in the list is a conversation, so that the reader can reach it"), + + # A run of a run. The two controls are one conversation and the runs of THAT one, so the + # run of a run is a conversation the reader can pick. + ("a run of a run is taken for a run of it", + [(" const flip = (layer) => (layer === CONVERSATION ? RUN : CONVERSATION);", + " const flip = (layer) => RUN;")], + "a run of a run is a conversation too, because the run pick shows the runs of ONE conversation"), # ---- the page ---- # diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py index 36aee4c7..b69a6ba3 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py @@ -18,9 +18,12 @@ resolver gives it, and a version that skips the containment check builds an entry from a path outside the session store. The second reads the workspace column: a runtime database written before that column existed has none, and a version that reads it without asking whether it is -there takes the whole list down with it. The third is the identity of that path: it can sit -inside the sessions root and still belong to a different session, and a version that does not -check would list that other session twice and lose the conversation the page is about. +there takes the whole list down with it. The third is the identity of that path, and it has two +parts now: the directory name can sit inside the sessions root and still belong to a different +session, and so can the manifest the runtime wrote beside the transcript -- a directory whose +name says one session and whose manifest says another is the other session's directory, and a +version that does not check would list that other session twice and lose the conversation the +page is about. Three more are on the page. The badge is applied to every row, the badge is applied to none, and the badge is placed in front of the name -- the third of those still contains the right @@ -70,14 +73,23 @@ " const ordered = currentEntry ? [...same, ...other, currentEntry] : [...same, ...other];")], "and the current conversation is first"), - # The identity of the recorded path. Containment is not enough: a path can sit inside the - # sessions root and still belong to a different session, and then the row that gets injected - # is that other session's transcript -- listed twice, while the conversation the page is - # about is nowhere in the list. - ("the injected row is not checked against the session it is supposed to be", + # The identity of the recorded path, in two parts now. The directory name is one of them: + # a path that points at another session's directory is refused, and this is the check that + # keeps one directory from being listed twice. The manifest is the other, and it is below. + ("the recorded path is not checked against the session its directory name encodes", [(" if (decodeSessionIdFromDirName(basename(dir)) !== expectedSessionId) {", " if (false) {")], - "a recorded path that belongs to another session is refused, so no session is listed twice"), + "a recorded path whose directory name names another session is refused even when its manifest agrees"), + + # The other part: the manifest the runtime wrote beside the transcript. It is the authority + # on which session a directory holds -- the service reads a session's identity from it -- so + # a directory whose name says one session and whose manifest says another is the OTHER + # session's directory, and accepting it puts the current conversation's id on somebody + # else's messages. + ("the recorded path is not checked against the session its manifest names", + [(" const identity = await readSessionIdentity(dir);\n if (identity.id !== expectedSessionId) {", + " const identity = { id: expectedSessionId };\n if (identity.id !== expectedSessionId) {")], + "a recorded path whose manifest names another session is refused"), # The partition, dropped. Every session comes back in one group, which is the plain recency # order -- the order the picker had before the current conversation was pinned. diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs index c06a5053..1833b851 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -28,11 +28,14 @@ * not on screen is a run the reader cannot open. * 3. They are ordered by when each one last ran, most recent first — the one that ran last is * the one a reader is most likely to want, and it is the one the other order buries. - * 4. The layer is decided by `parent_session_id` and by nothing else. `purpose` says what a - * session IS (a scheduled run, a background task), which is a different question from what - * it hangs off, and the two are orthogonal: a sub-agent is a task AND somebody's child. - * A judge that used `purpose` would call a scheduled run a sub-agent, and a sub-agent - * spawned by something other than a task a main session. + * 4. The layer is decided by `parent_session_id` and by the chain it forms, and by nothing + * else. `purpose` says what a session IS (a scheduled run, a background task), which is a + * different question from what it hangs off, and the two are orthogonal: a sub-agent is a + * task AND somebody's child. A judge that used `purpose` would call a scheduled run a + * sub-agent, and a sub-agent spawned by something other than a task a main session. + * The chain is what says whether a parent is a conversation: a run's parent is one, and a + * cycle has no conversation at the end of it, so the two sessions that name each other are + * two conversations rather than two runs of each other that the reader cannot reach. * * The fourth rule is the one that decides the other three, so the fixture below contains a row * that is a child with no purpose, and a row that is a main session whose purpose says 「task」. @@ -336,6 +339,14 @@ try { assert('a sub-agent with no purpose at all is still a sub-agent', kidsOfMain0.includes(purposeLess) && !mainIds.includes(purposeLess), kidsOfMain0.includes(purposeLess) ? 'in the sub-agent list' : 'missing'); + // The sub-agent whose purpose says what it is. It has a parent AND a purpose, and a judge + // that read the purpose column instead of the parent column would move it into the wrong + // layer -- so the two columns are read against each other here, and not only against the + // row that has just one of them. + const purposeSaying = KID(1); + assert('a sub-agent whose purpose says 「sub-agent」 is still a sub-agent', + kidsOfMain0.includes(purposeSaying) && !mainIds.includes(purposeSaying), + kidsOfMain0.includes(purposeSaying) ? 'in the sub-agent list' : 'missing'); // And the sub-agents hang off the main session they name — the join key is one id, the same // id the main row carries. assert('and each sub-agent hangs off the main session it names', @@ -672,6 +683,16 @@ try { !raw.includes('D:\\private\\workspace-one') && !raw.includes('D:/private/workspace-one'), raw.slice(0, 200)); + // The map itself, before the layer decision has anything to say about it: a parent that names + // no session is not a parent. The route clears the parent of every session it puts in the + // conversation layer, so the wire would look the same whether this check is here or not -- + // which is exactly why it is read off the map the read produces, and not off the wire. + const rawTitles = await readSessionTitles(home, ['mvs_phantom_child', 'mvs_hostile_child']); + assert('and the map itself carries no parent that names no session', + rawTitles.get('mvs_phantom_child')?.parent === null + && rawTitles.get('mvs_hostile_child')?.parent === null, + JSON.stringify([rawTitles.get('mvs_phantom_child')?.parent, rawTitles.get('mvs_hostile_child')?.parent])); + if (instance) await instance.dispose(); controller.abort(); if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; @@ -680,6 +701,219 @@ try { failures.push('a parent that names no session -- ' + (error && error.message)); } +/* ---------- 9: two sessions that name each other as parent ---------- */ +// +// The parent column is a foreign key into the runtime database's own session table, and a +// foreign key can point back: A's parent is B, B's parent is A, and both have a row, so both +// are real parents of a real session. The column alone cannot tell a run from a conversation +// there. Read it straight, both of them are runs -- of each other -- so neither is a +// conversation, the conversation list comes back with neither in it, and the two are in +// neither control: the reader cannot reach either one. +// +// The layer is decided by the chain, and a chain ends at a conversation. A cycle ends at no +// conversation, so it is dissolved: both sessions are conversations, and their parent columns +// are dropped, because a conversation that names a parent names a run. +// +// The fixture holds three sessions: the two that name each other, and one run of the first of +// them -- a run of a conversation that sits on a cycle is still a run of it. + +const CYCLE_ALPHA = 'mvs_cycle_alpha'; +const CYCLE_BRAVO = 'mvs_cycle_bravo'; +const CYCLE_RUN = 'mvs_cycle_run'; + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-cycle-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const alpha = await makeSession(sessionsRoot, 0, CYCLE_ALPHA, BASE_TIME - 10 * 60_000); + const bravo = await makeSession(sessionsRoot, 1, CYCLE_BRAVO, BASE_TIME - 20 * 60_000); + const run = await makeSession(sessionsRoot, 2, CYCLE_RUN, BASE_TIME - 30 * 60_000); + runtimeDb(home, [ + { + sessionId: CYCLE_ALPHA, + relative: alpha.relative, + parent: CYCLE_BRAVO, + kind: 'task', + purpose: 'local-background-task:alpha', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 10 * 60_000 + }, + { + sessionId: CYCLE_BRAVO, + relative: bravo.relative, + parent: CYCLE_ALPHA, + kind: 'task', + purpose: 'local-background-task:bravo', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 20 * 60_000 + }, + { + sessionId: CYCLE_RUN, + relative: run.relative, + parent: CYCLE_ALPHA, + kind: 'task', + purpose: 'local-background-task:run', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 30 * 60_000 + } + ]); + + const { start } = await import(pathToFileURL(SERVER).href); + const previousRoot = process.env.MMC_TRAJECTORY_ROOT; + process.env.MMC_TRAJECTORY_ROOT = home; + const controller = new AbortController(); + const noop = () => {}; + let base = null; + let instance = null; + for (let attempt = 0; attempt < 12 && !base; attempt += 1) { + const port = 19900 + Math.floor(Math.random() * 300) + attempt; + try { + instance = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + dataDir: PLUGIN_ROOT, + generationId: 'guard', + listen: { host: '127.0.0.1', port }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop } + }); + base = 'http://127.0.0.1:' + port; + } catch (error) { + // The port was taken. Another one is tried. + } + } + if (!base) throw new Error('could not bind a port for the cycle route'); + const response = await fetch(base + '/api/sessions?limit=20'); + const body = await response.json(); + const sessionIds = (body.sessions || []).map((row) => row.id); + const childIds = (body.children || []).map((row) => row.id); + + assert('a session whose parent names it back is a conversation, not a run', + sessionIds.includes(CYCLE_ALPHA) && sessionIds.includes(CYCLE_BRAVO), + sessionIds.join('|')); + assert('and it is in neither run list', + !childIds.includes(CYCLE_ALPHA) && !childIds.includes(CYCLE_BRAVO), + childIds.join('|')); + // The run of the cycle member still names it -- that is what a run names -- so this is about + // the two rows of the cycle themselves: a conversation that names a parent names a run, and + // these two are conversations. + const parentNamedBy = (id) => { + const row = (body.sessions || []).concat(body.children || []).find((entry) => entry.id === id); + return row ? row.sessionParentId : null; + }; + assert('and neither of them names a parent on the wire', + !parentNamedBy(CYCLE_ALPHA) && !parentNamedBy(CYCLE_BRAVO), + String(parentNamedBy(CYCLE_ALPHA)) + ' / ' + String(parentNamedBy(CYCLE_BRAVO))); + assert('and a run of a session that sits on a cycle is still a run of it', + childIds.includes(CYCLE_RUN) + && (body.children || []).some((row) => row.id === CYCLE_RUN && row.sessionParentId === CYCLE_ALPHA), + childIds.join('|')); + assert('and the two of them are in the conversation list only once each', + sessionIds.filter((id) => id === CYCLE_ALPHA).length === 1 + && sessionIds.filter((id) => id === CYCLE_BRAVO).length === 1, + sessionIds.join('|')); + + if (instance) await instance.dispose(); + controller.abort(); + if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; + else process.env.MMC_TRAJECTORY_ROOT = previousRoot; +} catch (error) { + failures.push('two sessions that name each other as parent -- ' + (error && error.message)); +} + +/* ---------- 10: a parent the payload does not know, and a run of a run ---------- */ +// +// Two more shapes of the chain, and the same rule answers both. +// +// - The parent has a row in the runtime database but no directory in the pool. It is a real +// session and a real conversation, but the reader cannot pick it -- and the run pick only +// ever shows the runs of the conversation that was picked. A run of it would therefore be a +// run of a conversation that is not in the list, which is a run nobody can reach. It is a +// conversation itself, so that it is somewhere the reader can reach. +// - The parent is itself a run. The two controls are one conversation and the runs of THAT +// one, so a run of a run goes in the conversation layer as well. +// +// Both are read off the function that decides the layer, and off the map it corrects: the +// parent column of a conversation is cleared, because a conversation that names a parent names +// a run. + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-unknown-parent-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const root = await makeSession(sessionsRoot, 0, 'mvs_chain_root', BASE_TIME - 10 * 60_000); + const run = await makeSession(sessionsRoot, 1, 'mvs_chain_run', BASE_TIME - 20 * 60_000); + const deeper = await makeSession(sessionsRoot, 2, 'mvs_chain_deeper', BASE_TIME - 30 * 60_000); + const unknown = await makeSession(sessionsRoot, 3, 'mvs_chain_unknown', BASE_TIME - 40 * 60_000); + runtimeDb(home, [ + { + sessionId: 'mvs_chain_root', + relative: root.relative, + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 10 * 60_000 + }, + { + sessionId: 'mvs_chain_run', + relative: run.relative, + parent: 'mvs_chain_root', + kind: 'task', + purpose: 'local-background-task:run', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 20 * 60_000 + }, + { + sessionId: 'mvs_chain_deeper', + relative: deeper.relative, + parent: 'mvs_chain_run', + kind: 'task', + purpose: 'local-background-task:deeper', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 30 * 60_000 + }, + { + sessionId: 'mvs_chain_unknown', + relative: unknown.relative, + parent: 'mvs_chain_absent', + kind: 'task', + purpose: 'local-background-task:unknown', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 40 * 60_000 + }, + { + // The parent of the last one. It has a row in the runtime database, so the parent column + // is a parent by the foreign key -- and it has no directory in the pool, so the walk never + // asked about it and the map does not have it. + sessionId: 'mvs_chain_absent', + relative: '2026/10/20/99-00-00-session_absent', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 50 * 60_000 + } + ]); + + const candidates = await collectSessionCandidates(sessionsRoot); + const ids = candidates.allEntries + .map((entry) => decodeSessionIdFromDirName(basename(entry.dir))).filter(Boolean); + const titles = await readSessionTitles(home, ids); + const layers = splitSessionLayers(candidates.allEntries, titles); + const mainIds = idsOf(layers.mains); + const kidIds = [...layers.childrenByParent.keys()] + .flatMap((parent) => idsOfBucket(layers.childrenByParent.get(parent))); + + assert('a run whose parent is not in the list is a conversation, so that the reader can reach it', + mainIds.includes('mvs_chain_unknown') && !kidIds.includes('mvs_chain_unknown'), + mainIds.join('|') + ' / ' + kidIds.join('|')); + assert('and it names no parent, because a conversation that names a parent names a run', + titles.get('mvs_chain_unknown')?.parent === null, + String(titles.get('mvs_chain_unknown')?.parent)); + assert('a run of a run is a conversation too, because the run pick shows the runs of ONE conversation', + mainIds.includes('mvs_chain_deeper') && !kidIds.includes('mvs_chain_deeper'), + mainIds.join('|') + ' / ' + kidIds.join('|')); + assert('and the run it hangs off is still a run of the conversation above it', + kidIds.includes('mvs_chain_run') + && [...layers.childrenByParent.keys()].every((parent) => parent === 'mvs_chain_root'), + [...layers.childrenByParent.keys()].join('|')); +} catch (error) { + failures.push('a parent the payload does not know, and a run of a run -- ' + (error && error.message)); +} + console.log(pass + ' assertions passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs index 06946750..ac50b7ac 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs @@ -447,6 +447,89 @@ assert('the kind tag is written on every row, not only on the rows that have one tagWrite !== null && tagInsideMeta === null, tagWrite === null ? 'no unconditional tag write found' : String(tagInsideMeta && tagInsideMeta[0])); +/* ---------- 13: a recorded path whose own manifest names another session ---------- */ +// +// The directory name is where a session id is written down, and for a long time it was the only +// place the recorded path was checked against. But the service reads a session's identity from +// the manifest the runtime wrote beside its transcript, and the manifest is the authority: a +// directory whose name encodes one session and whose manifest names another is the OTHER +// session's directory, whatever the folder is called. +// +// So the two can disagree, and both directions are refused here: +// +// - the manifest names another session -- the name is right and the manifest is wrong, which +// is the case the identity function itself is there for; +// - the name names another session -- the manifest is right and the name is wrong, and the +// directory the walk already lists under its own name would be injected a second time +// under the id the manifest gives it. +// +// The third case is the one that has to keep working: a manifest that agrees with the name is +// accepted, because a refusal there would take the current conversation out of the list. + +try { + // The manifest names another session. The directory's NAME encodes the current session, so + // the name check passes and the manifest is the one that has to catch it. + const a = await store(31); + await writeFile(join(a.made[30].dir, 'manifest.json'), + JSON.stringify({ sessionId: ID(0), createdAtMs: 1 }), 'utf8'); + runtimeDb(a.home, ['parent_session_id', 'purpose', 'workspace_dir'], [ + { sessionId: ID(30), relative: a.made[30].relative, workspace: 'workspace-one' }, + ]); + const wrongManifest = await visibleFor(a.sessionsRoot, a.home, 20); + assert('a recorded path whose manifest names another session is refused', + !wrongManifest.visible.some((entry) => entry.dir === a.made[30].dir), + dirsOf(wrongManifest.visible).join('|')); + assert('and the list is still the plain recency order', + sameOrder(wrongManifest.visible, a.made.slice(0, 20)), + dirsOf(wrongManifest.visible).join('|')); +} catch (error) { + failures.push('a recorded path whose manifest names another session is refused; ' + + 'the section threw before it could answer -- ' + + String(error && error.message ? error.message : error)); +} + +try { + // The name names another session and the manifest agrees with the CURRENT one. The + // directory is the walk's own row for a different session, so accepting it would list that + // one directory twice: once as the injected row and once as the walk's row. + const b = await store(31); + await writeFile(join(b.made[0].dir, 'manifest.json'), + JSON.stringify({ sessionId: ID(30), createdAtMs: 1 }), 'utf8'); + runtimeDb(b.home, ['parent_session_id', 'purpose', 'workspace_dir'], [ + { sessionId: ID(30), relative: b.made[0].relative, workspace: 'workspace-one' }, + ]); + const wrongName = await visibleFor(b.sessionsRoot, b.home, 20); + assert('a recorded path whose directory name names another session is refused even when its manifest agrees', + new Set(dirsOf(wrongName.visible)).size === wrongName.visible.length, + dirsOf(wrongName.visible).join('|')); + assert('and the list is still the plain recency order', + sameOrder(wrongName.visible, b.made.slice(0, 20)), + dirsOf(wrongName.visible).join('|')); +} catch (error) { + failures.push('a recorded path whose directory name names another session is refused; ' + + 'the section threw before it could answer -- ' + + String(error && error.message ? error.message : error)); +} + +try { + // The manifest agrees with the name. This is the ordinary case, and it is here so that a + // refusal cannot be bought by a check that refuses everything. + const c = await store(31); + await writeFile(join(c.made[30].dir, 'manifest.json'), + JSON.stringify({ sessionId: ID(30), createdAtMs: 1 }), 'utf8'); + runtimeDb(c.home, ['parent_session_id', 'purpose', 'workspace_dir'], [ + { sessionId: ID(30), relative: c.made[30].relative, workspace: 'workspace-one' }, + ]); + const rightManifest = await visibleFor(c.sessionsRoot, c.home, 20); + assert('a recorded path whose manifest agrees with its name is accepted, and the current conversation is first', + rightManifest.visible[0]?.dir === c.made[30].dir, + dirsOf(rightManifest.visible).slice(0, 3).join('|')); +} catch (error) { + failures.push('a recorded path whose manifest agrees with its name is accepted; ' + + 'the section threw before it could answer -- ' + + String(error && error.message ? error.message : error)); +} + for (const home of homes) rmSync(home, { recursive: true, force: true }); console.log('[session-picker] ' + pass + ' passed, ' + failures.length + ' failed'); From 89375ef21f1d5669f9b13e5eabbd8d0cbee7df02 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 18:54:20 +0800 Subject: [PATCH 14/95] The rows a session list publishes are built, and a directory is a session's only when its manifest says so A row came out of a cache and was patched with the runtime's facts, so a session whose row had left the runtime database went on publishing the title, the kind and the parent it had last time, while the validator, which reads the same rows, reported none of them. The row is built instead: every runtime fact is written on every pass, and a row with no runtime facts carries none of them -- the label falls back to the one the filesystem gives, and the kind, the parent and the tag are null. The layer of a session was decided by whether its parent was in the map, and the map holds sessions the reader cannot pick: the current conversation is in it even when it has no transcript in the store. The membership test is about the walk now, so a parent the reader cannot reach is a parent the reader cannot reach, whatever the database remembers about it. Two paths took a directory's name for a session's identity: the picker promoting the current conversation, and the route that serves the latest transcript binding the current session to it. Both confirm the directory against the manifest beside the transcript now, through one function, so a directory whose name says one session and whose manifest says another is that other session's directory. --- .../mmc-trajectory/miniapp/node/server.mjs | 92 ++++++++++++++----- 1 file changed, 71 insertions(+), 21 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 8dba7a96..1430dfe8 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -1253,6 +1253,35 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSe } } +/** + * The entry that holds a session: found by the id its directory name decodes to, and confirmed by + * the manifest beside its transcript. + * + * The directory name is a claim; the manifest is the authority. `readSessionIdentity` prefers + * `manifest.sessionId`, so a directory whose name encodes one session and whose manifest names + * another is the OTHER session's directory, whatever the folder is called. Two shortcuts used to + * take the name's word for it -- the picker promoting the current conversation into the list, and + * the 'latest' resolver choosing which transcript to serve -- and both could put one session's id + * on another session's messages. + * + * So the name is used to FIND the candidate and the manifest is used to CONFIRM it. A candidate + * whose manifest names another session is not this session's entry, and the search goes on: another + * directory may carry this id honestly. + * + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries + * @param {string} sessionId + * @returns {Promise<{ dir: string, sizeBytes: number, lastActiveAt: number } | null>} + */ +export async function verifiedEntryForSessionId(entries, sessionId) { + if (!sessionId) return null; + for (const candidate of entries) { + if (decodeSessionIdFromDirName(basename(candidate.dir)) !== sessionId) continue; + const identity = await readSessionIdentity(candidate.dir); + if (identity.id === sessionId) return candidate; + } + return null; +} + /** * The visible set the picker shows, and the order it is shown in. * @@ -1307,9 +1336,13 @@ export async function buildVisibleSessionList(entries, current, titles, sessions : null; let currentEntry = null; if (!currentParent) { - currentEntry = entries.find((entry) => decodeSessionIdFromDirName(basename(entry.dir)) === currentId) || null; + // The current conversation's own row, found among the walk's rows and CONFIRMED by the + // manifest beside the transcript: the directory name is only a claim. A directory whose name + // says this session and whose manifest says another one is that other session's directory, + // and promoting it would put this session's id on somebody else's messages. + currentEntry = await verifiedEntryForSessionId(entries, currentId); if (!currentEntry && current.relativeDir) { - // Not in the recent window -- build the entry straight from the recorded path. The session it + // Not in the walk -- build the entry straight from the recorded path. The session it // is supposed to be is named, so a path that turns out to be a different session's directory // is refused instead of being shown under this id. const built = await entryFromRelativeDir(sessionsRoot, current.relativeDir, currentId); @@ -1385,6 +1418,16 @@ export function splitSessionLayers(entries, titles) { const mains = []; const childrenByParent = new Map(); const map = titles instanceof Map ? titles : new Map(); + // The sessions this payload knows about: the ones with a directory in the walk. NOT the ones + // with a row in the runtime database -- the map holds sessions that have no transcript here, + // the current conversation among them, and a parent that is only a row is not something the + // reader can pick, so a run hanging off it would be a run the reader cannot reach. The rule is + // about what the reader can reach, so the membership test is about the walk. + const known = new Set(); + for (const entry of entries) { + const id = decodeSessionIdFromDirName(basename(entry.dir)); + if (id) known.add(id); + } /** * The session a session hangs off, or null when it hangs off nothing. @@ -1440,7 +1483,7 @@ export function splitSessionLayers(entries, titles) { chain.push(cur); if (onCycle.has(cur)) { base = CONVERSATION; break; } const parent = parentOf(cur); - if (!parent || !map.has(parent)) { base = CONVERSATION; break; } + if (!parent || !known.has(parent)) { base = CONVERSATION; break; } cur = parent; } let layer = base; @@ -3620,23 +3663,26 @@ export async function start(context) { const describeRow = async (entry) => { const descriptor = await describeSession(entry); const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir))); - if (meta) { - if (meta.title) descriptor.label = meta.title; - descriptor.sessionKind = meta.kind; - descriptor.sessionParentId = meta.parent; - } - // What this session IS, as a token the page can render: a lightweight exchange, a - // scheduled run, a background task. The raw purpose stays on this side of the wire -- - // it is free text, and free text is not something to hand a reader's browser. + // The row is BUILT, not patched. The descriptor comes out of a cache and is shared, so + // writing the runtime's facts onto it leaves the last writer's facts on it for the next + // reader -- and a session whose row has left the runtime database goes on publishing the + // title, the kind and the parent it had last time, while the validator, which reads the + // same rows, reports none of them. A body and an ETag that disagree about the same row is + // how a badge outlives the fact it was derived from. // - // Assigned unconditionally, and that is the point. The descriptor comes out of a cache, - // so a session whose row is missing from the runtime database keeps the tag it was - // given last time -- while the validator, which reads the same rows, reports no tag for - // it. A body and an ETag that disagree about the same row is how a badge outlives the - // fact it was derived from, so the field is written on every pass, not only when there - // is something to write. - descriptor.sessionTag = meta ? sessionTagFor(meta.kind, meta.purpose) : null; - return descriptor; + // So every runtime fact is written on every pass, and a row with no runtime facts carries + // none of them: the label falls back to the one the filesystem gives, and the kind, the + // parent and the tag are null. + return { + ...descriptor, + label: meta && meta.title ? meta.title : descriptor.label, + sessionKind: meta ? meta.kind : null, + sessionParentId: meta ? meta.parent : null, + // What this session IS, as a token the page can render: a lightweight exchange, a + // scheduled run, a background task. The raw purpose stays on this side of the wire -- + // it is free text, and free text is not something to hand a reader's browser. + sessionTag: meta ? sessionTagFor(meta.kind, meta.purpose) : null, + }; }; const described = []; for (const entry of visible) { @@ -3693,8 +3739,12 @@ export async function start(context) { // a worker or cron session often has the newest file even while the user reads another one. const current = await resolveCurrentConversation(location.root); if (current) { - entry = entries.find((candidate) => basename(candidate.dir).includes('session_') - && decodeSessionIdFromDirName(basename(candidate.dir)) === current.sessionId) ?? null; + // The current session's own row, found by the id its directory name decodes to and + // CONFIRMED by the manifest beside the transcript: the name is a claim, the manifest is + // the authority. A directory whose name says this session and whose manifest says another + // one is that other session's directory, and binding this session's id to it would serve + // somebody else's messages under this session's name. + entry = await verifiedEntryForSessionId(entries, current.sessionId); if (!entry && current.relativeDir) { // Not in the recent window — build the entry straight from the recorded path. That value // is a database column rather than a name this app chose, so it earns the same From 576a271e253b9cc516bcc0e6e378dd8442cf5df5 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 18:54:34 +0800 Subject: [PATCH 15/95] Four rules about what a row may say get owners that can fail A row whose runtime facts are gone no longer publishes them -- checked on the response body and on the body the service reuses for the next request, after the caches have had their time. A run whose parent the reader cannot pick is a conversation of its own. A directory whose manifest names another session is neither promoted nor bound to the current session, in both window positions. The workspace path is looked for in the parsed body, where JSON cannot hide it by escaping a backslash. The control that leaked the workspace also leaked the purpose, so it was caught by the purpose assertion and the workspace assertion was never asked whether it could fail. The two have a control each now, and each is caught by the assertion that is about it. --- .../tests/mutate-list-cache-ttl.py | 6 +- .../tests/mutate-session-layers.py | 91 +++++- .../tests/mutate-session-picker.py | 4 +- .../tests/verify-list-cache.mjs | 30 +- .../tests/verify-session-layers.mjs | 278 +++++++++++++++++- .../tests/verify-session-picker.mjs | 18 +- 6 files changed, 388 insertions(+), 39 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py index 4dc483ec..7e4d0999 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-list-cache-ttl.py @@ -185,9 +185,9 @@ [(" parent: parentNames ? parent : null,\n", "")], 'the parent row is mapped, and an empty id is no parent rather than a parent named ""'), - ("the descriptor is built without the parent", - [(" descriptor.sessionParentId = meta.parent;", "")], - "the descriptor carries the parent onto the row the page receives"), + ("the row is built without the parent", + [(" sessionParentId: meta ? meta.parent : null,", "")], + "the row the page receives carries the parent, on every row"), # This one is the most likely to be reintroduced by accident: it is a whole-field removal in # a function whose job is to decide what may leave the service, and nothing else in it fails. diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py index c2e5f30e..e5a147a7 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -20,8 +20,18 @@ * a run's parent is a CONVERSATION, so a cycle is dissolved into two conversations, a run of a run is a conversation, and a run whose parent is not in the list is a conversation -- all three because a run the reader cannot reach is a run that is gone; + * "not in the list" means not in the WALK: the runtime database remembering a parent the reader + cannot pick does not make that parent pickable; + * a row is built, not patched, so a session whose row leaves the runtime database stops + publishing the title, the kind and the parent it had; + * a directory is a session's directory only when its name AND its manifest say so, so the + picker does not promote the current conversation on somebody else's directory and the + 'latest' route does not bind it to one; * a parent that names no session is not a parent, so it is not published and the run it was on is a conversation; + * the raw purpose and the workspace each have to be caught by their own assertion, because + one control that leaked both was caught by the purpose one and never asked the workspace + one; * the page groups its runs into a map with no prototype, so a parent id that names an inherited object property cannot break the grouping; * the pool is the whole walk, not the thirty most recent sessions; @@ -158,10 +168,21 @@ # asked about it, so the map does not have it, and the run of it is a run of a # conversation the reader cannot pick. ("a run whose parent is not in the list is taken for a run of it", - [(" if (!parent || !map.has(parent)) { base = CONVERSATION; break; }", - " if (!parent) { base = CONVERSATION; break; }\n if (!map.has(parent)) { base = RUN; break; }")], + [(" if (!parent || !known.has(parent)) { base = CONVERSATION; break; }", + " if (!parent) { base = CONVERSATION; break; }\n if (!known.has(parent)) { base = RUN; break; }")], "a run whose parent is not in the list is a conversation, so that the reader can reach it"), + # The same rule, one step further: a parent the walk does not hold is a parent the reader + # cannot pick, WHETHER or not the runtime database still has a row for it. The route asks + # about the current conversation even when that conversation has no transcript in the store, + # so the map holds parents the reader can never reach -- and a run hanging off one of them is + # a run of nothing. The membership test is about the WALK, not about what the database + # remembers. + ("the layer is decided by what the database remembers rather than by what the walk holds", + [(" if (!parent || !known.has(parent)) { base = CONVERSATION; break; }", + " if (!parent || !map.has(parent)) { base = CONVERSATION; break; }")], + "a run whose parent is the current conversation but has no transcript of its own is a conversation"), + # A run of a run. The two controls are one conversation and the runs of THAT one, so the # run of a run is a conversation the reader can pick. ("a run of a run is taken for a run of it", @@ -169,6 +190,45 @@ " const flip = (layer) => RUN;")], "a run of a run is a conversation too, because the run pick shows the runs of ONE conversation"), + # ---- what a row is made of ---- + # + # The descriptor behind a row comes out of a cache, so the runtime's facts must be written + # onto a row that is BUILT, not onto the cached object: a session whose row leaves the + # runtime database has to stop publishing the title, the kind and the parent it had. + ("a row that lost its runtime facts goes on publishing them", + [(" return {\n" + " ...descriptor,\n" + " label: meta && meta.title ? meta.title : descriptor.label,\n" + " sessionKind: meta ? meta.kind : null,\n" + " sessionParentId: meta ? meta.parent : null,", + " if (meta) {\n" + " if (meta.title) descriptor.label = meta.title;\n" + " descriptor.sessionKind = meta.kind;\n" + " descriptor.sessionParentId = meta.parent;\n" + " }\n" + " return {\n" + " ...descriptor,\n" + " label: meta && meta.title ? meta.title : descriptor.label,\n" + " sessionKind: meta ? meta.kind : (descriptor.sessionKind ?? null),\n" + " sessionParentId: meta ? meta.parent : (descriptor.sessionParentId ?? null),")], + "a row whose runtime metadata disappears loses its title kind and parent across cache expiry"), + + # ---- whose directory this is ---- + # + # The directory name is a claim; the manifest beside the transcript is the authority. Two + # shortcuts took the name's word for it -- the picker promoting the current conversation, and + # the 'latest' resolver choosing the transcript to serve -- and both could put one session's + # id on another session's messages. + ("the picker promotes the current conversation on a directory that holds another session", + [(" currentEntry = await verifiedEntryForSessionId(entries, currentId);", + " currentEntry = entries.find((entry) => decodeSessionIdFromDirName(basename(entry.dir)) === currentId) || null;")], + "the current conversation is not promoted on a directory that holds another session"), + + ("the latest route binds the current session to a directory that holds another session", + [(" entry = await verifiedEntryForSessionId(entries, current.sessionId);", + " entry = entries.find((candidate) => decodeSessionIdFromDirName(basename(candidate.dir)) === current.sessionId) ?? null;")], + "the latest route does not serve a directory that holds another session as the current one"), + # ---- the page ---- # # The two picks. The conversation pick holds conversations; the run pick holds the runs of @@ -334,16 +394,29 @@ # The raw purpose and the workspace. Both are in the runtime database and neither may cross # the wire: the purpose is free text a person wrote, and the workspace is a location on the # reader's own disk. The page gets a token for the first and nothing for the second. - ("the raw purpose and the workspace cross the wire", - [(" descriptor.sessionParentId = meta.parent;", - " descriptor.sessionParentId = meta.parent;\n" - " descriptor.sessionPurpose = meta.purpose;\n" - " descriptor.sessionWorkspace = meta.workspace;"), + # + # They are TWO mutations, not one. A single control that leaked both was caught by the + # purpose assertion, so the workspace assertion was never asked whether it could fail -- and + # it could not: it looked for the path in the raw bytes, where JSON has escaped every + # backslash, so the literal it searched for was not in the response at all. The workspace + # alone has to be caught by the workspace assertion, by name. + ("the raw purpose crosses the wire", + [(" sessionParentId: meta ? meta.parent : null,", + " sessionParentId: meta ? meta.parent : null,\n" + " sessionPurpose: meta ? meta.purpose : null,"), (" sessionParentId: value.sessionParentId ?? null,", " sessionParentId: value.sessionParentId ?? null,\n" - " sessionPurpose: value.sessionPurpose ?? null,\n" - " sessionWorkspace: value.sessionWorkspace ?? null,")], + " sessionPurpose: value.sessionPurpose ?? null,")], "the raw purpose does not cross the wire"), + + ("the workspace crosses the wire", + [(" sessionParentId: meta ? meta.parent : null,", + " sessionParentId: meta ? meta.parent : null,\n" + " sessionWorkspace: meta ? meta.workspace : null,"), + (" sessionParentId: value.sessionParentId ?? null,", + " sessionParentId: value.sessionParentId ?? null,\n" + " sessionWorkspace: value.sessionWorkspace ?? null,")], + "and neither does the workspace path"), ] ORIGINALS = { diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py index b69a6ba3..12e16f2e 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py @@ -145,8 +145,8 @@ # validator -- which reads the same rows -- reports none. The body and the ETag then # disagree about the same row, and only the body reaches the page. ("the kind tag is only written on the rows that have one", - [(" descriptor.sessionTag = meta ? sessionTagFor(meta.kind, meta.purpose) : null;", - " if (meta) descriptor.sessionTag = sessionTagFor(meta.kind, meta.purpose);")], + [(" sessionTag: meta ? sessionTagFor(meta.kind, meta.purpose) : null,", + " sessionTag: meta ? sessionTagFor(meta.kind, meta.purpose) : (descriptor.sessionTag ?? null),")], "the kind tag is written on every row, not only on the rows that have one"), # ---- the badge, on the page ---- diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs index fdc7ced2..ea911d14 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-list-cache.mjs @@ -222,19 +222,25 @@ assert('the purpose row is mapped, and an empty value is no purpose', assert('the workspace row is mapped, and an empty value is no workspace', /workspace: typeof row\.workspace_dir === 'string' && row\.workspace_dir\.trim\(\) \? row\.workspace_dir\.trim\(\) : null,/.test(server)); -assert('the descriptor carries the parent onto the row the page receives', - /descriptor\.sessionParentId = meta\.parent;/.test(server)); - -// The tag is written on EVERY row, not only on the rows that have a database entry. The -// descriptor comes out of a cache, so a row that loses its entry keeps the tag it was given -// last time, while the validator — which reads the same rows — reports none for it. A body and -// an ETag that disagree about one row is how a badge outlives the fact it was derived from, -// and the page only ever sees the body. +// The parent the page groups a session's runs under is the one the runtime database holds, and +// the row that carries it is BUILT rather than patched: the descriptor comes out of a cache and +// is shared, so a runtime fact written onto it outlives the fact it was derived from. The parent +// is therefore a field of the row that is built, written on every pass, and null when the +// runtime database holds no row for that session. +assert('the row the page receives carries the parent, on every row', + /sessionParentId: meta \? meta\.parent : null,/.test(server)); + +// The tag is written on EVERY row, not only on the rows that have a database entry -- the same +// shape as the parent above, and for the same reason. A body and an ETag that disagree about +// one row is how a badge outlives the fact it was derived from, and the page only ever sees the +// body. // -// The shape is what makes it unconditional: a `? :` with the tag on one side and null on the -// other cannot be inside a `if (meta)` block without becoming a second statement. -assert('the descriptor carries the kind badge onto the row the page receives, on every row', - /descriptor\.sessionTag = meta \? sessionTagFor\(meta\.kind, meta\.purpose\) : null;/.test(server)); +// The shape is what makes it unconditional: the tag is a FIELD of the row that is built, and its +// else-branch is null -- "no row to derive one from" -- rather than a statement that only runs +// when there is a row. An object property is not a statement, so it cannot be conditional in the +// way the old spelling could. +assert('the row the page receives carries the kind badge, on every row', + /sessionTag: meta \? sessionTagFor\(meta\.kind, meta\.purpose\) : null,/.test(server)); // `?? null` rather than a bare copy: the page decides by whether this field is SET, so a missing // key and a present-but-undefined one have to land in the same place as a row that has no parent. diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs index 1833b851..03cc11c9 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -171,6 +171,53 @@ function runtimeDb(home, rows, leases) { db.close(); } +/** + * The service, started against a store, and stopped again. + * + * The port is picked from this guard's own band and a taken one is simply tried again. The store + * is named through the environment, because that is how the service finds it -- the same way the + * runtime finds it. + * + * @param {string} home + * @returns {Promise<{ base: string, dispose: () => Promise }>} + */ +async function startService(home) { + const { start } = await import(pathToFileURL(SERVER).href); + const previousRoot = process.env.MMC_TRAJECTORY_ROOT; + process.env.MMC_TRAJECTORY_ROOT = home; + const controller = new AbortController(); + const noop = () => {}; + let instance = null; + let base = null; + for (let attempt = 0; attempt < 12 && !base; attempt += 1) { + const port = 19900 + Math.floor(Math.random() * 300) + attempt; + try { + instance = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + dataDir: PLUGIN_ROOT, + generationId: 'guard', + listen: { host: '127.0.0.1', port }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop } + }); + base = 'http://127.0.0.1:' + port; + } catch (error) { + // The port was taken. Another one is tried. + } + } + if (!base) throw new Error('could not bind a port for the route'); + return { + base, + dispose: async () => { + if (instance) await instance.dispose(); + controller.abort(); + if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; + else process.env.MMC_TRAJECTORY_ROOT = previousRoot; + } + }; +} + /** * The store, the database and the layers the service builds from them. * @@ -672,16 +719,32 @@ try { !childIds.includes('mvs_hostile_child') && !parentsOnTheWire.includes(HOSTILE_PARENT), childIds.join('|') + ' / ' + parentsOnTheWire.join('|')); - // The two values that must never cross the wire at all, checked on the bytes rather than on - // the parsed body. The purpose is free text the runtime wrote, and the workspace is a - // location on the reader's own disk; the page gets a token for the first and nothing for the - // second. Both are in this fixture, so both are absent here. + // The two values that must never cross the wire at all. The purpose is free text the runtime + // wrote, and the workspace is a location on the reader's own disk; the page gets a token for + // the first and nothing for the second. Both are in this fixture, so both are absent here. assert('the raw purpose does not cross the wire', !raw.includes('local-background-task:orphan'), raw.slice(0, 200)); + // The workspace is looked for in the PARSED body, not in the raw bytes, and that is the whole + // point of the check. JSON escapes a backslash, so a path that is spelled + // `D:\private\workspace-one` on disk is spelled `D:\\private\\workspace-one` in the response, and + // a search for the path as it is on disk finds nothing in it. A guard that looks in the wrong + // spelling passes forever, which is what happened here: the workspace could cross the wire and + // this assertion stayed green, because the only mutation that leaked it also leaked the purpose + // and was caught by the assertion above. So every string in the body is searched instead, which + // is the spelling the value actually has once it is parsed. + const everyString = (value, out) => { + if (typeof value === 'string') { out.push(value); return out; } + if (Array.isArray(value)) { for (const item of value) everyString(item, out); return out; } + if (value && typeof value === 'object') { + for (const key of Object.keys(value)) { out.push(key); everyString(value[key], out); } + } + return out; + }; + const wireStrings = everyString(body, []).join('\n'); assert('and neither does the workspace path', - !raw.includes('D:\\private\\workspace-one') && !raw.includes('D:/private/workspace-one'), - raw.slice(0, 200)); + !wireStrings.includes('D:\\private\\workspace-one') && !wireStrings.includes('D:/private/workspace-one'), + wireStrings.slice(0, 200)); // The map itself, before the layer decision has anything to say about it: a parent that names // no session is not a parent. The route clears the parent of every session it puts in the @@ -914,6 +977,209 @@ try { failures.push('a parent the payload does not know, and a run of a run -- ' + (error && error.message)); } +/* ---------- 11: the current conversation that has no transcript of its own ---------- */ +// +// The map the layer is decided from holds the sessions with a ROW in the runtime database, and the +// route asks about the current conversation even when that conversation has no transcript in the +// store -- it wants the title and the workspace of it. So the map can name a parent the reader +// cannot pick: a row, not a directory. +// +// A run hanging off such a parent is a run of a conversation that is not in the list, and the run +// pick only ever shows the runs of the conversation that was picked. So the run is a conversation +// itself, exactly like a run whose parent has no row at all. The rule is about what the reader can +// reach, and a parent that is not in the list is a parent the reader cannot reach -- whether it is +// missing from the map or missing from the walk. + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-current-parent-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const child = await makeSession(sessionsRoot, 0, 'mvs_current_child', BASE_TIME - 20 * 60_000); + // The parent has a row in the runtime database and NO directory in the store. It is the current + // conversation -- the newest row, and a plain conversation -- so the route asks the database + // about it even though there is nothing on disk to read. + runtimeDb(home, [ + { + sessionId: 'mvs_current_parent', + relative: '2026/10/20/99-00-00-session_absent', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME + }, + { + sessionId: 'mvs_current_child', + relative: child.relative, + parent: 'mvs_current_parent', + kind: 'task', + purpose: 'local-background-task:child', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 20 * 60_000 + } + ]); + + const service = await startService(home); + try { + const body = await (await fetch(service.base + '/api/sessions?limit=20')).json(); + const sessionIds = (body.sessions || []).map((row) => row.id); + const childIds = (body.children || []).map((row) => row.id); + const parentsOnTheWire = (body.sessions || []).concat(body.children || []) + .map((row) => row.sessionParentId).filter(Boolean); + assert('a run whose parent is the current conversation but has no transcript of its own is a conversation', + sessionIds.includes('mvs_current_child') && !childIds.includes('mvs_current_child'), + sessionIds.join('|') + ' / ' + childIds.join('|')); + assert('and it names no parent, because the conversation it would hang off is not one the reader can pick', + !parentsOnTheWire.includes('mvs_current_parent'), + parentsOnTheWire.join('|')); + } finally { + await service.dispose(); + } +} catch (error) { + failures.push('the current conversation that has no transcript of its own -- ' + (error && error.message)); +} + +/* ---------- 12: a row whose runtime metadata disappears ---------- */ +// +// The descriptor behind a row comes out of a cache, and the runtime's facts -- the title, the +// kind, the parent -- used to be written ONTO that cached object. So a session whose row left the +// runtime database went on publishing the facts it had last time, while the validator, which +// reads the same rows, reported none of them: the body and the ETag disagreed about the same +// row, and the reader saw a parent and a kind for a session that no longer had either. +// +// The row is built, not patched, so the cached descriptor keeps the facts the filesystem gives. +// And the check is on the wire, after the caches have had their time, because the point is what +// the reader is shown. + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-vanished-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const parent = await makeSession(sessionsRoot, 0, 'mvs_vanish_parent', BASE_TIME - 10 * 60_000); + const child = await makeSession(sessionsRoot, 1, 'mvs_vanish_child', BASE_TIME - 20 * 60_000); + runtimeDb(home, [ + { + sessionId: 'mvs_vanish_parent', + relative: parent.relative, + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 10 * 60_000 + }, + { + sessionId: 'mvs_vanish_child', + relative: child.relative, + parent: 'mvs_vanish_parent', + kind: 'task', + purpose: 'local-background-task:child', + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 20 * 60_000 + } + ]); + + const service = await startService(home); + try { + const first = await (await fetch(service.base + '/api/sessions?limit=20')).json(); + const firstChild = (first.children || []).find((row) => row.id === 'mvs_vanish_child'); + assert('and the row carried the runtime facts while the runtime database still had them', + Boolean(firstChild) + && firstChild.sessionKind === 'task' + && firstChild.sessionParentId === 'mvs_vanish_parent', + JSON.stringify(firstChild)); + + // The runtime database forgets the run. Its transcript is still on disk, so the walk still + // finds it and the reader must still be able to reach it -- as a plain conversation, with + // nothing left over from the row that is gone. + const db = new DatabaseSync(join(home, 'v2', 'sqlite', 'runtime-state.sqlite')); + db.exec("delete from local_runtime_sessions where session_id = 'mvs_vanish_child'"); + db.close(); + // The map is held for three seconds and the descriptor for five, so the first wait is the one + // that decides when the row is seen to be gone. + await new Promise((resolve) => setTimeout(resolve, 3200)); + const second = await (await fetch(service.base + '/api/sessions?limit=20')).json(); + const secondChild = (second.sessions || []).find((row) => row.id === 'mvs_vanish_child'); + assert('a row whose runtime metadata disappears loses its title kind and parent across cache expiry', + Boolean(firstChild) + && Boolean(secondChild) + && secondChild.sessionKind === null + && secondChild.sessionParentId === null + && secondChild.sessionTag === null + && secondChild.label !== firstChild.label, + JSON.stringify([firstChild, secondChild])); + + // And the body the service reuses for the next request says the same thing: a body that is + // kept under its tag must not be able to keep publishing facts the map has already dropped. + const third = await (await fetch(service.base + '/api/sessions?limit=20')).json(); + const thirdChild = (third.sessions || []).find((row) => row.id === 'mvs_vanish_child'); + assert('and the body the service reuses for the next request says the same thing', + Boolean(secondChild) + && Boolean(thirdChild) + && thirdChild.sessionKind === null + && thirdChild.sessionParentId === null + && thirdChild.sessionTag === null + && thirdChild.label === secondChild.label, + JSON.stringify([secondChild, thirdChild])); + } finally { + await service.dispose(); + } +} catch (error) { + failures.push('a row whose runtime metadata disappears -- ' + (error && error.message)); +} + +/* ---------- 13: a directory that holds another session ---------- */ +// +// The directory name is where a session id is written down, and the manifest beside the +// transcript is where the runtime writes it too. The service reads a session's identity from the +// manifest, so a directory whose name says one session and whose manifest says another is the +// OTHER session's directory, whatever the folder is called. +// +// Two shortcuts took the name's word for it: the picker promoting the current conversation, and +// the 'latest' resolver choosing which transcript to serve. The recorded path was checked -- but +// it is only consulted when the directory is NOT in the window, so the window's own shortcut was +// never checked at all, and both of them could put one session's id on another session's +// messages. The route has to be asked over HTTP here, because the function is not the route. + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-mismatch-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + // The other session is the newest FILE, so the walk's own order starts with it, and the current + // conversation is the newest ROW. The two orders are independent, and the fixture needs them + // apart: the promotion has to be visible as a change of the first row. + const other = await makeSession(sessionsRoot, 0, 'mvs_mismatch_other', BASE_TIME); + const current = await makeSession(sessionsRoot, 1, 'mvs_mismatch_current', BASE_TIME - 60_000); + // The current conversation's own directory says it is the current conversation, and the manifest + // beside its transcript says it is somebody else. + await writeFile(join(current.dir, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_mismatch_stranger', createdAtMs: 1 }), 'utf8'); + runtimeDb(home, [ + { + sessionId: 'mvs_mismatch_other', + relative: other.relative, + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME - 60_000 + }, + { + sessionId: 'mvs_mismatch_current', + relative: current.relative, + workspace: 'D:\\private\\workspace-one', + updatedAtMs: BASE_TIME + } + ]); + + const service = await startService(home); + try { + const list = await (await fetch(service.base + '/api/sessions?limit=20')).json(); + const firstRow = (list.sessions || [])[0]; + assert('the current conversation is not promoted on a directory that holds another session', + Boolean(firstRow) && firstRow.id !== 'mvs_mismatch_stranger', + JSON.stringify(firstRow)); + const trajectory = await (await fetch(service.base + '/api/trajectory?session=latest')).json(); + assert('the latest route does not serve a directory that holds another session as the current one', + Boolean(trajectory.session) && trajectory.session.id !== 'mvs_mismatch_stranger', + JSON.stringify(trajectory.session)); + assert('and it does not claim the runtime binding for it either', + Boolean(trajectory.session) && trajectory.session.binding !== 'runtime-db', + String(trajectory.session && trajectory.session.binding)); + } finally { + await service.dispose(); + } +} catch (error) { + failures.push('a directory that holds another session -- ' + (error && error.message)); +} + console.log(pass + ' assertions passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs index ac50b7ac..19ce6fd2 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs @@ -435,14 +435,18 @@ assert('the kind badge is the one field that says what kind of session this is', strip !== null && /sessionTag: value\.sessionTag \?\? null,/.test(strip[0])); // The tag in the body and the tag in the ETag have to come from the same derivation, and the -// body has to write it on EVERY row. The descriptor comes out of a cache, so a row that loses -// its database entry keeps the tag it was given last time, while the validator -- which reads -// the same rows -- reports none. A body and an ETag that disagree about one row is how a badge -// outlives the fact it was derived from, and the page cannot see the disagreement: it only -// has the body. -const tagWrite = /descriptor\.sessionTag = meta \? sessionTagFor\(meta\.kind, meta\.purpose\) : null;/ +// body has to write it on EVERY row. A body and an ETag that disagree about one row is how a +// badge outlives the fact it was derived from, and the page cannot see the disagreement: it +// only has the body. +// +// The row is BUILT, not patched, so the tag is one of the fields the built row carries, and it +// is carried whether or not there is a row to derive it from. A descriptor that were written +// onto would keep the tag it was given last time -- it comes out of a cache and is shared -- and +// a row that loses its database entry would go on publishing it, while the validator, which +// reads the same rows, reports none. +const tagWrite = /sessionTag: meta \? sessionTagFor\(meta\.kind, meta\.purpose\) : null,/ .exec(serverText); -const tagInsideMeta = /if \(meta\) \{[^}]*descriptor\.sessionTag/.exec(serverText); +const tagInsideMeta = /if \(meta\) \{[^}]*sessionTag/.exec(serverText); assert('the kind tag is written on every row, not only on the rows that have one', tagWrite !== null && tagInsideMeta === null, tagWrite === null ? 'no unconditional tag write found' : String(tagInsideMeta && tagInsideMeta[0])); From 4e58d8d5f89a25deb10b6cbc705310541fe0365d Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 20:26:37 +0800 Subject: [PATCH 16/95] A directory is a session's directory only when its name and its manifest agree A published row took its identity from one place and its metadata from another: the id came from the manifest beside the transcript, while the title, the kind and the parent came from the id the directory's name decodes to, and the layer of a session was decided against that same decoded id. A directory whose name says one session and whose manifest says another therefore produced a row that named the manifest's session and carried the name's metadata, and a child session whose parent was that name was published under a parent no reader could pick -- neither control could reach it. The walk now refuses a directory whose name and manifest disagree. It is the judgment the recorded path and the latest binding already made, extended to the walk rather than written a second time, so one rule says which directory holds which session. A directory that fails it is not published at all: not in the session list, not in the run list, and not served by id. The id it claims is not among the published ones, so a run hanging off it becomes a conversation of its own, by the rule that a parent the reader cannot reach is not a parent. --- .../mmc-trajectory/miniapp/node/server.mjs | 45 +++++++++++++++++-- 1 file changed, 42 insertions(+), 3 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 1430dfe8..1aa0e342 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -793,6 +793,23 @@ export async function readSessionIdentity(dir) { }; } +/** + * The session id the manifest beside the transcript names, or null when there is no manifest. + * + * A directory's name and the manifest beside its transcript are two copies of the same id. This + * reads only the manifest -- the catalogue beside it says nothing about who the directory holds + * -- and it says nothing when the manifest is absent, because a directory with no manifest is + * judged by its name alone. + * + * @param {string} dir + * @returns {Promise} + */ +async function readManifestSessionId(dir) { + const manifest = await readJsonIfPresent(dir, 'manifest.json'); + const id = manifest && typeof manifest.sessionId === 'string' ? manifest.sessionId.trim() : ''; + return id || null; +} + /** * The whole candidate set from one walk, as two products of the same traversal. * @@ -862,11 +879,33 @@ export async function collectSessionCandidates(sessionsRoot) { } }; await walk(sessionsRoot, 0); - entries.sort((a, b) => b.lastActiveAt - a.lastActiveAt); + + // A directory is a session's directory only when its name and the manifest beside its + // transcript name that session: they are two copies of the same id, and the rule the recorded + // path and the 'latest' binding already follow -- the name AND the manifest -- is the one that + // belongs here too. + // + // A directory where the two disagree is nobody's. Publishing it would put one session's id on + // another session's transcript, and the row would mix the manifest's id with the name's + // metadata -- two sources in one row. So it does not enter the walk: the list does not publish + // it, the validator does not see it, and the trajectory route does not serve it. + // + // And the id it claims is not among the published ones, so it is not a parent the reader can + // pick -- a run hanging off it becomes a conversation of its own, by the rule that a parent the + // reader cannot reach is not a parent. + const verdicts = await Promise.all(entries.map(async (entry) => { + const claimed = decodeSessionIdFromDirName(basename(entry.dir)); + // A name that does not decode is still a real session; it is judged by its name alone. + if (!claimed) return true; + const manifestId = await readManifestSessionId(entry.dir); + return !(manifestId && manifestId !== claimed); + })); + const publishable = entries.filter((_entry, index) => verdicts[index]); + publishable.sort((a, b) => b.lastActiveAt - a.lastActiveAt); /** @type {Map>} */ const byDecodedId = new Map(); - for (const entry of entries) { + for (const entry of publishable) { const id = decodeSessionIdFromDirName(basename(entry.dir)); // A name that does not decode is still a real session; it just has no fast path and is // found the slow way. Dropping it here would turn "no fast path" into "does not exist". @@ -880,7 +919,7 @@ export async function collectSessionCandidates(sessionsRoot) { // what the picker used to draw from; the list of MAIN sessions needs a deeper pool than that, // because a sub-agent sits in that window and would otherwise take a main session' place. // The walk already stat'ed every one of these, so the deeper pool costs no extra disk work. - return { recentEntries: entries.slice(0, MAX_SESSION_LIST), allEntries: entries, byDecodedId, truncated }; + return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable, byDecodedId, truncated }; } /** From c02fde8b47edc472c1b0d7c2b84111669439b38c Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 20:26:47 +0800 Subject: [PATCH 17/95] The rules about which directory is a session's get owners that can fail Five assertions at the route level: the disagreeing directory is not in the list, its database row is not published under any id, a run of it becomes a conversation the reader can pick, it is not left in the run list either, and the route does not serve that directory either. One mutation turns the walk's refusal off, and it has to be caught by name. Two mutations had stopped changing anything, because the walk now removes those directories before the two shortcuts can reach them. They are anchored on the checks that are still alive instead, and the fixture that covered one of them was rebuilt so that it still covers it: the current conversation now has no directory of its own in the store, which leaves the recorded path as the only way in and the directory name the only thing that can refuse it. The check that says the id index is built from the decoded directory name was looking for that expression in the whole function, and the walk now decodes a name of its own when it decides whether a directory is publishable. It is pinned to the index construction instead. The four anchors in the reachability control were re-anchored to the walk as it is written now: a stale anchor stops that control before the mutations after it are run at all. --- .../tests/mutate-session-layers.py | 39 ++++++++++--- .../tests/mutate-session-reachability.py | 31 +++++----- .../tests/verify-session-layers.mjs | 57 +++++++++++++++++++ .../tests/verify-session-picker.mjs | 22 ++++--- .../tests/verify-session-reachability.mjs | 11 +++- 5 files changed, 129 insertions(+), 31 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py index e5a147a7..d4b3b710 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -215,20 +215,45 @@ # ---- whose directory this is ---- # - # The directory name is a claim; the manifest beside the transcript is the authority. Two - # shortcuts took the name's word for it -- the picker promoting the current conversation, and - # the 'latest' resolver choosing the transcript to serve -- and both could put one session's - # id on another session's messages. + # The directory name is a claim; the manifest beside the transcript is the authority. A + # directory where the two disagree is nobody's, and the walk now refuses it outright -- so + # the route never sees one, and the only way it can still reach such a directory is the + # recorded path, which is outside the walk. That is where the check below still has work to + # do, and these two controls now turn THAT off. + # + # The first one takes the recorded path's manifest check away: the picker then promotes the + # current conversation onto a directory that holds another session, and the guard says so. ("the picker promotes the current conversation on a directory that holds another session", - [(" currentEntry = await verifiedEntryForSessionId(entries, currentId);", - " currentEntry = entries.find((entry) => decodeSessionIdFromDirName(basename(entry.dir)) === currentId) || null;")], + [(" const identity = await readSessionIdentity(dir);\n" + " if (identity.id !== expectedSessionId) {\n" + " return { entry: null, rejected: true };\n" + " }", + " const identity = await readSessionIdentity(dir);")], "the current conversation is not promoted on a directory that holds another session"), + # The second one takes the walk's refusal away AND makes the 'latest' route take the + # directory's name for its word -- which is what the route did before it confirmed the + # manifest. Both together put the current session's id on somebody else's transcript. ("the latest route binds the current session to a directory that holds another session", - [(" entry = await verifiedEntryForSessionId(entries, current.sessionId);", + [(" return !(manifestId && manifestId !== claimed);", + " return true;"), + (" entry = await verifiedEntryForSessionId(entries, current.sessionId);", " entry = entries.find((candidate) => decodeSessionIdFromDirName(basename(candidate.dir)) === current.sessionId) ?? null;")], "the latest route does not serve a directory that holds another session as the current one"), + # The same rule asked of EVERY directory, not only of the two shortcuts. The walk drops a + # directory whose name and manifest disagree, because a row published from it would carry the + # manifest's id and the name's metadata -- two sources in one row -- and because the id it + # claims is not among the published ones, so a run hanging off it would be a run of a + # conversation the reader cannot select. + # + # Taking the walk's refusal away brings all of it back at once: the row with the mixed + # sources, and the run that hangs off a conversation nobody can pick. + ("a directory whose name and manifest disagree is published anyway", + [(" return !(manifestId && manifestId !== claimed);", + " return true;")], + "a directory whose name and manifest disagree is not published"), + # ---- the page ---- # # The two picks. The conversation pick holds conversations; the run pick holds the runs of diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py index 36c312d3..845b3637 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py @@ -11,11 +11,16 @@ answer a question it has no standing to answer. The two that look like harmless simplifications are the dangerous ones. Dropping the manifest -re-check on the index hit is faster and passes every existing guard, because nothing else in -the suite has a directory whose name disagrees with its manifest — it just serves the wrong -conversation under an id the reader typed. And replacing the `session_scan_budget_exhausted` -branch with a plain 404 is a one-line change that turns "we stopped looking" back into "it does -not exist", which is the same lie the original slice told, just further from its cause. +re-check on the index hit is faster, and since the walk started refusing a directory whose name +and its manifest disagree, no fixture in the suite can see it any more: such a directory is not +in the index to be served in the first place. What the re-check still holds is the row that is +about to be served -- a manifest rewritten under the walk is caught there instead of in the +reader's face -- and it is pinned by the shape of that line rather than by a fixture, because +no fixture rewrites a manifest between the walk and the route. The judgment that a disagreeing +directory never reaches the index at all is pinned behaviourally, in `mutate-session-layers.py`, +where the walk itself is broken. And replacing the `session_scan_budget_exhausted` branch with +a plain 404 is a one-line change that turns "we stopped looking" back into "it does not +exist", which is the same lie the original slice told, just further from its cause. Anchors are multi-line and the tree is LF, so they carry bare \\n and the files are read and written with newline=''. No anchor may contain a line ending of its own. @@ -43,13 +48,13 @@ # index dropped. The second layer's pool is the whole walk, and the reachability fix's # product is the index: losing either one is the shape this mutation puts back. ("the walk returns one truncated list again", - [(" return { recentEntries: entries.slice(0, MAX_SESSION_LIST), allEntries: entries, byDecodedId, truncated };", - " return { recentEntries: entries.slice(0, MAX_SESSION_LIST), allEntries: entries.slice(0, MAX_SESSION_LIST), byDecodedId: new Map(), truncated };")], + [(" return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable, byDecodedId, truncated };", + " return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable.slice(0, MAX_SESSION_LIST), byDecodedId: new Map(), truncated };")], "the id index is not the recent list", SERVER), ("the recent list is no longer capped", - [(" return { recentEntries: entries.slice(0, MAX_SESSION_LIST), allEntries: entries, byDecodedId, truncated };", - " return { recentEntries: entries, allEntries: entries, byDecodedId, truncated };")], + [(" return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable, byDecodedId, truncated };", + " return { recentEntries: publishable, allEntries: publishable, byDecodedId, truncated };")], "the recent list is still capped at MAX_SESSION_LIST", SERVER), # ---- the index stops being built from the walk ---- @@ -57,16 +62,16 @@ # Correct-looking, and now the picker pays for the reachability fix: the index is derived # from the slice, so a session outside the window has no fast path again. ("the index is derived from the recent list instead of the walk", - [(" for (const entry of entries) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" + [(" for (const entry of publishable) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" " // A name that does not decode is still a real session;", - " for (const entry of entries.slice(0, MAX_SESSION_LIST)) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" + " for (const entry of publishable.slice(0, MAX_SESSION_LIST)) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" " // A name that does not decode is still a real session;")], "the id index is not the recent list", SERVER), ("the index is no longer built at all", - [(" for (const entry of entries) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" + [(" for (const entry of publishable) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" " // A name that does not decode is still a real session;", - " for (const entry of entries) {\n const id = null;\n" + " for (const entry of publishable) {\n const id = null;\n" " // A name that does not decode is still a real session;")], "the index is built from the decoded directory name", SERVER), diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs index 03cc11c9..9e53bb35 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -1180,6 +1180,63 @@ try { failures.push('a directory that holds another session -- ' + (error && error.message)); } +/* ---------- 14: a directory whose name and manifest disagree ---------- */ +// +// A directory's name and the manifest beside its transcript are two copies of the same id. When +// they disagree, the directory is nobody's: publishing it puts one session's id on another +// session's transcript, and the row mixes the manifest's id with the name's metadata -- two +// sources in one row. +// +// And the id it claims is not among the published ones, so it is not a parent the reader can +// pick. A run hanging off it is a run of a conversation that is not in the list, and neither +// control can reach it -- the conversation pick has no row to select, and the run pick asks for +// the bucket of a conversation that is not there. +// +// The rule under test is the one the recorded path and the 'latest' binding already follow: the +// name AND the manifest have to name that session. Here it is asked of every directory. + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-join-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const ambiguous = await makeSession(sessionsRoot, 0, 'mvs_join_name', BASE_TIME); + const child = await makeSession(sessionsRoot, 1, 'mvs_join_child', BASE_TIME - 60_000); + // The directory says one session; the manifest beside its transcript says another. + await writeFile(join(ambiguous.dir, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_join_manifest', createdAtMs: 1 }), 'utf8'); + runtimeDb(home, [ + { sessionId: 'mvs_join_name', relative: ambiguous.relative, workspace: 'D:\\private\\workspace-one', updatedAtMs: BASE_TIME }, + { sessionId: 'mvs_join_child', relative: child.relative, parent: 'mvs_join_name', kind: 'task', updatedAtMs: BASE_TIME - 60_000 } + ]); + + const service = await startService(home); + try { + const list = await (await fetch(service.base + '/api/sessions?limit=20')).json(); + const rows = list.sessions || []; + const ids = rows.map((row) => row.id); + assert('a directory whose name and manifest disagree is not published', + !ids.includes('mvs_join_name') && !ids.includes('mvs_join_manifest'), + JSON.stringify(ids)); + assert('and its database row is not published under any id', + !rows.some((row) => row.label === '会话 name'), + JSON.stringify(rows.map((row) => row.label))); + const childRow = rows.find((row) => row.id === 'mvs_join_child'); + assert('a run of a directory that is not published becomes a conversation the reader can pick', + Boolean(childRow) && childRow.sessionParentId === null, + JSON.stringify(childRow)); + assert('and it is not left in the run list either', + !(list.children || []).some((row) => row.id === 'mvs_join_child'), + JSON.stringify(list.children || [])); + const served = await fetch(service.base + '/api/trajectory?session=mvs_join_name'); + assert('and the trajectory route does not serve that directory either', + served.status === 404, + 'status ' + served.status); + } finally { + await service.dispose(); + } +} catch (error) { + failures.push('a directory whose name and manifest disagree -- ' + (error && error.message)); +} + console.log(pass + ' assertions passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs index 19ce6fd2..771bc266 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs @@ -463,9 +463,9 @@ assert('the kind tag is written on every row, not only on the rows that have one // // - the manifest names another session -- the name is right and the manifest is wrong, which // is the case the identity function itself is there for; -// - the name names another session -- the manifest is right and the name is wrong, and the -// directory the walk already lists under its own name would be injected a second time -// under the id the manifest gives it. +// - the name names another session -- the manifest is right and the name is wrong. The walk +// does not list that directory at all any more (its two ids disagree), but the recorded path +// can still point at it, and the check that refuses it is the one on the name. // // The third case is the one that has to keep working: a manifest that agrees with the name is // accepted, because a refusal there would take the current conversation out of the list. @@ -493,10 +493,14 @@ try { } try { - // The name names another session and the manifest agrees with the CURRENT one. The - // directory is the walk's own row for a different session, so accepting it would list that - // one directory twice: once as the injected row and once as the walk's row. - const b = await store(31); + // The name names another session and the manifest agrees with the CURRENT one. + // + // The current conversation has no directory of its own in this store, so the path the runtime + // recorded for it is the only way it could be shown at all -- and that path points at a + // directory that belongs to somebody else. The manifest agrees, so the manifest check passes + // it; the NAME is the check that refuses it, and without that check this conversation would + // be listed with another session's transcript under it. + const b = await store(30); await writeFile(join(b.made[0].dir, 'manifest.json'), JSON.stringify({ sessionId: ID(30), createdAtMs: 1 }), 'utf8'); runtimeDb(b.home, ['parent_session_id', 'purpose', 'workspace_dir'], [ @@ -504,10 +508,10 @@ try { ]); const wrongName = await visibleFor(b.sessionsRoot, b.home, 20); assert('a recorded path whose directory name names another session is refused even when its manifest agrees', - new Set(dirsOf(wrongName.visible)).size === wrongName.visible.length, + !wrongName.visible.some((entry) => entry.dir === b.made[0].dir), dirsOf(wrongName.visible).join('|')); assert('and the list is still the plain recency order', - sameOrder(wrongName.visible, b.made.slice(0, 20)), + sameOrder(wrongName.visible, b.made.slice(1, 21)), dirsOf(wrongName.visible).join('|')); } catch (error) { failures.push('a recorded path whose directory name names another session is refused; ' diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs index 5cf86028..59f0ace0 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs @@ -22,7 +22,10 @@ * 3. The index is complete relative to the walk. A directory outside the recent 30 is still in * `byDecodedId`, which is the 116-session case measured above. * 4. Building the index costs no I/O. The walk already stat'ed `messages.jsonl` for every - * candidate, so the id comes from the name — this asserts no new file read was introduced. + * candidate, so the id comes from the name — this asserts no new file read was introduced + * by the index itself. The walk does read each candidate's manifest now, to decide whether + * that directory is publishable at all; that read belongs to the admission step, and the + * index still reads nothing. * 5. A decoded name is a claim, not a fact. `readSessionIdentity` prefers `manifest.sessionId`, * so the route re-reads the identity of the directory it picked and compares before serving. * Without this, a directory renamed on disk would be served under the wrong id. @@ -118,8 +121,12 @@ assert('the index maps an id to a directory that exists on disk', const walkBody = /export async function collectSessionCandidates\([\s\S]*?\n\}/.exec(SERVER); assert('collectSessionCandidates is the exported traversal', walkBody !== null); if (walkBody) { + // Scoped to the index construction rather than to the whole function: the walk reads each + // candidate's manifest to decide whether that directory is publishable at all, so the decoded + // name appears in the function more than once, and a check that only asks "does this function + // mention the decoded name" is satisfied by the admission step instead of the index. assert('the index is built from the decoded directory name', - walkBody[0].includes('decodeSessionIdFromDirName(basename(entry.dir))')); + /const byDecodedId = new Map\(\);[\s\S]{0,300}?const id = decodeSessionIdFromDirName\(basename\(entry\.dir\)\);/.test(walkBody[0])); assert('building the index opens no file of its own', !/byDecodedId[\s\S]{0,400}?\b(readFile|readJsonIfPresent|stat)\s*\(/.test(walkBody[0])); assert('the walk stats messages.jsonl exactly once per candidate', From 7f74dcc157570de2a5fff4d6cda61dd4b0f84e00 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 21:46:31 +0800 Subject: [PATCH 18/95] A directory is published under one id, and everything downstream reads that one A directory whose name does not decode is admitted by the walk on the strength of the manifest beside its transcript, and that admission is supposed to make it a session. The grouping set and the runtime-facts lookup still read the id decoded from the directory name, so such a directory was published under its manifest's id while the layer and the metadata did not know that id: a run hanging off it became a conversation of its own, and the parent lost the title and the kind the runtime database still had for it. The slow path of the explicit-id route had the mirror problem. It re-read the manifest of each candidate and admitted the id that manifest now claimed, without asking whether that id still agreed with the directory's name, so a candidate the walk had admitted under one id could be served under another one as soon as its manifest was rewritten inside the walk cache -- and the id it used to claim stopped working. The walk now decides the id a directory is published under once, when it reads the manifest it reads anyway, and puts it on the entry. The grouping set, the runtime-facts lookup, the row, the validator, the index the route resolves an id through and the recorded path all read that one field instead of decoding the name again. The index is keyed by the published id, so every id the list shows is an id the route can resolve; a directory whose name does not decode is keyed by the manifest's id, and one with no manifest at all by the folder name it can be reached by. A name that does not decode names no session, which is not the same as disagreeing with one, so such a directory is identified by its manifest alone. And the slow path now asks the same rule the walk asks, so a directory whose name and manifest disagree is served under neither id -- not the one its manifest now claims, and not the one the walk admitted it under. --- .../mmc-trajectory/miniapp/node/server.mjs | 199 +++++++++++------- 1 file changed, 128 insertions(+), 71 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 1aa0e342..672fb493 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -781,7 +781,7 @@ export async function readSessionIdentity(dir) { const manifest = await readJsonIfPresent(dir, 'manifest.json'); const catalog = await readJsonIfPresent(dir, 'history-catalog.json'); const manifestId = manifest && typeof manifest.sessionId === 'string' ? manifest.sessionId.trim() : ''; - const id = manifestId || decodeSessionIdFromDirName(basename(dir)) || basename(dir); + const id = publishedSessionId(manifestId, dir); const createdAtMs = manifest && Number.isFinite(Number(manifest.createdAtMs)) ? Number(manifest.createdAtMs) : null; const generations = parseCatalogGenerations(catalog); return { @@ -793,6 +793,26 @@ export async function readSessionIdentity(dir) { }; } +/** + * The id a directory is published under. + * + * One writer, and the same answer everywhere. The manifest beside the transcript names the + * session when it is there, and the manifest is the authority: a directory's name is a claim, + * not a fact. The name names the session when it decodes, and the folder's own name is the + * last resort -- a directory whose name does not decode and that has no manifest has no id of + * its own, and its folder name is what it can be reached by. + * + * A name that does not decode names NO session, which is not the same as disagreeing with one: + * such a directory is identified by its manifest alone, and it is still a real session. + * + * @param {string | null | undefined} manifestId + * @param {string} dir + * @returns {string} + */ +export function publishedSessionId(manifestId, dir) { + return manifestId || decodeSessionIdFromDirName(basename(dir)) || basename(dir); +} + /** * The session id the manifest beside the transcript names, or null when there is no manifest. * @@ -824,18 +844,20 @@ async function readManifestSessionId(dir) { * * - `recentEntries` — sorted by mtime, capped at `MAX_SESSION_LIST`. This is what the picker * shows, and it stays exactly as small as it was. - * - `byDecodedId` — every candidate directory, keyed by the id decoded from its own name. - * Decoding reads no file: the walk already stat'ed `messages.jsonl` for every candidate, so - * the index costs no I/O the walk was not doing anyway. + * - `bySessionId` — every candidate directory, keyed by the id that directory is published + * under. The walk decides that id once, when it reads the manifest it already reads to decide + * whether the directory is publishable at all, so the index costs no I/O the walk was not + * doing anyway -- and every id the list shows is an id this index can resolve. * * Two properties this must not lose: * - * - A decoded name is a *claim*, not a fact. `readSessionIdentity` prefers `manifest.sessionId` - * over the directory name, so every caller of this index still reads the identity of the one - * directory it picked and checks the id before serving it. - * - Two directories can decode to the same id. `Map.set` would silently keep whichever came - * last, which makes the answer depend on `readdir` order. Duplicates are therefore kept as a - * list and resolved against the manifests, not discarded and not guessed. + * - The id is a *decision*, not a guess. The walk refuses a directory whose name and manifest + * disagree, so a directory in this index is one session's directory; and every caller still + * reads the identity of the one directory it picked and checks the id before serving it, + * because the manifest can change under the walk. + * - Two directories can hold the same id. `Map.set` would silently keep whichever came last, + * which makes the answer depend on `readdir` order. Duplicates are therefore kept as a list + * and resolved against the manifests, not discarded and not guessed. * * `truncated` says the walk hit `MAX_SESSION_CANDIDATES` and stopped early. That budget is a * resource guard, and it must not be allowed to masquerade as "this session does not exist" — @@ -843,9 +865,9 @@ async function readManifestSessionId(dir) { * truthful one. * * @param {string} sessionsRoot - * @returns {Promise<{ recentEntries: Array<{ dir: string, sizeBytes: number, lastActiveAt: number }>, - * allEntries: Array<{ dir: string, sizeBytes: number, lastActiveAt: number }>, - * byDecodedId: Map>, + * @returns {Promise<{ recentEntries: Array<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string }>, + * allEntries: Array<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string }>, + * bySessionId: Map>, * truncated: boolean }>} */ export async function collectSessionCandidates(sessionsRoot) { @@ -893,33 +915,42 @@ export async function collectSessionCandidates(sessionsRoot) { // And the id it claims is not among the published ones, so it is not a parent the reader can // pick -- a run hanging off it becomes a conversation of its own, by the rule that a parent the // reader cannot reach is not a parent. - const verdicts = await Promise.all(entries.map(async (entry) => { + // + // The id every directory is PUBLISHED under is decided here, once. Everything downstream -- + // the title lookup, the layer, the row, the index the route resolves an id through -- reads it + // off the entry instead of decoding the name again, because a name that does not decode names + // no session at all: the manifest is then the only identity that directory has, and a parent + // identified that way still holds its runs, still has its title, and is still openable by the + // id the list shows. + const judged = await Promise.all(entries.map(async (entry) => { const claimed = decodeSessionIdFromDirName(basename(entry.dir)); - // A name that does not decode is still a real session; it is judged by its name alone. - if (!claimed) return true; const manifestId = await readManifestSessionId(entry.dir); - return !(manifestId && manifestId !== claimed); + // A name that does not decode names no session, so there is nothing for the manifest to + // disagree with: the manifest is the only identity there is. + const holds = !claimed || !manifestId || manifestId === claimed; + return { entry, holds, sessionId: publishedSessionId(manifestId, entry.dir) }; })); - const publishable = entries.filter((_entry, index) => verdicts[index]); + const publishable = judged.filter((one) => one.holds) + .map((one) => ({ ...one.entry, sessionId: one.sessionId })); publishable.sort((a, b) => b.lastActiveAt - a.lastActiveAt); - /** @type {Map>} */ - const byDecodedId = new Map(); + /** @type {Map>} */ + const bySessionId = new Map(); for (const entry of publishable) { - const id = decodeSessionIdFromDirName(basename(entry.dir)); - // A name that does not decode is still a real session; it just has no fast path and is - // found the slow way. Dropping it here would turn "no fast path" into "does not exist". - if (!id) continue; - const bucket = byDecodedId.get(id); + // The key is the id the entry is published under, so every id the list shows is an id this + // index can resolve. A name that does not decode names no session, and the folder name is + // what that directory can be reached by -- so it is in here under that name rather than + // missing, which would turn "no fast path" into "does not exist". + const bucket = bySessionId.get(entry.sessionId); if (bucket) bucket.push(entry); - else byDecodedId.set(id, [entry]); + else bySessionId.set(entry.sessionId, [entry]); } // `allEntries` is the same walk, uncut. The 30-entry window is what 'latest' falls back to and // what the picker used to draw from; the list of MAIN sessions needs a deeper pool than that, // because a sub-agent sits in that window and would otherwise take a main session' place. // The walk already stat'ed every one of these, so the deeper pool costs no extra disk work. - return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable, byDecodedId, truncated }; + return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable, bySessionId, truncated }; } /** @@ -1257,6 +1288,10 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSe if (dir !== root && !dir.startsWith(root + sep)) return { entry: null, rejected: true }; try { const fileStat = await stat(join(dir, 'messages.jsonl')); + // The id of the session this directory holds. It travels with the entry, like it does for + // the walk's own rows: a row built from a recorded path is published under an id too, and a + // row with no id is a row the page cannot group, label or select. + const identity = await readSessionIdentity(dir); if (expectedSessionId) { // The directory name is not the only place the session id is written down, and it is not // the authority either. The service reads a session's identity from the manifest the @@ -1273,18 +1308,25 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSe // another session's messages -- the same defect the name check exists for, one column // further in. // - // A directory with no manifest is judged by its name alone, which is the case that was - // always here. - if (decodeSessionIdFromDirName(basename(dir)) !== expectedSessionId) { + // A name that does not decode names NO session, and that is not the same as disagreeing + // with one: there is nothing for the manifest to disagree with, so the manifest decides + // alone. A directory with no manifest is judged by its name alone, which is the case that + // was always here. + const claimed = decodeSessionIdFromDirName(basename(dir)); + if (claimed && claimed !== expectedSessionId) { return { entry: null, rejected: true }; } - const identity = await readSessionIdentity(dir); if (identity.id !== expectedSessionId) { return { entry: null, rejected: true }; } } return { - entry: { dir, sizeBytes: fileStat.size, lastActiveAt: Math.floor(fileStat.mtimeMs) }, + entry: { + dir, + sizeBytes: fileStat.size, + lastActiveAt: Math.floor(fileStat.mtimeMs), + sessionId: identity.id, + }, rejected: false, }; } catch { @@ -1293,28 +1335,22 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSe } /** - * The entry that holds a session: found by the id its directory name decodes to, and confirmed by + * The entry that holds a session: found by the id the walk published it under, and confirmed by * the manifest beside its transcript. * - * The directory name is a claim; the manifest is the authority. `readSessionIdentity` prefers - * `manifest.sessionId`, so a directory whose name encodes one session and whose manifest names - * another is the OTHER session's directory, whatever the folder is called. Two shortcuts used to - * take the name's word for it -- the picker promoting the current conversation into the list, and - * the 'latest' resolver choosing which transcript to serve -- and both could put one session's id - * on another session's messages. + * The id is the walk's decision, and the walk refuses a directory whose name and manifest + * disagree -- so a candidate in the index holds the session it is indexed under. The manifest is + * read again anyway, because it can change under the walk, and a directory that has stopped + * holding that session must not be served as it. * - * So the name is used to FIND the candidate and the manifest is used to CONFIRM it. A candidate - * whose manifest names another session is not this session's entry, and the search goes on: another - * directory may carry this id honestly. - * - * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string }>} entries * @param {string} sessionId - * @returns {Promise<{ dir: string, sizeBytes: number, lastActiveAt: number } | null>} + * @returns {Promise<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string } | null>} */ export async function verifiedEntryForSessionId(entries, sessionId) { if (!sessionId) return null; for (const candidate of entries) { - if (decodeSessionIdFromDirName(basename(candidate.dir)) !== sessionId) continue; + if (candidate.sessionId !== sessionId) continue; const identity = await readSessionIdentity(candidate.dir); if (identity.id === sessionId) return candidate; } @@ -1348,7 +1384,7 @@ export async function verifiedEntryForSessionId(entries, sessionId) { * and promoting it here would put one session in both controls at once: the conversation pick * would offer a run, and the run pick would offer the same run again. * - * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string }>} entries * @param {{ sessionId: string, relativeDir?: string | null } | null} current * @param {Map} titles * @param {string} sessionsRoot @@ -1394,7 +1430,7 @@ export async function buildVisibleSessionList(entries, current, titles, sessions const same = []; const other = []; for (const entry of rest) { - const workspace = workspaceOfSession(titles, decodeSessionIdFromDirName(basename(entry.dir))); + const workspace = workspaceOfSession(titles, entry.sessionId); if (currentWorkspace !== null && sameWorkspacePath(workspace, currentWorkspace)) same.push(entry); else other.push(entry); } @@ -1449,9 +1485,9 @@ export async function buildVisibleSessionList(entries, current, titles, sessions * value. A copy would leave one of the three reading the uncorrected one, and the body and the * ETag would then describe two different lists. * - * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} entries + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string }>} entries * @param {Map} titles - * @returns {{ mains: Array<{ dir: string, sizeBytes: number, lastActiveAt: number }>, childrenByParent: Map> }} + * @returns {{ mains: Array<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string }>, childrenByParent: Map> }} */ export function splitSessionLayers(entries, titles) { const mains = []; @@ -1462,10 +1498,14 @@ export function splitSessionLayers(entries, titles) { // the current conversation among them, and a parent that is only a row is not something the // reader can pick, so a run hanging off it would be a run the reader cannot reach. The rule is // about what the reader can reach, so the membership test is about the walk. + // + // The membership is the id each directory is PUBLISHED under, which is not always the id its + // name decodes to: a name that does not decode names no session, and the manifest beside the + // transcript is then the only identity that directory has. A parent identified that way is in + // the list the reader can pick, so the runs hanging off it are its runs. const known = new Set(); for (const entry of entries) { - const id = decodeSessionIdFromDirName(basename(entry.dir)); - if (id) known.add(id); + if (entry.sessionId) known.add(entry.sessionId); } /** @@ -1547,7 +1587,7 @@ export function splitSessionLayers(entries, titles) { } for (const entry of entries) { - const id = decodeSessionIdFromDirName(basename(entry.dir)); + const id = entry.sessionId; const parent = id ? parentOf(id) : null; if (!parent) { mains.push(entry); @@ -2934,16 +2974,16 @@ export function etagMatches(header, etag) { * move would answer 304 for a body the caller never held — and the body it never held is the one * with the new sub-agent in it. * - * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} visible + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string }>} visible * @param {Map} titles * @param {number} limit - * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number }>} [children] + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string }>} [children] * @returns {string} */ export function sessionsValidator(visible, titles, limit, children) { const parts = ['limit=' + limit]; for (const entry of visible) { - const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir))); + const meta = titles.get(entry.sessionId); const title = meta ? String(meta.title ?? "") : ""; const kind = meta ? String(meta.kind ?? "") : ""; const parent = meta ? String(meta.parent ?? "") : ""; @@ -2965,7 +3005,7 @@ export function sessionsValidator(visible, titles, limit, children) { // each group. The parent id is in the tuple because the grouping is part of what the page // shows — the same sub-agent under a different main is a different page. for (const entry of children || []) { - const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir))); + const meta = titles.get(entry.sessionId); const title = meta ? String(meta.title ?? "") : ""; const kind = meta ? String(meta.kind ?? "") : ""; const parent = meta ? String(meta.parent ?? "") : ""; @@ -3504,11 +3544,14 @@ export async function start(context) { }; const describeSession = async (entry) => { - const cacheKey = `${entry.dir}|${entry.sizeBytes}|${entry.lastActiveAt}`; + // The id is the walk's decision, and the walk read the manifest to make it, so the row says + // the same thing about the session whether it is read here or by the layer or by the + // validator. The id is in the cache key: a manifest that changes changes the id, and a + // descriptor cached under the old one would publish the old session's id. + const cacheKey = `${entry.dir}|${entry.sizeBytes}|${entry.lastActiveAt}|${entry.sessionId}`; const cached = descriptorCache.get(cacheKey); if (cached && Date.now() - cached.at < SESSION_CACHE_TTL_MS) return cached.value; const identity = await readSessionIdentity(entry.dir); - const cacheId = `${entry.dir}|${identity.id}`; let peek = { messages: 0, userMessages: 0, assistantMessages: 0, toolResults: 0, toolCalls: 0, thinkingBlocks: 0, turns: 0 }; let label = null; let startedAt = identity.createdAtMs; @@ -3527,8 +3570,8 @@ export async function start(context) { context.logger.warn('miniapp.trajectory.read_error', { reason: 'peek_failed', code: errorCode(error) }); } const value = { - id: identity.id, - label: label ?? `会话 ${String(identity.id).slice(4, 12)}`, + id: entry.sessionId, + label: label ?? `会话 ${String(entry.sessionId).slice(4, 12)}`, // A prefix count is a prefix count. Reported as the session's message count it is simply // untrue, so when the prefix was not the whole file these are absent rather than approximate — // and the page's job is to print nothing rather than to print a number it cannot stand behind. @@ -3539,7 +3582,7 @@ export async function start(context) { // here, and /api/trajectory's is. Same name, different scope. Pinned by tests/verify-list-cache.mjs. sizeBytes: entry.sizeBytes, turnCount: peekIsWhole ? peek.turns : null, - cacheId, + cacheId: `${entry.dir}|${entry.sessionId}`, }; descriptorCache.set(cacheKey, { at: Date.now(), value }); if (descriptorCache.size > SESSION_CACHE_ENTRIES) { @@ -3652,7 +3695,11 @@ export async function start(context) { // Real titles from the runtime database, for every candidate and for the current conversation, // which may be outside the walk. The workspace column rides along because the visible set is // ordered by it, and the parent column rides along because the layer is decided by it. - const ids = allEntries.map((entry) => decodeSessionIdFromDirName(basename(entry.dir))).filter(Boolean); + // + // The ids are the ones the walk published those directories under -- not the ids their names + // happen to decode to. A directory whose name does not decode is identified by its manifest, + // and the runtime database knows it under that id. + const ids = allEntries.map((entry) => entry.sessionId).filter(Boolean); if (current && current.sessionId && ids.indexOf(current.sessionId) < 0) ids.push(current.sessionId); const titles = await sessionTitles(ids); @@ -3701,7 +3748,9 @@ export async function start(context) { // fields they carry: the same title, the same kind, the same tag, the same floor. const describeRow = async (entry) => { const descriptor = await describeSession(entry); - const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir))); + // The runtime facts are looked up under the id the row is published under, so the row's + // id and the row's facts are the same session by construction rather than by agreement. + const meta = titles.get(entry.sessionId); // The row is BUILT, not patched. The descriptor comes out of a cache and is shared, so // writing the runtime's facts onto it leaves the last writer's facts on it for the next // reader -- and a session whose row has left the runtime database goes on publishing the @@ -3767,7 +3816,7 @@ export async function start(context) { return { ok: false, status: 503, code: 'trajectory_unavailable', message: '未找到本地会话数据目录' }; } const entries = candidates.recentEntries; - if (entries.length === 0 && candidates.byDecodedId.size === 0) { + if (entries.length === 0 && candidates.bySessionId.size === 0) { return { ok: false, status: 503, code: 'trajectory_unavailable', message: '未找到本地会话数据目录' }; } const requested = url.searchParams.get('session'); @@ -3805,15 +3854,16 @@ export async function start(context) { } else { // Two ways in, and the order matters. // - // Fast path: the id was decoded from a directory name during the walk, so the directory is + // Fast path: the id is the one the walk published that directory under, so the directory is // already known and no scanning is needed. This is what makes a session outside the recent // 30 reachable at all — before this, the only directories this branch could see were the 30 // the picker shows, so 116 of 146 sessions on this machine answered 404 to their own id. // - // The decoded name is still only a claim. `readSessionIdentity` prefers `manifest.sessionId`, - // so a directory whose name disagrees with its manifest must not be served under the - // requested id; it falls through to the slow path instead, which re-checks every candidate. - const fastPath = candidates.byDecodedId.get(requested); + // The walk's decision is not the last word. The manifest can change under the walk, so the + // identity of the directory is read again and compared: a directory that has stopped + // holding this session is not served under it, and the request falls through to the slow + // path, which re-checks every candidate. + const fastPath = candidates.bySessionId.get(requested); let picked = null; if (fastPath) { // Duplicates are kept as a list rather than collapsed, because collapsing them with @@ -3835,7 +3885,14 @@ export async function start(context) { const matched = []; for (const candidate of entries) { const candidateIdentity = await readSessionIdentity(candidate.dir); - if (candidateIdentity.id === requested || basename(candidate.dir) === requested) { + // The same rule the walk applies, asked again of the directory that is in hand: a + // directory whose name decodes to one session and whose manifest names another is + // nobody's directory, and it is not served under either id. The walk decided this + // before the manifest changed, so the manifest is read again here and the answer + // follows the manifest that is on disk now. + const claimed = decodeSessionIdFromDirName(basename(candidate.dir)); + const holds = !claimed || claimed === candidateIdentity.id; + if (holds && (candidateIdentity.id === requested || basename(candidate.dir) === requested)) { matched.push({ entry: candidate, identity: candidateIdentity }); break; } From 3576cb55ec2bb668a28a81cb0cd1f28208bbf6d7 Mon Sep 17 00:00:00 2001 From: avatasia Date: Wed, 21 Oct 2026 21:46:46 +0800 Subject: [PATCH 19/95] The rules about the published id get owners that can fail A route-level fixture now asks the two cases that were open: a parent identified by its manifest alone keeps its runs, keeps its title, and the id the list shows is the id the route resolves; and a candidate whose manifest changes under a cached walk is served under neither id. The list fixture also says that every row the list publishes carries an id, since a row with no id cannot be grouped, labelled or selected. The reachability fixture was rewritten for the new index: it is keyed by the id the walk published, a directory whose name does not decode is in it under the id it is published under, and one with no manifest at all is in it under the folder name it can be reached by. The mutation controls grew with the rules. The layer control asks the grouping set, the runtime-facts lookup and the slow path to go back to the name, and asks the published row to lose its id; the picker control asks the recorded path to refuse a directory whose name does not decode and to build an entry with no id; the reachability control asks the index to go back to the decoded name. The anchors that the renames moved were re-anchored, because a stale anchor stops its control before the mutations after it are run at all. One mutation moved house. The rule that an entry built from a recorded path carries the session id sat in the layer control, where the visible set is built from the whole walk and the recorded path is only consulted for a directory the walk never reached -- so the mutation changed nothing any assertion could see. It sits in the picker control now, whose visible set is built from the window of thirty, where that path is reached. --- .../tests/mutate-session-layers.py | 68 +++++++++++--- .../tests/mutate-session-picker.py | 34 ++++++- .../tests/mutate-session-reachability.py | 45 +++++---- .../tests/mutate-sessions-etag.py | 16 ++-- .../tests/verify-session-layers.mjs | 91 +++++++++++++++++++ .../tests/verify-session-picker.mjs | 40 ++++++++ .../tests/verify-session-reachability.mjs | 74 +++++++++------ .../tests/verify-sessions-etag.mjs | 13 ++- 8 files changed, 306 insertions(+), 75 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py index d4b3b710..641b41af 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -224,36 +224,78 @@ # The first one takes the recorded path's manifest check away: the picker then promotes the # current conversation onto a directory that holds another session, and the guard says so. ("the picker promotes the current conversation on a directory that holds another session", - [(" const identity = await readSessionIdentity(dir);\n" - " if (identity.id !== expectedSessionId) {\n" + [(" if (identity.id !== expectedSessionId) {\n" " return { entry: null, rejected: true };\n" " }", - " const identity = await readSessionIdentity(dir);")], + " // the manifest is not checked here")], "the current conversation is not promoted on a directory that holds another session"), + # The row every page reads carries the id it is published under. Without it the row cannot be + # grouped, labelled or selected -- and the id is the only handle the page has on a session. + # + # The id has one writer on this path. The walk decided WHICH id a directory is published + # under, and the row takes it here; a version that leaves it out hands the page rows it + # cannot act on, and every promise the two controls make -- group a run under its + # conversation, label a row, select it -- is a promise about that id. + # + # The recorded path builds rows too, and it has its own mutation in the picker's control: the + # visible set there is built from the window of thirty, so the recorded path is reachable and + # a rule about it can be asked. Here the visible set is built from the WHOLE walk, so the + # recorded path is only consulted for a directory the walk never reached -- and a mutation + # there changed nothing any assertion could see. + ("the published row carries no id", + [(" id: entry.sessionId,", " id: null,")], + "every row the list publishes carries an id"), + # The second one takes the walk's refusal away AND makes the 'latest' route take the # directory's name for its word -- which is what the route did before it confirmed the # manifest. Both together put the current session's id on somebody else's transcript. ("the latest route binds the current session to a directory that holds another session", - [(" return !(manifestId && manifestId !== claimed);", - " return true;"), + [(" const holds = !claimed || !manifestId || manifestId === claimed;", + " const holds = true;"), (" entry = await verifiedEntryForSessionId(entries, current.sessionId);", " entry = entries.find((candidate) => decodeSessionIdFromDirName(basename(candidate.dir)) === current.sessionId) ?? null;")], "the latest route does not serve a directory that holds another session as the current one"), # The same rule asked of EVERY directory, not only of the two shortcuts. The walk drops a - # directory whose name and manifest disagree, because a row published from it would carry the - # manifest's id and the name's metadata -- two sources in one row -- and because the id it - # claims is not among the published ones, so a run hanging off it would be a run of a - # conversation the reader cannot select. + # directory whose name and manifest disagree, because the id it claims is not among the + # published ones, so a run hanging off it would be a run of a conversation the reader cannot + # select -- and because a route holding a cached walk could serve it under either id. # - # Taking the walk's refusal away brings all of it back at once: the row with the mixed - # sources, and the run that hangs off a conversation nobody can pick. + # Taking the walk's refusal away brings all of it back at once. ("a directory whose name and manifest disagree is published anyway", - [(" return !(manifestId && manifestId !== claimed);", - " return true;")], + [(" const holds = !claimed || !manifestId || manifestId === claimed;", + " const holds = true;")], "a directory whose name and manifest disagree is not published"), + # The id a directory is published under, asked of the layer. A parent whose folder name does + # not decode is identified by its manifest, and that is the id it is published under -- so it + # is among the sessions the reader can pick, and the runs hanging off it stay its runs. + # + # Reading the id off the name again puts it out of that set: the parent is not in the list, + # so the runs hanging off it are demoted into the conversation list, where the reader has to + # go looking for them. + ("the layer decides the parent from the directory name instead of the published id", + [(" for (const entry of entries) {\n if (entry.sessionId) known.add(entry.sessionId);\n }", + " for (const entry of entries) {\n const claimed = decodeSessionIdFromDirName(basename(entry.dir));\n if (claimed) known.add(claimed);\n }")], + "a published manifest-identified parent keeps its runs when its directory name does not decode"), + + # The same id, asked of the runtime facts. The row is looked up under the id the row is + # published under, so a parent identified by its manifest keeps the title and the kind the + # runtime database has for it. + ("the runtime facts are looked up by the directory name instead of the published id", + [(" const meta = titles.get(entry.sessionId);", + " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));")], + "and the parent keeps the title the runtime database has for it"), + + # And the same rule, asked of the route. The walk's refusal is a decision made when the walk + # ran; the manifest can change under a cached walk, and then the directory is nobody's + # directory and the route must not serve it under the id it now claims. + ("the route serves a candidate whose manifest disagrees with its name", + [(" const holds = !claimed || claimed === candidateIdentity.id;", + " const holds = true;")], + "a cached candidate whose manifest now disagrees with its name is not served under either id"), + # ---- the page ---- # # The two picks. The conversation pick holds conversations; the run pick holds the runs of diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py index 12e16f2e..4723b7a7 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py @@ -76,21 +76,49 @@ # The identity of the recorded path, in two parts now. The directory name is one of them: # a path that points at another session's directory is refused, and this is the check that # keeps one directory from being listed twice. The manifest is the other, and it is below. + # + # A name that does not decode is not a contradiction -- it says nothing at all -- so the + # check is "the name has to be this session IF it names one", and the manifest decides the + # rest. ("the recorded path is not checked against the session its directory name encodes", - [(" if (decodeSessionIdFromDirName(basename(dir)) !== expectedSessionId) {", + [(" if (claimed && claimed !== expectedSessionId) {", " if (false) {")], "a recorded path whose directory name names another session is refused even when its manifest agrees"), + ("the recorded path refuses a directory whose name does not decode", + [(" if (claimed && claimed !== expectedSessionId) {", + " if (claimed !== expectedSessionId) {")], + "a recorded path to a directory whose name does not decode is accepted when its manifest names the session"), + # The other part: the manifest the runtime wrote beside the transcript. It is the authority # on which session a directory holds -- the service reads a session's identity from it -- so # a directory whose name says one session and whose manifest says another is the OTHER # session's directory, and accepting it puts the current conversation's id on somebody # else's messages. ("the recorded path is not checked against the session its manifest names", - [(" const identity = await readSessionIdentity(dir);\n if (identity.id !== expectedSessionId) {", - " const identity = { id: expectedSessionId };\n if (identity.id !== expectedSessionId) {")], + [(" if (identity.id !== expectedSessionId) {\n" + " return { entry: null, rejected: true };\n" + " }", + " // the manifest is not checked here")], "a recorded path whose manifest names another session is refused"), + # The id travels with the entry. The recorded path is the ONLY way the current conversation can + # be in the list at all when its directory is outside the window the picker draws from, so an + # entry built from it that carries no id is a row the page cannot group, label or select. And + # the entry is the only holder of the id: the directory's name does not decode, so the + # manifest is the identity, and nothing else in the path says which session this is. + # + # This one is here, in the picker's control, because this is where the recorded path is + # reachable: the visible set is built from the window of thirty, and a session outside it is + # reached by its recorded path or not at all. It was in the layers' control before, where the + # visible set is built from the WHOLE walk -- so the recorded path is only consulted there for + # a directory the walk never reached, and the mutation changed nothing that any assertion + # could see. A mutation that cannot fail is worse than one that survives: it reports a rule + # nobody is checking. + ("the entry built from a recorded path carries no id", + [(" sessionId: identity.id,", " sessionId: null,")], + "the entry it builds carries the session id, not just the directory"), + # The partition, dropped. Every session comes back in one group, which is the plain recency # order -- the order the picker had before the current conversation was pinned. ("the workspace partition is dropped", diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py index 845b3637..c3df853f 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-reachability.py @@ -48,13 +48,13 @@ # index dropped. The second layer's pool is the whole walk, and the reachability fix's # product is the index: losing either one is the shape this mutation puts back. ("the walk returns one truncated list again", - [(" return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable, byDecodedId, truncated };", - " return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable.slice(0, MAX_SESSION_LIST), byDecodedId: new Map(), truncated };")], + [(" return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable, bySessionId, truncated };", + " return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable.slice(0, MAX_SESSION_LIST), bySessionId: new Map(), truncated };")], "the id index is not the recent list", SERVER), ("the recent list is no longer capped", - [(" return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable, byDecodedId, truncated };", - " return { recentEntries: publishable, allEntries: publishable, byDecodedId, truncated };")], + [(" return { recentEntries: publishable.slice(0, MAX_SESSION_LIST), allEntries: publishable, bySessionId, truncated };", + " return { recentEntries: publishable, allEntries: publishable, bySessionId, truncated };")], "the recent list is still capped at MAX_SESSION_LIST", SERVER), # ---- the index stops being built from the walk ---- @@ -62,26 +62,31 @@ # Correct-looking, and now the picker pays for the reachability fix: the index is derived # from the slice, so a session outside the window has no fast path again. ("the index is derived from the recent list instead of the walk", - [(" for (const entry of publishable) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" - " // A name that does not decode is still a real session;", - " for (const entry of publishable.slice(0, MAX_SESSION_LIST)) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" - " // A name that does not decode is still a real session;")], + [(" for (const entry of publishable) {\n // The key is the id the entry is published under,", + " for (const entry of publishable.slice(0, MAX_SESSION_LIST)) {\n // The key is the id the entry is published under,")], "the id index is not the recent list", SERVER), - ("the index is no longer built at all", - [(" for (const entry of publishable) {\n const id = decodeSessionIdFromDirName(basename(entry.dir));\n" - " // A name that does not decode is still a real session;", - " for (const entry of publishable) {\n const id = null;\n" - " // A name that does not decode is still a real session;")], - "the index is built from the decoded directory name", SERVER), + # The index keyed by nothing: every directory lands under one key, and the id the list shows + # is not an id the index can resolve. + ("the index is keyed by nothing", + [(" const bucket = bySessionId.get(entry.sessionId);", + " const bucket = bySessionId.get(null);")], + "the index is keyed by the id the walk published the entry under", SERVER), + + # The index keyed by the name again. A directory the manifest identifies is not in it under + # that id any more, so the id the list shows cannot be resolved. + ("the index is keyed by the decoded name again", + [(" else bySessionId.set(entry.sessionId, [entry]);", + " else bySessionId.set(decodeSessionIdFromDirName(basename(entry.dir)), [entry]);")], + "a directory the manifest identifies is published under that id", SERVER), # ---- duplicates silently resolved by directory order ---- # # `Map.set` keeps the last writer. `readdir` order is not sorted and is not stable across # platforms, so this is a Map that answers differently on two machines with the same store. ("duplicate ids are collapsed by directory order", - [(" const bucket = byDecodedId.get(id);\n if (bucket) bucket.push(entry);\n else byDecodedId.set(id, [entry]);", - " byDecodedId.set(id, [entry]);")], + [(" const bucket = bySessionId.get(entry.sessionId);\n if (bucket) bucket.push(entry);\n else bySessionId.set(entry.sessionId, [entry]);", + " bySessionId.set(entry.sessionId, [entry]);")], "duplicate ids are kept as a list rather than collapsed", SERVER), # ---- a decoded name promoted to an identity ---- @@ -139,8 +144,8 @@ # A second way to name a session, and the only one that covers a directory whose name does # not decode at all. ("a full directory basename can no longer name a session", - [(" if (candidateIdentity.id === requested || basename(candidate.dir) === requested) {", - " if (candidateIdentity.id === requested) {")], + [(" if (holds && (candidateIdentity.id === requested || basename(candidate.dir) === requested)) {", + " if (holds && candidateIdentity.id === requested) {")], "a full directory basename can still name a session", SERVER), # ---- the picker starts paying for the fix ---- @@ -153,7 +158,7 @@ # thirty-row window would take a main session' place. ("the list route starts building from the id index", [(" const allEntries = candidates.allEntries;\n", - " const allEntries = [...candidates.byDecodedId.values()].flat();\n")], + " const allEntries = [...candidates.bySessionId.values()].flat();\n")], "the list route builds from the whole walk", SERVER), # The trajectory route keeps the thirty most recent sessions for its 'latest' branch, and it @@ -161,7 +166,7 @@ # trajectory route is the one that pays for a descriptor per candidate. ("the trajectory route starts building from the id index", [(" const entries = candidates.recentEntries;\n", - " const entries = [...candidates.byDecodedId.values()].flat();\n")], + " const entries = [...candidates.bySessionId.values()].flat();\n")], "the trajectory route keeps the recent list for the latest branch", SERVER), # ---- the page names a session it is not showing ---- diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py index 8664f73d..cf77f044 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py @@ -37,10 +37,10 @@ # rather than guess which one it meant. ("a rename stops moving the tag", [(" for (const entry of visible) {\n" - " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const meta = titles.get(entry.sessionId);\n" " const title = meta ? String(meta.title ?? \"\") : \"\";\n", " for (const entry of visible) {\n" - " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const meta = titles.get(entry.sessionId);\n" " const title = \"\";\n")], "a renamed session moves the tag", SERVER), @@ -48,11 +48,11 @@ # tag is offering the reader a worker's session as if it were their conversation. ("a changed session kind stops moving the tag", [(" for (const entry of visible) {\n" - " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const meta = titles.get(entry.sessionId);\n" " const title = meta ? String(meta.title ?? \"\") : \"\";\n" " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n", " for (const entry of visible) {\n" - " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const meta = titles.get(entry.sessionId);\n" " const title = meta ? String(meta.title ?? \"\") : \"\";\n" " const kind = \"\";\n")], "a changed session kind moves the tag, even with no title", SERVER), @@ -160,12 +160,12 @@ # the title and the kind, so nothing ever asked the question. ("the validator stops covering the parent the body sends", [(" for (const entry of visible) {\n" - " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const meta = titles.get(entry.sessionId);\n" " const title = meta ? String(meta.title ?? \"\") : \"\";\n" " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n" " const parent = meta ? String(meta.parent ?? \"\") : \"\";\n", " for (const entry of visible) {\n" - " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const meta = titles.get(entry.sessionId);\n" " const title = meta ? String(meta.title ?? \"\") : \"\";\n" " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n" " const parent = \"\";\n")], @@ -190,7 +190,7 @@ # reader can see, and the 304 that follows is a statement about bytes nobody re-computed. ("the validator stops covering the kind badge the body sends", [(" for (const entry of visible) {\n" - " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const meta = titles.get(entry.sessionId);\n" " const title = meta ? String(meta.title ?? \"\") : \"\";\n" " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n" " const parent = meta ? String(meta.parent ?? \"\") : \"\";\n" @@ -200,7 +200,7 @@ " // did not move would answer 304 for a body the caller never held.\n" " const tag = meta ? String(sessionTagFor(meta.kind, meta.purpose) ?? \"\") : \"\";\n", " for (const entry of visible) {\n" - " const meta = titles.get(decodeSessionIdFromDirName(basename(entry.dir)));\n" + " const meta = titles.get(entry.sessionId);\n" " const title = meta ? String(meta.title ?? \"\") : \"\";\n" " const kind = meta ? String(meta.kind ?? \"\") : \"\";\n" " const parent = meta ? String(meta.parent ?? \"\") : \"\";\n" diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs index 9e53bb35..173484e0 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -1166,6 +1166,10 @@ try { assert('the current conversation is not promoted on a directory that holds another session', Boolean(firstRow) && firstRow.id !== 'mvs_mismatch_stranger', JSON.stringify(firstRow)); + assert('every row the list publishes carries an id', + (list.sessions || []).every((row) => typeof row.id === 'string' && row.id !== '') + && (list.children || []).every((row) => typeof row.id === 'string' && row.id !== ''), + JSON.stringify((list.sessions || []).map((row) => row.id))); const trajectory = await (await fetch(service.base + '/api/trajectory?session=latest')).json(); assert('the latest route does not serve a directory that holds another session as the current one', Boolean(trajectory.session) && trajectory.session.id !== 'mvs_mismatch_stranger', @@ -1237,6 +1241,93 @@ try { failures.push('a directory whose name and manifest disagree -- ' + (error && error.message)); } +/* ---------- 15: the identity a directory is published under ---------- */ +// +// A directory's name may not decode at all. The runtime writes the id into the folder name, but +// the manifest beside the transcript is the authority, and a name that decodes to nothing names +// no session -- so the manifest is the only identity such a directory has, and that identity is +// the id it is published under. +// +// Two things break when the service reads the id off the name instead: the parent is not among +// the sessions the reader can pick, so the runs hanging off it are demoted into the conversation +// list, and the row loses the title and the kind the runtime database has for it. +// +// And the rule has to hold when the manifest changes under a walk that is already cached: a +// directory whose manifest no longer agrees with its name is nobody's directory, and the route +// must not serve it under the id it now claims -- nor under the one it used to claim. + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-identified-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + // The parent's folder name does not decode. The manifest beside its transcript is the only + // identity it has. + const legacy = join(sessionsRoot, '2026', '10', '20', '00-00-00-session_legacy'); + await mkdir(legacy, { recursive: true }); + await writeFile(join(legacy, 'messages.jsonl'), + JSON.stringify({ role: 'user', content: 'p' }) + '\n', 'utf8'); + await writeFile(join(legacy, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_identified_parent', createdAtMs: 1 }), 'utf8'); + const legacyStamp = new Date(BASE_TIME); + await utimes(join(legacy, 'messages.jsonl'), legacyStamp, legacyStamp); + const child = await makeSession(sessionsRoot, 1, 'mvs_identified_child', BASE_TIME - 60_000); + runtimeDb(home, [ + { sessionId: 'mvs_identified_parent', relative: '2026/10/20/00-00-00-session_legacy', workspace: 'D:\\private\\workspace-one', updatedAtMs: BASE_TIME }, + { sessionId: 'mvs_identified_child', relative: child.relative, parent: 'mvs_identified_parent', kind: 'task', updatedAtMs: BASE_TIME - 60_000 } + ]); + + const service = await startService(home); + try { + const list = await (await fetch(service.base + '/api/sessions?limit=20')).json(); + const rows = list.sessions || []; + const parentRow = rows.find((row) => row.id === 'mvs_identified_parent'); + const childRow = (list.children || []).find((row) => row.id === 'mvs_identified_child'); + assert('a published manifest-identified parent keeps its runs when its directory name does not decode', + Boolean(parentRow) && Boolean(childRow) && childRow.sessionParentId === 'mvs_identified_parent', + JSON.stringify([parentRow, childRow])); + assert('and the parent keeps the title the runtime database has for it', + Boolean(parentRow) && parentRow.label === '会话 rent', + JSON.stringify(parentRow)); + const served = await fetch(service.base + '/api/trajectory?session=mvs_identified_parent'); + assert('and the id the list shows is the id the route resolves', + served.status === 200, + 'status ' + served.status); + } finally { + await service.dispose(); + } +} catch (error) { + failures.push('a parent identified by its manifest -- ' + (error && error.message)); +} + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-swapped-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const one = await makeSession(sessionsRoot, 0, 'mvs_swapped_one', BASE_TIME); + await writeFile(join(one.dir, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_swapped_one', createdAtMs: 1 }), 'utf8'); + runtimeDb(home, [ + { sessionId: 'mvs_swapped_one', relative: one.relative, updatedAtMs: BASE_TIME } + ]); + + const service = await startService(home); + try { + // The walk is primed while the manifest still agrees with the name. + await (await fetch(service.base + '/api/sessions?limit=20')).json(); + // Now the manifest beside the same transcript names another session. The walk is still + // cached, so the route is holding the directory it admitted a moment ago. + await writeFile(join(one.dir, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_swapped_other', createdAtMs: 1 }), 'utf8'); + const underNew = await fetch(service.base + '/api/trajectory?session=mvs_swapped_other'); + const underOld = await fetch(service.base + '/api/trajectory?session=mvs_swapped_one'); + assert('a cached candidate whose manifest now disagrees with its name is not served under either id', + underNew.status === 404 && underOld.status === 404, + JSON.stringify([underNew.status, underOld.status])); + } finally { + await service.dispose(); + } +} catch (error) { + failures.push('a manifest that changes under a cached walk -- ' + (error && error.message)); +} + console.log(pass + ' assertions passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs index 771bc266..6109ce35 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-picker.mjs @@ -538,6 +538,46 @@ try { + String(error && error.message ? error.message : error)); } +/* ---------- 14: a recorded path to a directory the name does not identify ---------- */ +// +// The recorded path is consulted when the current conversation is not among the sessions the +// picker draws from, and then it is the only way that conversation can be shown at all. A +// directory's name may not decode -- the manifest beside its transcript names the session, and +// the name says nothing. The name is a claim, and an empty claim is not a contradiction: refusing +// the path because of it would take the current conversation out of the list. + +try { + const d = await store(31); + // One directory older than every other, so it is outside the window of thirty the picker + // draws from, and its name does not decode: the manifest beside its transcript is the only + // identity it has. + const legacy = join(d.sessionsRoot, '2026', '10', '20', '00-00-00-session_legacy'); + await mkdir(legacy, { recursive: true }); + await writeFile(join(legacy, 'messages.jsonl'), + JSON.stringify({ role: 'user', content: 'x' }) + '\n', 'utf8'); + await writeFile(join(legacy, 'manifest.json'), + JSON.stringify({ sessionId: ID(30), createdAtMs: 1 }), 'utf8'); + const stamp = new Date(BASE_TIME - 31 * 60_000); + await utimes(join(legacy, 'messages.jsonl'), stamp, stamp); + runtimeDb(d.home, ['parent_session_id', 'purpose', 'workspace_dir'], [ + { sessionId: ID(30), relative: '2026/10/20/00-00-00-session_legacy', workspace: 'workspace-one' }, + ]); + const legacyPath = await visibleFor(d.sessionsRoot, d.home, 20); + assert('a recorded path to a directory whose name does not decode is accepted when its manifest names the session', + legacyPath.visible[0]?.dir === legacy, + dirsOf(legacyPath.visible).slice(0, 3).join('|')); + // The directory is only half of it. The name of that directory says nothing, so the manifest is + // the only identity it has -- and the entry has to CARRY that identity, or the row the page + // draws has no id to group, label or select it by. The entry is built here and nowhere else, so + // this is the only place that can see whether it kept the id. + assert('the entry it builds carries the session id, not just the directory', + legacyPath.visible[0]?.sessionId === ID(30), + String(legacyPath.visible[0]?.sessionId)); +} catch (error) { + failures.push('a recorded path to a directory whose name does not decode -- ' + + String(error && error.message ? error.message : error)); +} + for (const home of homes) rmSync(home, { recursive: true, force: true }); console.log('[session-picker] ' + pass + ' passed, ' + failures.length + ' failed'); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs index 59f0ace0..6555cd87 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-reachability.mjs @@ -20,19 +20,20 @@ * 2. The index is not the list. `recentEntries` stays at `MAX_SESSION_LIST`, so widening id * resolution costs the picker nothing. * 3. The index is complete relative to the walk. A directory outside the recent 30 is still in - * `byDecodedId`, which is the 116-session case measured above. + * `bySessionId`, which is the 116-session case measured above. * 4. Building the index costs no I/O. The walk already stat'ed `messages.jsonl` for every - * candidate, so the id comes from the name — this asserts no new file read was introduced - * by the index itself. The walk does read each candidate's manifest now, to decide whether - * that directory is publishable at all; that read belongs to the admission step, and the - * index still reads nothing. - * 5. A decoded name is a claim, not a fact. `readSessionIdentity` prefers `manifest.sessionId`, - * so the route re-reads the identity of the directory it picked and compares before serving. - * Without this, a directory renamed on disk would be served under the wrong id. + * candidate, so the id comes from the walk's own decision — this asserts no new file read was + * introduced by the index itself. The walk does read each candidate's manifest now, to decide + * whether that directory is publishable at all; that read belongs to the admission step, and + * the index still reads nothing. + * 5. The published id is a decision, not a guess. The manifest can change under the walk, so + * the route re-reads the identity of the directory it picked and compares before serving. + * Without this, a directory whose manifest changed would be served under the wrong id. * 6. Duplicate ids are not collapsed. `Map.set` would keep whichever came last and make the * answer depend on `readdir` order; the index keeps a list so the manifests can decide. - * 7. A name that does not decode is still a real session. Dropping it from the index would turn - * "no fast path" into "does not exist". + * 7. A name that does not decode is still a real session. It names no session, so the manifest + * beside it is the only identity it has, and the folder name is what it is reached by. + * Dropping it from the index would turn "reached the slow way" into "does not exist". * 8. The scan budget is a resource guard, not a correctness boundary. When the walk stops early * the route must say so, because "the budget ran out" and "no such session" are different * answers and only one of them is true. @@ -106,39 +107,39 @@ assert('the recent list does not reach the 31st session', * ===================================================================== */ assert('the id index is not the recent list', - candidates.byDecodedId instanceof Map && candidates.byDecodedId.size === NAME_COUNT); + candidates.bySessionId instanceof Map && candidates.bySessionId.size === NAME_COUNT); assert('a session outside the recent list is still resolvable by id', - candidates.byDecodedId.has(made[35].id)); + candidates.bySessionId.has(made[35].id)); assert('the newest session is resolvable by id too', - candidates.byDecodedId.has(made[0].id)); + candidates.bySessionId.has(made[0].id)); assert('the index maps an id to a directory that exists on disk', - (candidates.byDecodedId.get(made[35].id) || [])[0]?.dir === made[35].dir); + (candidates.bySessionId.get(made[35].id) || [])[0]?.dir === made[35].dir); /* ===================================================================== - * 4 — the index is built from names, not from new reads + * 4 — the index is keyed by the id the walk published, not by new reads * ===================================================================== */ const walkBody = /export async function collectSessionCandidates\([\s\S]*?\n\}/.exec(SERVER); assert('collectSessionCandidates is the exported traversal', walkBody !== null); if (walkBody) { // Scoped to the index construction rather than to the whole function: the walk reads each - // candidate's manifest to decide whether that directory is publishable at all, so the decoded - // name appears in the function more than once, and a check that only asks "does this function - // mention the decoded name" is satisfied by the admission step instead of the index. - assert('the index is built from the decoded directory name', - /const byDecodedId = new Map\(\);[\s\S]{0,300}?const id = decodeSessionIdFromDirName\(basename\(entry\.dir\)\);/.test(walkBody[0])); + // candidate's manifest to decide whether that directory is publishable at all, so a check that + // only asks "does this function mention the id" is satisfied by the admission step instead of + // the index. + assert('the index is keyed by the id the walk published the entry under', + /const bySessionId = new Map\(\);[\s\S]{0,600}?bySessionId\.get\(entry\.sessionId\)/.test(walkBody[0])); assert('building the index opens no file of its own', - !/byDecodedId[\s\S]{0,400}?\b(readFile|readJsonIfPresent|stat)\s*\(/.test(walkBody[0])); + !/bySessionId[\s\S]{0,400}?\b(readFile|readJsonIfPresent|stat)\s*\(/.test(walkBody[0])); assert('the walk stats messages.jsonl exactly once per candidate', (walkBody[0].match(/stat\(join\(child, 'messages\.jsonl'\)\)/g) || []).length === 1); } /* ===================================================================== - * 5 — a decoded name is a claim; the manifest gets the last word + * 5 — the published id is a decision; the manifest gets the last word * ===================================================================== */ -assert('the explicit-id branch consults the decoded-id index', - /const fastPath = candidates\.byDecodedId\.get\(requested\);/.test(SERVER)); +assert('the explicit-id branch consults the id index', + /const fastPath = candidates\.bySessionId\.get\(requested\);/.test(SERVER)); assert('an index hit still reads the directory identity', /for \(const candidate of fastPath\) \{\s*\n\s*const candidateIdentity = await readSessionIdentity\(candidate\.dir\);/.test(SERVER)); assert('a directory whose manifest disagrees is not served under the requested id', @@ -147,7 +148,7 @@ assert('an ambiguous id is not resolved by directory order', /if \(verified\.length === 1\) picked = verified\[0\];\s*\n\s*else if \(verified\.length > 1\)/.test(SERVER) && /reason: 'ambiguous_session_id'/.test(SERVER)); assert('duplicate ids are kept as a list rather than collapsed', - /const bucket = byDecodedId\.get\(id\);\s*\n\s*if \(bucket\) bucket\.push\(entry\);\s*\n\s*else byDecodedId\.set\(id, \[entry\]\);/.test(SERVER)); + /const bucket = bySessionId\.get\(entry\.sessionId\);\s*\n\s*if \(bucket\) bucket\.push\(entry\);\s*\n\s*else bySessionId\.set\(entry\.sessionId, \[entry\]\);/.test(SERVER)); /* ===================================================================== * 6 — a name that does not decode is still a session @@ -163,9 +164,26 @@ await utimes(join(odd, 'messages.jsonl'), oddStamp, oddStamp); const withOdd = await collectSessionCandidates(root); assert('a session whose name does not decode is still a candidate', withOdd.recentEntries.some((e) => e.dir === odd)); -assert('a non-decoding name simply gets no fast path, and is not treated as absent', +assert('a non-decoding name is published under the folder name it is reached by, and is not treated as absent', decodeSessionIdFromDirName('00-00-99-000-session_!!!not-base64!!!') === null - && withOdd.byDecodedId.size === NAME_COUNT); + && withOdd.bySessionId.size === NAME_COUNT + 1 + && (withOdd.bySessionId.get('00-00-99-000-session_!!!not-base64!!!') || [])[0]?.dir === odd); + +// A directory the manifest identifies and the name does not. The manifest is the only identity +// such a directory has, so that identity is the id it is published under -- and an id the list +// shows is an id the route has to be able to resolve. +const named = join(root, '2026', '01', '03', '00-00-98-000-session_legacy'); +await mkdir(named, { recursive: true }); +await writeFile(join(named, 'messages.jsonl'), JSON.stringify({ role: 'user', content: 'named' }) + '\n', 'utf8'); +await writeFile(join(named, 'manifest.json'), JSON.stringify({ sessionId: 'sess_manifest_only' }), 'utf8'); +const namedStamp = new Date(Date.UTC(2026, 0, 3)); +await utimes(join(named, 'messages.jsonl'), namedStamp, namedStamp); +const withNamed = await collectSessionCandidates(root); +assert('a directory the manifest identifies is published under that id', + (withNamed.bySessionId.get('sess_manifest_only') || [])[0]?.dir === named + && decodeSessionIdFromDirName('00-00-98-000-session_legacy') === null); +assert('and the id the walk published it under is the id the entry carries', + (withNamed.allEntries.find((e) => e.dir === named) || {}).sessionId === 'sess_manifest_only'); /* ===================================================================== * 7 — the scan budget is a guard, and it says so @@ -200,7 +218,7 @@ assert('a walk that runs out of candidate budget reports itself as truncated', assert('the budget still bounds the recent list', exhausted.recentEntries.length === 30 && exhausted.recentEntries.length <= budget); assert('an over-budget store indexes no more than the budget allows', - exhausted.byDecodedId.size <= budget && exhausted.byDecodedId.size > 0); + exhausted.bySessionId.size <= budget && exhausted.bySessionId.size > 0); await rm(budgetRoot, { recursive: true, force: true }); /* ===================================================================== diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs index 0b6ca150..739e8c00 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs @@ -55,9 +55,11 @@ function dirFor(id) { const A = 'mvs_0000000000000000000000000000aaaa'; const B = 'mvs_0000000000000000000000000000bbbb'; +// Shaped like the entries the walk returns: the id the directory is published under travels with +// it, because that is the id the runtime facts are looked up under. const visible = [ - { dir: dirFor(A), sizeBytes: 1024, lastActiveAt: 1700000000 }, - { dir: dirFor(B), sizeBytes: 2048, lastActiveAt: 1700000001 }, + { dir: dirFor(A), sizeBytes: 1024, lastActiveAt: 1700000000, sessionId: A }, + { dir: dirFor(B), sizeBytes: 2048, lastActiveAt: 1700000001, sessionId: B }, ]; const noTitles = new Map(); const titled = new Map([ @@ -139,7 +141,12 @@ assert('a null title and an absent one agree — they render the same row', // A session entering or leaving the visible slice. assert('one more session in view moves the tag', - sessionsValidator([...visible, { dir: dirFor('mvs_0000000000000000000000000000cccc'), sizeBytes: 1, lastActiveAt: 1 }], noTitles, 20) + sessionsValidator([...visible, { + dir: dirFor('mvs_0000000000000000000000000000cccc'), + sizeBytes: 1, + lastActiveAt: 1, + sessionId: 'mvs_0000000000000000000000000000cccc', + }], noTitles, 20) !== sessionsValidator(visible, noTitles, 20)); /* ---------- 2: the request itself is part of the identity ---------- */ From ca78443ae68175a6900a3772c8a3a5b419c8ce3d Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 01:05:45 +0800 Subject: [PATCH 20/95] A cache that holds an answer has to say which answer it is holding The list's validator covered the directory, the size, the mtime, the title, the kind, the parent and the tag -- but not the id the walk published. A manifest rewritten from one id to another left the validator where it was, so a conditional request was answered 304 with the tag of the old id, and the body the service reused for the next request kept publishing the old id while the route reported that id as 404. The trajectory cache key had the same gap. It carried the directory, the lineage, the size, the mtime, the catalog bytes and the binding, but not the identity the body is about: one directory holding two ids shared one entry, so the body of the first id could be served for the second, with the same ETag. And the `latest` fallback asked no question at all. The explicit-id route asks, on both its fast and its slow path, whether the directory still holds the session it was published under; the fallback took the first candidate of the cached walk instead, so a candidate whose manifest no longer agreed with its name was served anyway. The validator now names the published id, for the main sessions and for the sub-agents. The trajectory cache key now names the identity the body is about. The fallback now asks the same question the other two paths ask, and reports the session as unavailable when no candidate still holds one. --- .../mmc-trajectory/miniapp/node/server.mjs | 62 +++++++++++++++++-- 1 file changed, 57 insertions(+), 5 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 672fb493..5a7b6b4b 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -1357,6 +1357,33 @@ export async function verifiedEntryForSessionId(entries, sessionId) { return null; } +/** + * The newest candidate that still holds the session it was published under. + * + * The walk's admission is a decision made when the walk ran, and the manifest beside a + * transcript can change under it. The explicit-id routes ask again before they serve a + * candidate -- the fast path re-reads the identity, the slow path asks the same question the + * walk asks -- so a directory whose manifest no longer agrees with its name is refused by + * both. The `latest` fallback used to take the first entry of the window as it stood, which + * served exactly that directory: the same transcript, under the id its manifest now claims, + * while every explicit id for it was refused. One rule, one question, three places. + * + * The question is the one the id itself answers: the id the directory holds now -- the + * manifest's, or the name's when the manifest is gone -- has to be the id it was published + * under. A manifest that changes from one id to another moves that answer and nothing else, + * which is exactly why it has to be asked again rather than read off the walk. + * + * @param {ReadonlyArray<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string }>} entries + * @returns {Promise<{ dir: string, sizeBytes: number, lastActiveAt: number, sessionId: string } | null>} + */ +export async function firstHoldingEntry(entries) { + for (const candidate of entries) { + const identity = await readSessionIdentity(candidate.dir); + if (identity.id === candidate.sessionId) return candidate; + } + return null; +} + /** * The visible set the picker shows, and the order it is shown in. * @@ -2999,7 +3026,14 @@ export function sessionsValidator(visible, titles, limit, children) { // different separator cannot fix that, because no character is excluded from a TEXT value. // JSON escapes the ambiguous characters, so the tuple's boundaries survive anything the store // can return, and no field needs a second representation to be safe. - parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag])); + // + // The published id is in the tuple because it is in the payload: the row's id IS the id the + // walk published this directory under, and the body the reader receives says it. A manifest + // that changes from one id to another moves the published id and moves nothing else here -- + // the directory, the transcript, the runtime row and the mtime are all the same -- so without + // this field the validator would answer 304 for a body that names a session the reader is no + // longer looking at, and the reused body would keep publishing the old id indefinitely. + parts.push(JSON.stringify([entry.sessionId, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag])); } // The sub-agents, in the order the payload carries them: most recently active first inside // each group. The parent id is in the tuple because the grouping is part of what the page @@ -3010,7 +3044,7 @@ export function sessionsValidator(visible, titles, limit, children) { const kind = meta ? String(meta.kind ?? "") : ""; const parent = meta ? String(meta.parent ?? "") : ""; const tag = meta ? String(sessionTagFor(meta.kind, meta.purpose) ?? "") : ""; - parts.push(JSON.stringify([parent, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, tag])); + parts.push(JSON.stringify([entry.sessionId, parent, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, tag])); } return parts.join("\n"); } @@ -3037,8 +3071,15 @@ export function trajectoryCacheFamily(dir, filter) { * The cache key for one rendered trajectory. Every component comes from a `stat`, a small * manifest/catalog file, or the already-resolved session binding — never from reading * `messages.jsonl`, which is the whole point: a matching key proves the file did not grow. + * + * The identity the directory holds is in it, because the body says which session it is about. + * A manifest that changes from one id to another leaves the directory, the transcript, the + * lineage and the mtime exactly as they were, so a key made of those would match and the + * cached body of the OLD id would be served for the NEW one — a body whose `session.id` names + * a session nobody asked for. The key has to cover the same facts the body carries. + * * @param {{ dir: string, sizeBytes: number, lastActiveAt: number }} entry - * @param {{ catalogBytes?: number | null }} identity + * @param {{ id?: string | null, catalogBytes?: number | null }} identity * @param {any} binding * @returns {string} */ @@ -3060,7 +3101,7 @@ export function trajectoryCacheKey(entry, identity, binding, filter) { .join(','); // The requested slice is part of the answer too: the same session stitched and unfiltered is // two different bodies and must never share an entry. - return `${trajectoryCacheFamily(entry.dir, filter)}${lineage}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`; + return `${trajectoryCacheFamily(entry.dir, filter)}${identity.id ?? ''}|${lineage}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`; } /** * The `?since=` cursor, or null when there is not a usable one. @@ -3850,7 +3891,18 @@ export async function start(context) { } if (entry) binding = current; } - if (!entry) entry = entries[0]; + if (!entry) { + // The window's first entry -- but only while it still holds the session it was + // published under. The explicit-id routes ask that again before they serve a + // candidate, and the manifest can change under the walk, so the fallback has to ask + // it too: a directory whose manifest no longer agrees with its name is nobody's + // directory, and serving it here while both explicit ids refuse it would be the + // same transcript under an id the reader cannot ask for. + entry = await firstHoldingEntry(entries); + if (!entry) { + return { ok: false, status: 404, code: 'session_not_found', message: '找不到可用的会话' }; + } + } } else { // Two ways in, and the order matters. // From eea9c0fc6e21c02c4640613e7ac78ced66445041 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 01:05:56 +0800 Subject: [PATCH 21/95] Guard the id in the validator, in the cache key and in the fallback Each of the three rules gets a check of its own, and each check has a mutation control that is caught by name. - the list publishes the id the manifest names; a changed published id moves the list validator and replaces its body, and the body the service reuses for the next request says the same thing; `latest` never serves a cached candidate whose manifest now disagrees with its name; an identity-only manifest change cannot reuse another session identity's trajectory body - the kind the runtime database holds for a parent session survives - a changed published session id moves the tag - the cache key is read as the set of components it is made of rather than as one frozen line of source: the family prefix comes from the same helper the cache uses, the key names the identity the body is about, and the key names the lineage it stitches. Two of those checks read no source at all -- two identities are two keys, and an identity is not the same key as no identity The anchors of the mutation controls were re-anchored where the material they take apart grew a field, and the two that take the cache key apart were re-anchored to the key as it is written now. An anchor that no longer resolves stops the control before the mutations after it are run at all. --- .../tests/mutate-session-layers.py | 34 ++++ .../tests/mutate-sessions-etag.py | 23 ++- .../tests/mutate-trajectory-cache.py | 12 +- .../tests/verify-session-layers.mjs | 161 ++++++++++++++++++ .../tests/verify-sessions-etag.mjs | 13 ++ .../tests/verify-trajectory-cache.mjs | 25 ++- 6 files changed, 257 insertions(+), 11 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py index 641b41af..b0d5090a 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -296,6 +296,40 @@ " const holds = true;")], "a cached candidate whose manifest now disagrees with its name is not served under either id"), + # ---- the published id, after the walk has published it ---- + # + # The id is not only carried by the row: two caches are keyed by it, and a cache that does + # not know the id keeps answering with the answer it built for the id before. Three places, + # three shapes of the same question -- and each of them is a transition, so each one is + # caught by a fixture that waits for the caches to expire before it asks. + + # The list's validator. It carries the directory, the transcript's size and mtime, and the + # runtime's facts -- and the published id is the one thing it left out. A manifest that + # changes from one id to another moves the id and nothing else, so the tag stays put, the + # body the service reuses goes on publishing the old id, and the route serves that same id + # as a 404. + ("the list validator stops covering the published id", + [(" parts.push(JSON.stringify([entry.sessionId, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", + " parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n")], + "a changed published session id moves the list validator and replaces its body"), + + # The `latest` fallback. The explicit-id routes ask again whether a candidate still holds + # the session it was published under, because the manifest can change under the walk; the + # fallback used to take the first entry of the window as it stood, so the one candidate + # that both explicit ids refuse was the one `latest` served. + ("the latest fallback stops asking whether the candidate still holds its session", + [(" entry = await firstHoldingEntry(entries);", " entry = entries[0];")], + "latest never serves a cached candidate whose manifest now disagrees with its name"), + + # The trajectory's cache key. It carries the directory, the lineage, the size, the mtime and + # the binding -- and the identity is the one thing it left out. Two ids for the same + # directory therefore share one entry, and the body built for the first is served for the + # second, with the first one's validator. + ("the trajectory cache key stops covering the identity", + [("`${trajectoryCacheFamily(entry.dir, filter)}${identity.id ?? ''}|${lineage}|", + "`${trajectoryCacheFamily(entry.dir, filter)}${lineage}|")], + "an identity-only manifest change cannot reuse another session identity's trajectory body"), + # ---- the page ---- # # The two picks. The conversation pick holds conversations; the run pick holds the runs of diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py index cf77f044..29a37c3e 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py @@ -69,18 +69,29 @@ # representation with no ambiguity in it survives both, so there are two cases and each owns the # assertion that names it. ("the fields are concatenated with no separator at all", - [(" parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", - " parts.push(\"\" + entry.dir + entry.sizeBytes + entry.lastActiveAt" + [(" parts.push(JSON.stringify([entry.sessionId, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", + " parts.push(\"\" + entry.sessionId + entry.dir + entry.sizeBytes + entry.lastActiveAt" " + title + kind + parent + tag);")], "a title/kind split cannot be hidden inside the separator", SERVER), # mtime is one of the three facts the descriptor cache keys on, and the only one that moves # when a session is rewritten in place without growing. ("a newer mtime stops moving the tag", - [(" parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", - " parts.push(JSON.stringify([entry.dir, entry.sizeBytes, title, kind, parent, tag]));\n")], + [(" parts.push(JSON.stringify([entry.sessionId, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", + " parts.push(JSON.stringify([entry.sessionId, entry.dir, entry.sizeBytes, title, kind, parent, tag]));\n")], "a newer mtime moves the tag", SERVER), + # The published id. It is in the row the page receives -- a row's id IS the id the walk + # published that directory under -- and a manifest that changes from one id to another moves + # that id and nothing else: the directory, the transcript's size and mtime, and the runtime + # row are all the same. A material without the id therefore answers 304 for a body that names + # a session the reader is no longer looking at, and the body the service reuses keeps + # publishing the old one until some other fact happens to move. + ("the published id stops being part of the material", + [(" parts.push(JSON.stringify([entry.sessionId, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", + " parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n")], + "a changed published session id moves the tag", SERVER), + # ---- the 304 that is not earned, because it is decided too late ---- # # Correct answer, wrong bill: every descriptor is built and then thrown away. The validator @@ -178,8 +189,8 @@ # `String.fromCharCode(0)` rather than an escape in the replacement, because the point of the # mutation is the character, not how it is spelled. ("the fields go back to being joined on a bare NUL", - [(" parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", - " parts.push(entry.dir + \"|\" + entry.sizeBytes + \"|\" + entry.lastActiveAt" + [(" parts.push(JSON.stringify([entry.sessionId, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n", + " parts.push(entry.sessionId + \"|\" + entry.dir + \"|\" + entry.sizeBytes + \"|\" + entry.lastActiveAt" " + \"|\" + title + String.fromCharCode(0) + kind" " + String.fromCharCode(0) + parent + tag);")], "a separator-like character inside a value cannot merge two readable rows into one tag", SERVER), diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py index 1a007d1e..f2442a7b 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py @@ -91,14 +91,20 @@ ( "the key stops carrying the active file's size, so a grown session serves a stale body", SERVER, - " return `${trajectoryCacheFamily(entry.dir, filter)}${lineage}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", - " return `${trajectoryCacheFamily(entry.dir, filter)}${lineage}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", + " return `${trajectoryCacheFamily(entry.dir, filter)}${identity.id ?? ''}|${lineage}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", + " return `${trajectoryCacheFamily(entry.dir, filter)}${identity.id ?? ''}|${lineage}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", ), ( "the key stops carrying the lineage, so a compaction that only rotated snapshots serves a stale body", SERVER, + " return `${trajectoryCacheFamily(entry.dir, filter)}${identity.id ?? ''}|${lineage}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", + " return `${trajectoryCacheFamily(entry.dir, filter)}${identity.id ?? ''}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", + ), + ( + "the key stops naming the identity, so one session's body can be served for another", + SERVER, + " return `${trajectoryCacheFamily(entry.dir, filter)}${identity.id ?? ''}|${lineage}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", " return `${trajectoryCacheFamily(entry.dir, filter)}${lineage}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", - " return `${trajectoryCacheFamily(entry.dir, filter)}${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs index 173484e0..6d78bd1f 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -1287,6 +1287,9 @@ try { assert('and the parent keeps the title the runtime database has for it', Boolean(parentRow) && parentRow.label === '会话 rent', JSON.stringify(parentRow)); + assert('and the kind the runtime database has for it', + Boolean(parentRow) && parentRow.sessionKind === 'conversation', + JSON.stringify(parentRow)); const served = await fetch(service.base + '/api/trajectory?session=mvs_identified_parent'); assert('and the id the list shows is the id the route resolves', served.status === 200, @@ -1328,6 +1331,164 @@ try { failures.push('a manifest that changes under a cached walk -- ' + (error && error.message)); } +/* ---------- 16: the published id, after the walk has published it ---------- */ +// +// The walk decides the id a directory is published under, and everything downstream reads that +// one id. But the id is not only carried by the row: it is also what two CACHES are keyed by, +// and a cache that does not know the id will keep answering with the answer it built for the +// id before. +// +// Three places, three shapes of the same question: +// +// - the list's validator. It carries the directory, the transcript's size and mtime, and the +// runtime's facts -- but not the published id. So a manifest that changes from one id to +// another moves the id and nothing else, the validator does not move, and the body the +// service reuses goes on publishing the old id while the route serves that same id as +// 404. +// - the `latest` fallback. The explicit-id routes ask again whether a candidate still holds +// the session it was published under, because the manifest can change under the walk. The +// fallback used to take the first entry of the window as it stood, so the one candidate +// both explicit ids refuse was the one `latest` served. +// - the trajectory's cache key. It carries the directory, the lineage, the size, the mtime +// and the binding -- but not the identity. So two ids for the same directory share one +// entry, and the body built for the first is served for the second. +// +// Each of them is a transition, so each is asked over HTTP and after the caches have had their +// time: the walk is held for three seconds, and the first wait is the one that decides when a +// new id is seen. + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-reid-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + // The name does not decode, so the manifest beside the transcript is the only identity this + // directory has -- and the published id is therefore the only thing that can change when + // the manifest changes. Nothing else about the directory does: same transcript, same size, + // same mtime, and no runtime row for it at all. + const legacy = join(sessionsRoot, '2026', '10', '20', '00-00-00-session_legacy'); + await mkdir(legacy, { recursive: true }); + await writeFile(join(legacy, 'messages.jsonl'), + JSON.stringify({ role: 'user', content: 'x' }) + '\n', 'utf8'); + await writeFile(join(legacy, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_reid_one', createdAtMs: 1 }), 'utf8'); + const stamp = new Date(BASE_TIME); + await utimes(join(legacy, 'messages.jsonl'), stamp, stamp); + + const service = await startService(home); + try { + const first = await fetch(service.base + '/api/sessions?limit=20'); + const firstEtag = first.headers.get('etag'); + const firstBody = await first.json(); + assert('the list publishes the id the manifest names', + (firstBody.sessions || []).some((row) => row.id === 'mvs_reid_one'), + JSON.stringify((firstBody.sessions || []).map((row) => row.id))); + + // The manifest now names another session. + await writeFile(join(legacy, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_reid_two', createdAtMs: 1 }), 'utf8'); + await new Promise((resolve) => setTimeout(resolve, 3200)); + + const conditional = await fetch(service.base + '/api/sessions?limit=20', { + headers: { 'If-None-Match': firstEtag } + }); + const conditionalEtag = conditional.headers.get('etag'); + const conditionalText = await conditional.text(); + let conditionalBody = null; + try { conditionalBody = JSON.parse(conditionalText); } catch { conditionalBody = null; } + const conditionalIds = (conditionalBody && conditionalBody.sessions || []).map((row) => row.id); + assert('a changed published session id moves the list validator and replaces its body', + conditional.status !== 304 + && conditionalEtag !== firstEtag + && conditionalIds.includes('mvs_reid_two') + && !conditionalIds.includes('mvs_reid_one'), + JSON.stringify([conditional.status, conditionalEtag, firstEtag, conditionalIds])); + + // And the body the service reuses for the next request says the same thing. + const unconditional = await (await fetch(service.base + '/api/sessions?limit=20')).json(); + const unconditionalIds = (unconditional.sessions || []).map((row) => row.id); + assert('and the body the service reuses for the next request says the same thing', + unconditionalIds.includes('mvs_reid_two') && !unconditionalIds.includes('mvs_reid_one'), + JSON.stringify(unconditionalIds)); + } finally { + await service.dispose(); + } +} catch (error) { + failures.push('a published id that changes -- ' + (error && error.message)); +} + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-latest-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + // The newest session, and the one the fallback would take without asking. The healthy one + // is older, so the fallback has to skip past the broken one to reach it. + const broken = await makeSession(sessionsRoot, 0, 'mvs_latest_broken', BASE_TIME); + await makeSession(sessionsRoot, 1, 'mvs_latest_healthy', BASE_TIME - 60_000); + await writeFile(join(broken.dir, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_latest_broken', createdAtMs: 1 }), 'utf8'); + + const service = await startService(home); + try { + // The walk is primed while the manifest still agrees with the name. + await (await fetch(service.base + '/api/sessions?limit=20')).json(); + // Now the manifest beside the same transcript names another session. The walk is still + // cached, so the route is holding the directory it admitted a moment ago. + await writeFile(join(broken.dir, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_latest_other', createdAtMs: 1 }), 'utf8'); + const underOld = await fetch(service.base + '/api/trajectory?session=mvs_latest_broken'); + const underNew = await fetch(service.base + '/api/trajectory?session=mvs_latest_other'); + const latest = await fetch(service.base + '/api/trajectory?session=latest'); + const latestBody = latest.status === 200 ? await latest.json() : null; + const explicitRefused = underOld.status === 404 && underNew.status === 404; + const latestServedAnother = Boolean(latestBody) && Boolean(latestBody.session) + && latestBody.session.id === 'mvs_latest_healthy'; + assert('latest never serves a cached candidate whose manifest now disagrees with its name', + explicitRefused && (latest.status !== 200 || latestServedAnother), + JSON.stringify([underOld.status, underNew.status, latest.status, + latestBody && latestBody.session && latestBody.session.id])); + } finally { + await service.dispose(); + } +} catch (error) { + failures.push('the latest fallback -- ' + (error && error.message)); +} + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-trajcache-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + const legacy = join(sessionsRoot, '2026', '10', '20', '00-00-00-session_legacy'); + await mkdir(legacy, { recursive: true }); + await writeFile(join(legacy, 'messages.jsonl'), + JSON.stringify({ role: 'user', content: 'x' }) + '\n', 'utf8'); + await writeFile(join(legacy, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_traj_one', createdAtMs: 1 }), 'utf8'); + const stamp = new Date(BASE_TIME); + await utimes(join(legacy, 'messages.jsonl'), stamp, stamp); + + const service = await startService(home); + try { + const first = await fetch(service.base + '/api/trajectory?session=mvs_traj_one'); + const firstEtag = first.headers.get('etag'); + const firstBody = await first.json(); + // Only the manifest changes. The directory, the transcript, the lineage and the mtime are + // all the same, so a cache key made of those would match and the body built for the first + // id would be served for the second. + await writeFile(join(legacy, 'manifest.json'), + JSON.stringify({ sessionId: 'mvs_traj_two', createdAtMs: 1 }), 'utf8'); + await new Promise((resolve) => setTimeout(resolve, 3200)); + const second = await fetch(service.base + '/api/trajectory?session=mvs_traj_two'); + const secondEtag = second.headers.get('etag'); + const secondBody = second.status === 200 ? await second.json() : null; + assert('an identity-only manifest change cannot reuse another session identity\'s trajectory body', + Boolean(firstBody.session) && firstBody.session.id === 'mvs_traj_one' + && Boolean(secondBody && secondBody.session) && secondBody.session.id === 'mvs_traj_two' + && secondEtag !== firstEtag, + JSON.stringify([firstBody.session, secondBody && secondBody.session, firstEtag, secondEtag])); + } finally { + await service.dispose(); + } +} catch (error) { + failures.push('a trajectory body that belongs to another identity -- ' + (error && error.message)); +} + console.log(pass + ' assertions passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs index 739e8c00..ddda6b3a 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs @@ -82,6 +82,19 @@ assert('a newer mtime moves the tag', sessionsValidator([{ ...visible[0], lastActiveAt: 1700000002 }, visible[1]], noTitles, 20) !== sessionsValidator(visible, noTitles, 20)); +// The published id. It is in the row the page receives -- a row's id IS the id the walk +// published that directory under -- and a manifest that changes from one id to another moves +// that id and nothing else: the directory, the transcript's size and mtime, and the runtime +// row are all the same. The two maps here differ in the published id and in nothing else, so +// this cannot pass on the directory or on the mtime: a tag that did not move with the id +// would answer 304 for a body that names a session the reader is no longer looking at. +const reid = [ + { dir: dirFor(A), sizeBytes: 1024, lastActiveAt: 1700000000, sessionId: A }, + { dir: dirFor(B), sizeBytes: 2048, lastActiveAt: 1700000001, sessionId: 'mvs_0000000000000000000000000000dddd' }, +]; +assert('a changed published session id moves the tag', + sessionsValidator(visible, noTitles, 20) !== sessionsValidator(reid, noTitles, 20)); + // The rename. Both halves of a row's metadata, because the picker renders both — and each has to // be isolated, or the assertion passes on the other field and the one it names is untested. These // two maps differ in the title and in nothing else. diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs index 96b0d362..cc0a20e2 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs @@ -71,6 +71,16 @@ const rotatedIdentity = { assert('rotating the lineage changes the key', trajectoryCacheKey(entry(1000, 5000), rotatedIdentity, null, null) !== keyFresh); +// The identity the directory holds is part of the key, because the body says which session it is +// about. A manifest that changes from one id to another leaves the directory, the transcript and +// the mtime exactly as they were, so a key made of those would still match — and the cached body +// of the OLD id would be served for the NEW one. +const keyIdentityA = trajectoryCacheKey(entry(1000, 5000), { ...identity, id: 'sess-a' }, null, null); +const keyIdentityB = trajectoryCacheKey(entry(1000, 5000), { ...identity, id: 'sess-b' }, null, null); +assert('two identities are two keys, so one identity cannot reuse the other body', + keyIdentityA !== keyIdentityB); +assert('an identity is not the same key as no identity at all', keyIdentityA !== keyFresh); + /* ===================================================================== * 2 — growth replaces its predecessor instead of stacking beside it * ===================================================================== */ @@ -157,8 +167,19 @@ assert('the route no longer evicts by insertion order itself', !/while \(trajectoryCache\.size > TRAJECTORY_CACHE_ENTRIES\)/.test(startBody)); assert('the family helper is exported so the key and the cache cannot drift apart', SERVER.includes('export function trajectoryCacheFamily(dir, filter)')); -assert('the key is built from the same helper, so the prefix is one expression', - SERVER.includes('`${trajectoryCacheFamily(entry.dir, filter)}${lineage}|')); +// The key is read as the set of components it is made of, not as one frozen line of source. A key +// that grew a field is still the same key, and an assertion that pinned two neighbouring fields +// together would have called that growth a regression — the shape is an accident of how the +// template was written, the components are the contract. +const keyFnAt = SERVER.indexOf('export function trajectoryCacheKey('); +const keyFnEnd = keyFnAt >= 0 ? SERVER.indexOf('\n}', keyFnAt) : -1; +const keyFnBody = keyFnAt >= 0 && keyFnEnd > keyFnAt ? SERVER.slice(keyFnAt, keyFnEnd) : ''; +assert('the key is built from the same helper, so the prefix cannot drift from the cache', + keyFnBody.includes('${trajectoryCacheFamily(entry.dir, filter)}')); +assert('the key names the identity the body carries, so one identity cannot reuse another body', + keyFnBody.includes("${identity.id ?? ''}")); +assert('the key names the lineage it stitches, so a rotated snapshot cannot reuse an old body', + keyFnBody.includes('${lineage}')); assert('the route still releases both caches on dispose', /trajectoryCache\.clear\(\);/.test(startBody) && /parsedRows\.clear\(\);/.test(startBody)); From 44b1c460ede3c2df9ebfe40c62a2c0526f3886c2 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 02:22:58 +0800 Subject: [PATCH 22/95] A replaced request writes nothing, and a read names the session it is about The trajectory poll is a second implementation of the rule the session list already follows: a request that has been replaced writes neither a body nor a validator. It wrote its validator in the response handler, before the sequence gate, so a 304 that arrived late put its own older tag over the newer one. The body on screen and the tag that described it were then two different things -- the next poll asked the server about the old body and kept the new one on screen. The poll now has the shape the list already has: the response handler carries the answer out, and one writer, after the gate, writes the validator. And the two secondary reads asked for a different session than the one they were caching. A record index is a position inside one body, so the raw read and the detail read have to ask about the session that body was built for. The page's own cache key already named the displayed session; the requests asked for "latest", which is the session that is current now. The runtime can make another session current while the page still shows this one, and the page then showed that other session's words under this session's record. Both requests now name the displayed session, so the key and the request say the same thing. --- .../mmc-trajectory/miniapp/client/index.html | 54 +++++++++++++++---- 1 file changed, 43 insertions(+), 11 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index ca2899e7..17d962b6 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -4032,6 +4032,29 @@

检视

return state.followLatest || !state.sessionId ? 'latest' : encodeURIComponent(state.sessionId); } + /** + * The session the body on screen is about, for a read that addresses a position inside it. + * + * A record index is a position within one body, so a request carrying that index has to ask + * about the session that body was built for. `sessionParam()` answers a different question + * — which session is current — and while the page follows the newest conversation that is + * `latest`, which is not necessarily the body on screen: the runtime can have made another + * session current while the page still shows this one. + * + * The client's own cache key already says which body a value belongs to — `rawCacheKey` + * names the displayed session — so asking the server for `latest` stored one session's + * record under another session's key, and the page showed the other session's words. + * + * @returns {string} + */ + function displayedSessionParam() { + var payload = state.payload; + var session = payload && payload.session ? payload.session : null; + var sid = session && typeof session.id === 'string' ? session.id : ''; + if (sid) return encodeURIComponent(sid); + return sessionParam(); + } + /** * The session list, on its own validator. * @@ -4177,10 +4200,13 @@

检视

// Nothing changed since the copy already on screen. Keep every piece of state // — no re-flatten, no re-render — and let the next poll do the same cheaply. if (response.status === 304) { + // The validator is written by the one writer below, and that runs after the + // sequence gate. A request that has been replaced must write nothing at all: a + // 304 that arrived late used to put its own older tag over the newer one, so the + // body on screen and the tag that described it were two different things — and + // the next poll asked the server about the old one and kept the new one. var fresh = response.headers.get('etag'); - if (fresh) state.etag = fresh; - state.etagFor = cacheKey; - return null; + return { changed: false, body: null, etag: fresh || state.etag }; } return response .json() @@ -4197,15 +4223,21 @@

检视

} if (!body || typeof body !== 'object') throw new Error('api'); var tag = response.headers.get('etag'); - state.etag = tag || ''; - state.etagFor = cacheKey; - return body; + return { changed: true, body: body, etag: tag || '' }; }); }) - .then(function (payload) { + .then(function (result) { + // A late answer must not win. Every request carries a sequence number, and only + // the newest one may write — the same rule the session list follows. if (seq !== refreshSeq) return; inflight = null; - if (payload === null) { + // One writer for the trajectory's validator, on the one path that carried an + // answer. A request that has been replaced writes neither a body nor a tag: the + // tag is what the next poll asks the server with, so a stale one leaves the page + // describing a body it is no longer showing. + state.etag = result.etag; + state.etagFor = cacheKey; + if (!result.changed) { // 304: the previous render is still accurate. Only clear a pending loading // state, because nothing on screen should have been replaced. // @@ -4223,7 +4255,7 @@

检视

if (cleared) render(); return; } - applyPayload(payload, manual); + applyPayload(result.body, manual); }) .catch(function (error) { if (seq !== refreshSeq) return; @@ -8626,7 +8658,7 @@

检视

function requestRaw(index) { var url = '/api/trajectory/raw?index=' + encodeURIComponent(String(index)) - + '&session=' + sessionParam(); + + '&session=' + displayedSessionParam(); if (state.generation !== null) { url += '&generation=' + encodeURIComponent(String(state.generation)); } @@ -8717,7 +8749,7 @@

检视

function requestDetail(index) { var url = '/api/trajectory/record?index=' + encodeURIComponent(String(index)) - + '&session=' + sessionParam(); + + '&session=' + displayedSessionParam(); if (state.generation !== null) { url += '&generation=' + encodeURIComponent(String(state.generation)); } From 401d66e06f89f323b105d2adb6506817339e3e11 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 02:23:11 +0800 Subject: [PATCH 23/95] Ask the trajectory poll and the secondary reads what they are about - two overlapping trajectory polls are driven against a controllable transport, and a late 304 and a late 200 must both leave the newer answer alone; the poll after them asks the server about the body that is on screen - the raw read and the detail read are driven against a payload, and both must name the session on screen rather than whichever one is current now - the sub-agent's published id moves the list validator on its own: the main session's id had an assertion of its own, and the sub-agent's tuple is a different tuple - the controls beside them: three for the page and one for the validator, and one control each for the raw read and the detail read -- two requests, and a control that changed both would leave the detail assertion never asked whether it could fail --- .../mmc-trajectory/tests/client-check.mjs | 136 ++++++++++++++++++ .../tests/mutate-session-layers.py | 44 ++++++ .../tests/mutate-sessions-etag.py | 9 ++ .../tests/verify-sessions-etag.mjs | 20 +++ 4 files changed, 209 insertions(+) diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index 195c7759..0c0a6fa7 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -1641,6 +1641,142 @@ if (!fetchSessionsSrc || !inflightDecl || !seqDecl) { requests[6].headers['If-None-Match'], undefined); } +// The trajectory poll is the second implementation of the same rule the session list follows, +// and it used to write the validator BEFORE its sequence gate. A 304 that arrived late then +// put its own older tag over the newer one, so the body on screen and the tag that described +// it were two different things: the next poll asked the server about the old body and kept the +// new one on screen. +// +// The counterexample, driven against the shipped function: the page holds A, two polls are in +// flight, the newer one answers first with B and B's tag, and the older one answers with a 304 +// carrying A's tag. The old tag must not survive that. +const fetchTrajectorySrc = fnSource('function fetchTrajectory(manual)'); +const sessionParamSrc = fnSource('function sessionParam()'); +const trajInflightDecl = /var inflight = null;/.exec(html); +const trajSeqDecl = /var refreshSeq = 0;/.exec(html); +if (!fetchTrajectorySrc || !sessionParamSrc || !trajInflightDecl || !trajSeqDecl) { + failures.push('trajectory poll: could not lift fetchTrajectory and the state it keeps'); +} else { + vm.runInContext( + trajInflightDecl[0] + '\n' + trajSeqDecl[0] + '\n' + sessionParamSrc + '\n' + fetchTrajectorySrc, + sandbox + ); + + const trajRequests = []; + sandbox.fetch = function (url, options) { + const entry = { + url: url, + signal: options && options.signal, + headers: (options && options.headers) || {}, + settle: null + }; + entry.promise = new Promise((resolve) => { entry.settle = resolve; }); + trajRequests.push(entry); + return entry.promise; + }; + sandbox.AbortController = AbortController; + sandbox.state = { + payload: null, generation: null, etag: '', etagFor: '', since: 0, sinceFor: '', + loading: false, followLatest: true, sessionId: '', hostSessionId: '' + }; + sandbox.renderLoading = () => {}; + sandbox.noteFresh = () => false; + sandbox.render = () => {}; + const applied = []; + sandbox.applyPayload = function (body) { applied.push(body); }; + + const trajAnswer = (entry, body, etag) => entry.settle({ + status: 200, + ok: true, + headers: { get: (name) => (name === 'etag' ? (etag || '') : '') }, + json: () => Promise.resolve(body) + }); + const trajNotModified = (entry, etag) => entry.settle({ + status: 304, + ok: false, + headers: { get: (name) => (name === 'etag' ? (etag || '') : '') } + }); + + const older = sandbox.fetchTrajectory(false); + const newer = sandbox.fetchTrajectory(false); + check('the second poll stops the first', trajRequests[0].signal.aborted, true); + trajAnswer(trajRequests[1], { session: { id: 'mvs_b' } }, '"tag-b"'); + await newer; + check('the newer body is the one on screen', applied.map((b) => b.session.id), ['mvs_b']); + check('and the validator describes it', sandbox.state.etag, '"tag-b"'); + + // The older answer arrives late and carries the older tag. + trajNotModified(trajRequests[0], '"tag-a"'); + await older; + check('a replaced trajectory request writes no validator', sandbox.state.etag, '"tag-b"'); + check('and it leaves the body on screen alone', + applied.map((b) => b.session.id), ['mvs_b']); + + // And the poll after it asks the server about the body that is on screen. + sandbox.fetchTrajectory(false); + check('the next poll carries the validator of the body on screen', + trajRequests[2].headers['If-None-Match'], '"tag-b"'); + + // The same rule for a late 200: a replaced request must not put its body on screen either. + const lateOld = sandbox.fetchTrajectory(false); + const lateNew = sandbox.fetchTrajectory(false); + trajAnswer(trajRequests[4], { session: { id: 'mvs_c' } }, '"tag-c"'); + await lateNew; + trajAnswer(trajRequests[3], { session: { id: 'mvs_a' } }, '"tag-a"'); + await lateOld; + check('a replaced trajectory request writes no body either', + applied.map((b) => b.session.id), ['mvs_b', 'mvs_c']); + check('and the validator is still the newer one', sandbox.state.etag, '"tag-c"'); +} + +// A record index is a position within ONE body, so a read that carries that index has to ask +// about the session that body was built for. The client already says so in its own cache key — +// `rawCacheKey` names the displayed session — but the request used to ask for `latest`, which +// is "whichever session is current now" and not necessarily the one on screen. The runtime can +// make another session current while the page still shows this one, and the page then showed +// that other session's words under this session's record. +const requestRawSrc = fnSource('function requestRaw(index)'); +const requestDetailSrc = fnSource('function requestDetail(index)'); +const displayedSessionSrc = fnSource('function displayedSessionParam()'); +if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc) { + failures.push('secondary reads: could not lift requestRaw, requestDetail and the session they ask for'); +} else { + vm.runInContext(displayedSessionSrc + '\n' + requestRawSrc + '\n' + requestDetailSrc, sandbox); + + const secondary = []; + sandbox.fetch = function (url) { + secondary.push(url); + return Promise.resolve({ + ok: true, + status: 200, + json: () => Promise.resolve({ raw: 'ALPHA BODY', index: 1, text: 'ALPHA BODY' }) + }); + }; + // The page is following the newest conversation, and it is showing A. The runtime may already + // have made B current — the page does not know that, and it must not care: A is on screen. + sandbox.state = { + payload: { session: { id: 'mvs_displayed' } }, + generation: null, etag: '"tag"', followLatest: true, sessionId: '', hostSessionId: '' + }; + await sandbox.requestRaw(1); + await sandbox.requestDetail(1); + check('a raw read asks for the session on screen', + /[?&]session=mvs_displayed(&|$)/.test(secondary[0]), true); + check('and it does not ask for whatever is current now', + /[?&]session=latest(&|$)/.test(secondary[0]), false); + check('a detail read asks for the session on screen', + /[?&]session=mvs_displayed(&|$)/.test(secondary[1]), true); + check('and it does not ask for whatever is current now', + /[?&]session=latest(&|$)/.test(secondary[1]), false); + + // With nothing on screen there is no session to name, and the read falls back to the session + // parameter rather than sending an empty one. + sandbox.state = { payload: null, generation: null, etag: '', followLatest: true, sessionId: '', hostSessionId: '' }; + await sandbox.requestRaw(1); + check('with no body on screen the read falls back to the current session', + /[?&]session=latest(&|$)/.test(secondary[2]), true); +} + console.log('[2] pure helpers: ' + pass + ' passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py index b0d5090a..62c95e9c 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-layers.py @@ -518,6 +518,50 @@ " sessionParentId: value.sessionParentId ?? null,\n" " sessionWorkspace: value.sessionWorkspace ?? null,")], "and neither does the workspace path"), + + # ---- the trajectory poll: the same rule the list follows, in a second implementation ---- + # + # The list's validator is written by one writer, after the sequence gate. The trajectory poll + # is a second implementation of that promise and it wrote its validator before the gate, so a + # 304 that arrived late put its own older tag over the newer one: the body on screen and the + # tag that described it were two different things, and the next poll asked the server about + # the old body and kept the new one on screen. + ("a late 304 writes the trajectory's own validator over the newer one", + [(" var fresh = response.headers.get('etag');\n" + " return { changed: false, body: null, etag: fresh || state.etag };", + " var fresh = response.headers.get('etag');\n" + " if (fresh) state.etag = fresh;\n" + " state.etagFor = cacheKey;\n" + " return { changed: false, body: null, etag: state.etag };")], + "a replaced trajectory request writes no validator", + CLIENT), + + # ---- the secondary reads: a record index is a position inside ONE body ---- + # + # The raw read and the detail read both address a position inside the body on screen, so they + # have to ask about the session that body was built for. The client's own cache key already + # says so -- `rawCacheKey` names the displayed session -- but the request used to ask for + # `latest`, which is "whichever session is current now". The runtime can make another session + # current while the page still shows this one, and the page then showed that other session's + # words under this session's record. + # + # They are TWO controls, not one: the raw read and the detail read are two requests, and a + # control that changed both would leave the detail assertion never asked whether it could fail. + ("a raw read stops asking for the session on screen", + [(" var url = '/api/trajectory/raw?index=' + encodeURIComponent(String(index))\n" + " + '&session=' + displayedSessionParam();", + " var url = '/api/trajectory/raw?index=' + encodeURIComponent(String(index))\n" + " + '&session=' + sessionParam();")], + "a raw read asks for the session on screen", + CLIENT), + + ("a detail read stops asking for the session on screen", + [(" var url = '/api/trajectory/record?index=' + encodeURIComponent(String(index))\n" + " + '&session=' + displayedSessionParam();", + " var url = '/api/trajectory/record?index=' + encodeURIComponent(String(index))\n" + " + '&session=' + sessionParam();")], + "a detail read asks for the session on screen", + CLIENT), ] ORIGINALS = { diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py index 29a37c3e..09c2a056 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-sessions-etag.py @@ -92,6 +92,15 @@ " parts.push(JSON.stringify([entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, parent, tag]));\n")], "a changed published session id moves the tag", SERVER), + # The sub-agent's tuple is a different tuple, so the rule has to be asked of it separately. + # A material without the sub-agent's published id answers 304 for a body that names a session + # the reader is no longer looking at, and the body the service reuses keeps publishing the old + # one until some other fact happens to move. + ("the published sub-agent id stops being part of the material", + [(" parts.push(JSON.stringify([entry.sessionId, parent, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, tag]));\n", + " parts.push(JSON.stringify([parent, entry.dir, entry.sizeBytes, entry.lastActiveAt, title, kind, tag]));\n")], + "a changed published sub-agent id moves the tag", SERVER), + # ---- the 304 that is not earned, because it is decided too late ---- # # Correct answer, wrong bill: every descriptor is built and then thrown away. The validator diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs index ddda6b3a..d1f5dbdf 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-sessions-etag.mjs @@ -95,6 +95,26 @@ const reid = [ assert('a changed published session id moves the tag', sessionsValidator(visible, noTitles, 20) !== sessionsValidator(reid, noTitles, 20)); +// The published id of a SUB-AGENT. The main session's id has its own assertion above, and the +// sub-agent's tuple is a different tuple: a rule that holds for one of them does not hold for +// the other unless it is asked of both. Removing the sub-agent's id from its tuple left every +// verifier green -- the tag then answered 304 for a body that named a session the reader was no +// longer looking at, and no assertion noticed. +// +// The two child lists below differ in the published id and in nothing else: the same directory, +// the same size, the same mtime, and no runtime facts for either id. +const childIdA = 'mvs_0000000000000000000000000000eeee'; +const childIdB = 'mvs_0000000000000000000000000000ffff'; +const childWithIdA = [ + { dir: dirFor(childIdA), sizeBytes: 512, lastActiveAt: 1700000002, sessionId: childIdA }, +]; +const childWithIdB = [ + { dir: dirFor(childIdA), sizeBytes: 512, lastActiveAt: 1700000002, sessionId: childIdB }, +]; +assert('a changed published sub-agent id moves the tag', + sessionsValidator(visible, noTitles, 20, childWithIdA) + !== sessionsValidator(visible, noTitles, 20, childWithIdB)); + // The rename. Both halves of a row's metadata, because the picker renders both — and each has to // be isolated, or the assertion passes on the other field and the one it names is untested. These // two maps differ in the title and in nothing else. From a47482d43553ddd562a76158ad026dc86aa12e81 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 02:45:18 +0800 Subject: [PATCH 24/95] The READMEs say what the layer rule is, not the earlier version of it The two READMEs described the upper select as "the rows with no `parent_session_id`". That was an earlier and simpler version of the rule: the layer is decided by the chain of parents, so a session whose parent names no session, whose parent chain ends in a cycle, or whose parent is itself a run is in the conversation select too, and a row that ends up in that layer carries no parent. The description was narrower than the behaviour, and the rule that made it narrower is part of this change. Both READMEs now describe the chain, name the three shapes that have no conversation at the end of it, and say that a row in the conversation layer carries no parent. No implementation byte moved with them. --- plugins/avatasia/mmc-trajectory/README.md | 2 +- plugins/avatasia/mmc-trajectory/README.zh-CN.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/README.md b/plugins/avatasia/mmc-trajectory/README.md index 896be6bb..fe3f23cc 100644 --- a/plugins/avatasia/mmc-trajectory/README.md +++ b/plugins/avatasia/mmc-trajectory/README.md @@ -131,7 +131,7 @@ The runtime database is opened in read-only mode. If it is missing, unreadable, - A single `messages.jsonl` is read up to 64 MB. Larger sessions are truncated and the page says so. - The ledger does not use virtual scrolling; every rendered row is a real element. The page therefore opens on the newest 200 records, and the window bar button switches to the whole session. The button names what pressing it will do, not the current state: 「显示全部」 at the default and 「显示最新 200 条」 once everything is showing, while the count beside it reports what is on screen. That full view is opt-in for a reason: on a 9,900-record session it is roughly 57,000 elements rebuilt every 2.5s poll, which was measured to freeze the page rather than be read. 「向上加载更早的 N 条」 widens the window in steps and a poll does not undo it. A sticky window bar under the overview strip offers that choice behind four explicit controls: switch the strip between equal width and a real time axis, load 200 earlier records, jump back to the latest 200, and show everything. "Show everything" builds every record in the session into the DOM at once and gets noticeably slower on large sessions; incoming records do not knock it back down to 200. - `messageCount` and `turnCount` in the session picker are estimates derived from a bounded prefix of the file; exact values come from the selected session. -- The session picker is **two selects in a cascade**. The upper one, 「会话」, lists conversations only — the ones with no `parent_session_id` in the runtime database. The lower one, 「子代理」, lists the runs of the conversation that was picked: it is **empty by default**, it has values once a conversation is picked, and it is empty and disabled while the page follows the newest conversation. Measured on this machine, 13 of the 20 rows were sub-agent runs, so a reader could not tell which row was their own conversation; those 13 runs were also where that conversation actually happened. Nothing is hidden, only moved — the runs are not rows of the conversation list any more, they are rows of the run list under it, and the conversation list is now the twenty most recent conversations rather than the twenty most recent sessions of any kind. Every run is listed, most recently active first, and pressing one switches the page to it. The layer is decided by `parent_session_id`, which is independent of the 「定时 / 任务 / 轻量」 markers. The parent's name is still not written out: the parent id comes down with the row, and the page groups by that one id rather than doing a lookup that could fail quietly. Picking a run shows **that run's own trajectory** — a run is a session with its own transcript, and the parent's transcript does not contain its turns — and the conversation select stays on its parent. +- The session picker is **two selects in a cascade**. The upper one, 「会话」, lists conversations only — the sessions whose parent, followed up the chain, is a conversation. A session with no parent is one of them, and so is one whose parent names no session, whose parent chain ends in a cycle, or whose parent is itself a run: the layer follows the chain rather than the one column, and a row that ends up in the conversation layer carries no parent. The lower one, 「子代理」, lists the runs of the conversation that was picked: it is **empty by default**, it has values once a conversation is picked, and it is empty and disabled while the page follows the newest conversation. Measured on this machine, 13 of the 20 rows were sub-agent runs, so a reader could not tell which row was their own conversation; those 13 runs were also where that conversation actually happened. Nothing is hidden, only moved — the runs are not rows of the conversation list any more, they are rows of the run list under it, and the conversation list is now the twenty most recent conversations rather than the twenty most recent sessions of any kind. Every run is listed, most recently active first, and pressing one switches the page to it. The layer is decided by the chain of parents, which is independent of the 「定时 / 任务 / 轻量」 markers. The parent's name is still not written out: the parent id comes down with the row, and the page groups by that one id rather than doing a lookup that could fail quietly. Picking a run shows **that run's own trajectory** — a run is a session with its own transcript, and the parent's transcript does not contain its turns — and the conversation select stays on its parent. - The picker also says what a session **is**, not only what it is called: a scheduled run carries 「定时」, a background task 「任务」, and a tool-less lightweight conversation 「轻量」, so none of them can be mistaken for a conversation you wrote. They stay in the list and stay selectable — a marker is a marker, not a filter. The conversation the runtime database reports as current comes first, then the sessions of its own workspace, then the rest; the workspace path and the raw purpose string are compared on the server and never reach the page. - Light theme token coverage is verified — every `--mcode-*` token the page consumes is defined for both themes — but its rendered appearance was not visually checked; the page was exercised under a dark system preference. diff --git a/plugins/avatasia/mmc-trajectory/README.zh-CN.md b/plugins/avatasia/mmc-trajectory/README.zh-CN.md index 874107fa..e16f9c18 100644 --- a/plugins/avatasia/mmc-trajectory/README.zh-CN.md +++ b/plugins/avatasia/mmc-trajectory/README.zh-CN.md @@ -117,7 +117,7 @@ MiniMax Code 不会告诉 Mini App 是哪段对话打开了它的页面：传给 - 单个 `messages.jsonl` 最多读取 64 MB。更大的会话会被截断，页面会明确说明。 - 台里没有用虚拟滚动，每一行都是真实的 DOM 元素。页面默认只展示最新 200 条，窗口条上的按钮可以切到全量；全量是可选的，因为它很贵 —— 在这个会话上约 9,900 条、约 5.7 万个元素，每 2.5 秒轮询重建一次，实测会直接卡死页面。总览条正下方的吸顶窗口栏提供三个显式控件：「总览轴」「向上加载更早的 200 条」，以及在最新 200 条与全量之间切换的那一个按钮（它写的是按下去会变成什么，默认为「显示全部」，全量时为「显示最新 200 条」；当前窗口由右边的计数条报告，两者回答的是不同问题）。「总览轴」只改总览条的横轴——等宽（每条记录一样宽，完全忽略时间）或真实时间轴（宽度等于该记录的真实耗时）；它不筛选、不排序，台账里的行和页面上的任何计数都不变。「向上加载更早的 200 条」每点一次就多加一页，轮询不会把已经加宽的窗口重置回去；点切换按钮也不会自动滚动页面，读者正在看的那一行会留在原地。 - 会话选择器里的 `messageCount` / `turnCount` 是从文件的有界前缀估算的；精确值以选中会话后展示的为准。 -- 会话选择器是**级联的两个下拉框**。上面那个「会话」只列主会话 —— 运行时库里 `parent_session_id` 为空的那些；下面那个「子代理」只列所选那段对话的子代理，**默认是空**，选中主会话之后才有值，跟随最新时是空的且不可选。实测本机 20 行里有 13 行是子代理，读者根本分不出哪一行是自己的对话，而那 13 行恰恰就是这段对话真正发生的地方。它们没有被藏起来，只是从「平铺在会话里」变成「会话列表下面的子代理列表」：会话列表现在列的是最近 20 段主会话，而不是最近 20 个任意种类的会话；子代理有多少列多少，按各自最后一次活跃的时间倒序，最活跃的排在最上面，点其中一条就切到那一条。层级由 `parent_session_id` 决定，与「定时 / 任务 / 轻量」那套标记互相独立。父会话的名字仍然不写：父 id 由服务端随这一行带下来，页面按同一个 id 归组，不做一次可能悄悄失败的查找。选了子代理就显示**那一次运行自己的**轨迹 —— 每个子代理是一段独立的会话，有自己的 transcript，父会话的轨迹里并不包含它的轮次；此时上面那个下拉框停在它的父会话上。 +- 会话选择器是**级联的两个下拉框**。上面那个「会话」只列主会话 —— 沿父链往上走、终点是一段主会话的那些。父会话为空的是主会话；父会话指向一个不存在的会话、父链成环、或者父本身是子代理的，也是主会话 —— 层级看的是链，不是那一列，而落到会话层的行不带父会话。下面那个「子代理」只列所选那段对话的子代理，**默认是空**，选中主会话之后才有值，跟随最新时是空的且不可选。实测本机 20 行里有 13 行是子代理，读者根本分不出哪一行是自己的对话，而那 13 行恰恰就是这段对话真正发生的地方。它们没有被藏起来，只是从「平铺在会话里」变成「会话列表下面的子代理列表」：会话列表现在列的是最近 20 段主会话，而不是最近 20 个任意种类的会话；子代理有多少列多少，按各自最后一次活跃的时间倒序，最活跃的排在最上面，点其中一条就切到那一条。层级由父链决定，与「定时 / 任务 / 轻量」那套标记互相独立。父会话的名字仍然不写：父 id 由服务端随这一行带下来，页面按同一个 id 归组，不做一次可能悄悄失败的查找。选了子代理就显示**那一次运行自己的**轨迹 —— 每个子代理是一段独立的会话，有自己的 transcript，父会话的轨迹里并不包含它的轮次；此时上面那个下拉框停在它的父会话上。 - 会话选择器不只说会话**叫什么**，还说它**是什么**：定时任务标「定时」，后台任务标「任务」，没有工具的轻量会话标「轻量」，于是它们不会被误认成你自己开的对话。它们仍然在列表里、仍然可以选 —— 标记是标记，不是过滤。运行时库报告为当前的那段对话排在最前，然后是同一工作区的会话，再然后是其它；工作区路径和 purpose 原文只在服务端比较，不会发到页面。 - 浅色主题的 token 覆盖已验证（页面用到的每个 `--mcode-*` 变量在两套主题下都有定义），但渲染外观未做视觉检查；功能验收是在深色系统偏好下进行的。 From b19e86251296246a8a0a9af889faf7f510246d55 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 06:02:49 +0800 Subject: [PATCH 25/95] The family finds the body, and the search names the session on screen MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A cache key that names its requester cannot also decide which entry a read looks for. The `bind` segment carries the runtime binding that a `latest` request resolved; a request that names a session by its id resolves none and computes a different key for the same directory. The body the poll stored and the body the read looked for were therefore never the same entry, and reading one record of the conversation on screen answered 「会话数据已释放，请重新加载」 for a session the service was holding -- and the client's own retry did not rescue it, because the retry reloads the current conversation rather than the displayed one. The key names a version and the family names the question, and every entry already records the family it answers for. The two secondary readers now look the exact key up first and the family second, so a poll that asked for `latest` and a read that asked for that session by its id reach the same entry -- before the runtime moves on, after it, and after a reload. The search was the third place that asked about the current session rather than the displayed one; the other two were changed in the previous round. It now asks about the displayed session, and it checks that the answer names that session and that slice before the answer's indexes may filter the body on screen. An answer about another session leaves the previous result standing and says the search failed, which is a different sentence from 「matched nothing」. --- .../mmc-trajectory/miniapp/client/index.html | 17 +++++- .../mmc-trajectory/miniapp/node/server.mjs | 54 +++++++++++++++++-- 2 files changed, 67 insertions(+), 4 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 17d962b6..5617454e 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -4299,7 +4299,10 @@

检视

searchController = controller; var seq = (searchSeq += 1); state.searchPending = true; - var session = sessionParam(); + // The search is about the body on screen, so it asks about THAT session — the same + // question the raw and detail reads ask. Asking for the current session instead would + // answer with another session's records, and their indexes would then filter these rows. + var session = displayedSessionParam(); var url = '/api/trajectory/search?session=' + session + '&q=' + encodeURIComponent(asked) + (state.generation === null ? '' : '&generation=' + encodeURIComponent(String(state.generation))); @@ -4323,6 +4326,18 @@

检视

if (seq !== searchSeq) return; searchController = null; state.searchPending = false; + // The answer has to be about the body on screen BEFORE its indexes may filter that + // body. The server names the session it answered for, so the two can be compared: + // if they differ, the hits describe another session's records, and installing them + // would report a miss for a row that matches and a hit for one that does not. + var answered = body && body.session ? body.session : null; + var displayed = state.payload && state.payload.session ? state.payload.session : null; + if (!answered || !displayed || answered.id !== displayed.id + || answered.generation !== state.generation) { + var mismatch = new Error('session_mismatch'); + mismatch.code = 'session_mismatch'; + throw mismatch; + } state.searchHits = searchHitSet(body.hits); state.searchTotal = Number.isInteger(body.total) ? body.total : 0; state.searchTruncated = body.truncated === true; diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 5a7b6b4b..cb77106a 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -3346,6 +3346,34 @@ export function createTrajectoryCache(limit) { entries.set(key, hit); return hit.value; }, + /** + * The entry that answers for this family, if there is one. + * + * The key names a VERSION; the family names the QUESTION. Two requests that name the same + * directory and the same slice ask the same question, and the answer to it is the entry this + * cache holds -- but the key it was stored under can name a different version of it, because + * the key also carries the requester's context. A poll that asked for `latest` and a read that + * asked for that session by its id are the same reader of the same body, and they have to + * reach the same entry: the first stores the body under a key that carries the binding it + * resolved, the second looks it up under a key that does not, and the two keys differ for + * reasons that have nothing to do with the body. + * + * A family holds exactly one entry -- storing a newer one drops its predecessor -- so what is + * found here is the current answer to that question. + * + * @param {string} family + * @returns {any | null} + */ + getByFamily(family) { + for (const [key, hit] of entries) { + if (hit.family !== family) continue; + // Re-reading is the only signal that an entry is still hot, so it refreshes recency. + entries.delete(key); + entries.set(key, hit); + return hit.value; + } + return null; + }, /** * @param {string} key * @param {string} family the (session, slice) this answers for; a newer version of the same @@ -3987,7 +4015,20 @@ export async function start(context) { // Everything needed to decide "did this change?" is already in hand. Building the key // before the read is what lets an unchanged session skip the read entirely. - return { ok: true, entry, binding, identity, generations, filter, cacheKey: trajectoryCacheKey(entry, identity, binding, filter) }; + // + // The family travels with it: the key names the version and the family names the question, + // and a read that asks for this session by its id has to be able to reach the entry a poll + // stored for it even though the two computed different keys. + return { + ok: true, + entry, + binding, + identity, + generations, + filter, + cacheKey: trajectoryCacheKey(entry, identity, binding, filter), + family: trajectoryCacheFamily(entry.dir, filter), + }; }; /** @@ -4577,7 +4618,12 @@ export async function start(context) { sendError(response, 400, 'record_index_required', '缺少记录序号'); return; } - const cached = trajectoryCache.get(ctx.cacheKey); + // The key names the version and the family names the question. A read that names the + // session by its id and a poll that asked for `latest` are the same reader of the same + // body, so the second one falls back to the family's entry instead of reporting that + // nothing is cached: the two computed different keys because one of them carried the + // binding it resolved and the other did not, and that has nothing to do with the body. + const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family); if (!cached) { sendError(response, 503, 'trajectory_not_cached', '会话数据已释放，请重新加载'); return; @@ -4677,7 +4723,9 @@ export async function start(context) { sendError(response, 400, 'record_index_required', '缺少记录序号'); return; } - const cached = trajectoryCache.get(ctx.cacheKey); + // The key names the version and the family names the question — see `handleRaw` for why the + // family is the fallback. + const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family); if (!cached) { sendError(response, 503, 'trajectory_not_cached', '会话数据已释放，请重新加载'); return; From 00c16aa56d80845c34b936d64722c3aca9da7c89 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 06:02:49 +0800 Subject: [PATCH 26/95] The body of the session on screen is reachable, and the answer has to name it - the trajectory-cache guard now drives the route over HTTP: a store with two conversations and a runtime database that says which one is live, a poll that loads one of them, a read of one of its records by that conversation's id, the runtime moving on, and the read again -- before the switch, after it, and after a reload - it asserts the pieces separately as well: that the request resolves the family, that the cache can be asked for one, that reading by family refreshes recency, and that each of the two readers falls back to it - the client guard now asks the search about the session on screen, and checks the answer's session and slice against the displayed payload before its indexes may filter it -- including the other direction, that the answer about the conversation the page moved to is installed - the controls beside them: five for the cache -- one per piece, and one each for the two readers, since a control that changed both would leave the other assertion never asked whether it could fail -- and three for the search --- .../mmc-trajectory/tests/client-check.mjs | 113 +++++++- .../tests/mutate-search-client.py | 32 +++ .../tests/mutate-trajectory-cache.py | 64 +++++ .../tests/verify-trajectory-cache.mjs | 243 +++++++++++++++++- 4 files changed, 443 insertions(+), 9 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index 0c0a6fa7..1fbfbe1c 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -227,7 +227,8 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); const netCtx = { console, AbortController, setTimeout, clearTimeout, encodeURIComponent }; vm.createContext(netCtx); const needed = ['function isCount(value)', 'function formatInt(value)', 'function searchHitSet(hits)', - 'function sessionParam()', 'function searchStatusLabel(searchState)', 'function runSearch(needle)']; + 'function sessionParam()', 'function displayedSessionParam()', 'function searchStatusLabel(searchState)', + 'function runSearch(needle)']; const sources = needed.map((sig) => fnSource(sig)); check('the shipped search functions are all still findable by signature', needed.filter((sig, i) => !sources[i]), []); @@ -237,7 +238,10 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); // page's state is, because nothing else about the page's state is what is under test. 'var state = { query: "retry me", sessionId: null, hostSessionId: null, followLatest: true,', ' generation: null, searchHits: null, searchTotal: 0, searchTruncated: false,', - ' searchPending: false, searchError: null };', + ' searchPending: false, searchError: null,', + // The body on screen. `displayedSessionParam()` reads it and the answer is checked against + // it, so a harness with no body on screen cannot ask either question. + ' payload: { session: { id: "mvs_displayed" } } };', 'var searchSeq = 0;', 'var searchController = null;', // The two renderers paint the page; neither is what is being tested and neither has a DOM here. @@ -255,13 +259,25 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); for (let i = 0; i < 3; i += 1) await new Promise((done) => setImmediate(done)); }; - netCtx.fetch = () => Promise.reject(Object.assign(new Error('api'), { code: 'search_failed' })); + // Every URL the search asked for, so the question it asked can be read off the requests + // rather than out of the source. + const asked = []; + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.reject(Object.assign(new Error('api'), { code: 'search_failed' })); + }; netCtx.runSearch('retry me'); await settle(); const afterFailure = netCtx.searchStatusLabel(netCtx.state); - netCtx.fetch = () => Promise.resolve({ ok: true, status: 200, - json: () => Promise.resolve({ hits: [{ index: 7 }], total: 1, truncated: false }) }); + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ hits: [{ index: 7 }], total: 1, truncated: false, + // The server names the session it answered for. That name is what the page checks the + // answer against before it lets the hits filter the body on screen. + session: { id: 'mvs_displayed', generation: null } }) }); + }; netCtx.runSearch('retry me'); await settle(); @@ -271,6 +287,93 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); check('so the label reports the answer instead of the stale failure', netCtx.searchStatusLabel(netCtx.state), '命中 1 条'); check('and the hit set it describes is the new one', [...netCtx.state.searchHits], [7]); + + /* The search has to be about the body on screen, and the answer has to say so. + * + * The search used to ask for `latest`, which is a different question from 「what is on + * screen」: the runtime can make another conversation current while the page still shows + * this one, and the answer then describes another session's records. The hits are indexes, + * so they FILTER this body's rows against another body's matches — a row that matches + * reads as a miss, and a row that does not read as a hit. Both are wrong in opposite + * directions, which is why the number of hits alone never showed the defect. + * + * Two things have to hold and neither one is enough: + * + * - the request names the displayed session, and + * - the answer is checked against it. + * + * A request that names the displayed session can still be answered about another one if + * the runtime moved in between, and an answer that is checked can still be about another + * session if the wrong question was asked. Both halves are asserted below. + */ + check('the search asks about the session on screen', + /[?&]session=mvs_displayed(&|$)/.test(asked[1]), true); + check('and it does not ask about whatever is current now', + /[?&]session=latest(&|$)/.test(asked[1]), false); + + // The answer comes back about another session. The hits must not be installed: they are + // indexes into another body's records, and installing them would report this body's rows + // against another body's matches. + const installed = [...netCtx.state.searchHits]; + asked.length = 0; + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ hits: [{ index: 99 }], total: 1, truncated: false, + session: { id: 'mvs_other', generation: null } }) }); + }; + netCtx.runSearch('retry me'); + await settle(); + check('an answer about another session does not install its hit set', + [...netCtx.state.searchHits], installed); + check('and it does not report the other session\'s count either', + netCtx.state.searchTotal, 1); + check('it says the search failed, so the reader is told there is no answer', + netCtx.state.searchError, 'session_mismatch'); + check('and the label shows the failure rather than the hit count', + netCtx.searchStatusLabel(netCtx.state), '搜索失败，以下为上次结果'); + check('and the request it made still named the session on screen', + /[?&]session=mvs_displayed(&|$)/.test(asked[0]), true); + + // The same session, but another slice of it. A slice is a different body, so an answer + // about the whole lineage is not an answer about the slice on screen. + netCtx.state.generation = 3; + asked.length = 0; + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ hits: [{ index: 98 }], total: 1, truncated: false, + session: { id: 'mvs_displayed', generation: null } }) }); + }; + netCtx.runSearch('retry me'); + await settle(); + check('an answer about another slice of the same session does not install its hit set', + [...netCtx.state.searchHits], installed); + check('and the request named the slice that is on screen', + /[?&]generation=3(&|$)/.test(asked[0]), true); + check('and it failed for the same reason', + netCtx.state.searchError, 'session_mismatch'); + + // The other direction, because a check that rejects every answer is not a check. The page + // follows the runtime to another conversation, and the answer about THAT conversation is + // the one that belongs to the body on screen now. + netCtx.state.payload = { session: { id: 'mvs_displayed_b' } }; + netCtx.state.generation = null; + asked.length = 0; + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ hits: [{ index: 5 }], total: 1, truncated: false, + session: { id: 'mvs_displayed_b', generation: null } }) }); + }; + netCtx.runSearch('retry me'); + await settle(); + check('the answer about the conversation the page moved to is installed', + [...netCtx.state.searchHits], [5]); + check('and there is no failure, because this time there is an answer', + netCtx.state.searchError, null); + check('and the request asked about the conversation on screen at that moment', + /[?&]session=mvs_displayed_b(&|$)/.test(asked[0]), true); } /* flatten + signature */ diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py index a31ddd6b..668f37ef 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py @@ -99,6 +99,38 @@ " state.searchError = null;\n resetWindowMode();", " resetWindowMode();", ), + # ---- the search is about the body on screen --------------------------------------- + # + # The hits the server sends are INDEXES into one body's records, so they only mean + # anything against that body. The search used to ask for `latest` -- "whichever session is + # current now" -- and the runtime can make another conversation current while the page + # still shows this one. The answer then described another conversation's records, and the + # indexes filtered this body's rows against another body's matches: a row that matches + # read as a miss and a row that does not read as a hit. The count alone never showed it. + # + # Three controls, because the promise has two halves and one of them has two names. The + # request has to ask about the displayed session, and the ANSWER has to be checked against + # it -- by id and by the slice -- before its indexes may filter that body. A control that + # dropped both halves at once would leave the id half and the slice half never asked + # whether they could fail. + ( + "the search asks about the current session instead of the one on screen", + " var session = displayedSessionParam();", + " var session = sessionParam();", + ), + ( + "the search stops checking which session the answer is about", + " if (!answered || !displayed || answered.id !== displayed.id\n" + " || answered.generation !== state.generation) {", + " if (!answered || !displayed\n" + " || answered.generation !== state.generation) {", + ), + ( + "the search stops checking which slice of the session the answer is about", + " if (!answered || !displayed || answered.id !== displayed.id\n" + " || answered.generation !== state.generation) {", + " if (!answered || !displayed || answered.id !== displayed.id) {", + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py index f2442a7b..0c4d6b37 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py @@ -106,6 +106,70 @@ " return `${trajectoryCacheFamily(entry.dir, filter)}${identity.id ?? ''}|${lineage}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", " return `${trajectoryCacheFamily(entry.dir, filter)}${lineage}|${entry.sizeBytes}|${entry.lastActiveAt}|${identity.catalogBytes ?? ''}|${bind}`;", ), + # ---- the family as the way back to the body --------------------------------------- + # + # The key names the version and the family names the question. A poll that asked for + # `latest` and a read that asked for that session by its id are the same reader of the + # same body, but the two computed different keys -- the poll's key carries the binding it + # resolved and the read's does not -- so the read missed a body the service was holding + # and answered 「会话数据已释放，请重新加载」 for a conversation on screen. + # + # Five controls, because the promise is carried by five separate pieces and a control + # that broke two of them would leave the others never asked whether they could fail: + # the request has to resolve the family, the cache has to be able to look one up, the + # lookup has to refresh recency, and EACH of the two readers has to use it. + ( + "the request stops resolving the family, so neither reader can look one up", + SERVER, + " family: trajectoryCacheFamily(entry.dir, filter),\n", + "", + ), + ( + "the raw read stops falling back to the family, so a record of the conversation on screen is unreachable", + SERVER, + " // The key names the version and the family names the question — see `handleRaw` for why the\n" + " // family is the fallback.\n" + " const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family);", + " // The key names the version and the family names the question — see `handleRaw` for why the\n" + " // family is the fallback.\n" + " const cached = trajectoryCache.get(ctx.cacheKey);", + ), + ( + "the record read stops falling back to the family, so the four fields of a record are unreachable", + SERVER, + " // binding it resolved and the other did not, and that has nothing to do with the body.\n" + " const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family);", + " // binding it resolved and the other did not, and that has nothing to do with the body.\n" + " const cached = trajectoryCache.get(ctx.cacheKey);", + ), + ( + "the cache stops finding an entry by its family, so the fallback finds nothing", + SERVER, + " getByFamily(family) {\n" + " for (const [key, hit] of entries) {\n" + " if (hit.family !== family) continue;\n" + " // Re-reading is the only signal that an entry is still hot, so it refreshes recency.\n" + " entries.delete(key);\n" + " entries.set(key, hit);\n" + " return hit.value;\n" + " }\n" + " return null;\n" + " },", + " getByFamily(family) {\n" + " return null;\n" + " },", + ), + ( + "reading by family stops refreshing recency, so the family being read is the first evicted", + SERVER, + " if (hit.family !== family) continue;\n" + " // Re-reading is the only signal that an entry is still hot, so it refreshes recency.\n" + " entries.delete(key);\n" + " entries.set(key, hit);\n" + " return hit.value;", + " if (hit.family !== family) continue;\n" + " return hit.value;", + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs index cc0a20e2..e4293347 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs @@ -12,17 +12,34 @@ * and storing a new one drops its predecessor. Everything that decides whether the answer changed * stays in the key; everything that decides what the answer IS stays in the family. * - * Read-only: imports the pure helpers, never starts the runtime. + * Read-only: imports the pure helpers, and the last section starts the route against a store + * and a runtime database built here. * mutate-trajectory-cache.py breaks each of these in the shipped file. */ -import { readFileSync } from 'node:fs'; +import { mkdirSync, mkdtempSync, readFileSync } from 'node:fs'; +import { mkdir, writeFile, utimes } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; import { createTrajectoryCache, + start, trajectoryCacheFamily, trajectoryCacheKey, } from '../miniapp/node/server.mjs'; const SERVER = readFileSync(new URL('../miniapp/node/server.mjs', import.meta.url), 'utf8'); +/** The package root, so the route serves the same page the reader gets. */ +const PLUGIN_ROOT = fileURLToPath(new URL('..', import.meta.url)); + +// `node:sqlite` is still flagged experimental, and the flag goes to stderr the moment the module +// is first loaded. The runner reads `stdout + stderr` and prints the LAST non-empty line, so the +// warning would land where this guard's own result line should be. +const emitWarning = process.emitWarning; +process.emitWarning = (warning, ...rest) => ( + String(warning).startsWith('SQLite is an experimental feature') + ? undefined + : emitWarning.call(process, warning, ...rest)); let pass = 0; const failures = []; @@ -151,6 +168,24 @@ hot.set(keyOf('newcomer'), trajectoryCacheFamily('/store/newcomer', null), { bod assert('a family that was read again survives the next eviction', hot.get(keyOf('wanted'))?.body === 'wanted'); assert('the family that was NOT touched is the one evicted', hot.get(keyOf('other')) === null); +// Reading by family refreshes recency too, and for the same reason: an entry that was just +// read is hot, and a family the reader keeps asking for must not be the one evicted. A lookup +// that found the entry but left it where it was would make the family that is being read the +// first to go. +const hotFamily = createTrajectoryCache(2); +const familyKeyOf = (name) => trajectoryCacheKey( + { dir: `/store/${name}`, sizeBytes: 1, lastActiveAt: 1 }, identity, null, null, +); +hotFamily.set(familyKeyOf('wanted'), trajectoryCacheFamily('/store/wanted', null), { body: 'wanted' }); +hotFamily.set(familyKeyOf('other'), trajectoryCacheFamily('/store/other', null), { body: 'other' }); +// Touched by family, not by key: the reader asked the family and never computed this key. +hotFamily.getByFamily(trajectoryCacheFamily('/store/wanted', null)); +hotFamily.set(familyKeyOf('newcomer'), trajectoryCacheFamily('/store/newcomer', null), { body: 'newcomer' }); +assert('a family that was read by family survives the next eviction', + hotFamily.getByFamily(trajectoryCacheFamily('/store/wanted', null))?.body === 'wanted'); +assert('and the family that was not read is the one evicted', + hotFamily.getByFamily(trajectoryCacheFamily('/store/other', null)) === null); + /* ===================================================================== * 5 — structure: the cache is per-runtime, and the route names the family * ===================================================================== */ @@ -183,9 +218,209 @@ assert('the key names the lineage it stitches, so a rotated snapshot cannot reus assert('the route still releases both caches on dispose', /trajectoryCache\.clear\(\);/.test(startBody) && /parsedRows\.clear\(\);/.test(startBody)); +// The key names the version; the family names the question. A poll that asked for `latest` +// and a read that asked for that session by its id are the same reader of the same body, +// but they computed different keys: the poll's key carries the binding it resolved and the +// read's does not, and that has nothing to do with the body. So the read has to be able to +// ask for the family when the exact key misses. +// +// The two shapes are asserted separately on purpose. A guard that only checked the family +// would pass with a route that never stored one; a guard that only checked the store would +// pass with a route that never looked one up. +assert('the cache can be asked for the family it answers for', + typeof createTrajectoryCache(LIMIT).getByFamily === 'function'); +assert('the request resolves the family it will look up', + startBody.includes('family: trajectoryCacheFamily(entry.dir, filter)')); +assert('both readers fall back to the family when the exact key misses', + (startBody.match(/trajectoryCache\.get\(ctx\.cacheKey\) \?\? trajectoryCache\.getByFamily\(ctx\.family\)/g) || []).length === 2); + +// The family is what the two keys share, so it has to be looked up by the family and not by +// the key: an entry stored under one key is found under another whose family is the same. +const shared = createTrajectoryCache(LIMIT); +const boundKey = trajectoryCacheKey(entry(1000, 5000), identity, { sessionId: 'sess-a', title: 'A' }, null); +const unboundKey = trajectoryCacheKey(entry(1000, 5000), identity, null, null); +assert('the same body under two bindings is two keys', + boundKey !== unboundKey); +shared.set(boundKey, familyAll, { body: 'shared' }); +assert('and the exact key still finds it', shared.get(boundKey)?.body === 'shared'); +assert('the other key does not', + shared.get(unboundKey) === null); +assert('but the family finds it, which is what the readers do', + shared.getByFamily(familyAll)?.body === 'shared'); +assert('and a different family does not', + shared.getByFamily(familySlice) === null); + +/* ===================================================================== + * 6 — over HTTP: the body of the session on screen is reachable by its id + * ===================================================================== */ + +// The cache is where the body is held and the readers are where it is handed out, so the +// promise checked here is the one a reader sees: the conversation on screen can be opened, +// record by record. +// +// It used to answer 「会话数据已释放，请重新加载」 for a conversation the service was holding. +// The page polls `/api/trajectory?session=latest` and the body is stored under a key that +// carries the binding `latest` resolved; the page then reads one record through +// `/api/trajectory/raw?session=`, and that request carries no binding at +// all — so it computed a different key, missed, and reported that the session was gone. +// +// The client's own retry does not rescue it, and that is the part worth checking: the retry +// reloads the CURRENT conversation, which is not the one on screen, and asks again with the +// same key that missed. So the read after a reload is the same failure. +// +// The fixture carries a runtime database, and that is not decoration. The binding only +// exists when the runtime database answered; without it `latest` falls back to the newest +// file, the poll carries no binding either, and the two requests compute the SAME key — so +// the defect is invisible on a store with no runtime database, and a fixture without one +// would pass for the wrong reason. + +const REACH_ALPHA = 'mvs_reach_alpha'; +const REACH_BRAVO = 'mvs_reach_bravo'; + +try { + const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-reach-')); + const sessionsRoot = join(home, 'v2', 'sessions'); + + /** One conversation on disk: a transcript, the manifest that names it, and an mtime. */ + const makeConversation = async (id, body, mtimeMs) => { + const name = '00-00-00-session_' + Buffer.from(id, 'utf8').toString('base64url'); + const dir = join(sessionsRoot, '2026', '10', '22', name); + await mkdir(dir, { recursive: true }); + await writeFile(join(dir, 'messages.jsonl'), JSON.stringify({ + message_id: 'm1', + turn_id: 't1', + message: { role: 'user', content: body, timestamp: 1791600000000 }, + }) + '\n', 'utf8'); + await writeFile(join(dir, 'manifest.json'), JSON.stringify({ sessionId: id }), 'utf8'); + const stamp = new Date(mtimeMs); + await utimes(join(dir, 'messages.jsonl'), stamp, stamp); + return { id, dir, relative: '2026/10/22/' + name }; + }; + + const alpha = await makeConversation(REACH_ALPHA, 'ALPHA BODY', 1791600000000); + const bravo = await makeConversation(REACH_BRAVO, 'BRAVO BODY', 1791500000000); + + // The runtime database, which is the only place on this machine that knows which + // conversation is live. ALPHA is live first; BRAVO becomes live later in this section. + mkdirSync(join(home, 'v2', 'sqlite'), { recursive: true }); + const { DatabaseSync } = await import('node:sqlite'); + const dbPath = join(home, 'v2', 'sqlite', 'runtime-state.sqlite'); + const db = new DatabaseSync(dbPath); + db.exec('create table local_runtime_sessions (session_id text, title text, status text, ' + + 'history_relative_dir text, session_kind text, archived integer, updated_at_ms integer, ' + + 'parent_session_id text, purpose text, workspace_dir text)'); + db.exec('create table local_runtime_session_locks (session_id text, expires_at_ms integer)'); + const insert = db.prepare('insert into local_runtime_sessions values (?,?,?,?,?,0,?,null,null,null)'); + insert.run(REACH_ALPHA, 'Alpha', 'idle', alpha.relative, 'conversation', 200); + insert.run(REACH_BRAVO, 'Bravo', 'idle', bravo.relative, 'conversation', 100); + db.close(); + + const previousRoot = process.env.MMC_TRAJECTORY_ROOT; + process.env.MMC_TRAJECTORY_ROOT = home; + const controller = new AbortController(); + const noop = () => {}; + let instance = null; + let base = null; + for (let attempt = 0; attempt < 12 && !base; attempt += 1) { + const port = 19900 + Math.floor(Math.random() * 300) + attempt; + try { + instance = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + dataDir: PLUGIN_ROOT, + generationId: 'guard', + listen: { host: '127.0.0.1', port }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop }, + }); + base = 'http://127.0.0.1:' + port; + } catch (error) { + // The port was taken. Another one is tried; a guard that gave up here would be a + // guard that sometimes checks nothing. + } + } + if (!base) throw new Error('could not bind a port for the reachability route'); + + const get = async (path) => { + const response = await fetch(base + path); + let body = null; + try { body = await response.json(); } catch { body = null; } + return { status: response.status, body }; + }; + const why = (answer) => 'status ' + answer.status + ' ' + JSON.stringify(answer.body); + + try { + // 1 — the poll the page makes, and the read it makes right after it. + const poll = await get('/api/trajectory?session=latest'); + assert('the poll answers with the conversation the runtime says is live', + Boolean(poll.body && poll.body.session) && poll.body.session.id === REACH_ALPHA, + why(poll)); + + const rawById = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA); + assert('a record of the conversation on screen is readable by that conversation\'s id', + rawById.status === 200, why(rawById)); + assert('and it is that conversation\'s own message', + JSON.stringify(rawById.body && rawById.body.raw ? rawById.body.raw : '').includes('ALPHA BODY'), + why(rawById)); + + const detailById = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA); + assert('the four fields of a record are readable by that conversation\'s id', + detailById.status === 200, why(detailById)); + + // 2 — the runtime moves on while the page is still showing the first conversation. + const moved = new DatabaseSync(dbPath); + moved.exec('update local_runtime_sessions set updated_at_ms = 300 where session_id = ' + + '\'' + REACH_BRAVO + '\''); + moved.close(); + const poll2 = await get('/api/trajectory?session=latest'); + assert('the next poll follows the runtime to the conversation that is live now', + Boolean(poll2.body && poll2.body.session) && poll2.body.session.id === REACH_BRAVO, + why(poll2)); + + const rawAfter = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA); + assert('the conversation still on screen is still readable by its id', + rawAfter.status === 200, why(rawAfter)); + assert('and it is still its own message, not the other conversation\'s', + JSON.stringify(rawAfter.body && rawAfter.body.raw ? rawAfter.body.raw : '').includes('ALPHA BODY'), + why(rawAfter)); + + const detailAfter = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA); + assert('and its records\' fields are still readable by its id', + detailAfter.status === 200, why(detailAfter)); + + // 3 — the retry. The client's answer to a 503 is to reload the trajectory and ask again, + // so the read after a reload is the read a reader would make. + const reload = await get('/api/trajectory?session=latest'); + assert('the reload answers', reload.status === 200, why(reload)); + const rawRetry = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA); + assert('the read after a reload still finds the conversation on screen', + rawRetry.status === 200, why(rawRetry)); + const detailRetry = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA); + assert('and so does the record read', + detailRetry.status === 200, why(detailRetry)); + } finally { + // The server is released here and not at the end of the happy path: an assertion that + // throws would otherwise leave it listening, and the guard would hang instead of + // reporting. + if (instance) await instance.dispose(); + controller.abort(); + if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; + else process.env.MMC_TRAJECTORY_ROOT = previousRoot; + } +} catch (error) { + failures.push('the body of the session on screen is reachable by its id -- ' + + (error && error.message)); +} + if (failures.length) { console.error(`\n[trajectory-cache] ${failures.length} FAILED, ${pass} passed`); for (const f of failures) console.error(' - ' + f); - process.exit(1); + // `process.exitCode` rather than `process.exit(1)`: the route was just asked for, and an + // abrupt exit tears down the socket the answer arrived on, which on this platform ends in a + // libuv assertion and a crash instead of an exit code. A guard that CRASHES has not caught + // anything -- it has stopped -- and a mutation control that only saw a crash would not know + // which rule it broke. + process.exitCode = 1; +} else { + console.log(`\n[trajectory-cache] ${pass} assertions passed`); } -console.log(`\n[trajectory-cache] ${pass} assertions passed`); From 34b1e0eba0027b0d7b5309a9a4cd1f0c63858a0e Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 07:51:22 +0800 Subject: [PATCH 27/95] The body on screen says which slice it is, and the handoff keeps its session A generation the reader picked and the body on screen are two different things, and they come apart when the load of the picked generation fails: the selection says one thing and the body says another. The search asked about the selection, so its answer carried indexes of a slice that was not on screen -- and those indexes then filtered the rows the reader was looking at, reporting a miss for a row that matches and a hit for one that does not. The payload now says which slice it IS, instead of how many generations it happens to show, so the page can ask the server about the body it is showing. The search, the raw read, the record read and the browser's own raw cache key all follow it, and the answer's slice is checked against the body's before its indexes may filter that body. A cache entry now records which session it is about. The family names the question -- the directory and the slice -- and one family is asked by many readers, so the session cannot live in the family; but the fallback may not hand back another session's body either, and a directory can stop holding one session and start holding another. A read for the new session now says the body is not held, rather than answering with the old session's words. The handoff captures the session together with the record it is about. Its two reads are asynchronous, and the text it hands to the Host names the conversation -- a session read after the reads would name the conversation the reader had already moved on to, over another conversation's message. --- .../mmc-trajectory/miniapp/client/index.html | 70 ++++++++++++++++--- .../mmc-trajectory/miniapp/node/server.mjs | 40 ++++++++--- 2 files changed, 89 insertions(+), 21 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 5617454e..d1870a2e 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -4055,6 +4055,28 @@

检视

return sessionParam(); } + /** + * The slice of history the body on screen is, for a request that addresses a position + * inside it. + * + * A record index is a position within one body, and a body is one slice of one session's + * history. The reader's SELECTION and the body on screen are two different things: the + * reader can pick a generation, and the load of that generation can fail and leave the + * previous body on screen. A request built from the selection would then ask about a slice + * the page is not showing, and the answer's indexes would filter rows that are not there. + * + * So this reads the slice the payload says, which is the slice it was built for. The + * service sends it as the payload's own generation, and it means the same thing there as + * it does in a search answer: `null` is the whole lineage. + * + * @returns {number | null} + */ + function displayedGeneration() { + var payload = state.payload; + var session = payload && payload.session ? payload.session : null; + return session && typeof session.generation === 'number' ? session.generation : null; + } + /** * The session list, on its own validator. * @@ -4302,10 +4324,13 @@

检视

// The search is about the body on screen, so it asks about THAT session — the same // question the raw and detail reads ask. Asking for the current session instead would // answer with another session's records, and their indexes would then filter these rows. + // The slice is asked about the same way: the reader's selection and the body on screen + // are two different things, and a selection whose load failed leaves the old body up. var session = displayedSessionParam(); + var generation = displayedGeneration(); var url = '/api/trajectory/search?session=' + session + '&q=' + encodeURIComponent(asked) - + (state.generation === null ? '' : '&generation=' + encodeURIComponent(String(state.generation))); + + (generation === null ? '' : '&generation=' + encodeURIComponent(String(generation))); fetch(url, { signal: controller.signal, headers: { Accept: 'application/json' } }) .then(function (response) { @@ -4327,13 +4352,26 @@

检视

searchController = null; state.searchPending = false; // The answer has to be about the body on screen BEFORE its indexes may filter that - // body. The server names the session it answered for, so the two can be compared: - // if they differ, the hits describe another session's records, and installing them - // would report a miss for a row that matches and a hit for one that does not. + // body. The server names the session and the slice it answered for, so the two can + // be compared: if they differ, the hits describe another body's records, and + // installing them would report a miss for a row that matches and a hit for one that + // does not. + // + // The slice is compared against the body's own, not against the reader's selection: + // a selection whose load failed leaves the previous body on screen, and an answer + // about the selected slice would then filter rows that are not the ones on screen. var answered = body && body.session ? body.session : null; var displayed = state.payload && state.payload.session ? state.payload.session : null; + // The two slices are compared as the page reads them: a number names one generation and + // nothing names the whole lineage. So a body that does not carry the field at all and a + // body that carries the whole lineage are the same question, and only a real difference + // between two real generations is a mismatch. + var answeredSlice = answered && typeof answered.generation === 'number' + ? answered.generation : null; + var displayedSlice = displayed && typeof displayed.generation === 'number' + ? displayed.generation : null; if (!answered || !displayed || answered.id !== displayed.id - || answered.generation !== state.generation) { + || answeredSlice !== displayedSlice) { var mismatch = new Error('session_mismatch'); mismatch.code = 'session_mismatch'; throw mismatch; @@ -8663,7 +8701,10 @@

检视

var payload = state.payload; var session = payload && payload.session ? payload.session : null; var sid = session && typeof session.id === 'string' ? session.id : ''; - var gen = state.generation === null ? 'all' : String(state.generation); + // The slice the value belongs to is the slice the body on screen is, for the same + // reason the session is: a record index is a position inside one body, and the body on + // screen is not always the body the reader last selected. + var gen = displayedGeneration() === null ? 'all' : String(displayedGeneration()); // The validator belongs in this key. A record index is only a position within one version // of one body: the same index in the next version of the session is a different message. // Without the tag, a body that changed since the raw was read kept being served from here @@ -8674,8 +8715,9 @@

检视

function requestRaw(index) { var url = '/api/trajectory/raw?index=' + encodeURIComponent(String(index)) + '&session=' + displayedSessionParam(); - if (state.generation !== null) { - url += '&generation=' + encodeURIComponent(String(state.generation)); + var generation = displayedGeneration(); + if (generation !== null) { + url += '&generation=' + encodeURIComponent(String(generation)); } return fetch(url, { headers: { Accept: 'application/json' } }).then(function (response) { return response.json().then( @@ -8765,8 +8807,9 @@

检视

function requestDetail(index) { var url = '/api/trajectory/record?index=' + encodeURIComponent(String(index)) + '&session=' + displayedSessionParam(); - if (state.generation !== null) { - url += '&generation=' + encodeURIComponent(String(state.generation)); + var generation = displayedGeneration(); + if (generation !== null) { + url += '&generation=' + encodeURIComponent(String(generation)); } return fetch(url, { headers: { Accept: 'application/json' } }).then(function (response) { return response.json().then( @@ -8986,6 +9029,11 @@

检视

updateHandoffButton(); return; } + // The session is captured with the record, because the two reads below are asynchronous + // and the page can be showing another conversation by the time they answer. A label that + // names the session on screen at that moment would put one conversation's name on + // another conversation's message. + var session = state.payload ? state.payload.session : null; var record = entry.record; dom.handoffBtn.disabled = true; inspectorSay('正在读取原文…'); @@ -9006,7 +9054,7 @@

检视

: fetched.raw; var raw = truncateJson(rawSource, HANDOFF_RAW_LIMIT); var text = buildHandoffMarkdown( - state.payload ? state.payload.session : null, + session, entry.turn, record, raw.text, diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index cb77106a..e118bc95 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -3330,7 +3330,7 @@ export function createParsedRowsCache(budget) { * @param {number} limit */ export function createTrajectoryCache(limit) { - /** @type {Map} */ + /** @type {Map} */ const entries = new Map(); return { get size() { return entries.size; }, @@ -3347,7 +3347,7 @@ export function createTrajectoryCache(limit) { return hit.value; }, /** - * The entry that answers for this family, if there is one. + * The entry that answers for this family AND for this session, if there is one. * * The key names a VERSION; the family names the QUESTION. Two requests that name the same * directory and the same slice ask the same question, and the answer to it is the entry this @@ -3361,12 +3361,21 @@ export function createTrajectoryCache(limit) { * A family holds exactly one entry -- storing a newer one drops its predecessor -- so what is * found here is the current answer to that question. * + * The session is asked for as well, and that is not a second question: the family names the + * DIRECTORY and the slice, and a directory can stop holding one session and start holding + * another. The manifest beside the transcript is the authority on that, it can change under a + * cached walk, and the walk's own decision is not the last word. An entry built for the old + * session is therefore not an answer about the new one, however well the directory and the + * slice match -- and its decoded messages are the old session's messages. + * * @param {string} family + * @param {string} identityId the session the entry has to be about * @returns {any | null} */ - getByFamily(family) { + getByFamily(family, identityId) { for (const [key, hit] of entries) { if (hit.family !== family) continue; + if (hit.identityId !== identityId) continue; // Re-reading is the only signal that an entry is still hot, so it refreshes recency. entries.delete(key); entries.set(key, hit); @@ -3378,12 +3387,13 @@ export function createTrajectoryCache(limit) { * @param {string} key * @param {string} family the (session, slice) this answers for; a newer version of the same * family replaces the older one instead of stacking beside it + * @param {string} identityId the session this body is about * @param {any} value */ - set(key, family, value) { + set(key, family, identityId, value) { const previous = entries.get(key); if (previous !== undefined) entries.delete(key); - entries.set(key, { at: Date.now(), family, value }); + entries.set(key, { at: Date.now(), family, identityId, value }); for (const [existingKey, existing] of entries) { if (existingKey !== key && existing.family === family) entries.delete(existingKey); } @@ -4024,6 +4034,9 @@ export async function start(context) { entry, binding, identity, + // The session this request is about, for the reader that has to tell one session's cached + // body from another's. + identityId: identity.id ?? '', generations, filter, cacheKey: trajectoryCacheKey(entry, identity, binding, filter), @@ -4260,7 +4273,12 @@ export async function start(context) { truncated, redact, generations, - generation: shown.length === 1 ? shown[0].generation : null, + // The slice this body IS, which is the slice that was asked for. It used to be "the one + // generation the body happens to show", and that is a different question: a body built for + // the whole lineage and a body built for one generation have to be told apart, because the + // page compares a search answer against the body on screen before it lets that answer's + // indexes filter it. + generation: filter, orphans: lineage.orphans, identity: identityTable, openerByTurn: view.openerByTurn, @@ -4319,7 +4337,7 @@ export async function start(context) { // same while it goes stale. Everything that decides whether the answer changed stays in the // key, so a newer version of one family replaces its predecessor instead of stacking beside it. // `decoded` is filled in by /raw, at most RAW_DECODE_CACHE_ENTRIES lines' worth. - trajectoryCache.set(cacheKey, trajectoryCacheFamily(entry.dir, filter), { at: Date.now(), etag, body, raws, rowPos: rowPosByRecord, decoded: new Map() }); + trajectoryCache.set(cacheKey, trajectoryCacheFamily(entry.dir, filter), identity.id ?? '', { at: Date.now(), etag, body, raws, rowPos: rowPosByRecord, decoded: new Map() }); // Nothing below reads `rows` or `rowLocators`, and `payload` holds its own projections rather // than the rows, so this is where the object graph stops being reachable. A sealed generation's // rows are still held by the parsed-rows cache — that is the point of that cache — but the @@ -4623,7 +4641,9 @@ export async function start(context) { // body, so the second one falls back to the family's entry instead of reporting that // nothing is cached: the two computed different keys because one of them carried the // binding it resolved and the other did not, and that has nothing to do with the body. - const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family); + // The session is part of the question: a directory can stop holding one session and start + // holding another, and the old session's body is not an answer about the new one. + const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family, ctx.identityId); if (!cached) { sendError(response, 503, 'trajectory_not_cached', '会话数据已释放，请重新加载'); return; @@ -4724,8 +4744,8 @@ export async function start(context) { return; } // The key names the version and the family names the question — see `handleRaw` for why the - // family is the fallback. - const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family); + // family is the fallback, and why the session is part of it. + const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family, ctx.identityId); if (!cached) { sendError(response, 503, 'trajectory_not_cached', '会话数据已释放，请重新加载'); return; From 6739f89fe2bfdbdf05394b5f11b4c313e51b4cc4 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 07:51:40 +0800 Subject: [PATCH 28/95] The guards that own the slice, the session identity and the handoff - the trajectory-cache guard grew a section that drives the route over HTTP for the session identity: a directory whose name does not decode, so the manifest beside the transcript is the only identity it has; a load of that conversation; a read of one of its records that warms the decode cache; the manifest and the transcript replaced; and a read for the new conversation, which must not answer with the old one's message - it asserts the pieces separately as well: that an entry is found for the session it is about and not for another one, and that the route records the session while the request resolves it. Those two shape assertions come after the HTTP section on purpose -- the behaviour has to be asked first, or a mutation is reported as caught while the behaviour it is about was never asked whether it could fail - the client guard grew three sections: the search's slice, where the body on screen and the generation the reader picked come apart; the secondary reads' slice, including the browser's own raw cache key; and the handoff, where a deferred raw read lets the page move to another conversation while the read is still in flight - the controls beside them: three for the cache -- the lookup that no longer checks the session, the route that no longer records it, the request that no longer resolves it -- four for the search client, one each for the search, the raw read, the record read and the cache key -- and a new control file for the handoff, with one that reads the session after the reads and one that does not read it at all - the raw-locators guard stopped taking a fixed number of characters after the cache key function and now takes the whole function: a window that stops being the function when a comment grows fails on the edit rather than on the defect it is about - the cache control also started requiring that a failure NAME a rule, the way the other controls already did. It printed the name before, but did not insist on one. --- .../mmc-trajectory/tests/client-check.mjs | 171 ++++++++++++++++-- .../tests/mutate-handoff-identity.py | 121 +++++++++++++ .../tests/mutate-search-client.py | 44 ++++- .../tests/mutate-trajectory-cache.py | 80 ++++++-- .../tests/verify-raw-locators.mjs | 8 +- .../tests/verify-trajectory-cache.mjs | 159 ++++++++++++++-- 6 files changed, 529 insertions(+), 54 deletions(-) create mode 100644 plugins/avatasia/mmc-trajectory/tests/mutate-handoff-identity.py diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index 1fbfbe1c..86fd5bb9 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -227,8 +227,8 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); const netCtx = { console, AbortController, setTimeout, clearTimeout, encodeURIComponent }; vm.createContext(netCtx); const needed = ['function isCount(value)', 'function formatInt(value)', 'function searchHitSet(hits)', - 'function sessionParam()', 'function displayedSessionParam()', 'function searchStatusLabel(searchState)', - 'function runSearch(needle)']; + 'function sessionParam()', 'function displayedSessionParam()', 'function displayedGeneration()', + 'function searchStatusLabel(searchState)', 'function runSearch(needle)']; const sources = needed.map((sig) => fnSource(sig)); check('the shipped search functions are all still findable by signature', needed.filter((sig, i) => !sources[i]), []); @@ -240,8 +240,9 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); ' generation: null, searchHits: null, searchTotal: 0, searchTruncated: false,', ' searchPending: false, searchError: null,', // The body on screen. `displayedSessionParam()` reads it and the answer is checked against - // it, so a harness with no body on screen cannot ask either question. - ' payload: { session: { id: "mvs_displayed" } } };', + // it, so a harness with no body on screen cannot ask either question. Its `generation` is + // the slice the body IS: null is the whole lineage, a number is that one generation. + ' payload: { session: { id: "mvs_displayed", generation: null } } };', 'var searchSeq = 0;', 'var searchController = null;', // The two renderers paint the page; neither is what is being tested and neither has a DOM here. @@ -336,14 +337,20 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); /[?&]session=mvs_displayed(&|$)/.test(asked[0]), true); // The same session, but another slice of it. A slice is a different body, so an answer - // about the whole lineage is not an answer about the slice on screen. + // about one slice is not an answer about another. + // + // The slice on screen is the body the page is SHOWING, not the generation the reader picked. + // The two come apart when the load of a picked generation fails: the selection says one thing + // and the body on screen says another, and only the body's own slice is the question the + // search may ask and answer. + netCtx.state.payload = { session: { id: 'mvs_displayed', generation: 3 } }; netCtx.state.generation = 3; asked.length = 0; netCtx.fetch = (url) => { asked.push(String(url)); return Promise.resolve({ ok: true, status: 200, json: () => Promise.resolve({ hits: [{ index: 98 }], total: 1, truncated: false, - session: { id: 'mvs_displayed', generation: null } }) }); + session: { id: 'mvs_displayed', generation: 2 } }) }); }; netCtx.runSearch('retry me'); await settle(); @@ -354,10 +361,51 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); check('and it failed for the same reason', netCtx.state.searchError, 'session_mismatch'); + // The reader picks another generation and the load of it FAILS. The body on screen is still + // the old one; the selection is the new one. A search that asked about the selection would be + // asking about a body that is not there, and the answer it got back would carry indexes of + // THAT body — which then filtered the rows the reader is actually looking at. + netCtx.state.payload = { session: { id: 'mvs_displayed', generation: 0 } }; + netCtx.state.generation = 1; + asked.length = 0; + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ hits: [{ index: 1 }], total: 1, truncated: false, + session: { id: 'mvs_displayed', generation: 0 } }) }); + }; + netCtx.runSearch('retry me'); + await settle(); + check('the search asks about the slice on screen, not the one the reader picked', + /[?&]generation=0(&|$)/.test(asked[0]), true); + check('and it does not ask about the selection', + /[?&]generation=1(&|$)/.test(asked[0]), false); + check('so the answer about the body on screen is installed', + [...netCtx.state.searchHits], [1]); + check('and there is no failure, because the answer and the body agree', + netCtx.state.searchError, null); + + // The other direction, because a check that accepts every answer is not a check either. An + // answer about the SELECTED slice is not an answer about the body on screen, and its indexes + // must not filter that body. + asked.length = 0; + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ hits: [{ index: 42 }], total: 1, truncated: false, + session: { id: 'mvs_displayed', generation: 1 } }) }); + }; + netCtx.runSearch('retry me'); + await settle(); + check('an answer about the selected slice does not install its hit set', + [...netCtx.state.searchHits], [1]); + check('and it says the answer was about another body', + netCtx.state.searchError, 'session_mismatch'); + // The other direction, because a check that rejects every answer is not a check. The page // follows the runtime to another conversation, and the answer about THAT conversation is // the one that belongs to the body on screen now. - netCtx.state.payload = { session: { id: 'mvs_displayed_b' } }; + netCtx.state.payload = { session: { id: 'mvs_displayed_b', generation: null } }; netCtx.state.generation = null; asked.length = 0; netCtx.fetch = (url) => { @@ -376,6 +424,67 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); /[?&]session=mvs_displayed_b(&|$)/.test(asked[0]), true); } +/* handoff: the session the record came from, across a navigation. + * + * The handoff reads two things asynchronously — the record's raw message and its four bulk + * fields — and the page can be showing another conversation by the time they answer. The text + * that goes to the Host NAMES the session, so a session read at the END of the reads would put + * one conversation's name on another conversation's message: the reader would be told which + * conversation they are looking at, and told wrong. + * + * This runs the shipped `handoffSelected` over a deferred raw read and moves the page while + * that read is in flight, which is the only moment the two can differ. */ +{ + const handoffSources = { + handoff: fnSource('function handoffSelected()'), + selected: fnSource('function selectedRecord()'), + bridge: fnSource('function hostBridge()'), + }; + check('the shipped handoff functions are all still findable by signature', + Object.keys(handoffSources).filter((key) => !handoffSources[key]), []); + + if (handoffSources.handoff && handoffSources.selected && handoffSources.bridge) { + const handoffCtx = { console, setTimeout, clearTimeout }; + vm.createContext(handoffCtx); + // The page's own pure helpers, so the markdown is the shipped one and not a copy of it. + vm.runInContext(block, handoffCtx); + const appended = []; + // The Host bridge, so `hostBridge()` finds a message sink without being stubbed away. + handoffCtx.window = { mcode: { miniapp: { message: { append: (text) => { appended.push(text); } } } } }; + handoffCtx.dom = { handoffBtn: { disabled: false }, inspectorStatus: { textContent: '' } }; + handoffCtx.state = { + selectedId: 'rec-1', + flat: [{ + id: 'rec-1', + record: { id: 'rec-1', index: 1, kind: 'user', role: 'user', timestamp: 1791600000000 }, + turn: { id: 't1', index: 3 }, + }], + // The conversation the selected record belongs to. + payload: { session: { id: 'mvs_a', label: 'A 会话' } }, + }; + let releaseRaw = null; + handoffCtx.rawSourceFor = () => new Promise((resolve) => { releaseRaw = resolve; }); + handoffCtx.fetchDetail = () => Promise.resolve(null); + handoffCtx.inspectorSay = () => {}; + handoffCtx.stripVolatile = (record) => record; + vm.runInContext( + handoffSources.handoff + '\n' + handoffSources.selected + '\n' + handoffSources.bridge, + handoffCtx, + ); + + handoffCtx.handoffSelected(); + // The page moves to another conversation while the record's raw read is still in flight. + handoffCtx.state.payload = { session: { id: 'mvs_b', label: 'B 会话' } }; + if (releaseRaw) releaseRaw({ raw: '{"kind":"user"}', omitted: false }); + for (let i = 0; i < 5; i += 1) await new Promise((done) => setImmediate(done)); + + check('the handoff names the session the selected record came from', + appended.length === 1 && appended[0].includes('mvs_a'), true); + check('and not the conversation the page moved to while the read was pending', + appended.length === 1 && appended[0].includes('mvs_b'), false); + } +} + /* flatten + signature */ check('flatten empty', sandbox.flattenTurns(undefined), []); check('flatten junk', sandbox.flattenTurns([{ id: 'a' }, null, { id: 'b', records: [null, { id: 'r' }] }]).length, 1); @@ -1140,11 +1249,14 @@ const checks = [ if (!src) return false; return /state\.searchHits = null;/.test(src) && /state\.searchError = null;/.test(src); })()], - ['a search is asked about the slice the reader is looking at', (() => { + ['a search is asked about the slice the body on screen is', (() => { const src = fnSource('function runSearch(needle)'); if (!src) return false; return /'\/api\/trajectory\/search\?session='/.test(src) - && /&generation=' \+ encodeURIComponent\(String\(state\.generation\)\)/.test(src); + && /&generation=' \+ encodeURIComponent\(String\(generation\)\)/.test(src) + // The slice the reader PICKED is a different question, and the page has to ask the one + // that belongs to the body it is showing. + && !/String\(state\.generation\)/.test(src); })()], // Clicking the window button must not move the page. It scrolled to the document tail // whenever it narrowed the window, on the theory that "back to the tail" should land on the @@ -1841,10 +1953,14 @@ if (!fetchTrajectorySrc || !sessionParamSrc || !trajInflightDecl || !trajSeqDecl const requestRawSrc = fnSource('function requestRaw(index)'); const requestDetailSrc = fnSource('function requestDetail(index)'); const displayedSessionSrc = fnSource('function displayedSessionParam()'); -if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc) { +const displayedGenerationSrc = fnSource('function displayedGeneration()'); +const rawCacheKeySrc = fnSource('function rawCacheKey(index)'); +if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGenerationSrc + || !rawCacheKeySrc) { failures.push('secondary reads: could not lift requestRaw, requestDetail and the session they ask for'); } else { - vm.runInContext(displayedSessionSrc + '\n' + requestRawSrc + '\n' + requestDetailSrc, sandbox); + vm.runInContext(displayedSessionSrc + '\n' + displayedGenerationSrc + '\n' + + requestRawSrc + '\n' + requestDetailSrc + '\n' + rawCacheKeySrc, sandbox); const secondary = []; sandbox.fetch = function (url) { @@ -1858,7 +1974,7 @@ if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc) { // The page is following the newest conversation, and it is showing A. The runtime may already // have made B current — the page does not know that, and it must not care: A is on screen. sandbox.state = { - payload: { session: { id: 'mvs_displayed' } }, + payload: { session: { id: 'mvs_displayed', generation: null } }, generation: null, etag: '"tag"', followLatest: true, sessionId: '', hostSessionId: '' }; await sandbox.requestRaw(1); @@ -1872,12 +1988,41 @@ if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc) { check('and it does not ask for whatever is current now', /[?&]session=latest(&|$)/.test(secondary[1]), false); + // The slice on screen is not the generation the reader picked. A selection whose load failed + // leaves the previous body up, and a read that asks about the selection asks about a body + // that is not there — its answer is about records the reader cannot see. + sandbox.state = { + payload: { session: { id: 'mvs_displayed', generation: 2 } }, + generation: 7, etag: '"tag"', followLatest: true, sessionId: '', hostSessionId: '' + }; + await sandbox.requestRaw(1); + await sandbox.requestDetail(1); + check('a raw read asks for the slice on screen', + /[?&]generation=2(&|$)/.test(secondary[2]), true); + check('and not for the one the reader picked', + /[?&]generation=7(&|$)/.test(secondary[2]), false); + check('a detail read asks for the slice on screen too', + /[?&]generation=2(&|$)/.test(secondary[3]), true); + check('and not for the one the reader picked', + /[?&]generation=7(&|$)/.test(secondary[3]), false); + + // The browser's own raw cache is keyed the same way, and for the same reason: a record index + // is a position inside one body, and the body on screen is not always the body the reader last + // selected. A key built from the selection would serve the previous slice's message under the + // current slice's record — and the reader would be reading another generation's words. + check('the raw cache key names the slice on screen', + sandbox.rawCacheKey(1).includes('#2#'), true); + check('and not the one the reader picked', + sandbox.rawCacheKey(1).includes('#7#'), false); + check('and it still names the session on screen', + sandbox.rawCacheKey(1).startsWith('mvs_displayed#'), true); + // With nothing on screen there is no session to name, and the read falls back to the session // parameter rather than sending an empty one. sandbox.state = { payload: null, generation: null, etag: '', followLatest: true, sessionId: '', hostSessionId: '' }; await sandbox.requestRaw(1); check('with no body on screen the read falls back to the current session', - /[?&]session=latest(&|$)/.test(secondary[2]), true); + /[?&]session=latest(&|$)/.test(secondary[4]), true); } console.log('[2] pure helpers: ' + pass + ' passed, ' + failures.length + ' failed'); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-handoff-identity.py b/plugins/avatasia/mmc-trajectory/tests/mutate-handoff-identity.py new file mode 100644 index 00000000..c3d05878 --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-handoff-identity.py @@ -0,0 +1,121 @@ +"""Mutation control for the session the handoff names. + +The handoff reads two things asynchronously — the record's untouched message and its four bulk +fields — and the page can be showing another conversation by the time they answer. The report it +hands to the Host NAMES the session, so the session has to be the one the selected record came +from, captured before the reads and not looked up after them. + +Reading it after them is what the first mutation below does, and it does it with getters rather +than with a moved line: the object handed to the markdown is captured, but its fields are read +when the markdown is built — which is after the reads have answered, and by then the page may be +showing somebody else's conversation. The getter is not how anyone writes code; it is how this +one defect is reproduced in a single edit, so the control cannot be satisfied by an +implementation that merely happens to look right. + +The second mutation is the blunter half: no session at all. + +Both have to be caught by the same named assertion in client-check.mjs, which runs the shipped +`handoffSelected` over a deferred raw read and moves the page while that read is in flight. +""" +import subprocess +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parent +CLIENT = ROOT / "miniapp" / "client" / "index.html" +PROBE = HERE / "client-check.mjs" + +MUTATIONS = [ + ( + "the handoff reads the session off the page after its reads, so a navigation names the wrong conversation", + " var session = state.payload ? state.payload.session : null;", + " var session = { get id() { return state.payload.session.id; }, get label() { return state.payload.session.label; } };", + ), + ( + "the handoff names no conversation at all", + " var session = state.payload ? state.payload.session : null;", + " var session = null;", + ), +] + + +def read_exact(path): + """Normalise to LF for matching, and report the file's own newline so it can be put back.""" + data = path.read_bytes() + text = data.decode("utf-8").replace("\r\n", "\n") + return text, ("\r\n" if b"\r\n" in data else "\n") + + +def write_exact(path, text, newline): + if newline == "\r\n": + text = text.replace("\n", "\r\n") + path.write_bytes(text.encode("utf-8")) + + +def run_probe(): + proc = subprocess.run( + ["node", str(PROBE)], + cwd=str(ROOT), + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + ) + return proc.returncode, (proc.stdout or "") + (proc.stderr or "") + + +def main(): + code, output = run_probe() + if code != 0: + print("baseline probe does not pass; fix that first") + print(output) + return 1 + print("baseline clean") + + original = CLIENT.read_bytes() + missed = [] + unnamed = [] + for label, old, new in MUTATIONS: + text, newline = read_exact(CLIENT) + if old not in text: + print(f"SKIP {label}: anchor not found") + missed.append(label) + continue + if old == new: + print(f"SKIP {label}: the mutation changes nothing") + missed.append(label) + continue + write_exact(CLIENT, text.replace(old, new, 1), newline) + try: + code, output = run_probe() + finally: + CLIENT.write_bytes(original) + if code == 0: + print(f"MISSED {label}: the probe still passed") + missed.append(label) + continue + first = next((line.strip() for line in output.splitlines() if line.strip().startswith("- ")), None) + if first is None: + print(f"UNNAMED {label}: the probe failed without naming a rule") + unnamed.append(label) + else: + print(f"caught {label}\n -> {first}") + + if CLIENT.read_bytes() != original: + print(f"NOT RESTORED {CLIENT}") + return 1 + print("file restored byte-for-byte") + + if missed or unnamed: + if missed: + print(f"\n{len(missed)} mutation(s) got through") + if unnamed: + print(f"{len(unnamed)} mutation(s) failed without naming a rule") + return 1 + print(f"\nall {len(MUTATIONS)} mutations caught, each by name") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py index 668f37ef..dbe521bf 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py @@ -74,7 +74,7 @@ ), ( "the needle is sent without the generation, so a slice search answers about the whole lineage", - " + (state.generation === null ? '' : '&generation=' + encodeURIComponent(String(state.generation)));", + " + (generation === null ? '' : '&generation=' + encodeURIComponent(String(generation)));", " + '';", ), ( @@ -121,16 +121,52 @@ ( "the search stops checking which session the answer is about", " if (!answered || !displayed || answered.id !== displayed.id\n" - " || answered.generation !== state.generation) {", + " || answeredSlice !== displayedSlice) {", " if (!answered || !displayed\n" - " || answered.generation !== state.generation) {", + " || answeredSlice !== displayedSlice) {", ), ( "the search stops checking which slice of the session the answer is about", " if (!answered || !displayed || answered.id !== displayed.id\n" - " || answered.generation !== state.generation) {", + " || answeredSlice !== displayedSlice) {", " if (!answered || !displayed || answered.id !== displayed.id) {", ), + # ---- the slice is the one the BODY is, not the one the reader picked --------------- + # + # The reader's selection and the body on screen are two different things, and they come + # apart when the load of a picked generation FAILS: the selection says one thing and the + # body says another. Every read that asks the server about "a record inside this body" has + # to ask about the BODY's own slice — the search, the raw read, the record read, and the + # browser's own cache key. A request built from the selection asks about a body that is not + # there, and the answer it gets back carries indexes of THAT body. + ( + "the search asks about the generation the reader picked, not the slice on screen", + " var session = displayedSessionParam();\n var generation = displayedGeneration();", + " var session = displayedSessionParam();\n var generation = state.generation;", + ), + ( + "the raw read asks about the generation the reader picked, not the slice on screen", + " var url = '/api/trajectory/raw?index=' + encodeURIComponent(String(index))\n" + " + '&session=' + displayedSessionParam();\n" + " var generation = displayedGeneration();", + " var url = '/api/trajectory/raw?index=' + encodeURIComponent(String(index))\n" + " + '&session=' + displayedSessionParam();\n" + " var generation = state.generation;", + ), + ( + "the record read asks about the generation the reader picked, not the slice on screen", + " var url = '/api/trajectory/record?index=' + encodeURIComponent(String(index))\n" + " + '&session=' + displayedSessionParam();\n" + " var generation = displayedGeneration();", + " var url = '/api/trajectory/record?index=' + encodeURIComponent(String(index))\n" + " + '&session=' + displayedSessionParam();\n" + " var generation = state.generation;", + ), + ( + "the raw cache key names the generation the reader picked, not the slice on screen", + " var gen = displayedGeneration() === null ? 'all' : String(displayedGeneration());", + " var gen = state.generation === null ? 'all' : String(state.generation);", + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py index 0c4d6b37..f49c08d4 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py @@ -47,8 +47,8 @@ ( "the route names the whole key as the family, so every version is its own family", SERVER, - "trajectoryCache.set(cacheKey, trajectoryCacheFamily(entry.dir, filter), { at: Date.now(), etag, body, raws, rowPos: rowPosByRecord, decoded: new Map() });", - "trajectoryCache.set(cacheKey, cacheKey, { at: Date.now(), etag, body, raws, rowPos: rowPosByRecord, decoded: new Map() });", + "trajectoryCache.set(cacheKey, trajectoryCacheFamily(entry.dir, filter), identity.id ?? '', { at: Date.now(), etag, body, raws, rowPos: rowPosByRecord, decoded: new Map() });", + "trajectoryCache.set(cacheKey, cacheKey, identity.id ?? '', { at: Date.now(), etag, body, raws, rowPos: rowPosByRecord, decoded: new Map() });", ), ( "the family stops naming the slice, so one session's generations evict each other", @@ -128,26 +128,29 @@ "the raw read stops falling back to the family, so a record of the conversation on screen is unreachable", SERVER, " // The key names the version and the family names the question — see `handleRaw` for why the\n" - " // family is the fallback.\n" - " const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family);", + " // family is the fallback, and why the session is part of it.\n" + " const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family, ctx.identityId);", " // The key names the version and the family names the question — see `handleRaw` for why the\n" - " // family is the fallback.\n" + " // family is the fallback, and why the session is part of it.\n" " const cached = trajectoryCache.get(ctx.cacheKey);", ), ( "the record read stops falling back to the family, so the four fields of a record are unreachable", SERVER, - " // binding it resolved and the other did not, and that has nothing to do with the body.\n" - " const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family);", - " // binding it resolved and the other did not, and that has nothing to do with the body.\n" + " // The session is part of the question: a directory can stop holding one session and start\n" + " // holding another, and the old session's body is not an answer about the new one.\n" + " const cached = trajectoryCache.get(ctx.cacheKey) ?? trajectoryCache.getByFamily(ctx.family, ctx.identityId);", + " // The session is part of the question: a directory can stop holding one session and start\n" + " // holding another, and the old session's body is not an answer about the new one.\n" " const cached = trajectoryCache.get(ctx.cacheKey);", ), ( "the cache stops finding an entry by its family, so the fallback finds nothing", SERVER, - " getByFamily(family) {\n" + " getByFamily(family, identityId) {\n" " for (const [key, hit] of entries) {\n" " if (hit.family !== family) continue;\n" + " if (hit.identityId !== identityId) continue;\n" " // Re-reading is the only signal that an entry is still hot, so it refreshes recency.\n" " entries.delete(key);\n" " entries.set(key, hit);\n" @@ -163,13 +166,47 @@ "reading by family stops refreshing recency, so the family being read is the first evicted", SERVER, " if (hit.family !== family) continue;\n" + " if (hit.identityId !== identityId) continue;\n" " // Re-reading is the only signal that an entry is still hot, so it refreshes recency.\n" " entries.delete(key);\n" " entries.set(key, hit);\n" " return hit.value;", " if (hit.family !== family) continue;\n" + " if (hit.identityId !== identityId) continue;\n" " return hit.value;", ), + # ---- the session the entry is about ------------------------------------------------ + # + # The family names the QUESTION — the directory and the slice — and one family is asked by + # many readers, so the family cannot also carry the session. The entry remembers which + # session it is about, and the lookup names the session it wants. + # + # Three controls, because the promise has three pieces and a control that broke two of them + # would leave the third never asked whether it could fail: the lookup has to CHECK the + # session, the route has to RECORD it, and the request has to RESOLVE it. + # + # This is the defect the acceptance review found: a directory can stop holding one session + # and start holding another, and the family then finds the OLD session's entry — whose + # decoded messages are the old session's messages. A read for the new session answered with + # the old one's words. + ( + "the family lookup stops checking the session the entry is about", + SERVER, + " if (hit.identityId !== identityId) continue;\n", + "", + ), + ( + "the route stops recording the session the entry is about", + SERVER, + "trajectoryCache.set(cacheKey, trajectoryCacheFamily(entry.dir, filter), identity.id ?? '', { at: Date.now(), etag, body, raws, rowPos: rowPosByRecord, decoded: new Map() });", + "trajectoryCache.set(cacheKey, trajectoryCacheFamily(entry.dir, filter), '', { at: Date.now(), etag, body, raws, rowPos: rowPosByRecord, decoded: new Map() });", + ), + ( + "the request stops resolving the session it will look up", + SERVER, + " identityId: identity.id ?? '',\n", + " identityId: '',\n", + ), ] @@ -208,6 +245,7 @@ def main(): originals = {path: path.read_bytes() for path in {m[1] for m in MUTATIONS}} missed = [] + unnamed = [] for label, path, old, new in MUTATIONS: text, newline = read_exact(path) if old not in text: @@ -222,11 +260,18 @@ def main(): if code == 0: print(f"MISSED {label}: the probe still passed") missed.append(label) + continue + # A probe that failed without naming a rule has not said WHICH rule broke, so it has not + # shown that the guard owns this mutation. It is weaker than "caught", and it is reported + # as its own thing rather than folded into the count of survivors. + first = next( + (line.strip() for line in output.splitlines() if line.strip().startswith("- ")), + None, + ) + if first is None: + print(f"UNNAMED {label}: the probe failed without naming a rule") + unnamed.append(label) else: - first = next( - (line.strip() for line in output.splitlines() if line.strip().startswith("- ")), - "(no detail)", - ) print(f"caught {label}\n -> {first}") for path, data in originals.items(): @@ -235,10 +280,13 @@ def main(): return 1 print("all files restored byte-for-byte") - if missed: - print(f"\n{len(missed)} mutation(s) got through") + if missed or unnamed: + if missed: + print(f"\n{len(missed)} mutation(s) got through") + if unnamed: + print(f"{len(unnamed)} mutation(s) failed without naming a rule") return 1 - print(f"\nall {len(MUTATIONS)} mutations caught") + print(f"\nall {len(MUTATIONS)} mutations caught, each by name") return 0 diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-raw-locators.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-raw-locators.mjs index 0faf2caa..67785eb9 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-raw-locators.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-raw-locators.mjs @@ -275,7 +275,13 @@ assert('the decode cache is per entry, so a new body starts empty', const rawCacheKeyAt = CLIENT.indexOf('function rawCacheKey(index)'); assert('rawCacheKey exists', rawCacheKeyAt > 0); -const keyBody = CLIENT.slice(rawCacheKeyAt, rawCacheKeyAt + 900); +// The whole function, not a fixed number of characters. A window of N characters stops being +// the function the moment a comment grows, and the assertion then fails on the edit rather +// than on the defect it is about — which is how a guard starts reporting somebody else's work. +const keyEnd = rawCacheKeyAt >= 0 ? CLIENT.indexOf('\n }', rawCacheKeyAt) : -1; +const keyBody = rawCacheKeyAt >= 0 && keyEnd > rawCacheKeyAt + ? CLIENT.slice(rawCacheKeyAt, keyEnd) + : ''; assert("the page's raw cache key includes the validator", /state\.etag/.test(keyBody)); assert('and still the session and the index', keyBody.includes("sid + '#' + gen + '#'")); // A raw the server refuses is a hint, not a broken tab, so there is deliberately no second chance diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs index e4293347..d44d3eaa 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs @@ -54,6 +54,8 @@ function assert(name, cond) { const LIMIT = 4; const DIR = '/store/session-a'; +/** The session the entries in the unit section below are about. */ +const SID = 'sess-unit'; const entry = (size, mtime) => ({ dir: DIR, sizeBytes: size, lastActiveAt: mtime }); const identity = { catalogBytes: 1234, generations: [{ generation: 0, fileName: 'g0', byteLength: 10 }] }; @@ -103,10 +105,10 @@ assert('an identity is not the same key as no identity at all', keyIdentityA !== * ===================================================================== */ const cache = createTrajectoryCache(LIMIT); -cache.set(keyFresh, familyAll, { body: 'v1' }); +cache.set(keyFresh, familyAll, SID, { body: 'v1' }); check('one store makes one entry', cache.size, 1); for (let size = 2000; size <= 9000; size += 1000) { - cache.set(trajectoryCacheKey(entry(size, size), identity, null, null), familyAll, { body: `v${size}` }); + cache.set(trajectoryCacheKey(entry(size, size), identity, null, null), familyAll, SID, { body: `v${size}` }); } assert('nine versions of one question leave one entry', cache.size === 1); assert('the survivor is the newest, not the oldest', @@ -123,11 +125,11 @@ const keyA1 = trajectoryCacheKey(entry(1000, 5000), identity, null, null); const keyA2 = trajectoryCacheKey(entry(2000, 6000), identity, null, null); const keySlice = trajectoryCacheKey(entry(1000, 5000), identity, null, 3); const keyOther = trajectoryCacheKey({ ...entry(1000, 5000), dir: '/store/session-b' }, identity, null, null); -mixed.set(keyA1, familyAll, { body: 'a1' }); -mixed.set(keyA2, familyAll, { body: 'a2' }); +mixed.set(keyA1, familyAll, SID, { body: 'a1' }); +mixed.set(keyA2, familyAll, SID, { body: 'a2' }); check('growth in one family did not disturb the other family', mixed.size, 1); -mixed.set(keySlice, familySlice, { body: 's3' }); -mixed.set(keyOther, trajectoryCacheFamily('/store/session-b', null), { body: 'b1' }); +mixed.set(keySlice, familySlice, SID, { body: 's3' }); +mixed.set(keyOther, trajectoryCacheFamily('/store/session-b', null), SID, { body: 'b1' }); check('two sessions and two slices coexist', mixed.size, 3); assert('the unfiltered session kept its newest body', mixed.get(keyA2)?.body === 'a2'); assert('the slice kept its own body', mixed.get(keySlice)?.body === 's3'); @@ -143,7 +145,7 @@ for (let i = 0; i < LIMIT + 3; i += 1) { const dir = `/store/session-${i}`; const fam = trajectoryCacheFamily(dir, null); families.push(fam); - bounded.set(trajectoryCacheKey({ dir, sizeBytes: 100, lastActiveAt: 1 }, identity, null, null), fam, { body: `b${i}` }); + bounded.set(trajectoryCacheKey({ dir, sizeBytes: 100, lastActiveAt: 1 }, identity, null, null), fam, SID, { body: `b${i}` }); } assert('more sessions than the bound still respect it', bounded.size === LIMIT); assert('the oldest session was the one evicted', bounded.get(trajectoryCacheKey( @@ -160,11 +162,11 @@ const hot = createTrajectoryCache(2); const keyOf = (name) => trajectoryCacheKey( { dir: `/store/${name}`, sizeBytes: 1, lastActiveAt: 1 }, identity, null, null, ); -hot.set(keyOf('wanted'), trajectoryCacheFamily('/store/wanted', null), { body: 'wanted' }); -hot.set(keyOf('other'), trajectoryCacheFamily('/store/other', null), { body: 'other' }); +hot.set(keyOf('wanted'), trajectoryCacheFamily('/store/wanted', null), SID, { body: 'wanted' }); +hot.set(keyOf('other'), trajectoryCacheFamily('/store/other', null), SID, { body: 'other' }); // Touched, so recency order becomes [other, wanted] rather than [wanted, other]. hot.get(keyOf('wanted')); -hot.set(keyOf('newcomer'), trajectoryCacheFamily('/store/newcomer', null), { body: 'newcomer' }); +hot.set(keyOf('newcomer'), trajectoryCacheFamily('/store/newcomer', null), SID, { body: 'newcomer' }); assert('a family that was read again survives the next eviction', hot.get(keyOf('wanted'))?.body === 'wanted'); assert('the family that was NOT touched is the one evicted', hot.get(keyOf('other')) === null); @@ -176,15 +178,15 @@ const hotFamily = createTrajectoryCache(2); const familyKeyOf = (name) => trajectoryCacheKey( { dir: `/store/${name}`, sizeBytes: 1, lastActiveAt: 1 }, identity, null, null, ); -hotFamily.set(familyKeyOf('wanted'), trajectoryCacheFamily('/store/wanted', null), { body: 'wanted' }); -hotFamily.set(familyKeyOf('other'), trajectoryCacheFamily('/store/other', null), { body: 'other' }); +hotFamily.set(familyKeyOf('wanted'), trajectoryCacheFamily('/store/wanted', null), SID, { body: 'wanted' }); +hotFamily.set(familyKeyOf('other'), trajectoryCacheFamily('/store/other', null), SID, { body: 'other' }); // Touched by family, not by key: the reader asked the family and never computed this key. -hotFamily.getByFamily(trajectoryCacheFamily('/store/wanted', null)); -hotFamily.set(familyKeyOf('newcomer'), trajectoryCacheFamily('/store/newcomer', null), { body: 'newcomer' }); +hotFamily.getByFamily(trajectoryCacheFamily('/store/wanted', null), SID); +hotFamily.set(familyKeyOf('newcomer'), trajectoryCacheFamily('/store/newcomer', null), SID, { body: 'newcomer' }); assert('a family that was read by family survives the next eviction', - hotFamily.getByFamily(trajectoryCacheFamily('/store/wanted', null))?.body === 'wanted'); + hotFamily.getByFamily(trajectoryCacheFamily('/store/wanted', null), SID)?.body === 'wanted'); assert('and the family that was not read is the one evicted', - hotFamily.getByFamily(trajectoryCacheFamily('/store/other', null)) === null); + hotFamily.getByFamily(trajectoryCacheFamily('/store/other', null), SID) === null); /* ===================================================================== * 5 — structure: the cache is per-runtime, and the route names the family @@ -232,7 +234,7 @@ assert('the cache can be asked for the family it answers for', assert('the request resolves the family it will look up', startBody.includes('family: trajectoryCacheFamily(entry.dir, filter)')); assert('both readers fall back to the family when the exact key misses', - (startBody.match(/trajectoryCache\.get\(ctx\.cacheKey\) \?\? trajectoryCache\.getByFamily\(ctx\.family\)/g) || []).length === 2); + (startBody.match(/trajectoryCache\.get\(ctx\.cacheKey\) \?\? trajectoryCache\.getByFamily\(ctx\.family, ctx\.identityId\)/g) || []).length === 2); // The family is what the two keys share, so it has to be looked up by the family and not by // the key: an entry stored under one key is found under another whose family is the same. @@ -241,14 +243,50 @@ const boundKey = trajectoryCacheKey(entry(1000, 5000), identity, { sessionId: 's const unboundKey = trajectoryCacheKey(entry(1000, 5000), identity, null, null); assert('the same body under two bindings is two keys', boundKey !== unboundKey); -shared.set(boundKey, familyAll, { body: 'shared' }); +shared.set(boundKey, familyAll, SID, { body: 'shared' }); assert('and the exact key still finds it', shared.get(boundKey)?.body === 'shared'); assert('the other key does not', shared.get(unboundKey) === null); assert('but the family finds it, which is what the readers do', - shared.getByFamily(familyAll)?.body === 'shared'); + shared.getByFamily(familyAll, SID)?.body === 'shared'); assert('and a different family does not', - shared.getByFamily(familySlice) === null); + shared.getByFamily(familySlice, SID) === null); + +/* ===================================================================== + * 5b — the family names the QUESTION; the session names the ANSWER + * ===================================================================== */ + +// A family holds exactly one entry, and that entry is about one session. A directory can stop +// holding one session and start holding another: the manifest beside the transcript is the +// authority on that, and it can change under a cached walk. An entry built for the old session +// is therefore not an answer about the new one, however well the directory and the slice match +// — and its decoded messages are the OLD session's messages. +// +// The family cannot carry the session, because one family is one question and the question is +// asked by many readers of many sessions: the poll that asks for `latest` and the read that +// asks for one session by its id are the same reader of the same body. So the entry remembers +// which session it is about, and the lookup names the session it wants. + +const identityAware = createTrajectoryCache(LIMIT); +identityAware.set('key-for-a', familyAll, 'sess-a', { body: 'a' }); +assert('an entry is found for the session it is about', + identityAware.getByFamily(familyAll, 'sess-a')?.body === 'a'); +assert('and it is NOT found for another session', + identityAware.getByFamily(familyAll, 'sess-b') === null); +assert('the exact key still finds it, because the key names the session too', + identityAware.get('key-for-a')?.body === 'a'); +assert('and a family that was never asked for is still empty', + identityAware.getByFamily(familySlice, 'sess-a') === null); + +// The other direction, because a check that rejects everything is not a check: the SAME session +// has to be reachable by the family, which is what the readers rely on. +const sameSession = createTrajectoryCache(LIMIT); +sameSession.set('key-one', familyAll, 'sess-a', { body: 'one' }); +sameSession.set('key-two', familyAll, 'sess-a', { body: 'two' }); +assert('a newer version of the same family replaces its predecessor', + sameSession.size === 1); +assert('and the survivor is the newer one', + sameSession.getByFamily(familyAll, 'sess-a')?.body === 'two'); /* ===================================================================== * 6 — over HTTP: the body of the session on screen is reachable by its id @@ -300,6 +338,26 @@ try { const alpha = await makeConversation(REACH_ALPHA, 'ALPHA BODY', 1791600000000); const bravo = await makeConversation(REACH_BRAVO, 'BRAVO BODY', 1791500000000); + // A directory whose name does not decode names no session, so the manifest beside it is the + // only identity it has — and that identity can change under a cached walk. This is the shape + // the family fallback has to be careful about: the directory and the slice still match, and + // the session underneath is a different one. + const LEGACY_A = 'mvs_legacy_alpha'; + const LEGACY_B = 'mvs_legacy_bravo'; + const legacyDir = join(sessionsRoot, '2026', '10', '22', '00-00-00-session_legacy'); + const writeLegacy = async (id, body, mtimeMs) => { + await mkdir(legacyDir, { recursive: true }); + await writeFile(join(legacyDir, 'messages.jsonl'), JSON.stringify({ + message_id: 'm1', + turn_id: 't1', + message: { role: 'user', content: body, timestamp: 1791600000000 }, + }) + '\n', 'utf8'); + await writeFile(join(legacyDir, 'manifest.json'), JSON.stringify({ sessionId: id }), 'utf8'); + const stamp = new Date(mtimeMs); + await utimes(join(legacyDir, 'messages.jsonl'), stamp, stamp); + }; + await writeLegacy(LEGACY_A, 'ALPHA LEGACY', 1791600000000); + // The runtime database, which is the only place on this machine that knows which // conversation is live. ALPHA is live first; BRAVO becomes live later in this section. mkdirSync(join(home, 'v2', 'sqlite'), { recursive: true }); @@ -398,6 +456,52 @@ try { const detailRetry = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA); assert('and so does the record read', detailRetry.status === 200, why(detailRetry)); + + // 4 — the session under a directory can change, and the fallback has to notice. + // + // The family names the directory and the slice. The session is a different thing: a + // directory can stop holding one session and start holding another, and the manifest + // beside the transcript is the authority on that. The family lookup then finds the OLD + // session's entry, and that entry's decoded messages are the old session's messages — so + // a read for the new session answers with the old one's words. + const legacyLoad = await get('/api/trajectory?session=' + LEGACY_A); + assert('the conversation whose name does not decode loads by the id its manifest gives', + legacyLoad.status === 200 && Boolean(legacyLoad.body && legacyLoad.body.session) + && legacyLoad.body.session.id === LEGACY_A, why(legacyLoad)); + + const legacyRaw = await get('/api/trajectory/raw?index=1&session=' + LEGACY_A); + assert('and its record reads back', + legacyRaw.status === 200, why(legacyRaw)); + assert('with its own message', + JSON.stringify(legacyRaw.body && legacyRaw.body.raw ? legacyRaw.body.raw : '') + .includes('ALPHA LEGACY'), why(legacyRaw)); + + // The directory keeps its name and its place; the manifest and the transcript change. + await writeLegacy(LEGACY_B, 'BRAVO LEGACY', 1791700000000); + + const staleRaw = await get('/api/trajectory/raw?index=1&session=' + LEGACY_B); + assert('the read for the new session does not answer with the old session\'s message', + !(staleRaw.status === 200 + && JSON.stringify(staleRaw.body && staleRaw.body.raw ? staleRaw.body.raw : '') + .includes('ALPHA LEGACY')), why(staleRaw)); + assert('and it says the body is not held, rather than serving another session\'s', + staleRaw.status === 503, why(staleRaw)); + + const legacyDetail = await get('/api/trajectory/record?index=1&session=' + LEGACY_B); + assert('and the record read refuses it the same way', + legacyDetail.status === 503, why(legacyDetail)); + + // After a reload the new session is held, and then it reads back. + const legacyReload = await get('/api/trajectory?session=' + LEGACY_B); + assert('the new session loads after the swap', + legacyReload.status === 200 && Boolean(legacyReload.body && legacyReload.body.session) + && legacyReload.body.session.id === LEGACY_B, why(legacyReload)); + const legacyRawAfter = await get('/api/trajectory/raw?index=1&session=' + LEGACY_B); + assert('and then its own record reads back', + legacyRawAfter.status === 200, why(legacyRawAfter)); + assert('with the new message, not the old one', + JSON.stringify(legacyRawAfter.body && legacyRawAfter.body.raw ? legacyRawAfter.body.raw : '') + .includes('BRAVO LEGACY'), why(legacyRawAfter)); } finally { // The server is released here and not at the end of the happy path: an assertion that // throws would otherwise leave it listening, and the guard would hang instead of @@ -412,6 +516,21 @@ try { + (error && error.message)); } +/* ===================================================================== + * 7 — shape: the route records and resolves the session, too + * ===================================================================== */ + +// These two are read out of the source rather than run, and they are HERE rather than in +// section 5 on purpose: the behavioural section above is the owner of the promise they check, +// so a mutation that breaks the route must be caught by the reader's observable behaviour and +// only then by the shape. A shape check that fires first would report a mutation as caught +// while the behaviour it is about was never asked. + +assert('the request resolves the session the family lookup is about, because a family is not a session', + startBody.includes("identityId: identity.id ?? ''")); +assert('and the route records that session on the entry it stores', + (startBody.match(/trajectoryCache\.set\(cacheKey, trajectoryCacheFamily\(entry\.dir, filter\), identity\.id \?\? ''/g) || []).length === 1); + if (failures.length) { console.error(`\n[trajectory-cache] ${failures.length} FAILED, ${pass} passed`); for (const f of failures) console.error(' - ' + f); From beba2b56f94fa9eee441e11c07835a58ac58e552 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 10:16:50 +0800 Subject: [PATCH 29/95] A position read says which record of the body it means The session and the slice answer "which body". They do not answer "which version of the record at this index", and a transcript rewritten under a running page leaves the indexes alone while it changes the records behind them. The search answer is rebuilt from the current disk while the page filters the body it is showing, so a hit could mark a record as matching when it is a different record; and a secondary read named a position without naming the record behind it, so it could serve another record's content under the number the reader had open. Every record of a body now carries a version: the digest of the line it came from, together with its position inside that line. It is stable when the session grows -- the change that happens on every poll -- and it changes when the line is rewritten. A version over the body as a whole would have made every record on screen stale on every poll, which turns the most common change into a failure; a version per record does not. The search answer's hits carry the same version for the same record, and the page compares a hit's version with the record it is about to filter before installing the hit set. A hit whose record is not on screen is dropped and is not counted, so the number the reader sees and the rows that are highlighted agree. The two secondary reads carry the version in the request. The service answers one only when the record it would serve is the record that was asked for, and refuses otherwise: a read that names no version is refused, and a read whose version does not match the record now at that index is refused rather than answered with it. Old bodies are not kept -- the service declines to serve a record that moved, and the page says so. The route now publishes the slice it is for instead of how many generations it happens to show, and the body and the search answer name the same slice. That field was published to the reader while no assertion asked what it said. One comment in the page claimed that an answer's identity was checked before its indexes filtered the body. It checked the session and the slice and never the version. It now says what it does. --- .../mmc-trajectory/miniapp/client/index.html | 152 +++++++++++++- .../miniapp/node/search-segments.mjs | 19 +- .../mmc-trajectory/miniapp/node/server.mjs | 188 +++++++++++++++--- 3 files changed, 322 insertions(+), 37 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index d1870a2e..2a7c5875 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -3772,6 +3772,13 @@

检视

// Why the last search could not be answered, or null. 「没搜到」 and 「没搜成」 are // different states and the count line has to be able to tell them apart. searchError: null, + // How many times this needle has been re-asked against a freshly fetched body because the + // answer turned out to be about a different version of the records. Bounded, because a + // session that is being written to keeps moving — the counter is what stops the page from + // chasing it forever, and it is reset by the next answer that does apply. + searchRefresh: 0, + // The needle `searchRefresh` belongs to. A new needle is a new question and starts over. + searchRefreshFor: '', // Time range, in milliseconds since the session started — the same zero the ledger's // `+893 分 21 秒` column counts from. `null` on either side means that end is open, // and both null is the default: no range, whole session. @@ -4077,6 +4084,50 @@

检视

return session && typeof session.generation === 'number' ? session.generation : null; } + /** + * The records of the body on screen, by their index. + * + * A record index is a position inside ONE body, so anything that names an index has to be + * able to say which record of that body it means. This is the same set the ledger draws — + * the flattened body, not a second copy of it. + * + * @returns {Map} + */ + function displayedRecordsByIndex() { + var map = new Map(); + var flat = Array.isArray(state.flat) ? state.flat : []; + for (var i = 0; i < flat.length; i++) { + var entry = flat[i]; + var record = entry && entry.record ? entry.record : entry; + if (record && typeof record.index === 'number') map.set(record.index, record); + } + return map; + } + + /** + * The version of one record of the body on screen. + * + * The version is what makes index N in this body the same record as index N in another one. + * A transcript rewritten under a running page leaves the indexes alone and changes the + * records behind them, so the version is the only thing that tells the two apart — and a + * request that names an index has to carry it, or the service cannot tell which of the two + * records it is being asked about. + * + * @param {number} index + * @returns {string} the version, or '' when this body carries none + */ + function displayedRecordVersion(index) { + var flat = Array.isArray(state.flat) ? state.flat : []; + for (var i = 0; i < flat.length; i++) { + var entry = flat[i]; + var record = entry && entry.record ? entry.record : entry; + if (record && record.index === index) { + return typeof record.v === 'string' ? record.v : ''; + } + } + return ''; + } + /** * The session list, on its own validator. * @@ -4314,9 +4365,17 @@

检视

state.searchPending = false; state.searchTotal = 0; state.searchTruncated = false; + state.searchRefresh = 0; + state.searchRefreshFor = ''; searchSeq += 1; return; } + // A new needle is a new question, and the counter that bounds the re-asking belongs to + // the question that started it. + if (state.searchRefreshFor !== asked) { + state.searchRefreshFor = asked; + state.searchRefresh = 0; + } var controller = new AbortController(); searchController = controller; var seq = (searchSeq += 1); @@ -4352,14 +4411,19 @@

检视

searchController = null; state.searchPending = false; // The answer has to be about the body on screen BEFORE its indexes may filter that - // body. The server names the session and the slice it answered for, so the two can - // be compared: if they differ, the hits describe another body's records, and - // installing them would report a miss for a row that matches and a hit for one that - // does not. + // body. Three things have to agree, and each of them answers a different question: // - // The slice is compared against the body's own, not against the reader's selection: - // a selection whose load failed leaves the previous body on screen, and an answer - // about the selected slice would then filter rows that are not the ones on screen. + // - WHICH SESSION. The server names the session it answered for, and a body of + // another session's records would report a miss for a row that matches. + // - WHICH SLICE. The slice is compared against the body's own, not against the + // reader's selection: a selection whose load failed leaves the previous body on + // screen, and an answer about the selected slice would then filter rows that are + // not the ones on screen. + // - WHICH VERSION. The session and the slice say WHICH BODY; they say nothing about + // whether the record at a given index is still the same record. A transcript + // rewritten under a running page leaves the indexes alone and changes the records + // behind them, and an answer about the rewritten body would then light up a row + // whose text does not contain the needle at all. var answered = body && body.session ? body.session : null; var displayed = state.payload && state.payload.session ? state.payload.session : null; // The two slices are compared as the page reads them: a number names one generation and @@ -4376,9 +4440,68 @@

检视

mismatch.code = 'session_mismatch'; throw mismatch; } - state.searchHits = searchHitSet(body.hits); - state.searchTotal = Number.isInteger(body.total) ? body.total : 0; + // Every hit is checked against the body on screen, one by one, because a hit is a + // POSITION and a position only names a record when the version agrees. + // + // A hit whose position is on screen and whose version differs is about another version + // of that record: the whole answer is refused rather than partly installed, because + // the answer is about one body and the page is showing another. The body is then + // fetched once more and the same question is put again against the current body. + // + // A hit whose position is NOT on screen is about a record this body does not have — the + // session grew after the body was built. It cannot be shown, so it is not counted + // either: the number the reader sees has to be the number of rows that are lit up. + var byIndex = displayedRecordsByIndex(); + var hits = Array.isArray(body.hits) ? body.hits : []; + var applicable = []; + var wrongVersion = false; + for (var h = 0; h < hits.length; h += 1) { + var hit = hits[h]; + if (!hit || typeof hit.index !== 'number') continue; + var shown = byIndex.get(hit.index); + if (!shown) continue; + if (typeof shown.v === 'string' && shown.v !== '' && shown.v !== hit.v) { + wrongVersion = true; + break; + } + applicable.push(hit); + } + if (wrongVersion) { + // Bounded, because a session that is being written to keeps moving: after a couple + // of tries the page stops chasing it, says that the answer does not apply, and the + // next poll's own re-run takes over. + if (state.searchRefresh < 2) { + state.searchRefresh += 1; + state.searchError = null; + state.searchPending = true; + // The whole body, not a tail: a delta carries the records after the cursor, and + // the record that moved is not one of them. + state.since = 0; + state.sinceFor = ''; + state.etag = ''; + state.etagFor = ''; + fetchTrajectory(false).then(function () { + runSearch(state.query); + }, function () { + state.searchPending = false; + state.searchError = 'search_version_mismatch'; + renderLedger(); + }); + return; + } + state.searchError = 'search_version_mismatch'; + renderLedger(); + return; + } + state.searchHits = searchHitSet(applicable); state.searchTruncated = body.truncated === true; + // The count is the number of matches IN THE BODY ON SCREEN, so it is the number of rows + // the reader can see lit up. A truncated answer is the one exception: there the server's + // total is the real count and the rows on screen are only the first N of them, which + // the 「已截断」 mark says. + state.searchTotal = state.searchTruncated + ? (Number.isInteger(body.total) ? body.total : 0) + : applicable.length; // An answer that arrived retires the failure that preceded it. This line is here // because the two other places that clear the field are the two a READER triggers — // typing in the box and pressing clear — so nothing else could. A poll that re-runs the @@ -4387,6 +4510,7 @@

检视

// session: the label reads the error first, so the stale one outranked the fresh hit // set it was describing. state.searchError = null; + state.searchRefresh = 0; resetWindowMode(); renderLedger(); }) @@ -8719,6 +8843,12 @@

检视

if (generation !== null) { url += '&generation=' + encodeURIComponent(String(generation)); } + // WHICH RECORD. The index is a position inside the body on screen, and the service may be + // holding a newer body in which that position is another record — so the version of the + // record the reader opened travels with the request, and the service can tell the two + // apart instead of serving the newer record under the older one's number. + var version = displayedRecordVersion(index); + if (version) url += '&v=' + encodeURIComponent(version); return fetch(url, { headers: { Accept: 'application/json' } }).then(function (response) { return response.json().then( function (body) { return { ok: response.ok, body: body }; }, @@ -8811,6 +8941,10 @@

检视

if (generation !== null) { url += '&generation=' + encodeURIComponent(String(generation)); } + // The same record, the same way: the version travels with the request so the service can + // refuse when the position it would answer is a different record than the one opened. + var version = displayedRecordVersion(index); + if (version) url += '&v=' + encodeURIComponent(version); return fetch(url, { headers: { Accept: 'application/json' } }).then(function (response) { return response.json().then( function (body) { return { ok: response.ok, body: body }; }, diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs index ff1ef67c..41c9b661 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs @@ -43,8 +43,14 @@ import { createHash } from 'node:crypto'; import { mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises'; import { join } from 'node:path'; -/** Bumped when the on-disk shape changes in a way an older reader cannot survive. */ -export const SEGMENT_SCHEMA_VERSION = 3; +/** + * Bumped when the on-disk shape changes in a way an older reader cannot survive. + * + * 4: every line carries the record's version (`v`). A hit is a POSITION, and a position is only the + * same record in another body if the version says so — so a segment written before this would hand + * the page hits it cannot check, and the page would have to take them on faith. + */ +export const SEGMENT_SCHEMA_VERSION = 4; /** * The fields a search reads, in the page's order. @@ -206,6 +212,10 @@ export function buildSegment(input) { kind: typeof record.kind === 'string' ? record.kind : '', generation: typeof record.generation === 'number' ? record.generation : null, summary: typeof record.summary === 'string' ? record.summary : '', + // The version of the record this line is. It travels with the hit because a hit is a position + // and the page has to be able to tell whether that position still names the same record in the + // body it is showing. + v: typeof record.v === 'string' ? record.v : '', fileIdx: typeof record.fileIdx === 'number' ? record.fileIdx : -1, }) + '\t' + searchSurfaceOf(record)); } @@ -256,11 +266,11 @@ export function scanSegment(surface, needle) { * * @param {string} surface the whole segment text * @param {number[]} ordinals as returned by {@link scanSegment} - * @returns {Array<{ index: number, turnIndex: number | null, kind: string, summary: string, generation: number | null } | null>} + * @returns {Array<{ index: number, turnIndex: number | null, kind: string, summary: string, generation: number | null, v: string } | null>} */ export function readSegmentHits(surface, ordinals) { const wanted = new Set(ordinals); - /** @type {Map} */ + /** @type {Map} */ const found = new Map(); if (wanted.size === 0) return []; let start = 0; @@ -286,6 +296,7 @@ export function readSegmentHits(surface, ordinals) { kind: typeof parsed.kind === 'string' ? parsed.kind : '', summary: typeof parsed.summary === 'string' ? parsed.summary : '', generation: typeof parsed.generation === 'number' ? parsed.generation : null, + v: typeof parsed.v === 'string' ? parsed.v : '', }); } } diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index e118bc95..668f9d4c 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -346,6 +346,63 @@ export function lineDigest(bytes) { return createHash('sha256').update(bytes).digest('hex').slice(0, LINE_DIGEST_CHARS); } +/** + * The version of ONE record: what makes index N in one body the same record as index N in another. + * + * A record index is a POSITION, and a position means nothing without the body it was counted in. + * Two bodies of the same session at the same slice can hold different records at the same index — + * a transcript rewritten under a running page is the ordinary way that happens — and a reader who + * is shown 「命中 1 条」 pointing at a row whose text does not contain the needle has been told + * something false. + * + * So a version belongs to a position read the same way the session and the slice do: a request + * that names an index names the record it means, and the service can tell the two outcomes apart + * instead of serving the newer body under the older body's number. + * + * It is built from the LINE's digest and the record's position inside that line, and from nothing + * else. That matters twice over: + * + * - The line digest is already computed for every locator, so this costs no extra reading. + * - It is stable under APPEND. A session that grows does not rewrite the lines it already has, so + * the records a reader is looking at keep their version. Otherwise every poll would expire every + * row on screen, and the page would spend its life refusing its own answers — the common case + * would be the broken one. + * + * @param {string | null | undefined} digest the line's digest + * @param {number | null | undefined} rowPos the record's position inside that line + * @returns {string} + */ +export function recordVersionToken(digest, rowPos) { + const line = typeof digest === 'string' ? digest : ''; + const pos = Number.isInteger(rowPos) ? rowPos : 0; + return createHash('sha256').update(`${line}:${pos}`).digest('hex').slice(0, 16); +} + +/** + * Stamps every record of a built payload with its version. + * + * One place, because there are two callers that must agree: the body the page reads, and the + * search answer that filters it. A body whose records carry one version and an answer whose hits + * carry another is two answers to 「这条记录是哪一版」, and the page is then the only place that + * could notice — which is exactly where the check must not live. + * + * @param {{ turns?: Array }} payload + * @param {Map} locators record index -> its line's locator + * @returns {typeof payload} + */ +export function stampRecordVersions(payload, locators) { + const turns = payload && Array.isArray(payload.turns) ? payload.turns : []; + for (const turn of turns) { + const records = turn && Array.isArray(turn.records) ? turn.records : []; + for (const record of records) { + if (!record || typeof record !== 'object') continue; + const locator = locators && typeof locators.get === 'function' ? locators.get(record.index) : undefined; + record.v = recordVersionToken(locator ? locator.digest : null, record.rowPos); + } + } + return payload; +} + /** * Cheap single-pass counters used by the session list; values are approximate by design. * @@ -1863,10 +1920,10 @@ export function slimPayloadForWire(payload) { * @param {{ turns: Array }} payload * @param {string} needle already normalised and folded * @param {number} limit - * @returns {{ hits: Array<{ index: number, turnIndex: number, kind: string, summary: string, generation: number | null }>, total: number }} + * @returns {{ hits: Array<{ index: number, turnIndex: number, kind: string, summary: string, generation: number | null, v: string }>, total: number }} */ export function collectSearchHits(payload, needle, limit) { - /** @type {Array<{ index: number, turnIndex: number, kind: string, summary: string, generation: number | null }>} */ + /** @type {Array<{ index: number, turnIndex: number, kind: string, summary: string, generation: number | null, v: string }>} */ const hits = []; let total = 0; const turns = payload && Array.isArray(payload.turns) ? payload.turns : []; @@ -1884,6 +1941,10 @@ export function collectSearchHits(payload, needle, limit) { // without opening every row the needle found. summary: typeof record.summary === 'string' ? record.summary : '', generation: typeof record.generation === 'number' ? record.generation : null, + // The version of the record this hit names. The index alone is a position, and the position + // is only the same record in another body if the version says so — the page checks it + // against the body it is showing before it lets the hit filter anything. + v: typeof record.v === 'string' ? record.v : '', }); } } @@ -4317,6 +4378,11 @@ export async function start(context) { const locator = rowLocators[rowIndex]; if (locator !== undefined) raws.set(recordIndex, locator); } + // The version of every record, stamped before the body is serialized. A record index is a + // position inside ONE body, and the same position in a newer body can be a different record: + // the page asks about positions all the time (the search answer, `/raw`, `/record`), so the + // position carries the version of the record it means. See `recordVersionToken`. + stampRecordVersions(payload, raws); // Last thing before serializing, and the only place the bulk fields are ever taken off a // record. It happens here rather than at `push()` because the search, which runs in this @@ -4402,20 +4468,31 @@ export async function start(context) { const sessionKey = searchSessionKey(entry.dir, ctx.identity.id); const viewKey = searchViewKey(filter); + // ONE answer, sent by whichever path found the hits. + // + // Two paths answering the same request with two different response shapes is not a faster + // version of the endpoint, it is a second endpoint — and the second one is where the slice + // the answer is about would have gone stale. The slice travels with every answer, in the + // same word the body uses, so the page can compare the two. + // + // @param {Array} found the hits + // @param {number} foundTotal every match, which is not every hit + const sendAnswer = (found, foundTotal) => { + sendJson(response, 200, { + query: asked, + hits: found, + total: foundTotal, + truncated: foundTotal > found.length, + session: { id: ctx.identity.id, generation: filter }, + }); + }; + // Before anything expensive. The lineage walk above is stat-driven and cheap; everything after // this line is the rebuild, and on the largest session measured here it was 3.7 seconds for a // search that returns at most 500 rows. const onDisk = await searchFromArtifacts({ sessionKey, viewKey, files, needle }); if (onDisk !== null) { - sendJson(response, 200, { - query: asked, - hits: onDisk.hits, - total: onDisk.total, - truncated: onDisk.total > onDisk.hits.length, - // The same field the rebuild sends. Two paths answering the same request with two different - // response shapes is not a faster version of the endpoint, it is a second endpoint. - session: { id: ctx.identity.id, generation: filter }, - }); + sendAnswer(onDisk.hits, onDisk.total); return; } @@ -4435,8 +4512,18 @@ export async function start(context) { }); const rowLocators = view.occurrences.map((occurrence) => occurrence.locator); const rowIndexByRecord = splitRawRecords(payload.turns); + // The record -> its line's locator, so every record can be stamped with its version before it + // is serialized or written into a segment. The search answer and the body the page filters with + // have to carry the SAME version for the same record, or the page is comparing two answers. + /** @type {Map} */ + const locatorsByRecord = new Map(); + for (const [recordIndex, rowIndex] of rowIndexByRecord) { + const locator = rowLocators[rowIndex]; + if (locator !== undefined) locatorsByRecord.set(recordIndex, locator); + } + stampRecordVersions(payload, locatorsByRecord); - /** @type {Array<{ index: number, turnIndex: number, kind: string, summary: string, generation: number | null }>} */ + /** @type {Array<{ index: number, turnIndex: number, kind: string, summary: string, generation: number | null, v: string }>} */ const hits = []; let total = 0; const searched = collectSearchHits(payload, needle, MAX_SEARCH_HITS); @@ -4456,13 +4543,7 @@ export async function start(context) { activeFileName: walked.activeFileName, }).catch(() => undefined); - sendJson(response, 200, { - query: asked, - hits, - total, - truncated: total > hits.length, - session: { id: ctx.identity.id, generation: filter }, - }); + sendAnswer(hits, total); // Nothing below reads the view, and the payload holds projections rather than rows, so this is // where the object graph stops being reachable. view.rows.length = 0; @@ -4625,6 +4706,43 @@ export async function start(context) { * that moved under the cache answers 「该记录所在的文件已变化」 here too, instead of projecting a * line that is no longer the one the page is looking at. */ + /** + * The refusal a position read gets when it does not name the record it means. + * + * A position read names an INDEX, and an index is a position inside ONE body. The body the page + * is showing and the body the service would serve can be two different bodies — a transcript + * rewritten under a running page is the ordinary way that happens — and then the same index + * names two different records. The page says which one it means by sending the record's + * version; the service compares it with the version of the record it would serve, so the two + * outcomes are told apart instead of the newer body being served under the older body's number. + * + * TWO REFUSALS, not one: a read that does not say is a read the service cannot judge, and a + * read that says the wrong thing is a read about a record that is not there. They are different + * answers, so they are asked separately — and the page's retry can tell them apart. + * + * @param {URL} url + * @returns {{ status: number, code: string, message: string } | null} the refusal, or null + */ + const missingRecordVersion = (url) => { + const asked = url.searchParams.get('v'); + if (typeof asked === 'string' && asked !== '') return null; + return { status: 400, code: 'record_version_required', message: '缺少记录版本，请重新加载会话' }; + }; + + /** + * The refusal a position read gets when it names a record the service would not serve. + * + * @param {URL} url + * @param {{ digest?: string } | undefined} locator the record's line locator + * @param {number} rowPos the record's position inside that line + * @returns {{ status: number, code: string, message: string } | null} the refusal, or null + */ + const mismatchedRecordVersion = (url, locator, rowPos) => { + const asked = url.searchParams.get('v'); + if (asked === recordVersionToken(locator ? locator.digest : null, rowPos)) return null; + return { status: 409, code: 'record_version_mismatch', message: '该记录所在的版本已变化，请重新加载会话' }; + }; + const handleRecord = async (response, url) => { const ctx = await resolveTrajectoryRequest(url); if (!ctx.ok) { @@ -4654,6 +4772,14 @@ export async function start(context) { sendError(response, 404, 'record_not_found', '找不到该记录'); return; } + // Which record of which version the page is asking about. The entry in the cache may be a + // newer body than the one the page is showing, and then the index names another record. + const refused = missingRecordVersion(url) ?? mismatchedRecordVersion(url, locator, rowPos); + if (refused) { + context.logger.warn('miniapp.trajectory.read_error', { reason: refused.code }); + sendError(response, refused.status, refused.code, refused.message); + return; + } const found = await readRecordRow(locator); if ('error' in found) { context.logger.warn('miniapp.trajectory.read_error', { reason: `record_${found.reason}` }); @@ -4683,6 +4809,10 @@ export async function start(context) { } sendJson(response, 200, { index: wanted, + // No version here. The version's job was to make the REQUEST unambiguous, and the service + // above either served the record that was asked for or refused: an answer that repeated the + // version would be a field the page never reads, and a field nobody reads is a promise with + // no owner. The body carries the version of every record, and the page sends it back. // No `kind`. Measured over 6568 records of this machine's store: re-projecting one row // returns the same four values every time, but it disagrees about `kind` for an opener — // a `steer` row projected alone has no earlier turn to have been interrupted by, so it @@ -4750,6 +4880,21 @@ export async function start(context) { sendError(response, 503, 'trajectory_not_cached', '会话数据已释放，请重新加载'); return; } + const locator = cached.raws.get(wanted); + const rowPos = cached.rowPos.get(wanted); + if (locator === undefined || typeof locator.filePath !== 'string' || typeof rowPos !== 'number') { + sendError(response, 404, 'record_not_found', '找不到该记录'); + return; + } + // Which record of which version the page is asking about. The entry in the cache may be a + // newer body than the one the page is showing, and then the index names another record — and + // the decoded message kept below would be that other record's message. + const refused = missingRecordVersion(url) ?? mismatchedRecordVersion(url, locator, rowPos); + if (refused) { + context.logger.warn('miniapp.trajectory.read_error', { reason: refused.code }); + sendError(response, refused.status, refused.code, refused.message); + return; + } // Re-reading a record the reader has already opened is the common case, and a decoded message // is far smaller than a session, so the entry keeps the ones it has already paid for. const alreadyDecoded = cached.decoded.get(wanted); @@ -4757,11 +4902,6 @@ export async function start(context) { sendJson(response, 200, { index: wanted, raw: alreadyDecoded.value, omitted: alreadyDecoded.omitted }); return; } - const locator = cached.raws.get(wanted); - if (locator === undefined || typeof locator.filePath !== 'string') { - sendError(response, 404, 'record_not_found', '找不到该记录'); - return; - } const found = await readRecordRow(locator); if ('error' in found) { context.logger.warn('miniapp.trajectory.read_error', { reason: `raw_${found.reason}` }); From 80cae5225e597eb9468f786321fd1dc0a76a3746 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 10:17:05 +0800 Subject: [PATCH 30/95] The guards that own a record's version and the route's slice The cache guard now asks which version of a record it is serving. Every record of a body carries one, the search answer's hits carry the same one for the same record, and a secondary read that names an older version is refused instead of answering with the record that now sits at that index. A read that names no version at all is refused too, because the service cannot tell "the record you meant" from "whatever is at that index now". The body names the slice it is for, and the search answer names the same slice as the body it answers about. The client harness now asks whether a search answer for a newer body can filter the displayed older body, and what the page does when it cannot: the answer is not installed at all, the body is fetched again and the question asked again, at most twice, and after that the search reports a failure rather than an answer. A hit whose record is not on screen is dropped, is not counted, and does not start a refetch. The guard's own store moved to a temporary directory. A search writes its segment cache under the data directory, and the package directory installs onto every reader's machine, so a guard that left a cache in there was a guard that shipped a cache. The mutation controls grew to match what the guards now own: seven in the cache control -- the body stops stamping its records, either secondary read stops checking the version, a read that names no version is answered anyway, the hits stop carrying the version, the body stops naming its slice, and the search answer names a generation when no slice was asked for -- and three in the client control: the search stops comparing the version of the record a hit names, and the two reads stop naming the version of the record they ask for. --- .../mmc-trajectory/tests/client-check.mjs | 100 +++++++- .../tests/mutate-search-client.py | 30 ++- .../tests/mutate-trajectory-cache.py | 84 +++++++ .../tests/verify-search-endpoint.mjs | 10 +- .../tests/verify-trajectory-cache.mjs | 215 ++++++++++++++++-- 5 files changed, 403 insertions(+), 36 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index 86fd5bb9..e322754f 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -228,6 +228,7 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); vm.createContext(netCtx); const needed = ['function isCount(value)', 'function formatInt(value)', 'function searchHitSet(hits)', 'function sessionParam()', 'function displayedSessionParam()', 'function displayedGeneration()', + 'function displayedRecordsByIndex()', 'function displayedRecordVersion(index)', 'function searchStatusLabel(searchState)', 'function runSearch(needle)']; const sources = needed.map((sig) => fnSource(sig)); check('the shipped search functions are all still findable by signature', @@ -238,10 +239,16 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); // page's state is, because nothing else about the page's state is what is under test. 'var state = { query: "retry me", sessionId: null, hostSessionId: null, followLatest: true,', ' generation: null, searchHits: null, searchTotal: 0, searchTruncated: false,', - ' searchPending: false, searchError: null,', + ' searchPending: false, searchError: null, searchRefresh: 0, searchRefreshFor: "",', // The body on screen. `displayedSessionParam()` reads it and the answer is checked against // it, so a harness with no body on screen cannot ask either question. Its `generation` is // the slice the body IS: null is the whole lineage, a number is that one generation. + // + // `flat` is the same body's records, which is what a hit is checked against: a hit names a + // POSITION, and the position only names a record if the version on both sides agrees. + ' flat: [ { turn: {}, record: { index: 1, v: "version-one" } },', + ' { turn: {}, record: { index: 5, v: "version-five" } },', + ' { turn: {}, record: { index: 7, v: "version-seven" } } ],', ' payload: { session: { id: "mvs_displayed", generation: null } } };', 'var searchSeq = 0;', 'var searchController = null;', @@ -254,10 +261,19 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); + ' is not in index.html any more); }')), ].join('\n'), netCtx); - const settle = async () => { + const settle = async (turns = 3) => { // fetch -> response.json() -> then -> then -> then is all microtasks plus the json() promise. // Three macrotask turns drain all of them, so the count does not have to be right. - for (let i = 0; i < 3; i += 1) await new Promise((done) => setImmediate(done)); + for (let i = 0; i < turns; i += 1) await new Promise((done) => setImmediate(done)); + }; + + // The page re-asks against the body it fetches when an answer turns out to be about another + // version of the records. The fetch itself is not what is under test here, so it is a stub that + // counts the calls: how many times the page was willing to chase the answer IS the behaviour. + netCtx.refreshes = 0; + netCtx.fetchTrajectory = function () { + netCtx.refreshes += 1; + return Promise.resolve(); }; // Every URL the search asked for, so the question it asked can be read off the requests @@ -274,7 +290,7 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); netCtx.fetch = (url) => { asked.push(String(url)); return Promise.resolve({ ok: true, status: 200, - json: () => Promise.resolve({ hits: [{ index: 7 }], total: 1, truncated: false, + json: () => Promise.resolve({ hits: [{ index: 7, v: 'version-seven' }], total: 1, truncated: false, // The server names the session it answered for. That name is what the page checks the // answer against before it lets the hits filter the body on screen. session: { id: 'mvs_displayed', generation: null } }) }); @@ -371,7 +387,7 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); netCtx.fetch = (url) => { asked.push(String(url)); return Promise.resolve({ ok: true, status: 200, - json: () => Promise.resolve({ hits: [{ index: 1 }], total: 1, truncated: false, + json: () => Promise.resolve({ hits: [{ index: 1, v: 'version-one' }], total: 1, truncated: false, session: { id: 'mvs_displayed', generation: 0 } }) }); }; netCtx.runSearch('retry me'); @@ -411,7 +427,7 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); netCtx.fetch = (url) => { asked.push(String(url)); return Promise.resolve({ ok: true, status: 200, - json: () => Promise.resolve({ hits: [{ index: 5 }], total: 1, truncated: false, + json: () => Promise.resolve({ hits: [{ index: 5, v: 'version-five' }], total: 1, truncated: false, session: { id: 'mvs_displayed_b', generation: null } }) }); }; netCtx.runSearch('retry me'); @@ -422,6 +438,64 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); netCtx.state.searchError, null); check('and the request asked about the conversation on screen at that moment', /[?&]session=mvs_displayed_b(&|$)/.test(asked[0]), true); + + /* The version. A hit is a POSITION, and the same position in two bodies can be two different + * records: a transcript rewritten under the page leaves the indexes alone and changes the + * records behind them. The session and the slice say WHICH BODY the answer is about; they say + * nothing about whether the record at a given index is still the same record. + * + * So an answer about the rewritten body would light up a row whose text does not contain the + * needle at all — the page would be saying 「命中 1 条」 about a row that does not match. The + * version on the hit and the version on the body have to agree before a hit may filter + * anything. */ + const beforeVersion = [...netCtx.state.searchHits]; + netCtx.state.payload = { session: { id: 'mvs_displayed', generation: null } }; + netCtx.state.generation = null; + netCtx.state.searchRefresh = 0; + netCtx.state.searchRefreshFor = ''; + netCtx.refreshes = 0; + asked.length = 0; + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ hits: [{ index: 7, v: 'a-newer-version' }], total: 1, truncated: false, + session: { id: 'mvs_displayed', generation: null } }) }); + }; + netCtx.runSearch('retry me'); + await settle(24); + check('a search answer for a newer body cannot filter the displayed older body', + [...netCtx.state.searchHits], beforeVersion); + check('and the page asks again against the body it fetches, a bounded number of times', + netCtx.refreshes, 2); + check('and then it says the answer does not apply to what is on screen', + netCtx.state.searchError, 'search_version_mismatch'); + check('and the label shows the failure rather than a count it cannot stand behind', + netCtx.searchStatusLabel(netCtx.state), '搜索失败，以下为上次结果'); + + // A hit about a record this body does not have — the session grew after the body was built. + // It cannot be shown, so it is not counted either: the number the reader sees has to be the + // number of rows that are lit up. + netCtx.state.searchRefresh = 0; + netCtx.state.searchRefreshFor = ''; + netCtx.refreshes = 0; + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ + hits: [{ index: 7, v: 'version-seven' }, { index: 4242, v: 'a-record-not-on-screen' }], + total: 2, truncated: false, + session: { id: 'mvs_displayed', generation: null } }) }); + }; + netCtx.runSearch('retry me'); + await settle(); + check('a hit about a record that is not on screen is not installed', + [...netCtx.state.searchHits], [7]); + check('and the count is the number of rows that are lit up', + netCtx.state.searchTotal, 1); + check('and no failure is reported, because that answer did apply', + netCtx.state.searchError, null); + check('and the page did not go chasing a body it did not need to', + netCtx.refreshes, 0); } /* handoff: the session the record came from, across a navigation. @@ -1954,12 +2028,14 @@ const requestRawSrc = fnSource('function requestRaw(index)'); const requestDetailSrc = fnSource('function requestDetail(index)'); const displayedSessionSrc = fnSource('function displayedSessionParam()'); const displayedGenerationSrc = fnSource('function displayedGeneration()'); +const displayedRecordVersionSrc = fnSource('function displayedRecordVersion(index)'); const rawCacheKeySrc = fnSource('function rawCacheKey(index)'); if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGenerationSrc - || !rawCacheKeySrc) { + || !displayedRecordVersionSrc || !rawCacheKeySrc) { failures.push('secondary reads: could not lift requestRaw, requestDetail and the session they ask for'); } else { vm.runInContext(displayedSessionSrc + '\n' + displayedGenerationSrc + '\n' + + displayedRecordVersionSrc + '\n' + requestRawSrc + '\n' + requestDetailSrc + '\n' + rawCacheKeySrc, sandbox); const secondary = []; @@ -1973,8 +2049,13 @@ if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGen }; // The page is following the newest conversation, and it is showing A. The runtime may already // have made B current — the page does not know that, and it must not care: A is on screen. + // + // `flat` is the body on screen's records. A read that names an index has to name the RECORD + // behind that index, and the record's version is what says which one it is: the same position + // in a newer body can be a different record entirely. sandbox.state = { payload: { session: { id: 'mvs_displayed', generation: null } }, + flat: [{ turn: {}, record: { index: 1, v: 'the-version-of-record-one' } }], generation: null, etag: '"tag"', followLatest: true, sessionId: '', hostSessionId: '' }; await sandbox.requestRaw(1); @@ -1987,12 +2068,17 @@ if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGen /[?&]session=mvs_displayed(&|$)/.test(secondary[1]), true); check('and it does not ask for whatever is current now', /[?&]session=latest(&|$)/.test(secondary[1]), false); + check('a raw read names the version of the record it is asking for', + /[?&]v=the-version-of-record-one(&|$)/.test(secondary[0]), true); + check('and a detail read names it too', + /[?&]v=the-version-of-record-one(&|$)/.test(secondary[1]), true); // The slice on screen is not the generation the reader picked. A selection whose load failed // leaves the previous body up, and a read that asks about the selection asks about a body // that is not there — its answer is about records the reader cannot see. sandbox.state = { payload: { session: { id: 'mvs_displayed', generation: 2 } }, + flat: [{ turn: {}, record: { index: 1, v: 'the-version-of-record-one' } }], generation: 7, etag: '"tag"', followLatest: true, sessionId: '', hostSessionId: '' }; await sandbox.requestRaw(1); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py index dbe521bf..409eb6ad 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py @@ -96,8 +96,8 @@ # it was describing. The page showed the right rows under the wrong sentence. ( "an answer that arrives leaves the failure that preceded it standing", - " state.searchError = null;\n resetWindowMode();", - " resetWindowMode();", + " state.searchError = null;\n state.searchRefresh = 0;\n resetWindowMode();", + " state.searchRefresh = 0;\n resetWindowMode();", ), # ---- the search is about the body on screen --------------------------------------- # @@ -167,6 +167,32 @@ " var gen = displayedGeneration() === null ? 'all' : String(displayedGeneration());", " var gen = state.generation === null ? 'all' : String(state.generation);", ), + # ---- the version of the record a position names ------------------------------------ + # + # The session and the slice say WHICH BODY an answer is about. They say nothing about + # whether the record at a given index is still the same record: a transcript rewritten under + # the page leaves the indexes alone and changes the records behind them. So the version is + # checked before a hit may filter anything, and it travels with every read that names a + # position. + ( + "the search stops checking which version of the record a hit names", + " if (typeof shown.v === 'string' && shown.v !== '' && shown.v !== hit.v) {", + " if (false) {", + ), + ( + "the raw read stops naming the version of the record it asks for", + " // apart instead of serving the newer record under the older one's number.\n" + " var version = displayedRecordVersion(index);\n" + " if (version) url += '&v=' + encodeURIComponent(version);", + " // apart instead of serving the newer record under the older one's number.", + ), + ( + "the record read stops naming the version of the record it asks for", + " // refuse when the position it would answer is a different record than the one opened.\n" + " var version = displayedRecordVersion(index);\n" + " if (version) url += '&v=' + encodeURIComponent(version);", + " // refuse when the position it would answer is a different record than the one opened.", + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py index f49c08d4..f9aa9a32 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py @@ -207,6 +207,90 @@ " identityId: identity.id ?? '',\n", " identityId: '',\n", ), + # ---- the version of a record, which is what a position read has to name ------------ + # + # A record index is a POSITION, and the same position in two bodies can be two different + # records: a transcript rewritten under a running page leaves the indexes alone and changes + # the records behind them. The session and the slice say WHICH BODY an answer is about; the + # version says WHICH RECORD an index names. Without it, a read for the old body is answered + # with the new body's record under the old body's number. + ( + "the body stops stamping its records with their version", + SERVER, + " stampRecordVersions(payload, raws);\n", + "", + ), + ( + "the raw read stops checking which version of the record it is serving", + SERVER, + " const refused = missingRecordVersion(url) ?? mismatchedRecordVersion(url, locator, rowPos);\n" + " if (refused) {\n" + " context.logger.warn('miniapp.trajectory.read_error', { reason: refused.code });\n" + " sendError(response, refused.status, refused.code, refused.message);\n" + " return;\n" + " }\n" + " // Re-reading a record the reader has already opened is the common case", + " const refused = missingRecordVersion(url);\n" + " if (refused) {\n" + " context.logger.warn('miniapp.trajectory.read_error', { reason: refused.code });\n" + " sendError(response, refused.status, refused.code, refused.message);\n" + " return;\n" + " }\n" + " // Re-reading a record the reader has already opened is the common case", + ), + ( + "the record read stops checking which version of the record it is serving", + SERVER, + " const refused = missingRecordVersion(url) ?? mismatchedRecordVersion(url, locator, rowPos);\n" + " if (refused) {\n" + " context.logger.warn('miniapp.trajectory.read_error', { reason: refused.code });\n" + " sendError(response, refused.status, refused.code, refused.message);\n" + " return;\n" + " }\n" + " const found = await readRecordRow(locator);\n" + " if ('error' in found) {\n" + " context.logger.warn('miniapp.trajectory.read_error', { reason: `record_${found.reason}` });", + " const refused = missingRecordVersion(url);\n" + " if (refused) {\n" + " context.logger.warn('miniapp.trajectory.read_error', { reason: refused.code });\n" + " sendError(response, refused.status, refused.code, refused.message);\n" + " return;\n" + " }\n" + " const found = await readRecordRow(locator);\n" + " if ('error' in found) {\n" + " context.logger.warn('miniapp.trajectory.read_error', { reason: `record_${found.reason}` });", + ), + ( + "a read that does not name the record it means is answered anyway", + SERVER, + " if (typeof asked === 'string' && asked !== '') return null;\n" + " return { status: 400, code: 'record_version_required', message: '缺少记录版本，请重新加载会话' };", + " if (true) return null;\n" + " return { status: 400, code: 'record_version_required', message: '缺少记录版本，请重新加载会话' };", + ), + ( + "the search stops carrying the version of the record each hit names", + SERVER, + " v: typeof record.v === 'string' ? record.v : '',\n", + " v: '',\n", + ), + # ---- the slice the body IS, which the page compares every answer against ----------- + # + # The service publishes which slice a body is, and the page compares the slice of every + # answer it gets against the slice of the body on screen. If the two words mean different + # things, the comparison is between two different questions and a mismatch means nothing. + ( + "the body stops naming the slice it is for, and names the generations it happens to show", + SERVER, + " generation: filter,\n", + " generation: shown.length === 1 ? shown[0].generation : null,\n", + ), + ( + "the search answer names a generation when no slice was asked for", + SERVER, + " session: { id: ctx.identity.id, generation: filter },\n", + " session: { id: ctx.identity.id, generation: filter === null ? 0 : filter },\n", + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs index 26018461..11323fac 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs @@ -541,8 +541,14 @@ assert('and the artifact root holds segments and nothing else', artifactDirs.length === 1 && artifactDirs[0] === 'search'); // Two paths answering one request is only a performance change if they answer it identically, // and the response body is part of the answer the page reads. -check('both paths send the same session field', - (SERVER.match(/session: \{ id: ctx\.identity\.id, generation: filter \}/g) || []).length, 2); +// +// The two paths do not answer separately any more: they build the answer in ONE place and send +// it. So the question is not "do they agree" — it is "is there only one place for them to +// disagree in". A second copy of the answer is a second answer, and the copy nobody thinks +// about is the one that eventually stops answering. +const searchAnswerShape = (SERVER.match(/session: \{ id: ctx\.identity\.id, generation: filter \}/g) || []).length; +check('the search answer is built in one place, so the two paths cannot answer differently', + searchAnswerShape, 1); // Scoped to handleSearch's own text: the same two lines also appear in the trajectory route, and // a whole-file indexOf would compare a search against an unrelated request's body. const handleSearchSrc = SERVER.slice( diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs index d44d3eaa..cd5904e8 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs @@ -358,6 +358,26 @@ try { }; await writeLegacy(LEGACY_A, 'ALPHA LEGACY', 1791600000000); + // A conversation whose transcript is rewritten under a page that is still showing the old one. + // The session identity does not move: the same directory, the same manifest, the same id — only + // the line behind record 1 is a different line. That is what makes this the hard case: the + // index still says 1, and the position is still a position, but it names a different record. + const VERSIONED = 'mvs_versioned'; + const versionedDir = join(sessionsRoot, '2026', '10', '22', '00-00-00-session_' + + Buffer.from(VERSIONED, 'utf8').toString('base64url')); + const writeVersioned = async (body, mtimeMs) => { + await mkdir(versionedDir, { recursive: true }); + await writeFile(join(versionedDir, 'messages.jsonl'), JSON.stringify({ + message_id: 'm1', + turn_id: 't1', + message: { role: 'user', content: body, timestamp: 1791600000000 }, + }) + '\n', 'utf8'); + await writeFile(join(versionedDir, 'manifest.json'), JSON.stringify({ sessionId: VERSIONED }), 'utf8'); + const stamp = new Date(mtimeMs); + await utimes(join(versionedDir, 'messages.jsonl'), stamp, stamp); + }; + await writeVersioned('ALPHA BODY', 1791600000000); + // The runtime database, which is the only place on this machine that knows which // conversation is live. ALPHA is live first; BRAVO becomes live later in this section. mkdirSync(join(home, 'v2', 'sqlite'), { recursive: true }); @@ -379,25 +399,40 @@ try { const noop = () => {}; let instance = null; let base = null; - for (let attempt = 0; attempt < 12 && !base; attempt += 1) { - const port = 19900 + Math.floor(Math.random() * 300) + attempt; - try { - instance = await start({ - pluginId: 'mmc-trajectory', - pluginRoot: PLUGIN_ROOT, - dataDir: PLUGIN_ROOT, - generationId: 'guard', - listen: { host: '127.0.0.1', port }, - signal: controller.signal, - logger: { debug: noop, info: noop, warn: noop, error: noop }, - }); - base = 'http://127.0.0.1:' + port; - } catch (error) { - // The port was taken. Another one is tried; a guard that gave up here would be a - // guard that sometimes checks nothing. + // Starting the service is one thing that happens twice in this section, so it is one function. + // The second start is not a repeat of the first: it is how the service stops believing the walk + // it cached (the walk holds the session list for three seconds, so a rewritten transcript is + // invisible to it until that expires). Restarting drops that cache in one step instead of + // waiting it out. + const startService = async () => { + for (let attempt = 0; attempt < 12; attempt += 1) { + const port = 19900 + Math.floor(Math.random() * 300) + attempt; + try { + const started = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + // The temporary store, not the plugin directory. A search writes its segment cache + // under the data dir, and the plugin directory installs onto every reader's machine — + // a guard that leaves a cache in there is a guard that ships a cache. + dataDir: home, + generationId: 'guard', + listen: { host: '127.0.0.1', port }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop }, + }); + return { instance: started, base: 'http://127.0.0.1:' + port }; + } catch (error) { + // The port was taken. Another one is tried; a guard that gave up here would be a + // guard that sometimes checks nothing. + } } + throw new Error('could not bind a port for the reachability route'); + }; + { + const started = await startService(); + instance = started.instance; + base = started.base; } - if (!base) throw new Error('could not bind a port for the reachability route'); const get = async (path) => { const response = await fetch(base + path); @@ -407,6 +442,21 @@ try { }; const why = (answer) => 'status ' + answer.status + ' ' + JSON.stringify(answer.body); + // The version of one record of a body. A position read names an INDEX, and an index is a + // position inside ONE body — so a read has to say which record of that body it means, and this + // is what it says. The page takes it from the body it is showing, which is the only body it + // has. + const versionOf = (body, index) => { + const turns = body && Array.isArray(body.turns) ? body.turns : []; + for (const turn of turns) { + const records = Array.isArray(turn && turn.records) ? turn.records : []; + for (const record of records) { + if (record && record.index === index) return typeof record.v === 'string' ? record.v : ''; + } + } + return ''; + }; + try { // 1 — the poll the page makes, and the read it makes right after it. const poll = await get('/api/trajectory?session=latest'); @@ -414,17 +464,53 @@ try { Boolean(poll.body && poll.body.session) && poll.body.session.id === REACH_ALPHA, why(poll)); - const rawById = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA); + // The slice the body IS. The service publishes it, and the page compares every answer it + // gets against it, so it has to be the slice the body was BUILT FOR — the one that was asked + // for, and the whole lineage when none was. The old wording answered a different question + // (「how many generations does this body happen to show」) and gave a single-generation + // session's default view the number 0 — so the same body published two different slices + // depending on which of its generations happened to be in it, and an answer about the whole + // lineage could not be compared with the body it was supposed to filter. + assert('the body names the slice it is for: the whole lineage when no slice was asked for', + Boolean(poll.body && poll.body.session) && poll.body.session.generation === null, why(poll)); + + // Every record of the body carries its version, and the search answer's hits carry the SAME + // one for the same record. Two answers to 「which record is at this index」 that disagree are + // what let a newer body's hits filter an older body's rows. + const alphaVersion = versionOf(poll.body, 1); + assert('every record of the body carries a version', + alphaVersion !== '', 'version of record 1: ' + JSON.stringify(alphaVersion)); + const alphaSearch = await get('/api/trajectory/search?session=' + REACH_ALPHA + '&q=ALPHA'); + assert('the search answers with hits', + alphaSearch.status === 200 && Array.isArray(alphaSearch.body && alphaSearch.body.hits) + && alphaSearch.body.hits.length > 0, why(alphaSearch)); + const alphaHit = alphaSearch.body.hits.find((hit) => hit.index === 1); + assert('and a hit names the same version of the record it points at', + Boolean(alphaHit) && alphaHit.v === alphaVersion, + 'hit: ' + JSON.stringify(alphaHit) + ' body: ' + alphaVersion); + assert('and the search answer names the same slice as the body it is about', + Boolean(alphaSearch.body && alphaSearch.body.session) + && alphaSearch.body.session.generation === poll.body.session.generation, + 'search: ' + JSON.stringify(alphaSearch.body && alphaSearch.body.session) + + ' body: ' + JSON.stringify(poll.body && poll.body.session)); + + const rawById = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA + '&v=' + alphaVersion); assert('a record of the conversation on screen is readable by that conversation\'s id', rawById.status === 200, why(rawById)); assert('and it is that conversation\'s own message', JSON.stringify(rawById.body && rawById.body.raw ? rawById.body.raw : '').includes('ALPHA BODY'), why(rawById)); - const detailById = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA); + const detailById = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA + '&v=' + alphaVersion); assert('the four fields of a record are readable by that conversation\'s id', detailById.status === 200, why(detailById)); + // A read that does not say which record it means is not answered: the service cannot tell + // 「the record you meant」 from 「whatever is at that index now」, and guessing is the defect. + // (These two sit after the version scenario on purpose: the two refusals are two rules, and a + // control that takes the whole check away breaks both of them — so the one that is about the + // VERSION has to be asked first, or the control would be attributed to the wrong rule.) + // 2 — the runtime moves on while the page is still showing the first conversation. const moved = new DatabaseSync(dbPath); moved.exec('update local_runtime_sessions set updated_at_ms = 300 where session_id = ' @@ -435,14 +521,14 @@ try { Boolean(poll2.body && poll2.body.session) && poll2.body.session.id === REACH_BRAVO, why(poll2)); - const rawAfter = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA); + const rawAfter = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA + '&v=' + alphaVersion); assert('the conversation still on screen is still readable by its id', rawAfter.status === 200, why(rawAfter)); assert('and it is still its own message, not the other conversation\'s', JSON.stringify(rawAfter.body && rawAfter.body.raw ? rawAfter.body.raw : '').includes('ALPHA BODY'), why(rawAfter)); - const detailAfter = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA); + const detailAfter = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA + '&v=' + alphaVersion); assert('and its records\' fields are still readable by its id', detailAfter.status === 200, why(detailAfter)); @@ -450,10 +536,10 @@ try { // so the read after a reload is the read a reader would make. const reload = await get('/api/trajectory?session=latest'); assert('the reload answers', reload.status === 200, why(reload)); - const rawRetry = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA); + const rawRetry = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA + '&v=' + alphaVersion); assert('the read after a reload still finds the conversation on screen', rawRetry.status === 200, why(rawRetry)); - const detailRetry = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA); + const detailRetry = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA + '&v=' + alphaVersion); assert('and so does the record read', detailRetry.status === 200, why(detailRetry)); @@ -469,7 +555,8 @@ try { legacyLoad.status === 200 && Boolean(legacyLoad.body && legacyLoad.body.session) && legacyLoad.body.session.id === LEGACY_A, why(legacyLoad)); - const legacyRaw = await get('/api/trajectory/raw?index=1&session=' + LEGACY_A); + const legacyVersion = versionOf(legacyLoad.body, 1); + const legacyRaw = await get('/api/trajectory/raw?index=1&session=' + LEGACY_A + '&v=' + legacyVersion); assert('and its record reads back', legacyRaw.status === 200, why(legacyRaw)); assert('with its own message', @@ -496,12 +583,90 @@ try { assert('the new session loads after the swap', legacyReload.status === 200 && Boolean(legacyReload.body && legacyReload.body.session) && legacyReload.body.session.id === LEGACY_B, why(legacyReload)); - const legacyRawAfter = await get('/api/trajectory/raw?index=1&session=' + LEGACY_B); + const legacyRawAfter = await get('/api/trajectory/raw?index=1&session=' + LEGACY_B + '&v=' + versionOf(legacyReload.body, 1)); assert('and then its own record reads back', legacyRawAfter.status === 200, why(legacyRawAfter)); assert('with the new message, not the old one', JSON.stringify(legacyRawAfter.body && legacyRawAfter.body.raw ? legacyRawAfter.body.raw : '') .includes('BRAVO LEGACY'), why(legacyRawAfter)); + + // 5 — the transcript is rewritten under a page that is still showing the old one. + // + // The session does not change: the same directory, the same manifest, the same id. Only the + // line behind record 1 is a different line now. The index still says 1 and the position is + // still a position — but it names a DIFFERENT record, and the old one is what the reader is + // looking at. + // + // The service is holding the newer body by the time the read arrives (the page's next poll + // loaded it), so the line it would read is the new one and its own digest check passes. What + // stops it from answering is the version the request carries: the reader says which record it + // meant, and the service compares that with the record it would serve. + const versionLoad = await get('/api/trajectory?session=' + VERSIONED); + assert('the conversation whose transcript will be rewritten loads', + versionLoad.status === 200 && Boolean(versionLoad.body && versionLoad.body.session) + && versionLoad.body.session.id === VERSIONED, why(versionLoad)); + const oldVersion = versionOf(versionLoad.body, 1); + assert('and its record carries a version', + oldVersion !== '', 'version: ' + JSON.stringify(oldVersion)); + + await writeVersioned('BRAVO BODY', 1791800000000); + // The service now has to be holding the NEWER body, or the read below is not the read the + // reader would make. A fresh service instance is how it stops believing the walk it cached: + // the walk holds the session list for three seconds, so a rewritten transcript is invisible + // to it until that expires, and restarting drops that cache in one step. + await instance.dispose(); + { + const restarted = await startService(); + instance = restarted.instance; + base = restarted.base; + } + const versionReload = await get('/api/trajectory?session=' + VERSIONED); + assert('the rewritten conversation loads again', + versionReload.status === 200, why(versionReload)); + assert('and the body it serves is the rewritten one', + JSON.stringify(versionReload.body).includes('BRAVO BODY'), why(versionReload)); + const newVersion = versionOf(versionReload.body, 1); + assert('and the record behind the same index is a different version of itself', + newVersion !== '' && newVersion !== oldVersion, + 'old: ' + oldVersion + ' new: ' + newVersion); + + const staleVersionRaw = await get('/api/trajectory/raw?index=1&session=' + VERSIONED + '&v=' + oldVersion); + assert('a secondary read from an older body never serves a newer body\'s record', + !(staleVersionRaw.status === 200 + && JSON.stringify(staleVersionRaw.body && staleVersionRaw.body.raw ? staleVersionRaw.body.raw : '') + .includes('BRAVO BODY')), why(staleVersionRaw)); + assert('and it says the version moved, rather than answering with the other record', + staleVersionRaw.status === 409 && staleVersionRaw.body + && staleVersionRaw.body.error === 'record_version_mismatch', why(staleVersionRaw)); + + const staleVersionDetail = await get('/api/trajectory/record?index=1&session=' + VERSIONED + '&v=' + oldVersion); + assert('and the record read refuses it the same way', + staleVersionDetail.status === 409 && staleVersionDetail.body + && staleVersionDetail.body.error === 'record_version_mismatch', why(staleVersionDetail)); + + const currentVersionRaw = await get('/api/trajectory/raw?index=1&session=' + VERSIONED + '&v=' + newVersion); + assert('while the read that names the current version still answers', + currentVersionRaw.status === 200 + && JSON.stringify(currentVersionRaw.body && currentVersionRaw.body.raw ? currentVersionRaw.body.raw : '') + .includes('BRAVO BODY'), why(currentVersionRaw)); + + // And the other refusal: a read that does not say which record it means is not answered + // either. The service cannot tell 「the record you meant」 from 「whatever is at that index + // now」, and guessing is the defect. + // + // The body is loaded again first: a restarted service holds no body until one is asked for, + // and a read against an empty cache is refused for that reason rather than this one. + const alphaReload = await get('/api/trajectory?session=' + REACH_ALPHA); + assert('and the service still answers for the conversation on screen', + alphaReload.status === 200, why(alphaReload)); + const rawNoVersion = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA); + assert('a read that does not name the record it means is refused', + rawNoVersion.status === 400 && rawNoVersion.body && rawNoVersion.body.error === 'record_version_required', + why(rawNoVersion)); + const detailNoVersion = await get('/api/trajectory/record?index=1&session=' + REACH_ALPHA); + assert('and the record read refuses it the same way', + detailNoVersion.status === 400 && detailNoVersion.body && detailNoVersion.body.error === 'record_version_required', + why(detailNoVersion)); } finally { // The server is released here and not at the end of the happy path: an assertion that // throws would otherwise leave it listening, and the guard would hang instead of From 6eb841ba0ad587e4411c87e215acb4bf144d3733 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 11:08:23 +0800 Subject: [PATCH 31/95] The runtime refuses a data dir that would put its artifacts inside the package The search segments are derived state. They belong to the runtime's own data dir, and the runtime's data dir is not the plugin package: the package installs onto every reader's machine, so anything written inside it is published along with it. start() took context.dataDir on trust, so a caller that passed the package root had the segments land in the directory that ships. The refusal names the path it refused, because the caller is the one that has to change it. It also accepts everything outside the package, because a runtime that refuses to start is its own kind of outage, and the sibling of the package whose name merely starts with the package's name is outside it. The containment test is one shared function, isUnderDirectory, which the artifact path and the session path already needed and used to answer with a string prefix test each. A prefix test on Windows also matches a sibling whose name starts with the root's name, and it does not depend on the two paths having been spelled with the same letter case. The plugin ignores the directory its artifacts would land in, and the shipped harnesses hand the runtime a temporary data dir instead of the package. The anchor in tests/mutate-session-picker.py moves with the containment test it was pinning. --- plugins/avatasia/mmc-trajectory/.gitignore | 8 +- .../mmc-trajectory/miniapp/node/server.mjs | 120 ++++++++++++------ .../tests/mutate-session-picker.py | 2 +- .../tests/verify-runtime-diag.mjs | 11 +- .../tests/verify-session-layers.mjs | 19 ++- 5 files changed, 116 insertions(+), 44 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/.gitignore b/plugins/avatasia/mmc-trajectory/.gitignore index 1e95f573..1b285758 100644 --- a/plugins/avatasia/mmc-trajectory/.gitignore +++ b/plugins/avatasia/mmc-trajectory/.gitignore @@ -11,4 +11,10 @@ __pycache__/ # mid-mutation is caught on the next one instead of being inferred from whatever the source looks # like afterwards. The snapshot is machine-specific bookkeeping and this directory is the package # that ships to readers, so it must never be tracked. -tests/.shipped-snapshot.json \ No newline at end of file +tests/.shipped-snapshot.json + +# Derived artifacts. The runtime writes the search segments under its own data dir, which is +# outside this package; the runtime refuses a data dir that lands in here (see `start()` in +# miniapp/node/server.mjs). This entry is the second half of that rule: if a caller ever got +# through, the directory must not be tracked and must not be part of what ships. +trajectory-artifacts/ \ No newline at end of file diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 668f9d4c..49cec58b 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -3,7 +3,7 @@ import { open, readdir, readFile, stat } from 'node:fs/promises'; import { existsSync } from 'node:fs'; import { createServer } from 'node:http'; -import { basename, join, resolve, sep } from 'node:path'; +import { basename, isAbsolute, join, relative, resolve, sep } from 'node:path'; import { homedir } from 'node:os'; import { createHash } from 'node:crypto'; import { @@ -698,6 +698,31 @@ export function activeCatalogGeneration(catalog, generations) { return generations.length > 0 ? generations[generations.length - 1].generation : null; } +/** + * Whether `candidate` is `root` itself, or something under it. + * + * One definition, for every place that has to answer "is this path inside that one": the artifact + * file a catalog may name, the session directory a relative path may point at, and the data dir a + * caller may hand over. + * + * It compares the two paths rather than testing one string against the other, because a prefix test + * on the strings also matches a sibling whose name merely starts with the root's name (`D:\pkg-extra` + * against `D:\pkg`), and because it does not depend on the two paths having been spelled with the + * same letter case. + * + * @param {string} root + * @param {string} candidate + * @returns {boolean} + */ +export function isUnderDirectory(root, candidate) { + const base = resolve(root); + const target = resolve(candidate); + if (target === base) return true; + const inside = relative(base, target); + if (isAbsolute(inside)) return false; + return inside !== '..' && !inside.startsWith(`..${sep}`); +} + /** * Resolves a catalog artifact to a readable file inside the session directory. * @@ -715,7 +740,7 @@ export function resolveArtifactPath(dir, artifact) { if (!SAFE_ARTIFACT_NAME.test(artifact.fileName) || artifact.fileName.includes('..')) return null; const root = resolve(dir); const candidate = resolve(root, artifact.active ? artifact.fileName : join('snapshots', artifact.fileName)); - if (candidate !== root && !candidate.startsWith(root + sep)) return null; + if (!isUnderDirectory(root, candidate)) return null; return candidate; } @@ -1342,7 +1367,7 @@ export async function entryFromRelativeDir(sessionsRoot, relativeDir, expectedSe if (typeof relativeDir !== 'string' || !relativeDir.trim()) return { entry: null, rejected: false }; const root = resolve(sessionsRoot); const dir = resolve(root, relativeDir.trim().replace(/\\/g, '/')); - if (dir !== root && !dir.startsWith(root + sep)) return { entry: null, rejected: true }; + if (!isUnderDirectory(root, dir)) return { entry: null, rejected: true }; try { const fileStat = await stat(join(dir, 'messages.jsonl')); // The id of the session this directory holds. It travels with the entry, like it does for @@ -3575,10 +3600,33 @@ function sendError(response, status, code, message) { sendJson(response, status, { error: code, message }); } +/** + * The plugin package installs onto every reader's machine, so nothing derived may be written inside + * it. `context.dataDir` is the runtime's own data directory; a caller that points it at the + * package, or at anything under it, has the search segments land in a directory that ships to + * readers. + * + * The refusal names the path it refused, because the caller is the one that has to change it, and a + * refusal that does not say which path was refused is a refusal nobody can act on. + * + * @param {MiniAppContext} context + */ +function assertDataDirOutsidePlugin(context) { + const dataDir = typeof context.dataDir === 'string' ? context.dataDir : ''; + if (!dataDir) throw new Error('the runtime did not give this plugin a data dir to write into'); + const pluginRoot = typeof context.pluginRoot === 'string' ? context.pluginRoot : ''; + if (isUnderDirectory(pluginRoot, dataDir)) { + throw new Error( + `the data dir must not be inside the plugin package: ${resolve(dataDir)} is under ${resolve(pluginRoot)}` + ); + } +} + /** * @param {MiniAppContext} context */ export async function start(context) { + assertDataDirOutsidePlugin(context); const location = resolveSessionsRoot(); const secrets = [location.sessionsRoot, location.root, join(homedir(), '.minimax')]; const redact = createRedactor(secrets); @@ -4673,39 +4721,6 @@ export async function start(context) { void entry; }; - /** - * One record's untouched message, fetched only when something actually reads it. - * - * The body no longer has the parsed rows behind it, so the line is read back out of the file it - * came from. That is a seek and a digest over one line, against the alternative of holding every - * session's object graph in memory for the life of the page — measured at 233 MB of the 378 MB - * this process held while one session was open. - * - * The digest is what makes that safe to do. Between building the body and answering this request - * the file can be rewritten; the trajectory entry is keyed on the file's size and mtime, so a - * rewrite that preserved both would otherwise be served the wrong message under the right - * record's name. Answering `raw_stale` says the session moved instead. - * - * A missing entry means this process restarted between the two calls; saying so is more honest - * than silently showing the normalized record in a panel labelled 原文. - */ - /** - * One record's four bulk fields, read on request. - * - * The page used to receive all four on every row of every poll — measured over this machine's - * whole store, 59.1% of the body: `result` 26.6%, `thinking` 13.2%, `arguments` 9.3%, `text` - * 8.9% — and used them to draw a ledger row and, when the reader opened one record, four tabs. - * It now receives four presence flags and asks for the values when it is going to show them. - * - * It re-projects the ONE stored line that record came from, rather than rebuilding the session. - * A line can expand into several records (its text, each thinking block, each tool call), so the - * answer has to name which of them — `rowPos`, stamped at `push()` — and rebuilding the whole - * session to return one field would cost exactly what this change exists to stop paying. - * - * The bytes come through the same locator `/raw` uses and carry the same digest check, so a file - * that moved under the cache answers 「该记录所在的文件已变化」 here too, instead of projecting a - * line that is no longer the one the page is looking at. - */ /** * The refusal a position read gets when it does not name the record it means. * @@ -4743,6 +4758,23 @@ export async function start(context) { return { status: 409, code: 'record_version_mismatch', message: '该记录所在的版本已变化，请重新加载会话' }; }; + /** + * One record's four bulk fields, read on request. + * + * The page used to receive all four on every row of every poll — measured over this machine's + * whole store, 59.1% of the body: `result` 26.6%, `thinking` 13.2%, `arguments` 9.3%, `text` + * 8.9% — and used them to draw a ledger row and, when the reader opened one record, four tabs. + * It now receives four presence flags and asks for the values when it is going to show them. + * + * It re-projects the ONE stored line that record came from, rather than rebuilding the session. + * A line can expand into several records (its text, each thinking block, each tool call), so the + * answer has to name which of them — `rowPos`, stamped at `push()` — and rebuilding the whole + * session to return one field would cost exactly what this change exists to stop paying. + * + * The bytes come through the same locator `/raw` uses and carry the same digest check, so a file + * that moved under the cache answers 「该记录所在的文件已变化」 here too, instead of projecting a + * line that is no longer the one the page is looking at. + */ const handleRecord = async (response, url) => { const ctx = await resolveTrajectoryRequest(url); if (!ctx.ok) { @@ -4862,6 +4894,22 @@ export async function start(context) { return { row, sizeBytes: bytes.length }; }; + /** + * One record's untouched message, fetched only when something actually reads it. + * + * The body no longer has the parsed rows behind it, so the line is read back out of the file it + * came from. That is a seek and a digest over one line, against the alternative of holding every + * session's object graph in memory for the life of the page — measured at 233 MB of the 378 MB + * this process held while one session was open. + * + * The digest is what makes that safe to do. Between building the body and answering this request + * the file can be rewritten; the trajectory entry is keyed on the file's size and mtime, so a + * rewrite that preserved both would otherwise be served the wrong message under the right + * record's name. Answering `raw_stale` says the session moved instead. + * + * A missing entry means this process restarted between the two calls; saying so is more honest + * than silently showing the normalized record in a panel labelled 原文. + */ const handleRaw = async (response, url) => { const ctx = await resolveTrajectoryRequest(url); if (!ctx.ok) { diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py index 4723b7a7..b2d9e55a 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-session-picker.py @@ -164,7 +164,7 @@ # chose, so it gets the same check a catalog file name gets -- and a version that skips it # builds an entry from a directory outside the session store. ("a recorded path that escapes the sessions root is used anyway", - [(" if (dir !== root && !dir.startsWith(root + sep)) return { entry: null, rejected: true };", + [(" if (!isUnderDirectory(root, dir)) return { entry: null, rejected: true };", " if (false) return { entry: null, rejected: true };")], "a recorded path that escapes the sessions root is refused"), diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-runtime-diag.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-runtime-diag.mjs index bdc655e1..2de634ea 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-runtime-diag.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-runtime-diag.mjs @@ -64,6 +64,14 @@ function assert(name, cond, detail) { } const home = mkdtempSync(join(mkdtempSync(join(tmpdir(), 'mmc-trajectory-diag-')), '.minimax')); +/** + * Where the runtime may write what it derives. + * + * Not the package: the runtime refuses a data dir inside the package, because anything written + * there is part of what ships to every reader. A harness that handed it the package would put the + * search segments in the published directory, so the data dir here is a temporary one outside it. + */ +const dataHome = mkdtempSync(join(tmpdir(), 'mmc-trajectory-datadir-')); try { const where = resolveSessionsRoot({ MMC_TRAJECTORY_ROOT: home }); const redact = createRedactor([where.sessionsRoot, where.root, join(homedir(), '.minimax')]); @@ -223,7 +231,7 @@ try { instance = await start({ pluginId: 'mmc-trajectory', pluginRoot: PLUGIN_ROOT, - dataDir: PLUGIN_ROOT, + dataDir: dataHome, generationId: 'guard', listen: { host: '127.0.0.1', port }, signal: controller.signal, @@ -255,6 +263,7 @@ try { } } finally { rmSync(home, { recursive: true, force: true }); + rmSync(dataHome, { recursive: true, force: true }); } console.log('[runtime-diag] ' + pass + ' passed, ' + failures.length + ' failed'); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs index 6d78bd1f..3f5ca8d5 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-session-layers.mjs @@ -72,6 +72,15 @@ const SERVER = fileURLToPath(new URL('../miniapp/node/server.mjs', import.meta.u const CLIENT = fileURLToPath(new URL('../miniapp/client/index.html', import.meta.url)); /** The package root, so the route serves the same page the reader gets. */ const PLUGIN_ROOT = fileURLToPath(new URL('..', import.meta.url)); +/** + * Where the runtime is allowed to write what it derives. + * + * Not the package: the runtime refuses a data dir inside the package, and the reason it refuses is + * that anything written there ships to every reader. A harness that handed it the package would + * write the search segments into the directory that is published, so the data dir here is a + * temporary directory outside the package. + */ +const DATA_DIR = mkdtempSync(join(tmpdir(), 'mmc-trajectory-datadir-')); // `node:sqlite` is still flagged experimental, and the flag goes to stderr the moment the module // is first loaded. The runner reads `stdout + stderr` and prints the LAST non-empty line, so the @@ -195,7 +204,7 @@ async function startService(home) { instance = await start({ pluginId: 'mmc-trajectory', pluginRoot: PLUGIN_ROOT, - dataDir: PLUGIN_ROOT, + dataDir: DATA_DIR, generationId: 'guard', listen: { host: '127.0.0.1', port }, signal: controller.signal, @@ -454,7 +463,7 @@ try { instance = await start({ pluginId: 'mmc-trajectory', pluginRoot: PLUGIN_ROOT, - dataDir: PLUGIN_ROOT, + dataDir: DATA_DIR, generationId: 'guard', listen: { host: '127.0.0.1', port }, signal: controller.signal, @@ -584,7 +593,7 @@ try { instance = await start({ pluginId: 'mmc-trajectory', pluginRoot: PLUGIN_ROOT, - dataDir: PLUGIN_ROOT, + dataDir: DATA_DIR, generationId: 'guard', listen: { host: '127.0.0.1', port }, signal: controller.signal, @@ -683,7 +692,7 @@ try { instance = await start({ pluginId: 'mmc-trajectory', pluginRoot: PLUGIN_ROOT, - dataDir: PLUGIN_ROOT, + dataDir: DATA_DIR, generationId: 'guard', listen: { host: '127.0.0.1', port }, signal: controller.signal, @@ -833,7 +842,7 @@ try { instance = await start({ pluginId: 'mmc-trajectory', pluginRoot: PLUGIN_ROOT, - dataDir: PLUGIN_ROOT, + dataDir: DATA_DIR, generationId: 'guard', listen: { host: '127.0.0.1', port }, signal: controller.signal, From a786b6bcbac58f83b49457a6df9f997b96558109 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 11:08:38 +0800 Subject: [PATCH 32/95] The guard that owns where the derived artifacts may land MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit tests/verify-artifact-root.mjs holds the rule from three sides: a data dir that IS the package is refused, a data dir under it is refused too, and a data dir outside it — or a sibling whose name merely starts with the package's name — is accepted. It also holds the two owners around the refusal: the package ignores the directory its artifacts would land in, and no shipped harness hands the package over as its data dir. tests/mutate-artifact-root.py puts each of those back, five in all, and requires the guard to report it by name before the file is restored byte for byte. A refusal that is too broad is its own kind of broken, so one of the five turns the check into a prefix test and expects the sibling to be refused. --- .../tests/mutate-artifact-root.py | 166 ++++++++++++++++++ .../tests/verify-artifact-root.mjs | 154 ++++++++++++++++ 2 files changed, 320 insertions(+) create mode 100644 plugins/avatasia/mmc-trajectory/tests/mutate-artifact-root.py create mode 100644 plugins/avatasia/mmc-trajectory/tests/verify-artifact-root.mjs diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-artifact-root.py b/plugins/avatasia/mmc-trajectory/tests/mutate-artifact-root.py new file mode 100644 index 00000000..1208d542 --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-artifact-root.py @@ -0,0 +1,166 @@ +"""Mutation controls for verify-artifact-root.mjs. + +Each entry below puts one of the rule's owners back the way it was before the rule existed, and +requires the guard to report it BY NAME before the file is restored byte for byte. A mutation +reported as survived means the guard it exercised is not testing anything, and the fix is the +guard, never this list. + +Five owners are covered, and they are not the same kind of thing: + + - the runtime refusing a data dir inside the package at all; + - the refusal reaching a directory UNDER the package, not just the package itself; + - the refusal not being so broad that it turns into a refusal of a sibling whose name merely + starts with the package's name; + - the package ignoring the directory its artifacts would land in; + - the shipped harnesses not handing the package over as their data dir. + +Nothing here needs a network, a service, or a specific machine's paths. +""" +import subprocess +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parent +PROBE = HERE / "verify-artifact-root.mjs" + +SERVER = "miniapp/node/server.mjs" +GITIGNORE = ".gitignore" +LAYERS = "tests/verify-session-layers.mjs" + +MUTATIONS = [ + # ---- the runtime stops refusing ------------------------------------------------------ + ( + "the runtime stops refusing a data dir inside the package", + SERVER, + " assertDataDirOutsidePlugin(context);\n", + "", + "the runtime refuses a data dir that is the plugin package", + ), + # ---- the refusal has to reach a directory under it, not only the package itself ------- + ( + "the refusal only catches the package itself, not a directory under it", + SERVER, + " if (isUnderDirectory(pluginRoot, dataDir)) {", + " if (resolve(dataDir) === resolve(pluginRoot)) {", + "the runtime refuses a data dir under the plugin package", + ), + # ---- and it must not be so broad that it refuses everything -------------------------- + ( + "the refusal is a prefix test, so a sibling of the package is refused too", + SERVER, + " if (isUnderDirectory(pluginRoot, dataDir)) {", + " if (resolve(dataDir).startsWith(resolve(pluginRoot))) {", + "the runtime accepts a sibling of the plugin package whose name starts with its name", + ), + # ---- the package ignores where its artifacts would land ------------------------------ + ( + "the package ships without ignoring the directory its artifacts would land in", + GITIGNORE, + "\ntrajectory-artifacts/", + "", + "the package ignores the directory its artifacts would land in", + ), + # ---- and no shipped harness hands the package over ----------------------------------- + ( + "a shipped harness hands the package to the runtime as its data dir", + LAYERS, + " const port = 19400 + Math.floor(Math.random() * 400) + attempt;\n" + " try {\n" + " instance = await start({\n" + " pluginId: 'mmc-trajectory',\n" + " pluginRoot: PLUGIN_ROOT,\n" + " dataDir: DATA_DIR,", + " const port = 19400 + Math.floor(Math.random() * 400) + attempt;\n" + " try {\n" + " instance = await start({\n" + " pluginId: 'mmc-trajectory',\n" + " pluginRoot: PLUGIN_ROOT,\n" + " dataDir: PLUGIN_ROOT,", + "no shipped harness hands the package to the runtime as its data dir", + ), +] + + +def read_exact(path): + """Normalise to LF for matching, and report the file's own newline so it can be put back.""" + data = path.read_bytes() + text = data.decode("utf-8").replace("\r\n", "\n") + return text, ("\r\n" if b"\r\n" in data else "\n") + + +def write_exact(path, text, newline): + if newline == "\r\n": + text = text.replace("\n", "\r\n") + path.write_bytes(text.encode("utf-8")) + + +def run_probe(): + proc = subprocess.run( + ["node", str(PROBE)], + cwd=str(ROOT), + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + ) + return proc.returncode, (proc.stdout or "") + (proc.stderr or "") + + +def main(): + code, output = run_probe() + if code != 0: + print("baseline probe does not pass; fix that first") + print(output) + return 1 + print("baseline clean") + + originals = {rel: (ROOT / rel).read_bytes() for rel in (SERVER, GITIGNORE, LAYERS)} + + missed = [] + unnamed = [] + for label, rel, old, new, expected in MUTATIONS: + target = ROOT / rel + text, newline = read_exact(target) + if text.count(old) != 1: + print(f"SKIP {label}: the anchor resolves {text.count(old)} times, expected exactly 1") + missed.append(label) + continue + if old == new: + print(f"SKIP {label}: the mutation changes nothing") + missed.append(label) + continue + write_exact(target, text.replace(old, new, 1), newline) + try: + code, output = run_probe() + finally: + write_exact(target, text, newline) + if code == 0: + print(f"MISSED {label}: the probe still passed") + missed.append(label) + continue + if expected not in output: + print(f"UNNAMED {label}: the probe failed without naming 「{expected}」") + unnamed.append(label) + continue + first = next((line.strip() for line in output.splitlines() if line.strip().startswith("- ")), None) + print(f"caught {label}\n -> {first}") + + for rel, original in originals.items(): + if (ROOT / rel).read_bytes() != original: + print(f"NOT RESTORED {rel}") + return 1 + print("files restored byte-for-byte") + + if missed or unnamed: + if missed: + print(f"\n{len(missed)} mutation(s) got through") + if unnamed: + print(f"\n{len(unnamed)} mutation(s) failed without naming a rule") + return 1 + print(f"\nall {len(MUTATIONS)} mutations caught, each by name") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-artifact-root.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-artifact-root.mjs new file mode 100644 index 00000000..61913b9d --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/verify-artifact-root.mjs @@ -0,0 +1,154 @@ +/** + * The derived artifacts must land under the runtime's own data dir, and nowhere else. + * + * The plugin package installs onto every reader's machine. The search segments are derived state: + * they belong to the runtime's data dir, and the runtime's data dir is not the package. Nothing + * stopped a caller from handing `context.dataDir` the package itself, and when one did, the + * segments landed in a directory that was published to every reader. + * + * So the refusal is the rule, and it is checked from three sides here: + * + * 1. a data dir that IS the package is refused, and the refusal names the path it refused; + * 2. a data dir UNDER the package is refused too — the root is only the obvious case; + * 3. a data dir OUTSIDE the package is accepted, and so is a sibling whose name merely starts with + * the package's name. A refusal that is too broad is its own kind of broken: it would stop the + * runtime from starting at all. + * + * Two more owners carry the other half of the rule. The package ignores the directory its artifacts + * would land in, so that a caller who got through leaves nothing that is tracked. And no shipped + * harness hands the package to the runtime as its data dir, because that is how the segments got + * there in the first place. + * + * `tests/mutate-artifact-root.py` puts each of those back and requires this guard to report it by + * name. Nothing here needs a network or a particular machine's paths. + */ +import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join, resolve } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +const PLUGIN_ROOT = fileURLToPath(new URL('..', import.meta.url)); +/** The package root without its trailing separator, so a sibling can be spelled next to it. */ +const PKG = resolve(PLUGIN_ROOT); +const SERVER = fileURLToPath(new URL('../miniapp/node/server.mjs', import.meta.url)); +const GITIGNORE = readFileSync(new URL('../.gitignore', import.meta.url), 'utf8'); + +let pass = 0; +const failures = []; +function assert(name, cond, detail) { + if (cond) pass += 1; + else failures.push(detail ? `${name} -- ${detail}` : name); +} + +const REFUSAL = 'the data dir must not be inside the plugin package'; + +const noop = () => {}; +const controller = new AbortController(); +const previousRoot = process.env.MMC_TRAJECTORY_ROOT; + +/** + * Start the runtime against a data dir and report what happened. + * + * The refusal is told apart from any other failure by its message, because a port that was already + * taken is not an answer about the data dir and must not be read as one. + * + * @param {string} dataDir + * @returns {Promise<{ accepted: boolean, refused: boolean, message: string }>} + */ +async function startWith(dataDir) { + for (let attempt = 0; attempt < 12; attempt += 1) { + const port = 21500 + Math.floor(Math.random() * 400) + attempt; + try { + const instance = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + dataDir, + generationId: 'guard', + listen: { host: '127.0.0.1', port }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop } + }); + await instance.dispose(); + return { accepted: true, refused: false, message: '' }; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + if (message.includes(REFUSAL)) return { accepted: false, refused: true, message }; + if (/EADDRINUSE|address already in use/i.test(message)) continue; + return { accepted: false, refused: false, message }; + } + } + return { accepted: false, refused: false, message: 'could not bind a port for the runtime' }; +} + +/** Every `.mjs` in this directory is a harness that may start the runtime. */ +const harnesses = readdirSync(fileURLToPath(new URL('.', import.meta.url))) + .filter((name) => name.endsWith('.mjs')); + +const sessionsHome = mkdtempSync(join(tmpdir(), 'mmc-trajectory-artifact-home-')); +const outsideDir = mkdtempSync(join(tmpdir(), 'mmc-trajectory-artifact-out-')); +mkdirSync(join(sessionsHome, 'v2', 'sessions'), { recursive: true }); +process.env.MMC_TRAJECTORY_ROOT = sessionsHome; + +const { start } = await import(pathToFileURL(SERVER).href); + +try { + // --- 1. the package itself is refused, and the refusal says which path ----------------- + const atRoot = await startWith(PKG); + assert('the runtime refuses a data dir that is the plugin package', atRoot.refused, atRoot.message); + assert('the refusal names the path it refused', + atRoot.message.includes(PKG), atRoot.message); + + // --- 2. so is a directory under it ----------------------------------------------------- + const underRoot = join(PKG, 'tests'); + const under = await startWith(underRoot); + assert('the runtime refuses a data dir under the plugin package', under.refused, under.message); + assert('the refusal names the path it refused', + under.message.includes(resolve(underRoot)), under.message); + + // --- 3. a data dir outside the package is accepted ------------------------------------- + const outside = await startWith(outsideDir); + assert('the runtime accepts a data dir outside the plugin package', outside.accepted, outside.message); + + // A sibling whose name starts with the package's name is outside it. A refusal that matched on + // the string rather than on the path would turn this into a refusal of everything. + const sibling = `${PKG}-sibling`; + const beside = await startWith(sibling); + assert('the runtime accepts a sibling of the plugin package whose name starts with its name', + beside.accepted, beside.message); + + // --- 4. the package holds none of them ------------------------------------------------- + assert('the package holds no derived artifacts', + !existsSync(join(PKG, 'trajectory-artifacts'))); + + // --- 5. and it ignores the directory they would land in -------------------------------- + const ignored = GITIGNORE.split(/\r?\n/).map((line) => line.trim()); + assert('the package ignores the directory its artifacts would land in', + ignored.includes('trajectory-artifacts/') || ignored.includes('trajectory-artifacts'), + ignored.join(' | ')); + + // --- 6. and no shipped harness hands the package over as the data dir ------------------ + const offenders = []; + for (const name of harnesses) { + const text = readFileSync(join(fileURLToPath(new URL('.', import.meta.url)), name), 'utf8'); + for (const line of text.split(/\r?\n/)) { + if (/dataDir\s*[:=]\s*PLUGIN_ROOT/.test(line)) offenders.push(`${name}: ${line.trim()}`); + } + } + assert('no shipped harness hands the package to the runtime as its data dir', + offenders.length === 0, offenders.join(' | ')); +} finally { + controller.abort(); + rmSync(sessionsHome, { recursive: true, force: true }); + rmSync(outsideDir, { recursive: true, force: true }); + if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; + else process.env.MMC_TRAJECTORY_ROOT = previousRoot; +} + +if (failures.length) { + console.log(`\n [artifact-root] ${failures.length} failure(s):`); + for (const line of failures) console.log(` - ${line}`); + console.log(''); + process.exitCode = 1; +} else { + console.log(`[artifact-root] ${pass} assertions passed`); +} From 7c5eeaa8a40cc5de6093a6f2c6c15e103d501a1e Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 13:03:11 +0800 Subject: [PATCH 33/95] The ledger holds its rows only for the groups the reader is on, and a search names the records it answers about MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 「显示全部」built every row of every group into the page, and the poll threw them away and built them again two and a half seconds later. On a session of about 48,000 records that was 443,237 nodes and 560 MB of the renderer's private memory, rising past 1.2 GB while a rebuild ran with the old page still alive. The table is still rebuilt on every poll — the reader's place in the session is restored from it, and that is not something to give up. What changes is that a group the reader is not on no longer holds its rows. It keeps the height it had, so the scrollbar still describes the whole session, and its rows come back when the reader reaches it. The height is the hard part, because of two things about the browser. contain-intrinsic-size sizes an element the browser has skipped, and a group near the viewport is rendered rather than skipped: a group whose rows are gone collapses to its header, and the document under the reader loses everything they have scrolled past — measured at 79,288 px for the groups around the viewport and 257,378 px more for the viewport's own group. A released group therefore holds its height explicitly, from the height it had while it was holding its rows. The number of groups holding rows at once is bounded as well, so a reader who scrolls through three hundred groups in a minute is not holding all of them, and the rows of a group are built in one place, so the build and the observer that refills a group cannot disagree about what a row is. A search answers about a named set of record versions and says so when the set it was asked about is not the set it has: the page sends a fingerprint folded over the version tokens of the records it is showing, the service recomputes it from the records it has, and a mismatch is refused rather than answered about. A record's version is built from the line's digest rather than from its body, so a body rewritten in place is a different record, and a segment's source identity now carries the file's mtime, because a file rewritten in place at the same length is a different file to everything else that keys on it. --- .../mmc-trajectory/miniapp/client/index.html | 544 ++++++++++++++++-- .../miniapp/node/search-segments.mjs | 66 ++- .../mmc-trajectory/miniapp/node/server.mjs | 134 ++++- 3 files changed, 679 insertions(+), 65 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 2a7c5875..d2bdc54f 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -1409,6 +1409,47 @@ content-visibility: auto; } + /* A group that is holding its height but not its rows. + + The rows are gone, so the box is empty — and an empty box is a box of no height, which is + how the document under the reader loses the height of everything they have scrolled past. + The height is written onto the group from JavaScript, from the height it had while it was + holding its rows, and it is the only thing that holds the box while the browser is + RENDERING the group rather than skipping it: `contain-intrinsic-size` sizes an element + the browser has skipped, and a group near the viewport is rendered. + + Measured, on a real session of 47,937 records: releasing the rows of the groups around + the viewport took the scroll height down by 79,288 px, and releasing the viewport's own + group took another 257,378 px. Pinned by tests/verify-ledger-lifecycle.mjs. */ + .turn-group.is-released .row-list { + min-height: 2px; + } + + /* The group is on its way back. The bar stands where the rows are coming, so the reader + sees a line rather than a hole; the group's own height is held by its min-height, so the + bar costs the document nothing. */ + .turn-group.is-released .row-list::after { + content: ''; + display: block; + height: 2px; + margin: 10px 0; + border-radius: 1px; + background: var(--mcode-accent, currentColor); + opacity: .3; + animation: ledger-release 1.2s ease-in-out infinite; + } + + @keyframes ledger-release { + 0%, 100% { opacity: .1; transform: scaleX(.3); } + 50% { opacity: .55; transform: scaleX(1); } + } + + @media (prefers-reduced-motion: reduce) { + .turn-group.is-released .row-list::after { + animation: none; + } + } + .turn-head { position: sticky; /* Below every band above it, measured. It used to pin at 0 with z-index 2 and simply got @@ -4105,27 +4146,86 @@

检视

} /** - * The version of one record of the body on screen. + * The identity of a SET of records: a fold over their version tokens, in the order they are + * shown. + * + * A version token says 「this one record is this one version」. A search answer is about a SET + * of them — it says 「these match and the rest do not」 — and the part that matters to the + * reader is the part it does NOT name. So the set needs an identity of its own, and it has to + * be one the service can compute over ITS records and compare. + * + * Two properties, and both of them are the point: + * + * - **Stable under APPEND.** A session that grows does not change the records it already + * has, so the fold over the first N records is the same before and after an append. A + * body-level identity (an ETag, a digest of the whole body) changes every time the session + * is written to, and then every answer stops applying every few seconds — the common case + * would be the broken one. + * - **Different when a record is rewritten.** The token is a digest of that record's own + * bytes, so a record whose text was replaced under the page has a different token, and the + * fold over the same positions has a different answer. + * + * The fold is ordered, so a set that was reordered is a different set. Two lanes of 32 bits + * each. This is the same function as `recordSetFingerprint` in the service, and + * `tests/client-check.mjs` pins that the two agree — two implementations of one question is + * two answers, and the difference between them is nobody's to explain. + * + * @param {Array} versions + * @returns {string} 16 hex characters + */ + function recordSetFingerprint(versions) { + var h1 = 0x811c9dc5 >>> 0; + var h2 = 0x9e3779b9 >>> 0; + var list = Array.isArray(versions) ? versions : []; + for (var i = 0; i < list.length; i += 1) { + var text = typeof list[i] === 'string' ? list[i] : ''; + for (var j = 0; j < text.length; j += 1) { + var code = text.charCodeAt(j); + h1 = Math.imul(h1 ^ code, 16777619) >>> 0; + h2 = Math.imul(h2 ^ code, 2246822519) >>> 0; + } + // A separator, so that ["ab", "c"] and ["a", "bc"] are two different sets. + h1 = Math.imul(h1 ^ 0x1f, 16777619) >>> 0; + h2 = Math.imul(h2 ^ 0x1f, 2246822519) >>> 0; + } + return (h1 >>> 0).toString(16).padStart(8, '0') + (h2 >>> 0).toString(16).padStart(8, '0'); + } + + /** + * The version tokens of the records on screen, in the order they are shown. * - * The version is what makes index N in this body the same record as index N in another one. - * A transcript rewritten under a running page leaves the indexes alone and changes the - * records behind them, so the version is the only thing that tells the two apart — and a - * request that names an index has to carry it, or the service cannot tell which of the two - * records it is being asked about. + * The COUNT comes with them, because the service folds over ITS first N records and N is + * what says which of its records the fold is about. A page that is behind because the + * session grew has FEWER records, not different ones, so its first N are still the same N. * - * @param {number} index - * @returns {string} the version, or '' when this body carries none + * @returns {Array} */ - function displayedRecordVersion(index) { + function displayedRecordVersions() { + var out = []; var flat = Array.isArray(state.flat) ? state.flat : []; for (var i = 0; i < flat.length; i++) { var entry = flat[i]; var record = entry && entry.record ? entry.record : entry; - if (record && record.index === index) { - return typeof record.v === 'string' ? record.v : ''; - } + out.push(record && typeof record.v === 'string' ? record.v : ''); } - return ''; + return out; + } + + /** + * The version of one record, as the record itself carries it. + * + * The record, and NOT the body on screen. A read that comes back 「this session is not + * cached」 reloads the body and asks again — and the body it reloads is a DIFFERENT one, so a + * version read off the screen at that moment belongs to the new body's record, and the retry + * would be asking about a different record than the one the reader opened. The record the + * reader opened is the one that carries the answer to 「which record is this」, so it is the + * one that carries the version. + * + * @param {Record} record + * @returns {string} the version, or '' when the record carries none + */ + function recordVersionOf(record) { + return record && typeof record.v === 'string' ? record.v : ''; } /** @@ -4357,6 +4457,45 @@

检视

* * @param {string} needle */ + /** + * An answer that is not about the body on screen. + * + * One place, because there are two ways to find that out, and they mean the same thing: a + * hit whose position holds a different record, and a service that says the body it searched + * is not this one. Both mean the answer cannot be installed, and both are answered the same + * way — the body is fetched once more and the same question is put again against it. + * + * Bounded, because a session that is being written to keeps moving: after a couple of tries + * the page stops chasing it, says that the answer does not apply, and the next poll's own + * re-run takes over. + * + * @returns {boolean} whether the page is going to ask again + */ + function searchAnswerDoesNotApply() { + if (state.searchRefresh < 2) { + state.searchRefresh += 1; + state.searchError = null; + state.searchPending = true; + // The whole body, not a tail: a delta carries the records after the cursor, and the + // record that moved is not one of them. + state.since = 0; + state.sinceFor = ''; + state.etag = ''; + state.etagFor = ''; + fetchTrajectory(false).then(function () { + runSearch(state.query); + }, function () { + state.searchPending = false; + state.searchError = 'search_version_mismatch'; + renderLedger(); + }); + return true; + } + state.searchError = 'search_version_mismatch'; + renderLedger(); + return false; + } + function runSearch(needle) { if (searchController) searchController.abort(); var asked = String(needle == null ? '' : needle).trim(); @@ -4387,9 +4526,17 @@

检视

// are two different things, and a selection whose load failed leaves the old body up. var session = displayedSessionParam(); var generation = displayedGeneration(); + // WHICH BODY. The answer is about a SET of records, and the part of it that matters to + // the reader is the part it does not name: a record this body has and the other body does + // not match would be dimmed as if it did not contain the needle, with no error anywhere. + // So the request says which set it is about — how many records the page has, and a + // fingerprint of their versions — and the service answers only if it searched THAT set. + var versions = displayedRecordVersions(); var url = '/api/trajectory/search?session=' + session + '&q=' + encodeURIComponent(asked) - + (generation === null ? '' : '&generation=' + encodeURIComponent(String(generation))); + + (generation === null ? '' : '&generation=' + encodeURIComponent(String(generation))) + + '&n=' + versions.length + + '&fp=' + encodeURIComponent(recordSetFingerprint(versions)); fetch(url, { signal: controller.signal, headers: { Accept: 'application/json' } }) .then(function (response) { @@ -4467,30 +4614,10 @@

检视

applicable.push(hit); } if (wrongVersion) { - // Bounded, because a session that is being written to keeps moving: after a couple - // of tries the page stops chasing it, says that the answer does not apply, and the - // next poll's own re-run takes over. - if (state.searchRefresh < 2) { - state.searchRefresh += 1; - state.searchError = null; - state.searchPending = true; - // The whole body, not a tail: a delta carries the records after the cursor, and - // the record that moved is not one of them. - state.since = 0; - state.sinceFor = ''; - state.etag = ''; - state.etagFor = ''; - fetchTrajectory(false).then(function () { - runSearch(state.query); - }, function () { - state.searchPending = false; - state.searchError = 'search_version_mismatch'; - renderLedger(); - }); - return; - } - state.searchError = 'search_version_mismatch'; - renderLedger(); + // The whole answer is refused rather than partly installed, because the answer is + // about one body and the page is showing another. The body is then fetched once + // more and the same question is put again against the current body. + searchAnswerDoesNotApply(); return; } state.searchHits = searchHitSet(applicable); @@ -4519,6 +4646,13 @@

检视

searchController = null; state.searchPending = false; if (error && error.name === 'AbortError') return; + // The service refused because the body it searched is not the one on screen. That is + // the same thing a hit with the wrong version says, and it is answered the same way: + // fetch the body again and ask the question against it. + if (error && error.code === 'search_version_mismatch') { + searchAnswerDoesNotApply(); + return; + } // A search that could not be answered leaves the previous answer standing rather than // emptying the table: 「没搜到」 and 「没搜成」 are different, and only one of them is // true. The count line carries the reason. @@ -6850,6 +6984,231 @@

检视

return state.limit; } + /* ---------------- ledger group lifecycle ---------------- */ + + /** + * How long a group may keep its rows after the reader has left it. + * + * Long enough that scrolling back and forth over the same stretch does not rebuild it + * every time, short enough that the rows of a session the reader has moved past are not + * held for the rest of the session. + */ + var LEDGER_IDLE_MS = 15000; + + /** + * How many groups may hold their rows at once. + * + * The cap is what makes the memory bounded rather than merely reduced. A reader who + * scrolls through three hundred groups in a minute has been on all of them, and the idle + * timeout alone would not release them fast enough to stop it. Past the cap the oldest one + * goes, which is the order the reader would choose. + */ + var LEDGER_LIVE_GROUPS = 6; + + /** + * What is known about each turn group: whether it is holding its rows, when the reader was + * last on it, and how tall it was. + * + * Keyed by the turn's key, so it survives the poll that rebuilds the table. The DOM is + * thrown away and rebuilt every two and a half seconds; this is what carries the reader's + * place in it across that, and it is why the reader does not have to start again at the top + * of the session on every tick. + * + * @type {Map} + */ + var ledgerState = new Map(); + + /** + * The group each section was built from. + * + * A section that scrolls into view has to be filled, and filling it needs the records it + * stands for. The table is rebuilt on every poll, so this is rebuilt with it — a section + * from the previous table is not in here any more, and filling it would put rows into a + * table that no longer exists. + * + * @type {Map} + */ + var ledgerContent = new Map(); + + /** + * The observer that fills a group when the reader reaches it. + * + * One observer for the whole table: the sections are rebuilt on every poll, so an observer + * per section would have to be rebuilt with them. + */ + var ledgerObserver = null; + + /** + * The state of one group, created on first sight. + * + * @param {string} key + * @returns {{live: boolean, lastSeen: number, height: number}} + */ + function ledgerStateOf(key) { + var state = ledgerState.get(key); + if (!state) { + state = { live: false, lastSeen: 0, height: 0 }; + ledgerState.set(key, state); + } + return state; + } + + /** + * Whether a group should be holding its rows right now. + * + * Three things make a group live: the reader is on it, the reader was on it recently and + * the idle timeout has not passed, and it is the first group of the window — which is where + * the reader arrives, and the only source of the heights every other group is predicted + * from. + * + * @param {string} key + * @param {boolean} isFirst + * @returns {boolean} + */ + function ledgerHoldsRows(key, isFirst) { + if (isFirst) return true; + var state = ledgerState.get(key); + if (!state || !state.live) return false; + return Date.now() - state.lastSeen < LEDGER_IDLE_MS; + } + + /** + * Decide which groups give up their rows. + * + * Two rules, and the order matters: a group the reader has left goes when the idle + * timeout has passed, and when more groups are live than the cap allows, the oldest goes + * first. The second is what bounds the memory. + * + * This only decides. The rows themselves are taken away by the next build, which is the + * one place that writes them, so there is no second writer of the rows and nothing to keep + * in step. + * + * @param {string} keepKey the group the reader is on, which is never released + */ + function releaseStaleGroups(keepKey) { + var now = Date.now(); + var live = []; + ledgerState.forEach(function (state, key) { + if (!state.live) return; + if (key === keepKey) { + live.push({ key: key, at: state.lastSeen }); + return; + } + if (now - state.lastSeen >= LEDGER_IDLE_MS) { + state.live = false; + return; + } + live.push({ key: key, at: state.lastSeen }); + }); + if (live.length <= LEDGER_LIVE_GROUPS) return; + live.sort(function (a, b) { return a.at - b.at; }); + for (var i = 0; i < live.length - LEDGER_LIVE_GROUPS; i += 1) { + var state = ledgerState.get(live[i].key); + if (state) state.live = false; + } + } + + /** + * Put a group's rows back into it. + * + * The group is in view, so this is what the reader is about to read, and the rows go in + * before the next paint — so there is no frame in which the group is empty. + * + * @param {HTMLElement} section + */ + function fillLedgerGroup(section) { + var group = ledgerContent.get(section); + if (!group) return; + var state = ledgerStateOf(group.key); + state.live = true; + state.lastSeen = Date.now(); + if (section.getAttribute('data-rows') === 'live') return; + var list = section.querySelector('ul.row-list'); + if (!list) return; + fillGroupRows(list, group); + section.setAttribute('data-rows', 'live'); + section.classList.remove('is-released'); + section.style.minHeight = ''; + // The height it really has, now that it is laid out for real. A group the browser has + // skipped measures by its own contain-intrinsic-size, so this reads the prediction back + // rather than the truth — which is why it is the remembered height and not the only one. + sizeBuiltGroup(section); + state.height = Math.round(section.getBoundingClientRect().height); + } + + /** + * Watch the groups, so a group the reader scrolls to gets its rows. + * + * The observer is disconnected and rebuilt on every poll, because the sections are: a + * section from the previous table is not in `ledgerContent` any more. + */ + function watchLedgerGroups() { + if (!ledgerObserver) { + ledgerObserver = new IntersectionObserver(function (entries) { + for (var i = 0; i < entries.length; i += 1) { + if (!entries[i].isIntersecting) continue; + fillLedgerGroup(entries[i].target); + } + }); + } else { + ledgerObserver.disconnect(); + } + var sections = dom.ledger.querySelectorAll(':scope > section.turn-group'); + for (var i = 0; i < sections.length; i += 1) ledgerObserver.observe(sections[i]); + } + + /** + * Hold a group's height while it is not holding its rows. + * + * `contain-intrinsic-size` sizes a group the browser has skipped, and `min-height` sizes + * one it is rendering. A group near the viewport is rendered, so it needs the second, and + * without it the document under the reader loses the height of everything they have + * scrolled past — measured at 79,288 px for the groups around the viewport, and 257,378 px + * more for the viewport's own group. + * + * @param {HTMLElement} section + * @param {number} height + */ + function holdLedgerHeight(section, height) { + if (!(height > 0)) return; + section.style.minHeight = Math.round(height) + 'px'; + } + + /** + * Build a group's rows into its list. + * + * The rows of a group are built in one place, and the table has two writers for them: the + * build, which makes the groups the reader is on, and the observer, which fills a group the + * reader has just scrolled to. A second copy of this loop would be two definitions of what + * a row is, and the page has already paid for that once. + * + * A `null` list counts the compaction bands without building anything, which is what a + * group that is not holding its rows needs: its height is predicted from how many bands it + * has, and a prediction that ignored them would be short by every band in it. + * + * @param {HTMLElement|null} list + * @param {{turn: any, entries: any[]}} group + * @returns {number} how many of the entries are compaction bands + */ + function fillGroupRows(list, group) { + var rails = list ? railStates(group.entries) : null; + var bands = 0; + for (var j = 0; j < group.entries.length; j += 1) { + var isBand = group.entries[j].record.kind === 'compaction'; + if (isBand) bands += 1; + if (!list) continue; + var item = el('li'); + if (isBand) { + item.className = 'compaction-cell'; + item.appendChild(buildCompactionBand(group.entries[j])); + } else { + item.appendChild(buildRow(group.entries[j], rails[j])); + } + list.appendChild(item); + } + return bands; + } + /* ---------------- batched row build ---------------- */ /** @@ -6974,6 +7333,10 @@

检视

only fixes the first build after a page load. This line is what fixes the rest. */ ledger.classList.remove('is-skipping-offscreen'); clear(ledger); + // The sections the last build made are gone with them, and the map that says which group + // each one stands for has to go with them: a section that is no longer in the table must + // not be filled when the observer reports it as in view. + ledgerContent.clear(); if (state.loading) { ledger.appendChild(buildSkeleton()); @@ -7119,6 +7482,11 @@

检视

var buildTotal = 0; for (var b = 0; b < groups.length; b += 1) buildTotal += groups[b].entries.length; + // The groups the reader has left give up their rows, and the ones past the cap give them + // up oldest first. Decided here and applied by this build — the one place that writes + // rows — so there is no second writer to keep in step with this one. + if (groups.length) releaseStaleGroups(turnKey(groups[0].turn)); + function buildStep() { // A newer renderLedger() owns the ledger now. Stop without touching it: the rows half // built here belong to a table that no longer exists, and the exit that replaced it has @@ -7134,18 +7502,24 @@

检视

var list = el('ul', 'row-list'); // Every entry that survived the filters becomes a row. There is no second pass // that hides anything, so what is on screen is exactly what was selected. - var rails = railStates(group.entries); - var bands = 0; - for (var j = 0; j < group.entries.length; j += 1) { - var item = el('li'); - if (group.entries[j].record.kind === 'compaction') { - item.className = 'compaction-cell'; - item.appendChild(buildCompactionBand(group.entries[j])); - bands += 1; - } else { - item.appendChild(buildRow(group.entries[j], rails[j])); - } - list.appendChild(item); + // + // The rows are built for the groups the reader is on and released for the rest. What + // the reader gives up is the DOM, which is the whole cost: measured on a session of + // 47,937 records, the rows of 「显示全部」 are 438,986 nodes and about 470 MB of the + // renderer's private memory. What the reader does not give up is the height — a + // released group keeps the box it had, so the document under them does not change + // size when they scroll. + var key = turnKey(group.turn); + var holds = ledgerHoldsRows(key, build.at === 1); + var bands = fillGroupRows(holds ? list : null, group); + if (holds) { + section.setAttribute('data-rows', 'live'); + var held = ledgerStateOf(key); + held.live = true; + held.lastSeen = Date.now(); + } else { + section.setAttribute('data-rows', 'released'); + section.classList.add('is-released'); } section.appendChild(list); build.rows += group.entries.length; @@ -7155,8 +7529,17 @@

检视

// how a scrollbar ends up shorter than the table it is supposed to describe. On the // first batch build.heights is still null, so this is a no-op and sizeBuiltGroup() // below is what covers those groups. - applyIntrinsicSize(section, group.entries.length, bands, build.heights); + var predicted = applyIntrinsicSize(section, group.entries.length, bands, build.heights); + if (!holds) { + // The height the group had while it was holding its rows, if it ever was; the + // prediction otherwise. A group that has never been on screen has no measured + // height, and the prediction is the best answer there is — the same answer the + // scrollbar is already giving. + var known = ledgerStateOf(key).height; + holdLedgerHeight(section, known > 0 ? known : predicted); + } ledger.appendChild(section); + ledgerContent.set(section, { key: key, turn: group.turn, entries: group.entries, bands: bands }); built += 1; } @@ -7178,7 +7561,20 @@

检视

// viewport and get measured for real. if (!build.heights) { build.heights = calibrateRowHeights(ledger); - var firstBatch = ledger.querySelectorAll('.turn-group'); + // The groups that were built without their rows have nothing to be sized from, and a + // group left unsized collapses to nothing the moment the class goes on. They are + // sized now, from the calibration, with the same arithmetic every other group is + // sized with. + var dormant = ledger.querySelectorAll('.turn-group[data-rows="released"]'); + for (var d = 0; d < dormant.length; d += 1) { + var content = ledgerContent.get(dormant[d]); + if (!content) continue; + var height = applyIntrinsicSize(dormant[d], content.entries.length, content.bands, build.heights); + holdLedgerHeight(dormant[d], height); + } + // The groups that were holding their rows are the ones that can be measured. A group + // without rows measures as its header, and writing that down would collapse it. + var firstBatch = ledger.querySelectorAll('.turn-group[data-rows="live"]'); for (var f = 0; f < firstBatch.length && f < LEDGER_BATCH_GROUPS; f += 1) { sizeBuiltGroup(firstBatch[f]); } @@ -7198,6 +7594,10 @@

检视

// measuring has to stay the last word, and a trailing statement is the one place where // a future synchronous call would quietly measure an empty page. flushDeferredPoll(); + // The groups are watched from here, so a group the reader scrolls to gets its rows back + // before the next paint. Before this the sections do not exist — the table was empty a + // moment ago — and an observer pointed at nothing sees nothing. + watchLedgerGroups(); // Measured here rather than at the end of render(): this is the one exit every path that // changes what the table shows goes through — the filters that call render() and the nine // that call renderLedger() directly. A filter that grows the band therefore cannot leave @@ -7712,6 +8112,16 @@

检视

/** The topmost row that intersects the viewport, as {id, top}, or null. */ function firstVisibleRow() { + // The group the reader is on has to be holding its rows before the anchor can be read out + // of it. A released group has no rows, and an anchor read from the next live group down + // is an anchor to the wrong place — the reader would be moved to wherever the first group + // that still has rows happens to start. + var sections = dom.ledger.querySelectorAll(':scope > section.turn-group'); + for (var s = 0; s < sections.length; s += 1) { + if (sections[s].getBoundingClientRect().bottom <= 0) continue; + if (sections[s].getAttribute('data-rows') !== 'live') fillLedgerGroup(sections[s]); + break; + } var nodes = dom.ledger.querySelectorAll('[data-row-id]'); for (var i = 0; i < nodes.length; i += 1) { var rect = nodes[i].getBoundingClientRect(); @@ -7801,11 +8211,16 @@

检视

* * `auto` on the left is what lets the browser prefer the size it has already rendered once * the reader has scrolled past — so scrolling down and back does not accumulate drift. + * + * The height is returned as well as written, because a group that is not holding its rows + * needs the same number on its `min-height`: `contain-intrinsic-size` sizes a group the + * browser has skipped, and a group near the viewport is rendered rather than skipped. */ function applyIntrinsicSize(section, rows, bands, heights) { - if (!heights) return; + if (!heights) return 0; var h = heights.head + rows * heights.row + bands * heights.band; section.style.containIntrinsicSize = 'auto ' + Math.round(h) + 'px'; + return h; } /** @@ -8836,7 +9251,7 @@

检视

return sid + '#' + gen + '#' + (state.etag || '') + '#' + index; } - function requestRaw(index) { + function requestRaw(index, version) { var url = '/api/trajectory/raw?index=' + encodeURIComponent(String(index)) + '&session=' + displayedSessionParam(); var generation = displayedGeneration(); @@ -8847,7 +9262,10 @@

检视

// holding a newer body in which that position is another record — so the version of the // record the reader opened travels with the request, and the service can tell the two // apart instead of serving the newer record under the older one's number. - var version = displayedRecordVersion(index); + // + // The version is the one the RECORD carries, and that is the whole point: a read that is + // retried after a reload must ask about the same record it asked about the first time, or + // the retry is a different question wearing the same question's clothes. if (version) url += '&v=' + encodeURIComponent(version); return fetch(url, { headers: { Accept: 'application/json' } }).then(function (response) { return response.json().then( @@ -8879,7 +9297,7 @@

检视

var key = rawCacheKey(index); if (Object.prototype.hasOwnProperty.call(rawCache, key)) return Promise.resolve(rawCache[key]); if (rawPending[key]) return rawPending[key]; - var attempt = requestRaw(index).then(function (result) { + var attempt = requestRaw(index, recordVersionOf(record)).then(function (result) { delete rawPending[key]; if (result.ok && result.body && Object.prototype.hasOwnProperty.call(result.body, 'raw')) { var entry = { raw: result.body.raw, omitted: result.body.omitted === true }; @@ -8934,7 +9352,7 @@

检视

var detailCache = {}; var detailPending = {}; - function requestDetail(index) { + function requestDetail(index, version) { var url = '/api/trajectory/record?index=' + encodeURIComponent(String(index)) + '&session=' + displayedSessionParam(); var generation = displayedGeneration(); @@ -8942,8 +9360,8 @@

检视

url += '&generation=' + encodeURIComponent(String(generation)); } // The same record, the same way: the version travels with the request so the service can - // refuse when the position it would answer is a different record than the one opened. - var version = displayedRecordVersion(index); + // refuse when the position it would answer is a different record than the one opened — and + // it is the record's own version, so a retry after a reload asks about the same record. if (version) url += '&v=' + encodeURIComponent(version); return fetch(url, { headers: { Accept: 'application/json' } }).then(function (response) { return response.json().then( @@ -8969,7 +9387,7 @@

检视

var key = rawCacheKey(index); if (Object.prototype.hasOwnProperty.call(detailCache, key)) return Promise.resolve(detailCache[key]); if (detailPending[key]) return detailPending[key]; - var attempt = requestDetail(index).then(function (result) { + var attempt = requestDetail(index, recordVersionOf(record)).then(function (result) { delete detailPending[key]; if (result.ok && result.body && typeof result.body.index === 'number') { var entry = { @@ -9513,6 +9931,12 @@

检视

function selectSession(id) { state.generation = null; state.sessionId = id; + // The groups of the last session are not the groups of this one, and a turn key means + // nothing across two sessions. What is known about them goes with them, so the first + // build of the new session starts from nothing rather than from another session's + // heights. + ledgerState.clear(); + ledgerContent.clear(); setFollow(false); } diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs index 41c9b661..f7f6833f 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs @@ -61,7 +61,7 @@ export const SEGMENT_SCHEMA_VERSION = 4; */ export const SEARCH_FIELDS = ['summary', 'text', 'result', 'thinking', 'toolName']; -/** @typedef {{ fileName: string, generation: number | null, size: number, revision: string | null, truncated?: boolean }} SegmentSource */ +/** @typedef {{ fileName: string, generation: number | null, size: number, mtimeMs?: number | null, revision: string | null, truncated?: boolean }} SegmentSource */ /** @typedef {{ schemaVersion: number, generation: number | null, view: string, recordCount: number, surfaceBytes: number, surfaceDigest: string, source: SegmentSource[] }} SegmentManifest */ @@ -451,6 +451,61 @@ export function createSegmentStore(root) { }; } +/** + * The version tokens of a segment's records, in order, up to a limit. + * + * Every line of a segment carries the version of the record it is, so the versions are already + * there — this reads them out rather than keeping a second copy of them. The reading is by hand + * instead of by parsing each line's whole answer, because a search over a big segment would then + * parse every line it does not need; `tests/verify-search-segments.mjs` pins that this and the + * parsed answer agree. + * + * @param {string} surface + * @param {number} [limit] stop after this many records + * @returns {Array} + */ +export function segmentVersions(surface, limit) { + /** @type {Array} */ + const out = []; + if (typeof surface !== 'string' || surface === '') return out; + const wanted = Number.isInteger(limit) && limit > 0 ? limit : Infinity; + let start = 0; + while (start <= surface.length && out.length < wanted) { + const nl = surface.indexOf('\n', start); + const end = nl === -1 ? surface.length : nl; + const tab = surface.indexOf('\t', start); + if (tab >= 0 && tab <= end) out.push(versionInAnswer(surface.slice(start, tab))); + if (nl === -1) break; + start = nl + 1; + } + return out; +} + +/** + * The version token inside one line's answer. + * + * The answer is JSON and the version is one of its fields. Reading that one field by name is + * what keeps a search from parsing every line of a segment; `buildSegment` writes the field, and + * the guard pins that the two readings agree. + * + * @param {string} answer the JSON prefix of one segment line + * @returns {string} + */ +function versionInAnswer(answer) { + const at = answer.indexOf('"v":"'); + if (at < 0) return ''; + const from = at + 5; + let end = from; + while (end < answer.length) { + const ch = answer.charAt(end); + // A backslash escapes the next character, so it cannot be the end of the value. + if (ch === '\\') { end += 2; continue; } + if (ch === '"') break; + end += 1; + } + return answer.slice(from, end); +} + /** * Whether the generation a segment was built from is still the generation on disk. * @@ -470,6 +525,15 @@ export function sameSource(stored, live) { if (a.fileName !== b.fileName) return false; if (a.generation !== b.generation) return false; if (a.size !== b.size) return false; + // The mtime is this process's idea of a file's identity: the parsed-rows cache and the + // trajectory entry both key on (size, mtime), so a file that was rewritten IN PLACE — the same + // length, different bytes — is a different file to everything else in here. Leaving the mtime + // out of this comparison is what let a same-size rewrite keep a segment built from the old + // bytes, and the search then answered from those bytes as if they were the current ones. + // + // A sealed file is not written to, so its mtime does not move on its own; comparing it costs + // a rebuild only when something touched a file it had no reason to touch. + if (a.mtimeMs !== b.mtimeMs) return false; // Same reason as in the index: a generation read short by our own cap produces a segment that // covers a prefix, while `size` is the whole file. Trusting it would answer with a smaller // `total` and say nothing about being wrong. diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 49cec58b..597b53ae 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -18,6 +18,7 @@ import { normaliseWhitespace, readSegmentHits, scanSegment, + segmentVersions, searchSurfaceOf, } from './search-segments.mjs'; @@ -403,6 +404,85 @@ export function stampRecordVersions(payload, locators) { return payload; } +/** + * The identity of a SET of records: a fold over their version tokens, in the order they are shown. + * + * A version token says 「this one record is this one version」. It does not say anything about a + * set of them, and a set is what a search answer is about: an answer says 「these records match and + * the rest do not」, and the part that matters to the reader is the part it does NOT name. + * + * Two properties, and both of them are the point: + * + * - **It is stable under APPEND.** A session that grows does not change the records it already + * has, so the fold over the first N records is the same before and after an append. This is + * the same reason the version token is built from the line's digest rather than from the body: + * a body-level identity (an ETag, a digest of the whole body) changes every time the session + * is written to, and then every answer stops applying every few seconds — the common case + * would be the broken one. + * - **It changes when a record is rewritten.** The token is a digest of that record's own bytes, + * so a record whose text was replaced under the page has a different token, and the fold over + * the same positions has a different answer. + * + * The fold is ordered, so a set that was reordered is a different set. Two lanes of 32 bits each: + * a collision is not a thing a reader can notice, and one lane of 32 is one thing a reader could + * notice. + * + * @param {Array} versions the records' version tokens, in order + * @returns {string} 16 hex characters + */ +export function recordSetFingerprint(versions) { + let h1 = 0x811c9dc5 >>> 0; + let h2 = 0x9e3779b9 >>> 0; + const list = Array.isArray(versions) ? versions : []; + for (let i = 0; i < list.length; i += 1) { + const text = typeof list[i] === 'string' ? list[i] : ''; + for (let j = 0; j < text.length; j += 1) { + const code = text.charCodeAt(j); + h1 = Math.imul(h1 ^ code, 16777619) >>> 0; + h2 = Math.imul(h2 ^ code, 2246822519) >>> 0; + } + // A separator, so that ["ab", "c"] and ["a", "bc"] are two different sets. + h1 = Math.imul(h1 ^ 0x1f, 16777619) >>> 0; + h2 = Math.imul(h2 ^ 0x1f, 2246822519) >>> 0; + } + return hex8(h1) + hex8(h2); +} + +/** + * Eight hex characters, most significant first. + * @param {number} value + * @returns {string} + */ +function hex8(value) { + return (value >>> 0).toString(16).padStart(8, '0'); +} + +/** + * The version tokens of a payload's records, in the order the ledger shows them. + * + * The order is the payload's, not the index order: the ledger draws turn by turn, and a fold over + * a different order is a different answer to 「这批记录是哪一批」. + * + * @param {{ turns?: Array }} payload + * @param {number} [limit] stop after this many records + * @returns {Array} + */ +export function payloadRecordVersions(payload, limit) { + /** @type {Array} */ + const out = []; + const cap = Number.isInteger(limit) && limit >= 0 ? limit : Infinity; + const turns = payload && Array.isArray(payload.turns) ? payload.turns : []; + for (const turn of turns) { + const records = turn && Array.isArray(turn.records) ? turn.records : []; + for (const record of records) { + if (!record || typeof record !== 'object') continue; + out.push(typeof record.v === 'string' ? record.v : ''); + if (out.length >= cap) return out; + } + } + return out; +} + /** * Cheap single-pass counters used by the session list; values are approximate by design. * @@ -4505,6 +4585,25 @@ export async function start(context) { const needle = foldNeedle(asked); const { entry, filter } = ctx; + // WHICH BODY THE ANSWER IS ABOUT. The page says which body that is: the records it has, in + // order, as a fingerprint. The service folds its own records the same way and compares, and + // refuses to answer when the two are not the same set. + // + // Without this the service can only answer about the body on disk, and the page then filters + // its own body with an answer about another one. What hurts is the part the answer does NOT + // name: a record this body has and that body does not match gets dimmed as if it did not + // contain the needle, with no error anywhere, and the reader is told the record is not there. + // + // The count matters as much as the fingerprint. The fold is over the FIRST N records, because + // the page is behind whenever the session grew since its last poll — and a session that grew + // has MORE records, not different ones, so the first N are still the same N. + const count = Number.parseInt(url.searchParams.get('n') ?? '', 10); + const askedFingerprint = (url.searchParams.get('fp') ?? '').trim(); + if (!Number.isInteger(count) || count < 0 || !/^[0-9a-f]{16}$/.test(askedFingerprint)) { + sendError(response, 400, 'search_fingerprint_required', '会话已更新，请重新加载后再搜索'); + return; + } + let walked; try { walked = await collectLineage(entry, ctx.generations); @@ -4538,8 +4637,15 @@ export async function start(context) { // Before anything expensive. The lineage walk above is stat-driven and cheap; everything after // this line is the rebuild, and on the largest session measured here it was 3.7 seconds for a // search that returns at most 500 rows. - const onDisk = await searchFromArtifacts({ sessionKey, viewKey, files, needle }); + const onDisk = await searchFromArtifacts({ sessionKey, viewKey, files, needle, count }); if (onDisk !== null) { + // The segments ARE the body this answer is about, so their records decide whether it is + // about the body the page is showing. The fold is over the first N, for the reason given + // above: a page that is behind has fewer records, not different ones. + if (onDisk.fingerprint !== askedFingerprint) { + sendError(response, 409, 'search_version_mismatch', '该记录所在的版本已变化，请重新加载会话'); + return; + } sendAnswer(onDisk.hits, onDisk.total); return; } @@ -4571,6 +4677,13 @@ export async function start(context) { } stampRecordVersions(payload, locatorsByRecord); + // The same check, on the same records, on the path that rebuilt them. The fold is over the + // first N records of THIS body, which is what the page folded over the first N of its own. + if (recordSetFingerprint(payloadRecordVersions(payload, count)) !== askedFingerprint) { + sendError(response, 409, 'search_version_mismatch', '该记录所在的版本已变化，请重新加载会话'); + return; + } + /** @type {Array<{ index: number, turnIndex: number, kind: string, summary: string, generation: number | null, v: string }>} */ const hits = []; let total = 0; @@ -4617,12 +4730,17 @@ export async function start(context) { * The session's record index is NOT read. Each segment line carries its own answer, so a needle * matching 14 records out of 36,525 reads those 14 lines rather than parsing 34.2 MB of index. * - * @param {{ sessionKey: string, viewKey: string, files: Array, needle: string }} input + * @param {{ sessionKey: string, viewKey: string, files: Array, needle: string, count: number }} input */ - const searchFromArtifacts = async ({ sessionKey, viewKey, files, needle }) => { + const searchFromArtifacts = async ({ sessionKey, viewKey, files, needle, count }) => { /** @type {Array<{ index: number, turnIndex: number | null, kind: string, summary: string, generation: number | null }>} */ const hits = []; let total = 0; + // The version tokens of the records these segments cover, in the order the ledger shows them. + // They are what the fingerprint is folded over, and the fingerprint is what says whether this + // answer is about the body the page is showing. + /** @type {Array} */ + const versions = []; // `files` is the lineage walk's own output, so a generation that is no longer in the lineage // has no entry here and no segment is read — a leftover on disk cannot keep it searchable. for (let fileIdx = 0; fileIdx < files.length; fileIdx += 1) { @@ -4638,12 +4756,20 @@ export async function start(context) { total += 1; if (hits.length < MAX_SEARCH_HITS) hits.push(hit); } + // The versions of this generation's records, up to the number the page asked about. Read + // from the surface the segment already holds rather than from a second copy of it: the + // version is on every line, and a second copy would be a second thing to keep in step. + if (versions.length < count) { + for (const version of segmentVersions(segment.surface, count - versions.length)) { + versions.push(version); + } + } } // Ascending ledger order, because the segments were read oldest generation first and a hit's // place in that list is what the page scrolls to. Sorting by the index itself is the order the // rebuild produces, and two answers that differ only in order are two answers. hits.sort((a, b) => a.index - b.index); - return { hits, total }; + return { hits, total, fingerprint: recordSetFingerprint(versions) }; }; /** From 72759e97e7a38dc4722e17b8f2caad4125d976fb Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 13:03:12 +0800 Subject: [PATCH 34/95] The guards that own the ledger's rows and the search route's version tests/verify-ledger-lifecycle.mjs runs the shipped lifecycle in a sandbox with a fake DOM, and asserts the two sides of it: a group the reader is not on builds no rows, and a group that has given them up keeps the height it had, puts them back when the reader reaches it, and is read out of the anchor in the right place. The number of groups holding rows at once is bounded and the oldest go first, and a compaction record is a band rather than a row. tests/mutate-ledger-lifecycle.py puts each of those back, twenty in all, and requires the guard to report it by name before the file is restored byte for byte. One of them turns the idle timeout off, one removes the bound, one sorts the release order the wrong way round, one reads the anchor without filling the group the reader is on, and one breaks the branch that renders a compaction record as a band. The band rule is the one that changed shape. The routing of a checkpoint to its band moved out of the build and into the function that builds a group's rows, which put the test and the branch that uses it two lines apart, and the assertion that owned it was checking that the word appeared rather than that it decided. The assertion now reads the branch, so breaking the branch and breaking the word are different things, and the mutation that breaks it is re-anchored onto the branch rather than onto the word. tests/verify-trajectory-cache.mjs and tests/verify-search-segments.mjs own the other half: a search answer belongs to the set of record versions it was asked about, the route says so when it is not, a rendered body may not keep four versions of one question alive, a search segment may not cross a record boundary, and a scan is resolved in one walk. --- .../mmc-trajectory/tests/client-check.mjs | 174 ++++++- .../tests/mutate-compaction-turn.py | 9 +- .../tests/mutate-intrinsic-size.py | 4 +- .../tests/mutate-ledger-lifecycle.py | 225 +++++++++ .../tests/mutate-search-client.py | 74 ++- .../tests/mutate-search-endpoint.py | 4 +- .../tests/mutate-search-segments.py | 20 + .../tests/mutate-trajectory-cache.py | 38 ++ .../tests/verify-ledger-lifecycle.mjs | 426 ++++++++++++++++++ .../tests/verify-search-endpoint.mjs | 20 +- .../tests/verify-search-segments.mjs | 50 ++ .../tests/verify-trajectory-cache.mjs | 113 ++++- 12 files changed, 1120 insertions(+), 37 deletions(-) create mode 100644 plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py create mode 100644 plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index e322754f..f2df8f44 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -7,6 +7,10 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { execFileSync } from 'node:child_process'; import vm from 'node:vm'; +// The service's own fold over a set of records. The page has a copy of it — two copies of one +// question is two answers, and the difference between them is nobody's to explain — so the two are +// compared here rather than trusted. +import { recordSetFingerprint } from '../miniapp/node/server.mjs'; const HTML = fileURLToPath(new URL('../miniapp/client/index.html', import.meta.url)); const SERVER = fileURLToPath(new URL('../miniapp/node/server.mjs', import.meta.url)); @@ -228,8 +232,9 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); vm.createContext(netCtx); const needed = ['function isCount(value)', 'function formatInt(value)', 'function searchHitSet(hits)', 'function sessionParam()', 'function displayedSessionParam()', 'function displayedGeneration()', - 'function displayedRecordsByIndex()', 'function displayedRecordVersion(index)', - 'function searchStatusLabel(searchState)', 'function runSearch(needle)']; + 'function displayedRecordsByIndex()', 'function displayedRecordVersions()', + 'function recordSetFingerprint(versions)', 'function searchStatusLabel(searchState)', + 'function searchAnswerDoesNotApply()', 'function runSearch(needle)']; const sources = needed.map((sig) => fnSource(sig)); check('the shipped search functions are all still findable by signature', needed.filter((sig, i) => !sources[i]), []); @@ -496,6 +501,51 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); netCtx.state.searchError, null); check('and the page did not go chasing a body it did not need to', netCtx.refreshes, 0); + + /* WHICH SET the answer is about. + * + * A search answer is about a SET of records: it says 「these match and the rest do not」, and + * the part that matters to the reader is the part it does NOT name. So the request has to say + * which set — how many records the page has, and a fingerprint of their versions — and the + * service has to answer only if it searched THAT set. Without it, an answer about another body + * dims a record the reader can see, and says nothing about it. + */ + check('the search names how many records the body on screen has', + /[?&]n=3(&|$)/.test(asked[0]), true); + check('and a fingerprint of their versions', + /[?&]fp=[0-9a-f]{16}(&|$)/.test(asked[0]), true); + check('and the page folds a set of records into the same answer the service does', + netCtx.recordSetFingerprint(['version-one', 'version-five', 'version-seven']), + recordSetFingerprint(['version-one', 'version-five', 'version-seven'])); + check('and a set that is not the same is not the same answer', + netCtx.recordSetFingerprint(['version-one', 'version-five']) + === netCtx.recordSetFingerprint(['version-one', 'version-seven']), false); + check('and the count is appended, so the request still names the session first', + /[?&]session=mvs_displayed(&|$)/.test(asked[0]), true); + + // The service refuses: the body it searched is not the body on screen. That is the same thing a + // hit with the wrong version says, and it has to be answered the same way — the answer is not + // installed, the body is fetched once more, and the same question is put again against it. + const beforeRefusal = [...netCtx.state.searchHits]; + netCtx.state.searchRefresh = 0; + netCtx.state.searchRefreshFor = ''; + netCtx.refreshes = 0; + asked.length = 0; + netCtx.fetch = (url) => { + asked.push(String(url)); + return Promise.resolve({ ok: false, status: 409, + json: () => Promise.resolve({ error: 'search_version_mismatch', message: 'x' }) }); + }; + netCtx.runSearch('retry me'); + await settle(24); + check('an answer the service refuses is not installed', + [...netCtx.state.searchHits], beforeRefusal); + check('and the page asks again against a body it fetches, a bounded number of times', + netCtx.refreshes, 2); + check('and then it says the answer does not apply to what is on screen', + netCtx.state.searchError, 'search_version_mismatch'); + check('and the label shows the failure rather than a count it cannot stand behind', + netCtx.searchStatusLabel(netCtx.state), '搜索失败，以下为上次结果'); } /* handoff: the session the record came from, across a navigation. @@ -1386,15 +1436,35 @@ const checks = [ // A checkpoint is drawn as a band, not as a row. It has to be routed there in the one place // the ledger builds its list — routing it anywhere else would leave the row grid laying it out // like ordinary work while the CSS sits unused. + // + // That place is fillGroupRows(), which the build calls: the rows of a group are built by the + // build and by the observer that fills a group the reader scrolls to, and a second copy of the + // loop would be two definitions of what a row is. Both halves are checked — a routing rule in a + // function the ledger does not call is a rule about nothing. ['a compaction record renders as a band, not as a row', (() => { if (!/function buildCompactionBand\(entry\)/.test(html)) return false; + const fill = fnSource('function fillGroupRows(list, group)'); + if (!fill) return false; const render = fnSource('function renderLedger()'); if (!render) return false; - const loop = render.slice(render.indexOf('var rails = railStates(')); - if (!loop) return false; - return /group\.entries\[j\]\.record\.kind === 'compaction'/.test(loop) - && /item\.className = 'compaction-cell'/.test(loop) - && /item\.appendChild\(buildCompactionBand\(group\.entries\[j\]\)\)/.test(loop); + // The test and the branch it decides have to be one thing. + // + // Checking that the word 'compaction' appears in the function is not the same as checking + // that a compaction record is rendered as a band: a version that computes the test and then + // ignores it says the same word and renders the record as a row. That is not a hypothetical + // — the two were once the same line, and splitting them (the test into a variable, the + // branch into an `if`) left the old assertion checking a word rather than a decision. + // + // So the band branch is taken out of the function and read on its own, and the branch has to + // be the one the test opens. The row branch is checked too: a version that renders EVERY + // record as a band has the band branch right and is wrong in the other direction. + const band = fill.match(/if \(isBand\) \{([\s\S]*?)\n {10}\} else \{/); + return /var isBand = group\.entries\[j\]\.record\.kind === 'compaction';/.test(fill) + && !!band + && /item\.className = 'compaction-cell'/.test(band[1]) + && /item\.appendChild\(buildCompactionBand\(group\.entries\[j\]\)\)/.test(band[1]) + && /item\.appendChild\(buildRow\(group\.entries\[j\], rails\[j\]\)\)/.test(fill) + && /fillGroupRows\(holds \? list : null, group\)/.test(render); })()], // The band is a real control: it carries the same data-row-id the inspector looks up, so // clicking it opens the checkpoint's details rather than doing nothing. The two halves are @@ -1527,6 +1597,11 @@ const layerSrc = layerFns.map((name) => fnSource(name)); if (layerSrc.some((src) => !src)) { failures.push('session layers: could not lift ' + layerFns.filter((_, i) => !layerSrc[i]).join(', ')); } else { + // selectSession() drops what the ledger knows about the groups of the session it is leaving, so + // the two maps it uses have to exist here. They are the page's own, lifted as empty ones: what + // is under test is which session is chosen, not what the ledger remembers about it. + sandbox.ledgerState = new Map(); + sandbox.ledgerContent = new Map(); vm.runInContext(layerSrc.join('\n'), sandbox); } @@ -2024,18 +2099,18 @@ if (!fetchTrajectorySrc || !sessionParamSrc || !trajInflightDecl || !trajSeqDecl // is "whichever session is current now" and not necessarily the one on screen. The runtime can // make another session current while the page still shows this one, and the page then showed // that other session's words under this session's record. -const requestRawSrc = fnSource('function requestRaw(index)'); -const requestDetailSrc = fnSource('function requestDetail(index)'); +const requestRawSrc = fnSource('function requestRaw(index, version)'); +const requestDetailSrc = fnSource('function requestDetail(index, version)'); const displayedSessionSrc = fnSource('function displayedSessionParam()'); const displayedGenerationSrc = fnSource('function displayedGeneration()'); -const displayedRecordVersionSrc = fnSource('function displayedRecordVersion(index)'); +const recordVersionOfSrc = fnSource('function recordVersionOf(record)'); const rawCacheKeySrc = fnSource('function rawCacheKey(index)'); if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGenerationSrc - || !displayedRecordVersionSrc || !rawCacheKeySrc) { + || !recordVersionOfSrc || !rawCacheKeySrc) { failures.push('secondary reads: could not lift requestRaw, requestDetail and the session they ask for'); } else { vm.runInContext(displayedSessionSrc + '\n' + displayedGenerationSrc + '\n' - + displayedRecordVersionSrc + '\n' + + recordVersionOfSrc + '\n' + requestRawSrc + '\n' + requestDetailSrc + '\n' + rawCacheKeySrc, sandbox); const secondary = []; @@ -2058,8 +2133,8 @@ if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGen flat: [{ turn: {}, record: { index: 1, v: 'the-version-of-record-one' } }], generation: null, etag: '"tag"', followLatest: true, sessionId: '', hostSessionId: '' }; - await sandbox.requestRaw(1); - await sandbox.requestDetail(1); + await sandbox.requestRaw(1, 'the-version-of-record-one'); + await sandbox.requestDetail(1, 'the-version-of-record-one'); check('a raw read asks for the session on screen', /[?&]session=mvs_displayed(&|$)/.test(secondary[0]), true); check('and it does not ask for whatever is current now', @@ -2081,8 +2156,8 @@ if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGen flat: [{ turn: {}, record: { index: 1, v: 'the-version-of-record-one' } }], generation: 7, etag: '"tag"', followLatest: true, sessionId: '', hostSessionId: '' }; - await sandbox.requestRaw(1); - await sandbox.requestDetail(1); + await sandbox.requestRaw(1, 'the-version-of-record-one'); + await sandbox.requestDetail(1, 'the-version-of-record-one'); check('a raw read asks for the slice on screen', /[?&]generation=2(&|$)/.test(secondary[2]), true); check('and not for the one the reader picked', @@ -2106,11 +2181,76 @@ if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGen // With nothing on screen there is no session to name, and the read falls back to the session // parameter rather than sending an empty one. sandbox.state = { payload: null, generation: null, etag: '', followLatest: true, sessionId: '', hostSessionId: '' }; - await sandbox.requestRaw(1); + await sandbox.requestRaw(1, 'the-version-of-record-one'); check('with no body on screen the read falls back to the current session', /[?&]session=latest(&|$)/.test(secondary[4]), true); } +/* A read that is retried must ask about the SAME record. + * + * The retry exists because the service let go of the session: it reloads the body and asks again. + * The body it reloads is a DIFFERENT one — the transcript moved while the tab was open — so a + * version read off the screen at that moment belongs to the NEW body's record at that index, and + * the retry is a different question wearing the same question's clothes. The service would answer + * it, because it is a perfectly good question about a perfectly good record: the wrong one. + * + * So the version travels with the RECORD the reader opened, and the reload does not change it. */ +{ + const fetchRawSrc = fnSource('function fetchRaw(record, retried)'); + const fetchDetailSrc = fnSource('function fetchDetail(record, retried)'); + const requestDetailSrc2 = fnSource('function requestDetail(index, version)'); + const isCountSrc = fnSource('function isCount(value)'); + if (!fetchRawSrc || !fetchDetailSrc || !requestRawSrc || !requestDetailSrc2 || !isCountSrc + || !recordVersionOfSrc) { + failures.push('secondary read retry: could not lift fetchRaw, fetchDetail and the requests they make'); + } else { + vm.runInContext('var rawCache = {};\nvar rawPending = {};\nvar detailCache = {};\nvar detailPending = {};\n' + + isCountSrc + '\n' + recordVersionOfSrc + '\n' + + requestRawSrc + '\n' + requestDetailSrc2 + '\n' + + fetchRawSrc + '\n' + fetchDetailSrc, sandbox); + + const retried = []; + let attempts = 0; + sandbox.fetch = function (url) { + retried.push(String(url)); + attempts += 1; + if (attempts === 1 || attempts === 3) { + return Promise.resolve({ ok: false, status: 503, + json: () => Promise.resolve({ error: 'trajectory_not_cached' }) }); + } + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ raw: 'ALPHA BODY', index: 1, text: 'ALPHA BODY', omitted: false }) }); + }; + // The reload is what brings the NEW body: the same index, another record. A version read off + // the screen after this point is the new record's version, and the retry would be asking + // about it. + sandbox.fetchTrajectory = function () { + sandbox.state.flat = [{ turn: {}, record: { index: 1, v: 'a-newer-version' } }]; + return Promise.resolve(); + }; + sandbox.state = { + payload: { session: { id: 'mvs_displayed', generation: null } }, + flat: [{ turn: {}, record: { index: 1, v: 'the-version-of-record-one' } }], + generation: null, etag: '"tag"', followLatest: true, sessionId: '', hostSessionId: '' + }; + const opened = { index: 1, v: 'the-version-of-record-one' }; + await sandbox.fetchRaw(opened); + await sandbox.fetchDetail(opened); + check('the first read names the version of the record the reader opened', + /[?&]v=the-version-of-record-one(&|$)/.test(retried[0] || ''), true); + check('a read that is retried asks about that same record', + /[?&]v=the-version-of-record-one(&|$)/.test(retried[1] || ''), true); + check('and not about the record that happens to be at that index after the reload', + /[?&]v=a-newer-version(&|$)/.test(retried[1] || ''), false); + check('and the same for the record\'s fields', + /[?&]v=the-version-of-record-one(&|$)/.test(retried[2] || ''), true); + check('a field read that is retried asks about that same record too', + /[?&]v=the-version-of-record-one(&|$)/.test(retried[3] || ''), true); + check('and not about the record that happens to be at that index after the reload', + /[?&]v=a-newer-version(&|$)/.test(retried[3] || ''), false); + } +} + console.log('[2] pure helpers: ' + pass + ' passed, ' + failures.length + ' failed'); if (failures.length) { console.log('FAILURES:\n - ' + failures.join('\n - ')); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-compaction-turn.py b/plugins/avatasia/mmc-trajectory/tests/mutate-compaction-turn.py index 2253f34b..3b314ddc 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-compaction-turn.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-compaction-turn.py @@ -60,10 +60,15 @@ " var opened = false", ), ( + # Re-anchored: the routing moved out of renderLedger() and into fillGroupRows(), which + # is the one place a group's rows are built. The rule did not change, only where it + # lives. The decision and the branch that uses it are two lines apart here, so the + # branch is what gets broken — and the guard that owns the rule now reads the branch + # rather than the word. "client: the ledger stops routing checkpoints to the band", CLIENT, - " if (group.entries[j].record.kind === 'compaction') {", - " if (false) {", + " if (isBand) {\n", + " if (false) {\n", ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-intrinsic-size.py b/plugins/avatasia/mmc-trajectory/tests/mutate-intrinsic-size.py index 68c523a4..bb69d290 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-intrinsic-size.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-intrinsic-size.py @@ -79,14 +79,14 @@ # Every group after the calibration stops being sized, which is the same collapse applied to # 127 of 152 groups instead of 25 — and the count of sized groups is what the measurement reads. ("later groups are left unsized", - [(" applyIntrinsicSize(section, group.entries.length, bands, build.heights);\n", "")], + [(" var predicted = applyIntrinsicSize(section, group.entries.length, bands, build.heights);\n", "")], "a group is sized before it is appended, in the row loop"), # Predicting before there is anything to predict from. Dropping the guard does not write a # wrong number — it throws on `null.head`, and the throw happens while the very first batch is # being built, before a single row reaches the ledger. ("the guard against sizing without heights is dropped", - [(" function applyIntrinsicSize(section, rows, bands, heights) {\n if (!heights) return;\n", + [(" function applyIntrinsicSize(section, rows, bands, heights) {\n if (!heights) return 0;\n", " function applyIntrinsicSize(section, rows, bands, heights) {\n")], "declining to size without heights is not a crash"), diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py new file mode 100644 index 00000000..f2704f7f --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -0,0 +1,225 @@ +"""Break the per-group lifecycle in the shipped client and require verify-ledger-lifecycle.mjs +to report each break by name. + +The lifecycle has two halves that pull in opposite directions, and a version of this that keeps +one and loses the other is worse than not doing it at all: + + - a group the reader is not on must not be holding its rows, or 47,937 records are 438,986 + nodes and about 470 MB of the renderer's private memory; + - a group that has given up its rows must still be the height it was, or the document under + the reader loses everything they have scrolled past — measured at 79,288 px for the groups + around the viewport and 257,378 px more for the viewport's own group. + +Each mutation below breaks one of those, or one of the transitions between them, and the guard +has to name the rule it broke. A mutation that is only "the file no longer parses" proves +nothing, so the guard has to exit non-zero *and* report the rule by name. + +Anchors are multi-line and index.html is LF, so they are written with bare \\n and the file is +read and written with newline='' — matching on normalised text while preserving the file's own +endings. +""" +import io +import os +import subprocess +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +CLIENT = os.path.join(os.path.dirname(HERE), "miniapp", "client", "index.html") +PROBE = os.path.join(HERE, "verify-ledger-lifecycle.mjs") + +# Every anchor below is text the rule is written on — a statement, a condition, an argument +# list. Anchoring on prose would mean an edit that merely rewords a comment reported as drift, +# and an edit that breaks the rule without touching the prose went unnoticed. +# +# `expect` is the name of the assertion the guard has to report. It is a list where one break +# owns more than one rule: removing the class removal breaks both the state the reader sees +# and the rule that says the state is turned off. +MUTATIONS = [ + # The idle timeout is what releases a group the reader has walked away from. Without it a + # group stays live for as long as the session is open, and the bound below becomes the only + # thing standing between the reader and 470 MB. + ("a group the reader has left never gives its rows up", + [(" if (now - state.lastSeen >= LEDGER_IDLE_MS) {\n", + " if (false) {\n")], + "a group the reader has left gives its rows up once the idle time has passed"), + + # The bound is what makes the memory bounded rather than merely reduced: a reader who + # scrolls through three hundred groups in a minute has been on all of them. + ("the number of groups holding their rows is never bounded", + [(" if (live.length <= LEDGER_LIVE_GROUPS) return;\n", + " if (true) return;\n")], + "the number of groups holding their rows is bounded"), + + # The oldest goes first, which is the order the reader would choose. Sorting the other way + # releases the group the reader is on and keeps the one they left long ago. + ("the newest group is released instead of the oldest", + [(" live.sort(function (a, b) { return a.at - b.at; });\n", + " live.sort(function (a, b) { return b.at - a.at; });\n")], + "the oldest groups are the ones that go when the bound is passed"), + + # The height is the reader's place in the session. `contain-intrinsic-size` sizes a group + # the browser has skipped, and a group near the viewport is rendered — so without an + # explicit height the box collapses to its 39 px header and the document shrinks under + # the reader. + ("a released group loses the height it had", + [(" section.style.minHeight = Math.round(height) + 'px';\n", + " section.style.minHeight = '';\n")], + "a released group keeps the height it had"), + + # A group with no measured height gets no reserved height either. Writing 0px would be + # worse than writing nothing: a group sized 0px can never come back into the viewport, so + # it is never re-rendered, and the ledger stays collapsed for good. + ("a group with no height to keep is given one anyway", + [(" if (!(height > 0)) return;\n", + " if (false) return;\n")], + "a group with no height to keep is not given one"), + + # The rows come back when the reader reaches the group. This is the whole of what the + # reader is owed for having given them up. + ("the rows do not come back when the reader reaches the group", + [(" fillGroupRows(list, group);\n", "")], + "a group the reader reaches puts its rows back"), + + # The released state is what the reader sees while the group is empty. + ("the released state is left on after the rows come back", + [(" section.classList.remove('is-released');\n", "")], + ["a group that has its rows back drops the released state", + "the released state is turned off when the rows come back"]), + + # The reserved height is given back when the group has its rows again, or the group is + # taller than the rows it holds. + ("the reserved height is not given back after the rows come back", + [(" section.style.minHeight = '';\n", "")], + "a group that has its rows back gives its reserved height back"), + + # A group that is already holding its rows is filled by the build and then again by the + # observer, and the second pass appends every row a second time. + ("a group that is already holding its rows is filled a second time", + [(" if (section.getAttribute('data-rows') === 'live') return;\n", "")], + "a group that is already holding its rows is not filled a second time"), + + # The first group of a window is where the reader arrives, and every other group's height + # is predicted from it. If it does not hold its rows there is nothing to predict from. + ("the first group of a window does not hold its rows", + [(" if (isFirst) return true;\n", + " if (isFirst) return false;\n")], + "the first group of a window always holds its rows"), + + # The anchor is read out of the group the reader is on. A released group has no rows, and + # an anchor read from the next group down that still has rows is an anchor to the wrong + # place — the reader is moved to wherever that group starts. + ("the anchor is read without filling the group the reader is on", + [(" if (sections[s].getAttribute('data-rows') !== 'live') fillLedgerGroup(sections[s]);\n", + "")], + "the group the reader is on is filled before the anchor is read out of it"), + + # The build asks for no rows for a group it is not holding. This is the one that carries + # the whole memory win: asking for them and then hiding them is the same cost. + ("the build asks for the rows of every group", + [(" var bands = fillGroupRows(holds ? list : null, group);\n", + " var bands = fillGroupRows(list, group);\n")], + "the build asks for no rows for a group it is not holding"), + + # A group the build is not holding is given a height to keep. + ("a group the build is not holding is given no height to keep", + [(" holdLedgerHeight(section, known > 0 ? known : predicted);\n", "")], + "a group the build is not holding is given a height to keep"), + + # The two states are what the rest of the page reads: the build says which, and the + # observer fills the other. + ("a group the build is holding does not say so", + [(" section.setAttribute('data-rows', 'live');\n", "")], + "a group the build is holding says so"), + ("a group the build is not holding does not say so", + [(" section.setAttribute('data-rows', 'released');\n", "")], + "a group the build is not holding says so too"), + + # The released state is what the reader sees while the group is empty, so the rule that + # turns it on has to exist in the build and not only in the stylesheet. + ("the released state is never turned on", + [(" section.classList.add('is-released');\n", "")], + "the released state is turned on by the build"), + + # The bands of a released group are counted even though its rows are not built: its height + # is predicted from how many bands it has, and a prediction that ignored them would be + # short by every band in it. + ("the bands of a released group stop being counted", + [(" if (isBand) bands += 1;\n", + " if (isBand && list) bands += 1;\n")], + "the bands of a released group are still counted"), + + # A compaction record is a band, not a row, and the test that says so has to be the one the + # branch uses. A version that computes the test and then ignores it says the same word and + # renders the record as a row — which is why this break is in the branch rather than in the + # word. + ("a compaction record is rendered as a row instead of a band", + [(" if (isBand) {\n", " if (false) {\n")], + "a compaction record is rendered as a band, not as a row"), + + # The map that says which group each section stands for has to go with the sections. A + # section that is no longer in the table must not be filled when the observer reports it + # as in view — it would be filled with rows that belong to a table that no longer exists. + ("the ledger does not forget the groups it just emptied", + [(" ledgerContent.clear();\n\n if (state.loading) {\n" + " ledger.appendChild(buildSkeleton());\n", + " if (state.loading) {\n ledger.appendChild(buildSkeleton());\n")], + "the ledger drops what it knew about the groups it just emptied"), + + # A turn key means nothing across two sessions, so the groups of the session being left + # have to go with it. + ("the groups of the session being left are not forgotten", + [(" ledgerState.clear();\n ledgerContent.clear();\n", + " ledgerContent.clear();\n")], + "the groups of the session being left are forgotten with it"), +] + +original = io.open(CLIENT, encoding="utf-8", newline="").read() + + +def run(): + proc = subprocess.run(["node", PROBE], cwd=HERE, + capture_output=True, text=True, encoding="utf-8", errors="replace") + return proc.returncode, proc.stdout + proc.stderr + + +ok = True +code, baseline = run() +base_ok = code == 0 and "0 failed" in baseline +print(("ok " if base_ok else "FAIL ") + "baseline green") +ok = ok and base_ok + +caught = [] +for title, edits, expect in MUTATIONS: + for old, _ in edits: + hits = original.count(old) + if hits != 1: + raise SystemExit("FAIL anchor for %r matched %d times" % (title, hits)) + names = expect if isinstance(expect, list) else [expect] + try: + mutated = original + for old, new in edits: + mutated = mutated.replace(old, new, 1) + io.open(CLIENT, "w", encoding="utf-8", newline="").write(mutated) + during_code, during = run() + named = all(n in during for n in names) + red = during_code != 0 + # A guard that crashes is not a guard that caught anything: the rule has to be reported + # by name, or the mutation only proved that the file no longer parses. + print(("ok " if (named and red) else "FAIL ") + "guard catches: " + title) + if not named: + print(" no assertion reported %r" % (names,)) + for line in during.splitlines()[-4:]: + print(" " + line.strip()) + elif not red: + print(" the rule was reported but the guard still exited 0") + caught.append(named and red) + finally: + io.open(CLIENT, "w", encoding="utf-8", newline="").write(original) + +restored = io.open(CLIENT, encoding="utf-8", newline="").read() +after_code, after = run() +after_ok = after_code == 0 and "0 failed" in after +print(("ok " if restored == original else "FAIL ") + "client restored byte for byte") +print(("ok " if after_ok else "FAIL ") + "green again after restore") + +sys.exit(0 if (ok and all(caught) and restored == original and after_ok) else 1) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py index 409eb6ad..eca4f024 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py @@ -74,8 +74,8 @@ ), ( "the needle is sent without the generation, so a slice search answers about the whole lineage", - " + (generation === null ? '' : '&generation=' + encodeURIComponent(String(generation)));", - " + '';", + " + (generation === null ? '' : '&generation=' + encodeURIComponent(String(generation)))", + " + ''", ), ( "the hit set is built from the summaries rather than the indexes, so it can never match a row", @@ -181,17 +181,75 @@ ), ( "the raw read stops naming the version of the record it asks for", - " // apart instead of serving the newer record under the older one's number.\n" - " var version = displayedRecordVersion(index);\n" + " // the retry is a different question wearing the same question's clothes.\n" " if (version) url += '&v=' + encodeURIComponent(version);", - " // apart instead of serving the newer record under the older one's number.", + " // the retry is a different question wearing the same question's clothes.", ), ( "the record read stops naming the version of the record it asks for", - " // refuse when the position it would answer is a different record than the one opened.\n" - " var version = displayedRecordVersion(index);\n" + " // The same record, the same way: the version travels with the request so the service can\n" + " // refuse when the position it would answer is a different record than the one opened — and\n" + " // it is the record's own version, so a retry after a reload asks about the same record.\n" " if (version) url += '&v=' + encodeURIComponent(version);", - " // refuse when the position it would answer is a different record than the one opened.", + " // The same record, the same way: the version travels with the request so the service can\n" + " // refuse when the position it would answer is a different record than the one opened — and\n" + " // it is the record's own version, so a retry after a reload asks about the same record.", + ), + # ---- the version travels with the RECORD, not with the screen ------------------------ + # + # A read that comes back 「this session is not cached」 reloads the body and asks again. The + # body it reloads is a DIFFERENT one — the transcript moved while the tab was open — so a + # version read off the screen at that moment belongs to the NEW body's record at that index, + # and the retry is a different question wearing the same question's clothes. The service + # answers it, because it is a perfectly good question about a perfectly good record: the + # wrong one. + # + # So the version comes from the record the reader opened, and this control is the old way + # round: the version of whatever is at that index NOW. + ( + "a read takes the version of the record now on screen instead of the one the reader opened", + " function recordVersionOf(record) {\n" + " return record && typeof record.v === 'string' ? record.v : '';\n" + " }", + " function recordVersionOf(record) {\n" + " var flat = Array.isArray(state.flat) ? state.flat : [];\n" + " for (var i = 0; i < flat.length; i += 1) {\n" + " var entry = flat[i];\n" + " var shown = entry && entry.record ? entry.record : entry;\n" + " if (shown && shown.index === (record && record.index)) {\n" + " return typeof shown.v === 'string' ? shown.v : '';\n" + " }\n" + " }\n" + " return '';\n" + " }", + ), + # ---- the answer has to be about the SET the page is showing -------------------------- + # + # A search answer says 「these records match and the rest do not」, and the part of it that + # matters to the reader is the part it does NOT name: a record this body has and that body + # does not match gets dimmed as if it did not contain the needle, with nothing saying so. + # + # So the request says which set it is about — how many records the page has, and a + # fingerprint of their versions — and the service answers only if it searched THAT set. + ( + "the search stops saying which set of records it is about", + " + '&n=' + versions.length\n" + " + '&fp=' + encodeURIComponent(recordSetFingerprint(versions));", + " + '';", + ), + ( + "the search says how many records it is about, but not which ones", + " + '&n=' + versions.length\n" + " + '&fp=' + encodeURIComponent(recordSetFingerprint(versions));", + " + '&n=' + versions.length;", + ), + ( + "a refusal from the service is treated as an ordinary failure instead of asking again", + " if (error && error.code === 'search_version_mismatch') {\n" + " searchAnswerDoesNotApply();\n" + " return;\n" + " }", + "", ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py index 53959d55..a4b27862 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py @@ -119,8 +119,8 @@ ), ( "the artifact path reports how many it sent instead of how many matched", - " return { hits, total };", - " return { hits, total: hits.length };", + " return { hits, total, fingerprint: recordSetFingerprint(versions) };", + " return { hits, total: hits.length, fingerprint: recordSetFingerprint(versions) };", ), ( # The summary a hit answers with now lives on the segment line, so the mutation that swaps it diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py index 6980c659..08430d68 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py @@ -97,6 +97,26 @@ " if (typeof a.revision === 'string' && a.revision.length > 0\n && typeof b.revision === 'string' && b.revision.length > 0 && a.revision !== b.revision) {\n return false;\n }\n", "", ), + ( + # The one that was still wrong after the revision comparison was fixed: the ACTIVE file has + # no catalog entry, so it has no revision to compare, and a transcript rewritten in place + # keeps its length — so "ALPHA BODY" became "BRAVO BODY" and every comparison in the file + # said the same file. The mtime is the other half of this process's idea of a file's + # identity (the parsed-rows cache and the trajectory entry both key on (size, mtime)), so + # leaving it out is what let a segment built from the old bytes answer for the new ones. + "the mtime stops being compared, so a rewrite that kept the length is reused", + " if (a.mtimeMs !== b.mtimeMs) return false;\n", + "", + ), + ( + # The versions are read out of the lines by hand, so that a search over a big segment does + # not parse every line it does not need. A hand-reading is a second way to answer 「which + # version is this record」, and the two have to agree — the service folds them into the + # fingerprint that says whether an answer is about the body the page is showing. + "the versions of a segment's records are read as something else", + " if (tab >= 0 && tab <= end) out.push(versionInAnswer(surface.slice(start, tab)));", + " if (tab >= 0 && tab <= end) out.push(String(out.length));", + ), # ---- the rest of the contract --------------------------------------------------------- ( "the needle floor is dropped, so one character is a search", diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py index f9aa9a32..b047edf5 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py @@ -291,6 +291,44 @@ " session: { id: ctx.identity.id, generation: filter },\n", " session: { id: ctx.identity.id, generation: filter === null ? 0 : filter },\n", ), + # ---- the answer has to be about the SET the page is showing -------------------------- + # + # A search answer says 「these records match and the rest do not」, and the part of it that + # matters to the reader is the part it does NOT name: a record this body has and that body + # does not match gets dimmed as if it did not contain the needle, with nothing saying so. + # + # So the request says which set it is about — how many records the page has, and a fingerprint + # of their versions — and the service answers only if it searched THAT set. Three controls, + # because the promise has three halves: the set the page names has to be checked, the set the + # SEGMENTS stand for has to be checked too (the fast path is a different body by another + # route), and a request that names no set at all has to be refused rather than guessed at. + ( + "the search answers about a set the page did not name", + SERVER, + " if (recordSetFingerprint(payloadRecordVersions(payload, count)) !== askedFingerprint) {\n" + " sendError(response, 409, 'search_version_mismatch', '该记录所在的版本已变化，请重新加载会话');\n" + " return;\n" + " }\n", + "", + ), + ( + "the search never checks the set the stored segments stand for", + SERVER, + " if (onDisk.fingerprint !== askedFingerprint) {\n" + " sendError(response, 409, 'search_version_mismatch', '该记录所在的版本已变化，请重新加载会话');\n" + " return;\n" + " }\n", + "", + ), + ( + "a search that does not say which set it is about is answered anyway", + SERVER, + " if (!Number.isInteger(count) || count < 0 || !/^[0-9a-f]{16}$/.test(askedFingerprint)) {\n" + " sendError(response, 400, 'search_fingerprint_required', '会话已更新，请重新加载后再搜索');\n" + " return;\n" + " }\n", + "", + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs new file mode 100644 index 00000000..756f7e9f --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -0,0 +1,426 @@ +/** + * The ledger keeps its rows for the groups the reader is on and gives the rest up — and the + * promise that comes with giving them up is two-sided. + * + * The one side everybody can see: a group the reader is not on must not be holding its rows. + * Measured on a real session of 47,937 records, the rows of 「显示全部」 are 438,986 nodes and + * about 470 MB of the renderer's private memory, so "hold everything" is the whole cost. + * + * The other side is the one that is easy to lose: a group that has given up its rows must still + * be *there*. The rows are the DOM; the height is the reader's place in a session they have + * scrolled through, and it is what the scrollbar is describing. Two facts about the browser + * make this harder than it looks: + * + * - `contain-intrinsic-size` sizes an element the browser has SKIPPED. A group near the + * viewport is RENDERED, and a rendered group with no rows is 39 px tall — its header. So + * releasing the rows of the groups around the viewport took the scroll height down by + * 79,288 px, and releasing the viewport's own group took another 257,378 px. + * - The rows of a group come back when the reader reaches it, so the anchor the page restores + * the scroll position from has to be read out of the group the reader is actually on. Read + * from the next group down that still has rows, it is an anchor to the wrong place. + * + * So the rules here are: the rows of a group the reader is not on are not built; the height of + * that group is kept; the rows come back when the reader arrives; and the group that is live is + * bounded in number, which is what makes the memory bounded rather than merely reduced. + * + * These are asserted by running the shipped functions in a sandbox with a fake DOM, not by + * reading the source: a source-shaped check cannot tell a group that is released from one that + * is released *and keeps its height*, which is the difference between a smaller page and a + * broken one. + * + * mutate-ledger-lifecycle.py breaks each rule in the shipped file and requires this guard to + * report it by name, so a rule nothing tests cannot be added here. + */ +import { readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import vm from 'node:vm'; + +const HTML = fileURLToPath(new URL('../miniapp/client/index.html', import.meta.url)); +const html = readFileSync(HTML, 'utf8'); + +let pass = 0; +const failures = []; +function assert(name, cond) { + if (cond) pass += 1; + else failures.push(name); +} + +function functionSource(name) { + const at = html.indexOf('function ' + name + '('); + if (at < 0) return null; + let i = html.indexOf('{', at); + let depth = 0; + for (let j = i; j < html.length; j += 1) { + if (html[j] === '{') depth += 1; + else if (html[j] === '}') { + depth -= 1; + if (depth === 0) return html.slice(at, j + 1); + } + } + return null; +} + +/* ---------- the shipped pieces, lifted ---------- */ + +const LIFECYCLE_FNS = [ + 'ledgerStateOf', + 'ledgerHoldsRows', + 'releaseStaleGroups', + 'fillLedgerGroup', + 'holdLedgerHeight', + 'fillGroupRows', + 'firstVisibleRow' +]; + +const fnSrc = LIFECYCLE_FNS.map((name) => functionSource(name)); +const missing = LIFECYCLE_FNS.filter((_, i) => !fnSrc[i]); +if (missing.length) { + console.error(`[ledger-lifecycle] 0 passed, ${missing.length} failed`); + for (const m of missing) console.error(' FAIL the shipped script has no ' + m + '()'); + process.exit(1); +} + +// The two limits are declarations, not functions, so they are lifted on their own. They are +// numbers the shipped code reads, and the guard below uses the shipped values rather than +// numbers written here — a copy of the cap in this file would keep passing after the shipped +// one changed. +const idleDecl = html.match(/^[ \t]*var LEDGER_IDLE_MS = [^;]+;/m); +const capDecl = html.match(/^[ \t]*var LEDGER_LIVE_GROUPS = [^;]+;/m); +assert('the shipped script declares the idle timeout', !!idleDecl); +assert('the shipped script declares the cap on groups holding their rows', !!capDecl); +if (!idleDecl || !capDecl) { + console.error(`[ledger-lifecycle] ${pass} passed, ${failures.length} failed`); + for (const f of failures) console.error(' FAIL ' + f); + process.exit(1); +} + +/* ---------- a DOM small enough to reason about ---------- */ + +function makeEl(tag) { + const el = { + tagName: String(tag).toUpperCase(), + children: [], + attrs: Object.create(null), + style: {}, + classes: new Set(), + appendChild(child) { el.children.push(child); return child; }, + setAttribute(name, value) { el.attrs[name] = String(value); }, + getAttribute(name) { return name in el.attrs ? el.attrs[name] : null; }, + querySelector(selector) { return queryOne(el, selector); }, + getBoundingClientRect() { return { height: 240, width: 900, top: 0, bottom: 240 }; } + }; + el.classList = { + add(name) { el.classes.add(name); }, + remove(name) { el.classes.delete(name); }, + contains(name) { return el.classes.has(name); } + }; + return el; +} + +function queryOne(root, selector) { + const parts = selector.split('.'); + const tag = parts.shift(); + for (const child of root.children) { + if (!child || !child.children) continue; + const ok = (!tag || child.tagName === tag.toUpperCase()) + && parts.every((c) => child.classes.has(c)); + if (ok) return child; + const deeper = queryOne(child, selector); + if (deeper) return deeper; + } + return null; +} + +function entriesOf(count, bands, prefix) { + const entries = []; + for (let i = 0; i < count; i += 1) { + entries.push({ record: { kind: 'message', id: prefix + i } }); + } + for (let i = 0; i < bands; i += 1) { + entries.push({ record: { kind: 'compaction', id: prefix + 'c' + i } }); + } + return entries; +} + +/** + * A fresh sandbox with the shipped lifecycle code in it. + * + * The clock is a variable rather than the real Date, because the idle timeout is a rule about + * time and a test that has to wait fifteen seconds to check it is a test nobody runs. + */ +function harness() { + const box = { clock: 1000, built: [], sized: [] }; + const sandbox = { + console, + Math, + JSON, + Date: { now: () => box.clock }, + ledgerState: new Map(), + ledgerContent: new Map(), + el: (tag) => { const e = makeEl(tag); box.built.push(e); return e; }, + railStates: (entries) => entries.map(() => null), + buildRow: (entry) => { + const e = makeEl('div'); + e.classes.add('row'); + e.setAttribute('data-row-id', entry.record.id); + return e; + }, + buildCompactionBand: (entry) => { + const e = makeEl('div'); + e.classes.add('band'); + e.setAttribute('data-row-id', entry.record.id); + return e; + }, + sizeBuiltGroup: (section) => { box.sized.push(section); } + }; + vm.createContext(sandbox); + vm.runInContext( + idleDecl[0] + '\n' + capDecl[0] + '\n' + fnSrc.join('\n'), + sandbox + ); + box.sandbox = sandbox; + return box; +} + +/* ---------- 1. the rows of a group the reader is not on are not built ---------- */ + +{ + const h = harness(); + const s = h.sandbox; + const group = { key: 'g', turn: { index: 3 }, entries: entriesOf(5, 2, 'r') }; + const bands = s.fillGroupRows(null, group); + assert('a released group builds no rows', h.built.length === 0); + assert('the bands of a released group are still counted, because its height depends on them', + bands === 2); +} + +{ + const h = harness(); + const s = h.sandbox; + const list = makeEl('ul'); + const group = { key: 'g', turn: { index: 3 }, entries: entriesOf(5, 2, 'r') }; + const bands = s.fillGroupRows(list, group); + assert('a group that is holding its rows builds every one of them', list.children.length === 7); + assert('the bands are counted the same way either way', bands === 2); +} + +/* ---------- 1b. a compaction record is a band, not a row ---------- */ + +{ + const h = harness(); + const s = h.sandbox; + const list = makeEl('ul'); + const group = { key: 'g', turn: { index: 3 }, entries: entriesOf(2, 1, 'r') }; + s.fillGroupRows(list, group); + const isBand = (li) => li.children.some((n) => n.classes.has('band')); + const isRow = (li) => li.children.some((n) => n.classes.has('row')); + assert('a compaction record is rendered as a band, not as a row', + list.children.filter(isBand).length === 1 && list.children.filter(isRow).length === 2); +} + +/* ---------- 2. a released group keeps the height it had ---------- */ + +{ + const h = harness(); + const s = h.sandbox; + const section = makeEl('section'); + s.holdLedgerHeight(section, 3643); + assert('a released group keeps the height it had', section.style.minHeight === '3643px'); +} + +{ + const h = harness(); + const s = h.sandbox; + const section = makeEl('section'); + s.holdLedgerHeight(section, 0); + assert('a group with no height to keep is not given one', + section.style.minHeight === undefined); +} + +/* ---------- 3. a group the reader left gives its rows up, and the count is bounded ---------- */ + +{ + const h = harness(); + const s = h.sandbox; + s.ledgerState.set('a', { live: true, lastSeen: 1000, height: 0 }); + + h.clock = 1000 + s.LEDGER_IDLE_MS - 1; + s.releaseStaleGroups('somewhere-else'); + assert('a group the reader is still within reach of keeps its rows', + s.ledgerState.get('a').live === true); + + h.clock = 1000 + s.LEDGER_IDLE_MS; + s.releaseStaleGroups('somewhere-else'); + assert('a group the reader has left gives its rows up once the idle time has passed', + s.ledgerState.get('a').live === false); +} + +{ + const h = harness(); + const s = h.sandbox; + const n = s.LEDGER_LIVE_GROUPS + 2; + for (let i = 0; i < n; i += 1) { + s.ledgerState.set('g' + i, { live: true, lastSeen: 1000 + i, height: 0 }); + } + s.releaseStaleGroups('g' + (n - 1)); + const live = [...s.ledgerState.values()].filter((v) => v.live).length; + assert('the number of groups holding their rows is bounded', + live === s.LEDGER_LIVE_GROUPS); + assert('the oldest groups are the ones that go when the bound is passed', + s.ledgerState.get('g0').live === false + && s.ledgerState.get('g1').live === false + && s.ledgerState.get('g' + (n - 1)).live === true); +} + +{ + const h = harness(); + const s = h.sandbox; + assert('the first group of a window always holds its rows, because every height is predicted from it', + s.ledgerHoldsRows('never-seen', true) === true); + assert('a group nobody has been on does not hold its rows', + s.ledgerHoldsRows('never-seen', false) === false); +} + +/* ---------- 4. the rows come back when the reader reaches the group ---------- */ + +{ + const h = harness(); + const s = h.sandbox; + const section = makeEl('section'); + const list = makeEl('ul'); + list.classes.add('row-list'); + section.appendChild(list); + section.setAttribute('data-rows', 'released'); + section.classList.add('is-released'); + section.style.minHeight = '3643px'; + + const entries = entriesOf(4, 0, 'r'); + s.ledgerContent.set(section, { key: 'g1', turn: { index: 1 }, entries, bands: 0 }); + s.fillLedgerGroup(section); + + assert('a group the reader reaches puts its rows back', list.children.length === 4); + assert('a group that has its rows back says so', section.getAttribute('data-rows') === 'live'); + assert('a group that has its rows back drops the released state', + section.classList.contains('is-released') === false); + assert('a group that has its rows back gives its reserved height back', + section.style.minHeight === ''); + assert('a group that has its rows back is remembered as live', + s.ledgerState.get('g1').live === true); +} + +{ + const h = harness(); + const s = h.sandbox; + const section = makeEl('section'); + const list = makeEl('ul'); + list.classes.add('row-list'); + list.appendChild(makeEl('li')); + section.appendChild(list); + section.setAttribute('data-rows', 'live'); + + s.ledgerContent.set(section, { key: 'g2', turn: { index: 2 }, entries: entriesOf(4, 0, 'r'), bands: 0 }); + s.fillLedgerGroup(section); + assert('a group that is already holding its rows is not filled a second time', + list.children.length === 1); +} + +{ + const h = harness(); + const s = h.sandbox; + const section = makeEl('section'); + let threw = null; + try { s.fillLedgerGroup(section); } catch (err) { threw = err; } + assert('a section the ledger does not know about is left alone', + threw === null && section.children.length === 0); +} + +/* ---------- 5. the anchor is read out of the group the reader is on ---------- */ + +{ + const h = harness(); + const s = h.sandbox; + + const here = makeEl('section'); + const hereList = makeEl('ul'); + hereList.classes.add('row-list'); + here.appendChild(hereList); + here.setAttribute('data-rows', 'released'); + here.classList.add('is-released'); + here.getBoundingClientRect = () => ({ height: 240, width: 900, top: 0, bottom: 240 }); + + const below = makeEl('section'); + const belowList = makeEl('ul'); + belowList.classes.add('row-list'); + below.appendChild(belowList); + below.setAttribute('data-rows', 'live'); + below.getBoundingClientRect = () => ({ height: 240, width: 900, top: 240, bottom: 480 }); + s.fillGroupRows(belowList, { key: 'b', turn: { index: 9 }, entries: entriesOf(2, 0, 'b') }); + + s.ledgerContent.set(here, { key: 'h', turn: { index: 8 }, entries: entriesOf(2, 0, 'h'), bands: 0 }); + + const sections = [here, below]; + const ledger = makeEl('div'); + ledger.querySelectorAll = (selector) => { + if (selector === ':scope > section.turn-group') return sections; + if (selector === '[data-row-id]') { + const out = []; + for (const sec of sections) { + const list = sec.querySelector('ul.row-list'); + if (!list) continue; + for (const item of list.children) { + for (const node of item.children) { + if (node.getAttribute('data-row-id') !== null) out.push(node); + } + } + } + return out; + } + return []; + }; + s.dom = { ledger }; + + const anchor = s.firstVisibleRow(); + assert('the group the reader is on is filled before the anchor is read out of it', + hereList.children.length === 2); + assert('the anchor comes from the group the reader is on, not from the next one down', + !!anchor && String(anchor.id).startsWith('h')); +} + +/* ---------- 6. the shipped build asks for exactly this ---------- */ + +const buildStepSrc = functionSource('buildStep') || ''; +assert('the build asks for no rows for a group it is not holding', + /fillGroupRows\(holds \? list : null, group\)/.test(buildStepSrc)); +assert('a group the build is not holding is given a height to keep', + /if \(!holds\) \{[\s\S]{0,400}?holdLedgerHeight\(section,/.test(buildStepSrc)); +assert('a group the build is holding says so', + /setAttribute\('data-rows', 'live'\)/.test(buildStepSrc)); +assert('a group the build is not holding says so too', + /setAttribute\('data-rows', 'released'\)/.test(buildStepSrc)); + +const renderSrc = functionSource('renderLedger') || ''; +assert('the ledger drops what it knew about the groups it just emptied', + /clear\(ledger\);/.test(renderSrc) && /ledgerContent\.clear\(\);/.test(renderSrc)); + +const selectSrc = functionSource('selectSession') || ''; +assert('the groups of the session being left are forgotten with it', + /ledgerState\.clear\(\);/.test(selectSrc) && /ledgerContent\.clear\(\);/.test(selectSrc)); + +const styleEnd = html.indexOf(''); +const css = (styleEnd > 0 ? html.slice(0, styleEnd) : '').replace(/\/\*[\s\S]*?\*\//g, ''); +assert('the released state is something the reader can see', + /\.turn-group\.is-released/.test(css)); +assert('the released state is turned on by the build, so the rule is not dead', + /classList\.add\('is-released'\)/.test(buildStepSrc)); +assert('the released state is turned off when the rows come back', + /classList\.remove\('is-released'\)/.test(functionSource('fillLedgerGroup') || '')); + +/* ---------- report ---------- */ + +if (failures.length) { + console.error(`[ledger-lifecycle] ${pass} passed, ${failures.length} failed`); + for (const f of failures) console.error(' FAIL ' + f); + process.exit(1); +} +// "0 failed" on success is what mutate-ledger-lifecycle.py reads to decide the baseline is green. +console.log(`[ledger-lifecycle] ${pass} passed, 0 failed`); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs index 11323fac..0cf564a7 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs @@ -499,7 +499,25 @@ const artifactsSrc = SERVER.slice( assert('and the shipped artifact path counts every match rather than what it sent', /total \+= 1;/.test(artifactsSrc) && /if \(hits\.length < MAX_SEARCH_HITS\) hits\.push\(hit\);/.test(artifactsSrc) - && /return \{ hits, total \};/.test(artifactsSrc)); + && /return \{ hits, total, fingerprint: recordSetFingerprint\(versions\) \};/.test(artifactsSrc)); + +// The answer is about a SET of records, and the part of it that matters to the reader is the part +// it does NOT name. So the service has to be able to say which set it searched, and the page has +// to be able to check it: the fingerprint is folded over the records' own version tokens, over as +// many of them as the page asked about. +// +// The fold is over the FIRST N, and that is not a detail. A page that is behind because the +// session grew has fewer records, not different ones, so its first N are still the same N — and a +// body-level identity (an ETag, a digest of the whole body) would change on every append and +// make every answer stop applying every few seconds. +assert('the artifact path folds the versions of the records it covers', + /segmentVersions\(segment\.surface, count - versions\.length\)/.test(artifactsSrc) + && /const versions = \[\];/.test(artifactsSrc)); +assert('and the route refuses an answer that is not about the set the page asked about', + /search_version_mismatch/.test(SERVER) + && /recordSetFingerprint\(payloadRecordVersions\(payload, count\)\)/.test(SERVER)); +assert('and a request that does not say which set it is about is refused rather than guessed at', + /search_fingerprint_required/.test(SERVER)); // The route resolves a whole scan in ONE call, because resolving one ordinal at a time walks the // segment once per hit. That version is not a slower version of this one, it is the version that diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs index d0e61c3c..72a54f0a 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs @@ -42,6 +42,7 @@ import { readSegmentHits, sameSource, scanSegment, + segmentVersions, searchSurfaceOf, } from '../miniapp/node/search-segments.mjs'; @@ -478,6 +479,8 @@ try { (await store.read('s', 'all', 'g0', [{ ...SOURCE[0], size: 101 }])) === null); assert('a source whose revision changed refuses the segment', (await store.read('s', 'all', 'g0', [{ ...SOURCE[0], revision: 'sha256:' + 'f'.repeat(64) }])) === null); + assert('a source whose mtime changed refuses the segment, so a rewrite that kept the length is not reused', + (await store.read('s', 'all', 'g0', [{ ...SOURCE[0], mtimeMs: 999 }])) === null); assert('a renamed source refuses the segment', (await store.read('s', 'all', 'g0', [{ ...SOURCE[0], fileName: 'x.jsonl' }])) === null); assert('a source that went away refuses the segment', @@ -561,6 +564,53 @@ assert('the comparison is in this module and not only in the caller', /if \(\(a\.truncated === true\) !== \(b\.truncated === true\)\) return false;/.test( source.split('export function sameSource')[1] || '')); +/* --------------------------------------------------------------------- + * The source identity has to see a rewrite that kept the LENGTH. + * + * `size` is the whole file whether or not the read covered it, and a transcript rewritten in + * place keeps its length: "ALPHA BODY" becomes "BRAVO BODY" and the file is the same size. Every + * other part of this process treats (size, mtime) as a file's identity — the parsed-rows cache + * keys on it and the trajectory entry keys on it — so a file that was rewritten in place is a + * DIFFERENT file to everything else in here, and a segment built from the old bytes must not be + * served as if it were built from the new ones. + * + * The old wording of this rule said "identity, not mtime", because a sealed file is not written + * to and a reader that treated a touch as a change would rebuild correct bytes. That was true and + * it was the wrong question: the file that is written to is the active one, and it is exactly the + * one whose bytes can change under a size that did not. + * --------------------------------------------------------------------- */ +const ACTIVE_SOURCE = { fileName: 'messages.jsonl', generation: null, size: 500, mtimeMs: 1000, revision: null }; +check('an active file rewritten in place is not the same source', + sameSource([{ ...ACTIVE_SOURCE }], [{ ...ACTIVE_SOURCE, mtimeMs: 2000 }]), false); +check('and the same file untouched is', + sameSource([{ ...ACTIVE_SOURCE }], [{ ...ACTIVE_SOURCE }]), true); +check('and a rewrite that also changed the length is not either', + sameSource([{ ...ACTIVE_SOURCE }], [{ ...ACTIVE_SOURCE, size: 501, mtimeMs: 2000 }]), false); +check('and a sealed file is judged the same way, because a sealed file can be rewritten too', + sameSource([{ fileName: 'g1.jsonl', generation: 1, size: 500, mtimeMs: 1000, revision: 'r1' }], + [{ fileName: 'g1.jsonl', generation: 1, size: 500, mtimeMs: 2000, revision: 'r1' }]), false); + +/* --------------------------------------------------------------------- + * The versions of the records a segment covers, read back out of the lines themselves. + * + * The service folds them into the fingerprint that says whether an answer is about the body the + * page is showing, and the answer it compares against carries the same versions. Two readings of + * one thing would be two answers, so the reading that is done by hand is pinned against the one + * that parses the line. + * --------------------------------------------------------------------- */ +const versionedRecords = RECORDS.map((r, i) => ({ ...r, index: i, v: 'version-' + i })); +const versionedSurface = buildSegment({ records: versionedRecords }).surface; +check('the versions of a segment can be read back in order', + segmentVersions(versionedSurface, versionedRecords.length), + versionedRecords.map((r) => r.v)); +check('and only as many as were asked for', + segmentVersions(versionedSurface, 2), ['version-0', 'version-1']); +check('and the reading agrees with the answer the line carries', + segmentVersions(versionedSurface, versionedRecords.length), + readSegmentHits(versionedSurface, versionedRecords.map((_, i) => i)).map((h) => (h === null ? '' : h.v))); +check('and an empty surface has no versions', + segmentVersions('', 10), []); + /* --------------------------------------------------------------------- */ if (failures.length) { diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs index cd5904e8..34d2fbf0 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs @@ -17,16 +17,18 @@ * mutate-trajectory-cache.py breaks each of these in the shipped file. */ import { mkdirSync, mkdtempSync, readFileSync } from 'node:fs'; -import { mkdir, writeFile, utimes } from 'node:fs/promises'; +import { mkdir, readdir, readFile, writeFile, utimes } from 'node:fs/promises'; import { tmpdir } from 'node:os'; -import { join } from 'node:path'; +import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; import { createTrajectoryCache, + recordSetFingerprint, start, trajectoryCacheFamily, trajectoryCacheKey, } from '../miniapp/node/server.mjs'; +import { digestOf } from '../miniapp/node/search-segments.mjs'; const SERVER = readFileSync(new URL('../miniapp/node/server.mjs', import.meta.url), 'utf8'); /** The package root, so the route serves the same page the reader gets. */ @@ -444,8 +446,9 @@ try { // The version of one record of a body. A position read names an INDEX, and an index is a // position inside ONE body — so a read has to say which record of that body it means, and this - // is what it says. The page takes it from the body it is showing, which is the only body it - // has. + // is what it says. The page takes it from the RECORD the reader opened, which came from the body + // it is showing: the record is the one that survives a reload, so a read that is retried after + // one still asks about the same record. const versionOf = (body, index) => { const turns = body && Array.isArray(body.turns) ? body.turns : []; for (const turn of turns) { @@ -457,6 +460,47 @@ try { return ''; }; + // The SET of records a body is, as the page names it: the records' own versions, in the order + // the ledger shows them. The service folds its own records the same way, and an answer it + // cannot tie to this set is refused — an answer about another body says nothing about this + // body's omissions, and the rows it does not name are the ones the reader cannot see dimmed. + const versionsOf = (body) => { + const out = []; + const turns = body && Array.isArray(body.turns) ? body.turns : []; + for (const turn of turns) { + const records = Array.isArray(turn && turn.records) ? turn.records : []; + for (const record of records) { + out.push(record && typeof record.v === 'string' ? record.v : ''); + } + } + return out; + }; + + // The segments a search published, found by walking the data dir. Publishing them is a side + // effect that nobody awaits, so the test waits for the file rather than for a promise. + const waitForSegments = async (dir) => { + const walk = async (current, found) => { + let entries = []; + try { + entries = await readdir(current, { withFileTypes: true }); + } catch { + return found; + } + for (const entry of entries) { + const full = join(current, entry.name); + if (entry.isDirectory()) await walk(full, found); + else if (entry.name === 'surface.txt') found.push(full); + } + return found; + }; + for (let attempt = 0; attempt < 60; attempt += 1) { + const found = await walk(dir, []); + if (found.length > 0) return found; + await new Promise((done) => setTimeout(done, 50)); + } + return []; + }; + try { // 1 — the poll the page makes, and the read it makes right after it. const poll = await get('/api/trajectory?session=latest'); @@ -480,7 +524,9 @@ try { const alphaVersion = versionOf(poll.body, 1); assert('every record of the body carries a version', alphaVersion !== '', 'version of record 1: ' + JSON.stringify(alphaVersion)); - const alphaSearch = await get('/api/trajectory/search?session=' + REACH_ALPHA + '&q=ALPHA'); + const alphaVersions = versionsOf(poll.body); + const alphaSearch = await get('/api/trajectory/search?session=' + REACH_ALPHA + '&q=ALPHA' + + '&n=' + alphaVersions.length + '&fp=' + recordSetFingerprint(alphaVersions)); assert('the search answers with hits', alphaSearch.status === 200 && Array.isArray(alphaSearch.body && alphaSearch.body.hits) && alphaSearch.body.hits.length > 0, why(alphaSearch)); @@ -494,6 +540,63 @@ try { 'search: ' + JSON.stringify(alphaSearch.body && alphaSearch.body.session) + ' body: ' + JSON.stringify(poll.body && poll.body.session)); + // WHICH SET the answer is about. An answer says 「these match and the rest do not」, and the + // part that matters to the reader is the part it does NOT name: a record this body has and + // that body does not match would be dimmed as if it did not contain the needle, with nothing + // anywhere saying so. So the request names the set, and a set that is not this one is refused. + const otherSet = await get('/api/trajectory/search?session=' + REACH_ALPHA + '&q=ALPHA' + + '&n=' + alphaVersions.length + '&fp=' + recordSetFingerprint(['a-different-record'])); + assert('a search about a set the body on screen is not is refused', + otherSet.status === 409 && Boolean(otherSet.body) + && otherSet.body.error === 'search_version_mismatch', why(otherSet)); + const unnamedSet = await get('/api/trajectory/search?session=' + REACH_ALPHA + '&q=ALPHA'); + assert('and a search that does not say which set it is about is refused rather than guessed at', + unnamedSet.status === 400 && Boolean(unnamedSet.body) + && unnamedSet.body.error === 'search_fingerprint_required', why(unnamedSet)); + // The fold is over the FIRST N records, so a page that is behind — the session grew since its + // last poll — is still answered: its first N records are the same N, and the answer applies + // to them. A body-level identity would have refused this, and the refusal would have been the + // common case rather than the rare one. + const behind = await get('/api/trajectory/search?session=' + REACH_ALPHA + '&q=ALPHA' + + '&n=1&fp=' + recordSetFingerprint(alphaVersions.slice(0, 1))); + assert('a page that is behind is still answered, because its first records are the same ones', + behind.status === 200, why(behind)); + + // The same refusal, on the path that answers from the stored segments — and the PROOF that + // this path was the one taken. + // + // The segments are the cache the next search is answered from, and the answer is about the set + // THEY stand for. So the test edits the version on one of their lines and fixes the digest: + // the segment still matches its manifest, so it is still usable, and now the set it stands for + // is not the set the page is showing. A rebuild would have answered 200 — the records on disk + // are the ones the page has — so a 409 can only have come from the segments. And the other way + // round: naming the EDITED set is accepted, which a rebuild would have refused. Between the + // two, the path is not a guess. + const segments = await waitForSegments(home); + assert('the first search published its segments, so the next one is answered from them', + segments.length > 0, 'segments under ' + home); + if (segments.length > 0) { + const surfacePath = segments[0]; + const manifestPath = join(dirname(surfacePath), 'manifest.json'); + const surface = await readFile(surfacePath, 'utf8'); + const manifest = JSON.parse(await readFile(manifestPath, 'utf8')); + const edited = surface.replace(/"v":"[0-9a-f]{16}"/, '"v":"ffffffffffffffff"'); + await writeFile(surfacePath, edited); + manifest.surfaceDigest = digestOf(edited); + await writeFile(manifestPath, JSON.stringify(manifest)); + const editedVersions = alphaVersions.slice(); + editedVersions[0] = 'ffffffffffffffff'; + const viaSegments = await get('/api/trajectory/search?session=' + REACH_ALPHA + '&q=ALPHA' + + '&n=' + editedVersions.length + '&fp=' + recordSetFingerprint(editedVersions)); + assert('a search answered from the segments is accepted when the page names the set they stand for', + viaSegments.status === 200, why(viaSegments)); + const aboutTheOtherSet = await get('/api/trajectory/search?session=' + REACH_ALPHA + '&q=ALPHA' + + '&n=' + alphaVersions.length + '&fp=' + recordSetFingerprint(alphaVersions)); + assert('and refused when the page names a set they do not stand for', + aboutTheOtherSet.status === 409 && Boolean(aboutTheOtherSet.body) + && aboutTheOtherSet.body.error === 'search_version_mismatch', why(aboutTheOtherSet)); + } + const rawById = await get('/api/trajectory/raw?index=1&session=' + REACH_ALPHA + '&v=' + alphaVersion); assert('a record of the conversation on screen is readable by that conversation\'s id', rawById.status === 200, why(rawById)); From 1f296ed1ae0f020ebbc942452b097266d46e020d Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 14:05:48 +0800 Subject: [PATCH 35/95] An answer that is standing is retired when the records behind it change, and a read's key names the record it is about MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A body can change without changing its shape. The page decides whether anything moved by comparing counters — the message count, the turn count, the usage, the duration — because rebuilding the table every couple of seconds is wasteful when nothing changed. A record's text rewritten in place leaves every one of those counters exactly where it was, so the counters said that nothing moved while the records on screen were different ones. The answer standing on screen was about the old ones, and it says 「these records match and the rest do not」, so the part a reader notices is the part it does not name: a row that no longer contains the needle stayed lit up, a row that now contains it stayed dimmed, and nothing said so — no error, no pending state, the count line still read 「命中 1 条」. The request side already knew this. The search request names the set it is about — how many records the body on screen has, and a fingerprint of their versions — and the service refuses a search about another set. But a refusal only happens if a request is made, and on this path none is: the answer is not re-asked, it is left standing. So the page now records which set its answer was computed against, and puts the question again when a body arrives whose records differ. The set is recorded rather than recomputed, so a session that is merely being written to does not re-ask on every tick. The version of a record travelled with the request but not with the key that decides whose answer a caller gets. That key was built out of what is on screen — the body, the slice, the position — so after a reload it could not tell two reads apart: the read of the record the reader opened, and the read of the record that had taken its place at the same position, which the rebuilt panel starts on its own. The retry of the first found the second's pending entry and took its answer for its own, so the reader opened one record and the page handed back another's body. The key now carries the caller's version, which is the same version the request carries: a retried read issues its own request, and what it gets is the service's refusal of a version that does not match the record it would serve. --- .../mmc-trajectory/miniapp/client/index.html | 80 +++++++++++++++++-- 1 file changed, 73 insertions(+), 7 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index d2bdc54f..8041c49c 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -3820,6 +3820,17 @@

检视

searchRefresh: 0, // The needle `searchRefresh` belongs to. A new needle is a new question and starts over. searchRefreshFor: '', + // WHICH SET the search answer on screen is about: the session and the slice of the body it + // was computed against, and a fingerprint of that body's records. + // + // A body can change without changing its shape — a record's text rewritten under the page + // keeps every counter the same — so the shape of the body cannot say whether the answer + // still covers the records on screen. This does: it is the same set the request names, so + // the standing answer and the request cannot disagree about which records it is about. + // + // Written in exactly one place: `runSearch`, when the question is put. Read in exactly one + // place: `applyPayload`, which is the only writer of the body on screen. + searchSet: '', // Time range, in milliseconds since the session started — the same zero the ledger's // `+893 分 21 秒` column counts from. `null` on either side means that end is open, // and both null is the default: no range, whole session. @@ -4211,6 +4222,25 @@

检视

return out; } + /** + * WHICH SET the page's search answer is about. + * + * The session and the slice say WHICH BODY; the fingerprint says WHICH RECORDS of it. A body + * can change without changing its shape — a record's text rewritten under the page keeps every + * counter the same — so a comparison of counters cannot see that the records behind an answer + * are not the ones it was computed about. This is the same set the search request names, so + * the standing answer and the request cannot disagree about which records it covers. + * + * @param {Array} [versions] the versions to fold, when the caller already has them + * @returns {string} + */ + function displayedRecordSet(versions) { + var list = Array.isArray(versions) ? versions : displayedRecordVersions(); + return displayedSessionParam() + '#' + + (displayedGeneration() === null ? 'all' : String(displayedGeneration())) + + '#' + recordSetFingerprint(list); + } + /** * The version of one record, as the record itself carries it. * @@ -4506,6 +4536,9 @@

检视

state.searchTruncated = false; state.searchRefresh = 0; state.searchRefreshFor = ''; + // No question, no set. An answer about a set that is no longer on screen would filter + // rows by a question nobody is asking. + state.searchSet = ''; searchSeq += 1; return; } @@ -4532,6 +4565,10 @@

检视

// So the request says which set it is about — how many records the page has, and a // fingerprint of their versions — and the service answers only if it searched THAT set. var versions = displayedRecordVersions(); + // The set this question is about, recorded before it is asked. The body on screen can be + // replaced while the answer is in flight, and a poll that finds the set has moved has to + // know that the answer it is waiting for is about the old one. + state.searchSet = displayedRecordSet(versions); var url = '/api/trajectory/search?session=' + session + '&q=' + encodeURIComponent(asked) + (generation === null ? '' : '&generation=' + encodeURIComponent(String(generation))) @@ -4857,7 +4894,20 @@

检视

// the old answer would show the reader rows that never matched. Re-running it here is the // whole of the rule; doing it at the reader would leave the poll path stale, which is the // same mistake the window pinning above used to make. - if (changed && state.query) runSearch(state.query); + // + // WHICH SET, and not only which shape. A body can change without changing its shape: a + // record's text rewritten under the page keeps every counter the same, so `changed` is + // false and the answer standing on screen is still about the old text — it would light up + // a row whose text does not contain the needle, and nothing anywhere would say so. The + // set the answer is about is the set of record versions it was computed against, which is + // the same set the request names; so the standing answer and the request cannot disagree + // about which records it covers. + // + // The set is only compared once a question has been asked (`state.searchSet` is empty + // until then), and a question that was never put has no answer to retire. + if (state.query && (changed || (state.searchSet && displayedRecordSet() !== state.searchSet))) { + runSearch(state.query); + } render(); if (manual) announce('已刷新，正在显示最新数据。'); } @@ -9236,7 +9286,7 @@

检视

var rawCache = {}; var rawPending = {}; - function rawCacheKey(index) { + function rawCacheKey(index, version) { var payload = state.payload; var session = payload && payload.session ? payload.session : null; var sid = session && typeof session.id === 'string' ? session.id : ''; @@ -9248,7 +9298,15 @@

检视

// of one body: the same index in the next version of the session is a different message. // Without the tag, a body that changed since the raw was read kept being served from here // forever, and the page had no way to notice. Pinned by tests/verify-raw-lazy.mjs. - return sid + '#' + gen + '#' + (state.etag || '') + '#' + index; + // + // And the RECORD's own version belongs in it too. This key is what decides whose answer a + // caller gets, and it used to be built out of what is on screen — the validator, the slice, + // the position — and none of those say which RECORD is being read. So a read that is + // retried after a reload found the pending read of the record that had taken its place + // (the rebuild of the panel starts one) and took its answer for its own: the reader + // opened ALPHA and the page handed back CHARLIE. The version is the caller's, not the + // screen's — the same rule the request itself follows. + return sid + '#' + gen + '#' + (state.etag || '') + '#' + index + '#' + (version || ''); } function requestRaw(index, version) { @@ -9294,10 +9352,15 @@

检视

function fetchRaw(record, retried) { var index = record && record.index; if (!isCount(index) || index < 1) return Promise.reject(new Error('record_index_missing')); - var key = rawCacheKey(index); + // The version of the record the READER opened, read off that record and not off the + // screen — and it decides the key, because the key is what says whose answer a caller + // gets. A version read off the screen at this moment would belong to the record that has + // taken this one's place, and the retry would be answered with it. + var version = recordVersionOf(record); + var key = rawCacheKey(index, version); if (Object.prototype.hasOwnProperty.call(rawCache, key)) return Promise.resolve(rawCache[key]); if (rawPending[key]) return rawPending[key]; - var attempt = requestRaw(index, recordVersionOf(record)).then(function (result) { + var attempt = requestRaw(index, version).then(function (result) { delete rawPending[key]; if (result.ok && result.body && Object.prototype.hasOwnProperty.call(result.body, 'raw')) { var entry = { raw: result.body.raw, omitted: result.body.omitted === true }; @@ -9384,10 +9447,13 @@

检视

function fetchDetail(record, retried) { var index = record && record.index; if (!isCount(index) || index < 1) return Promise.reject(new Error('record_index_missing')); - var key = rawCacheKey(index); + // The same version, and for the same reason: the key is the same key, and it decides + // whose answer a caller gets. + var version = recordVersionOf(record); + var key = rawCacheKey(index, version); if (Object.prototype.hasOwnProperty.call(detailCache, key)) return Promise.resolve(detailCache[key]); if (detailPending[key]) return detailPending[key]; - var attempt = requestDetail(index, recordVersionOf(record)).then(function (result) { + var attempt = requestDetail(index, version).then(function (result) { delete detailPending[key]; if (result.ok && result.body && typeof result.body.index === 'number') { var entry = { From 0cee0b37f6be8a1d5033bc0de9b210267ec0be57 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 14:06:02 +0800 Subject: [PATCH 36/95] The guards that own the two rules, and the controls that break them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two rules were standing without an owner, and the way that showed up was a check that could not fail. The rule that a standing search answer has to be retired when the records behind it change was owned by an assertion that looked for the text of a condition — `if (changed && state.query)` — rather than for the behaviour the condition stands for. A record rewritten in place keeps every counter, so `changed` was false, the condition was never reached, and the assertion was green about a rule that was broken. So the shape check is gone and two behavioural ones have taken its place. The first drives the shipped `applyPayload` with a body whose records changed while every counter stayed the same: the question standing on screen must be put again, a body that did not change must not put it again, and a body that did not change must not put it again on the polls that follow either — so a check that retires the answer on every tick cannot pass as a check. The second lets the panel that rebuilds on a reloaded body start its own read of the record now at that position, while the read of the record the reader opened is still in flight: the retried read must not take the other record's pending answer, must not be answered with the other record's body, and must have asked about the record the reader opened on a request of its own — for the raw read and for the field read, which share the key. The controls follow the rules. One undoes the retirement of a standing answer when the records change without the shape changing; one takes the record's version out of the key, so a retried read joins a newer record's pending read. Both are caught by name. Two anchors that had stopped resolving were re-anchored rather than dropped: the condition that re-asks the search after a changed payload, and the key that carries the validator — an anchor that no longer resolves stops the control before the mutations after it are run at all, which is a worse failure than a red one. --- .../mmc-trajectory/tests/client-check.mjs | 162 +++++++++++++++++- .../tests/mutate-raw-locators.py | 4 +- .../tests/mutate-search-client.py | 24 ++- .../tests/verify-raw-locators.mjs | 7 +- .../tests/verify-record-detail.mjs | 8 +- 5 files changed, 190 insertions(+), 15 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index f2df8f44..0fa05b12 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -233,7 +233,8 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); const needed = ['function isCount(value)', 'function formatInt(value)', 'function searchHitSet(hits)', 'function sessionParam()', 'function displayedSessionParam()', 'function displayedGeneration()', 'function displayedRecordsByIndex()', 'function displayedRecordVersions()', - 'function recordSetFingerprint(versions)', 'function searchStatusLabel(searchState)', + 'function displayedRecordSet(versions)', 'function recordSetFingerprint(versions)', + 'function searchStatusLabel(searchState)', 'function searchAnswerDoesNotApply()', 'function runSearch(needle)']; const sources = needed.map((sig) => fnSource(sig)); check('the shipped search functions are all still findable by signature', @@ -245,6 +246,7 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); 'var state = { query: "retry me", sessionId: null, hostSessionId: null, followLatest: true,', ' generation: null, searchHits: null, searchTotal: 0, searchTruncated: false,', ' searchPending: false, searchError: null, searchRefresh: 0, searchRefreshFor: "",', + ' searchSet: "",', // The body on screen. `displayedSessionParam()` reads it and the answer is checked against // it, so a harness with no body on screen cannot ask either question. Its `generation` is // the slice the body IS: null is the whole lineage, a number is that one generation. @@ -548,6 +550,96 @@ check('no kinds match', sandbox.hasKind({ kind: 'system' }, ['user']), false); netCtx.searchStatusLabel(netCtx.state), '搜索失败，以下为上次结果'); } +/* The answer has to be about the records that are ON SCREEN — and a body can change without + * changing its shape. + * + * The change detector is a cheap one, on purpose: it compares structural counters, because + * rebuilding 200+ rows every 2.5s is wasteful when nothing changed. But a record's TEXT can be + * rewritten under the page while every one of those counters stays the same — the same message + * count, the same turn count, the same timestamps, the same usage. The body on screen is then a + * different SET of records, and the answer standing on screen is about the old ones: the reader + * is told 「命中 1 条」 about a row that does not contain the needle, and nothing anywhere says + * so. + * + * The request side already knows this — it names the set it is asking about, and the service + * refuses a search about another set. But a refusal only happens if a REQUEST is made, and here + * none is: the answer is not re-asked, it is simply left standing. + * + * So this runs the real `applyPayload` with a body that was rewritten in place, and asks what + * the page does about it. */ +{ + const applyCtx = { console, AbortController, setTimeout, clearTimeout, encodeURIComponent }; + vm.createContext(applyCtx); + const needed = ['function isCount(value)', 'function signatureOf(payload)', 'function flattenTurns(turns)', + 'function payloadCursor(payload)', 'function deltaMatchesNumbering(previous, delta)', + 'function recordSetFingerprint(versions)', 'function displayedRecordVersions()', + 'function displayedRecordSet(versions)', 'function displayedSessionParam()', + 'function displayedGeneration()', 'function applyPayload(body, manual)']; + const sources = needed.map((sig) => fnSource(sig)); + check('the shipped functions the answer-set rule needs are all still findable by signature', + needed.filter((sig, i) => !sources[i]), []); + vm.runInContext([ + 'var turnById = new Map();', + 'var state = { payload: null, flat: [], signature: "", loading: false, hardError: null,', + ' since: null, sinceFor: null, etag: "", etagFor: "", query: "", searchSet: "" };', + // The re-ask is what is under test, so it is recorded rather than performed — and the set is + // recorded the way the shipped `runSearch` records it, because the page keeps answering + // about the set it last asked about. + 'var asked = [];', + 'function runSearch(needle) { asked.push(String(needle)); state.searchSet = displayedRecordSet(); }', + // The page paints itself; none of that is what is under test and none of it has a DOM here. + 'function render() {}', + 'function announce() {}', + 'function noteFresh() {}', + 'function rePinWindow() {}', + ...sources.map((src, i) => src || (needed[i] + ' { throw new Error(' + JSON.stringify(needed[i]) + + ' is not in index.html any more); }')), + ].join('\n'), applyCtx); + + // One record, and the body that holds it. The SHAPE is the same in every one of these bodies: + // the same session, the same counters, the same turn, the same one record. Only the record's own + // version token moves — and that is exactly what a rewrite in place looks like from here. + const bodyWith = (version, stats) => ({ + session: { id: 'mvs_displayed', generation: null }, + stats: Object.assign({ messages: 1, toolCalls: 0, errors: 0, durationMs: 1000, + tokens: { totalTokens: 10 } }, stats || {}), + turns: [{ id: 'turn-1', records: [{ index: 1, v: version, summary: 'ALPHA BODY' }] }] + }); + + // The body on screen, and a question that was answered for it. + const alpha = bodyWith('the-version-of-alpha'); + applyCtx.state.payload = alpha; + applyCtx.state.flat = applyCtx.flattenTurns(alpha.turns); + applyCtx.state.signature = applyCtx.signatureOf(alpha); + applyCtx.state.query = 'alpha'; + applyCtx.state.searchSet = applyCtx.displayedRecordSet(); + applyCtx.asked.length = 0; + + // The same body again: nothing moved, so the question is not put again. + applyCtx.applyPayload(bodyWith('the-version-of-alpha'), false); + check('a body that did not change does not put the question again', applyCtx.asked, []); + + // The record's text is rewritten in place. Every counter is the same; only the record's own + // version token is different, so the body on screen is a different SET of records. + applyCtx.applyPayload(bodyWith('the-version-of-charlie'), false); + check('a content-only payload change retires the previous search answer', + applyCtx.asked, ['alpha']); + + // And it is retired once, not on every poll afterwards. + applyCtx.asked.length = 0; + applyCtx.applyPayload(bodyWith('the-version-of-charlie'), false); + check('and the question is not put again on every poll after that', + applyCtx.asked, []); + + // The rule that was already there: a body whose SHAPE changed puts the question again, even + // when its records are the same ones. A change of counters is a change of body, and the answer + // on screen was computed against the old one. + applyCtx.asked.length = 0; + applyCtx.applyPayload(bodyWith('the-version-of-charlie', { durationMs: 2000 }), false); + check('a payload that changed re-runs the search rather than filtering by the old answer', + applyCtx.asked, ['alpha']); +} + /* handoff: the session the record came from, across a navigation. * * The handoff reads two things asynchronously — the record's raw message and its four bulk @@ -1363,11 +1455,6 @@ const checks = [ const gates = src.match(/if \(seq !== searchSeq\) return;/g) || []; return gates.length === 2 && /var seq = \(searchSeq \+= 1\);/.test(src); })()], - ['a payload that changed re-runs the search rather than filtering by the old answer', (() => { - const src = fnSource('function applyPayload(body, manual)'); - if (!src) return false; - return /if \(changed && state\.query\) runSearch\(state\.query\);/.test(src); - })()], ['clearing the filters drops the answer with them', (() => { const src = fnSource('function resetBaseFilters()'); if (!src) return false; @@ -2104,7 +2191,7 @@ const requestDetailSrc = fnSource('function requestDetail(index, version)'); const displayedSessionSrc = fnSource('function displayedSessionParam()'); const displayedGenerationSrc = fnSource('function displayedGeneration()'); const recordVersionOfSrc = fnSource('function recordVersionOf(record)'); -const rawCacheKeySrc = fnSource('function rawCacheKey(index)'); +const rawCacheKeySrc = fnSource('function rawCacheKey(index, version)'); if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGenerationSrc || !recordVersionOfSrc || !rawCacheKeySrc) { failures.push('secondary reads: could not lift requestRaw, requestDetail and the session they ask for'); @@ -2248,6 +2335,67 @@ if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGen /[?&]v=the-version-of-record-one(&|$)/.test(retried[3] || ''), true); check('and not about the record that happens to be at that index after the reload', /[?&]v=a-newer-version(&|$)/.test(retried[3] || ''), false); + + /* The retry must not JOIN another read. + * + * The reload brings a body in which the record at that position is a DIFFERENT one, and the + * panel that rebuilds on it asks for the record now on screen. The read of the record the + * reader opened is still in flight, and it has to ask about ITS OWN record: the two reads are + * at the same position, and the key that decides whose answer a caller gets is the only + * thing that can tell them apart. + * + * The key used to be built out of what is on screen — the validator, the slice, the position — + * so after the reload the retry of the old record and the new record's own read had the SAME + * key. The retry was then answered with the new record's body without asking anything: the + * reader opened ALPHA and the page handed back CHARLIE. The version belongs in the key, and + * it is the caller's version, not the screen's. */ + for (const [readName, what] of [['fetchRaw', 'a retried read'], + ['fetchDetail', 'a retried field read']]) { + vm.runInContext('rawCache = {}; rawPending = {}; detailCache = {}; detailPending = {};', sandbox); + const joined = []; + sandbox.state = { + payload: { session: { id: 'mvs_displayed', generation: null } }, + flat: [{ turn: {}, record: { index: 1, v: 'the-version-of-record-one' } }], + generation: null, etag: '"tag-old"', followLatest: true, sessionId: '', hostSessionId: '' + }; + sandbox.fetch = function (url) { + joined.push(String(url)); + // The first attempt is refused because the service let go of the session. + if (joined.length === 1) { + return Promise.resolve({ ok: false, status: 503, + json: () => Promise.resolve({ error: 'trajectory_not_cached' }) }); + } + // The old record's own request: the service refuses it, because the record at that + // position is not the one that was asked for. + if (/[?&]v=the-version-of-record-one(&|$)/.test(joined[joined.length - 1])) { + return Promise.resolve({ ok: false, status: 409, + json: () => Promise.resolve({ error: 'raw_stale' }) }); + } + // The record now on screen, which the rebuild of the panel asks for. + return Promise.resolve({ ok: true, status: 200, + json: () => Promise.resolve({ raw: 'CHARLIE BODY longer', index: 1, text: 'CHARLIE BODY longer' }) }); + }; + sandbox.fetchTrajectory = function () { + // The new body is on screen, with its own validator, and the panel rebuilds on it: that + // starts a read of the record now at that position. + sandbox.state.etag = '"tag-new"'; + sandbox.state.flat = [{ turn: {}, record: { index: 1, v: 'the-version-of-record-two' } }]; + sandbox.fetchRaw({ index: 1, v: 'the-version-of-record-two' }); + return Promise.resolve(); + }; + let refused = null; + let handed = null; + try { + handed = await sandbox[readName]({ index: 1, v: 'the-version-of-record-one' }); + } catch (err) { + refused = err && err.code ? err.code : String(err); + } + check(what + ' cannot join a newer record\'s pending read after reload', + refused, 'raw_stale'); + check(what + ' is not answered with the other record\'s body', handed, null); + check(what + ' asked about the record the reader opened, on a request of its own', + joined.filter((u) => /[?&]v=the-version-of-record-one(&|$)/.test(u)).length, 2); + } } } diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-raw-locators.py b/plugins/avatasia/mmc-trajectory/tests/mutate-raw-locators.py index ff474010..5f6e915d 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-raw-locators.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-raw-locators.py @@ -159,8 +159,8 @@ ( "client: the page's raw cache key drops the validator, so a moved-on session keeps its old messages", CLIENT, - " return sid + '#' + gen + '#' + (state.etag || '') + '#' + index;", - " return sid + '#' + gen + '#' + index;", + " return sid + '#' + gen + '#' + (state.etag || '') + '#' + index + '#' + (version || '');", + " return sid + '#' + gen + '#' + index + '#' + (version || '');", ), ( "client: a refused raw is reloaded and retried, turning a hint into a reload the reader never asked for", diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py index eca4f024..2af5251b 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py @@ -8,7 +8,11 @@ - telling 「no answer yet」 apart from 「matched nothing」; - letting only the newest request write, since the answer now arrives out of order; - re-asking when the payload changed, since the answer names indexes of the payload that was - there before. + there before; + - re-asking when the RECORDS changed but the shape did not, since the answer names a set of + records and not only a set of indexes — a record rewritten in place keeps every counter; + - keying a secondary read by the record's own version, so a retry after a reload cannot take + the pending answer of the record that took its place. Each is a change the page renders differently and none of them throws. """ @@ -54,8 +58,22 @@ ), ( "a changed payload keeps the old hit set, so the new table is filtered by the old answer", - " if (changed && state.query) runSearch(state.query);", - "", + " if (state.query && (changed || (state.searchSet && displayedRecordSet() !== state.searchSet))) {\n" + " runSearch(state.query);\n" + " }", + " if (state.query && (state.searchSet && displayedRecordSet() !== state.searchSet)) {\n" + " runSearch(state.query);\n" + " }", + ), + ( + "a content-only payload change does not retire the previous search answer", + " if (state.query && (changed || (state.searchSet && displayedRecordSet() !== state.searchSet))) {", + " if (state.query && changed) {", + ), + ( + "a read joins a newer record's pending read after reload, because the key does not carry the record's version", + " return sid + '#' + gen + '#' + (state.etag || '') + '#' + index + '#' + (version || '');", + " return sid + '#' + gen + '#' + (state.etag || '') + '#' + index;", ), ( "clearing the filters leaves the hit set behind, so the next query filters by a stale answer", diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-raw-locators.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-raw-locators.mjs index 67785eb9..e0fee559 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-raw-locators.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-raw-locators.mjs @@ -273,7 +273,7 @@ assert('the decode cache is per entry, so a new body starts empty', * key the page keeps showing the previous version's message for the same record. * ===================================================================== */ -const rawCacheKeyAt = CLIENT.indexOf('function rawCacheKey(index)'); +const rawCacheKeyAt = CLIENT.indexOf('function rawCacheKey(index, version)'); assert('rawCacheKey exists', rawCacheKeyAt > 0); // The whole function, not a fixed number of characters. A window of N characters stops being // the function the moment a comment grows, and the assertion then fails on the edit rather @@ -284,6 +284,11 @@ const keyBody = rawCacheKeyAt >= 0 && keyEnd > rawCacheKeyAt : ''; assert("the page's raw cache key includes the validator", /state\.etag/.test(keyBody)); assert('and still the session and the index', keyBody.includes("sid + '#' + gen + '#'")); +// The key says whose answer a caller gets, so it has to say which RECORD is being read: the +// validator and the position do not, and a read that was retried after a reload found the pending +// read of the record that had taken its place. +assert("and the record's own version, so a read cannot take another record's answer", + keyBody.includes("(version || '')")); // A raw the server refuses is a hint, not a broken tab, so there is deliberately no second chance // for it: `raw_stale` stays out of the reload-and-retry path and lands on the same line every // other raw failure uses. What must not happen is the opposite — a stale answer being kept. diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-record-detail.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-record-detail.mjs index 46b2d57e..4d70fb1b 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-record-detail.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-record-detail.mjs @@ -432,9 +432,13 @@ assert('and the page never asks for the duplicate field any more', !/record\.error\b/.test(CLIENT)); // The detail read is a second source of the answer, so it has to be per record and invalidated // with the same key the raw read uses — a cached value from a previous version of the body is the -// exact failure `/raw` already had one guard for. +// exact failure `/raw` already had one guard for. The key carries the record's own version, so +// the two paths have to build it the same way; the assertion compares the two calls instead of +// naming one of them, so a change to the key's arguments cannot pass on one path alone. +const keyCalls = CLIENT.match(/var key = rawCacheKey\([^)]*\);/g) || []; assert('the detail cache is keyed like the raw one', - CLIENT.includes('var key = rawCacheKey(index);') && CLIENT.includes('var detailCache = {};')); + keyCalls.length >= 2 && keyCalls.every((call) => call === keyCalls[0]) + && CLIENT.includes('var detailCache = {};')); assert('and a failed detail read is reported rather than drawn as empty', CLIENT.includes('读取失败，这条记录的详细内容暂时取不到。')); From c27557d1695576b5647f2ee6c4e1ca39564a9a27 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 19:02:44 +0800 Subject: [PATCH 37/95] A rebuild does not keep a group the reader has left alive, and a tail is answered only for the records the page holds MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rebuilding an off-screen group renewed that group's reader-idle lease, so a group the reader had already left stayed alive and went on holding its rows. The build step no longer writes the lease, and the group the window is anchored on is counted against the same ceiling as the groups the reader is on, so the ceiling means the same thing whichever group happens to be anchored. A tail request asked for "everything after the cursor" without saying which records the page was holding, and the service answered with whatever it had. The page now states the record set it holds; the service refuses a tail it cannot answer for exactly that set and names the refusal; the page recognises that refusal as one and only as one, and reloads the whole body rather than show an answer that does not belong to the copy on screen. A refusal is only produced on the path that rebuilds the body. Every refusal makes the page fetch the whole body again, which costs about 41 MB for the sessions measured here; that cost is accepted in this change rather than hidden. The comment in the shipped client that records what holding every row of every group in the DOM costs now carries one measurement instead of several. In a session of about 50,600 records the page holds 14,801 DOM nodes, and the renderer's private memory read between 199 MB and 268 MB, measured on 2026-10-22 at the middle of the session. The node count repeats across runs; the memory figure does not, so it is written as a range — the same build read 194.1 MB and 269.8 MB at the other two positions of that afternoon, a spread of 39% — and these are the page's numbers: the service's own memory was measured separately, later, and is not reduced by this change. --- .../mmc-trajectory/miniapp/client/index.html | 130 +++++++++++++++--- .../mmc-trajectory/miniapp/node/server.mjs | 61 ++++++++ 2 files changed, 169 insertions(+), 22 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 8041c49c..0e32aa01 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -4241,6 +4241,39 @@

检视

+ '#' + recordSetFingerprint(list); } + /** + * The set of records the page holds, as a request states it. + * + * The same question the search request asks, and for the same reason: the answer is about a + * SET of records, and a body can change without changing its shape — a record rewritten in + * place keeps every counter the same, so how many records the page has says nothing about + * whether its copies are still the service's. The versions do. + * + * @returns {{ count: number, fingerprint: string }} + */ + function heldRecordSet() { + var versions = displayedRecordVersions(); + return { count: versions.length, fingerprint: recordSetFingerprint(versions) }; + } + + /** + * The query a tail is asked with: the cursor, and the set the page holds. + * + * The cursor says how far the page got; the set says which records that is. A tail asked + * for without the set is a tail the service cannot answer without guessing: the records + * before the cursor are not in the tail, so the service would have no way of knowing + * whether the page's copies of them are still its own — and a record rewritten in place + * keeps its index and its id, so nothing else would say so either. + * + * @returns {string} + */ + function tailQuery() { + var held = heldRecordSet(); + return '&since=' + encodeURIComponent(String(state.since)) + + '&n=' + held.count + + '&fp=' + encodeURIComponent(held.fingerprint); + } + /** * The version of one record, as the record itself carries it. * @@ -4383,8 +4416,12 @@

检视

// and the delta only ever arrives when the session really did grow. A manual refresh // drops the cursor as well as the validator: it is asking for the truth from the // server, and a tail is not that. + // + // The tail says which set the page holds as well, because the records before the + // cursor are not in the tail: the service can only answer one if it knows that the + // page's copies of those records are still its own. + (!manual && state.sinceFor === cacheKey && state.since > 0 - ? '&since=' + encodeURIComponent(String(state.since)) : ''); + ? tailQuery() : ''); var headers = { Accept: 'application/json' }; // A manual refresh must always reach the server: drop the validator so the // response cannot be revalidated into a 304 the user did not ask for. @@ -4464,6 +4501,15 @@

检视

if (seq !== refreshSeq) return; inflight = null; if (error && error.name === 'AbortError') return; + // The service will not answer a tail to a page whose copy is not the one it has, and + // it will not answer a tail asked for without saying which set the page holds. + // Either way the page's copy cannot be laid on top of what comes back, so the only + // answer left is the whole body — asked for again, without the cursor and without + // the validator that belonged to the copy that is gone. + if (isStaleCopyRefusal(error)) { + reloadWholeBody(); + return; + } handleTrajectoryError(error); }); } @@ -4846,6 +4892,38 @@

检视

return false; } + /** + * The copy on screen is not the one the service has: the whole body is asked for again. + * + * The cursor goes with it — a cursor into a copy that no longer exists — and so does the + * validator, which described that copy. What comes back is the whole body, which is the + * one answer that cannot be laid on top of a stale copy. + */ + function reloadWholeBody() { + state.since = null; + state.sinceFor = null; + state.loading = true; + render(); + fetchTrajectory(true); + } + + /** + * Whether the service's answer is a refusal to answer a tail to this page's copy. + * + * Two refusals, and both of them mean the same thing: the records the page holds are not + * the ones the service has — either because they have changed under it, or because the + * request never said which set it was holding. Neither is a failed poll: the request was + * answered, and the answer is that it cannot be answered with a tail. + * + * @param {any} error + * @returns {boolean} + */ + function isStaleCopyRefusal(error) { + if (!error || typeof error.code !== 'string') return false; + return error.code === 'trajectory_version_mismatch' + || error.code === 'trajectory_fingerprint_required'; + } + function applyPayload(body, manual) { // A delta is only ever laid onto a payload this page actually has. Without one — // first load, a session switch, a manual refresh — the whole payload is the answer. @@ -4854,11 +4932,7 @@

检视

// BEFORE the merge and returns early, because the alternative — merging it and noticing // afterwards — leaves records in the wrong turn with nothing to undo them. if (hasDelta && !deltaMatchesNumbering(state.payload, body)) { - state.since = null; - state.sinceFor = null; - state.loading = true; - render(); - fetchTrajectory(true); + reloadWholeBody(); return; } var mergeable = hasDelta; @@ -7125,25 +7199,33 @@

检视

/** * Decide which groups give up their rows. * - * Two rules, and the order matters: a group the reader has left goes when the idle - * timeout has passed, and when more groups are live than the cap allows, the oldest goes - * first. The second is what bounds the memory. + * Three rules, and the order matters. A group the reader has left goes when the idle + * timeout has passed; when more groups are live than the cap allows, the oldest goes + * first; and the group the window is anchored on is none of the reader's visits at all, + * so it is counted rather than timed. + * + * That last one is what keeps the bound about the rows on screen. The anchored group + * holds its rows whatever the reader is doing — it is where the reader arrives, and every + * other group's height is predicted from it — so a bound on the reader's groups alone + * would leave one group holding its rows outside the count. Counting it here makes the + * two sets the same set: the groups that are live are the groups that hold their rows. * * This only decides. The rows themselves are taken away by the next build, which is the * one place that writes them, so there is no second writer of the rows and nothing to keep * in step. * - * @param {string} keepKey the group the reader is on, which is never released + * @param {string} keepKey the group the window is anchored on, which is never released */ function releaseStaleGroups(keepKey) { var now = Date.now(); var live = []; + // The anchored group is counted first and with the current time, so it is never the + // oldest one the cap reaches for. Its lease is not consulted: it does not have one. + ledgerStateOf(keepKey).live = true; + live.push({ key: keepKey, at: now }); ledgerState.forEach(function (state, key) { + if (key === keepKey) return; if (!state.live) return; - if (key === keepKey) { - live.push({ key: key, at: state.lastSeen }); - return; - } if (now - state.lastSeen >= LEDGER_IDLE_MS) { state.live = false; return; @@ -7554,19 +7636,23 @@

检视

// that hides anything, so what is on screen is exactly what was selected. // // The rows are built for the groups the reader is on and released for the rest. What - // the reader gives up is the DOM, which is the whole cost: measured on a session of - // 47,937 records, the rows of 「显示全部」 are 438,986 nodes and about 470 MB of the - // renderer's private memory. What the reader does not give up is the height — a - // released group keeps the box it had, so the document under them does not change - // size when they scroll. + // the reader gives up is the DOM, which is the whole cost: measured on one session of + // about 50,600 records (2026-10-22), the rows of 「显示全部」 were 463,122 DOM nodes and + // the renderer's private memory read 785 to 820 MB; with only the groups the reader is + // on holding their rows, the same session measured 14,801 nodes and 199 to 268 MB. + // Those are measurements of one session, not bounds. What the reader does not give up + // is the height — a released group keeps the box it had, so the document under them + // does not change size when they scroll. var key = turnKey(group.turn); + // The rebuild is not a visit. The poll runs every eight seconds whether the reader + // is there or not, so a build that stamped the lease would renew it forever: the + // idle timeout could never pass, and the group the reader had left would keep its + // row DOM for the rest of the session. The lease belongs to the observer, which + // only speaks when the reader is actually at the group. var holds = ledgerHoldsRows(key, build.at === 1); var bands = fillGroupRows(holds ? list : null, group); if (holds) { section.setAttribute('data-rows', 'live'); - var held = ledgerStateOf(key); - held.live = true; - held.lastSeen = Date.now(); } else { section.setAttribute('data-rows', 'released'); section.classList.add('is-released'); diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 597b53ae..f71424a7 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -483,6 +483,50 @@ export function payloadRecordVersions(payload, limit) { return out; } +/** + * Whether a tail may be answered to a page that holds a particular set of records. + * + * A tail carries the records AFTER the page's cursor. The records before it are not in the tail + * at all — the page keeps the copies it has — so the tail is only about the page's copy if that + * copy is still the service's. + * + * A record rewritten in place keeps its index and its id, so the cursor still points at the same + * record, the tail looks empty, and the page merges nothing: it keeps the old text and installs + * the new body's validator. Nothing else in the protocol can see it, which is why the tail names + * the set it is about. + * + * So the page says which set it holds — how many records, and a fingerprint of their versions — + * and the service folds its own first N the same way. The two have to agree before a tail is + * answered with a tail. The count is the FIRST N for the reason the search has it: a page that is + * behind because the session grew has fewer records, not different ones. + * + * `null` means the tail may be answered. Otherwise the refusal, with the status and the code the + * page answers it by. + * + * @param {{ turns?: Array }} payload the body the tail would be cut from + * @param {number} count how many records the page holds + * @param {string} askedFingerprint the fingerprint of those records' versions + * @returns {{ status: number, code: string, message: string } | null} + */ +export function deltaRefusal(payload, count, askedFingerprint) { + const fingerprint = typeof askedFingerprint === 'string' ? askedFingerprint.trim() : ''; + if (!Number.isInteger(count) || count < 0 || !/^[0-9a-f]{16}$/.test(fingerprint)) { + return { + status: 400, + code: 'trajectory_fingerprint_required', + message: '会话已更新，请重新加载后再继续', + }; + } + if (recordSetFingerprint(payloadRecordVersions(payload, count)) !== fingerprint) { + return { + status: 409, + code: 'trajectory_version_mismatch', + message: '该记录所在的版本已变化，请重新加载会话', + }; + } + return null; +} + /** * Cheap single-pass counters used by the session list; values are approximate by design. * @@ -4543,6 +4587,23 @@ export async function start(context) { // and not the server's work. Any reading of "41 MB → a few KB, so polling got faster" // has to carry that with it. Pinned by tests/verify-incremental.mjs. const since = parseSince(url.searchParams.get('since')); + if (since !== null) { + // WHICH SET the tail is about. The tail carries the records after the cursor; the ones + // before it stay as the page has them, so the tail is only about the page's copy if that + // copy is still this body's. A record rewritten in place keeps its index and its id, so + // the cursor still points at the same record and the tail looks empty — the page would + // keep the old text under the new body's validator, and nothing else in the protocol + // could see that it had changed. + const refusal = deltaRefusal( + payload, + Number.parseInt(url.searchParams.get('n') ?? '', 10), + url.searchParams.get('fp') ?? '' + ); + if (refusal) { + sendError(response, refusal.status, refusal.code, refusal.message); + return; + } + } const delta = since === null ? null : buildDeltaPayload(payload, since); sendJsonBody(response, 200, delta === null ? body : JSON.stringify(delta), { etag, revalidate: true }); }; From cf8dae27546e532f6d47cd335c95e43cc3acc007 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 19:02:44 +0800 Subject: [PATCH 38/95] The guards that own the rebuild's lease and the tail's record set The two rules that changed in the previous commit now have guards that ask about the behaviour rather than about the shape of the source, and each of those guards has a mutation control that breaks the rule and has to be reported by name. The guard that owns the shipped files no longer names them. It used to carry a hand-written list of eight files, and the six files that list had left out were passed over without saying so; the list is now derived from the package, the derivation checks that it did not come out empty, and a control that disables the derivation is caught by name. The set it derives matches the files the package ships, one for one. Two controls were reporting something other than what they measured. One looked for a hardcoded size anywhere in the page, so a measurement of "268 MB" in a comment counted as one; the criterion now asks for a number that starts where a number starts, which still catches a hardcoded size and no longer catches a measurement. The other decided that a probe had crashed by its exit code, but a guard that correctly fails exits non-zero, so every mutation was reported as a crash; a probe that reports the rule it found has not crashed, and a probe that reports nothing has. That control also had five mutations anchored on a file they did not touch, and one pair of mutations listed twice. The suite is green in full: 101 of 101 controls pass, and the package is restored byte for byte afterwards. The guards and controls named above are 41 and 23 in the ledger's own case, 92 and 29 for the incremental one, 100 and 32 for the trajectory cache, 50 and 54 for the shipped-file one, and 26 and 11 for the page-size one. --- .../mmc-trajectory/tests/client-check.mjs | 7 +- .../tests/mutate-incremental.py | 63 +++++-- .../tests/mutate-ledger-lifecycle.py | 38 +++- .../mmc-trajectory/tests/mutate-pr15-fixes.py | 91 ++++++++-- .../mmc-trajectory/tests/mutate-review16.py | 69 +++++++- .../tests/mutate-trajectory-cache.py | 30 ++++ .../tests/verify-incremental.mjs | 137 ++++++++++++++- .../tests/verify-ledger-lifecycle.mjs | 166 +++++++++++++++++- .../tests/verify-pr15-fixes.mjs | 10 +- .../mmc-trajectory/tests/verify-review16.mjs | 62 +++++-- .../tests/verify-trajectory-cache.mjs | 123 ++++++++++++- 11 files changed, 738 insertions(+), 58 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs index 0fa05b12..82ce653d 100644 --- a/plugins/avatasia/mmc-trajectory/tests/client-check.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/client-check.mjs @@ -2378,9 +2378,14 @@ if (!requestRawSrc || !requestDetailSrc || !displayedSessionSrc || !displayedGen sandbox.fetchTrajectory = function () { // The new body is on screen, with its own validator, and the panel rebuilds on it: that // starts a read of the record now at that position. + // + // THE SAME KIND OF READ as the one under test. A field read that is retried has to be + // tested against a COMPETING FIELD READ: the two live in different pending maps, so a + // raw read competing with a field read would leave the field's own map empty, and the + // rule — a retry must not join another read — would never have been asked of it. sandbox.state.etag = '"tag-new"'; sandbox.state.flat = [{ turn: {}, record: { index: 1, v: 'the-version-of-record-two' } }]; - sandbox.fetchRaw({ index: 1, v: 'the-version-of-record-two' }); + sandbox[readName]({ index: 1, v: 'the-version-of-record-two' }); return Promise.resolve(); }; let refused = null; diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-incremental.py b/plugins/avatasia/mmc-trajectory/tests/mutate-incremental.py index 6fe9d36b..9a4f0e02 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-incremental.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-incremental.py @@ -60,6 +60,22 @@ "the cursor after a delta is the server upTo, not a second walk", "state.since = mergeable ? body.delta.upTo : payloadCursor(payload);", "state.since = payloadCursor(payload);"), + # The set the tail is about, from the page's side. A record rewritten in place keeps its index + # and its id, so nothing else in the protocol can see that the page's copy is not the + # service's — the tail comes back empty and the page keeps the old text under the new body's + # validator. The service's half of this is in SERVER_CASES, because it edits server.mjs; a + # case that edits one file and sits in the other's list is a case whose anchor never resolves. + ("a tail is asked for without saying which set the page holds", + "a tail is asked for with the cursor and the set the page holds", + " + '&n=' + held.count\n", ""), + ("the page treats every failure as a refusal about its copy", + "a refusal about the page's copy is recognised as one, and only as one", + " return error.code === 'trajectory_version_mismatch'\n" + " || error.code === 'trajectory_fingerprint_required';\n", + " return Boolean(error);\n"), + ("the page asks for the whole body for something that is not a refusal", + "a tail the service will not answer leaves the page asking for the whole body", + " if (isStaleCopyRefusal(error)) {\n", " if (false) {\n"), ] SERVER_CASES = [ @@ -115,6 +131,17 @@ ("a whole turn ships and the count inherits the whole turn instead of the additions", "while still counting only what is new", " records += fresh;", " records += all.length;"), + + # The set the tail is about. A record rewritten in place keeps its index and its id, so + # nothing else in the protocol can see that the page's copy is not the service's: the tail + # comes back empty, and the page keeps the old text under the new body's validator. + ("a tail is answered to a page whose copy is not the one the service has", + "a content-only rewrite at the cursor cannot be acknowledged without replacing the displayed record", + "export function deltaRefusal(payload, count, askedFingerprint) {\n", + "export function deltaRefusal(payload, count, askedFingerprint) {\n if (payload) return null;\n"), + ("the refusal is computed and then ignored", + "the service refuses a tail whose set the page did not state, and says so", + " if (refusal) {\n", " if (false) {\n"), ] # The page side. A delta replayed against a copy that already has a record must be allowed to @@ -133,20 +160,31 @@ def run_probe(): ["node", str(PROBE)], cwd=str(ROOT), capture_output=True, text=True, encoding="utf-8", errors="replace", ) + lines = proc.stdout.splitlines() fired = [] - for line in proc.stdout.splitlines(): + for line in lines: if line.startswith("FAIL "): fired.append(line[5:].strip()) - # A crash is the probe failing to run, which the exit code says. It is *not* "no assertion - # fired" — this guard prints its FAIL lines only when there are any, so a passing probe - # prints none, and conflating the two turns every surviving mutation into a fake crash. - return proc.returncode == 0, fired, proc.stdout + proc.stderr + # Two facts, and they are not the same one. + # + # The exit code says whether the guard PASSED — and a guard that fails an assertion on purpose + # exits non-zero, so a non-zero code is what a CATCH looks like. Reading it as "the probe + # crashed" turns every catch into a crash, which is a control that reports nothing but crashes. + # + # Whether the probe REPORTED is the other fact. This guard prints its summary line whether it + # passed or failed, and a probe that died before its reporting block printed neither the + # summary nor any FAIL line. A run that produced no output contains no name, so a crash cannot + # be read as a catch. + # + # The verdict is the name. The exit code only says whether the baseline was clean. + reported = any("checks passed" in line for line in lines) + return proc.returncode, reported, fired, proc.stdout + proc.stderr def main(): originals = {CLIENT: CLIENT.read_bytes(), SERVER: SERVER.read_bytes()} - ok, fired, output = run_probe() - if not ok or fired: + code, reported, fired, output = run_probe() + if code != 0 or not reported or fired: print("baseline verify-incremental.mjs did not pass; refusing to run mutations") print(output) return 1 @@ -165,15 +203,18 @@ def main(): target.write_bytes(text.replace(old, new, 1).encode("utf-8")) try: - passed, fired, _ = run_probe() + code, reported, fired, _ = run_probe() finally: target.write_bytes(originals[target]) - if not ok: - failures.append("%s: CRASHED — the probe could not run; that is not a catch" % name) + # A probe that did not report died before it could name anything, and a run that produced + # no output contains no name — so neither of those is a catch. A probe that reported and + # named nothing failed nothing, which is a different thing: the rule still holds. + if not reported: + failures.append("%s: CRASHED — the probe did not report; that is not a catch" % name) continue if not fired: - failures.append("%s: NOT CAUGHT — probe still passed" % name) + failures.append("%s: NOT CAUGHT — the probe reported and passed anyway" % name) continue if expected not in fired: failures.append("%s: caught by %s, expected %r" % (name, fired, expected)) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py index f2704f7f..65220fd8 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -4,8 +4,8 @@ The lifecycle has two halves that pull in opposite directions, and a version of this that keeps one and loses the other is worse than not doing it at all: - - a group the reader is not on must not be holding its rows, or 47,937 records are 438,986 - nodes and about 470 MB of the renderer's private memory; + - a group the reader is not on must not be holding its rows, or one session of about 50,600 + records is 463,122 DOM nodes and 785 to 820 MB of the renderer's private memory; - a group that has given up its rows must still be the height it was, or the document under the reader loses everything they have scrolled past — measured at 79,288 px for the groups around the viewport and 257,378 px more for the viewport's own group. @@ -35,12 +35,40 @@ # owns more than one rule: removing the class removal breaks both the state the reader sees # and the rule that says the state is turned off. MUTATIONS = [ + # The build is not a visit. The poll runs every eight seconds whether the reader is there + # or not, so a build that stamps the reader-idle lease renews it forever: the fifteen + # second idle timeout can never pass, and the group the reader left keeps its row DOM for + # the rest of the session. + ("a rebuild renews the reader-idle lease of a group the reader has left", + [(" var holds = ledgerHoldsRows(key, build.at === 1);\n", + " var holds = ledgerHoldsRows(key, build.at === 1);\n" + " ledgerStateOf(key).lastSeen = Date.now();\n")], + ["rebuilding an offscreen group does not renew its reader-idle lease", + "the build does not write the reader-idle lease"]), + + # The window is anchored on one group, and that group holds its rows whatever the reader + # is doing — so it has to be counted among the groups that hold their rows. Counted with + # its own lease instead of the current time, a group the reader left long ago is the + # oldest one there is, and the cap reaches for it first. + ("the window's first group is released with the others", + [(" live.push({ key: keepKey, at: now });\n", + " live.push({ key: keepKey, at: ledgerStateOf(keepKey).lastSeen });\n")], + "the window's first group is one of the groups that hold their rows"), + + # The same group left out of the count altogether: it goes on holding its rows past the + # bound, so the bound is about one set of groups while the rows on screen are another. + ("the window's first group is not counted among the groups that hold their rows", + [(" ledgerStateOf(keepKey).live = true;\n" + " live.push({ key: keepKey, at: now });\n", + "")], + "the window's first group is one of the groups that hold their rows"), + # The idle timeout is what releases a group the reader has walked away from. Without it a # group stays live for as long as the session is open, and the bound below becomes the only - # thing standing between the reader and 470 MB. + # thing standing between the reader and 785 MB. ("a group the reader has left never gives its rows up", - [(" if (now - state.lastSeen >= LEDGER_IDLE_MS) {\n", - " if (false) {\n")], + [(" if (now - state.lastSeen >= LEDGER_IDLE_MS) {\n", + " if (false) {\n")], "a group the reader has left gives its rows up once the idle time has passed"), # The bound is what makes the memory bounded rather than merely reduced: a reader who diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-pr15-fixes.py b/plugins/avatasia/mmc-trajectory/tests/mutate-pr15-fixes.py index 3e5b3c7f..3ca1a18f 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-pr15-fixes.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-pr15-fixes.py @@ -1,8 +1,10 @@ """Mutation control for verify-pr15-fixes.mjs. Each mutation reintroduces exactly one bug the probe is supposed to catch. The probe must FAIL -every time, and the file must come back byte-for-byte identical afterwards. A mutation that the -probe still passes means the assertion was tautological. +every time, it must fail **on the assertion that owns the bug** -- a probe that fails for some +other reason has not shown that the rule is being tested -- and the file must come back +byte-for-byte identical afterwards. A mutation that the probe still passes means the assertion was +tautological. """ import subprocess import sys @@ -14,6 +16,10 @@ CLIENT = ROOT / "miniapp" / "client" / "index.html" PROBE = HERE / "verify-pr15-fixes.mjs" +# Each entry is (label, file, text to replace, what replaces it, the assertion that owns the bug). +# The last field is not decoration: a control that only asks "did the probe fail" is satisfied by a +# probe that fails for a reason nobody asked about, and the probe here fails on several assertions +# at once more than once. So the failure has to NAME the rule. MUTATIONS = [ ( # Only the "offered" half of the original label is still observable. The count is built @@ -27,24 +33,32 @@ " .map((entry) => ({ ...entry, logicalMessageCount: logicalPerGeneration.get(entry.generation) ?? 0 }));", " const generations = (Array.isArray(input.generations) ? input.generations : [])\n" " .map((entry) => ({ ...entry, logicalMessageCount: logicalPerGeneration.get(entry.generation) ?? 0 }));", + "the picker never offers the forked generation", ), ( "server: match orphans by generation number instead of file name", SERVER, " const orphanFiles = new Set(orphans.map((orphan) => orphan.fileName));", " const orphanFiles = new Set(orphans.map((orphan) => String(orphan.generation)));", + # This one is caught by the picker assertion rather than by the by-file rule: the set now + # carries generation numbers, so the filter — which still compares file names — no longer + # recognises the fork, and the picker offers it. The by-file rule itself needs both halves + # changed at once, which is what the last mutation in this list does. + "the picker never offers the forked generation", ), ( "server: truncation fires on any read that fell short of the file", SERVER, " return size > limit;", " return size >= limit - 1;", + "a file exactly at the cap is not", ), ( "server: truncation fires on any read that fell short of the file (strict)", SERVER, " return size > limit;", " return size !== limit;", + "the reported file is not truncation", ), ( # The verdict now lives where the parse result is built rather than at the point of @@ -54,30 +68,63 @@ SERVER, "truncated: truncatedByCap(head.size, limit),", "truncated: head.read < head.size,", + "the route actually calls the tested helper", ), ( "server: catalog-recorded size as the read limit again", SERVER, " const limit = MAX_MESSAGE_BYTES;", " const limit = Math.min(entry.sizeBytes, identity.catalogBytes ?? MAX_MESSAGE_BYTES, MAX_MESSAGE_BYTES);", + "no catalog-recorded size is used as a read limit any more", ), ( "server: probe reads a truncated head budget again", SERVER, "const head = await readHead(filePath, GENERATION_PROBE_BYTES);", "const head = await readHead(filePath, 512);", + "the probe reads a fixed head, not a catalog-derived budget", ), ( "server: stop sending the cap to the page", SERVER, " maxMessageBytes: MAX_MESSAGE_BYTES,\n", "", + "the server sends the cap it applied", + ), + ( + # The by-file rule is a two-part thing — the orphan set has to carry file names, and the + # filter has to compare file names — and either half alone leaves the counter-example + # standing: a set of generation numbers is not compared against anything, and a filter + # that compares generations against a set of file names removes nothing. So this mutation + # changes the half that can be changed on its own: the filter now also drops a generation + # that an orphan record NAMES BY NUMBER, and the counter-example — an orphan whose record + # says generation 2 while naming a different file — loses generation 2 with it. + "server: a generation an orphan names by number is dropped as well", + SERVER, + " .filter((entry) => !orphanFiles.has(entry.fileName))\n", + " .filter((entry) => !orphanFiles.has(entry.fileName)\n" + " && !orphans.some((orphan) => String(orphan.generation) === String(entry.generation)))\n", + "orphans are matched by file, not by generation number", ), ( "client: hardcode the number instead of rendering the cap", CLIENT, " dom.footerTruncated.textContent = '文件过大，仅显示前 ' + formatBytes(session.maxMessageBytes) + '。';", " dom.footerTruncated.textContent = '文件过大，仅显示前 8 MB。';", + "no hardcoded 8 MB survives in the markup", + ), + ( + # The footer is not the only place a size can be written down, and the rule is not about one + # string — it is about a number of its own standing where the page should be formatting the + # number it was sent. This one puts the same hardcoded cap in the session-volume line, so + # the assertion has to fire ON ITS OWN: the footer mutation above breaks a second + # assertion with it, and a control that only ever broke that one would never have asked + # whether this assertion can fail by itself. + "client: hardcode a size in the session-volume line too", + CLIENT, + " '会话体积 ' + formatBytes(state.payload.session ? state.payload.session.sizeBytes : null)", + " '会话体积 8 MB'", + "no hardcoded 8 MB survives in the markup", ), ] @@ -107,6 +154,29 @@ def run_probe(): return result.returncode, result.stdout + result.stderr +def failure_names(output): + """The assertions the probe named, in the order it named them. + + The probe prints one ` - ` line per failed assertion. Two of the two kinds of check + print differently — an `assert` prints the name alone, a `check` prints the name and then what + it got and what it wanted — so the name is the START of the line, and that is what is matched. + + A run that produced no output contains no name, so a probe that crashed cannot be read as a + catch — which is why the name is the verdict here rather than the exit code. + """ + names = [] + for line in output.splitlines(): + stripped = line.strip() + if stripped.startswith("- "): + names.append(stripped[2:].strip()) + return names + + +def named(names, expect): + """Whether the probe named this rule. The name may carry what it got and what it wanted.""" + return any(name.startswith(expect) for name in names) + + def main(): code, output = run_probe() if code != 0: @@ -117,7 +187,7 @@ def main(): originals = {path: path.read_bytes() for path in {m[1] for m in MUTATIONS}} missed = [] - for label, path, old, new in MUTATIONS: + for label, path, old, new, expect in MUTATIONS: text, newline = read_exact(path) if old not in text: print(f"SKIP {label}: anchor not found in {path.name}") @@ -128,15 +198,16 @@ def main(): code, output = run_probe() finally: path.write_bytes(originals[path]) + names = failure_names(output) if code == 0: print(f"MISSED {label}: the probe still passed") missed.append(label) + elif not named(names, expect): + print(f"MISSED {label}: the probe failed, but not on 「{expect}」") + print(" it failed on: " + ("; ".join(names) if names else "(nothing was named)")) + missed.append(label) else: - first = next( - (line.strip() for line in output.splitlines() if line.strip().startswith("-")), - "(no detail)", - ) - print(f"caught {label}\n -> {first}") + print(f"caught {label}\n -> {expect}") for path, data in originals.items(): if path.read_bytes() != data: @@ -147,9 +218,9 @@ def main(): if missed: print(f"\n{len(missed)} mutation(s) got through") return 1 - print(f"\nall {len(MUTATIONS)} mutations caught") + print(f"\nall {len(MUTATIONS)} mutations caught, each by name") return 0 if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file + sys.exit(main()) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py b/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py index a6adfd0e..53cd4e36 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py @@ -172,6 +172,65 @@ " * read rather than re-measured here: band is L2 and overview is L3, and", "no shipped file carries a private defect or layer number"), + # ---- the same three checks, over the six files the hand-written table had left out ---- + # + # The guard derives its shipped-text set from the package now, and this block is what makes that + # derivation a claim rather than a hope. Each of the six files the old table missed gets a + # defect planted in it and the guard has to name it -- otherwise "the set is derived" is a + # sentence, and the hole is the same hole one level down: the three checks above could have + # gone green with the defect fully back in any of these six files. + # + # The defects are spread across all three checks on purpose. Proving coverage with one check + # would leave the other two free to have quietly stopped reading the new files, which is the + # same failure the old table had. + # + # The anchor of each one is a string that occurs exactly once in its own file and in none of the + # others this control searches, so the mutation cannot land somewhere it was not aimed at. + ("the package manifest points at the document that left the package", + '"miniApp": "./miniapp/miniapp.json"', + '"miniApp": "./miniapp/miniapp.json",\n "docs": "docs/LAYERS.md"', + "nothing still points at a docs/ file that does not ship"), + + # chr(0) rather than an escape, so the byte is written literally instead of being re-read as + # text -- the same reason the server-source mutation above does it. + ("a NUL byte ships inside the mini app manifest", + '"lifecycle": "on-demand"', + '"lifecycle": "on-' + chr(0) + 'demand"', + "no shipped source file carries a NUL byte"), + + ("a private defect number comes back into the shipped type declarations", + "export type HostConnectorErrorDisposition =", + "export type HostConnectorErrorDisposition = // D8", + "no shipped file carries a private defect or layer number"), + + # The three configuration files that ship. The defect planted in each is a pointer to a + # document that does not ship -- the class the check exists for -- and the point of the three + # is coverage rather than plausibility: the licence and the two dotfiles were the three + # shipped text files no check had ever read. + ("the licence points at the document that left the package", + "Copyright (c) 2026 avatasia", + "Copyright (c) 2026 avatasia\n\nAdditional warranty terms: docs/STAGE1-SPEC.md", + "nothing still points at a docs/ file that does not ship"), + + ("the line-ending rules point at the document that left the package", + "*.png binary", + "*.png binary\n\n# the reasoning is in docs/LAYERS.md", + "nothing still points at a docs/ file that does not ship"), + + ("the ignore rules point at the document that left the package", + "trajectory-artifacts/", + "trajectory-artifacts/\n\n# see docs/LAYERS.md", + "nothing still points at a docs/ file that does not ship"), + + # And the deriver itself. If the walk stops returning anything -- a skip list that grew a + # name, a directory that moved -- all three checks read an empty set and none of them fires, + # which is a guard that cannot fail rather than a guard that passed. The self-check is what + # catches that, so it needs a mutation of its own or it is an assertion nobody can break. + ("the deriver stops deriving anything, so the three checks read nothing", + "if (DEV_TREES.has(entry.name)) continue;", + "if (true) continue;", + "the shipped text set is derived from the package, and covers the twelve text files that ship"), + # ---- 5. the cache stops being released ---- # Anchored without any line ending on purpose. The first version of these two used \r\n because # server.mjs is CRLF in the working tree — and that is exactly the trap: it only matched because @@ -526,14 +585,20 @@ def _node(): if replace_all: for rel in ("tests/run.mjs", "README.md", "README.zh-CN.md", "miniapp/client/index.html", "miniapp/node/server.mjs", - ".minimax-plugin/plugin.json", "tests/verify-review16.mjs"): + ".minimax-plugin/plugin.json", + ".gitattributes", ".gitignore", "LICENSE", "package.json", + "miniapp/miniapp.json", "miniapp/node/miniapp-api.ts", + "tests/verify-review16.mjs"): if old in read(rel): target = rel break else: for rel in ("tests/run.mjs", "README.md", "README.zh-CN.md", "miniapp/client/index.html", "miniapp/node/server.mjs", - ".minimax-plugin/plugin.json", "tests/verify-review16.mjs"): + ".minimax-plugin/plugin.json", + ".gitattributes", ".gitignore", "LICENSE", "package.json", + "miniapp/miniapp.json", "miniapp/node/miniapp-api.ts", + "tests/verify-review16.mjs"): if read(rel).count(old) == 1: target = rel break diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py index b047edf5..77f7e2c7 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py @@ -329,6 +329,36 @@ " }\n", "", ), + # ---- the tail, which is about the set the page holds ------------------------------- + # + # A tail carries the records AFTER the page's cursor; the ones before it stay as the page + # has them. So the tail is only about the page's copy if that copy is still the service's — + # and a record rewritten IN PLACE keeps its index and its id, so the cursor still points at + # the same record, the tail looks empty, and the page merges nothing: it keeps the old text + # and installs the new body's validator. Nothing else in the protocol could see it. + # + # Three controls, because the promise has three pieces: a tail to a page whose copy has + # changed has to be refused, a tail asked for without the set has to be refused rather than + # guessed at, and the path that finds the body already held has to answer with the WHOLE + # body — a tail from there would be a tail nobody checked. + ( + "a tail is answered to a page whose copy is not the one the service has", + SERVER, + " if (recordSetFingerprint(payloadRecordVersions(payload, count)) !== fingerprint) {\n", + " if (false) {\n", + ), + ( + "a tail asked for without saying which set the page holds is answered anyway", + SERVER, + " if (!Number.isInteger(count) || count < 0 || !/^[0-9a-f]{16}$/.test(fingerprint)) {\n", + " if (false) {\n", + ), + ( + "the cached path stops answering with the body it holds, so a tail is cut from it unchecked", + SERVER, + " sendJsonBody(response, 200, cached.body, { etag, revalidate: true });\n return;\n", + " // the body is rebuilt below\n", + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-incremental.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-incremental.mjs index c2590476..d63f02d8 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-incremental.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-incremental.mjs @@ -20,7 +20,7 @@ import { readFileSync } from 'node:fs'; import { fileURLToPath } from 'node:url'; import vm from 'node:vm'; -import { buildDeltaPayload, parseSince } from '../miniapp/node/server.mjs'; +import { buildDeltaPayload, parseSince, deltaRefusal, payloadRecordVersions, recordSetFingerprint } from '../miniapp/node/server.mjs'; const HERE = fileURLToPath(new URL('.', import.meta.url)); const FIXTURE = fileURLToPath(new URL('./fixtures/trajectory.json', import.meta.url)); @@ -358,13 +358,133 @@ assert('the cursor of an empty payload is zero, not NaN', JSON.stringify(page.mergeDelta(merged, tail)) === JSON.stringify(merged)); } +// -------------------------------------------------- the set a tail is about + +/* + * A tail carries the records AFTER the page's cursor. The records before it are not in the tail + * at all — the page keeps the copies it has — so the tail is only about the page's copy if that + * copy is still the service's. + * + * A record rewritten in place keeps its index and its id. The cursor therefore still points at + * the same record, the tail names the same id, and the tail is empty: the page would merge + * nothing, keep the old text, and install the new body's validator. Nothing else in the + * protocol could see it — which is why the tail names the set it is about. + * + * So the page says which set it holds — how many records, and a fingerprint of their versions — + * and the service folds its own first N the same way. The two have to agree before a tail is + * answered with a tail. + */ + +// A body of three records in two turns, with the versions the service stamps on them. +const F7_BODY = { + session: { id: 's1' }, + stats: { records: 3 }, + turns: [ + { id: 't1', index: 1, closed: true, records: [ + { id: 'r1', index: 1, v: '1111111111111111' }, + { id: 'r2', index: 2, v: '2222222222222222' }, + ] }, + { id: 't2', index: 2, closed: false, records: [ + { id: 'r3', index: 3, v: '3333333333333333' }, + ] }, + ], +}; +const F7_COUNT = 3; +const F7_FINGERPRINT = recordSetFingerprint(payloadRecordVersions(F7_BODY, F7_COUNT)); + +{ + assert('a tail is answered to a page holding the records the service has', + deltaRefusal(F7_BODY, F7_COUNT, F7_FINGERPRINT) === null); + + // The record AT the cursor, rewritten in place: the same index, the same id, a new version. + const atCursor = JSON.parse(JSON.stringify(F7_BODY)); + atCursor.turns[1].records[0].v = 'deadbeefdeadbeef'; + const refusalAtCursor = deltaRefusal(atCursor, F7_COUNT, F7_FINGERPRINT); + assert('a content-only rewrite at the cursor cannot be acknowledged without replacing the displayed record', + refusalAtCursor !== null); + assert('and the refusal says the version moved', + Boolean(refusalAtCursor) && refusalAtCursor.code === 'trajectory_version_mismatch'); + + // The same, one turn earlier. + const earlier = JSON.parse(JSON.stringify(F7_BODY)); + earlier.turns[0].records[0].v = 'deadbeefdeadbeef'; + assert('a content-only rewrite in an earlier turn cannot be acknowledged either', + deltaRefusal(earlier, F7_COUNT, F7_FINGERPRINT) !== null); + + // Why the check has to be there at all: in both cases the tail is empty, and the record it + // names at the cursor is the same one the page already holds — so every other part of the + // protocol says the tail is fine. + assert('the tail carries nothing in either case, so the page would have kept the old text', + buildDeltaPayload(atCursor, 3).turns.length === 0 + && buildDeltaPayload(earlier, 3).turns.length === 0); + assert('and the record it names at the cursor is the one the page already holds', + buildDeltaPayload(atCursor, 3).delta.basedOn.id === 'r3'); + + // A tail asked for without saying which set the page holds is refused rather than guessed at. + const unnamed = deltaRefusal(F7_BODY, F7_COUNT, ''); + assert('a tail asked for without the set the page holds is refused', + Boolean(unnamed) && unnamed.code === 'trajectory_fingerprint_required'); + assert('and that refusal is a bad request, not a version mismatch', + Boolean(unnamed) && unnamed.status === 400); +} + +// --------------------------------------------------------------- the page, run + +{ + const versions = ['1111111111111111', '2222222222222222', '3333333333333333']; + const client = { + console, + encodeURIComponent, + state: { since: 3, flat: versions.map((v) => ({ record: { v } })) }, + }; + vm.createContext(client); + for (const name of ['recordSetFingerprint', 'displayedRecordVersions', 'heldRecordSet', 'tailQuery']) { + const src = fnSource('function ' + name + '('); + if (!src) throw new Error('could not extract ' + name); + vm.runInContext(src, client); + } + const held = client.heldRecordSet(); + assert('the page states the set it holds as a count and a fingerprint', + held.count === 3 && /^[0-9a-f]{16}$/.test(held.fingerprint)); + assert('and the fingerprint it sends is the one the service folds over the same records', + held.fingerprint === F7_FINGERPRINT); + const query = client.tailQuery(); + assert('a tail is asked for with the cursor and the set the page holds', + /&since=3/.test(query) && /&n=3/.test(query) && /&fp=[0-9a-f]{16}/.test(query)); +} + +{ + const asked = []; + const client = { + console, + state: { since: 42, sinceFor: 'key', etag: 'tag', etagFor: 'key' }, + render() { client.rendered = (client.rendered || 0) + 1; }, + fetchTrajectory(manual) { asked.push(manual); }, + }; + vm.createContext(client); + for (const name of ['reloadWholeBody', 'isStaleCopyRefusal']) { + const src = fnSource('function ' + name + '('); + if (!src) throw new Error('could not extract ' + name); + vm.runInContext(src, client); + } + client.reloadWholeBody(); + assert('a page whose copy is not the one the service has drops the cursor and asks for the whole body again', + client.state.since === null && client.state.sinceFor === null + && asked.length === 1 && asked[0] === true); + assert('a refusal about the page\'s copy is recognised as one, and only as one', + client.isStaleCopyRefusal({ code: 'trajectory_version_mismatch' }) === true + && client.isStaleCopyRefusal({ code: 'trajectory_fingerprint_required' }) === true + && client.isStaleCopyRefusal({ code: 'search_version_mismatch' }) === false + && client.isStaleCopyRefusal(null) === false); +} + // --------------------------------------------------------------- the wiring // A cursor is only ever asked alongside the validator it was taken against. Sending one // without the other is how a page ends up asking a server for the tail of a payload it does // not have — which is a session switch, or a generation switch, or a manual refresh. assert('the request carries the cursor and the validator under one condition', - /!manual && state\.sinceFor === cacheKey && state\.since > 0[\s\S]{0,120}&since=/.test(html) + /!manual && state\.sinceFor === cacheKey && state\.since > 0/.test(html) && /state\.etag && state\.etagFor === cacheKey/.test(html)); assert('a manual refresh drops the cursor as well as the validator', /if \(manual\) \{\s*\n\s*state\.etag = '';\s*\n\s*state\.etagFor = '';\s*\n\s*state\.since = 0;\s*\n\s*state\.sinceFor = '';/.test(html)); @@ -404,6 +524,19 @@ assert('and it still sends the whole body when there is no cursor', assert('the cursor after a delta is the server upTo, not a second walk', /state\.since = mergeable \? body\.delta\.upTo : payloadCursor\(payload\);/.test(html)); +// The refusal is wired to the answer, not merely computed: a check nobody reads is not a check. +assert('the service refuses a tail whose set the page did not state, and says so', + /const refusal = deltaRefusal\(/.test(server) + && /if \(refusal\) \{/.test(server) + && /sendError\(response, refusal\.status, refusal\.code, refusal\.message\);/.test(server)); + +// And the page answers both refusals the only way it can: by asking for the whole body. A refusal +// it treated as a failed poll would leave the stale copy on screen, which is the defect the +// refusal exists to stop. +assert('a tail the service will not answer leaves the page asking for the whole body', + /if \(isStaleCopyRefusal\(error\)\) \{/.test(html) + && /reloadWholeBody\(\);/.test(html)); + // ---------------------------------------------------------------- reporting for (const name of failures) console.log('FAIL ' + name); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs index 756f7e9f..2449e7f7 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -3,8 +3,9 @@ * promise that comes with giving them up is two-sided. * * The one side everybody can see: a group the reader is not on must not be holding its rows. - * Measured on a real session of 47,937 records, the rows of 「显示全部」 are 438,986 nodes and - * about 470 MB of the renderer's private memory, so "hold everything" is the whole cost. + * Measured on one session of about 50,600 records (2026-10-22), the rows of 「显示全部」 were + * 463,122 DOM nodes and the renderer's private memory read 785 to 820 MB, so "hold everything" + * is the whole cost. * * The other side is the one that is easy to lose: a group that has given up its rows must still * be *there*. The rows are the DOM; the height is the reader's place in a session they have @@ -69,7 +70,11 @@ const LIFECYCLE_FNS = [ 'fillLedgerGroup', 'holdLedgerHeight', 'fillGroupRows', - 'firstVisibleRow' + 'firstVisibleRow', + // The build itself, because the rule this file has to prove is about what the build + // writes. A group the reader has left must not be kept alive by the rebuild — and the + // source of a function cannot say whether it stamps the reader's lease or not. + 'buildStep' ]; const fnSrc = LIFECYCLE_FNS.map((name) => functionSource(name)); @@ -86,9 +91,11 @@ if (missing.length) { // one changed. const idleDecl = html.match(/^[ \t]*var LEDGER_IDLE_MS = [^;]+;/m); const capDecl = html.match(/^[ \t]*var LEDGER_LIVE_GROUPS = [^;]+;/m); +const batchDecl = html.match(/^[ \t]*var LEDGER_BATCH_GROUPS = [^;]+;/m); assert('the shipped script declares the idle timeout', !!idleDecl); assert('the shipped script declares the cap on groups holding their rows', !!capDecl); -if (!idleDecl || !capDecl) { +assert('the shipped script declares how many groups a build does at a time', !!batchDecl); +if (!idleDecl || !capDecl || !batchDecl) { console.error(`[ledger-lifecycle] ${pass} passed, ${failures.length} failed`); for (const f of failures) console.error(' FAIL ' + f); process.exit(1); @@ -107,6 +114,7 @@ function makeEl(tag) { setAttribute(name, value) { el.attrs[name] = String(value); }, getAttribute(name) { return name in el.attrs ? el.attrs[name] : null; }, querySelector(selector) { return queryOne(el, selector); }, + querySelectorAll() { return []; }, getBoundingClientRect() { return { height: 240, width: 900, top: 0, bottom: 240 }; } }; el.classList = { @@ -175,13 +183,71 @@ function harness() { }; vm.createContext(sandbox); vm.runInContext( - idleDecl[0] + '\n' + capDecl[0] + '\n' + fnSrc.join('\n'), + idleDecl[0] + '\n' + capDecl[0] + '\n' + batchDecl[0] + '\n' + fnSrc.join('\n'), sandbox ); box.sandbox = sandbox; return box; } +/** + * The sandbox the build runs in. + * + * The build is a nested function inside renderLedger, so lifting it leaves everything it + * closes over undefined — and it is lifted here to be *run*, not to be read. The collaborators + * it needs are the ones a build touches: the ledger it appends to, the groups it is told to + * build, and the clock. The ones that would need a real browser (the observer, the anchor, + * the measurement) are stubs, because the rule under test is about the lease and the lease is + * decided before any of them run. + */ +function buildHarness() { + const h = harness(); + const s = h.sandbox; + s.ledgerBuildToken = 1; + s.build = { token: 1, at: 0, rows: 0, bands: 0, heights: null }; + s.groups = []; + s.anchor = null; + s.scrollY = 0; + s.ledger = makeEl('div'); + s.isCount = (n) => typeof n === 'number'; + s.DASH = '—'; + s.turnKey = (turn) => 'k' + turn.index; + s.buildTurnHead = () => makeEl('div'); + s.applyIntrinsicSize = () => 240; + s.calibrateRowHeights = () => ({ rows: 240 }); + s.setLedgerProgress = () => {}; + s.restoreRowAnchor = () => {}; + s.flushDeferredPoll = () => {}; + s.watchLedgerGroups = () => {}; + s.syncStickyOffsets = () => {}; + s.window = { requestAnimationFrame: () => {} }; + return h; +} + +/** + * One rebuild of the ledger, the way renderLedger does it. + * + * The groups are the window's, the first one is the group the window is anchored on, and the + * state carries over — a rebuild does not reset the reader's lease, and proving that it does + * not is the whole point of the rule below. + */ +function rebuild(h, groups) { + const s = h.sandbox; + s.ledger.children.length = 0; + s.ledgerContent.clear(); + s.groups = groups; + s.build.at = 0; + s.build.rows = 0; + s.build.bands = 0; + s.build.heights = null; + s.buildStep(); +} + +/** The section the build made for a group, by its order in the window. */ +function sectionOf(h, index) { + return h.sandbox.ledger.children[index]; +} + /* ---------- 1. the rows of a group the reader is not on are not built ---------- */ { @@ -262,6 +328,10 @@ function harness() { for (let i = 0; i < n; i += 1) { s.ledgerState.set('g' + i, { live: true, lastSeen: 1000 + i, height: 0 }); } + // The clock is past every lease above. The anchored group is counted with the current time + // — it has no visit of its own to be dated by — so a clock that sat behind the leases would + // make the anchored group look like the oldest one, which is the one thing it never is. + h.clock = 2000; s.releaseStaleGroups('g' + (n - 1)); const live = [...s.ledgerState.values()].filter((v) => v.live).length; assert('the number of groups holding their rows is bounded', @@ -272,6 +342,85 @@ function harness() { && s.ledgerState.get('g' + (n - 1)).live === true); } +/* ---------- 3b. the group the window is anchored on is counted among them ---------- */ + +/* + * Two sets, and they have to be the same set. + * + * The bound is about the groups that hold their rows: that is the memory. The groups that are + * live are the groups the reader is keeping alive, and the group the window is anchored on is + * not one of them — the reader arrives there and every other group's height is predicted from + * it, so it holds its rows whatever the reader is doing. It has no visit of its own to be + * dated by, so it is counted rather than timed. + * + * Left out of the count, it would go on holding its rows past the bound: the bound would be + * about one set while the rows on screen were another. That is what the assertion below is + * for, and it is a separate assertion because it is a separate rule. + */ + +{ + const h = harness(); + const s = h.sandbox; + h.clock = 5000; + // The anchored group: not live, and its last visit was long enough ago that the idle + // timeout would have taken it, if the idle timeout were about it. It is not. + s.ledgerState.set('anchor', { live: false, lastSeen: 1000, height: 0 }); + for (let i = 0; i < s.LEDGER_LIVE_GROUPS + 1; i += 1) { + s.ledgerState.set('g' + i, { live: true, lastSeen: 4000 + i, height: 0 }); + } + s.releaseStaleGroups('anchor'); + const live = [...s.ledgerState.values()].filter((v) => v.live).length; + assert('the window\'s first group is one of the groups that hold their rows', + s.ledgerState.get('anchor').live === true); + assert('the bound is about the groups holding their rows, and the anchored group is one of them', + live === s.LEDGER_LIVE_GROUPS); +} + +/* ---------- 3c. a rebuild is not a visit ---------- */ + +/* + * The poll runs every eight seconds whether the reader is there or not. A build that stamped + * the reader-idle lease would renew it every time — so the lease could never run out, the + * group the reader had left would keep its row DOM for the rest of the session, and the + * fifteen-second idle timeout would be a rule about nothing. + * + * The build is run here rather than read: what it writes is the rule, and the source of a + * function does not say whether it writes it. + */ + +{ + const h = buildHarness(); + const s = h.sandbox; + const groups = [ + { turn: { index: 1 }, entries: entriesOf(2, 0, 'a') }, + { turn: { index: 2 }, entries: entriesOf(2, 0, 'b') } + ]; + // The second group is the one the reader left: live, last seen at 1000, and nobody has + // been at it since. + s.ledgerState.set('k2', { live: true, lastSeen: 1000, height: 0 }); + + h.clock = 9000; + rebuild(h, groups); + assert('rebuilding an offscreen group does not renew its reader-idle lease', + s.ledgerState.get('k2').lastSeen === 1000); + assert('a group the reader has just left still holds its rows, because the idle time has not passed', + s.ledgerState.get('k2').live === true + && sectionOf(h, 1).getAttribute('data-rows') === 'live'); + + h.clock = 17000; + s.releaseStaleGroups('k1'); + rebuild(h, groups); + assert('a group the reader has left gives its rows up while the ledger keeps being rebuilt', + s.ledgerState.get('k2').live === false + && sectionOf(h, 1).getAttribute('data-rows') === 'released'); + + h.clock = 25000; + rebuild(h, groups); + assert('a group that has given its rows up does not get them back by being rebuilt', + s.ledgerState.get('k2').live === false + && sectionOf(h, 1).getAttribute('data-rows') === 'released'); +} + { const h = harness(); const s = h.sandbox; @@ -389,8 +538,15 @@ function harness() { /* ---------- 6. the shipped build asks for exactly this ---------- */ const buildStepSrc = functionSource('buildStep') || ''; +// The comments are stripped before the shape checks, so a comment that talks about the lease +// is not a lease write. A rule that only holds while nobody writes about it is not a rule. +const buildStepCode = buildStepSrc + .replace(/\/\*[\s\S]*?\*\//g, '') + .replace(/\/\/[^\n]*/g, ''); assert('the build asks for no rows for a group it is not holding', /fillGroupRows\(holds \? list : null, group\)/.test(buildStepSrc)); +assert('the build does not write the reader-idle lease', + !/lastSeen\s*=/.test(buildStepCode) && !/\.live\s*=/.test(buildStepCode)); assert('a group the build is not holding is given a height to keep', /if \(!holds\) \{[\s\S]{0,400}?holdLedgerHeight\(section,/.test(buildStepSrc)); assert('a group the build is holding says so', diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-pr15-fixes.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-pr15-fixes.mjs index 10bd252e..f85f1b6e 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-pr15-fixes.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-pr15-fixes.mjs @@ -129,8 +129,14 @@ check('the server sends the cap it applied', buildTrajectoryPayload(baseInput).s check('64 MB is what that cap is', MAX_MESSAGE_BYTES, 64 * 1024 * 1024); assert('the client renders the cap it was sent', /session\.maxMessageBytes/.test(CLIENT)); assert('the client formats it rather than restating a number', /formatBytes\(session\.maxMessageBytes\)/.test(CLIENT)); -assert('no hardcoded 8 MB survives in the markup', !/8 MB/.test(CLIENT)); -assert('no hardcoded 64 MB either — the value comes from the payload', !/64 MB/.test(CLIENT)); +// The cap is a number of its own, and the boundary is part of the rule. The client's own comments +// carry measurements — the ledger's cost is written down there, in megabytes — and a measurement +// whose number merely ENDS in 8 ("268 MB") is not a hardcoded cap. A scan that cannot tell the two +// apart is a guard that fails for the wrong reason, and it would fail on the file that documents +// this very change, which is the worst place for it to fail. So the number has to begin where a +// number begins, not at the tail of one. +assert('no hardcoded 8 MB survives in the markup', !/(? relative(PKG, p).split(/[\\/]/).join('/')); +const SHIPPED_TEXT = {}; +for (const rel of derivedRel) SHIPPED_TEXT[rel] = read(rel); +SHIPPED_TEXT['tests/make-fixtures.mjs'] = read('tests/make-fixtures.mjs'); +SHIPPED_TEXT['tests/client-check.mjs'] = read('tests/client-check.mjs'); + +// The deriver is what the three checks below now stand on, so it gets a check of its own. If the +// walk ever returns nothing -- a skip list that grew a name, a directory that moved -- all three +// read an empty set and none of them fires, and a guard that cannot fail is not a guard that +// passed. Twelve is the number of text files that ship today: adding one to the package must not +// trip this, losing one must. +assert('the shipped text set is derived from the package, and covers the twelve text files that ship', + derivedRel.length >= 12, 'derived ' + derivedRel.length); // This assertion was written over the two READMEs, because those are what a reader opens first, and // the two shipped source files were never added to it. They carried 19 of these pointers between diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs index 34d2fbf0..04bc724d 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs @@ -364,21 +364,31 @@ try { // The session identity does not move: the same directory, the same manifest, the same id — only // the line behind record 1 is a different line. That is what makes this the hard case: the // index still says 1, and the position is still a position, but it names a different record. + // + // TWO records in TWO turns, because the rule has two cases and one record can only ever be + // one of them: a record rewritten AT the cursor, and a record rewritten in a turn BEFORE it. + // The writer takes the lines one at a time, so a case can rewrite one of them and leave the + // other exactly as it was. const VERSIONED = 'mvs_versioned'; const versionedDir = join(sessionsRoot, '2026', '10', '22', '00-00-00-session_' + Buffer.from(VERSIONED, 'utf8').toString('base64url')); - const writeVersioned = async (body, mtimeMs) => { + const writeVersioned = async (lines, mtimeMs) => { await mkdir(versionedDir, { recursive: true }); - await writeFile(join(versionedDir, 'messages.jsonl'), JSON.stringify({ - message_id: 'm1', - turn_id: 't1', - message: { role: 'user', content: body, timestamp: 1791600000000 }, - }) + '\n', 'utf8'); + const text = lines.map((one) => JSON.stringify({ + message_id: one.id, + turn_id: one.turn, + message: { role: 'user', content: one.body, timestamp: 1791600000000 }, + }) + '\n').join(''); + await writeFile(join(versionedDir, 'messages.jsonl'), text, 'utf8'); await writeFile(join(versionedDir, 'manifest.json'), JSON.stringify({ sessionId: VERSIONED }), 'utf8'); const stamp = new Date(mtimeMs); await utimes(join(versionedDir, 'messages.jsonl'), stamp, stamp); }; - await writeVersioned('ALPHA BODY', 1791600000000); + const ALPHA_LINES = [ + { id: 'm1', turn: 't1', body: 'ALPHA BODY' }, + { id: 'm2', turn: 't2', body: 'ALPHA SECOND' }, + ]; + await writeVersioned(ALPHA_LINES, 1791600000000); // The runtime database, which is the only place on this machine that knows which // conversation is live. ALPHA is live first; BRAVO becomes live later in this section. @@ -475,6 +485,20 @@ try { } return out; }; + // The cursor a page would ask a tail from: the last record it holds. The same word the server + // uses for it, and the same number — a tail cut at the wrong one is a tail about the wrong + // records, and the point of this section is that the wrong one is refused rather than served. + const lastIndexOf = (body) => { + const turns = body && Array.isArray(body.turns) ? body.turns : []; + let last = 0; + for (const turn of turns) { + const records = Array.isArray(turn && turn.records) ? turn.records : []; + for (const record of records) { + if (record && typeof record.index === 'number' && record.index > last) last = record.index; + } + } + return last; + }; // The segments a search published, found by walking the data dir. Publishing them is a side // effect that nobody awaits, so the test waits for the file rather than for a promise. @@ -711,8 +735,18 @@ try { const oldVersion = versionOf(versionLoad.body, 1); assert('and its record carries a version', oldVersion !== '', 'version: ' + JSON.stringify(oldVersion)); - - await writeVersioned('BRAVO BODY', 1791800000000); + // The set the page holds, and the cursor it would ask a tail from: the records it has, in + // order, and the last of them. + const oldVersions = versionsOf(versionLoad.body); + const oldCursor = lastIndexOf(versionLoad.body); + assert('the conversation has the two records the two cases need', + oldVersions.length === 2 && oldCursor === 2, + 'versions: ' + oldVersions.length + ' cursor: ' + oldCursor); + + await writeVersioned([ + { id: 'm1', turn: 't1', body: 'BRAVO BODY' }, + { id: 'm2', turn: 't2', body: 'ALPHA SECOND' }, + ], 1791800000000); // The service now has to be holding the NEWER body, or the read below is not the read the // reader would make. A fresh service instance is how it stops believing the walk it cached: // the walk holds the session list for three seconds, so a rewritten transcript is invisible @@ -723,6 +757,35 @@ try { instance = restarted.instance; base = restarted.base; } + + // THE TAIL A PAGE ASKS FOR WHEN THE RECORD UNDER IT HAS CHANGED. + // + // The tail carries the records AFTER the page's cursor; the ones before it stay as the page + // has them. So the tail is only about the page's copy if that copy is still the service's — + // and a record rewritten in place keeps its index and its id, so the cursor still points at + // the same record and the tail looks empty. The page would merge nothing, keep the old text, + // and install the new body's validator, and nothing else in the protocol could see it. + // + // So the request says which set the page holds, and the service refuses a tail it cannot tie + // to that set. THIS IS THE FIRST REQUEST AFTER THE RESTART ON PURPOSE: the refusal belongs + // to the path that REBUILDS the body, and the path that finds the body already held answers + // with the whole body — which is safe, because a whole body replaces everything. + // + // The set the page holds is the one it had before the rewrite: record 1 was rewritten in + // place, in the turn BEFORE the cursor's turn. + const staleTail = await get('/api/trajectory?session=' + VERSIONED + '&since=' + oldCursor + + '&n=' + oldVersions.length + '&fp=' + recordSetFingerprint(oldVersions)); + assert('a tail asked for by a page whose record was rewritten in place is refused', + staleTail.status === 409 && Boolean(staleTail.body) + && staleTail.body.error === 'trajectory_version_mismatch', why(staleTail)); + // And the tail it is NOT given: the next request finds the body the service already holds, + // and is answered with the whole body. That is safe — a whole body replaces everything — and + // it is why the refusal belongs to the path that rebuilds: the cached path answers with a + // whole body, not with a tail, so there is no set for it to check. + const wholeBody = await get('/api/trajectory?session=' + VERSIONED + '&since=' + oldCursor); + assert('a tail asked for against a body the service already holds is answered with the whole body', + wholeBody.status === 200 && Boolean(wholeBody.body) && !wholeBody.body.delta, why(wholeBody)); + const versionReload = await get('/api/trajectory?session=' + VERSIONED); assert('the rewritten conversation loads again', versionReload.status === 200, why(versionReload)); @@ -770,6 +833,48 @@ try { assert('and the record read refuses it the same way', detailNoVersion.status === 400 && detailNoVersion.body && detailNoVersion.body.error === 'record_version_required', why(detailNoVersion)); + + // The other case of the same rule: the record AT THE CURSOR, rewritten in place. + // + // The page's set is the one it would hold now — the body after the first rewrite, whose + // record 2 is still the old one — and the record that changed under it is the one the cursor + // points at. The tail would be empty here too, and the id it names would still agree. + await writeVersioned([ + { id: 'm1', turn: 't1', body: 'BRAVO BODY' }, + { id: 'm2', turn: 't2', body: 'BRAVO SECOND' }, + ], 1791900000000); + await instance.dispose(); + { + const restarted = await startService(); + instance = restarted.instance; + base = restarted.base; + } + const cursorSet = versionsOf(versionReload.body); + const staleCursorTail = await get('/api/trajectory?session=' + VERSIONED + '&since=' + oldCursor + + '&n=' + cursorSet.length + '&fp=' + recordSetFingerprint(cursorSet)); + assert('a tail asked for by a page whose record AT THE CURSOR was rewritten in place is refused', + staleCursorTail.status === 409 && Boolean(staleCursorTail.body) + && staleCursorTail.body.error === 'trajectory_version_mismatch', why(staleCursorTail)); + + // And the request that names no set at all. The service cannot tell which records the page + // has, so it cannot answer with a tail: the records before the cursor are not in the tail, + // and the page would keep the copies it has without knowing whether they are still the + // service's. A whole body would be safe, but the body has to be rebuilt for it — and the + // rule is that a tail is only ever answered to a page that says which set it holds. + await writeVersioned([ + { id: 'm1', turn: 't1', body: 'CHARLIE BODY' }, + { id: 'm2', turn: 't2', body: 'BRAVO SECOND' }, + ], 1792000000000); + await instance.dispose(); + { + const restarted = await startService(); + instance = restarted.instance; + base = restarted.base; + } + const unnamedTail = await get('/api/trajectory?session=' + VERSIONED + '&since=' + oldCursor); + assert('a tail asked for without saying which set the page holds is refused rather than guessed at', + unnamedTail.status === 400 && Boolean(unnamedTail.body) + && unnamedTail.body.error === 'trajectory_fingerprint_required', why(unnamedTail)); } finally { // The server is released here and not at the end of the happy path: an assertion that // throws would otherwise leave it listening, and the guard would hang instead of From 94ebdf14e01ab3b2d6c666b243cf228a76086d7d Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 21:36:13 +0800 Subject: [PATCH 39/95] A released group keeps the height it had, brought up to date with what it holds now MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The height a group keeps was measured on the content it held when the reader was last on it, and a group is keyed by its turn — which keeps its key while its content changes. A late reply, or a band from a compaction, left the group holding the height of a content that was no longer there: the document under the reader came up short by those rows, and the page grew taller under the reader when the group came back. The measurement now records what it was a measurement of, and the difference between what the group holds now and what it held then is charged at the calibrated row and band heights. The group's own height is kept as it was and only the rows that changed are priced, which is the whole difference between this and falling back on the prediction — the prediction is priced from medians, and a group's real height runs to several times the median one. The build reads that measurement and does not write it. The measurement is the observer's, taken when the group is really laid out, and a build that stamped one would be writing down a height it never measured. The difference is priced at a median, so a group whose own rows are taller than the median one has its difference undercounted: the height is brought up to date, not re-measured. --- .../mmc-trajectory/miniapp/client/index.html | 103 +++++++++++++++--- 1 file changed, 88 insertions(+), 15 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 0e32aa01..2b952f51 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -1414,9 +1414,10 @@ The rows are gone, so the box is empty — and an empty box is a box of no height, which is how the document under the reader loses the height of everything they have scrolled past. The height is written onto the group from JavaScript, from the height it had while it was - holding its rows, and it is the only thing that holds the box while the browser is - RENDERING the group rather than skipping it: `contain-intrinsic-size` sizes an element - the browser has skipped, and a group near the viewport is rendered. + holding its rows — brought up to date with what the group holds now, since the turn keeps + its key while its content changes — and it is the only thing that holds the box while the + browser is RENDERING the group rather than skipping it: `contain-intrinsic-size` sizes an + element the browser has skipped, and a group near the viewport is rendered. Measured, on a real session of 47,937 records: releasing the rows of the groups around the viewport took the scroll height down by 79,288 px, and releasing the viewport's own @@ -7138,7 +7139,7 @@

检视

* place in it across that, and it is why the reader does not have to start again at the top * of the session on every tick. * - * @type {Map} + * @type {Map} */ var ledgerState = new Map(); @@ -7165,13 +7166,18 @@

检视

/** * The state of one group, created on first sight. * + * `heightRows` and `heightBands` say what the measured height was a measurement OF. A + * height on its own describes a past: the group can hold a different number of rows by + * the time the height is read again, and without the two counts the difference cannot be + * charged to anything. + * * @param {string} key - * @returns {{live: boolean, lastSeen: number, height: number}} + * @returns {{live: boolean, lastSeen: number, height: number, heightRows: number, heightBands: number}} */ function ledgerStateOf(key) { var state = ledgerState.get(key); if (!state) { - state = { live: false, lastSeen: 0, height: 0 }; + state = { live: false, lastSeen: 0, height: 0, heightRows: 0, heightBands: 0 }; ledgerState.set(key, state); } return state; @@ -7266,6 +7272,13 @@

检视

// rather than the truth — which is why it is the remembered height and not the only one. sizeBuiltGroup(section); state.height = Math.round(section.getBoundingClientRect().height); + // What it was holding when it was measured, so that the height can be brought up to + // date with the content the group holds now. The group is keyed by the turn, and the + // turn keeps its key while its content changes — a late reply, a band from a + // compaction — so a height that is not read together with its counts describes a + // content that is no longer there. + state.heightRows = group.entries.length; + state.heightBands = group.bands || 0; } /** @@ -7306,6 +7319,61 @@

检视

section.style.minHeight = Math.round(height) + 'px'; } + /** + * The height a released group should keep, brought up to date with what it holds now. + * + * The remembered height is a measurement of a past. It was taken when the group was + * holding a particular number of rows and bands, and the group can gain one of each + * after the reader has walked away from it — a late reply, a band from a compaction — + * because a group is keyed by its turn, and the turn keeps its key while its content + * changes. A height kept unchanged then describes content that is no longer there, and + * the document under the reader comes up short by exactly the rows that were added. + * + * So the difference between what the group holds now and what it held when it was + * measured is charged at the rates the prediction uses: a row's worth of height for + * every row added, a band's worth for every band. What is left of the measurement is + * the part that belongs to the group rather than to its content — its own header, its + * borders — and that is the part the prediction cannot know, because the prediction is + * priced from medians and this group's chrome is its own. + * + * The difference is local: the measured height is kept as it was, and only the rows + * that changed are priced. That is the whole difference between this and falling back + * on the prediction, which would price the whole group again — and a group's real + * height runs to several times the median one. + * + * The measurement is the observer's: it is taken when the group is really laid out. + * This reads it and never writes it, because a build that stamped one would be writing + * down a height it never measured. + * + * What this does not claim: the difference is priced at the calibrated heights, which + * are medians, so a group whose own rows are taller than the median one has its + * difference undercounted. The height is brought up to date; it is not a measurement of + * what the group holds now. + * + * @param {string} key + * @param {number} rows how many rows the group holds now + * @param {number} bands how many of them are compaction bands + * @param {number} predicted the height the shape of the group predicts + * @param {{head: number, row: number, band: number}|null} heights the calibrated heights + * @returns {number} + */ + function heldLedgerHeight(key, rows, bands, predicted, heights) { + var state = ledgerState.get(key); + if (!state || !(state.height > 0)) return predicted; + // A height measured without knowing what was in the group cannot be brought up to + // date: there is no difference to charge. + if (typeof state.heightRows !== 'number') return predicted; + // No calibration yet, so a difference in rows cannot be turned into pixels. The + // measured height is stale rather than right, and a stale height is better than none: + // the first batch is re-sized from the calibration as soon as there is one. + if (!heights) return state.height; + var addedRows = rows - state.heightRows; + var addedBands = bands - state.heightBands; + if (!addedRows && !addedBands) return state.height; + var held = state.height + addedRows * heights.row + addedBands * heights.band; + return held > 0 ? held : predicted; + } + /** * Build a group's rows into its list. * @@ -7640,9 +7708,15 @@

检视

// about 50,600 records (2026-10-22), the rows of 「显示全部」 were 463,122 DOM nodes and // the renderer's private memory read 785 to 820 MB; with only the groups the reader is // on holding their rows, the same session measured 14,801 nodes and 199 to 268 MB. - // Those are measurements of one session, not bounds. What the reader does not give up - // is the height — a released group keeps the box it had, so the document under them - // does not change size when they scroll. + // Those are measurements of one session, not bounds: each run reads three positions + // and the figure above is the middle-of-session one, and the same build read 194.1 MB + // at the first position of one run and 269.8 MB at the last position of the other — a + // spread of 39% for the same code on the same afternoon. The service's own memory was + // not measured in these runs at all. + // + // What the reader does not give up is the height — a released group keeps the box it + // had, brought up to date with what it holds now, so the document under them does not + // change size when they scroll. var key = turnKey(group.turn); // The rebuild is not a visit. The poll runs every eight seconds whether the reader // is there or not, so a build that stamped the lease would renew it forever: the @@ -7667,12 +7741,11 @@

检视

// below is what covers those groups. var predicted = applyIntrinsicSize(section, group.entries.length, bands, build.heights); if (!holds) { - // The height the group had while it was holding its rows, if it ever was; the - // prediction otherwise. A group that has never been on screen has no measured - // height, and the prediction is the best answer there is — the same answer the - // scrollbar is already giving. - var known = ledgerStateOf(key).height; - holdLedgerHeight(section, known > 0 ? known : predicted); + // The height the group had while it was holding its rows, brought up to date + // with the content it holds now; the prediction otherwise. A group that has + // never been on screen has no measured height, and the prediction is the best + // answer there is — the same answer the scrollbar is already giving. + holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights)); } ledger.appendChild(section); ledgerContent.set(section, { key: key, turn: group.turn, entries: group.entries, bands: bands }); From 6e84ab0322a9e12746c8dac9578a35c651ecebb4 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 21:36:13 +0800 Subject: [PATCH 40/95] The guards that own the height a released group keeps The rule that a released group's height has to be brought up to date with what it holds now is asked about the behaviour, not about the shape of the source, and each break of it has to be reported by name. Three groups are measured for real and then rebuilt: one that gained a row, one whose content did not change, and one that gained a band. The height the build gives the first is the measured height plus one row, the second is the measured height unchanged, and the third is the measured height plus one band. A rule that cannot tell a changed group from an unchanged one, or that charges a band as a row, is visible in those three numbers. The build must not write the height it computes: the measurement belongs to the observer, and a build that stamped one would write down a height it never measured. That is checked by running the build and reading the state afterwards, and by asking the build's own source whether it writes one. The control breaks five things: the difference is not charged, the measurement stops recording what it was a measurement of, the band count is not recorded, the build writes the height it computed, and the measurement is replaced by the prediction. Each is caught by name. The suite is green in full: 101 of 101 controls pass, and the package is restored byte for byte afterwards. The guard that owns this rule is 47 assertions and its control is 28 mutations. --- .../tests/mutate-ledger-lifecycle.py | 49 +++++++++- .../tests/verify-ledger-lifecycle.mjs | 95 ++++++++++++++++++- 2 files changed, 136 insertions(+), 8 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py index 65220fd8..d090b8d3 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -89,10 +89,10 @@ # the browser has skipped, and a group near the viewport is rendered — so without an # explicit height the box collapses to its 39 px header and the document shrinks under # the reader. - ("a released group loses the height it had", + ("a released group loses the height it is given", [(" section.style.minHeight = Math.round(height) + 'px';\n", " section.style.minHeight = '';\n")], - "a released group keeps the height it had"), + "a released group keeps the height it is given"), # A group with no measured height gets no reserved height either. Writing 0px would be # worse than writing nothing: a group sized 0px can never come back into the viewport, so @@ -150,7 +150,7 @@ # A group the build is not holding is given a height to keep. ("a group the build is not holding is given no height to keep", - [(" holdLedgerHeight(section, known > 0 ? known : predicted);\n", "")], + [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights));\n", "")], "a group the build is not holding is given a height to keep"), # The two states are what the rest of the page reads: the build says which, and the @@ -199,6 +199,49 @@ [(" ledgerState.clear();\n ledgerContent.clear();\n", " ledgerContent.clear();\n")], "the groups of the session being left are forgotten with it"), + + # The height a released group keeps has to describe the content it holds now. A group is + # keyed by its turn, and the turn keeps its key while its content changes — a late reply, a + # band from a compaction — so the height taken when the group was last on screen describes + # content that is no longer there. Without the difference being charged, the group keeps + # the height of a content it no longer holds, and the document under the reader comes up + # short by exactly the rows that were added. + ("a released group does not bring the height it keeps up to date", + [(" var held = state.height + addedRows * heights.row + addedBands * heights.band;\n", + " var held = state.height;\n")], + "a released group brings the height it keeps up to date with the rows it gained"), + + # A height that does not say what it was a measurement of cannot be brought up to date: + # there is no difference to charge, and the group keeps a height that describes nothing. + ("the height a group was measured at stops recording what it was holding", + [(" state.heightRows = group.entries.length;\n" + " state.heightBands = group.bands || 0;\n", + "")], + "the height a group was measured at records what it was holding"), + + # A band is not a row. It has its own height, and a group that gained one is taller by a + # band and not by a row — so a band that is not counted is a band that is not charged. + ("the band a group was measured at stops being recorded", + [(" state.heightBands = group.bands || 0;\n", " state.heightBands = 0;\n")], + "a released group is charged for the band it gained"), + + # The measured height belongs to the observer: it is taken when the group is really laid + # out. A build that stamped one would be writing down a height it never measured, and the + # next rebuild would read its own guess back as if it were a measurement. + ("the build writes the height it computed", + [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights));\n", + " ledgerStateOf(key).height = predicted;\n" + " holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights));\n")], + ["a rebuild does not change the height a group was measured at", + "the build does not write a measured height"]), + + # A group that has been on screen keeps the height it was measured at. The prediction is + # priced from medians, and a group's real height runs to several times the median one, so + # falling back on the prediction trades a real height for an average one. + ("a released group is given the prediction instead of the height it was measured at", + [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights));\n", + " holdLedgerHeight(section, predicted);\n")], + "a released group whose content has not changed keeps the height it was measured at"), ] original = io.open(CLIENT, encoding="utf-8", newline="").read() diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs index 2449e7f7..cb8537ee 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -69,6 +69,7 @@ const LIFECYCLE_FNS = [ 'releaseStaleGroups', 'fillLedgerGroup', 'holdLedgerHeight', + 'heldLedgerHeight', 'fillGroupRows', 'firstVisibleRow', // The build itself, because the rule this file has to prove is about what the build @@ -205,6 +206,7 @@ function buildHarness() { const s = h.sandbox; s.ledgerBuildToken = 1; s.build = { token: 1, at: 0, rows: 0, bands: 0, heights: null }; + s.buildTotal = 0; s.groups = []; s.anchor = null; s.scrollY = 0; @@ -214,7 +216,10 @@ function buildHarness() { s.turnKey = (turn) => 'k' + turn.index; s.buildTurnHead = () => makeEl('div'); s.applyIntrinsicSize = () => 240; - s.calibrateRowHeights = () => ({ rows: 240 }); + // The real calibration answers with the three heights it measured — the head, a row and a + // band — and the build prices a difference in rows with them. A stub that answers with any + // other shape is a stub of a different function. + s.calibrateRowHeights = () => ({ head: 40, row: 36, band: 60 }); s.setLedgerProgress = () => {}; s.restoreRowAnchor = () => {}; s.flushDeferredPoll = () => {}; @@ -240,6 +245,9 @@ function rebuild(h, groups) { s.build.rows = 0; s.build.bands = 0; s.build.heights = null; + // The total the build reports while it is running. It lives in renderLedger() beside the + // build, and a window longer than one batch is the case that reads it. + s.buildTotal = groups.reduce((n, g) => n + g.entries.length, 0); s.buildStep(); } @@ -284,14 +292,14 @@ function sectionOf(h, index) { list.children.filter(isBand).length === 1 && list.children.filter(isRow).length === 2); } -/* ---------- 2. a released group keeps the height it had ---------- */ +/* ---------- 2. a released group keeps the height it is given ---------- */ { const h = harness(); const s = h.sandbox; const section = makeEl('section'); s.holdLedgerHeight(section, 3643); - assert('a released group keeps the height it had', section.style.minHeight === '3643px'); + assert('a released group keeps the height it is given', section.style.minHeight === '3643px'); } { @@ -303,6 +311,74 @@ function sectionOf(h, index) { section.style.minHeight === undefined); } +/* ---------- 2b. the height a released group keeps is brought up to date with what it holds ---------- */ + +/** + * Measure a group the way the observer does: fill it, and take the height it really has. + * + * The counts the height was measured at are recorded by the same call that records the height, + * so a test that sets them by hand is testing a state the shipped code never produces. + */ +function measureGroup(h, key, entries, bands, height) { + const s = h.sandbox; + const section = makeEl('section'); + const list = makeEl('ul'); + list.classes.add('row-list'); + section.appendChild(list); + section.setAttribute('data-rows', 'released'); + section.getBoundingClientRect = () => ({ height, width: 900, top: 0, bottom: height }); + s.ledgerContent.set(section, { key, turn: { index: 1 }, entries, bands }); + s.fillLedgerGroup(section); + return section; +} + +/* + * The measured height is a measurement of a past. A group is keyed by its turn, and the turn + * keeps its key while its content changes — a late reply, a band from a compaction — so the + * height taken when the group was last on screen describes content that is no longer there. + * A group that keeps it unchanged is shorter than the content it is holding, and the + * document under the reader loses the rows that were added. + * + * The difference is priced locally: the measured height is kept, and only the rows that + * changed are charged at the calibrated row height. The three cases below are the three ways + * the content can differ — a row added, nothing changed, a band added — and the first two + * carry the same measured height, so a rule that cannot tell them apart is visible. + * + * The window has to be longer than one batch, because the first batch is priced from the + * prediction and the groups after it from the measurement. Twenty-five groups is one batch. + */ +{ + const h = buildHarness(); + const s = h.sandbox; + const groups = []; + for (let i = 0; i < 30; i += 1) { + groups.push({ turn: { index: i + 1 }, entries: entriesOf(3, 0, 'g' + i) }); + } + // Three groups measured for real, on three different contents: rows and no band, the same + // again, and rows with a band. + measureGroup(h, 'k26', entriesOf(3, 0, 'm26'), 0, 1000); + measureGroup(h, 'k27', entriesOf(3, 0, 'm27'), 0, 1000); + measureGroup(h, 'k28', entriesOf(2, 1, 'm28'), 1, 1000); + // And the content they hold now: a row added, nothing changed, a band added. + groups[25].entries = entriesOf(4, 0, 'g25'); + groups[26].entries = entriesOf(3, 0, 'g26'); + groups[27].entries = entriesOf(1, 2, 'g27'); + // The reader walked away from all three a long time ago, so they do not hold their rows. + h.clock = 100000; + + rebuild(h, groups); + s.buildStep(); + + assert('a released group brings the height it keeps up to date with the rows it gained', + sectionOf(h, 25).style.minHeight === '1036px'); + assert('a released group whose content has not changed keeps the height it was measured at', + sectionOf(h, 26).style.minHeight === '1000px'); + assert('a released group is charged for the band it gained', + sectionOf(h, 27).style.minHeight === '1060px'); + assert('a rebuild does not change the height a group was measured at', + s.ledgerState.get('k26').height === 1000); +} + /* ---------- 3. a group the reader left gives its rows up, and the count is bounded ---------- */ { @@ -443,8 +519,10 @@ function sectionOf(h, index) { section.classList.add('is-released'); section.style.minHeight = '3643px'; - const entries = entriesOf(4, 0, 'r'); - s.ledgerContent.set(section, { key: 'g1', turn: { index: 1 }, entries, bands: 0 }); + // Three rows and a band: four rows on screen, and a band that is priced as a band rather + // than as a row — so the counts the height was measured at are not the same number twice. + const entries = entriesOf(3, 1, 'r'); + s.ledgerContent.set(section, { key: 'g1', turn: { index: 1 }, entries, bands: 1 }); s.fillLedgerGroup(section); assert('a group the reader reaches puts its rows back', list.children.length === 4); @@ -455,6 +533,8 @@ function sectionOf(h, index) { section.style.minHeight === ''); assert('a group that has its rows back is remembered as live', s.ledgerState.get('g1').live === true); + assert('the height a group was measured at records what it was holding', + s.ledgerState.get('g1').heightRows === 4 && s.ledgerState.get('g1').heightBands === 1); } { @@ -547,6 +627,11 @@ assert('the build asks for no rows for a group it is not holding', /fillGroupRows\(holds \? list : null, group\)/.test(buildStepSrc)); assert('the build does not write the reader-idle lease', !/lastSeen\s*=/.test(buildStepCode) && !/\.live\s*=/.test(buildStepCode)); +// The measured height belongs to the observer, for the same reason the lease does: it is +// taken when the group is really laid out. A build that stamped one would be writing down a +// height it never measured, and the next rebuild would read its own guess back as one. +assert('the build does not write a measured height', + !/\.height\s*=/.test(buildStepCode)); assert('a group the build is not holding is given a height to keep', /if \(!holds\) \{[\s\S]{0,400}?holdLedgerHeight\(section,/.test(buildStepSrc)); assert('a group the build is holding says so', From 4df9a45e0c8c9cbe3dd70811c59c80b2aa2a8b28 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 22:14:44 +0800 Subject: [PATCH 41/95] A compaction band is charged as a band, and not as a row as well The height a released group keeps was brought up to date with the rows and the bands the group had gained. The two counts it kept were not two counts: the measurement recorded how many entries the group held, and a band is an entry as well as a row, so every band was counted once among the rows that had been added and once among the bands. A band was therefore charged a row's height and a band's height -- 36 px and 38 px on a real session, so 74 px for a band that is 38 px tall -- and the box stood a whole row taller than the content it was holding. That is the document lying about its own height in the other direction, which is the thing this change was meant to stop. The measurement now records the rows and the bands as two counts, and the rebuild prices the difference between them: a row's worth of height for every row added, a band's worth for every band, each counted once. The names say what the numbers are now -- the stored count is rows, and the band count is counted apart -- and the comment on the helper says the same. --- .../mmc-trajectory/miniapp/client/index.html | 22 ++++++++++++++----- 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 2b952f51..61561a02 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -7277,8 +7277,14 @@

检视

// turn keeps its key while its content changes — a late reply, a band from a // compaction — so a height that is not read together with its counts describes a // content that is no longer there. - state.heightRows = group.entries.length; - state.heightBands = group.bands || 0; + // + // The two counts are rows and bands, and they are counted apart. A band is an entry + // as well as a row, so a count of entries would charge every band once as a row and + // once as a band — the band at 36 px and 38 px would be charged 74 px, and the box + // would stand taller than the content it is holding. + var measuredBands = group.bands || 0; + state.heightRows = group.entries.length - measuredBands; + state.heightBands = measuredBands; } /** @@ -7331,7 +7337,10 @@

检视

* * So the difference between what the group holds now and what it held when it was * measured is charged at the rates the prediction uses: a row's worth of height for - * every row added, a band's worth for every band. What is left of the measurement is + * every row added, a band's worth for every band. The two counts are kept apart, so a + * band is charged as a band and not as a row as well — the two heights are close (36 px + * and 38 px, measured on a real session), and a band charged twice would leave the box + * a whole row taller than the content it is holding. What is left of the measurement is * the part that belongs to the group rather than to its content — its own header, its * borders — and that is the part the prediction cannot know, because the prediction is * priced from medians and this group's chrome is its own. @@ -7351,8 +7360,9 @@

检视

* what the group holds now. * * @param {string} key - * @param {number} rows how many rows the group holds now - * @param {number} bands how many of them are compaction bands + * @param {number} rows how many rows the group holds now; the compaction bands are + * counted apart and are not rows + * @param {number} bands how many compaction bands the group holds now * @param {number} predicted the height the shape of the group predicts * @param {{head: number, row: number, band: number}|null} heights the calibrated heights * @returns {number} @@ -7745,7 +7755,7 @@

检视

// with the content it holds now; the prediction otherwise. A group that has // never been on screen has no measured height, and the prediction is the best // answer there is — the same answer the scrollbar is already giving. - holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights)); + holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length - bands, bands, predicted, build.heights)); } ledger.appendChild(section); ledgerContent.set(section, { key: key, turn: group.turn, entries: group.entries, bands: bands }); From b2e0be1992dd12d7c61a85981f9b65bf7e1f5d32 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 22:14:44 +0800 Subject: [PATCH 42/95] The guards that own a band being charged once The band cases of the ledger guard now add a band without removing a row, so that the two counts cannot be one count. A group measured with three rows and no band, now holding three rows and one band, is charged for one band; a group measured with two rows and one band, now holding one row and two bands, is charged for one row lost and one band gained. The case that hid the double charge is the one that removed a row and added a band in the same step: the number of entries did not change, so a difference taken over the entries was zero, and the band was charged once instead of twice. That case is kept, with the number the correct arithmetic gives. The control breaks six things: the difference is not charged, the measurement stops recording what it was a measurement of, the band count is not recorded, the measurement records the entry count instead of the row count, the build writes the height it computed, and the measurement is replaced by the prediction. Each is caught by name. The suite is green in full: 101 of 101 checks pass, every guard and every control among them, and the package is restored byte for byte afterwards. The guard that owns this rule is 49 assertions and its control is 29 mutations. --- .../tests/mutate-ledger-lifecycle.py | 26 +++++++++---- .../tests/verify-ledger-lifecycle.mjs | 39 +++++++++++++------ 2 files changed, 46 insertions(+), 19 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py index d090b8d3..496aee42 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -150,7 +150,7 @@ # A group the build is not holding is given a height to keep. ("a group the build is not holding is given no height to keep", - [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights));\n", "")], + [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length - bands, bands, predicted, build.heights));\n", "")], "a group the build is not holding is given a height to keep"), # The two states are what the rest of the page reads: the build says which, and the @@ -214,24 +214,36 @@ # A height that does not say what it was a measurement of cannot be brought up to date: # there is no difference to charge, and the group keeps a height that describes nothing. ("the height a group was measured at stops recording what it was holding", - [(" state.heightRows = group.entries.length;\n" - " state.heightBands = group.bands || 0;\n", + [(" var measuredBands = group.bands || 0;\n" + " state.heightRows = group.entries.length - measuredBands;\n" + " state.heightBands = measuredBands;\n", "")], "the height a group was measured at records what it was holding"), # A band is not a row. It has its own height, and a group that gained one is taller by a # band and not by a row — so a band that is not counted is a band that is not charged. ("the band a group was measured at stops being recorded", - [(" state.heightBands = group.bands || 0;\n", " state.heightBands = 0;\n")], + [(" state.heightBands = measuredBands;\n", " state.heightBands = 0;\n")], "a released group is charged for the band it gained"), + # The measurement has to record rows and bands as two counts. A band is an entry as well as + # a row, so recording the entry count charges every band once as a row and once as a band: + # a band would be charged 36 px and 38 px, and the box would stand a whole row taller than + # the content it is holding — the document lying about its own height in the other + # direction. The guard's band cases add a band without removing a row, so this shows up. + ("the measurement records the entry count instead of the row count", + [(" state.heightRows = group.entries.length - measuredBands;\n", + " state.heightRows = group.entries.length;\n")], + ["a released group is charged for the band it gained", + "a released group that lost a row and gained a band is charged for both"]), + # The measured height belongs to the observer: it is taken when the group is really laid # out. A build that stamped one would be writing down a height it never measured, and the # next rebuild would read its own guess back as if it were a measurement. ("the build writes the height it computed", - [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights));\n", + [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length - bands, bands, predicted, build.heights));\n", " ledgerStateOf(key).height = predicted;\n" - " holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights));\n")], + " holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length - bands, bands, predicted, build.heights));\n")], ["a rebuild does not change the height a group was measured at", "the build does not write a measured height"]), @@ -239,7 +251,7 @@ # priced from medians, and a group's real height runs to several times the median one, so # falling back on the prediction trades a real height for an average one. ("a released group is given the prediction instead of the height it was measured at", - [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length, bands, predicted, build.heights));\n", + [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length - bands, bands, predicted, build.heights));\n", " holdLedgerHeight(section, predicted);\n")], "a released group whose content has not changed keeps the height it was measured at"), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs index cb8537ee..3388af34 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -340,9 +340,17 @@ function measureGroup(h, key, entries, bands, height) { * document under the reader loses the rows that were added. * * The difference is priced locally: the measured height is kept, and only the rows that - * changed are charged at the calibrated row height. The three cases below are the three ways - * the content can differ — a row added, nothing changed, a band added — and the first two - * carry the same measured height, so a rule that cannot tell them apart is visible. + * changed are charged at the calibrated row height. The cases below are the ways the content + * can differ — a row added, nothing changed, a band added, a first band gained, a row gone + * and a band gained. + * + * The rows and the bands are two counts, not one. A band is an entry as well as a row, so a + * count of entries charges every band once as a row and once as a band; the row and band + * heights are close (36 px and 38 px on a real session), so a band charged twice leaves the + * box a whole row taller than the content it holds — the document lying about its own height + * in the other direction. The cases are built so that this shows up: the band cases add a + * band without removing a row, and the last one removes a row and adds a band at the same + * time, which is the one a single count cannot tell. * * The window has to be longer than one batch, because the first batch is priced from the * prediction and the groups after it from the measurement. Twenty-five groups is one batch. @@ -351,19 +359,22 @@ function measureGroup(h, key, entries, bands, height) { const h = buildHarness(); const s = h.sandbox; const groups = []; - for (let i = 0; i < 30; i += 1) { + for (let i = 0; i < 31; i += 1) { groups.push({ turn: { index: i + 1 }, entries: entriesOf(3, 0, 'g' + i) }); } - // Three groups measured for real, on three different contents: rows and no band, the same - // again, and rows with a band. + // Five groups measured for real, each on the content it is compared against. measureGroup(h, 'k26', entriesOf(3, 0, 'm26'), 0, 1000); measureGroup(h, 'k27', entriesOf(3, 0, 'm27'), 0, 1000); measureGroup(h, 'k28', entriesOf(2, 1, 'm28'), 1, 1000); - // And the content they hold now: a row added, nothing changed, a band added. + measureGroup(h, 'k29', entriesOf(3, 0, 'm29'), 0, 1000); + measureGroup(h, 'k30', entriesOf(2, 1, 'm30'), 1, 1000); + // And the content they hold now. groups[25].entries = entriesOf(4, 0, 'g25'); groups[26].entries = entriesOf(3, 0, 'g26'); - groups[27].entries = entriesOf(1, 2, 'g27'); - // The reader walked away from all three a long time ago, so they do not hold their rows. + groups[27].entries = entriesOf(2, 2, 'g27'); + groups[28].entries = entriesOf(3, 1, 'g28'); + groups[29].entries = entriesOf(1, 2, 'g29'); + // The reader walked away from all of them a long time ago, so they do not hold their rows. h.clock = 100000; rebuild(h, groups); @@ -375,6 +386,10 @@ function measureGroup(h, key, entries, bands, height) { sectionOf(h, 26).style.minHeight === '1000px'); assert('a released group is charged for the band it gained', sectionOf(h, 27).style.minHeight === '1060px'); + assert('a released group that gained its first band is charged for the band', + sectionOf(h, 28).style.minHeight === '1060px'); + assert('a released group that lost a row and gained a band is charged for both', + sectionOf(h, 29).style.minHeight === '1024px'); assert('a rebuild does not change the height a group was measured at', s.ledgerState.get('k26').height === 1000); } @@ -519,8 +534,8 @@ function measureGroup(h, key, entries, bands, height) { section.classList.add('is-released'); section.style.minHeight = '3643px'; - // Three rows and a band: four rows on screen, and a band that is priced as a band rather - // than as a row — so the counts the height was measured at are not the same number twice. + // Three rows and a band: four items on screen — and the band is a band rather than a row, + // so the two counts the height was measured at are different numbers. const entries = entriesOf(3, 1, 'r'); s.ledgerContent.set(section, { key: 'g1', turn: { index: 1 }, entries, bands: 1 }); s.fillLedgerGroup(section); @@ -534,7 +549,7 @@ function measureGroup(h, key, entries, bands, height) { assert('a group that has its rows back is remembered as live', s.ledgerState.get('g1').live === true); assert('the height a group was measured at records what it was holding', - s.ledgerState.get('g1').heightRows === 4 && s.ledgerState.get('g1').heightBands === 1); + s.ledgerState.get('g1').heightRows === 3 && s.ledgerState.get('g1').heightBands === 1); } { From 073b804a1bc1a71d2973f366ff8a127c6dc725d9 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 23:07:24 +0800 Subject: [PATCH 43/95] The height a released group keeps now survives the default view MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two things took the rule apart, and both of them were in the default view — the one nobody opts into, where the page opens on the newest 200 records and 200 records span fewer turns than the 25 a build does at a time, so the whole ledger is the first batch. The prediction was priced with the entry count as its row count. A compaction band is an entry as well as a row, so every band was priced twice: once at the row's height and once at the band's own. The two are close — 36 px and 38 px on a real session — so the box stood a whole row taller than the content it was holding, which is the document lying about its own height in the other direction. The rows are counted apart from the bands now, at both call sites. And the pass that follows the calibration sized the released groups with the prediction, which undid the rule on every group of the first batch: the calibration does not exist until the batch is built, so the height a group was measured at was still the one it had when it was measured, and the pass replaced it with the prediction. That pass writes the height that is held now — the measured one, brought up to date with what the group holds — and a group that was never measured still gets the prediction. --- .../mmc-trajectory/miniapp/client/index.html | 29 +++++++++++++++++-- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index 61561a02..a726b5a5 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -7749,7 +7749,11 @@

检视

// how a scrollbar ends up shorter than the table it is supposed to describe. On the // first batch build.heights is still null, so this is a no-op and sizeBuiltGroup() // below is what covers those groups. - var predicted = applyIntrinsicSize(section, group.entries.length, bands, build.heights); + // + // The rows are counted apart from the bands. A band is an entry as well as a row, so + // a count of entries would price every band twice — once at the row height and once + // at the band's own — and the box would stand taller than what it is holding. + var predicted = applyIntrinsicSize(section, group.entries.length - bands, bands, build.heights); if (!holds) { // The height the group had while it was holding its rows, brought up to date // with the content it holds now; the prediction otherwise. A group that has @@ -7788,8 +7792,21 @@

检视

for (var d = 0; d < dormant.length; d += 1) { var content = ledgerContent.get(dormant[d]); if (!content) continue; - var height = applyIntrinsicSize(dormant[d], content.entries.length, content.bands, build.heights); - holdLedgerHeight(dormant[d], height); + // The rows are counted apart from the bands, for the same reason as above: a band + // is an entry as well as a row, and the prediction prices it once, at its own + // height. + var dormantRows = content.entries.length - content.bands; + var dormantPredicted = applyIntrinsicSize(dormant[d], dormantRows, content.bands, build.heights); + // And the height that is held is the measured one, brought up to date with what + // the group holds now — the same rule the build applies to the groups it writes + // above. A group that was never measured has nothing to bring up to date, and the + // prediction is the answer there. + // + // Writing the prediction here instead would undo the rule on every group of the + // first batch, and in the default view the first batch is the whole window: the + // page opens on the newest 200 records, and 200 records span fewer turns than + // LEDGER_BATCH_GROUPS. + holdLedgerHeight(dormant[d], heldLedgerHeight(content.key, dormantRows, content.bands, dormantPredicted, build.heights)); } // The groups that were holding their rows are the ones that can be measured. A group // without rows measures as its header, and writing that down would collapse it. @@ -8428,6 +8445,12 @@

检视

/** * A group's predicted height from its own shape: one head, then rows and bands. * + * The rows and the bands are two counts: `rows` is the number of rows ALONE, and the + * compaction bands are counted in `bands`. A band is an entry as well as a row, so + * passing the entry count as the row count would price every band twice — once at the + * row height and once at the band's own — and the box would stand taller than what it + * is holding. + * * `auto` on the left is what lets the browser prefer the size it has already rendered once * the reader has scrolled past — so scrolling down and back does not accumulate drift. * From 28375e469c438455a17a31336b557a746594c2e0 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 23:07:24 +0800 Subject: [PATCH 44/95] The guards that own the held height in the first batch The height cases so far drove the helper directly, and the build path they ran was stubbed: the prediction answered with a fixed number, and the ledger's own query answered with an empty list, so the pass that follows the calibration never ran in the guard at all. A rule that is only proved away from the path that decides it is a rule the default view does not have. The new case runs the build over a window of 26 groups, so the first batch is all of the first 25 and the 26th is the second batch, with the real prediction in it and the ledger's query answering for the groups the build released. It reads what a released group ends up holding: with the stub's calibrated heights (40, 36 and 60) the measured height brought up to date with the row or the band it gained is 1036 px and 1060 px, and the prediction would have been 184 px. It also reads the prediction itself, so a band priced once is priced once: 232 px for two rows and two bands, and 208 px for three rows and a band. Three new controls: the prediction charges a band as a row as well, the build charges a band as a row as well, and the calibration overwrites the height a released group keeps. Each is caught by name, and the package is restored byte for byte afterwards. One existing control moved with the code it anchors on: it breaks the sizing of a group by removing the line that sizes it, and that line now carries the row count without the bands. A control whose anchor is gone reports it and stops, rather than passing a mutation it never ran. The guard is 54 assertions and its control is 32 mutations. --- .../tests/mutate-intrinsic-size.py | 2 +- .../tests/mutate-ledger-lifecycle.py | 28 +++++++ .../tests/verify-ledger-lifecycle.mjs | 73 ++++++++++++++++++- 3 files changed, 100 insertions(+), 3 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-intrinsic-size.py b/plugins/avatasia/mmc-trajectory/tests/mutate-intrinsic-size.py index bb69d290..514863b3 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-intrinsic-size.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-intrinsic-size.py @@ -79,7 +79,7 @@ # Every group after the calibration stops being sized, which is the same collapse applied to # 127 of 152 groups instead of 25 — and the count of sized groups is what the measurement reads. ("later groups are left unsized", - [(" var predicted = applyIntrinsicSize(section, group.entries.length, bands, build.heights);\n", "")], + [(" var predicted = applyIntrinsicSize(section, group.entries.length - bands, bands, build.heights);\n", "")], "a group is sized before it is appended, in the row loop"), # Predicting before there is anything to predict from. Dropping the guard does not write a diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py index 496aee42..24de3128 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -254,6 +254,34 @@ [(" holdLedgerHeight(section, heldLedgerHeight(key, group.entries.length - bands, bands, predicted, build.heights));\n", " holdLedgerHeight(section, predicted);\n")], "a released group whose content has not changed keeps the height it was measured at"), + + # The prediction prices the bands on their own height. A band is an entry as well as a row, + # so a prediction that takes the entry count as the row count prices every band twice: once + # at the row's height and once at the band's own. The two are close — 36 px and 38 px on a + # real session — so the box stands a whole row taller than the content it is holding, which + # is the document lying about its own height in the other direction. + # + # The two call sites are separate breaks: the one the pass after the calibration makes, and + # the one the build makes for every group it writes. + ("the prediction charges a band as a row as well", + [("var dormantPredicted = applyIntrinsicSize(dormant[d], dormantRows, content.bands, build.heights);", + "var dormantPredicted = applyIntrinsicSize(dormant[d], content.entries.length, content.bands, build.heights);")], + "a band is priced once in the predicted height"), + + ("the build charges a band as a row as well", + [("var predicted = applyIntrinsicSize(section, group.entries.length - bands, bands, build.heights);", + "var predicted = applyIntrinsicSize(section, group.entries.length, bands, build.heights);")], + "the prediction of a group with a band does not charge the band as a row"), + + # The pass that follows the calibration sizes the groups the build released, and what it + # writes is the height that is HELD — the measured one, brought up to date with what the + # group holds now. Writing the prediction there undoes the rule on every group of the first + # batch, and in the default view the first batch is the whole window: the page opens on the + # newest 200 records, and 200 records span fewer turns than LEDGER_BATCH_GROUPS. + ("the calibration overwrites the height a released group keeps", + [("holdLedgerHeight(dormant[d], heldLedgerHeight(content.key, dormantRows, content.bands, dormantPredicted, build.heights));", + "holdLedgerHeight(dormant[d], dormantPredicted);")], + "a released group keeps the height it was measured at in the first batch"), ] original = io.open(CLIENT, encoding="utf-8", newline="").read() diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs index 3388af34..caefbbc5 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -72,6 +72,10 @@ const LIFECYCLE_FNS = [ 'heldLedgerHeight', 'fillGroupRows', 'firstVisibleRow', + // The prediction itself, because the arithmetic of the prediction is half of what this file + // has to prove: a band is an entry as well as a row, and a prediction that counts it as both + // prices it twice. + 'applyIntrinsicSize', // The build itself, because the rule this file has to prove is about what the build // writes. A group the reader has left must not be kept alive by the rebuild — and the // source of a function cannot say whether it stamps the reader's lease or not. @@ -201,7 +205,7 @@ function harness() { * the measurement) are stubs, because the rule under test is about the lease and the lease is * decided before any of them run. */ -function buildHarness() { +function buildHarness(options) { const h = harness(); const s = h.sandbox; s.ledgerBuildToken = 1; @@ -215,7 +219,11 @@ function buildHarness() { s.DASH = '—'; s.turnKey = (turn) => 'k' + turn.index; s.buildTurnHead = () => makeEl('div'); - s.applyIntrinsicSize = () => 240; + // The prediction is stubbed by default, so a case that does not care about its arithmetic does + // not have to know the calibrated heights. A case that does care asks for the real one: the + // arithmetic of the prediction is half of what this file has to prove — a band is an entry as + // well as a row, and a prediction that counts it as both prices it twice. + if (!(options && options.realPrediction)) s.applyIntrinsicSize = () => 240; // The real calibration answers with the three heights it measured — the head, a row and a // band — and the build prices a difference in rows with them. A stub that answers with any // other shape is a stub of a different function. @@ -394,6 +402,67 @@ function measureGroup(h, key, entries, bands, height) { s.ledgerState.get('k26').height === 1000); } +/* ---------- 2c. the first batch is the whole window in the default view ---------- */ + +/* + * The page opens on the newest 200 records, and 200 records span fewer turns than a batch is + * long — so in the default view the whole ledger IS the first batch. The first batch is the + * one whose groups are measured against the prediction: the calibration does not exist until + * the batch is built, and the pass that follows the calibration is the one that sizes the + * groups the build released. + * + * So the rule "a released group keeps the height it had, brought up to date with what it holds + * now" has to hold THERE as well — otherwise it holds only in the full view, which is the + * opt-in one, and the page nobody opted in is the one with the rule missing. + * + * These cases run the build path with the real prediction in it, and read what the released + * group ends up holding. The prediction is priced from the calibrated heights (40, 36 and 60 in + * the stub above), so the numbers below are the ones that stub produces: a group of four rows + * is predicted at 40 + 4*36 = 184, and a group of two rows and two bands at 40 + 2*36 + 2*60 + * = 232 — a band priced once, at its own height, and not at the row's height as well. + */ +{ + const h = buildHarness({ realPrediction: true }); + const s = h.sandbox; + // The ledger's own query, so the pass that follows the calibration finds the groups the + // build released. The fake DOM answers every query with an empty list, and a build whose + // calibration pass finds nothing never sizes the groups it released — so the case would + // pass for the wrong reason. + s.ledger.querySelectorAll = (selector) => { + if (!/data-rows\s*=\s*"released"/.test(selector)) return []; + return s.ledger.children.filter((c) => c.getAttribute('data-rows') === 'released'); + }; + const groups = []; + for (let i = 0; i < 26; i += 1) { + groups.push({ turn: { index: i + 1 }, entries: entriesOf(3, 0, 'g' + i) }); + } + // Two groups measured for real, each on the content it is compared against: one with three + // rows and no band, one with two rows and a band. + measureGroup(h, 'k2', entriesOf(3, 0, 'm2'), 0, 1000); + measureGroup(h, 'k3', entriesOf(2, 1, 'm3'), 1, 1000); + // And what they hold now: the first gained a row, the second gained a band. + groups[1].entries = entriesOf(4, 0, 'g1'); + groups[2].entries = entriesOf(2, 2, 'g2'); + // The group of the second batch, which has a band in it and was never measured. + groups[25].entries = entriesOf(3, 1, 'g25'); + // The reader walked away a long time ago, so they do not hold their rows. + h.clock = 100000; + + rebuild(h, groups); + s.buildStep(); + + assert('a released group keeps the height it was measured at in the first batch', + sectionOf(h, 1).style.minHeight === '1036px'); + assert('a released group is charged for the band it gained in the first batch', + sectionOf(h, 2).style.minHeight === '1060px'); + assert('the height a released group keeps is not the prediction', + sectionOf(h, 1).style.minHeight !== '184px'); + assert('a band is priced once in the predicted height', + sectionOf(h, 2).style.containIntrinsicSize === 'auto 232px'); + assert('the prediction of a group with a band does not charge the band as a row', + sectionOf(h, 25).style.containIntrinsicSize === 'auto 208px'); +} + /* ---------- 3. a group the reader left gives its rows up, and the count is bounded ---------- */ { From fabda814ad670cb5115f9362510b04b3f62b2d85 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 23:07:25 +0800 Subject: [PATCH 45/95] A limit of zero is zero records, on both paths MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cap on the fold over the first N records was checked after the push, so a limit of zero yielded one version. The page asks about the records it holds and the service folds its own first N the same way; with a limit of zero the two folds were over different sets — the page's empty set and the service's one record — so the fingerprint never matched and the request was refused for ever. The other path, which answers from the stored segments, folds the empty set for the same request and answers it. One question, two answers. The cap is checked before the push now, and the sibling fold in the segment reader gives the same reading of a limit of zero. A missing, negative or fractional limit still means every record, so the callers that ask for everything are unaffected. The guard that owns the rule now writes its expectation out by hand — the versions of the first three records and the fingerprint of that list — because a guard that folds its expectation with the function it polices cannot see an off-by-one inside it: the mistake moves the expectation and the answer together and the guard stays green. A control breaks the cap back into its old place, and the rule is reported by name. --- .../miniapp/node/search-segments.mjs | 17 +++- .../mmc-trajectory/miniapp/node/server.mjs | 8 +- .../tests/mutate-trajectory-cache.py | 87 +++++++++++++++++-- .../tests/verify-incremental.mjs | 22 ++++- 4 files changed, 122 insertions(+), 12 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs index f7f6833f..5455714c 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs @@ -461,14 +461,18 @@ export function createSegmentStore(root) { * parsed answer agree. * * @param {string} surface - * @param {number} [limit] stop after this many records + * @param {number} [limit] stop after this many records; zero is zero records, and a missing, + * negative or fractional limit is every record * @returns {Array} */ export function segmentVersions(surface, limit) { /** @type {Array} */ const out = []; if (typeof surface !== 'string' || surface === '') return out; - const wanted = Number.isInteger(limit) && limit > 0 ? limit : Infinity; + // The same rule as the payload's own fold over the first N records: an explicit limit of zero + // is zero records, and a missing, negative or fractional limit is every record. The two are two + // readings of one fact — the versions of the first N records — so they have to agree. + const wanted = Number.isInteger(limit) && limit >= 0 ? limit : Infinity; let start = 0; while (start <= surface.length && out.length < wanted) { const nl = surface.indexOf('\n', start); @@ -509,8 +513,13 @@ function versionInAnswer(answer) { /** * Whether the generation a segment was built from is still the generation on disk. * - * Identity, not mtime: a sealed file is not written to, and a reader that treated a touch as a - * change would rebuild correct bytes on every poll. + * Identity is the file name, the generation, the size, the mtime, whether the read was cut short, + * and the revision when both sides carry one: all of them have to agree for the segment to still + * be about the same bytes. + * + * The mtime IS part of that, deliberately. The parsed-rows cache and the trajectory entry both + * key on (size, mtime), so a file rewritten in place — the same length, different bytes — is a + * different file to everything else in here, and a touch is a change here as much as it is there. * * @param {SegmentManifest['source']} stored * @param {SegmentManifest['source']} live diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index f71424a7..f343b569 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -464,7 +464,8 @@ function hex8(value) { * a different order is a different answer to 「这批记录是哪一批」. * * @param {{ turns?: Array }} payload - * @param {number} [limit] stop after this many records + * @param {number} [limit] stop after this many records; zero is zero records, and a missing, + * negative or fractional limit is every record * @returns {Array} */ export function payloadRecordVersions(payload, limit) { @@ -476,8 +477,11 @@ export function payloadRecordVersions(payload, limit) { const records = turn && Array.isArray(turn.records) ? turn.records : []; for (const record of records) { if (!record || typeof record !== 'object') continue; - out.push(typeof record.v === 'string' ? record.v : ''); + // The cap is checked BEFORE the push, so a limit of zero is zero records. Checked after the + // push it would yield one version, and the fold over the page's empty set would then never + // match the service's own fold over its empty set — the request would be refused for ever. if (out.length >= cap) return out; + out.push(typeof record.v === 'string' ? record.v : ''); } } return out; diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py index 77f7e2c7..1b87a040 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py @@ -17,6 +17,11 @@ ROOT = HERE.parent SERVER = ROOT / "miniapp" / "node" / "server.mjs" PROBE = HERE / "verify-trajectory-cache.mjs" +# A case may name its own probe, and this is the other one: the cap on the fold over the first +# N records is owned by `verify-incremental.mjs`, which is where the assertion about it lives. A +# case that ran this file's probe instead would be asking a guard that never heard of the rule, +# and would report the mutation as survived. +INCREMENTAL_PROBE = HERE / "verify-incremental.mjs" FAMILY_EVICT = """ for (const [existingKey, existing] of entries) { if (existingKey !== key && existing.family === family) entries.delete(existingKey); @@ -359,6 +364,25 @@ " sendJsonBody(response, 200, cached.body, { etag, revalidate: true });\n return;\n", " // the body is rebuilt below\n", ), + # ---- the cap on the fold over the first N records --------------------------------- + # + # The cap is checked BEFORE the push, so a limit of zero is zero records. Checked after the + # push it is one, and the fold over the page's empty set then never matches the service's fold + # over its empty set -- so a request for a tail over no records is refused for ever. + # + # The two readings of "the versions of the first N records" are in two files -- this one in + # `server.mjs`, the other in `search-segments.mjs` -- and they have to agree, which is why the + # case below names the guard that owns the rule rather than this file's own probe. + ( + "the cap on the fold is checked after the push, so a limit of zero still yields one version", + SERVER, + " if (out.length >= cap) return out;\n" + " out.push(typeof record.v === 'string' ? record.v : '');\n", + " out.push(typeof record.v === 'string' ? record.v : '');\n" + " if (out.length >= cap) return out;\n", + "a limit of zero records is a limit of zero, not of one", + INCREMENTAL_PROBE, + ), ] @@ -375,9 +399,9 @@ def write_exact(path, text, newline): path.write_bytes(text.encode("utf-8")) -def run_probe(): +def run_probe(probe=PROBE): proc = subprocess.run( - ["node", str(PROBE)], + ["node", str(probe)], cwd=str(ROOT), capture_output=True, text=True, @@ -387,18 +411,61 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") +def reported_rules(output): + """The rules a probe named as broken. + + Two guards, two formats: `verify-trajectory-cache.mjs` prints ` - ` and + `verify-incremental.mjs` prints `FAIL `. Both name the rule, so a control that could + only read one of them would read the other's output as a failure that named nothing. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(text[2:].strip()) + elif text.startswith("FAIL "): + names.append(text[5:].strip()) + return names + + +def probes_used(): + """The probes the cases ask for, in the order they first appear.""" + used = [] + for case in MUTATIONS: + probe = case[5] if len(case) > 5 else PROBE + if probe not in used: + used.append(probe) + return used + + def main(): - code, output = run_probe() + code, output = run_probe(PROBE) if code != 0: print("baseline probe does not pass; fix that first") print(output) return 1 print("baseline clean") + # Every probe a case names is baselined before any mutation is applied: a probe that is + # already red cannot be read as a catch, and a control that ran one would report a mutation + # as survived for a reason that has nothing to do with it. + for probe in probes_used(): + if probe == PROBE: + continue + code, output = run_probe(probe) + if code != 0: + print(f"baseline {probe.name} does not pass; fix that first") + print(output) + return 1 + print(f"baseline {probe.name} clean") + originals = {path: path.read_bytes() for path in {m[1] for m in MUTATIONS}} missed = [] unnamed = [] - for label, path, old, new in MUTATIONS: + for case in MUTATIONS: + label, path, old, new = case[:4] + expect = case[4] if len(case) > 4 else None + probe = case[5] if len(case) > 5 else PROBE text, newline = read_exact(path) if old not in text: print(f"SKIP {label}: anchor not found in {path.name}") @@ -406,13 +473,23 @@ def main(): continue write_exact(path, text.replace(old, new, 1), newline) try: - code, output = run_probe() + code, output = run_probe(probe) finally: path.write_bytes(originals[path]) if code == 0: print(f"MISSED {label}: the probe still passed") missed.append(label) continue + if expect is not None: + # The case names the rule it is about, so the probe has to name that rule and not + # some other one that happened to go red in the same run. + names = reported_rules(output) + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") + continue # A probe that failed without naming a rule has not said WHICH rule broke, so it has not # shown that the guard owns this mutation. It is weaker than "caught", and it is reported # as its own thing rather than folded into the count of survivors. diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-incremental.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-incremental.mjs index d63f02d8..4d07ede7 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-incremental.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-incremental.mjs @@ -390,7 +390,27 @@ const F7_BODY = { ], }; const F7_COUNT = 3; -const F7_FINGERPRINT = recordSetFingerprint(payloadRecordVersions(F7_BODY, F7_COUNT)); +// The expectation is written out by hand, not folded from the function it polices. A guard that +// computes its expected fingerprint with `payloadRecordVersions` cannot see an off-by-one inside +// that function: the mistake would move the expectation and the answer together, and the guard +// would stay green. So the versions are listed here, and the fingerprint is their literal. +const F7_VERSIONS = ['1111111111111111', '2222222222222222', '3333333333333333']; +const F7_FINGERPRINT = 'c6f81774c9d5df24'; + +{ + assert('the fingerprint of the hand-written list is the one the page is asked for', + recordSetFingerprint(F7_VERSIONS) === F7_FINGERPRINT); + assert('the first N records are the ones the service folds over', + JSON.stringify(payloadRecordVersions(F7_BODY, F7_COUNT)) === JSON.stringify(F7_VERSIONS)); + // A limit of zero is zero records. The cap used to be checked after the push, so a limit of + // zero yielded one version — and the fold over the page's empty set then never matched the + // service's fold over its empty set, and the request was refused for ever. + assert('a limit of zero records is a limit of zero, not of one', + payloadRecordVersions(F7_BODY, 0).length === 0); + assert('and a limit of one record is the first record\'s version', + payloadRecordVersions(F7_BODY, 1).length === 1 + && payloadRecordVersions(F7_BODY, 1)[0] === F7_VERSIONS[0]); +} { assert('a tail is answered to a page holding the records the service has', From b2a9bd964dc4209457985e7eaae6bdaeee5d0420 Mon Sep 17 00:00:00 2001 From: avatasia Date: Thu, 22 Oct 2026 23:07:26 +0800 Subject: [PATCH 46/95] The badge says what the runtime marked it, not what it did not do MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The picker marks a lightweight conversation 「轻量」, and the marker comes from the runtime's own record of the session. Both READMEs called that session "tool-less" — a claim about the session's contents that nothing in this app checks, because the app never looks at whether a session used a tool. The sentence now says where the marker comes from, and stops claiming the rest. The comment above the reader that decides whether a segment is still about the same bytes said the opposite of the code next to it: it promised an identity that did not include the mtime, while the code compares the mtime on purpose, because the parsed-rows cache and the trajectory entry both key on size and mtime and a file rewritten in place is a different file to both. The comment now states the rule the code implements. --- plugins/avatasia/mmc-trajectory/README.md | 2 +- plugins/avatasia/mmc-trajectory/README.zh-CN.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/README.md b/plugins/avatasia/mmc-trajectory/README.md index fe3f23cc..3c64fbce 100644 --- a/plugins/avatasia/mmc-trajectory/README.md +++ b/plugins/avatasia/mmc-trajectory/README.md @@ -132,7 +132,7 @@ The runtime database is opened in read-only mode. If it is missing, unreadable, - The ledger does not use virtual scrolling; every rendered row is a real element. The page therefore opens on the newest 200 records, and the window bar button switches to the whole session. The button names what pressing it will do, not the current state: 「显示全部」 at the default and 「显示最新 200 条」 once everything is showing, while the count beside it reports what is on screen. That full view is opt-in for a reason: on a 9,900-record session it is roughly 57,000 elements rebuilt every 2.5s poll, which was measured to freeze the page rather than be read. 「向上加载更早的 N 条」 widens the window in steps and a poll does not undo it. A sticky window bar under the overview strip offers that choice behind four explicit controls: switch the strip between equal width and a real time axis, load 200 earlier records, jump back to the latest 200, and show everything. "Show everything" builds every record in the session into the DOM at once and gets noticeably slower on large sessions; incoming records do not knock it back down to 200. - `messageCount` and `turnCount` in the session picker are estimates derived from a bounded prefix of the file; exact values come from the selected session. - The session picker is **two selects in a cascade**. The upper one, 「会话」, lists conversations only — the sessions whose parent, followed up the chain, is a conversation. A session with no parent is one of them, and so is one whose parent names no session, whose parent chain ends in a cycle, or whose parent is itself a run: the layer follows the chain rather than the one column, and a row that ends up in the conversation layer carries no parent. The lower one, 「子代理」, lists the runs of the conversation that was picked: it is **empty by default**, it has values once a conversation is picked, and it is empty and disabled while the page follows the newest conversation. Measured on this machine, 13 of the 20 rows were sub-agent runs, so a reader could not tell which row was their own conversation; those 13 runs were also where that conversation actually happened. Nothing is hidden, only moved — the runs are not rows of the conversation list any more, they are rows of the run list under it, and the conversation list is now the twenty most recent conversations rather than the twenty most recent sessions of any kind. Every run is listed, most recently active first, and pressing one switches the page to it. The layer is decided by the chain of parents, which is independent of the 「定时 / 任务 / 轻量」 markers. The parent's name is still not written out: the parent id comes down with the row, and the page groups by that one id rather than doing a lookup that could fail quietly. Picking a run shows **that run's own trajectory** — a run is a session with its own transcript, and the parent's transcript does not contain its turns — and the conversation select stays on its parent. -- The picker also says what a session **is**, not only what it is called: a scheduled run carries 「定时」, a background task 「任务」, and a tool-less lightweight conversation 「轻量」, so none of them can be mistaken for a conversation you wrote. They stay in the list and stay selectable — a marker is a marker, not a filter. The conversation the runtime database reports as current comes first, then the sessions of its own workspace, then the rest; the workspace path and the raw purpose string are compared on the server and never reach the page. +- The picker also says what a session **is**, not only what it is called: a scheduled run carries 「定时」, a background task 「任务」, and a lightweight conversation 「轻量」 — the markers come from the runtime's own record of the session, not from anything this app checks — so none of them can be mistaken for a conversation you wrote. They stay in the list and stay selectable — a marker is a marker, not a filter. The conversation the runtime database reports as current comes first, then the sessions of its own workspace, then the rest; the workspace path and the raw purpose string are compared on the server and never reach the page. - Light theme token coverage is verified — every `--mcode-*` token the page consumes is defined for both themes — but its rendered appearance was not visually checked; the page was exercised under a dark system preference. ## Tested environment diff --git a/plugins/avatasia/mmc-trajectory/README.zh-CN.md b/plugins/avatasia/mmc-trajectory/README.zh-CN.md index e16f9c18..29eb0f4c 100644 --- a/plugins/avatasia/mmc-trajectory/README.zh-CN.md +++ b/plugins/avatasia/mmc-trajectory/README.zh-CN.md @@ -118,7 +118,7 @@ MiniMax Code 不会告诉 Mini App 是哪段对话打开了它的页面：传给 - 台里没有用虚拟滚动，每一行都是真实的 DOM 元素。页面默认只展示最新 200 条，窗口条上的按钮可以切到全量；全量是可选的，因为它很贵 —— 在这个会话上约 9,900 条、约 5.7 万个元素，每 2.5 秒轮询重建一次，实测会直接卡死页面。总览条正下方的吸顶窗口栏提供三个显式控件：「总览轴」「向上加载更早的 200 条」，以及在最新 200 条与全量之间切换的那一个按钮（它写的是按下去会变成什么，默认为「显示全部」，全量时为「显示最新 200 条」；当前窗口由右边的计数条报告，两者回答的是不同问题）。「总览轴」只改总览条的横轴——等宽（每条记录一样宽，完全忽略时间）或真实时间轴（宽度等于该记录的真实耗时）；它不筛选、不排序，台账里的行和页面上的任何计数都不变。「向上加载更早的 200 条」每点一次就多加一页，轮询不会把已经加宽的窗口重置回去；点切换按钮也不会自动滚动页面，读者正在看的那一行会留在原地。 - 会话选择器里的 `messageCount` / `turnCount` 是从文件的有界前缀估算的；精确值以选中会话后展示的为准。 - 会话选择器是**级联的两个下拉框**。上面那个「会话」只列主会话 —— 沿父链往上走、终点是一段主会话的那些。父会话为空的是主会话；父会话指向一个不存在的会话、父链成环、或者父本身是子代理的，也是主会话 —— 层级看的是链，不是那一列，而落到会话层的行不带父会话。下面那个「子代理」只列所选那段对话的子代理，**默认是空**，选中主会话之后才有值，跟随最新时是空的且不可选。实测本机 20 行里有 13 行是子代理，读者根本分不出哪一行是自己的对话，而那 13 行恰恰就是这段对话真正发生的地方。它们没有被藏起来，只是从「平铺在会话里」变成「会话列表下面的子代理列表」：会话列表现在列的是最近 20 段主会话，而不是最近 20 个任意种类的会话；子代理有多少列多少，按各自最后一次活跃的时间倒序，最活跃的排在最上面，点其中一条就切到那一条。层级由父链决定，与「定时 / 任务 / 轻量」那套标记互相独立。父会话的名字仍然不写：父 id 由服务端随这一行带下来，页面按同一个 id 归组，不做一次可能悄悄失败的查找。选了子代理就显示**那一次运行自己的**轨迹 —— 每个子代理是一段独立的会话，有自己的 transcript，父会话的轨迹里并不包含它的轮次；此时上面那个下拉框停在它的父会话上。 -- 会话选择器不只说会话**叫什么**，还说它**是什么**：定时任务标「定时」，后台任务标「任务」，没有工具的轻量会话标「轻量」，于是它们不会被误认成你自己开的对话。它们仍然在列表里、仍然可以选 —— 标记是标记，不是过滤。运行时库报告为当前的那段对话排在最前，然后是同一工作区的会话，再然后是其它；工作区路径和 purpose 原文只在服务端比较，不会发到页面。 +- 会话选择器不只说会话**叫什么**，还说它**是什么**：定时任务标「定时」，后台任务标「任务」，轻量会话标「轻量」（标记来自运行时对会话自己的记录，不是这个应用检查出来的），于是它们不会被误认成你自己开的对话。它们仍然在列表里、仍然可以选 —— 标记是标记，不是过滤。运行时库报告为当前的那段对话排在最前，然后是同一工作区的会话，再然后是其它；工作区路径和 purpose 原文只在服务端比较，不会发到页面。 - 浅色主题的 token 覆盖已验证（页面用到的每个 `--mcode-*` 变量在两套主题下都有定义），但渲染外观未做视觉检查；功能验收是在深色系统偏好下进行的。 ## 测试环境 From 9f5082241d4068ab4b6860ce8a937bcb29029822 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 00:37:44 +0800 Subject: [PATCH 47/95] =?UTF-8?q?=E5=AE=A2=E6=88=B7=E7=AB=AF=EF=BC=9A?= =?UTF-8?q?=E6=B4=BB=E7=BB=84=E9=87=8F=E5=88=B0=E7=9A=84=E9=AB=98=E5=BA=A6?= =?UTF-8?q?=E8=BF=9B=E7=8A=B6=E6=80=81=EF=BC=9B=E6=8C=81=E6=9C=89=E8=A1=8C?= =?UTF-8?q?=E7=9A=84=E7=BB=84=E6=95=B0=E4=B8=8A=E9=99=90=E5=9C=A8=E8=AE=BF?= =?UTF-8?q?=E5=AE=A2=E8=B7=AF=E5=BE=84=E4=B8=8A=E4=B9=9F=E6=88=90=E7=AB=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 第 29 轮第三遍评审报上来的两条客户端发现，这一笔是实现。 活组量到的高度没进状态 观察者回调在「行已经填好」时提前返回，而记录高度的三行在它后面。于是 build 出来就已经 live 的那一组（窗口的第一组总是 live）一次也没把测量值写进状态。下一次重建丢掉那个 DOM，组被释放时就只剩预测值：内容一行没变，预留高度却从量到的 1000 px 变成 208 px （这两个数是评审用守卫自己的标定费率做的例子）。 改法：把「填行」与「记测量」拆成两件事。行只填一次，测量两条路都记，并且记下它是 在什么内容上量的——行数与带数分开数，带只按带计一次。 持有行的组数上限只在重建路径上执行 上限的封顶动作只有一个执行点，而那个执行点只被重建路径调用。两次重建之间，访客 路径把访问过的组逐个标成 live 并填行，不施加任何上限，于是访问 7 个组就是 8 个组 同时持行，而上限是 6。 改法：封顶动作收成一个，两条路径共用同一个决定。访客路径上被封顶释放的组，当场丢掉 它的行并保留它量到的高度——否则文档会在读者脚下缩。读者刚到的那一组永远不是被释放 的那一组，最旧的先走。 这一条本来有两条路可选：在访客路径上也执行上限，或者把承诺重新定义成「只在重建时」的 目标并说明资源后果。选的是前者：这条承诺的价值恰恰是它在读者做任何事的时候都成立， 缩成「每次重建后」就是把它留给读者自己推。 代价写明：读者在两次重建之间（轮询 2.5 s 一趟）快速滚过很多组时，那些组会当场交出 它们的行，滚回去时行再回来。持有量的上界因此从「读者在一次轮询间隔内访问过的组数」 变成那 6 组——这比原来更紧，不是更松。 --- .../mmc-trajectory/miniapp/client/index.html | 156 ++++++++++++++---- 1 file changed, 125 insertions(+), 31 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index a726b5a5..a49eb7eb 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -7207,8 +7207,8 @@

检视

* * Three rules, and the order matters. A group the reader has left goes when the idle * timeout has passed; when more groups are live than the cap allows, the oldest goes - * first; and the group the window is anchored on is none of the reader's visits at all, - * so it is counted rather than timed. + * first; and the group the caller is holding on to is none of the reader's visits at + * all, so it is counted rather than timed. * * That last one is what keeps the bound about the rows on screen. The anchored group * holds its rows whatever the reader is doing — it is where the reader arrives, and every @@ -7216,34 +7216,53 @@

检视

* would leave one group holding its rows outside the count. Counting it here makes the * two sets the same set: the groups that are live are the groups that hold their rows. * - * This only decides. The rows themselves are taken away by the next build, which is the - * one place that writes them, so there is no second writer of the rows and nothing to keep - * in step. + * One decision and two callers, and that is the point of it. The bound is about what the + * reader is holding at this moment, not only about what a rebuild leaves behind: a reader + * who walks through more groups than the cap allows between two rebuilds is holding all + * of them until the next poll comes round. So the rebuild and the visit both come through + * here, and they cannot disagree about which groups go — the same order, the same count, + * the same group held back. * - * @param {string} keepKey the group the window is anchored on, which is never released + * This decides. The rows are taken away by the build, which writes the released groups + * as it writes the table, and by the visit, which takes the rows out of a table that is + * already on screen. + * + * @param {string} keepKey the group that holds its rows whatever else happens: the + * window's first group on a rebuild, the group the reader has just reached on a visit + * @returns {string[]} the groups that have to give up their rows */ function releaseStaleGroups(keepKey) { var now = Date.now(); var live = []; - // The anchored group is counted first and with the current time, so it is never the - // oldest one the cap reaches for. Its lease is not consulted: it does not have one. - ledgerStateOf(keepKey).live = true; - live.push({ key: keepKey, at: now }); + var doomed = []; ledgerState.forEach(function (state, key) { if (key === keepKey) return; if (!state.live) return; if (now - state.lastSeen >= LEDGER_IDLE_MS) { state.live = false; + doomed.push(key); return; } live.push({ key: key, at: state.lastSeen }); }); - if (live.length <= LEDGER_LIVE_GROUPS) return; + // The group the caller is holding on to is counted with the current time, so it is + // never the oldest one the cap reaches for. Its lease is not consulted: it does not + // have one. + // + // It is counted after the others, so that a visit in the same millisecond as another + // one — a tie the clock cannot break — cannot put it at the head of the queue. The + // group the reader has just reached is the one the cap must not take. + ledgerStateOf(keepKey).live = true; + live.push({ key: keepKey, at: now }); + if (live.length <= LEDGER_LIVE_GROUPS) return doomed; live.sort(function (a, b) { return a.at - b.at; }); for (var i = 0; i < live.length - LEDGER_LIVE_GROUPS; i += 1) { var state = ledgerState.get(live[i].key); - if (state) state.live = false; + if (!state) continue; + state.live = false; + doomed.push(live[i].key); } + return doomed; } /** @@ -7252,6 +7271,9 @@

检视

* The group is in view, so this is what the reader is about to read, and the rows go in * before the next paint — so there is no frame in which the group is empty. * + * The rows go in once. A group that is already holding them is not built a second time, + * and the second copy of every row is the whole cost of getting it wrong. + * * @param {HTMLElement} section */ function fillLedgerGroup(section) { @@ -7260,33 +7282,104 @@

检视

var state = ledgerStateOf(group.key); state.live = true; state.lastSeen = Date.now(); - if (section.getAttribute('data-rows') === 'live') return; - var list = section.querySelector('ul.row-list'); - if (!list) return; - fillGroupRows(list, group); - section.setAttribute('data-rows', 'live'); - section.classList.remove('is-released'); - section.style.minHeight = ''; + var alreadyLive = section.getAttribute('data-rows') === 'live'; + if (!alreadyLive) { + var list = section.querySelector('ul.row-list'); + if (!list) return; + fillGroupRows(list, group); + section.setAttribute('data-rows', 'live'); + section.classList.remove('is-released'); + section.style.minHeight = ''; + } + // The height the group really has, taken on both paths — see measureLedgerGroup(). + // A group that is already holding its rows and has a height of its own has nothing + // new to say: nothing about its content has changed, so the height it was measured + // at is the height it has. + if (!alreadyLive || !state.height) measureLedgerGroup(section, group); + // The cap, on the visit path as well as on the rebuild: the number of groups holding + // their rows is bounded at any moment, not only right after a rebuild. + releaseLedgerRows(releaseStaleGroups(group.key)); + } + + /** + * Record how tall a group really is, and what it was holding when it was measured. + * + * The measurement belongs to the observer, because the observer is what sees the group + * laid out for real. The build never measures: a group it holds has no height of its own + * until somebody looks at it, and a build that stamped one would be writing down a + * height it never measured. + * + * Both paths record it. The group the build made live — the first group of the window is + * always live — is visited by the observer like any other, and it is laid out for real at + * that moment. Without a measurement here it keeps no height of its own, and when the + * window widens and the group is released it falls back on the prediction: the document + * under the reader loses the height the group really had, over a group whose rows and + * bands did not change by one. + * + * The counts go with the height, because a height is a measurement OF something. The group + * is keyed by its turn, and the turn keeps its key while its content changes — a late + * reply, a band from a compaction — so a height that is not read together with its counts + * describes a content that is no longer there. + * + * The two counts are rows and bands, and they are counted apart. A band is an entry as + * well as a row, so a count of entries would charge every band once as a row and once as + * a band — the band at 36 px and 38 px would be charged 74 px, and the box would stand + * taller than the content it is holding. + * + * @param {HTMLElement} section + * @param {{key: string, entries: any[], bands?: number}} group + */ + function measureLedgerGroup(section, group) { + var state = ledgerStateOf(group.key); // The height it really has, now that it is laid out for real. A group the browser has // skipped measures by its own contain-intrinsic-size, so this reads the prediction back // rather than the truth — which is why it is the remembered height and not the only one. sizeBuiltGroup(section); state.height = Math.round(section.getBoundingClientRect().height); - // What it was holding when it was measured, so that the height can be brought up to - // date with the content the group holds now. The group is keyed by the turn, and the - // turn keeps its key while its content changes — a late reply, a band from a - // compaction — so a height that is not read together with its counts describes a - // content that is no longer there. - // - // The two counts are rows and bands, and they are counted apart. A band is an entry - // as well as a row, so a count of entries would charge every band once as a row and - // once as a band — the band at 36 px and 38 px would be charged 74 px, and the box - // would stand taller than the content it is holding. var measuredBands = group.bands || 0; state.heightRows = group.entries.length - measuredBands; state.heightBands = measuredBands; } + /** + * Take the rows out of the groups that have to give up their rows. + * + * The rebuild writes the released groups as it builds the table, so there is nothing to do + * there. A visit happens between two rebuilds, and the groups it releases are in a table + * that is already on screen: their rows have to go now, or the reader is holding more + * rows than the cap allows until the next poll comes round. + * + * The height is kept. The rows are the DOM; the height is the reader's place in a session + * they have scrolled through, and a group that has given up its rows has to keep it or + * the document shrinks under them. + * + * @param {string[]} keys the groups that have to give up their rows + */ + function releaseLedgerRows(keys) { + if (!keys.length) return; + var sections = dom.ledger.querySelectorAll(':scope > section.turn-group'); + for (var i = 0; i < sections.length; i += 1) { + var section = sections[i]; + var content = ledgerContent.get(section); + if (!content) continue; + if (keys.indexOf(content.key) < 0) continue; + var bands = content.bands || 0; + var rows = content.entries.length - bands; + // The height the group keeps is the one it was measured at. It is read before the + // rows go, because after them the box is the group's own head and nothing else. + var measured = Math.round(section.getBoundingClientRect().height); + var list = section.querySelector('ul.row-list'); + if (list) clear(list); + section.setAttribute('data-rows', 'released'); + section.classList.add('is-released'); + // The measured height, brought up to date with what the group holds now; the height + // read above otherwise. There is no calibration here to price a difference with — + // the calibration belongs to the build that is running — so the height the group + // was measured at is kept as it is, and the next rebuild brings it up to date. + holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, null)); + } + } + /** * Watch the groups, so a group the reader scrolls to gets its rows. * @@ -7693,8 +7786,9 @@

检视

for (var b = 0; b < groups.length; b += 1) buildTotal += groups[b].entries.length; // The groups the reader has left give up their rows, and the ones past the cap give them - // up oldest first. Decided here and applied by this build — the one place that writes - // rows — so there is no second writer to keep in step with this one. + // up oldest first. The same decision the visit path makes, so the two cannot disagree + // about which groups go; applied by this build, which writes the released groups as it + // writes the table. if (groups.length) releaseStaleGroups(turnKey(groups[0].turn)); function buildStep() { From 948c5e5e4058c27955e0063a968a6af6de6e0fd4 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 00:37:44 +0800 Subject: [PATCH 48/95] =?UTF-8?q?=E5=AE=A2=E6=88=B7=E7=AB=AF=EF=BC=9A?= =?UTF-8?q?=E6=B4=BB=E7=BB=84=E6=B5=8B=E9=87=8F=E4=B8=8E=E8=AE=BF=E5=AE=A2?= =?UTF-8?q?=E8=B7=AF=E5=BE=84=E5=B0=81=E9=A1=B6=E7=9A=84=E5=AE=88=E5=8D=AB?= =?UTF-8?q?=E4=B8=8E=E5=8F=98=E5=BC=82?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 上一笔的实现配守卫。四条行为断言，四条按名字抓住的变异控制。 新增的行为断言（tests/verify-ledger-lifecycle.mjs，54 -> 58 条）： - a group built live keeps its measured height when a wider window releases it - reader visits cannot increase the held row groups past the ceiling between rebuilds - a group the cap released holds no rows - a group the cap released keeps its reserved height 新增的变异控制（tests/mutate-ledger-lifecycle.py，32 -> 36 条，每条按名字抓住）： - a group the build made live never records the height it was measured at - reader visits do not enforce the cap on the groups holding their rows - a group the cap released keeps its rows - a group the cap released keeps no reserved height 证明：四条新断言在未修的代码上全红（[ledger-lifecycle] 54 passed, 4 failed），修后 58/58。 守卫为此把两个新函数可选地提取出来，这样它能跑在修前的客户端上——证明本身就是这个 守卫跑出来的。 顺带修好的守卫缺陷（不是评审报的，是写这条断言时撞上的）：守卫的假 DOM 里 el() 桩 丢掉了 class 名，于是 build 出来的 section 没有行列表，访客路径在沙箱里根本找不到它， 一直是个空操作。也就是说即便评审报的那条真的存在，守卫也永远看不见它——因为守卫 根本没让那条路跑起来。桩与那条 section 查询一并修好。 两条既有变异的锚随被指的代码移动了（封顶那行、行只填一次那行），变异名不变。 --- .../tests/mutate-ledger-lifecycle.py | 38 ++++- .../tests/verify-ledger-lifecycle.mjs | 139 +++++++++++++++++- 2 files changed, 171 insertions(+), 6 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py index 24de3128..7bfc4f36 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -74,8 +74,8 @@ # The bound is what makes the memory bounded rather than merely reduced: a reader who # scrolls through three hundred groups in a minute has been on all of them. ("the number of groups holding their rows is never bounded", - [(" if (live.length <= LEDGER_LIVE_GROUPS) return;\n", - " if (true) return;\n")], + [(" if (live.length <= LEDGER_LIVE_GROUPS) return doomed;\n", + " if (true) return doomed;\n")], "the number of groups holding their rows is bounded"), # The oldest goes first, which is the order the reader would choose. Sorting the other way @@ -123,7 +123,8 @@ # A group that is already holding its rows is filled by the build and then again by the # observer, and the second pass appends every row a second time. ("a group that is already holding its rows is filled a second time", - [(" if (section.getAttribute('data-rows') === 'live') return;\n", "")], + [(" var alreadyLive = section.getAttribute('data-rows') === 'live';\n", + " var alreadyLive = false;\n")], "a group that is already holding its rows is not filled a second time"), # The first group of a window is where the reader arrives, and every other group's height @@ -282,6 +283,37 @@ [("holdLedgerHeight(dormant[d], heldLedgerHeight(content.key, dormantRows, content.bands, dormantPredicted, build.heights));", "holdLedgerHeight(dormant[d], dormantPredicted);")], "a released group keeps the height it was measured at in the first batch"), + + # The build makes the first group of a window live, and it is laid out for real, but the + # build never measures — the measurement is the observer's. So the group the build made + # live has no height of its own until the observer looks at it, and the visit used to + # return early because the rows were already there. The group then fell back on the + # prediction when it was released: three rows and a band laid out at 1,000 px reserved + # 208 px, with not one row or band changed. + ("a group the build made live never records the height it was measured at", + [(" if (!alreadyLive || !state.height) measureLedgerGroup(section, group);\n", + " if (!alreadyLive) measureLedgerGroup(section, group);\n")], + "a group built live keeps its measured height when a wider window releases it"), + + # The bound is about what the reader is holding at this moment, not only about what a + # rebuild leaves behind. Without the cap on the visit path, a reader who walks through + # more groups than the cap allows between two rebuilds is holding all of them. + ("reader visits do not enforce the cap on the groups holding their rows", + [(" releaseLedgerRows(releaseStaleGroups(group.key));\n", "")], + "reader visits cannot increase the held row groups past the ceiling between rebuilds"), + + # A group the cap released gives up its rows at once. The rows are the DOM, and a group + # that has given them up must not still be holding them. + ("a group the cap released keeps its rows", + [(" if (list) clear(list);\n", "")], + "a group the cap released holds no rows"), + + # And it keeps the height it had. The height is the reader's place in a session they have + # scrolled through, and a group that has given up its rows without keeping it takes the + # document down under them. + ("a group the cap released keeps no reserved height", + [(" holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, null));\n", "")], + "a group the cap released keeps its reserved height"), ] original = io.open(CLIENT, encoding="utf-8", newline="").read() diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs index caefbbc5..be3a1716 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -72,6 +72,8 @@ const LIFECYCLE_FNS = [ 'heldLedgerHeight', 'fillGroupRows', 'firstVisibleRow', + // The emptying of a list, which is how a group gives up its rows. + 'clear', // The prediction itself, because the arithmetic of the prediction is half of what this file // has to prove: a band is an entry as well as a row, and a prediction that counts it as both // prices it twice. @@ -90,6 +92,18 @@ if (missing.length) { process.exit(1); } +/* + * The two helpers the live-group measurement and the release of a visited group live in. + * + * They are lifted when the client has them and left out when it does not, and that is on + * purpose: this guard has to be runnable against the client as it was before the fix, + * because the proof that the new assertions fail is a run of this guard against the broken + * code. A client without them is a client whose visited groups are never measured and never + * released, and the assertions below say so by failing. + */ +const OPTIONAL_FNS = ['measureLedgerGroup', 'releaseLedgerRows']; +const optionalSrc = OPTIONAL_FNS.map((name) => functionSource(name)).filter(Boolean); + // The two limits are declarations, not functions, so they are lifted on their own. They are // numbers the shipped code reads, and the guard below uses the shipped values rather than // numbers written here — a copy of the cap in this file would keep passing after the shipped @@ -116,10 +130,23 @@ function makeEl(tag) { style: {}, classes: new Set(), appendChild(child) { el.children.push(child); return child; }, + removeChild(child) { + const at = el.children.indexOf(child); + if (at >= 0) el.children.splice(at, 1); + return child; + }, + get firstChild() { return el.children.length ? el.children[0] : null; }, setAttribute(name, value) { el.attrs[name] = String(value); }, getAttribute(name) { return name in el.attrs ? el.attrs[name] : null; }, querySelector(selector) { return queryOne(el, selector); }, - querySelectorAll() { return []; }, + // The one query the shipped code makes on the ledger: the sections of the table, in the + // order they were built. Every other query answers with an empty list. + querySelectorAll(selector) { + if (selector === ':scope > section.turn-group') { + return el.children.filter((c) => c && c.tagName === 'SECTION' && c.classes.has('turn-group')); + } + return []; + }, getBoundingClientRect() { return { height: 240, width: 900, top: 0, bottom: 240 }; } }; el.classList = { @@ -170,7 +197,17 @@ function harness() { Date: { now: () => box.clock }, ledgerState: new Map(), ledgerContent: new Map(), - el: (tag) => { const e = makeEl(tag); box.built.push(e); return e; }, + // The ledger the table is in. The shipped code reads it when it has to take the rows out + // of a group the reader is no longer on. + dom: { ledger: makeEl('div') }, + // The shipped el(tag, className), class and all: the class is what a section is found + // by, and a list without one is a list the shipped code cannot find. + el: (tag, className) => { + const e = makeEl(tag); + if (className) String(className).split(/\s+/).forEach((c) => { if (c) e.classes.add(c); }); + box.built.push(e); + return e; + }, railStates: (entries) => entries.map(() => null), buildRow: (entry) => { const e = makeEl('div'); @@ -188,7 +225,7 @@ function harness() { }; vm.createContext(sandbox); vm.runInContext( - idleDecl[0] + '\n' + capDecl[0] + '\n' + batchDecl[0] + '\n' + fnSrc.join('\n'), + idleDecl[0] + '\n' + capDecl[0] + '\n' + batchDecl[0] + '\n' + fnSrc.join('\n') + '\n' + optionalSrc.join('\n'), sandbox ); box.sandbox = sandbox; @@ -215,6 +252,9 @@ function buildHarness(options) { s.anchor = null; s.scrollY = 0; s.ledger = makeEl('div'); + // The ledger the table is in, which is where the shipped code looks for the groups it has + // to take the rows out of. + s.dom = { ledger: s.ledger }; s.isCount = (n) => typeof n === 'number'; s.DASH = '—'; s.turnKey = (turn) => 'k' + turn.index; @@ -590,6 +630,99 @@ function measureGroup(h, key, entries, bands, height) { s.ledgerHoldsRows('never-seen', false) === false); } +/* ---------- 3d. a group the build made live keeps the height it was measured at ---------- */ + +/* + * The build makes the first group of a window live, and that group is laid out for real: + * every other group's height is predicted from it. But the build never measures — the + * measurement is the observer's — so the group the build made live has no height of its own + * until somebody looks at it. + * + * The observer does look at it: it is in view, so it is visited. And the visit used to return + * early, because the rows were already there. So the group never recorded its height, and + * when the window widened and the group was released it fell back on the prediction: a group + * of three rows and a band, predicted at 208 px, standing in for a group laid out at 1,000 px, + * with not one row or band changed. + * + * The case below is that one: a window whose first group is live, a real height on it, a + * visit, a wider window, the idle time, a rebuild — and the height the group reserves. + */ +{ + const h = buildHarness({ realPrediction: true }); + const s = h.sandbox; + // The ledger's own query, so the pass that follows the calibration finds the groups the + // build released. The fake DOM answers every query with an empty list, and a build whose + // calibration pass finds nothing never sizes the groups it released. + s.ledger.querySelectorAll = (selector) => { + if (!/data-rows\s*=\s*"released"/.test(selector)) return []; + return s.ledger.children.filter((c) => c.getAttribute('data-rows') === 'released'); + }; + const groups = [ + { turn: { index: 1 }, entries: entriesOf(3, 1, 'g0') }, + { turn: { index: 2 }, entries: entriesOf(2, 0, 'g1') } + ]; + h.clock = 1000; + rebuild(h, groups); + // The first group is live, and it is really laid out at 1,000 px. + const first = sectionOf(h, 0); + first.getBoundingClientRect = () => ({ height: 1000, width: 900, top: 0, bottom: 1000 }); + // The observer visits it: it is in view, and the rows are already there. + s.fillLedgerGroup(first); + // The window widens — a group comes in above it, so it is no longer the first one — and + // the reader walks away from it, so the idle time passes. + const wider = [ + { turn: { index: 0 }, entries: entriesOf(2, 0, 'w0') }, + { turn: { index: 1 }, entries: entriesOf(3, 1, 'g0') }, + { turn: { index: 2 }, entries: entriesOf(2, 0, 'g1') } + ]; + h.clock = 1000 + s.LEDGER_IDLE_MS + 1; + rebuild(h, wider); + assert('a group built live keeps its measured height when a wider window releases it', + sectionOf(h, 1).style.minHeight === '1000px'); +} + +/* ---------- 3e. the cap holds between the rebuilds too ---------- */ + +/* + * The bound is about what the reader is holding at this moment. The rebuild enforces it, and + * so does the visit: a reader who walks through more groups than the cap allows between two + * rebuilds would otherwise be holding all of them, and the promise the cap makes would hold + * only right after a poll. + * + * The visits below are one after another, and each is within the idle timeout of the one + * before it, so the idle rule is not what releases the groups — the cap is. And the groups + * that are counted are the sections that really hold their rows, not the flags in the state. + */ +{ + const h = buildHarness(); + const s = h.sandbox; + const n = s.LEDGER_LIVE_GROUPS + 2; + const groups = []; + for (let i = 0; i < n; i += 1) { + groups.push({ turn: { index: i + 1 }, entries: entriesOf(2, 0, 'g' + i) }); + } + h.clock = 1000; + rebuild(h, groups); + // The reader walks through the window, one group at a time. + for (let i = 0; i < n; i += 1) { + h.clock += 10; + s.fillLedgerGroup(sectionOf(h, i)); + } + const holding = s.ledger.children.filter((c) => { + const list = c.querySelector('ul.row-list'); + return c.getAttribute('data-rows') === 'live' && !!list && list.children.length > 0; + }); + assert('reader visits cannot increase the held row groups past the ceiling between rebuilds', + holding.length <= s.LEDGER_LIVE_GROUPS); + // And the group the cap released gives up its rows at once, and keeps the height it had. + const released = sectionOf(h, 0); + assert('a group the cap released holds no rows', + released.getAttribute('data-rows') === 'released' + && released.querySelector('ul.row-list').children.length === 0); + assert('a group the cap released keeps its reserved height', + typeof released.style.minHeight === 'string' && released.style.minHeight !== ''); +} + /* ---------- 4. the rows come back when the reader reaches the group ---------- */ { From 29b8b5dd73a8bb61b570be320c62cf4c8be6e4fb Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 00:37:45 +0800 Subject: [PATCH 49/95] =?UTF-8?q?=E6=9C=8D=E5=8A=A1=E7=AB=AF=EF=BC=9A?= =?UTF-8?q?=E7=BC=93=E5=AD=98=E6=90=9C=E7=B4=A2=E6=8C=89=E5=8F=B0=E9=87=8C?= =?UTF-8?q?=E5=BA=8F=E6=8A=98=E5=89=8D=20N=EF=BC=8C=E5=91=BD=E4=B8=AD?= =?UTF-8?q?=E5=90=8C=E5=BA=8F=EF=BC=9B=E6=AE=B5=E7=9B=96=E4=B8=8D=E6=BB=A1?= =?UTF-8?q?=20body=20=E6=97=B6=E4=B8=8D=E7=A1=AC=E7=AD=94?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 第 29 轮第三遍评审报上来的服务端那条发现，这一笔是实现。 重建路按逻辑轮序折前 N 条记录的版本（先按轮，再按轮内的记录序），artifact 路按物理 文件序折——段是一个代一个文件、读的时候从旧到新。一个早轮的记录落到后代文件时， body 的顺序是 [1,3,2]、段的顺序是 [1,2,3]，于是同一份没变的 body，缓存搜索折出的 指纹与重建搜索折出的不是同一个集合，请求被 409 拒答，真实客户端两次重试耗尽后显示 失败。（[1,3,2] 与 [1,2,3] 是评审给的最小输入。） 改法：段行里本来就带着轮号与记录号，所以轮序是可以重建的，不必放弃缓存路径。 - search-segments.mjs 新增导出 segmentLedgerPositions()：一次遍历读出每行的轮号、 记录号与版本，不逐行解析。 - server.mjs 新增导出 segmentRecordVersions()：按（轮号，记录号）排序后取前 N， 限值语义与重建路那一条一致（0 是空集，缺/负/分数是全部）。 - 缓存搜索按重建出来的台里序折前 N。 顺带两条，同一个族： - 那条把命中按记录号排序的代码，与它上面那句「这就是重建路给出的顺序」的注释，都是 错的：记录号不是台里序，早轮后到的记录号更大、台里却把它画在前。两条路现在共用同一 个排序。 - 盖不满 body 的段不硬答。段集合少覆盖了一条记录时，缓存路的集合就不是页面持有的那个 集合，指纹自然对不上，409 会换个理由再来一次。改法是让段自己说清它盖不满多少：段 旁边的那份清单带上这个数，读的时候不说明覆盖率的清单不认，缓存路拒答自称盖不满的 集合，于是请求退回重建路——重建路才是记录集合的权威。清单的格式变了，段的版本号 从 4 推到 5，磁盘上已有的旧产物会被拒一次、第一次搜索重建并重新发布。这是这条规则 的价格。 指纹拒答一个字都没有弱化：body 的记录真的变了，仍然拒答。 一处要写下来的事实：评审指的那条「没有 locator 的记录不在任何段里」，今天走 HTTP 路由 造不出来——每一行都被造了 locator，locator 的路径与谱系里那条是同一个解析路径，构建 payload 时也总是写下原始行号。这一半因此是对着手工搭出来的产物状态证明的，不是对着 一个请求证明的；发布侧那一半没有任何一趟路由能观察到，用结构断言钉住。 --- .../miniapp/node/search-segments.mjs | 133 ++++++++++++++- .../mmc-trajectory/miniapp/node/server.mjs | 152 +++++++++++++++--- 2 files changed, 261 insertions(+), 24 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs index 5455714c..2a61f075 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/search-segments.mjs @@ -49,8 +49,13 @@ import { join } from 'node:path'; * 4: every line carries the record's version (`v`). A hit is a POSITION, and a position is only the * same record in another body if the version says so — so a segment written before this would hand * the page hits it cannot check, and the page would have to take them on faith. + * + * 5: every manifest states how many of the body's records the segments do NOT cover. A record with + * no locator names no file, so it belongs to no generation's segment — and a set that covers fewer + * records than the body stands for a DIFFERENT set, whose fingerprint the page is not holding. A + * manifest written before this does not say, and a set that does not say cannot be used. */ -export const SEGMENT_SCHEMA_VERSION = 4; +export const SEGMENT_SCHEMA_VERSION = 5; /** * The fields a search reads, in the page's order. @@ -63,7 +68,7 @@ export const SEARCH_FIELDS = ['summary', 'text', 'result', 'thinking', 'toolName /** @typedef {{ fileName: string, generation: number | null, size: number, mtimeMs?: number | null, revision: string | null, truncated?: boolean }} SegmentSource */ -/** @typedef {{ schemaVersion: number, generation: number | null, view: string, recordCount: number, surfaceBytes: number, surfaceDigest: string, source: SegmentSource[] }} SegmentManifest */ +/** @typedef {{ schemaVersion: number, generation: number | null, view: string, recordCount: number, surfaceBytes: number, surfaceDigest: string, uncovered: number, source: SegmentSource[] }} SegmentManifest */ /** * One record's searchable text. @@ -175,6 +180,37 @@ export function fileIdxRowsFromPayload(payload, raws, files) { return out; } +/** + * How many of the body's records no segment covers. + * + * A record is filed under the generation its locator names, and a record with no locator names no + * file — so it is filed under no generation, and no segment holds it. The segments then cover fewer + * records than the body, and a shorter set is a DIFFERENT set: its fingerprint is the fingerprint + * of the records it does cover, and the page is holding the ones it does not. So the count is what + * the manifest states, and a reader that sees it knows the set cannot answer for the body. + * + * A row that is not a placement at all counts as uncovered. The count is read as 「how much is + * missing」, so the safe direction is the one that says something IS missing. + * + * @param {Array<{ fileIdx: number }>} rows one entry per record, from + * {@link fileIdxRowsFromPayload} + * @returns {number} + */ +export function uncoveredRecordCount(rows) { + const list = Array.isArray(rows) ? rows : []; + let uncovered = 0; + for (const row of list) { + if (!row || typeof row !== 'object') { + uncovered += 1; + continue; + } + if (typeof row.fileIdx !== 'number' || !Number.isFinite(row.fileIdx) || row.fileIdx < 0) { + uncovered += 1; + } + } + return uncovered; +} + /** * Builds one generation's searchable lines. * @@ -387,7 +423,7 @@ export function createSegmentStore(root) { * @param {string} view * @param {string} generationKey * @param {{ surface: string }} segment - * @param {{ generation: number | null, source: SegmentManifest['source'] }} meta + * @param {{ generation: number | null, source: SegmentManifest['source'], uncovered: number }} meta */ async write(sessionKey, view, generationKey, segment, meta) { const dir = dirOf(sessionKey, view, generationKey); @@ -402,6 +438,11 @@ export function createSegmentStore(root) { recordCount: segment.surface === '' ? 0 : segment.surface.split('\n').length, surfaceBytes: Buffer.byteLength(segment.surface), surfaceDigest: digestOf(segment.surface), + // How many of the body's records these segments do NOT cover. Written here so that a reader + // can tell a set that stands for the whole body from one that stands for part of it — see + // `uncoveredRecordCount` for what the count is, and the schema version above for what + // happens to a manifest that does not say. + uncovered: meta.uncovered, source: meta.source, }; await writeFile(`${join(dir, 'manifest.json')}.tmp`, `${JSON.stringify(manifest)}\n`, 'utf8'); @@ -431,6 +472,11 @@ export function createSegmentStore(root) { if (!manifest || typeof manifest !== 'object') return null; if (manifest.schemaVersion !== SEGMENT_SCHEMA_VERSION) return null; if (manifest.view !== view) return null; + // The manifest has to say how many of the body's records these segments do not cover. A + // manifest that does not say cannot be told apart from one that says 「none」, and a set that + // covers part of the body is not the set the page is holding — so a manifest that does not + // say is a manifest this reader cannot use. + if (!Number.isInteger(manifest.uncovered) || manifest.uncovered < 0) return null; if (!sameSource(manifest.source, source)) return null; let surface; try { @@ -460,6 +506,9 @@ export function createSegmentStore(root) { * parse every line it does not need; `tests/verify-search-segments.mjs` pins that this and the * parsed answer agree. * + * The route does not read the versions this way: it needs each record's LEDGER position as well, + * so it reads the two in one walk — see {@link segmentLedgerPositions}. + * * @param {string} surface * @param {number} [limit] stop after this many records; zero is zero records, and a missing, * negative or fractional limit is every record @@ -485,6 +534,84 @@ export function segmentVersions(surface, limit) { return out; } +/** + * The ledger position of every record a segment holds, and the version of each. + * + * A line carries the record's TURN and the record's own number in the body, and those two are + * what the ledger's order is made of: the ledger draws turn by turn, and inside a turn the + * records in the order they were emitted. So the two numbers on a line are enough to put that + * record back where the body has it. + * + * Why that matters: the segments are one file per generation and the body is not in file order. + * A record that belongs to an EARLIER turn and arrived after that turn was written lives in a + * LATER file, so the first N records of the body and the first N records of the files are two + * different sets. Reading the versions out of the files alone therefore answers about the wrong + * set, and the answer is refused for a body that did not change. + * + * The version comes along because the same walk has to hand it over: a second walk over the + * same surface would cost the same again, and the answer that needs them is the answer about + * THIS set of records. + * + * The numbers are read by hand, like the version: a search over a big segment would then parse + * every line it does not need. `index` and `turnIndex` are written before the summary, so the + * first one in a line's answer is the field and not something the summary says. + * + * @param {string} surface + * @returns {Array<{ turnIndex: number | null, index: number | null, v: string }>} one entry per + * record, in the order the lines sit in this surface + */ +export function segmentLedgerPositions(surface) { + /** @type {Array<{ turnIndex: number | null, index: number | null, v: string }>} */ + const out = []; + if (typeof surface !== 'string' || surface === '') return out; + let start = 0; + while (start <= surface.length) { + const nl = surface.indexOf('\n', start); + const end = nl === -1 ? surface.length : nl; + const tab = surface.indexOf('\t', start); + if (tab >= 0 && tab <= end) { + const answer = surface.slice(start, tab); + out.push({ + turnIndex: numberInAnswer(answer, '"turnIndex":'), + index: numberInAnswer(answer, '"index":'), + v: versionInAnswer(answer), + }); + } + if (nl === -1) break; + start = nl + 1; + } + return out; +} + +/** + * One number out of a line's answer. + * + * Read by hand for the same reason the version is: parsing the whole answer of every line of a + * big segment is the cost the hand-reading exists to avoid. The number ends at the first + * separator after it, which is where the field ends. + * + * An empty field is nothing, not a number: `Number('')` is zero, and a record that carries no + * number of its own must not come back as the record numbered zero. + * + * @param {string} answer the JSON prefix of one segment line + * @param {string} key the field to read, with its quotes and its colon + * @returns {number | null} the number, or nothing when the field is not there + */ +function numberInAnswer(answer, key) { + const at = answer.indexOf(key); + if (at < 0) return null; + let end = at + key.length; + let value = ''; + while (end < answer.length) { + const ch = answer.charAt(end); + if (ch === ',' || ch === '}') break; + value += ch; + end += 1; + } + const parsed = value.trim() === '' ? NaN : Number(value); + return Number.isFinite(parsed) ? parsed : null; +} + /** * The version token inside one line's answer. * diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index f343b569..20c1bbf2 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -18,8 +18,9 @@ import { normaliseWhitespace, readSegmentHits, scanSegment, - segmentVersions, + segmentLedgerPositions, searchSurfaceOf, + uncoveredRecordCount, } from './search-segments.mjs'; /** @typedef {import('./miniapp-api.js').MiniAppContext} MiniAppContext */ @@ -487,6 +488,92 @@ export function payloadRecordVersions(payload, limit) { return out; } +/** + * The ledger's order: the turns in the order the body numbers them, and inside a turn the + * records in the order they were emitted. + * + * ONE definition, because two answers have to agree on it. The fold over the first N records + * says which SET of records an answer is about, and the order the hits are reported in says how + * that answer reads. Both are the body's order — the ledger draws turn by turn — so both are + * this comparison. + * + * The record's own number is not the order. It is the order the records were EMITTED in, and a + * record that belongs to an earlier turn and arrived after that turn was written is emitted + * later than a record of a later turn: its number is larger, and it still comes first. That is + * why the turn comes first here, and why sorting by the number alone is the wrong order. + * + * @param {{ turnIndex?: number | null, index?: number | null }} a + * @param {{ turnIndex?: number | null, index?: number | null }} b + * @returns {number} + */ +export function byLedgerOrder(a, b) { + // A record with no turn of its own cannot be placed, and the callers that read these fields + // have already refused such a record. The fallback is here so that a comparison can never + // read `null` as a number. + const turnA = typeof a.turnIndex === 'number' ? a.turnIndex : Number.MAX_SAFE_INTEGER; + const turnB = typeof b.turnIndex === 'number' ? b.turnIndex : Number.MAX_SAFE_INTEGER; + if (turnA !== turnB) return turnA - turnB; + const indexA = typeof a.index === 'number' ? a.index : 0; + const indexB = typeof b.index === 'number' ? b.index : 0; + return indexA - indexB; +} + +/** + * The version tokens of the first N records, in the order the ledger shows them, read from the + * stored segments rather than from a rebuilt body. + * + * THE SAME FACT AS {@link payloadRecordVersions}, read the other way round. The body is in + * ledger order — turn by turn, and inside a turn the records in the order they were emitted — + * and the segments are not: they are one file per generation, read oldest first. A record that + * belongs to an EARLIER turn and arrived after that turn was written lives in a LATER file, so + * the first N records of the body and the first N records of the files are two different sets. + * + * That is not a detail of the fixture: a turn that spans a compaction is written like that, and + * a fold over the files answers about a set the page is not holding. The fingerprint then does + * not match the one the page asked for, and the service refuses a body that did not change. + * + * The positions on the lines are what puts the records back: the turn a record is in, and the + * record's own number in the body. Both are on the line, so nothing has to be rebuilt to know + * them — and the order is the body's, because the body numbers its turns in the order it draws + * them and its records in the order it emits them. + * + * `null` is the answer for a record that cannot be placed: a line that carries no turn or no + * record number cannot be put back into the ledger's order, and an answer about a set we cannot + * name is not an answer about the set the page holds. The caller falls back to the rebuild, + * which is the same answer it would have given before. + * + * @param {Array<{ turnIndex: number | null, index: number | null, v: string }>} positions the + * ledger position of every record the segments cover + * @param {number} [limit] stop after this many records; zero is zero records, and a missing, + * negative or fractional limit is every record + * @returns {Array | null} + */ +export function segmentRecordVersions(positions, limit) { + const list = Array.isArray(positions) ? positions : []; + /** @type {Array<{ turnIndex: number, index: number, v: string }>} */ + const placed = []; + for (const one of list) { + if (!one || typeof one !== 'object') return null; + const turnIndex = one.turnIndex; + const index = one.index; + if (typeof turnIndex !== 'number' || !Number.isFinite(turnIndex)) return null; + // A negative number is how the segment says 「this record has no number of its own」, the + // same way it says a record belongs to no file. Such a record cannot be placed inside its + // turn, and a fold that put it somewhere would be folding over a set nobody can name. + if (typeof index !== 'number' || !Number.isFinite(index) || index < 0) return null; + placed.push({ turnIndex, index, v: typeof one.v === 'string' ? one.v : '' }); + } + // The ledger's order: the turns in the order the body numbers them, and inside a turn the + // records in the order they were emitted. The record's own number is what says which is + // which inside a turn — the segments hold one turn's records in more than one file. + placed.sort(byLedgerOrder); + // The same rule as the payload's own fold over the first N records: an explicit limit of zero + // is zero records, and a missing, negative or fractional limit is every record. The two are two + // readings of one fact — the versions of the first N records — so they have to agree. + const wanted = Number.isInteger(limit) && limit >= 0 ? limit : Infinity; + return placed.slice(0, wanted).map((one) => one.v); +} + /** * Whether a tail may be answered to a page that holds a particular set of records. * @@ -4801,17 +4888,24 @@ export async function start(context) { /** @type {Array<{ index: number, turnIndex: number | null, kind: string, summary: string, generation: number | null }>} */ const hits = []; let total = 0; - // The version tokens of the records these segments cover, in the order the ledger shows them. - // They are what the fingerprint is folded over, and the fingerprint is what says whether this - // answer is about the body the page is showing. - /** @type {Array} */ - const versions = []; + // The ledger position of every record these segments cover: the turn it is in and its own + // number in the body. They are what the version tokens are folded over IN THE ORDER THE LEDGER + // SHOWS THEM — the segments are one file per generation, and the body is not in file order, + // so the first N records of the files are not the first N records of the body. + /** @type {Array<{ turnIndex: number | null, index: number | null, v: string }>} */ + const positions = []; // `files` is the lineage walk's own output, so a generation that is no longer in the lineage // has no entry here and no segment is read — a leftover on disk cannot keep it searchable. for (let fileIdx = 0; fileIdx < files.length; fileIdx += 1) { const file = files[fileIdx]; const segment = await segmentStore.read(sessionKey, viewKey, generationKey(file.generation), [file]); if (segment === null) return null; + // A set that does not cover the whole body is not the set the page is holding: the records + // no segment covers are missing from the fold, so the fingerprint is the fingerprint of a + // DIFFERENT set of records, and the answer would be refused for a body that did not change. + // The rebuild answers instead — it has the body, and the body is the authority on which + // records exist. + if (segment.manifest.uncovered > 0) return null; const ordinals = scanSegment(segment.surface, needle); const answers = readSegmentHits(segment.surface, ordinals); for (const hit of answers) { @@ -4821,19 +4915,27 @@ export async function start(context) { total += 1; if (hits.length < MAX_SEARCH_HITS) hits.push(hit); } - // The versions of this generation's records, up to the number the page asked about. Read - // from the surface the segment already holds rather than from a second copy of it: the - // version is on every line, and a second copy would be a second thing to keep in step. - if (versions.length < count) { - for (const version of segmentVersions(segment.surface, count - versions.length)) { - versions.push(version); - } - } + // The position and the version of every record this generation holds, read from the surface + // the segment already holds rather than from a second copy of it: both are on every line, + // and a second copy would be a second thing to keep in step. + for (const one of segmentLedgerPositions(segment.surface)) positions.push(one); } - // Ascending ledger order, because the segments were read oldest generation first and a hit's - // place in that list is what the page scrolls to. Sorting by the index itself is the order the - // rebuild produces, and two answers that differ only in order are two answers. - hits.sort((a, b) => a.index - b.index); + // The version tokens of the first N records, in the order the ledger shows them. This is the + // same fact the rebuild folds over the payload's own records, and the two have to agree: the + // fingerprint is what says whether this answer is about the body the page is showing. + const versions = segmentRecordVersions(positions, count); + // A record that cannot be placed in the ledger's order cannot be counted into a set, so the + // answer would be about a set nobody can name. The rebuild answers instead. + if (versions === null) return null; + // The ledger's order, which is the order the rebuild returns its hits in: the turns in the + // order the body numbers them, and inside a turn the records in the order they were + // emitted. The segments were read oldest generation first, so that is NOT the order they + // arrive in, and the record's own number is not it either — a record that belongs to an + // earlier turn and arrived after that turn was written is emitted later than a record of a + // later turn, and the ledger still draws it first. Two answers to one question that differ + // only in order are two answers, so the cached path reports its hits in the same order the + // rebuild does. + hits.sort(byLedgerOrder); return { hits, total, fingerprint: recordSetFingerprint(versions) }; }; @@ -4855,6 +4957,11 @@ export async function start(context) { if (locator !== undefined) locators.set(recordIndex, locator); } const fileIdxRows = fileIdxRowsFromPayload(payload, locators, files); + // How many of the body's records no segment will hold. A record with no locator names no + // file, so it belongs to no generation's segment — and the segments then cover fewer records + // than the body, which is a different set with a different fingerprint. The count goes into + // every manifest, so a reader can SEE that the set is short instead of answering from it. + const uncovered = uncoveredRecordCount(fileIdxRows); // One segment per generation, the active one included. §8.8.1 rule 5 excluded the active // generation on the grounds that a segment for it would be stale before it was written — true, @@ -4899,13 +5006,16 @@ export async function start(context) { // rebuild forever, for a generation that could never match anything. const generation = files[fileIdx].generation; const generationKeyValue = generationKey(generation); - // Already there and still matching its source: nothing to do, which is the common case on a - // session that is not being compacted right now. const existing = await segmentStore.read(sessionKey, viewKey, generationKeyValue, [files[fileIdx]]); - if (existing !== null) continue; + // Already there and still matching its source: nothing to do, which is the common case on a + // session that is not being compacted right now. The count has to agree as well: it is + // written into every manifest, and a set whose manifests say different things about what + // they cover is not a set a reader can take on faith. + if (existing !== null && existing.manifest.uncovered === uncovered) continue; const segment = buildSegment({ records }); await segmentStore.write(sessionKey, viewKey, generationKeyValue, segment, { generation, + uncovered, source: [files[fileIdx]], }); } From 394ba76d6b9531de603145f481a40ceccf4b50db Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 00:37:45 +0800 Subject: [PATCH 50/95] =?UTF-8?q?=E6=9C=8D=E5=8A=A1=E7=AB=AF=EF=BC=9A?= =?UTF-8?q?=E5=8F=B0=E9=87=8C=E5=BA=8F=E6=8A=98=E3=80=81=E5=91=BD=E4=B8=AD?= =?UTF-8?q?=E5=90=8C=E5=BA=8F=E3=80=81=E6=AE=B5=E7=9B=96=E4=B8=8D=E6=BB=A1?= =?UTF-8?q?=E6=97=B6=E9=80=80=E5=9B=9E=E9=87=8D=E5=BB=BA=E7=9A=84=E5=AE=88?= =?UTF-8?q?=E5=8D=AB=E4=B8=8E=E5=8F=98=E5=BC=82?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 上一笔的实现配守卫。断言与控制的名字逐字如下。 tests/verify-trajectory-cache.mjs 新增 6b 段（两代 fixture 下的端到端行为，112 -> 121 条）： - a cached search accepts the unchanged displayed set when an earlier turn gains a record in a later generation - a cached search returns its hits in the same order the rebuild returns them - a search whose segments do not cover the whole body is answered by the rebuild - and the rebuild reports every record that matches, not the ones the segments cover - the manifest beside it says how much of the body it does not cover - the publish counts the records no segment covers, and hands the count to the manifest tests/verify-search-segments.mjs 新增段行里的台里序位置与清单覆盖率那组断言 （136 -> 143 条），其中： - and says how much of the body it does not cover - a manifest that does not say what it does not cover is refused - a record whose file is not in the table is filed under no generation - and the records no segment covers are counted - a set that covers the whole body counts nothing tests/verify-search-endpoint.mjs 新增 segmentRecordVersions 的单元段（106 条）。 新增的变异控制，每条按名字抓住： - tests/mutate-trajectory-cache.py（34 -> 37 条） the cached path folds the versions in the order the files hold them, not the order the ledger draws them the cached path sorts its hits by the record's number, so it answers in a different order than the rebuild the cached path answers from a set of segments that does not cover the whole body the publish counts nothing, so a set that does not cover the body says it covers it - tests/mutate-search-segments.py（27 -> 29 条） a manifest that does not say what it does not cover is served anyway the records no segment covers are counted as covered - tests/mutate-search-endpoint.py（21 条） the limit on the fold over the first N records is read as something else 证明：把实现逐半回退、守卫跑红、再还原，还原后逐字节核对一致。退回「按文件序折」-> [trajectory-cache] 5 FAILED, 107 passed；退回「清单不说明覆盖率」-> [search-segments] 5 failure(s) 与 [trajectory-cache] 9 FAILED；退回「读的时候接受不说明覆盖率的清单」-> [search-segments] 1 failure(s)。 一条既有形状断言随实现改了措辞：它指的那行代码（按剩余条数读前若干条版本）已经不存在 了，新的期望钉的是台里序那条折，而它的注释写明了自己看不见顺序——顺序由上面那条端到端 的行为断言拥有。 一处既有变异的锚跟着代码挪了（段旁边多出来的那行清单），否则它会被报成死锚。 --- .../tests/mutate-search-endpoint.py | 13 +- .../tests/mutate-search-segments.py | 36 +++ .../tests/mutate-trajectory-cache.py | 58 ++++ .../tests/verify-search-endpoint.mjs | 56 +++- .../tests/verify-search-segments.mjs | 106 ++++++- .../tests/verify-trajectory-cache.mjs | 288 +++++++++++++++++- 6 files changed, 540 insertions(+), 17 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py index a4b27862..6ef90ae5 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py @@ -122,6 +122,15 @@ " return { hits, total, fingerprint: recordSetFingerprint(versions) };", " return { hits, total: hits.length, fingerprint: recordSetFingerprint(versions) };", ), + ( + # The two readings of "the versions of the first N records" have to agree, and the limit + # is read the same way in both: an explicit zero is zero records, and a missing, negative + # or fractional limit is every record. Read the other way round, a request for no records + # at all is answered about every record there is. + "the limit on the fold over the first N records is read as something else", + " const wanted = Number.isInteger(limit) && limit >= 0 ? limit : Infinity;", + " const wanted = Number.isInteger(limit) && limit > 0 ? limit : Infinity;", + ), ( # The summary a hit answers with now lives on the segment line, so the mutation that swaps it # for the kind belongs with the module that writes it — mutate-search-segments.py. @@ -167,8 +176,8 @@ # actually write instead of a revert. It trips both halves of the rule: the store's name and # the second directory it makes. "a second artifact store is written beside the segments, and nothing reads it", - " const segment = buildSegment({ records });\n await segmentStore.write(sessionKey, viewKey, generationKeyValue, segment, {\n generation,\n source: [files[fileIdx]],\n });", - " const segment = buildSegment({ records });\n await segmentStore.write(sessionKey, viewKey, generationKeyValue, segment, {\n generation,\n source: [files[fileIdx]],\n });\n const indexStore = createSegmentStore(join(artifactRoot, 'index'));\n await indexStore.write(sessionKey, viewKey, generationKeyValue, segment, {\n generation,\n source: [files[fileIdx]],\n });", + " const segment = buildSegment({ records });\n await segmentStore.write(sessionKey, viewKey, generationKeyValue, segment, {\n generation,\n uncovered,\n source: [files[fileIdx]],\n });", + " const segment = buildSegment({ records });\n await segmentStore.write(sessionKey, viewKey, generationKeyValue, segment, {\n generation,\n uncovered,\n source: [files[fileIdx]],\n });\n const indexStore = createSegmentStore(join(artifactRoot, 'index'));\n await indexStore.write(sessionKey, viewKey, generationKeyValue, segment, {\n generation,\n uncovered,\n source: [files[fileIdx]],\n });", ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py index 08430d68..3b12c310 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py @@ -117,6 +117,42 @@ " if (tab >= 0 && tab <= end) out.push(versionInAnswer(surface.slice(start, tab)));", " if (tab >= 0 && tab <= end) out.push(String(out.length));", ), + ( + # The ledger position of a record is what puts the records back where the body has them: + # the turn a record is in, and the record's own number in the body. The segments are one + # file per generation and the body is not in file order, so a fold over the files is a + # fold over a different set — and the answer is refused for a body that did not change. + # Reading the line's ordinal instead of the record's number makes the position say where + # the record sits IN THIS FILE, which is the order the wrong one. + "the record's own number is read as the line's ordinal, so the records cannot be put back in order", + " index: numberInAnswer(answer, '\"index\":'),", + " index: numberInAnswer(answer, '\"ordinal\":'),", + ), + ( + # The turn a record is in is the other half of the position. Read as the record's own + # number, the two halves of the position become one, and the turns of the body are gone. + "the turn a record is in is read as its own number, so the ledger's order cannot be rebuilt", + " turnIndex: numberInAnswer(answer, '\"turnIndex\":'),", + " turnIndex: numberInAnswer(answer, '\"index\":'),", + ), + ( + # The manifest has to say how many of the body's records the segments do NOT cover. A + # manifest that does not say cannot be told apart from one that says 「none」, and a set that + # covers part of the body is not the set the page is holding — so a reader that takes such a + # manifest for a complete one answers about a set nobody can name. + "a manifest that does not say what it does not cover is served anyway", + " if (!Number.isInteger(manifest.uncovered) || manifest.uncovered < 0) return null;\n", + "", + ), + ( + # The count itself: a record whose locator names no file in the table — and a record with no + # locator at all — belongs to no generation's segment, so no segment holds it. Counted as + # covered, the manifest says the set stands for the whole body, and the reader answers from + # a set that is one record short. + "the records no segment covers are counted as covered", + " if (typeof row.fileIdx !== 'number' || !Number.isFinite(row.fileIdx) || row.fileIdx < 0) {\n uncovered += 1;\n }", + " if (false) {\n uncovered += 1;\n }", + ), # ---- the rest of the contract --------------------------------------------------------- ( "the needle floor is dropped, so one character is a search", diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py index 1b87a040..7e50831a 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py @@ -364,6 +364,64 @@ " sendJsonBody(response, 200, cached.body, { etag, revalidate: true });\n return;\n", " // the body is rebuilt below\n", ), + # ---- the order the first N records are folded over --------------------------------- + # + # The answer is about a SET of records, and the set is the first N of the body — the body in + # LEDGER order, which is turn by turn. The segments are one file per generation, read oldest + # first, and a record that belongs to an EARLIER turn can live in a LATER file: the first N + # records of the files are then not the first N records of the body. A fold over the files + # answers about a different set, the fingerprint of the first N does not match the one the + # page asked for, and the service refuses a body that did not change at all. + # + # The control below breaks the reconstruction by sorting on the record's own number, which is + # the order the files hold — the mistake reads as a simplification, because a record's number + # does order the records of one turn. + ( + "the cached path folds the versions in the order the files hold them, not the order the ledger draws them", + SERVER, + " placed.sort(byLedgerOrder);", + " placed.sort((a, b) => a.index - b.index);", + "a cached search accepts the unchanged displayed set when an earlier turn gains a record in a later generation", + ), + ( + # The hits are reported in the ledger's order, which is the order the rebuild reports them + # in. The record's own number is NOT that order: a record that belongs to an earlier turn + # and arrived after that turn was written is emitted later than a record of a later turn, + # and the ledger still draws it first. Sorted by the number alone, the cached path answers + # the same question in a different order — and two answers that differ only in order are + # two answers. + "the cached path sorts its hits by the record's number, so it answers in a different order than the rebuild", + SERVER, + " hits.sort(byLedgerOrder);", + " hits.sort((a, b) => a.index - b.index);", + "a cached search returns its hits in the same order the rebuild returns them", + ), + # ---- a set of segments that does not cover the whole body ------------------------------- + # + # A record with no locator names no file, so it belongs to no generation's segment: the + # segments cover fewer records than the body, and the fold over them is a fold over a + # DIFFERENT set of records. The fingerprint of that set is not the one the page is holding, so + # the service refuses a body that did not change at all — and the reader is told the session + # moved when nothing moved. + # + # Two controls, one for each half. The reader's half is the refusal: without it the cached path + # answers from a set that is short. The publish's half is the count: a publish that counted + # nothing would write 「the set covers the whole body」 for a set that does not, and the reader + # would have nothing to refuse. + ( + "the cached path answers from a set of segments that does not cover the whole body", + SERVER, + " if (segment.manifest.uncovered > 0) return null;\n", + "", + "a search whose segments do not cover the whole body is answered by the rebuild", + ), + ( + "the publish counts nothing, so a set that does not cover the body says it covers it", + SERVER, + " const uncovered = uncoveredRecordCount(fileIdxRows);", + " const uncovered = 0;", + "the publish counts the records no segment covers, and hands the count to the manifest", + ), # ---- the cap on the fold over the first N records --------------------------------- # # The cap is checked BEFORE the push, so a limit of zero is zero records. Checked after the diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs index 0cf564a7..5dc170a4 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs @@ -32,6 +32,7 @@ import { needleIsUsable, normaliseWhitespace, selectIdentityView, + segmentRecordVersions, splitRawRecords, ROW_GENERATION, } from '../miniapp/node/server.mjs'; @@ -293,6 +294,44 @@ check('a hit carries the generation it is in', check('a payload with no turns answers nothing', collectSearchHits({}, foldNeedle('needle'), 500), { hits: [], total: 0 }); +/* ===================================================================== + * 3b — the fold over the first N records, read from the stored segments + * ===================================================================== */ + +// The same fact as the payload's own fold, read the other way round. The segments carry each +// record's LEDGER position — the turn it is in and its own number in the body — and the fold is +// over the first N in THAT order: the body's order, which is turn by turn. The segments are one +// file per generation, and a record that belongs to an EARLIER turn can live in a LATER file, so +// the order the segments are read in is not the order the ledger draws. +// +// The limit is read the way the payload's own fold reads it: an explicit zero is zero records, +// and a missing, negative or fractional limit is every record. The two are two readings of one +// fact — the versions of the first N records — so they have to agree. +const LEDGER_POSITIONS = [ + { turnIndex: 1, index: 1, v: 'version-1' }, + { turnIndex: 1, index: 3, v: 'version-3' }, + { turnIndex: 2, index: 2, v: 'version-2' }, +]; +check('the versions of the first N records are folded in the ledger\'s order', + segmentRecordVersions(LEDGER_POSITIONS, 3), ['version-1', 'version-3', 'version-2']); +check('and that order is the ledger\'s, not the order the segments were read in', + segmentRecordVersions([ + { turnIndex: 2, index: 2, v: 'version-2' }, + { turnIndex: 1, index: 3, v: 'version-3' }, + { turnIndex: 1, index: 1, v: 'version-1' }, + ], 3), ['version-1', 'version-3', 'version-2']); +check('and only as many as were asked for', + segmentRecordVersions(LEDGER_POSITIONS, 2), ['version-1', 'version-3']); +check('and a limit of zero is zero records', + segmentRecordVersions(LEDGER_POSITIONS, 0), []); +check('and a missing, negative or fractional limit is every record', + [segmentRecordVersions(LEDGER_POSITIONS).length, + segmentRecordVersions(LEDGER_POSITIONS, -1).length, + segmentRecordVersions(LEDGER_POSITIONS, 1.5).length], [3, 3, 3]); +check('and a record that cannot be placed is not folded over at all', + [segmentRecordVersions([{ turnIndex: null, index: 1, v: 'version-1' }], 1), + segmentRecordVersions([{ turnIndex: 1, index: -1, v: 'version-1' }], 1)], [null, null]); + /* ===================================================================== * 4 — a needle never crosses a record, and never gets that far if too short * ===================================================================== */ @@ -510,9 +549,20 @@ assert('and the shipped artifact path counts every match rather than what it sen // session grew has fewer records, not different ones, so its first N are still the same N — and a // body-level identity (an ETag, a digest of the whole body) would change on every append and // make every answer stop applying every few seconds. -assert('the artifact path folds the versions of the records it covers', - /segmentVersions\(segment\.surface, count - versions\.length\)/.test(artifactsSrc) - && /const versions = \[\];/.test(artifactsSrc)); +// The fold is over the FIRST N records IN THE ORDER THE LEDGER SHOWS THEM — turn by turn, and +// inside a turn the records in the order they were emitted. The segments are one file per +// generation, and the body is not in file order, so the artifact path has to put the records +// back where the body has them before it takes the first N. +// +// This assertion cannot see the ORDER: a fold over the files reads the same lines and leaves the +// same shape behind, and the difference only shows on a body whose turn grouping differs from +// its file order. The behaviour is owned by verify-trajectory-cache.mjs, which searches such a +// body through the real route and compares the answer with the rebuild's. +assert('the artifact path folds the versions of the records it covers, in the ledger\'s order', + /segmentLedgerPositions\(segment\.surface\)/.test(artifactsSrc) + && /const positions = \[\];/.test(artifactsSrc) + && /segmentRecordVersions\(positions, count\)/.test(artifactsSrc) + && /recordSetFingerprint\(versions\)/.test(artifactsSrc)); assert('and the route refuses an answer that is not about the set the page asked about', /search_version_mismatch/.test(SERVER) && /recordSetFingerprint\(payloadRecordVersions\(payload, count\)\)/.test(SERVER)); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs index 72a54f0a..5d39d635 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs @@ -35,6 +35,7 @@ import { capHits, createSegmentStore, digestOf, + fileIdxRowsFromPayload, foldNeedle, generationKey, needleIsUsable, @@ -42,8 +43,10 @@ import { readSegmentHits, sameSource, scanSegment, + segmentLedgerPositions, segmentVersions, searchSurfaceOf, + uncoveredRecordCount, } from '../miniapp/node/search-segments.mjs'; const SEGMENT_PATH = new URL('../miniapp/node/search-segments.mjs', import.meta.url); @@ -417,8 +420,9 @@ try { assert('nothing stored yet reads as no segment', (await store.read('s', 'all', 'g0', SOURCE)) === null); - const written = await store.write('s', 'all', 'g0', segment, { generation: 0, source: SOURCE }); + const written = await store.write('s', 'all', 'g0', segment, { generation: 0, uncovered: 0, source: SOURCE }); check('the manifest counts the records', written.recordCount, RECORDS.length); + check('and says how much of the body it does not cover', written.uncovered, 0); const good = await store.read('s', 'all', 'g0', SOURCE); assert('a complete segment reads back', good !== null); if (good !== null) { @@ -453,13 +457,13 @@ try { const counted = await store.read('s', 'all', 'g0', SOURCE); assert('and is read once the manifest counts it', counted !== null); if (counted !== null) check('with every line present', counted.surface.split('\n').length, RECORDS.length + 1); - await store.write('s', 'all', 'g0', segment, { generation: 0, source: SOURCE }); + await store.write('s', 'all', 'g0', segment, { generation: 0, uncovered: 0, source: SOURCE }); await rm(join(dir, 'manifest.json'), { force: true }); assert('a pair with no manifest is refused', (await store.read('s', 'all', 'g0', SOURCE)) === null); - await store.write('s', 'all', 'g0', segment, { generation: 0, source: SOURCE }); + await store.write('s', 'all', 'g0', segment, { generation: 0, uncovered: 0, source: SOURCE }); const manifestPath = join(dir, 'manifest.json'); const original = JSON.parse(await readFile(manifestPath, 'utf8')); for (const [patch, label] of [ @@ -469,6 +473,19 @@ try { await writeFile(manifestPath, JSON.stringify({ ...original, ...patch }), 'utf8'); assert(`a manifest from ${label} is refused`, (await store.read('s', 'all', 'g0', SOURCE)) === null); } + // What the manifest has to SAY. A set of segments that covers fewer records than the body stands + // for a different set — the records it does not cover are missing from the fold — and the page + // is holding the ones it does not. So the manifest states the shortfall, and a reader that sees + // a shortfall knows the set cannot answer for the body. + // + // A manifest that does not state it cannot be told apart from one that states 「none」, so it is + // refused rather than read as 「none」. + await writeFile(manifestPath, JSON.stringify({ ...original, uncovered: undefined }), 'utf8'); + assert('a manifest that does not say what it does not cover is refused', + (await store.read('s', 'all', 'g0', SOURCE)) === null); + await writeFile(manifestPath, JSON.stringify({ ...original, uncovered: 2 }), 'utf8'); + assert('and one that says it does not cover the whole body is read, so the reader can see it', + (await store.read('s', 'all', 'g0', SOURCE)) !== null); await writeFile(manifestPath, 'not json', 'utf8'); assert('an unreadable manifest is refused', (await store.read('s', 'all', 'g0', SOURCE)) === null); await writeFile(manifestPath, JSON.stringify(original), 'utf8'); @@ -489,7 +506,7 @@ try { (await store.read('s', 'all', 'g0', [{ ...SOURCE[0], revision: SOURCE[0].revision }])) !== null); // Two generations are two segments: one never answers for the other. - await store.write('s', 'all', 'g1', segment, { generation: 1, source: [{ ...SOURCE[0], generation: 1 }] }); + await store.write('s', 'all', 'g1', segment, { generation: 1, uncovered: 0, source: [{ ...SOURCE[0], generation: 1 }] }); const g0 = await store.read('s', 'all', 'g0', SOURCE); const g1 = await store.read('s', 'all', 'g1', [{ ...SOURCE[0], generation: 1 }]); check('two generations read back the same bytes', g0?.surface, g1?.surface); @@ -504,7 +521,35 @@ try { for (const line of storeFailures) failures.push(line); /* ===================================================================== - * 5 — the rest of the contract + * 5 — the coverage a manifest states + * ===================================================================== */ + +// A record is filed under the generation its locator names. A record with no locator names no file, +// so it is filed under NO generation — and no segment holds it, so the segments cover fewer records +// than the body. That is a shorter set, which is a different set: the records it does not cover are +// missing from the fold, and the page is holding those. So the shortfall is counted here and stated +// in the manifest, and a reader that sees it knows the set cannot answer for the body. +const PLACEMENT = { turns: [{ records: [{ index: 1 }, { index: 2 }, { index: 3 }] }] }; +const PLACED_FILES = [{ filePath: '/store/g0.jsonl' }]; +// The first record names the file in the table; the second names a file that is not in it; the third +// has no locator at all. The last two are the same state seen from two sides, and neither of them +// may be filed under a generation that did not produce them. +const PLACEMENT_RAWS = new Map([ + [1, { filePath: '/store/g0.jsonl' }], + [2, { filePath: '/store/elsewhere.jsonl' }], +]); +const placement = fileIdxRowsFromPayload(PLACEMENT, PLACEMENT_RAWS, PLACED_FILES); +check('a record whose file is not in the table is filed under no generation', + placement.map((row) => row.fileIdx), [0, -1, -1]); +check('and the records no segment covers are counted', + uncoveredRecordCount(placement), 2); +check('a set that covers the whole body counts nothing', + uncoveredRecordCount([{ fileIdx: 0 }, { fileIdx: 1 }]), 0); +check('a placement that is not a placement counts as missing, not as covered', + uncoveredRecordCount([{ fileIdx: 0 }, null, { fileIdx: -1 }]), 2); + +/* ===================================================================== + * 6 — the rest of the contract * ===================================================================== */ check('the active generation has a key of its own', generationKey(null), 'active'); @@ -611,6 +656,57 @@ check('and the reading agrees with the answer the line carries', check('and an empty surface has no versions', segmentVersions('', 10), []); +/* --------------------------------------------------------------------- + * The ledger position of a record, which is what puts the records back + * where the body has them. + * + * The segments are one file per generation and the body is not in file order: a record that + * belongs to an EARLIER turn and arrived after that turn was written lives in a LATER file. So + * the first N records of the body and the first N records of the files are two different sets, + * and the fold that says which set an answer is about has to be over the body's. + * + * The position is what makes the reconstruction possible without rebuilding anything: the turn + * a record is in, and the record's own number in the body. Both are on the line. + * --------------------------------------------------------------------- */ +{ + // A generation that holds two turns, and a second generation that holds a record which + // belongs to the FIRST turn — the shape that makes the two orders differ. + const sealed = buildSegment({ + records: [ + { index: 1, turnIndex: 1, kind: 'user', summary: 'first question', v: 'version-1' }, + { index: 2, turnIndex: 2, kind: 'user', summary: 'second question', v: 'version-2' }, + ], + }).surface; + const active = buildSegment({ + records: [ + { index: 3, turnIndex: 1, kind: 'user', summary: 'the late answer', v: 'version-3' }, + ], + }).surface; + check('a segment reads back the turn and the number of each of its records', + segmentLedgerPositions(sealed), [ + { turnIndex: 1, index: 1, v: 'version-1' }, + { turnIndex: 2, index: 2, v: 'version-2' }, + ]); + check('and the record that arrived for the earlier turn carries that turn', + segmentLedgerPositions(active), [ + { turnIndex: 1, index: 3, v: 'version-3' }, + ]); + check('and the reading agrees with the answer the line carries', + segmentLedgerPositions(sealed).map((one) => one.v), + readSegmentHits(sealed, [0, 1]).map((hit) => (hit === null ? '' : hit.v))); + check('and a segment with no records has no positions', + segmentLedgerPositions(''), []); + check('and a record with no turn of its own is reported as having none', + segmentLedgerPositions(buildSegment({ + records: [{ index: 7, kind: 'user', summary: 'no turn', v: 'version-7' }], + }).surface), [{ turnIndex: null, index: 7, v: 'version-7' }]); + // A field with nothing in it is nothing, not a number: `Number('')` is zero, and a record that + // carries no number of its own must not come back as the record numbered zero. + check('and a field with nothing in it is reported as having no number', + segmentLedgerPositions('{"index":,"turnIndex":1,"v":"version-1"}\ttext'), + [{ turnIndex: 1, index: null, v: 'version-1' }]); +} + /* --------------------------------------------------------------------- */ if (failures.length) { diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs index 04bc724d..4857e21f 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs @@ -17,7 +17,7 @@ * mutate-trajectory-cache.py breaks each of these in the shipped file. */ import { mkdirSync, mkdtempSync, readFileSync } from 'node:fs'; -import { mkdir, readdir, readFile, writeFile, utimes } from 'node:fs/promises'; +import { mkdir, readdir, readFile, stat, writeFile, utimes } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; @@ -390,6 +390,70 @@ try { ]; await writeVersioned(ALPHA_LINES, 1791600000000); + // A conversation whose turn grouping is not its file order. + // + // The ledger draws turn by turn, and inside a turn the records in the order they were + // emitted — so the body of this one is [m1, m3, m2]: the first turn's two records together, + // then the second turn's one. The generation files hold [m1, m2] and then [m3], because the + // record that belongs to the FIRST turn arrived after the rotation. + // + // The two orders read one fact two ways, and they are not the same order: the first N records + // of the body and the first N records of the files are two different sets. A search that + // folds the files answers about the wrong one, and refuses a body that did not change. + const LEDGER = 'mvs_ledger_order'; + const ledgerDir = join(sessionsRoot, '2026', '10', '22', '00-00-00-session_' + + Buffer.from(LEDGER, 'utf8').toString('base64url')); + const ledgerRow = (one) => JSON.stringify({ + message_id: one.id, + turn_id: one.turn, + message: { role: 'user', content: one.body, timestamp: 1791600000000 }, + }) + '\n'; + await mkdir(join(ledgerDir, 'snapshots'), { recursive: true }); + await writeFile(join(ledgerDir, 'snapshots', 'gen0.jsonl'), [ + { id: 'm1', turn: 't1', body: 'LEDGER ORDER FIRST QUESTION' }, + { id: 'm2', turn: 't2', body: 'LEDGER ORDER SECOND QUESTION' }, + ].map(ledgerRow).join(''), 'utf8'); + await writeFile(join(ledgerDir, 'messages.jsonl'), [ + { id: 'm3', turn: 't1', body: 'LEDGER ORDER LATE ANSWER' }, + ].map(ledgerRow).join(''), 'utf8'); + await writeFile(join(ledgerDir, 'manifest.json'), JSON.stringify({ sessionId: LEDGER }), 'utf8'); + // The catalog is what names the sealed generation. Without it the lineage is the active file + // alone, and there is no second generation for the turn grouping to disagree with. + await writeFile(join(ledgerDir, 'history-catalog.json'), JSON.stringify({ + artifacts: [ + { generation: 0, kind: 'snapshot', fileName: 'gen0.jsonl' }, + { generation: 1, kind: 'active', fileName: 'messages.jsonl' }, + ], + }), 'utf8'); + const ledgerStamp = new Date(1791600000000); + await utimes(join(ledgerDir, 'messages.jsonl'), ledgerStamp, ledgerStamp); + await utimes(join(ledgerDir, 'snapshots', 'gen0.jsonl'), ledgerStamp, ledgerStamp); + + // A conversation whose segment will not cover the whole body. + // + // A record with no locator names no file, so it belongs to no generation's segment — and the + // segments then cover fewer records than the body, which is a DIFFERENT set of records. The + // fold over that set is a fold over the wrong one, its fingerprint is not the one the page is + // holding, and the service refuses a body that did not change at all. + // + // Three records, so that a segment which covers two of them is visibly short. + const UNCOVERED = 'mvs_uncovered'; + const uncoveredDir = join(sessionsRoot, '2026', '10', '22', '00-00-00-session_' + + Buffer.from(UNCOVERED, 'utf8').toString('base64url')); + await mkdir(uncoveredDir, { recursive: true }); + await writeFile(join(uncoveredDir, 'messages.jsonl'), [ + { id: 'm1', turn: 't1', body: 'UNCOVERED SET FIRST' }, + { id: 'm2', turn: 't2', body: 'UNCOVERED SET SECOND' }, + { id: 'm3', turn: 't3', body: 'UNCOVERED SET THIRD' }, + ].map((one) => JSON.stringify({ + message_id: one.id, + turn_id: one.turn, + message: { role: 'user', content: one.body, timestamp: 1791600000000 }, + }) + '\n').join(''), 'utf8'); + await writeFile(join(uncoveredDir, 'manifest.json'), JSON.stringify({ sessionId: UNCOVERED }), 'utf8'); + const uncoveredStamp = new Date(1791600000000); + await utimes(join(uncoveredDir, 'messages.jsonl'), uncoveredStamp, uncoveredStamp); + // The runtime database, which is the only place on this machine that knows which // conversation is live. ALPHA is live first; BRAVO becomes live later in this section. mkdirSync(join(home, 'v2', 'sqlite'), { recursive: true }); @@ -502,29 +566,62 @@ try { // The segments a search published, found by walking the data dir. Publishing them is a side // effect that nobody awaits, so the test waits for the file rather than for a promise. - const waitForSegments = async (dir) => { - const walk = async (current, found) => { + const surfacePaths = async (dir) => { + const found = []; + const walk = async (current) => { let entries = []; try { entries = await readdir(current, { withFileTypes: true }); } catch { - return found; + return; } for (const entry of entries) { const full = join(current, entry.name); - if (entry.isDirectory()) await walk(full, found); + if (entry.isDirectory()) await walk(full); else if (entry.name === 'surface.txt') found.push(full); } - return found; }; + await walk(dir); + return found; + }; + const waitForSegments = async (dir) => { for (let attempt = 0; attempt < 60; attempt += 1) { - const found = await walk(dir, []); + const found = await surfacePaths(dir); if (found.length > 0) return found; await new Promise((done) => setTimeout(done, 50)); } return []; }; + // The surfaces that hold a given text. Every session's segments live under the same root, so + // they are told apart by what is IN them: the searchable text of the records they cover. + const waitForSegmentsWith = async (dir, marker, wanted) => { + for (let attempt = 0; attempt < 60; attempt += 1) { + const found = []; + for (const path of await surfacePaths(dir)) { + const text = await readFile(path, 'utf8'); + if (text.includes(marker)) found.push(path); + } + if (found.length >= wanted) return found; + await new Promise((done) => setTimeout(done, 50)); + } + return []; + }; + + // The manifest lands after the surface, so a segment that has just been published can be found + // before it can be read. Wait for the pair, not for the surface. + const waitForManifest = async (surfacePath) => { + const manifestPath = join(dirname(surfacePath), 'manifest.json'); + for (let attempt = 0; attempt < 60; attempt += 1) { + try { + return { manifestPath, manifest: JSON.parse(await readFile(manifestPath, 'utf8')) }; + } catch { + await new Promise((done) => setTimeout(done, 50)); + } + } + return null; + }; + try { // 1 — the poll the page makes, and the read it makes right after it. const poll = await get('/api/trajectory?session=latest'); @@ -875,6 +972,166 @@ try { assert('a tail asked for without saying which set the page holds is refused rather than guessed at', unnamedTail.status === 400 && Boolean(unnamedTail.body) && unnamedTail.body.error === 'trajectory_fingerprint_required', why(unnamedTail)); + + // 6b — the set the segments stand for, when the ledger's order is not the files' order. + // + // The search answers about a SET of records, and the set is the first N of the body — the + // body in LEDGER order, which is turn by turn. The segments are one file per generation, + // read oldest first, and those are two different orders whenever a record that belongs to an + // EARLIER turn lives in a LATER file. That is what a turn that spans a rotation looks like, + // and it is the conversation the fixture above holds. + // + // So the fold has to be over the body's order, reconstructed from the segments. A fold over + // the files answers about a different set, the fingerprint of the first N does not match the + // one the page asked for, and the service refuses a body that did not change at all. + const ledgerBody = await get('/api/trajectory?session=' + LEDGER); + assert('the conversation whose turn grouping differs from its file order loads', + ledgerBody.status === 200 && Boolean(ledgerBody.body && ledgerBody.body.turns), why(ledgerBody)); + // The order, written out by hand: the first turn holds the records 1 and 3, the second holds + // the record 2. The generation files hold 1 and 2, then 3. + const LEDGER_ORDER = [1, 3, 2]; + const FILE_ORDER = [1, 2, 3]; + check('the ledger draws the first turn with the record that arrived after the rotation', + (ledgerBody.body.turns || []).map((turn) => (turn.records || []).map((record) => record.index)), + [[1, 3], [2]]); + const ledgerVersionByIndex = new Map( + (ledgerBody.body.turns || []).flatMap((turn) => (turn.records || [])) + .map((record) => [record.index, record.v])); + // The set the page holds: the versions of its records, in the order the ledger draws them. + // The ORDER is the one written out above and the versions are the page's own, so the + // expectation cannot move together with the answer it is checking — a fold computed with the + // function under test would. + const ledgerVersions = LEDGER_ORDER.map((index) => ledgerVersionByIndex.get(index) ?? ''); + const fileVersions = FILE_ORDER.map((index) => ledgerVersionByIndex.get(index) ?? ''); + const ledgerFingerprint = recordSetFingerprint(ledgerVersions); + assert('and the two orders really are two different sets, so this case can tell them apart', + ledgerFingerprint !== recordSetFingerprint(fileVersions)); + + // The first search finds no segment to answer from, so the rebuild answers it — and + // publishes the segments the next search is answered from. + const rebuilt = await get('/api/trajectory/search?session=' + LEDGER + + '&q=LEDGER ORDER&n=' + ledgerVersions.length + '&fp=' + ledgerFingerprint); + assert('the search answers the set the page holds', + rebuilt.status === 200, why(rebuilt)); + check('and it reports the records that match', + (rebuilt.body && rebuilt.body.hits ? rebuilt.body.hits : []).map((hit) => hit.index), [1, 3, 2]); + const ledgerSurfaces = await waitForSegmentsWith(home, 'ledger order', 2); + assert('and the search published a segment for each of the two generations', + ledgerSurfaces.length === 2, 'surfaces holding the marker: ' + ledgerSurfaces.length); + + // THE REBUILD IS TAKEN OUT OF THE PICTURE, so that the answers below can only be the + // segments'. The active file's line stops parsing while the file keeps its length and its + // timestamp, so the segments still match their source — and a rebuild would see a body + // without the record that arrived late. The service is restarted because the rendered body + // is cached under the file's size and timestamp, and both of those are unchanged. + const ledgerActive = join(ledgerDir, 'messages.jsonl'); + const ledgerStat = await stat(ledgerActive); + const ledgerText = await readFile(ledgerActive, 'utf8'); + // The colon after the message id becomes a letter: the line stops being JSON, and the file + // keeps every byte it had — a segment is reusable while its source is the same file, and a + // file's identity here is its size and its timestamp. + await writeFile(ledgerActive, ledgerText.replace('"message_id":', '"message_id"X'), 'utf8'); + await utimes(ledgerActive, ledgerStat.mtimeMs / 1000, ledgerStat.mtimeMs / 1000); + await instance.dispose(); + { + const restarted = await startService(); + instance = restarted.instance; + base = restarted.base; + } + const onDisk = await get('/api/trajectory?session=' + LEDGER); + check('so the body on disk now holds only the records of the sealed generation', + (onDisk.body && onDisk.body.turns ? onDisk.body.turns : []) + .flatMap((turn) => (turn.records || [])).length, 2); + + const cached = await get('/api/trajectory/search?session=' + LEDGER + + '&q=LEDGER ORDER&n=' + ledgerVersions.length + '&fp=' + ledgerFingerprint); + assert('a cached search accepts the unchanged displayed set when an earlier turn gains a record in a later generation', + cached.status === 200, why(cached)); + // The hits come back in the ledger's order, which is the order the rebuild returns them in. + // The record's own number is NOT that order: the record that arrived for the first turn after + // the rotation is emitted last and drawn first, so the two orders really do differ here — the + // rebuild answers [1, 3, 2] and the segments hold [1, 2, 3]. + assert('a cached search returns its hits in the same order the rebuild returns them', + JSON.stringify(cached.body && cached.body.hits) === JSON.stringify(rebuilt.body && rebuilt.body.hits), + 'cached: ' + JSON.stringify(cached.body && cached.body.hits) + + ' rebuilt: ' + JSON.stringify(rebuilt.body && rebuilt.body.hits)); + check('and the answer is the one the rebuild gave', + cached.body && cached.body.hits, rebuilt.body && rebuilt.body.hits); + check('and the count is the count', + cached.body && cached.body.total, rebuilt.body && rebuilt.body.total); + + // A page that is behind is still answered, because its first records are the same ones. The + // body on disk holds two records, so a rebuild could not have answered this at all — which is + // what says the answer above came from the segments. + const ledgerBehind = await get('/api/trajectory/search?session=' + LEDGER + + '&q=LEDGER ORDER&n=2&fp=' + recordSetFingerprint(ledgerVersions.slice(0, 2))); + assert('and a page that is behind is still answered, because its first records are the same ones', + ledgerBehind.status === 200, why(ledgerBehind)); + + // And the refusal is still a refusal. The set the body on disk holds is the first two + // records in FILE order, which is not the first two in ledger order — so the segments do not + // stand for it, and a search that names it is refused rather than answered about the wrong + // set. A rebuild would have answered it, so the refusal says the segments answered. + const ledgerOtherSet = await get('/api/trajectory/search?session=' + LEDGER + + '&q=LEDGER ORDER&n=2&fp=' + recordSetFingerprint(fileVersions.slice(0, 2))); + assert('and a search that names a set the segments do not stand for is refused', + ledgerOtherSet.status === 409 && Boolean(ledgerOtherSet.body) + && ledgerOtherSet.body.error === 'search_version_mismatch', why(ledgerOtherSet)); + + // 6c — a set of segments that does not cover the whole body. + // + // The set a search is answered about is the first N records of the body, and the segments have + // to stand for THAT set. A record with no locator names no file, so it belongs to no + // generation's segment: the segments cover fewer records than the body, and the fold over them + // is a fold over a different set of records. The fingerprint of that set is not the one the + // page is holding, so the service refuses a body that did not change at all — and the reader is + // told the session moved when nothing moved. + // + // The route cannot produce that state today: every parsed row gets a locator, and a locator + // names a file that is in the lineage. So the state is put together by hand here — the state + // the segments would be in — and what is checked is the ANSWER: the search has to come from + // the rebuild, and it has to report every record that matches rather than the ones the + // segments happen to cover. + const uncoveredBody = await get('/api/trajectory?session=' + UNCOVERED); + const uncoveredVersions = versionsOf(uncoveredBody.body); + const uncoveredFingerprint = recordSetFingerprint(uncoveredVersions); + assert('the conversation whose segments will not cover the body holds three records', + uncoveredVersions.length === 3, 'records: ' + uncoveredVersions.length); + // The first search finds no segment to answer from, so the rebuild answers it — and publishes + // the segments the next search would be answered from. + const uncoveredRebuilt = await get('/api/trajectory/search?session=' + UNCOVERED + + '&q=UNCOVERED SET&n=' + uncoveredVersions.length + '&fp=' + uncoveredFingerprint); + assert('and it answers its search', uncoveredRebuilt.status === 200, why(uncoveredRebuilt)); + const uncoveredSurfaces = await waitForSegmentsWith(home, 'uncovered set', 1); + assert('the search published a segment for it', + uncoveredSurfaces.length === 1, 'surfaces holding the marker: ' + uncoveredSurfaces.length); + if (uncoveredSurfaces.length === 1) { + const pair = await waitForManifest(uncoveredSurfaces[0]); + assert('the manifest beside it says how much of the body it does not cover', pair !== null); + if (pair !== null) { + check('which is nothing, because the segment covers the whole body', + pair.manifest.uncovered, 0); + // The segment is made to cover LESS than the body: the last record is taken out of the + // surface, the manifest is told about the shorter surface, and the manifest is told that it + // does not cover the whole body. The digest is fixed, so the pair still matches itself — + // the only thing wrong with it is that it does not stand for the body the page is holding. + const surface = await readFile(uncoveredSurfaces[0], 'utf8'); + const shorter = surface.split('\n').slice(0, 2).join('\n'); + await writeFile(uncoveredSurfaces[0], shorter); + pair.manifest.surfaceDigest = digestOf(shorter); + pair.manifest.surfaceBytes = Buffer.byteLength(shorter); + pair.manifest.recordCount = 2; + pair.manifest.uncovered = 1; + await writeFile(pair.manifestPath, JSON.stringify(pair.manifest)); + const uncoveredAnswer = await get('/api/trajectory/search?session=' + UNCOVERED + + '&q=UNCOVERED SET&n=' + uncoveredVersions.length + '&fp=' + uncoveredFingerprint); + assert('a search whose segments do not cover the whole body is answered by the rebuild', + uncoveredAnswer.status === 200, why(uncoveredAnswer)); + check('and the rebuild reports every record that matches, not the ones the segments cover', + (uncoveredAnswer.body && uncoveredAnswer.body.hits ? uncoveredAnswer.body.hits : []) + .map((hit) => hit.index), [1, 2, 3]); + } + } } finally { // The server is released here and not at the end of the happy path: an assertion that // throws would otherwise leave it listening, and the guard would hang instead of @@ -904,6 +1161,23 @@ assert('the request resolves the session the family lookup is about, because a f assert('and the route records that session on the entry it stores', (startBody.match(/trajectoryCache\.set\(cacheKey, trajectoryCacheFamily\(entry\.dir, filter\), identity\.id \?\? ''/g) || []).length === 1); +/* ===================================================================== + * 8 — shape: the publish says how much of the body it does not cover + * ===================================================================== */ + +// The reader's half of the rule is behavioural and lives in section 6c: a set of segments that does +// not cover the whole body is refused, and the rebuild answers. The publisher's half is read out of +// the source instead, and the reason is the same one that put the state together by hand in 6c — +// the route cannot produce a record with no locator, so nothing that runs the route can observe what +// the publish computes. +// +// So this is the claim that the count the publish writes into the manifest IS the count of the +// records no segment covers: a publish that counted nothing would write 「the set covers the whole +// body」 for a set that does not, and the reader would answer from it. +assert('the publish counts the records no segment covers, and hands the count to the manifest', + startBody.includes('const uncovered = uncoveredRecordCount(fileIdxRows);') + && startBody.includes(' uncovered,\n')); + if (failures.length) { console.error(`\n[trajectory-cache] ${failures.length} FAILED, ${pass} passed`); for (const f of failures) console.error(' - ' + f); From 8adf329de6b7bf7b2c375f9154fa1d5940d00b91 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 03:30:23 +0800 Subject: [PATCH 51/95] Search: cap the cached path's hits AFTER the ledger's order, not during the walk The cached path walked the generations and kept the first N matches it met. The rebuilt path walks the body turn by turn and keeps the first N in the ledger's order. The two orders disagree whenever a record that belongs to an earlier turn arrived after that turn was written: the ledger draws it first, the generation walk meets it later. So the two paths answered with different SETS of records, with the same count and the same fingerprint - the one difference nothing downstream can notice, because the page turns the hits into a set of record numbers and lights up the rows in that set. The walk now collects every match, the hits are sorted into the ledger's order, and the cap is applied to that order. The count still counts every match, so it still says how many matched rather than how many were sent, and the fingerprint is still the fingerprint of the set the page is holding. The cost is that every match is held for the length of the walk. That is the same order of magnitude as the body this path exists to avoid reading, and it is released with the request. --- .../mmc-trajectory/miniapp/node/server.mjs | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 20c1bbf2..db59acd8 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -4913,7 +4913,14 @@ export async function start(context) { // does not agree with this reader. Answering without it would quietly drop a record. if (hit === null) return null; total += 1; - if (hits.length < MAX_SEARCH_HITS) hits.push(hit); + // Every match is kept, and the cap is applied later. The cap is about WHICH records the + // answer is about, and the records that come first in the ledger's order are not the + // records that come first in the generations: a record that belongs to an earlier turn and + // arrived after that turn was written sits in a later file, and the ledger still draws it + // first. Truncating here would keep the first N in the order the generations happened to + // be in — a different SET of records, with the same count and the same fingerprint, which + // is the hardest kind of difference for anything downstream to notice. + hits.push(hit); } // The position and the version of every record this generation holds, read from the surface // the segment already holds rather than from a second copy of it: both are on every line, @@ -4936,7 +4943,15 @@ export async function start(context) { // only in order are two answers, so the cached path reports its hits in the same order the // rebuild does. hits.sort(byLedgerOrder); - return { hits, total, fingerprint: recordSetFingerprint(versions) }; + // The cap is applied HERE, after the order is restored — the first N in the ledger's order is + // what the rebuilt path returns, and the two paths have to answer with the same records, not + // merely with the same number of them. + // + // The cost is that every match is held for the length of the walk. That is the same order of + // magnitude as the body this path exists to avoid reading, and it is released with the + // request. Capping the cache by bytes is a separate piece of work. + const capped = hits.length > MAX_SEARCH_HITS ? hits.slice(0, MAX_SEARCH_HITS) : hits; + return { hits: capped, total, fingerprint: recordSetFingerprint(versions) }; }; /** From b751f38a9005b2b2ef19975b874e7aaef65b7c35 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 03:30:24 +0800 Subject: [PATCH 52/95] Search: pin the cached path's hit set and order, and the cap's place in the walk - The endpoint's shape check now pins the new order: the ledger's order is applied to the hits before the cap is taken, and the old "keep the first N while walking" line is gone. The check was rewritten rather than dropped, because a check that pins a shape nobody ships is worse than no check. - The cache guard gains a section that builds a conversation with more matches than the cap allows, spread over two generations so that the ledger's order and the file order disagree about the first records. It asks the cached path and the rebuilt path the same question and compares the SET of records and the ORDER, not only the count. - Two mutations: one puts the cap back into the generation walk, one turns the ledger comparison around. Both are caught by name, and the comparison was covered from inside as well as from outside. - The endpoint control's case that reported how many it sent instead of how many matched is re-anchored on the line that now carries the count. It asks the same question as before; only the line it is written against moved. - The cache guard's transport is made explicit, because a transport failure used to be reported as a failed rule: a port is used once per run, the service is asked once before the section uses it, a transport failure is retried once, and a failure that survives names its cause. A dead pooled connection reads exactly like a broken service, and it is not one. --- .../tests/mutate-search-endpoint.py | 7 +- .../tests/mutate-trajectory-cache.py | 23 +++ .../tests/verify-search-endpoint.mjs | 14 +- .../tests/verify-trajectory-cache.mjs | 178 +++++++++++++++++- 4 files changed, 213 insertions(+), 9 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py index 6ef90ae5..80048cce 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py @@ -118,9 +118,12 @@ " if (segment === null) continue;", ), ( + # The rule is that `total` counts every match, so the cap cannot bound it: the answer + # carries the first N records of the ledger's order and says how many matched. The cap is + # applied to the HITS, after the order is restored, and `total` is counted on the walk. "the artifact path reports how many it sent instead of how many matched", - " return { hits, total, fingerprint: recordSetFingerprint(versions) };", - " return { hits, total: hits.length, fingerprint: recordSetFingerprint(versions) };", + " return { hits: capped, total, fingerprint: recordSetFingerprint(versions) };", + " return { hits: capped, total: capped.length, fingerprint: recordSetFingerprint(versions) };", ), ( # The two readings of "the versions of the first N records" have to agree, and the limit diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py index 7e50831a..fc75f259 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-cache.py @@ -396,6 +396,29 @@ " hits.sort((a, b) => a.index - b.index);", "a cached search returns its hits in the same order the rebuild returns them", ), + ( + # The cap on the hits is about WHICH records the answer is about. The rebuilt path keeps the + # first N it meets while it walks the body turn by turn, so it keeps the first N in the + # ledger's order. A path that caps while it walks the GENERATIONS keeps the first N in the + # order the generations happened to be in — and when the two orders disagree, that is a + # different SET of records, with the same count and the same fingerprint. Nothing downstream + # can notice: the page turns the hits into a set of record numbers and lights up those rows. + "the cached path caps the hits while it walks the generations, so it answers with a different set of records", + SERVER, + " hits.push(hit);", + " if (hits.length < MAX_SEARCH_HITS) hits.push(hit);", + "and it answers with the same records as the rebuild", + ), + ( + # The order itself, not the calls that use it. The two controls above change what the + # comparator is called with; this one changes what it does, so a comparator that had been + # quietly wrong would be caught even if every call site still said `byLedgerOrder`. + "the ledger's order compares the turns the wrong way round", + SERVER, + " if (turnA !== turnB) return turnA - turnB;", + " if (turnA !== turnB) return turnB - turnA;", + "a cached search accepts the unchanged displayed set when an earlier turn gains a record in a later generation", + ), # ---- a set of segments that does not cover the whole body ------------------------------- # # A record with no locator names no file, so it belongs to no generation's segment: the diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs index 5dc170a4..e2fe5f97 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-search-endpoint.mjs @@ -531,14 +531,22 @@ assert('one missing segment makes the whole search fall back rather than answer // The cap assertions above run against a copy of the loop written here, so they say the rule holds // and not that the shipped route holds it. These read the shipped function: `total` is incremented -// for every match, the push is the thing that is bounded, and the return hands back the count. +// for every match, the hits are put back in the ledger's order, the cap is taken AFTER that, and +// the return hands back the count. +// +// The cap used to be taken while the generations were being walked, which kept the first N in the +// order the generations happened to be in — a different SET of records, with the same count and the +// same fingerprint. This assertion pins the order of the two steps instead of the shape of one of +// them: the sort has to come before the slice, and the old early push must be gone. const artifactsSrc = SERVER.slice( SERVER.indexOf('const searchFromArtifacts = async'), SERVER.indexOf('const publishSearchArtifacts = async')); assert('and the shipped artifact path counts every match rather than what it sent', /total \+= 1;/.test(artifactsSrc) - && /if \(hits\.length < MAX_SEARCH_HITS\) hits\.push\(hit\);/.test(artifactsSrc) - && /return \{ hits, total, fingerprint: recordSetFingerprint\(versions\) \};/.test(artifactsSrc)); + && /return \{ hits: capped, total, fingerprint: recordSetFingerprint\(versions\) \};/.test(artifactsSrc)); +assert('and it takes the cap after it puts the hits back in the ledger\'s order, so the two paths answer with the same records', + /hits\.sort\(byLedgerOrder\);[\s\S]*hits\.slice\(0, MAX_SEARCH_HITS\)/.test(artifactsSrc) + && !/if \(hits\.length < MAX_SEARCH_HITS\) hits\.push\(hit\);/.test(artifactsSrc)); // The answer is about a SET of records, and the part of it that matters to the reader is the part // it does NOT name. So the service has to be able to say which set it searched, and the page has diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs index 4857e21f..8f58b8a3 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-cache.mjs @@ -429,6 +429,53 @@ try { await utimes(join(ledgerDir, 'messages.jsonl'), ledgerStamp, ledgerStamp); await utimes(join(ledgerDir, 'snapshots', 'gen0.jsonl'), ledgerStamp, ledgerStamp); + // A conversation with more matches than the answer may carry, and shaped so that the two orders + // disagree about which records come first. + // + // The hits are capped, and the cap is about WHICH records the answer is about. The rebuilt path + // walks the body turn by turn, so it keeps the first N in the LEDGER's order; a path that keeps + // the first N while it walks the GENERATIONS keeps the first N in the order the generations + // happened to be in. Those are different sets of records when a turn spans a rotation — and the + // difference is invisible: the count is the same, and the fingerprint is folded over the first N + // RECORDS, not over the hits. + // + // So this one is big enough for the cap to bite (901 records, all of them matching) and shaped so + // that the first turn's records mostly arrive in the LATER generation: the sealed generation holds + // the first turn's first record and the second turn's 300 records, and the active generation holds + // 600 more records of the first turn. + // + // The ledger draws the first turn first — [m1, then the 600 that arrived after the rotation] — + // and then the second turn's 300. The files hold the records by number: 1..301 in the sealed one, + // 302..901 in the active one. The first 500 in the ledger's order are therefore the first turn's + // 500, and the first 500 in file order are the first 500 by number: two different sets. + const CAP = 'mvs_ledger_cap'; + const capDir = join(sessionsRoot, '2026', '10', '22', '00-00-00-session_' + + Buffer.from(CAP, 'utf8').toString('base64url')); + await mkdir(join(capDir, 'snapshots'), { recursive: true }); + const capRow = (id, turn) => JSON.stringify({ + message_id: id, + turn_id: turn, + message: { role: 'user', content: 'CAP ORDER ' + id, timestamp: 1791600000000 }, + }) + '\n'; + const capSealed = [{ id: 'm1', turn: 't1' }]; + for (let i = 0; i < 300; i += 1) capSealed.push({ id: 'm' + (i + 2), turn: 't2' }); + const capActive = []; + for (let i = 0; i < 600; i += 1) capActive.push({ id: 'm' + (i + 302), turn: 't1' }); + await writeFile(join(capDir, 'snapshots', 'gen0.jsonl'), + capSealed.map((one) => capRow(one.id, one.turn)).join(''), 'utf8'); + await writeFile(join(capDir, 'messages.jsonl'), + capActive.map((one) => capRow(one.id, one.turn)).join(''), 'utf8'); + await writeFile(join(capDir, 'manifest.json'), JSON.stringify({ sessionId: CAP }), 'utf8'); + await writeFile(join(capDir, 'history-catalog.json'), JSON.stringify({ + artifacts: [ + { generation: 0, kind: 'snapshot', fileName: 'gen0.jsonl' }, + { generation: 1, kind: 'active', fileName: 'messages.jsonl' }, + ], + }), 'utf8'); + const capStamp = new Date(1791600000000); + await utimes(join(capDir, 'messages.jsonl'), capStamp, capStamp); + await utimes(join(capDir, 'snapshots', 'gen0.jsonl'), capStamp, capStamp); + // A conversation whose segment will not cover the whole body. // // A record with no locator names no file, so it belongs to no generation's segment — and the @@ -480,9 +527,19 @@ try { // it cached (the walk holds the session list for three seconds, so a rewritten transcript is // invisible to it until that expires). Restarting drops that cache in one step instead of // waiting it out. + // The ports this run has already used, and why they are not used twice. + // + // A service that is disposed and a new one that binds the SAME port is the same ORIGIN to the + // client, and the client pools its connections per origin: the next request reuses the socket + // the disposed service closed, and the answer is `fetch failed` — a transport artifact that + // reads exactly like a broken service. It is not rare here, because the port window is shared + // with every other guard that starts a service, so a crowded window pushes consecutive starts + // onto the same few ports. Each port is used once per run. + const usedPorts = new Set(); const startService = async () => { for (let attempt = 0; attempt < 12; attempt += 1) { - const port = 19900 + Math.floor(Math.random() * 300) + attempt; + const port = 19900 + Math.floor(Math.random() * 900) + attempt; + if (usedPorts.has(port)) continue; try { const started = await start({ pluginId: 'mmc-trajectory', @@ -496,7 +553,21 @@ try { signal: controller.signal, logger: { debug: noop, info: noop, warn: noop, error: noop }, }); - return { instance: started, base: 'http://127.0.0.1:' + port }; + usedPorts.add(port); + const startedBase = 'http://127.0.0.1:' + port; + // The service is asked once before the section starts using it. A service that is not + // answering is not a failure of the rule under test; it is a port that is not usable, and + // the next one is tried. Without this probe, "the service never came up" is reported as + // a failed assertion, which is the one thing a guard must not do. + try { + const probe = await fetch(startedBase + '/api/runtime'); + await probe.arrayBuffer(); + } catch (probeError) { + await started.dispose(); + usedPorts.delete(port); + continue; + } + return { instance: started, base: startedBase }; } catch (error) { // The port was taken. Another one is tried; a guard that gave up here would be a // guard that sometimes checks nothing. @@ -511,7 +582,21 @@ try { } const get = async (path) => { - const response = await fetch(base + path); + // One retry, and only for a TRANSPORT failure: the client reuses a pooled connection that + // the service has closed, and the retry is answered on a fresh one. A service that is broken + // answers with a status and a body, and that is what every assertion below reads — so the + // retry cannot hide a defect; it only stops a dead socket from being read as one. + let response = null; + let lastError = null; + for (let attempt = 0; attempt < 2; attempt += 1) { + try { + response = await fetch(base + path); + break; + } catch (error) { + lastError = error; + } + } + if (response === null) throw lastError; let body = null; try { body = await response.json(); } catch { body = null; } return { status: response.status, body }; @@ -1131,6 +1216,86 @@ try { (uncoveredAnswer.body && uncoveredAnswer.body.hits ? uncoveredAnswer.body.hits : []) .map((hit) => hit.index), [1, 2, 3]); } + // 6d — the cap, and the order the cap is taken in. + // + // The hits are capped, and the cap is about WHICH records the answer is about: the rebuilt path + // walks the body turn by turn and keeps the first N matches it meets, so it keeps the first N in + // the LEDGER's order. A path that keeps the first N while it walks the generations keeps the + // first N in the order the generations happened to be in — and when the two orders disagree, that + // is a different SET of records, with the same count and the same fingerprint. Nothing + // downstream can notice that: the page turns the hits into a set of record numbers and lights up + // the rows in that set. + const capBody = await get('/api/trajectory?session=' + CAP); + assert('the conversation with more matches than the cap holds has 901 records', + capBody.status === 200 && versionsOf(capBody.body).length === 901, + 'records: ' + (capBody.body ? versionsOf(capBody.body).length : 'no body')); + // The first three records of the body in the ledger's order: the first turn's first record, then + // the two that arrived after the rotation. The ORDER is written out by hand and the versions come + // from the body's own records, so the expectation cannot move together with the answer it checks. + const capVersionByIndex = new Map( + (capBody.body && capBody.body.turns ? capBody.body.turns : []).flatMap((turn) => (turn.records || [])) + .map((record) => [record.index, record.v])); + const capFirstThree = [1, 302, 303]; + const capFingerprint = recordSetFingerprint( + capFirstThree.map((index) => capVersionByIndex.get(index) ?? '')); + const capRebuilt = await get('/api/trajectory/search?session=' + CAP + + '&q=CAP ORDER&n=3&fp=' + capFingerprint); + assert('the search answers the set the page holds', capRebuilt.status === 200, why(capRebuilt)); + const capRebuiltHits = (capRebuilt.body && capRebuilt.body.hits ? capRebuilt.body.hits : []); + // The cap bites here: more records match than the answer may carry, and the count says so. + assert('and the answer carries the cap while the count is the whole number of matches', + capRebuiltHits.length === 500 && Boolean(capRebuilt.body) && capRebuilt.body.total === 901, + 'hits: ' + capRebuiltHits.length + ' total: ' + (capRebuilt.body ? capRebuilt.body.total : '?')); + // The first 500 in the ledger's order are the first turn's 500: its first record, and the 499 + // that arrived after the rotation. The first 500 in FILE order are the first 500 by number. + const capLedgerFirst500 = [1]; + for (let i = 302; i <= 800; i += 1) capLedgerFirst500.push(i); + const capFileFirst500 = []; + for (let i = 1; i <= 500; i += 1) capFileFirst500.push(i); + check('and they are the first turn\'s records, in the ledger\'s order', + capRebuiltHits.map((hit) => hit.index), capLedgerFirst500); + assert('and the fixture is one where the two orders disagree about the first records', + JSON.stringify(capLedgerFirst500) !== JSON.stringify(capFileFirst500)); + const capSurfaces = await waitForSegmentsWith(home, 'cap order', 2); + assert('and it published a segment for each of the two generations', + capSurfaces.length === 2, 'surfaces holding the marker: ' + capSurfaces.length); + + // THE REBUILD IS TAKEN OUT OF THE PICTURE, the same way as in 6b: every line of the active file + // stops parsing while the file keeps its length and its timestamp, so the segments still match + // their source — and a rebuild would see only the 301 records of the sealed generation, which + // cannot answer with 500 hits over 901 records. + const capActiveFile = join(capDir, 'messages.jsonl'); + const capStat = await stat(capActiveFile); + const capText = await readFile(capActiveFile, 'utf8'); + await writeFile(capActiveFile, capText.replace(/"message_id":/g, '"message_id"X'), 'utf8'); + await utimes(capActiveFile, capStat.mtimeMs / 1000, capStat.mtimeMs / 1000); + await instance.dispose(); + { + const restarted = await startService(); + instance = restarted.instance; + base = restarted.base; + } + const capCached = await get('/api/trajectory/search?session=' + CAP + + '&q=CAP ORDER&n=3&fp=' + capFingerprint); + assert('a capped search accepts the unchanged displayed set', + capCached.status === 200, why(capCached)); + const capCachedHits = (capCached.body && capCached.body.hits ? capCached.body.hits : []); + const capCachedIds = capCachedHits.map((hit) => hit.index); + const capRebuiltIds = capRebuiltHits.map((hit) => hit.index); + // The SET, not the order: two answers that differ only in which records they are about are two + // answers, and the page lights up the rows of the set the answer names. + // + // The two comparisons are `assert` rather than `check` on purpose: a control names the rule it + // broke, and the name a control reads is the whole line after the `- `, so a rule that carries + // its values with it can never be named. The values are in the two checks below. + const capCachedSet = digestOf(JSON.stringify(capCachedIds.slice().sort((a, b) => a - b))); + const capRebuiltSet = digestOf(JSON.stringify(capRebuiltIds.slice().sort((a, b) => a - b))); + assert('and it answers with the same records as the rebuild', capCachedSet === capRebuiltSet); + check('and the records it names are the first turn\'s, in the ledger\'s order', + capCachedIds.slice(0, 8), capRebuiltIds.slice(0, 8)); + assert('and in the same order as the rebuild', + digestOf(JSON.stringify(capCachedIds)) === digestOf(JSON.stringify(capRebuiltIds))); + check('and the count is the count', capCached.body && capCached.body.total, 901); } } finally { // The server is released here and not at the end of the happy path: an assertion that @@ -1142,8 +1307,13 @@ try { else process.env.MMC_TRAJECTORY_ROOT = previousRoot; } } catch (error) { + // The cause is named: a transport failure and a broken service look the same from here, and a + // reader who cannot tell them apart has to guess. `ECONNRESET` is the client's pooled + // connection being reused after the service closed it; `ECONNREFUSED` is a service that is not + // listening at all. failures.push('the body of the session on screen is reachable by its id -- ' - + (error && error.message)); + + (error && error.message) + + (error && error.cause && error.cause.code ? ' (' + error.cause.code + ')' : '')); } /* ===================================================================== From 6e2b106138fea5bb673b0871fc6fa63aa79adadc Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 03:30:24 +0800 Subject: [PATCH 53/95] Ledger: keep the window's first group on both paths, and price the rows a visit drops Two defects in the ledger's group lifecycle. The stale-group release protected ONE key. The rebuild path protected the window's first group and the visit path protected the group that had just arrived, so on the visit path the window's first group was cleared: the reader's own view of the top of the ledger was emptied by scrolling. The release now protects a SET of keys - the group that just arrived and the window's first group. The first group's key is read from the table's first segment rather than kept aside, because the table has one writer and its first segment is where that key is. A released group also kept the height it was measured at when it was on screen, and the rows it had dropped were not priced at all, so the ledger's own accounting of the space it used was wrong. The visit path now writes back the height the group has now, priced by the ledger's calibration - the one number the table is calibrated with - rather than by a measurement taken while the group was off screen, where the browser reports a prediction instead of a measurement. --- .../mmc-trajectory/miniapp/client/index.html | 141 ++++++++++++++---- 1 file changed, 114 insertions(+), 27 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index a49eb7eb..fee0f767 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -7155,6 +7155,21 @@

检视

*/ var ledgerContent = new Map(); + /** + * The row heights the last build calibrated: the head, a row and a band. + * + * The build measures them off the first batch, which is the only part of the table that + * has been laid out for real. The visit path needs them as well: it has to bring the + * height a released group keeps up to date with what the group holds now, and a group + * that has gained a row since it was measured is taller by a row's worth of height. + * + * One writer: the build, right after it has measured the batch. The visit path reads it + * and never writes it. + * + * @type {{head: number, row: number, band: number}|null} + */ + var ledgerCalibration = null; + /** * The observer that fills a group when the reader reaches it. * @@ -7207,36 +7222,55 @@

检视

* * Three rules, and the order matters. A group the reader has left goes when the idle * timeout has passed; when more groups are live than the cap allows, the oldest goes - * first; and the group the caller is holding on to is none of the reader's visits at - * all, so it is counted rather than timed. + * first; and the groups the caller is holding on to are none of the reader's visits at + * all, so they are counted rather than timed. * - * That last one is what keeps the bound about the rows on screen. The anchored group - * holds its rows whatever the reader is doing — it is where the reader arrives, and every - * other group's height is predicted from it — so a bound on the reader's groups alone - * would leave one group holding its rows outside the count. Counting it here makes the - * two sets the same set: the groups that are live are the groups that hold their rows. + * That last one is what keeps the bound about the rows on screen. The groups the caller + * holds on to hold their rows whatever the reader is doing — the window's first group is + * where the reader arrives and every other group's height is predicted from it — so a + * bound on the reader's groups alone would leave one group holding its rows outside the + * count. Counting them here makes the two sets the same set: the groups that are live + * are the groups that hold their rows. * * One decision and two callers, and that is the point of it. The bound is about what the * reader is holding at this moment, not only about what a rebuild leaves behind: a reader * who walks through more groups than the cap allows between two rebuilds is holding all * of them until the next poll comes round. So the rebuild and the visit both come through - * here, and they cannot disagree about which groups go — the same order, the same count, - * the same group held back. + * here, and they cannot disagree about which groups go — the same order, the same count. + * + * They do not hold on to the same groups, and the difference is the point. The rebuild + * holds on to the window's first group. The visit holds on to that group as well, and to + * the group the reader has just reached: a visit that protected only the group the + * reader had just reached would let the window's first group go — the idle timeout + * would take it, or the cap would reach for it as the oldest one there is — and the + * group every height is predicted from would lose its rows while the reader was on the + * page. * * This decides. The rows are taken away by the build, which writes the released groups * as it writes the table, and by the visit, which takes the rows out of a table that is * already on screen. * - * @param {string} keepKey the group that holds its rows whatever else happens: the - * window's first group on a rebuild, the group the reader has just reached on a visit + * @param {string[]} keepKeys the groups that hold their rows whatever else happens: the + * window's first group, and on a visit the group the reader has just reached as well. + * The group the reader has just reached is the last of them, so that a visit in the same + * millisecond as another one cannot be put at the head of the queue. * @returns {string[]} the groups that have to give up their rows */ - function releaseStaleGroups(keepKey) { + function releaseStaleGroups(keepKeys) { var now = Date.now(); var live = []; var doomed = []; + var kept = []; + var keptIndex = Object.create(null); + var keys = Array.isArray(keepKeys) ? keepKeys : [keepKeys]; + for (var k = 0; k < keys.length; k += 1) { + if (keys[k] === null || keys[k] === undefined) continue; + if (keptIndex[keys[k]]) continue; + keptIndex[keys[k]] = true; + kept.push(keys[k]); + } ledgerState.forEach(function (state, key) { - if (key === keepKey) return; + if (keptIndex[key]) return; if (!state.live) return; if (now - state.lastSeen >= LEDGER_IDLE_MS) { state.live = false; @@ -7245,15 +7279,18 @@

检视

} live.push({ key: key, at: state.lastSeen }); }); - // The group the caller is holding on to is counted with the current time, so it is - // never the oldest one the cap reaches for. Its lease is not consulted: it does not - // have one. + // The groups the caller is holding on to are counted with the current time, so they are + // never the oldest ones the cap reaches for. Their leases are not consulted: they do + // not have one. // - // It is counted after the others, so that a visit in the same millisecond as another - // one — a tie the clock cannot break — cannot put it at the head of the queue. The - // group the reader has just reached is the one the cap must not take. - ledgerStateOf(keepKey).live = true; - live.push({ key: keepKey, at: now }); + // They are counted after the others, so that a visit in the same millisecond as another + // one — a tie the clock cannot break — cannot put the group the reader has just reached + // at the head of the queue. The group the reader has just reached is the last of them, + // and the cap must not take it. + for (var j = 0; j < kept.length; j += 1) { + ledgerStateOf(kept[j]).live = true; + live.push({ key: kept[j], at: now }); + } if (live.length <= LEDGER_LIVE_GROUPS) return doomed; live.sort(function (a, b) { return a.at - b.at; }); for (var i = 0; i < live.length - LEDGER_LIVE_GROUPS; i += 1) { @@ -7265,6 +7302,42 @@

检视

return doomed; } + /** + * The key of the window's first group. + * + * The window's first group is the first section of the table: the build writes the groups + * in the order of the window, so the first section is the group the reader arrives at and + * the one every other group's height is predicted from. + * + * It is read out of the table rather than remembered in a variable of its own. A variable + * would be a second copy of the same fact, and the two copies would have to be kept in + * step; the table has one writer — the build — and its first section is the window's + * first group by construction. + * + * @returns {string|null} null when the table has no group in it + */ + function ledgerFirstGroupKey() { + var sections = dom.ledger.querySelectorAll(':scope > section.turn-group'); + if (!sections.length) return null; + var content = ledgerContent.get(sections[0]); + return content ? content.key : null; + } + + /** + * The groups that have to give up their rows before a rebuild writes the table. + * + * The rebuild and the visit make the same decision, so they come through the same + * function; this is the rebuild's half of it, and it is a function of its own so that the + * two halves cannot drift apart: the rebuild holds on to the window's first group, and + * the visit holds on to that group and to the group the reader has just reached. + * + * @param {Array<{turn: any}>} groups the window's groups, in order + */ + function releaseStaleGroupsBeforeRebuild(groups) { + if (!groups.length) return; + releaseStaleGroups([turnKey(groups[0].turn)]); + } + /** * Put a group's rows back into it. * @@ -7298,7 +7371,13 @@

检视

if (!alreadyLive || !state.height) measureLedgerGroup(section, group); // The cap, on the visit path as well as on the rebuild: the number of groups holding // their rows is bounded at any moment, not only right after a rebuild. - releaseLedgerRows(releaseStaleGroups(group.key)); + // + // Two groups are held on to: the group the reader has just reached, and the window's + // first group, which holds its rows whatever the reader is doing — it is where the + // reader arrives and the one every other group's height is predicted from. The group + // the reader has just reached is the last of the two, so that a tie the clock cannot + // break cannot put it at the head of the queue. + releaseLedgerRows(releaseStaleGroups([ledgerFirstGroupKey(), group.key])); } /** @@ -7373,10 +7452,15 @@

检视

section.setAttribute('data-rows', 'released'); section.classList.add('is-released'); // The measured height, brought up to date with what the group holds now; the height - // read above otherwise. There is no calibration here to price a difference with — - // the calibration belongs to the build that is running — so the height the group - // was measured at is kept as it is, and the next rebuild brings it up to date. - holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, null)); + // read above otherwise. The difference is priced at the row heights the last build + // calibrated, which is the same arithmetic the build uses. + // + // The height read above cannot be used on its own: a group the browser has skipped + // measures by its own contain-intrinsic-size, so for the groups that are off screen + // it is the prediction and not a measurement — and writing it would replace the + // height the group was really measured at with a guess. The calibration is what + // prices the difference, and it is what the build prices it with. + holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, ledgerCalibration)); } } @@ -7789,7 +7873,7 @@

检视

// up oldest first. The same decision the visit path makes, so the two cannot disagree // about which groups go; applied by this build, which writes the released groups as it // writes the table. - if (groups.length) releaseStaleGroups(turnKey(groups[0].turn)); + releaseStaleGroupsBeforeRebuild(groups); function buildStep() { // A newer renderLedger() owns the ledger now. Stop without touching it: the rows half @@ -7878,6 +7962,9 @@

检视

// viewport and get measured for real. if (!build.heights) { build.heights = calibrateRowHeights(ledger); + // The visit path prices the difference a released group has with the same + // calibration, so the last one is kept where the visit path can reach it. + ledgerCalibration = build.heights; // The groups that were built without their rows have nothing to be sized from, and a // group left unsized collapses to nothing the moment the class goes on. They are // sized now, from the calibration, with the same arithmetic every other group is From 35755c411b3949308584ab6c64062370d8555738 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 03:30:25 +0800 Subject: [PATCH 54/95] Ledger: pin the window's first group on both paths, and the height the visit path writes - The lifecycle guard now drives the rebuild path through the shipped release, so the cap on the groups that still hold their rows is checked on that path too, and not only on the visit path. - A group the cap released keeps the height it has now, and the visit path writes back the height the group was measured at. The check names the number, so a control can catch it by name. - The window's first group keeps its rows when the clock cannot break the tie. - Five mutations, each caught by name: the visit path drops the window's first group, the release protects one key instead of a set, the visit path writes back a stale height, the rebuild path stops releasing, and the tie is broken the wrong way round. --- .../tests/mutate-ledger-lifecycle.py | 75 ++++++- .../tests/verify-ledger-lifecycle.mjs | 192 +++++++++++++++++- 2 files changed, 247 insertions(+), 20 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py index 7bfc4f36..e3c52adf 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -51,15 +51,15 @@ # its own lease instead of the current time, a group the reader left long ago is the # oldest one there is, and the cap reaches for it first. ("the window's first group is released with the others", - [(" live.push({ key: keepKey, at: now });\n", - " live.push({ key: keepKey, at: ledgerStateOf(keepKey).lastSeen });\n")], + [(" live.push({ key: kept[j], at: now });\n", + " live.push({ key: kept[j], at: ledgerStateOf(kept[j]).lastSeen });\n")], "the window's first group is one of the groups that hold their rows"), # The same group left out of the count altogether: it goes on holding its rows past the # bound, so the bound is about one set of groups while the rows on screen are another. ("the window's first group is not counted among the groups that hold their rows", - [(" ledgerStateOf(keepKey).live = true;\n" - " live.push({ key: keepKey, at: now });\n", + [(" ledgerStateOf(kept[j]).live = true;\n" + " live.push({ key: kept[j], at: now });\n", "")], "the window's first group is one of the groups that hold their rows"), @@ -299,21 +299,76 @@ # rebuild leaves behind. Without the cap on the visit path, a reader who walks through # more groups than the cap allows between two rebuilds is holding all of them. ("reader visits do not enforce the cap on the groups holding their rows", - [(" releaseLedgerRows(releaseStaleGroups(group.key));\n", "")], + [(" releaseLedgerRows(releaseStaleGroups([ledgerFirstGroupKey(), group.key]));\n", "")], "reader visits cannot increase the held row groups past the ceiling between rebuilds"), + # The visit path holds on to the group the reader has just reached. It has to hold on to + # the window's first group as well: that group holds its rows whatever the reader is + # doing, it is where the reader arrives, and every other group's height is predicted from + # it. A visit that protected only the group the reader had just reached let the window's + # first group go — the idle timeout took it, or the cap reached for it as the oldest one — + # and the group the heights are predicted from lost its rows while the reader was on the + # page. + ("the visit path does not protect the window's first group", + [(" releaseLedgerRows(releaseStaleGroups([ledgerFirstGroupKey(), group.key]));\n", + " releaseLedgerRows(releaseStaleGroups([group.key]));\n")], + ["the window's first group keeps its rows after a round of visits", + "the window's first group keeps its rows when the reader comes back to another group"]), + # A group the cap released gives up its rows at once. The rows are the DOM, and a group # that has given them up must not still be holding them. ("a group the cap released keeps its rows", [(" if (list) clear(list);\n", "")], "a group the cap released holds no rows"), - # And it keeps the height it had. The height is the reader's place in a session they have - # scrolled through, and a group that has given up its rows without keeping it takes the - # document down under them. + # And it keeps the height it has now. The height is the reader's place in a session they + # have scrolled through, and a group that has given up its rows without keeping it takes + # the document down under them. ("a group the cap released keeps no reserved height", - [(" holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, null));\n", "")], - "a group the cap released keeps its reserved height"), + [(" holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, ledgerCalibration));\n", "")], + "a group the cap released keeps the height it has now"), + + # The height a released group keeps has to describe the content it holds now, and the + # visit path has no calibration of its own to price the difference with. Passing no + # calibration writes the old measurement back unchanged: a group measured at 1,000 px with + # three rows in it and holding four rows by the time it is released on the visit path keeps + # 1,000 px instead of 1,036 px, and the document under the reader comes up short by the row + # that was added. + ("the visit path writes back the height the group was measured at", + [(" holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, ledgerCalibration));\n", + " holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, null));\n")], + "a group the cap released keeps the height it has now"), + + # The rebuild is one of the two places that decides the bound, and the fixture's rebuild() + # calls the shipped function that does it. A rebuild that stopped releasing the groups the + # reader had left came out with more groups holding their rows than the cap allows, and + # every assertion in the guard still passed. + ("the rebuild does not release the groups that have to give up their rows", + [(" releaseStaleGroups([turnKey(groups[0].turn)]);\n", "")], + "after a rebuild, the number of groups holding their rows is at most the cap"), + + # And the call site itself: the rebuild asks for the release before it writes the table. + ("the rebuild does not ask for the release before it writes the table", + [(" releaseStaleGroupsBeforeRebuild(groups);\n", "")], + "the rebuild releases the groups that have to give up their rows before it writes the table"), + + # The groups the caller holds on to are counted after the ones the reader has visited, so + # that a tie the clock cannot break cannot put them at the head of the queue. The cap + # reaches for the oldest, and the group the reader has just reached must not be the one it + # reaches for. Counted first, the window's first group is the head of the queue and the + # cap takes it. + ("the groups the caller holds on to are counted before the ones the reader has visited", + [(" for (var j = 0; j < kept.length; j += 1) {\n" + " ledgerStateOf(kept[j]).live = true;\n" + " live.push({ key: kept[j], at: now });\n" + " }\n", ""), + (" ledgerState.forEach(function (state, key) {\n", + " for (var j = 0; j < kept.length; j += 1) {\n" + " ledgerStateOf(kept[j]).live = true;\n" + " live.push({ key: kept[j], at: now });\n" + " }\n" + " ledgerState.forEach(function (state, key) {\n")], + "the window's first group keeps its rows when the clock cannot break the tie"), ] original = io.open(CLIENT, encoding="utf-8", newline="").read() diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs index be3a1716..d2398b2a 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -67,6 +67,12 @@ const LIFECYCLE_FNS = [ 'ledgerStateOf', 'ledgerHoldsRows', 'releaseStaleGroups', + // The window's first group, read out of the table: the visit path holds it on to as well as + // the group the reader has just reached. + 'ledgerFirstGroupKey', + // The rebuild's half of the release decision, so that a rebuild here is the rebuild the + // client makes rather than an approximation of it. + 'releaseStaleGroupsBeforeRebuild', 'fillLedgerGroup', 'holdLedgerHeight', 'heldLedgerHeight', @@ -197,6 +203,9 @@ function harness() { Date: { now: () => box.clock }, ledgerState: new Map(), ledgerContent: new Map(), + // The row heights the last build calibrated. The visit path prices the difference a + // released group has with them, so the build writes them and the visit path reads them. + ledgerCalibration: null, // The ledger the table is in. The shipped code reads it when it has to take the rows out // of a group the reader is no longer on. dom: { ledger: makeEl('div') }, @@ -283,6 +292,12 @@ function buildHarness(options) { * The groups are the window's, the first one is the group the window is anchored on, and the * state carries over — a rebuild does not reset the reader's lease, and proving that it does * not is the whole point of the rule below. + * + * The release step is part of it, and it is the shipped one: the fixture calls the function + * the client calls, so a rebuild here is a rebuild of the client rather than an approximation + * of it. Leaving it out is what made this file blind to the rebuild's half of the bound — a + * build that stopped releasing the groups the reader had left came out with more groups + * holding their rows than the cap allows, and every assertion here still passed. */ function rebuild(h, groups) { const s = h.sandbox; @@ -296,6 +311,7 @@ function rebuild(h, groups) { // The total the build reports while it is running. It lives in renderLedger() beside the // build, and a window longer than one batch is the case that reads it. s.buildTotal = groups.reduce((n, g) => n + g.entries.length, 0); + s.releaseStaleGroupsBeforeRebuild(groups); s.buildStep(); } @@ -511,12 +527,12 @@ function measureGroup(h, key, entries, bands, height) { s.ledgerState.set('a', { live: true, lastSeen: 1000, height: 0 }); h.clock = 1000 + s.LEDGER_IDLE_MS - 1; - s.releaseStaleGroups('somewhere-else'); + s.releaseStaleGroups(['somewhere-else']); assert('a group the reader is still within reach of keeps its rows', s.ledgerState.get('a').live === true); h.clock = 1000 + s.LEDGER_IDLE_MS; - s.releaseStaleGroups('somewhere-else'); + s.releaseStaleGroups(['somewhere-else']); assert('a group the reader has left gives its rows up once the idle time has passed', s.ledgerState.get('a').live === false); } @@ -532,7 +548,7 @@ function measureGroup(h, key, entries, bands, height) { // — it has no visit of its own to be dated by — so a clock that sat behind the leases would // make the anchored group look like the oldest one, which is the one thing it never is. h.clock = 2000; - s.releaseStaleGroups('g' + (n - 1)); + s.releaseStaleGroups(['g' + (n - 1)]); const live = [...s.ledgerState.values()].filter((v) => v.live).length; assert('the number of groups holding their rows is bounded', live === s.LEDGER_LIVE_GROUPS); @@ -568,7 +584,7 @@ function measureGroup(h, key, entries, bands, height) { for (let i = 0; i < s.LEDGER_LIVE_GROUPS + 1; i += 1) { s.ledgerState.set('g' + i, { live: true, lastSeen: 4000 + i, height: 0 }); } - s.releaseStaleGroups('anchor'); + s.releaseStaleGroups(['anchor']); const live = [...s.ledgerState.values()].filter((v) => v.live).length; assert('the window\'s first group is one of the groups that hold their rows', s.ledgerState.get('anchor').live === true); @@ -608,7 +624,7 @@ function measureGroup(h, key, entries, bands, height) { && sectionOf(h, 1).getAttribute('data-rows') === 'live'); h.clock = 17000; - s.releaseStaleGroups('k1'); + s.releaseStaleGroups(['k1']); rebuild(h, groups); assert('a group the reader has left gives its rows up while the ledger keeps being rebuilt', s.ledgerState.get('k2').live === false @@ -692,6 +708,16 @@ function measureGroup(h, key, entries, bands, height) { * The visits below are one after another, and each is within the idle timeout of the one * before it, so the idle rule is not what releases the groups — the cap is. And the groups * that are counted are the sections that really hold their rows, not the flags in the state. + * + * The second group is the one the cap releases, and it carries the height rule with it. It is + * measured at 1,000 px with three rows in it; a late reply makes it four rows, and the rebuild + * that brings the reply writes the group with the four rows in it. The rebuild does not + * measure, so the group still carries the height it was measured at — 1,000 px for three rows — + * while the height it really has is 1,036 px. When the cap takes it on the visit path, the + * height it keeps has to be the height it has now: 1,036 px, the measured height and the row + * that was added, priced at the calibrated row height. The visit path used to write the old + * measurement back unchanged, so the group came up 36 px short of the content it was holding, + * and the document under the reader came up short by that much until the next rebuild. */ { const h = buildHarness(); @@ -699,10 +725,27 @@ function measureGroup(h, key, entries, bands, height) { const n = s.LEDGER_LIVE_GROUPS + 2; const groups = []; for (let i = 0; i < n; i += 1) { - groups.push({ turn: { index: i + 1 }, entries: entriesOf(2, 0, 'g' + i) }); + groups.push({ turn: { index: i + 1 }, entries: entriesOf(3, 0, 'g' + i) }); } h.clock = 1000; rebuild(h, groups); + // The reader is on the second group, and it is laid out for real at 1,000 px. + const second = sectionOf(h, 1); + second.getBoundingClientRect = () => ({ height: 1000, width: 900, top: 0, bottom: 1000 }); + s.fillLedgerGroup(second); + h.clock += 10; + s.fillLedgerGroup(sectionOf(h, 0)); + // A late reply: every group has a fourth row now, and the rebuild that brings it writes the + // groups with the four rows in it. + const grown = []; + for (let i = 0; i < n; i += 1) { + grown.push({ turn: { index: i + 1 }, entries: entriesOf(4, 0, 'g' + i) }); + } + h.clock += 10; + rebuild(h, grown); + // The second group is laid out at the height it really has now. + const secondGrown = sectionOf(h, 1); + secondGrown.getBoundingClientRect = () => ({ height: 1036, width: 900, top: 0, bottom: 1036 }); // The reader walks through the window, one group at a time. for (let i = 0; i < n; i += 1) { h.clock += 10; @@ -714,13 +757,136 @@ function measureGroup(h, key, entries, bands, height) { }); assert('reader visits cannot increase the held row groups past the ceiling between rebuilds', holding.length <= s.LEDGER_LIVE_GROUPS); - // And the group the cap released gives up its rows at once, and keeps the height it had. - const released = sectionOf(h, 0); + // The window's first group is where the reader arrives, and every other group's height is + // predicted from it, so it holds its rows whatever the reader is doing — including after a + // round of visits that fills more groups than the cap allows. A visit that held on to the + // group the reader had just reached and nothing else would let this one go: the idle + // timeout would take it, or the cap would reach for it as the oldest one there is, and the + // group the heights are predicted from would lose its rows while the reader was on the page. + const first = sectionOf(h, 0); + assert('the window\'s first group keeps its rows after a round of visits', + first.getAttribute('data-rows') === 'live' + && first.querySelector('ul.row-list').children.length > 0); + // And the group the cap released gives up its rows at once, and keeps the height it has now. + const released = sectionOf(h, 1); assert('a group the cap released holds no rows', released.getAttribute('data-rows') === 'released' && released.querySelector('ul.row-list').children.length === 0); - assert('a group the cap released keeps its reserved height', - typeof released.style.minHeight === 'string' && released.style.minHeight !== ''); + // The height is the number the group really has, not merely a non-empty string: 1,036 px, + // the 1,000 px it was measured at and the row that was added, priced at the calibrated row + // height. A shape check would pass on a height of 1 px, which is the opposite of what the + // reader is owed. + assert('a group the cap released keeps the height it has now', + released.style.minHeight === '1036px'); +} + +/* ---------- 3f. the window's first group survives the idle timeout ---------- */ + +/* + * The window's first group holds its rows whatever the reader is doing, and the visit path + * has to hold it on to as well as the group the reader has just reached. + * + * The case: the reader is on the first group, the observer gives it a lease, and the reader is + * away long enough for the idle timeout to pass. The reader comes back on the second group, + * and that one visit is enough to take the first group's rows away — the idle timeout reaches + * for it, because it is the oldest lease there is — and the group every height is predicted + * from loses its rows while the reader is on the page. + */ +{ + const h = buildHarness(); + const s = h.sandbox; + const groups = [ + { turn: { index: 1 }, entries: entriesOf(3, 0, 'g0') }, + { turn: { index: 2 }, entries: entriesOf(3, 0, 'g1') }, + { turn: { index: 3 }, entries: entriesOf(3, 0, 'g2') } + ]; + h.clock = 1000; + rebuild(h, groups); + // The reader is on the first group, and the observer gives it a lease. + s.fillLedgerGroup(sectionOf(h, 0)); + // The reader is away long enough for the idle timeout to pass, and comes back on the second + // group. One visit is enough to release the first one. + h.clock = 1000 + s.LEDGER_IDLE_MS; + s.fillLedgerGroup(sectionOf(h, 1)); + const first = sectionOf(h, 0); + assert('the window\'s first group keeps its rows when the reader comes back to another group', + first.getAttribute('data-rows') === 'live' + && first.querySelector('ul.row-list').children.length > 0); +} + +/* ---------- 3g. the rebuild applies the bound too ---------- */ + +/* + * The bound is about the groups that hold their rows, and the rebuild is one of the two + * places that decides it. + * + * The fixture's rebuild() used to leave that step out — it called the build and nothing else — + * so a build that stopped releasing the groups the reader had left was a build this file + * could not see: the table came out with more groups holding their rows than the cap allows, + * and every assertion in this file still passed. + * + * The case: the reader has been on six groups, which is the cap, and then the window widens — + * a group comes in above — so the rebuild's first group is a new one. The rebuild counts it, + * the count is one over the cap, and the oldest of the reader's groups goes. Without that step + * the new group is simply added and the table holds seven groups' rows. + */ +{ + const h = buildHarness(); + const s = h.sandbox; + const groups = []; + for (let i = 0; i < s.LEDGER_LIVE_GROUPS; i += 1) { + groups.push({ turn: { index: i + 1 }, entries: entriesOf(2, 0, 'g' + i) }); + } + h.clock = 1000; + rebuild(h, groups); + // The reader walks through the window: six groups, which is the cap. + for (let i = 0; i < s.LEDGER_LIVE_GROUPS; i += 1) { + h.clock += 10; + s.fillLedgerGroup(sectionOf(h, i)); + } + // The window widens: a group comes in above, so the rebuild's first group is a new one. + const wider = [{ turn: { index: 0 }, entries: entriesOf(2, 0, 'w0') }].concat(groups); + h.clock += 10; + rebuild(h, wider); + const holding = s.ledger.children.filter((c) => { + const list = c.querySelector('ul.row-list'); + return c.getAttribute('data-rows') === 'live' && !!list && list.children.length > 0; + }); + assert('after a rebuild, the number of groups holding their rows is at most the cap', + holding.length <= s.LEDGER_LIVE_GROUPS); +} + +/* ---------- 3h. a tie the clock cannot break ---------- */ + +/* + * The cap reaches for the oldest group, and when several groups share a millisecond the order + * the cap sees is the order the groups were counted in. The groups the caller holds on to are + * counted after the ones the reader has visited, so a tie cannot put them at the head of the + * queue — and the group the reader has just reached is the last of them, so the cap cannot + * reach for the group the reader is on. + * + * The case: the reader walks through the window without the clock moving, so every lease is + * the same age, and the count goes one over the cap on the last visit. Counted before the + * groups the reader has visited, the window's first group would be the head of that queue and + * the cap would take it. + */ +{ + const h = buildHarness(); + const s = h.sandbox; + const groups = []; + for (let i = 0; i < s.LEDGER_LIVE_GROUPS + 1; i += 1) { + groups.push({ turn: { index: i + 1 }, entries: entriesOf(2, 0, 'g' + i) }); + } + h.clock = 1000; + rebuild(h, groups); + // The reader walks through the window without the clock moving: every lease is the same age. + for (let i = 1; i < s.LEDGER_LIVE_GROUPS + 1; i += 1) { + s.fillLedgerGroup(sectionOf(h, i)); + } + const first = sectionOf(h, 0); + assert('the window\'s first group keeps its rows when the clock cannot break the tie', + first.getAttribute('data-rows') === 'live' + && first.querySelector('ul.row-list').children.length > 0); } /* ---------- 4. the rows come back when the reader reaches the group ---------- */ @@ -859,6 +1025,12 @@ assert('a group the build is not holding says so too', const renderSrc = functionSource('renderLedger') || ''; assert('the ledger drops what it knew about the groups it just emptied', /clear\(ledger\);/.test(renderSrc) && /ledgerContent\.clear\(\);/.test(renderSrc)); +// The rebuild asks for the release before it writes the table. The behavioural half of this +// rule is the assertion about the number of groups holding their rows after a rebuild; this +// one is the call site, which no behavioural check can reach: renderLedger() is too big to +// run here, so the fixture calls the shipped release function itself. +assert('the rebuild releases the groups that have to give up their rows before it writes the table', + /releaseStaleGroupsBeforeRebuild\(groups\);/.test(renderSrc)); const selectSrc = functionSource('selectSession') || ''; assert('the groups of the session being left are forgotten with it', From b9fe499d4f801bc94fa858aeb58acfba59e76143 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 05:57:59 +0800 Subject: [PATCH 55/95] A live ledger group measures itself again when its layout changes, and a window resize measures the live groups that are really rendered. A height is a measurement of a LAYOUT, and a layout can change while the content does not: the rows and the bands can be exactly the same and the height a different one -- the window is resized, the text scale is turned up, the group's own chrome becomes taller. The height a released group keeps was the remembered one, unchanged, so a group whose layout had changed stood shorter than the content it was holding, and the document under the reader came up short by exactly that much. Two things trigger a measurement now, and both of them are places where the group is really laid out: * a visit, when the group's laid-out height is not the height it was measured at; and * a window resize, for the live groups that are really rendered. The resize path is deferred to the next frame, because the layout a resize brings about is not there yet when the event arrives. It touches only the groups that are holding their rows: the rows are not built there, because the second copy of every row is the whole cost of getting it wrong. How the two paths tell a rendered group from a skipped one: the size the browser answers with while a group is off screen is the size written on it, so a group that reads back exactly the number it was given is being sized rather than laid out, and what it reads is a prediction. A prediction written down as a measurement is a guess the next rebuild reads back as a fact, so those groups are left alone. What this does NOT do, on purpose: * the build path still writes no measurement -- the build never sees a group laid out for real; * a group that was off screen when the layout changed, and that is never visited again, keeps the height it was last really measured at. That half cannot be reached at the moment the group is released, because off screen the browser answers with the group's own `contain-intrinsic-size`, which is a prediction. It is the same limit the build path has, and it is written down next to the code. The rows of a group are still built once, and the measurement is still written by one place only. --- .../mmc-trajectory/miniapp/client/index.html | 103 +++++++++++++++++- 1 file changed, 99 insertions(+), 4 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index fee0f767..da6cc14f 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -7365,10 +7365,19 @@

检视

section.style.minHeight = ''; } // The height the group really has, taken on both paths — see measureLedgerGroup(). - // A group that is already holding its rows and has a height of its own has nothing - // new to say: nothing about its content has changed, so the height it was measured - // at is the height it has. - if (!alreadyLive || !state.height) measureLedgerGroup(section, group); + // A group that is already holding its rows is measured again when its LAYOUT has + // changed, and the test for that is the layout and not the content: the rows and the + // bands can be exactly the same and the height a different one — the window resized, + // the text scale turned up, the group's own chrome taller. The height that was + // measured then describes a layout that is no longer there, and the height a released + // group keeps is that height unchanged, because the counts it was measured at have + // not changed either — so the group stands shorter than the content it is holding, + // and the document under the reader comes up short by exactly that much. + // + // The group the reader is on is rendered, so the height it has now is its layout and + // not the prediction an off-screen group is sized by; the off-screen groups are the + // calibration-priced path's business, and this does not touch them. + if (!alreadyLive || !state.height || ledgerLayoutChanged(section, state)) measureLedgerGroup(section, group); // The cap, on the visit path as well as on the rebuild: the number of groups holding // their rows is bounded at any moment, not only right after a rebuild. // @@ -7420,6 +7429,84 @@

检视

state.heightBands = measuredBands; } + /** + * Whether a group's layout is not the one its remembered height was measured at. + * + * The remembered height is a measurement of a layout that was there then, and a layout + * can change without the content changing: the window is resized, the text scale is + * turned up, the group's own chrome becomes taller. The rows and the bands can be + * exactly the same and the height a different one — and the height a released group + * keeps is the remembered one unchanged, because the counts it was measured at have not + * changed either. So the group stands shorter than the content it is holding, and the + * document under the reader comes up short by exactly that much. + * + * So the test is the height itself: the group has a height of its own now, and it is + * not the one that was remembered. + * + * A group with no remembered height has no layout to compare against, and this says + * nothing about it: the caller has a rule of its own for that case, and it is the one + * that has to stay — the group the build made live has to record the height it was + * measured at, because the build never measures. + * + * @param {HTMLElement} section + * @param {{height: number}} state + * @returns {boolean} + */ + function ledgerLayoutChanged(section, state) { + if (!state.height) return false; + return Math.round(section.getBoundingClientRect().height) !== state.height; + } + + var ledgerMeasureFrame = null; + + /** + * Measure the groups that are on screen again, after the window has changed shape. + * + * WHY: a height is a measurement of a layout, and a resize changes the layout of every + * group that is rendered without changing a row or a band in it. The measurement is the + * observer's, and the observer is not told that the window changed shape — so a group + * the reader is looking at keeps the height it was measured at, and that height + * describes the layout before the resize, until the reader scrolls away from it and + * back. And when the cap releases it, the height it reserves is that stale number, so + * the document under the reader comes up short by exactly the difference. + * + * The measurement is deferred to the next frame, because the layout a resize brought + * about is not there yet when the event arrives. + * + * What it does not claim: a group that was off screen when the layout changed, and is + * never visited again, keeps the height it was last really measured at. Off screen the + * browser answers with the group's own `contain-intrinsic-size`, and that number is a + * prediction — a prediction written down as a measurement is a guess the next rebuild + * reads back as a fact. So a group whose height is the size it was given is left alone, + * and this is the same limit the build path has. + */ + function refreshLedgerMeasurements() { + if (ledgerMeasureFrame !== null) return; + ledgerMeasureFrame = window.requestAnimationFrame(function () { + ledgerMeasureFrame = null; + var sections = dom.ledger.querySelectorAll(':scope > section.turn-group'); + for (var i = 0; i < sections.length; i += 1) { + var section = sections[i]; + // Only the groups that are holding their rows. A group that has given them up is + // sized by the prediction, and the height it keeps is the one it was measured at. + if (section.getAttribute('data-rows') !== 'live') continue; + var group = ledgerContent.get(section); + if (!group) continue; + // The size the browser answers with while a group is off screen is the size + // written on it, so a group that reads back exactly the number it was given is + // being sized rather than laid out — and what it reads is a prediction. + var sizes = String(section.style.containIntrinsicSize || '').match(/(-?[0-9.]+)px/g); + if (sizes) { + var predicted = Math.round(parseFloat(sizes[sizes.length - 1])); + if (Math.round(section.getBoundingClientRect().height) === predicted) continue; + } + // The rows are not built here: a group that is holding them is holding them, and + // the second copy of every row is the whole cost of getting it wrong. + measureLedgerGroup(section, group); + } + }); + } + /** * Take the rows out of the groups that have to give up their rows. * @@ -10751,6 +10838,14 @@

检视

window.addEventListener('scroll', scheduleInspectorDock, { passive: true }); window.addEventListener('resize', scheduleInspectorDock); + // The ledger's groups keep the height they were measured at, and a resize changes the + // layout of every group on screen without changing a row or a band in it. The + // measurement is the observer's, and the observer is not told that the window changed + // shape — so the groups the reader is looking at would keep a height of the layout + // before the resize, and the height a released group reserves would be that stale + // number. + window.addEventListener('resize', refreshLedgerMeasurements); + // Two buttons, one job each, and neither writes the field itself — each names the state its // click goes to and hands it to the one writer. The single button this replaced cycled // folded -> half -> expanded -> folded, which was correct and still wrong to use: reaching From 9b67991ed57c58b70c1adfc46bc8d882356acd12 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 05:58:00 +0800 Subject: [PATCH 56/95] Two rules about the ledger's measurements get an owner each, and a third one keeps the one it had. A live group that is holding its rows re-measures itself when its LAYOUT changes, not only when it has no measurement yet. The rule is owned by 'a live group refreshes its measurement after its layout changes', and the control 'a live group never refreshes its measurement after its layout changes' breaks the call site by dropping only the layout clause -- the fill and the "no height yet" clause stay -- so it suppresses only the refresh of a non-zero height. The height a released group keeps is owned as well: 'the height a released group keeps is the height its layout has now' pins the number the reader sees, and the control 'a live group never refreshes its measurement when the window is resized' breaks the resize path's measurement and nothing else. One consequence is written into the guard rather than papered over. With the measurement refreshed on a rendered group, the difference the height function prices is zero on that path, so the rule the control 'the visit path writes back the height the group was measured at' breaks now lives on the OFF-SCREEN path -- off screen the browser answers with the group's own `contain-intrinsic-size`, which is a prediction, and the height a group reserves is then the measurement it was really taken at, brought up to date with the rows it gained. The new case 'a group released off screen keeps the height it was measured at, brought up to date' owns that half, rather than the control being weakened. The counts: the guard goes from 63 assertions to 67, and the control from 41 cases to 43. Every case is caught by name, and the control restores the shipped source byte for byte. --- .../tests/mutate-ledger-lifecycle.py | 46 +++- .../tests/verify-ledger-lifecycle.mjs | 240 +++++++++++++++++- 2 files changed, 278 insertions(+), 8 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py index e3c52adf..b10aea63 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -291,10 +291,45 @@ # prediction when it was released: three rows and a band laid out at 1,000 px reserved # 208 px, with not one row or band changed. ("a group the build made live never records the height it was measured at", - [(" if (!alreadyLive || !state.height) measureLedgerGroup(section, group);\n", - " if (!alreadyLive) measureLedgerGroup(section, group);\n")], + [(" if (!alreadyLive || !state.height || ledgerLayoutChanged(section, state)) measureLedgerGroup(section, group);\n", + " if (!alreadyLive || ledgerLayoutChanged(section, state)) measureLedgerGroup(section, group);\n")], "a group built live keeps its measured height when a wider window releases it"), + # A height is a measurement of a layout, and a layout can change without the content + # changing: the window is resized, the text scale is turned up, the group's own chrome + # becomes taller. The rows and the bands stay exactly the same and the height is a + # different one — and the height a released group keeps is the remembered one unchanged, + # because the counts it was measured at have not changed either. So a group laid out at + # 1,500 px kept the 1,000 px it was measured at, and the document under the reader came + # up 500 px short of the group it stood for. + # + # The break takes away the refresh and nothing else: the group is still measured when it + # has no height of its own, and the rows are still built once. + ("a live group never refreshes its measurement after its layout changes", + [(" if (!alreadyLive || !state.height || ledgerLayoutChanged(section, state)) measureLedgerGroup(section, group);\n", + " if (!alreadyLive || !state.height) measureLedgerGroup(section, group);\n")], + "a live group refreshes its measurement after its layout changes"), + + # The window is resized and the client is told: the groups that are on screen have to be + # measured again, because a resize changes their layout without changing a row or a band + # in them. A client that does not re-measure them keeps the height it measured them at — + # a height of the layout before the resize — and when the cap releases the group, the + # height it reserves is that stale number, so the document under the reader comes up short + # by the difference. + # + # The break takes away the measurement the resize drives and nothing else: the rows are + # still built once, the cap still releases the same groups, and the measurement the visit + # path takes is untouched. + # + # The boundary, which this break does not touch either: a group that was off screen when + # the layout changed, and is never visited again, keeps the height it was last really + # measured at. Off screen the browser answers with the group's own contain-intrinsic-size, + # and that number is a prediction — a prediction written down as a measurement is a guess + # the next rebuild reads back as a fact. + ("a live group never refreshes its measurement when the window is resized", + [(" measureLedgerGroup(section, group);\n", "")], + "a live group refreshes its measurement when the window is resized"), + # The bound is about what the reader is holding at this moment, not only about what a # rebuild leaves behind. Without the cap on the visit path, a reader who walks through # more groups than the cap allows between two rebuilds is holding all of them. @@ -334,10 +369,15 @@ # three rows in it and holding four rows by the time it is released on the visit path keeps # 1,000 px instead of 1,036 px, and the document under the reader comes up short by the row # that was added. + # + # The group that witnesses it is one the reader has left: it is off screen, so the browser + # skips it and what it reads is the prediction, and the measured height brought up to date + # with the row that was added is the only answer that is not a guess. A group the reader + # is on is measured again when its layout changes, and its measurement is the current one. ("the visit path writes back the height the group was measured at", [(" holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, ledgerCalibration));\n", " holdLedgerHeight(section, heldLedgerHeight(content.key, rows, bands, measured, null));\n")], - "a group the cap released keeps the height it has now"), + "a group released off screen keeps the height it was measured at, brought up to date"), # The rebuild is one of the two places that decides the bound, and the fixture's rebuild() # calls the shipped function that does it. A rebuild that stopped releasing the groups the diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs index d2398b2a..ebfd7ad4 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -45,6 +45,14 @@ function assert(name, cond) { if (cond) pass += 1; else failures.push(name); } +// The values of an assertion, kept apart from its name. A control catches a rule by name, +// and a name that carries a number cannot be matched — so the name says which rule, and the +// numbers go here, where they say how far off it was. +function check(name, got, want) { + const ok = JSON.stringify(got) === JSON.stringify(want); + if (ok) pass += 1; + else failures.push(`${name}: got ${JSON.stringify(got)}, want ${JSON.stringify(want)}`); +} function functionSource(name) { const at = html.indexOf('function ' + name + '('); @@ -106,10 +114,37 @@ if (missing.length) { * because the proof that the new assertions fail is a run of this guard against the broken * code. A client without them is a client whose visited groups are never measured and never * released, and the assertions below say so by failing. + * + * The layout comparison is in here for the same reason: it is the half of the measurement + * that says a group whose layout has changed is measured again, and a client without it + * keeps the height it was measured at — which is what the assertion below catches. */ -const OPTIONAL_FNS = ['measureLedgerGroup', 'releaseLedgerRows']; +const OPTIONAL_FNS = ['measureLedgerGroup', 'releaseLedgerRows', 'ledgerLayoutChanged', + // The refresh the window's resize drives. It is optional for the same reason the other two + // are: a client without it never measures a group again, and the assertion below says so by + // failing. + 'refreshLedgerMeasurements']; const optionalSrc = OPTIONAL_FNS.map((name) => functionSource(name)).filter(Boolean); +/* + * The client's own registration of the resize handler, run in the sandbox. + * + * The case has to deliver a real 'resize' event, and an event only arrives if the client + * registered a handler for it. So the line the client ships is the line that runs here: the + * sandbox's window records the handlers, the case fires the event, and the shipped handler is + * what answers it. A client that does not register one has nothing to answer with, and the + * measurement stays the one it was taken at — which is the failure the new assertion is there + * to catch. + */ +const refreshSrc = functionSource('refreshLedgerMeasurements'); +const resizeRegistration = refreshSrc + ? /window\.addEventListener\(\s*'resize'\s*,\s*refreshLedgerMeasurements\s*\)\s*;/.exec(html) + : null; + +// The frame the deferred measurement is waiting in, lifted on its own — it is a declaration +// and not a function, and it is read by the shipped handler. +const measureFrameDecl = html.match(/^[ \t]*var ledgerMeasureFrame = [^;]+;/m); + // The two limits are declarations, not functions, so they are lifted on their own. They are // numbers the shipped code reads, and the guard below uses the shipped values rather than // numbers written here — a copy of the cap in this file would keep passing after the shipped @@ -195,7 +230,21 @@ function entriesOf(count, bands, prefix) { * time and a test that has to wait fifteen seconds to check it is a test nobody runs. */ function harness() { - const box = { clock: 1000, built: [], sized: [] }; + const box = { clock: 1000, built: [], sized: [], frames: [], listeners: {} }; + // The window the shipped code talks to. It records the handlers it is given and the frames + // it is asked for, so a case can deliver a real event and then run the frame the handler + // deferred its work to. + const win = { + addEventListener(type, handler) { + if (!box.listeners[type]) box.listeners[type] = []; + box.listeners[type].push(handler); + }, + requestAnimationFrame(fn) { + box.frames.push(fn); + return box.frames.length; + } + }; + box.window = win; const sandbox = { console, Math, @@ -209,6 +258,7 @@ function harness() { // The ledger the table is in. The shipped code reads it when it has to take the rows out // of a group the reader is no longer on. dom: { ledger: makeEl('div') }, + window: win, // The shipped el(tag, className), class and all: the class is what a section is found // by, and a list without one is a list the shipped code cannot find. el: (tag, className) => { @@ -230,11 +280,23 @@ function harness() { e.setAttribute('data-row-id', entry.record.id); return e; }, - sizeBuiltGroup: (section) => { box.sized.push(section); } + // The shipped sizeBuiltGroup() writes the group's own height on it, and that number is + // what the browser answers with while the group is off screen — so the fake DOM has to + // answer with it too, or the rule under test would be read against a DOM that does not + // behave like the browser's. + sizeBuiltGroup: (section) => { + box.sized.push(section); + section.style.containIntrinsicSize = + Math.round(section.getBoundingClientRect().height) + 'px'; + } }; vm.createContext(sandbox); + const registration = resizeRegistration ? resizeRegistration[0] + '\n' : ''; + const frameDecl = measureFrameDecl ? measureFrameDecl[0] + '\n' + : (refreshSrc ? 'var ledgerMeasureFrame = null;\n' : ''); vm.runInContext( - idleDecl[0] + '\n' + capDecl[0] + '\n' + batchDecl[0] + '\n' + fnSrc.join('\n') + '\n' + optionalSrc.join('\n'), + idleDecl[0] + '\n' + capDecl[0] + '\n' + batchDecl[0] + '\n' + frameDecl + + fnSrc.join('\n') + '\n' + optionalSrc.join('\n') + '\n' + registration, sandbox ); box.sandbox = sandbox; @@ -282,7 +344,9 @@ function buildHarness(options) { s.flushDeferredPoll = () => {}; s.watchLedgerGroups = () => {}; s.syncStickyOffsets = () => {}; - s.window = { requestAnimationFrame: () => {} }; + // The same window the harness made, so a case that delivers a resize event reaches the + // handler the client registered for it. + s.window = h.window; return h; } @@ -889,6 +953,172 @@ function measureGroup(h, key, entries, bands, height) { && first.querySelector('ul.row-list').children.length > 0); } +/* ---------- 3i. a live group refreshes its measurement after its layout changes ---------- */ + +/* + * A height is a measurement of a layout, and a layout can change without the content + * changing: the window is resized, the text scale is turned up, the group's own chrome + * becomes taller. The rows and the bands can be exactly the same and the height a different + * one. + * + * The remembered height was refreshed only when the content had changed, because the counts + * of rows and bands are what the difference was charged at. So a group whose layout changed + * and whose content did not kept the height it was measured at — and `heldLedgerHeight` + * handed that height back unchanged, because the counts it was measured at had not changed + * either. A group laid out at 1,500 px kept 1,000 px, and the document under the reader came + * up 500 px short of the group it stood for. + * + * The case is the reviewer's: eight groups of three rows, the second one laid out at + * 1,000 px and visited, then laid out at 1,500 px and visited again — the same three rows, + * the same number of bands — and then the reader walks through the rest of the window, so + * the cap takes the group away and the group has to keep the height it has now. + */ +{ + const h = buildHarness(); + const s = h.sandbox; + const n = 8; + const groups = []; + for (let i = 0; i < n; i += 1) { + groups.push({ turn: { index: i + 1 }, entries: entriesOf(3, 0, 'g' + i) }); + } + h.clock = 1000; + rebuild(h, groups); + // The reader is on the second group, and it is laid out for real at 1,000 px. + const second = sectionOf(h, 1); + let laidOut = 1000; + second.getBoundingClientRect = () => ({ height: laidOut, width: 900, top: 0, bottom: laidOut }); + s.fillLedgerGroup(second); + // The layout changes and nothing about the content does: the same three rows and the same + // number of bands, in a box that is 500 px taller. + laidOut = 1500; + s.fillLedgerGroup(second); + // The reader walks through the rest of the window, so the cap releases the group. + for (let i = 2; i < n; i += 1) { + h.clock += 10; + s.fillLedgerGroup(sectionOf(h, i)); + } + const released = sectionOf(h, 1); + // The rule, by name: the group was measured again, so the height it remembers is the one + // its layout has now. No numbers in the name — a control catches a rule by name. + assert('a live group refreshes its measurement after its layout changes', + s.ledgerState.get('k2').height === 1500); + // And the number the reader is owed: the height the released group keeps is the height it + // has now, and not the one it was measured at. + check('the height a released group keeps is the height its layout has now', + released.style.minHeight, '1500px'); +} + +/* ---------- 3j. a group released off screen keeps the height it was measured at ---------- */ + +/* + * The refresh above is the visitor's, and the visitor is on a group that is rendered: the + * height it has now is its layout and not the prediction an off-screen group is sized by. + * + * A group the reader has left is a different thing, and it is the calibration-priced path's + * business: it is off screen, the browser skips it, and what it reads is the prediction. The + * height it keeps is the one it was measured at, brought up to date with the rows it gained + * since — priced at the calibrated row height, because the difference between what it holds + * now and what it held when it was measured is a number of rows, and the prediction cannot + * know it. + * + * The case: the reader is on the second group and it is measured at 1,000 px with three rows + * in it; a late reply makes it four rows; the rebuild that brings the reply writes the group + * with the four rows in it and does not measure it; the reader walks away from it, so the + * browser skips it and it reads by its own contain-intrinsic-size; and the cap takes it on + * the visit path. The height it keeps has to be 1,036 px — the measured height and the row + * that was added — and not the prediction it is sized by off screen. + */ +{ + const h = buildHarness(); + const s = h.sandbox; + const n = s.LEDGER_LIVE_GROUPS + 2; + const groups = []; + for (let i = 0; i < n; i += 1) { + groups.push({ turn: { index: i + 1 }, entries: entriesOf(3, 0, 'g' + i) }); + } + h.clock = 1000; + rebuild(h, groups); + // The reader is on the second group, and it is laid out for real at 1,000 px. + const second = sectionOf(h, 1); + second.getBoundingClientRect = () => ({ height: 1000, width: 900, top: 0, bottom: 1000 }); + s.fillLedgerGroup(second); + // A late reply: the turn keeps its key and gains a row, and the rebuild that brings it + // writes the group with the four rows in it. The build never measures, so the height the + // group remembers is still the one of three rows. + const grown = []; + for (let i = 0; i < n; i += 1) { + grown.push({ turn: { index: i + 1 }, entries: entriesOf(4, 0, 'g' + i) }); + } + h.clock += 10; + rebuild(h, grown); + // The reader has walked away from it and the browser has skipped it: an off-screen group + // measures by its own contain-intrinsic-size, so what it reads now is the prediction. + const off = sectionOf(h, 1); + off.getBoundingClientRect = () => ({ height: 184, width: 900, top: 0, bottom: 184 }); + // The reader walks through the rest of the window — and not through that group, which is + // the one the reader has left — so the cap reaches the group the reader has left. + for (let i = 2; i < n; i += 1) { + h.clock += 10; + s.fillLedgerGroup(sectionOf(h, i)); + } + check('a group released off screen keeps the height it was measured at, brought up to date', + sectionOf(h, 1).style.minHeight, '1036px'); +} + +/* ---------- 3k. a live group refreshes its measurement when the window is resized ---------- */ + +/* + * The window is resized, and the layout of every group on screen changes with it: the rows + * and the bands stay exactly the same and the height is a different one. + * + * The visit path refreshes a group's measurement when the reader reaches it, and the reader + * does not reach the groups they are already looking at. So a group that was on screen before + * the resize kept the height it was measured at — a height of the layout before the resize — + * until the reader scrolled away from it and back. And when the cap released it, the height + * it reserved was that stale number, so the document under the reader came up short by + * exactly the difference. + * + * The case: eight groups of three rows, the second one laid out at 1,000 px and visited, the + * window resized so that it is laid out at 1,500 px, and the resize delivered to the client + * the way the browser delivers it — to the handler the client registered for it, with the + * measurement deferred to the next frame, because the layout a resize brought about is not + * there yet when the event arrives. The group is still rendered — the reader has not moved — + * so the height it has now is its layout and not the prediction. + * + * The boundary: a group that was OFF SCREEN when the layout changed, and is never visited + * again, keeps the height it was last really measured at. Off screen the browser answers with + * the group's own `contain-intrinsic-size`, and that number is a prediction — a prediction + * written down as a measurement is a guess the next rebuild reads back as a fact. This case + * claims nothing about that group; it is the same limit the build path has. + */ +{ + const h = buildHarness(); + const s = h.sandbox; + const n = 8; + const groups = []; + for (let i = 0; i < n; i += 1) { + groups.push({ turn: { index: i + 1 }, entries: entriesOf(3, 0, 'g' + i) }); + } + h.clock = 1000; + rebuild(h, groups); + // The reader is on the second group, and it is laid out for real at 1,000 px. + const second = sectionOf(h, 1); + let laidOut = 1000; + second.getBoundingClientRect = () => ({ height: laidOut, width: 900, top: 0, bottom: laidOut }); + s.fillLedgerGroup(second); + // The layout changes and nothing about the content does: the same three rows and the same + // number of bands, in a box that is 500 px taller. + laidOut = 1500; + // The window is resized. + for (const handler of (h.listeners.resize || [])) handler(); + const frames = h.frames.splice(0, h.frames.length); + for (const frame of frames) frame(); + // The rule, by name: the group was measured again, so the height it remembers is the one + // its layout has now. No numbers in the name — a control catches a rule by name. + assert('a live group refreshes its measurement when the window is resized', + s.ledgerState.get('k2').height === 1500); +} + /* ---------- 4. the rows come back when the reader reaches the group ---------- */ { From 4b45568fc8b4cfa34cc339458208c7980becb972 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 05:58:00 +0800 Subject: [PATCH 57/95] A rule in the search-segments guard could not fail, and now it can. The rule says a needle that is only in a record's ANSWER, and not in the text the page searches, finds nothing. The needle it used was taken from the first fixture record, and that record carried no `kind` -- so the needle was `undefined`, it folded to the empty string, and the scan answers an empty needle with nothing. The check therefore passed whatever the module did, which is the one thing a rule must never do: an assertion that cannot fail is a green light with no lamp behind it. The fixture record now carries a kind, so the needle is a real string that is on the line, in the answer part, and NOT in the searched text -- the kind is not one of the five searched fields. The check says those three things out loud -- the needle is in the answer, the needle is not in the text, the scan finds nothing -- so the last one cannot go back to being an empty assertion. This rule is older than the change in this branch: the fixture's missing field is in the base, so the rule has been green without proving anything since it was written. It is fixed here because the control that breaks the scan's per-line test breaks this rule too -- and a rule that cannot fail cannot be held by a control that claims to own it. --- .../tests/verify-search-segments.mjs | 25 ++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs index 5d39d635..a5b61df8 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-search-segments.mjs @@ -88,7 +88,11 @@ function pageRecordSearchText(record) { } const RECORDS = [ - { summary: 'the first answer', text: 'a longer Body of the reply', result: null, thinking: null, toolName: null }, + // The first record carries a KIND. The kind is part of a record's ANSWER and not one of the five + // searched fields, and the rule about a needle that is only in the answer needs one to point at — + // a needle taken from a record that has no kind is `undefined`, which folds to the empty string + // and answers nothing. That rule says the whole of it; this line is the other half. + { kind: 'assistant', summary: 'the first answer', text: 'a longer Body of the reply', result: null, thinking: null, toolName: null }, { summary: 'Reading a file', text: null, result: 'file contents here', thinking: null, toolName: 'read' }, { summary: 'Thinking it over', text: null, result: null, thinking: { text: 'weighing the options' }, toolName: null }, { summary: '', text: null, result: null, thinking: null, toolName: 'grep' }, @@ -292,10 +296,25 @@ assert('and there is no per-ordinal reader exported for a caller to reach for by // The JSON prefix holds the answer, so a needle must not be able to match it. Otherwise a search // for a record's kind would return every record of that kind whether its text says so or not. +// +// The needle has to be a REAL string, and it is one now. The first draft took it from a fixture +// record that carried no `kind`, so the needle was `undefined`, it folded to `''`, and `scanSegment` +// answers an empty needle with nothing — the rule then passed whatever the module did, which is the +// one thing a rule must never do. The fixture record carries a kind, so the needle is that kind: it +// is on the line, in the answer part, and it is NOT in the searchable text, because the kind is not +// one of the five searched fields. The first two thirds of the check say that out loud, so the last +// third cannot go back to being an empty assertion. const kindNeedle = RECORDS[0].kind; +const kindSurface = buildSegment({ + records: [{ kind: kindNeedle, summary: 'zzz nothing alike', text: null, result: null, thinking: null, toolName: null }], +}).surface; +const kindLine = kindSurface.split('\n')[0]; +const kindTab = kindLine.indexOf('\t'); +const kindAnswer = kindLine.slice(0, kindTab); +const kindText = kindLine.slice(kindTab + 1); check('a needle that is only in the answer, not in the text, finds nothing', - scanSegment(buildSegment({ records: [{ kind: kindNeedle, summary: 'zzz nothing alike', text: null, result: null, thinking: null, toolName: null }] }).surface, - foldNeedle(kindNeedle)), []); + [kindAnswer.includes(kindNeedle), kindText.includes(kindNeedle), scanSegment(kindSurface, foldNeedle(kindNeedle))], + [true, false, []]); // The direct statement of the line-number bug, as an assertion: for every record, that record's own // surface finds that record, AT THAT RECORD'S ORDINAL. A surface that kept the page's newline From cdfd81e52867f02b652c271eff58b06e30c1866a Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 05:58:00 +0800 Subject: [PATCH 58/95] The two search mutation controls now say which rule each case is about, and a case counts as caught only when the guard reported THAT name. Both controls used to take the first line of the guard's failure report that begins with `- ` and treat it as the name of the rule that broke. A case could therefore break one rule while the control printed another rule's name, and the control still counted the case as caught -- and its summary said "each by name". The other controls in the suite already carried the rule's name in each case and judged by it; these two were the ones that did not. Every case in both files now carries the name of the rule it breaks, and the runner collects the names the guard reported and asks whether the case's own name is among them. A case is caught only then; a case whose rule was not reported is reported as MISSED, with the names that were reported, and the control exits non-zero at the end. The proof that the old contract was worth nothing: in a copy of the package, a decoy failure line inserted into the guard's failure report made the old controls print the decoy as the owner of every case and still say "all 21 mutations caught, each by name" / "all 29 mutations caught, each by name". With the new contract, the same probe makes every case MISSED and the control exit non-zero. The decoy's own variant is reported honestly: with the decoy inserted and the real failures still printed, the mutation really did break the expected rule, so `caught` is the truthful answer -- what the old contract got wrong was the attribution, not a false pass. One case is added as well. The rule that a needle spanning two records matches nothing had no mutation of its own: the case that was supposed to own it joined whole segment lines, and every line carries a JSON prefix, so the join spliced the next record's prefix between the two texts and a needle crossing a record boundary still could not match. The new case joins the record TEXTS, which is the blob the boundary exists to prevent, and a needle that starts at the end of one record and finishes at the start of the next is found by it. --- .../tests/mutate-search-endpoint.py | 83 ++++++++++++- .../tests/mutate-search-segments.py | 117 ++++++++++++++++-- 2 files changed, 187 insertions(+), 13 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py index 80048cce..e2dee995 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-endpoint.py @@ -8,7 +8,13 @@ showing, and hands back an index no row occupies; - dropping one of the five searched fields narrows what a reader can find with no error at all; - counting only what fits under the cap turns "how many matched" into "how many did we send". + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the probe reported THAT rule: a probe that went red on some other assertion has not shown +that this guard owns this mutation. """ +import json +import re import subprocess import sys from pathlib import Path @@ -31,27 +37,32 @@ "the endpoint is wired to a scan of the stored rows rather than the projected view", " const searched = collectSearchHits(payload, needle, MAX_SEARCH_HITS);", " const searched = collectSearchHits({ turns: [{ index: 1, records: rows.map((row) => ({ index: 0, kind: 'reply', summary: String(row.message_id) })) }] }, needle, MAX_SEARCH_HITS);", + 'the endpoint answers through the one search helper', ), ( "a hit reports the record's stored id instead of the index the ledger gives it", " index: record.index,\n turnIndex: turn.index,", " index: record.id === null || record.id === undefined ? 0 : 1,\n turnIndex: turn.index,", + '"needle" finds the records the page found', ), ( "the endpoint projects the whole lineage instead of the slice that was asked for", " const view = selectIdentityView(identityTable, filter);\n const payload = buildTrajectoryPayload({\n session: { id: ctx.identity.id, createdAtMs: entry.lastActiveAt },", " const view = selectIdentityView(identityTable, null);\n const payload = buildTrajectoryPayload({\n session: { id: ctx.identity.id, createdAtMs: entry.lastActiveAt },", + 'both readers project the requested slice, not the whole lineage', ), # ---- the surface has to be the page's --------------------------------------------------- ( "the endpoint searches only the summary", " if (!searchSurfaceOf(record).includes(needle)) continue;", " if (!String(record.summary ?? '').includes(needle)) continue;", + '"read" finds the records the page found', ), ( "the endpoint folds the surface away, so a case-sensitive needle matches everything", " if (!searchSurfaceOf(record).includes(needle)) continue;", " if (!String(record.summary ?? '').toLowerCase().includes(needle) || true) continue;", + '"needle" finds the records the page found', ), # ---- how many --------------------------------------------------------------------------- ( @@ -63,43 +74,51 @@ if (!searchSurfaceOf(record).includes(needle)) continue; if (hits.length >= limit) break; total += 1;""", + 'and not what is counted', ), ( "the cap is applied before the count, so an unlimited search reports nothing", " total += 1;\n if (hits.length >= limit) continue;", " if (hits.length < limit) total += 1;\n if (hits.length >= limit) continue;", + 'and not what is counted', ), # ---- a needle never crosses a record ------------------------------------------------------ ( "records are joined into one blob before the scan, so a needle can cross a boundary", " for (const turn of turns) {\n const records = Array.isArray(turn.records) ? turn.records : [];\n for (const record of records) {\n if (!searchSurfaceOf(record).includes(needle)) continue;\n total += 1;", " const blob = turns.flatMap((t) => (Array.isArray(t.records) ? t.records : [])).map((r) => searchSurfaceOf(r)).join(' ');\n if (blob.includes(needle)) { hits.push({ index: 0, turnIndex: 1, kind: 'reply', summary: '', generation: null }); total += 1; }\n for (const turn of []) {\n const records = [];\n for (const record of records) {\n if (!searchSurfaceOf(record).includes(needle)) continue;\n total += 1;", + 'a needle spanning two records finds nothing', ), # ---- the route --------------------------------------------------------------------------- ( "the route is not registered, so a search answers 404", " if (url.pathname === '/api/trajectory/search') {", " if (url.pathname === '/api/trajectory/searchX') {", + 'the route exists and is not the trajectory route', ), ( "a needle too short is answered with everything instead of refused", " if (!needleIsUsable(asked)) {", " if (false) {", + 'a needle too short is refused with a status, not answered with everything', ), ( "the lineage is walked a second time, so the two readers can disagree about it", " walked = await collectLineage(entry, ctx.generations);\n } catch (error) {\n sendError(response, 500, 'trajectory_unreadable', lineageFailure(error));\n return;\n }\n const { occurrences, files } = walked;", " walked = await collectLineage(entry, ctx.generations);\n walked = await collectLineage(entry, ctx.generations);\n } catch (error) {\n sendError(response, 500, 'trajectory_unreadable', lineageFailure(error));\n return;\n }\n const { occurrences, files } = walked;", + 'the lineage is walked in one place, and both readers use it', ), ( "the session key becomes the session id, so an id becomes a directory name", " createHash('sha256').update(`${dir}\\n${sessionId}`).digest('hex').slice(0, 32);", " String(sessionId);", + 'the session key is a hash, so no session id becomes a directory name', ), ( "the artifacts are written beside the session history instead of under the data dir", " const artifactRoot = join(context.dataDir, 'trajectory-artifacts');", " const artifactRoot = join(process.cwd(), 'trajectory-artifacts');", + 'the artifacts are written under the runtime data dir', ), ( # Used to be the other way round: §8.8.1 rule 5 kept the active generation out of the store, @@ -109,6 +128,7 @@ "the active generation is skipped again, so a session still being written has nothing to scan", " for (let fileIdx = 0; fileIdx < files.length; fileIdx += 1) {\n const records = [];", " for (let fileIdx = 0; fileIdx < files.length; fileIdx += 1) {\n if (fileIdx === 0) continue;\n const records = [];", + 'every generation ends up with a segment: the only outer early exit is "already on disk", and it comes after the read', ), ( # The rule is that `total` counts every match, so a missing generation cannot be tolerated: @@ -116,6 +136,7 @@ "one missing segment no longer makes the search fall back", " if (segment === null) return null;", " if (segment === null) continue;", + 'one missing segment makes the whole search fall back rather than answer from the rest', ), ( # The rule is that `total` counts every match, so the cap cannot bound it: the answer @@ -124,6 +145,7 @@ "the artifact path reports how many it sent instead of how many matched", " return { hits: capped, total, fingerprint: recordSetFingerprint(versions) };", " return { hits: capped, total: capped.length, fingerprint: recordSetFingerprint(versions) };", + 'and the shipped artifact path counts every match rather than what it sent', ), ( # The two readings of "the versions of the first N records" have to agree, and the limit @@ -133,6 +155,7 @@ "the limit on the fold over the first N records is read as something else", " const wanted = Number.isInteger(limit) && limit >= 0 ? limit : Infinity;", " const wanted = Number.isInteger(limit) && limit > 0 ? limit : Infinity;", + 'and a limit of zero is zero records', ), ( # The summary a hit answers with now lives on the segment line, so the mutation that swaps it @@ -140,6 +163,7 @@ "writing the artifacts can fail the search that asked for them", " }).catch(() => undefined);", " });", + 'a failed artifact write cannot fail the search that asked for it', ), # ---- the scan is resolved in one batch --------------------------------------------------- ( @@ -164,6 +188,7 @@ " }\n" " return null;\n" " });", + 'the shipped artifact path resolves a whole scan in one batch call', ), ( # A hit that cannot be resolved is a segment written by something this reader disagrees @@ -171,6 +196,7 @@ "a hit the reader could not resolve is skipped instead of falling back", " if (hit === null) return null;", " if (hit === null) continue;", + 'a hit the scan named but the reader could not resolve falls back rather than being skipped', ), ( # The shape that was removed: a second store under the artifact root, filled on every @@ -181,6 +207,7 @@ "a second artifact store is written beside the segments, and nothing reads it", " const segment = buildSegment({ records });\n await segmentStore.write(sessionKey, viewKey, generationKeyValue, segment, {\n generation,\n uncovered,\n source: [files[fileIdx]],\n });", " const segment = buildSegment({ records });\n await segmentStore.write(sessionKey, viewKey, generationKeyValue, segment, {\n generation,\n uncovered,\n source: [files[fileIdx]],\n });\n const indexStore = createSegmentStore(join(artifactRoot, 'index'));\n await indexStore.write(sessionKey, viewKey, generationKeyValue, segment, {\n generation,\n uncovered,\n source: [files[fileIdx]],\n });", + 'and the artifact root holds segments and nothing else', ), ] @@ -210,6 +237,45 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + Two formats, as in `mutate-trajectory-cache.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names + + def main(): code, output = run_probe() if code != 0: @@ -221,7 +287,7 @@ def main(): original = SERVER.read_bytes() missed = [] unnamed = [] - for label, old, new in MUTATIONS: + for label, old, new, expect in MUTATIONS: text, newline = read_exact(SERVER) if old not in text: print(f"SKIP {label}: anchor not found") @@ -240,12 +306,19 @@ def main(): print(f"MISSED {label}: the probe still passed") missed.append(label) continue - first = next((line.strip() for line in output.splitlines() if line.strip().startswith("- ")), None) - if first is None: + names = reported_rules(output) + if not names: print(f"UNNAMED {label}: the probe failed without naming a rule") unnamed.append(label) - else: - print(f"caught {label}\n -> {first}") + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") if SERVER.read_bytes() != original: print(f"NOT RESTORED {SERVER}") diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py index 3b12c310..abec0631 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-segments.py @@ -7,7 +7,13 @@ The first six are the ones that matter: together they are the ways a segment can quietly stop being "the text the page was searching". A dropped field, a reordered field, a different separator, no folding and no whitespace rule each narrow what a reader can find, and none of them throws. + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the probe reported THAT rule: a probe that went red on some other assertion has not shown +that this guard owns this mutation. """ +import json +import re import subprocess import sys from pathlib import Path @@ -25,77 +31,114 @@ "the thinking field is dropped from the search surface", FIELDS, "export const SEARCH_FIELDS = ['summary', 'text', 'result', 'toolName'];", + 'a word in a thinking block is found', ), ( "the fields are read in a different order", FIELDS, "export const SEARCH_FIELDS = ['summary', 'result', 'text', 'thinking', 'toolName'];", + "the field list is the page's, in the page's order", ), ( "the fields are joined with nothing instead of a separator", " return normaliseWhitespace(parts.join('\\n')).toLowerCase();", " return normaliseWhitespace(parts.join('')).toLowerCase();", + 'and a needle spanning two fields of one record is found, which is the point of the space', ), ( "whitespace is not normalised, so the surface is five lines per record again", " return String(value).replace(/\\s+/g, ' ').trim();", " return String(value);", + 'a record with five fields is still one line', ), ( "the surface is not folded, so an upper-case needle finds nothing", " return normaliseWhitespace(parts.join('\\n')).toLowerCase();", " return normaliseWhitespace(parts.join('\\n'));", + 'the surface is folded once, at build time', ), ( "the needle is folded but not normalised, so what the reader typed decides the answer", " return normaliseWhitespace(typeof needle === 'string' ? needle : '').toLowerCase();", " return (typeof needle === 'string' ? needle : '').toLowerCase();", + 'a needle with odd whitespace finds what the tidy one finds', ), # ---- the boundary, and the ordinal it is only correct because of ---------------------- ( "the scan joins every line into one blob, so a needle can cross a record boundary", " const hits = [];\n let start = 0;\n let ordinal = 0;", " const hits = [];\n const flat = surface.replace(/\\n/g, ' ');\n if (flat.includes(needle)) hits.push(0);\n let start = 0;\n let ordinal = 0;", + 'and does not match either side', + ), + ( + # The hazard the case above cannot reach, and the one the rule about the boundary is really + # about. Every line carries a JSON prefix, so joining whole LINES splices the next record's + # answer between the two texts — and a needle crossing a record boundary still cannot match, + # which is why the case above is caught by another rule and not by that one. + # + # Joining the TEXTS — the searchable parts, with the prefix stripped — is the blob the + # boundary exists to prevent, and a needle that starts at the end of one record and finishes + # at the start of the next IS found by it. So this is the case that owns the rule, and it + # owns the structural rule about the scan not being a joined blob as well: the per-line test + # is gone from the module entirely. + "the scan joins the record texts, so a needle can cross a record boundary", + " if (tab >= 0 && line.slice(tab + 1).includes(needle)) hits.push(ordinal);", + " if (tab >= 0 && surface.split('\\n').map((l) => l.slice(l.indexOf('\\t') + 1)).join(' ').includes(needle)) hits.push(ordinal);", + 'a needle spanning two records matches nothing', ), ( # Was about empty lines in a plain surface, which the new format cannot produce: every # record's line starts with its answer, so an empty line means a record that is not there. # The hazard that replaced it is the prefix itself — a scan that tested the whole line would # answer from a record's KIND and SUMMARY rather than from the text being searched. + # + # The rule it is about is the one that says a needle which is only in the answer is not + # found. It could not be named before: the rule's own needle was `undefined` (the fixture + # record carried no kind), so it folded to the empty string and the rule passed whatever + # the module did. The fixture carries a kind now, so this case really breaks that rule. + # The structural rule about the scan not being a joined blob breaks with it — the module no + # longer holds the per-line test either — and the case above owns that one. "the scan tests the record's answer as well as its text", " if (tab >= 0 && line.slice(tab + 1).includes(needle)) hits.push(ordinal);", " if (tab >= 0 && line.includes(needle)) hits.push(ordinal);", + 'a needle that is only in the answer, not in the text, finds nothing', ), ( "the locators are numbered from one while the scan numbers from zero", " ordinal: i,", " ordinal: i + 1,", + 'the ordinal is the record order', ), ( "a locator of a record with no position claims file 0", " fileIdx: typeof record.fileIdx === 'number' ? record.fileIdx : -1,", " fileIdx: typeof record.fileIdx === 'number' ? record.fileIdx : 0,", + 'a record with no file number says -1, not 0', ), # ---- the artifact --------------------------------------------------------------------- ( "the manifest is published before the surface it describes", " await rename(`${join(dir, 'surface.txt')}.tmp`, join(dir, 'surface.txt'));", " await rename(`${join(dir, 'manifest.json')}.tmp`, join(dir, 'manifest.json'));", + 'the manifest is published last', ), ( "a manifest from another schema version is served anyway", " if (manifest.schemaVersion !== SEGMENT_SCHEMA_VERSION) return null;", "", + 'a manifest from another schema version is refused', ), ( "a surface with a line the manifest does not count is accepted", " if (lines !== manifest.recordCount) return null;", "", + 'a surface the manifest does not count is refused', ), ( "the runtime's own revision stops being compared, so a rewrite that kept the length is reused", " if (typeof a.revision === 'string' && a.revision.length > 0\n && typeof b.revision === 'string' && b.revision.length > 0 && a.revision !== b.revision) {\n return false;\n }\n", "", + 'a source whose revision changed refuses the segment', ), ( # The one that was still wrong after the revision comparison was fixed: the ACTIVE file has @@ -107,6 +150,7 @@ "the mtime stops being compared, so a rewrite that kept the length is reused", " if (a.mtimeMs !== b.mtimeMs) return false;\n", "", + 'a source whose mtime changed refuses the segment, so a rewrite that kept the length is not reused', ), ( # The versions are read out of the lines by hand, so that a search over a big segment does @@ -116,6 +160,7 @@ "the versions of a segment's records are read as something else", " if (tab >= 0 && tab <= end) out.push(versionInAnswer(surface.slice(start, tab)));", " if (tab >= 0 && tab <= end) out.push(String(out.length));", + 'the versions of a segment can be read back in order', ), ( # The ledger position of a record is what puts the records back where the body has them: @@ -127,6 +172,7 @@ "the record's own number is read as the line's ordinal, so the records cannot be put back in order", " index: numberInAnswer(answer, '\"index\":'),", " index: numberInAnswer(answer, '\"ordinal\":'),", + 'a segment reads back the turn and the number of each of its records', ), ( # The turn a record is in is the other half of the position. Read as the record's own @@ -134,6 +180,7 @@ "the turn a record is in is read as its own number, so the ledger's order cannot be rebuilt", " turnIndex: numberInAnswer(answer, '\"turnIndex\":'),", " turnIndex: numberInAnswer(answer, '\"index\":'),", + 'and the record that arrived for the earlier turn carries that turn', ), ( # The manifest has to say how many of the body's records the segments do NOT cover. A @@ -143,6 +190,7 @@ "a manifest that does not say what it does not cover is served anyway", " if (!Number.isInteger(manifest.uncovered) || manifest.uncovered < 0) return null;\n", "", + 'a manifest that does not say what it does not cover is refused', ), ( # The count itself: a record whose locator names no file in the table — and a record with no @@ -152,22 +200,26 @@ "the records no segment covers are counted as covered", " if (typeof row.fileIdx !== 'number' || !Number.isFinite(row.fileIdx) || row.fileIdx < 0) {\n uncovered += 1;\n }", " if (false) {\n uncovered += 1;\n }", + 'and the records no segment covers are counted', ), # ---- the rest of the contract --------------------------------------------------------- ( "the needle floor is dropped, so one character is a search", " return foldNeedle(needle).trim().length >= MIN_NEEDLE_CHARS;", " return foldNeedle(needle).length >= 0;", + 'a one-character needle is refused', ), ( "the hit cap is dropped, so a two-character needle returns the whole ledger", " return { ordinals: hits.slice(0, limit), total: hits.length, truncated: true };", " return { ordinals: hits, total: hits.length, truncated: true };", + 'hits are capped', ), ( "the active generation's segment is stored under a sealed generation's key", " return generation === null || generation === undefined ? 'active' : `g${generation}`;", " return generation === null || generation === undefined ? 'g0' : `g${generation}`;", + 'the active generation has a key of its own', ), # ---- a generation read short must not look like one read whole -------------------------- ( @@ -176,6 +228,7 @@ "the segment line reports the kind where the summary belongs", " summary: typeof record.summary === 'string' ? record.summary : '',", " summary: typeof record.kind === 'string' ? record.kind : '',", + "the summary it reads back is the record's own", ), ( # `size` is the whole file whether or not the read covered it, so this comparison is the @@ -185,6 +238,7 @@ "a source that was read short is trusted as if it had been read whole", " if ((a.truncated === true) !== (b.truncated === true)) return false;\n if (typeof a.revision === 'string' && a.revision.length > 0", " if (typeof a.revision === 'string' && a.revision.length > 0", + 'a source read short is not trusted as if it had been read whole', ), # ---- the answer is read back in ONE walk ----------------------------------------------- ( @@ -210,6 +264,7 @@ " }\n" " return found.get(ordinal) ?? null;\n" " });", + 'reading 200 answers out of 200 lines takes one walk (counted 21100 per-line string calls)', ), ( # The batch is a batch because the scan hands back every ordinal at once. Answering in the @@ -218,6 +273,7 @@ "the answers come back in the order they sit rather than the order asked", " return ordinals.map((ordinal) => found.get(ordinal) ?? null);", " return [...found.values()];", + 'the batch reader answers in the order asked, not the order they sit', ), ( # Structural, and the reason the first one is worth a mutation at all: a per-ordinal reader @@ -228,6 +284,7 @@ "export function generationKey(generation) {\n return generation === null || generation === undefined ? 'active' : `g${generation}`;\n}\n\n" "/** @deprecated one ordinal at a time — walks the whole segment to find it. */\n" "export function readSegmentHit(surface, ordinal) {\n return readSegmentHits(surface, [ordinal])[0] ?? null;\n}", + 'and there is no per-ordinal reader exported for a caller to reach for by mistake', ), # ---- a retired field must not come back through this door --------------------------- ( @@ -244,6 +301,7 @@ " byteStart: typeof record.byteStart === 'number' ? record.byteStart : -1,\n" " byteEnd: typeof record.byteEnd === 'number' ? record.byteEnd : -1,\n" " }) + '\\t' + searchSurfaceOf(record));", + 'no line carries a byte range any more', ), ] @@ -273,6 +331,45 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + Two formats, as in `mutate-trajectory-cache.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names + + def main(): code, output = run_probe() if code != 0: @@ -284,7 +381,7 @@ def main(): original = MODULE.read_bytes() missed = [] unnamed = [] - for label, old, new in MUTATIONS: + for label, old, new, expect in MUTATIONS: text, newline = read_exact(MODULE) if old not in text: print(f"SKIP {label}: anchor not found") @@ -303,15 +400,19 @@ def main(): print(f"MISSED {label}: the probe still passed") missed.append(label) continue - first = next( - (line.strip() for line in output.splitlines() if line.strip().startswith("- ")), - None, - ) - if first is None: + names = reported_rules(output) + if not names: print(f"UNNAMED {label}: the probe failed without naming a rule") unnamed.append(label) - else: - print(f"caught {label}\n -> {first}") + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") if MODULE.read_bytes() != original: print(f"NOT RESTORED {MODULE}") From a0c83b0cc89f72504ae03728f5230534b792bd64 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 05:58:01 +0800 Subject: [PATCH 59/95] A mutation control has to decide a case by the NAME of the rule that case broke, and nothing was there to stop that contract from being undone. The contract in two sentences. A mutation control that gives a case its own expected rule name must decide that case by MEMBERSHIP -- testing the case's own name against the collection of names the guard reported -- and must never decide it by taking ONE reported name and treating it as the rule's name. A control that carries no per-case expected rule name cannot say which rule fired, so it is reported as not using the name contract rather than failed: the contract is about how a case is judged, and a control that never claims ownership of a rule has nothing to judge by. Why it is a rule and not a style. The first `- ` line a guard prints is whatever that guard happened to fail on first. A control that takes that line and calls it "the rule" reports a case as caught whenever the guard went red for ANY reason -- a second assertion in the same file, a crash in an unrelated check, a stale expectation -- and the number it prints is then a number that means nothing. The reviewer found exactly this in two controls; the fix was to give every case its own expected rule name and to test that name by membership. What was missing is the thing that keeps the fix in place. What is added is a guard that reads the controls' own SOURCE, statically, rather than running them: running every control would rewrite shipped files, and this guard has to be able to run while a suite is in flight. The coverage set is DERIVED, never written out -- every tests/mutate-*.py in the package's tests directory -- so a control added tomorrow is covered without anybody editing this file. The two files the reviewer found are pinned by an ANCHOR, and that is a different thing from the coverage set: the anchor says those two must carry the contract, so relaxing either one -- including dropping the expected-name field, which takes it out of the set of controls that carry it -- is reported by name. The guard cannot be silenced either. It reads the set from its own directory, so a copy of it placed where no controls sit derives an empty set and FAILS, and a copy that sits beside one control fails as well. A guard that can report nothing and call it clean is not a guard. The four rules it states are these: 'a mutation control decides a case by membership, not by one reported name', 'the two reviewed mutation controls decide their cases by membership', 'the set of mutation controls is derived from the package and is not empty', and 'every mutation control in the package parses'. The guard also owes the reader a classification, and it prints one on every run: of the 52 mutation controls in the package, 39 decide their cases by membership, and 13 do not. Those 13 carry no per-case expected rule name, so they are reported rather than failed -- they have no ownership to judge by, and calling them failures would be a number that means nothing either. Giving each of those 13 a rule name is not done here; it is a separate piece of work, and the guard names them so the next reader does not have to rediscover them. The control that owns these rules has five cases, each caught by name. Three of them relax the decision inside the anchored controls -- 'the decision compares the case's expected name with the first name the guard reported', 'the decision drops the case's expected name and takes the first name the guard reported', and 'the case table drops the expected-name field'. The other two run the guard in a directory holding fewer controls -- 'the set of controls the guard derives is empty' and 'the set of controls the guard derives has been shrunk to one' -- which is what proves the guard cannot be silenced by making its own set smaller. --- .../tests/mutate-mutation-contracts.py | 291 +++++++++ .../tests/verify-mutation-contracts.py | 595 ++++++++++++++++++ 2 files changed, 886 insertions(+) create mode 100644 plugins/avatasia/mmc-trajectory/tests/mutate-mutation-contracts.py create mode 100644 plugins/avatasia/mmc-trajectory/tests/verify-mutation-contracts.py diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-mutation-contracts.py b/plugins/avatasia/mmc-trajectory/tests/mutate-mutation-contracts.py new file mode 100644 index 00000000..bafde2e6 --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-mutation-contracts.py @@ -0,0 +1,291 @@ +"""The control that proves the name contract is enforced, and that it cannot be silenced. + +`verify-mutation-contracts.py` is a guard about the OTHER mutation controls: it reads each of +them and says whether a case is judged by the NAME of the rule that case broke, or by one +reported name standing in for it. A guard that cannot fail is not a guard, so this file breaks +that guard five ways and requires it to fail by name every time: + + 1. the decision compares the case's own expected name with the FIRST name the guard reported; + 2. the decision drops the case's expected name and takes the first name the guard reported; + 3. the case table drops the expected-name field altogether; + 4. the set of controls the guard derives is empty; + 5. the set of controls the guard derives has been shrunk to one. + +Cases 1 to 3 are the relaxations of a real control -- `mutate-search-endpoint.py`, one of the two +the reviewer found -- and they are undone byte for byte. Cases 4 and 5 are the silencing: they +run a COPY of the guard from a directory that holds fewer controls than the package does, which +is what a guard that reads a hand-written list, or one whose list has drifted, looks like from +the inside. A guard that reported an empty set as clean would pass both of them, and the two +cases are here so that it cannot. + +Every case is caught BY NAME. The guard's failure lines are collected and the case's expected +rule name has to be among them: a guard that went red for some other reason has not shown that +it owns this mutation, which is the same rule this file is about. +""" + +import ast +import hashlib +import shutil +import subprocess +import sys +import tempfile +from pathlib import Path + +HERE = Path(__file__).resolve().parent +GUARD = HERE / "verify-mutation-contracts.py" +TARGET = HERE / "mutate-search-endpoint.py" + +# The rules the guard reports, spelled the way it spells them. +RULE_MEMBERSHIP = "a mutation control decides a case by membership, not by one reported name" +RULE_ANCHOR = "the two reviewed mutation controls decide their cases by membership" +RULE_DERIVED = "the set of mutation controls is derived from the package and is not empty" + +# The decision that is correct, and the two that are not. The case table is not touched by the +# first two: every case keeps its expected rule name, and only the decision changes. +DECISION_FROM = r""" # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") +""" + +DECISION_SINGLE = r""" # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + first = names[0] + if first != expect: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {first}") +""" + +DECISION_FIRST = r""" # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + first = names[0] + print(f"caught {label}\n -> {first}") +""" + + +def read_exact(path): + """Normalise to LF for matching, and report the file's own newline so it can be put back.""" + data = path.read_bytes() + text = data.decode("utf-8").replace("\r\n", "\n") + return text, ("\r\n" if b"\r\n" in data else "\n") + + +def write_exact(path, text, newline): + if newline == "\r\n": + text = text.replace("\n", "\r\n") + path.write_bytes(text.encode("utf-8")) + + +def digest(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def reported_rules(output): + """The rules the guard named. + + Two formats, as in `mutate-search-endpoint.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(text[2:].strip()) + elif text.startswith("FAIL "): + names.append(text[5:].strip()) + return names + + +def run_guard(script=None, cwd=None): + """Run the guard, or a copy of it, and report what it said.""" + command = [sys.executable, str(script if script is not None else GUARD)] + proc = subprocess.run( + command, + cwd=str(cwd if cwd is not None else HERE), + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + ) + return proc.returncode, (proc.stdout or "") + (proc.stderr or "") + + +# -------------------------------------------------------------------------------------- +# the three relaxations of a real control +# -------------------------------------------------------------------------------------- + +def relax_decision(text, replacement): + """Replace the decision that judges a case, and nothing else.""" + if DECISION_FROM not in text: + raise SystemExit("the decision this control relaxes is not in the file any more") + return text.replace(DECISION_FROM, replacement, 1) + + +def drop_expected_field(text): + """Delete the expected-name field from every case, and the decision that read it. + + The field is the fourth element of every tuple in the case table, and it is found through + the tree rather than through a pattern: the line each one sits on is read off the parse, so + a case whose text happens to look like another case's text cannot be mistaken for it. + """ + tree = ast.parse(text) + table = None + for node in ast.walk(tree): + if not isinstance(node, ast.Assign): + continue + for target in node.targets: + if isinstance(target, ast.Name) and target.id == "MUTATIONS": + table = node.value + if table is None: + raise SystemExit("the case table this control edits is not in the file any more") + + drop = set() + for element in table.elts: + if len(element.elts) < 4: + raise SystemExit("a case no longer carries an expected rule name") + field = element.elts[3] + if field.end_lineno != field.lineno: + raise SystemExit("an expected rule name now spans more than one line") + drop.add(field.lineno) + + kept = [line for number, line in enumerate(text.split("\n"), 1) if number not in drop] + out = "\n".join(kept) + out = out.replace("for label, old, new, expect in MUTATIONS:", + "for label, old, new in MUTATIONS:") + out = relax_decision(out, DECISION_FIRST) + # The name has to be gone as a NAME, not as a word: the file's own prose says "expect". + for node in ast.walk(ast.parse(out)): + if isinstance(node, ast.Name) and node.id == "expect": + raise SystemExit("the expected rule name is still read after the field was dropped") + return out + + +RELAXATIONS = [ + ("the decision compares the case's expected name with the first name the guard reported", + RULE_MEMBERSHIP, + lambda text: relax_decision(text, DECISION_SINGLE)), + ("the decision drops the case's expected name and takes the first name the guard reported", + RULE_ANCHOR, + lambda text: relax_decision(text, DECISION_FIRST)), + ("the case table drops the expected-name field", + RULE_ANCHOR, + drop_expected_field), +] + + +# -------------------------------------------------------------------------------------- +# the two silences +# -------------------------------------------------------------------------------------- + +def run_in_a_shorter_tree(label, expect, keep): + """Run a COPY of the guard from a directory that holds fewer controls than the package. + + The copy is the guard's own source, unchanged. What differs is the directory it sits in, and + that is the point: the set of controls it derives comes from where it stands, so a guard + that read a list, or one whose list had drifted, would find fewer controls here and report + on fewer. This case requires it to fail instead. + """ + workspace = tempfile.mkdtemp(prefix="mutation-contracts-") + try: + shutil.copy(GUARD, Path(workspace) / GUARD.name) + for name in keep: + shutil.copy(HERE / name, Path(workspace) / name) + code, output = run_guard(Path(workspace) / GUARD.name, cwd=workspace) + names = reported_rules(output) + if code == 0: + print(f"MISSED {label}: the guard passed with {len(keep)} control(s) in reach") + return False + if expect not in names: + print(f"MISSED {label}: the guard named {names} instead of {expect}") + return False + print(f"caught {label}\n -> {expect}") + return True + finally: + shutil.rmtree(workspace, ignore_errors=True) + + +def main(): + original_bytes = TARGET.read_bytes() + original_digest = digest(TARGET) + missed = [] + unnamed = [] + + # The guard as it stands has to pass, or the cases below prove nothing. + code, output = run_guard() + if code != 0: + print("the guard does not pass on the package as it stands; fix that first") + print(output) + return 1 + print("baseline clean") + + text, newline = read_exact(TARGET) + for label, expect, relax in RELAXATIONS: + relaxed = relax(text) + if relaxed == text: + print(f"SKIP {label}: the relaxation changes nothing") + missed.append(label) + continue + write_exact(TARGET, relaxed, newline) + try: + code, output = run_guard() + finally: + TARGET.write_bytes(original_bytes) + if code == 0: + print(f"MISSED {label}: the guard still passed") + missed.append(label) + continue + names = reported_rules(output) + if not names: + print(f"UNNAMED {label}: the guard failed without naming a rule") + unnamed.append(label) + continue + if expect not in names: + print(f"MISSED {label}: the guard named {names} instead of {expect}") + missed.append(label) + continue + if TARGET.name not in output: + print(f"MISSED {label}: the guard did not name {TARGET.name}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") + + # The guard cannot be silenced by the set it derives shrinking to nothing. + if not run_in_a_shorter_tree( + "the set of controls the guard derives is empty", + RULE_DERIVED, []): + missed.append("the set of controls the guard derives is empty") + + # Nor by the set shrinking to one control that happens to be correct. + if not run_in_a_shorter_tree( + "the set of controls the guard derives has been shrunk to one", + RULE_ANCHOR, ["mutate-search-endpoint.py"]): + missed.append("the set of controls the guard derives has been shrunk to one") + + if TARGET.read_bytes() != original_bytes: + print(f"NOT RESTORED {TARGET}") + return 1 + if digest(TARGET) != original_digest: + print(f"NOT RESTORED {TARGET}: the bytes are the same file but not the same hash") + return 1 + print("file restored byte-for-byte") + + if missed or unnamed: + if missed: + print(f"\n{len(missed)} mutation(s) got through") + if unnamed: + print(f"{len(unnamed)} mutation(s) failed without naming a rule") + return 1 + print(f"\nall {len(RELAXATIONS) + 2} relaxations caught, each by name") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-mutation-contracts.py b/plugins/avatasia/mmc-trajectory/tests/verify-mutation-contracts.py new file mode 100644 index 00000000..24dcfebc --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/verify-mutation-contracts.py @@ -0,0 +1,595 @@ +"""A mutation control must judge a case by the NAME of the rule that case broke. + +The rule this pins, in two sentences: + + A mutation control that gives a case its own expected rule name must decide that case by + MEMBERSHIP -- testing the case's own name against the collection of names the guard reported -- + and must never decide it by taking ONE reported name and treating it as the rule's name. + + A control that carries no per-case expected rule name cannot say which rule fired, so it is + reported as not using the name contract rather than failed: the contract is about how a case is + judged, and a control that never claims ownership of a rule has nothing to judge by. + +Why this is a rule and not a style: the first `- ` line a guard prints is whatever that guard +happened to fail on first. A control that takes that line and calls it "the rule" reports a case +as caught whenever the guard went red for ANY reason -- a second assertion in the same file, a +crash in an unrelated check, a stale expectation. Such a control still passes, and the number it +prints is a number that means nothing. The reviewer found exactly this in two controls; the fix +was to give every case its own expected rule name and to test that name by membership. This guard +is what stops the fix from being undone by accident, and it is a static reading of the control's +own source rather than a run of it: running every control would rewrite shipped files, and this +guard has to be able to run while a suite is in flight. + +The coverage set is DERIVED, never written out: every tests/mutate-*.py in this directory is +covered, so a control added tomorrow is covered without anybody editing this file. The two files +the reviewer found are pinned by an ANCHOR below, which is a different thing: the anchor says +those two must carry the contract, so that relaxing either one -- including dropping the +expected-name field, which takes it out of the set of controls that carry it -- is reported by +name. + +The set is read from this file's own directory, so a copy of this file placed somewhere with no +controls under it derives an empty set and FAILS. A guard that cannot be silenced is one that +cannot report nothing and call it clean. +""" + +import ast +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent + +# The coverage set is derived from the package: every mutation control that ships in this +# directory. Written out as a pattern, never as a list of names. +COVERAGE_PATTERN = "mutate-*.py" + +# The anchor. These two are the controls the reviewer found deciding a case by one reported +# name. They are named here on purpose and are NOT the coverage set: the set above is derived, +# and this is the one place a file name is written down. Both must carry the name contract, so +# relaxing either one -- or dropping the expected-name field, which removes it from the set of +# controls that carry it -- is reported by name. +ANCHOR = ("mutate-search-endpoint.py", "mutate-search-segments.py") + +# The rules, named once, so the output a failure carries is the same string every time. +RULE_MEMBERSHIP = "a mutation control decides a case by membership, not by one reported name" +RULE_ANCHOR = "the two reviewed mutation controls decide their cases by membership" +RULE_DERIVED = "the set of mutation controls is derived from the package and is not empty" +RULE_PARSE = "every mutation control in the package parses" + +# How a control decides. +BY_MEMBERSHIP = "membership" +BY_SINGLE_NAME = "single-name" +BY_NO_NAME = "no-name" + +# A slot holds the source text the control writes into a shipped file. Those are not rule names, +# and a control that tests one of them against the file it is editing is doing anchor work, not +# judging a case. +SOURCE_METHODS = ("replace", "sub", "count") + + +# -------------------------------------------------------------------------------------- +# reading a control +# -------------------------------------------------------------------------------------- + +def names_in(node): + """Every name an expression mentions.""" + return {n.id for n in ast.walk(node) if isinstance(n, ast.Name)} + + +def show(node): + try: + return ast.unparse(node) + except Exception: # pragma: no cover - only on an interpreter too old to unparse + return "" + + +def has_call(node): + """Whether an expression calls anything. A call is work, not a value.""" + return any(isinstance(n, ast.Call) for n in ast.walk(node)) + + +def case_tables(tree): + """The names that hold a list of cases: a module- or function-level list of tuples.""" + tables = {} + for node in ast.walk(tree): + if not isinstance(node, ast.Assign): + continue + value = node.value + if not isinstance(value, ast.List) or not value.elts: + continue + if not all(isinstance(e, ast.Tuple) and e.elts for e in value.elts): + continue + for target in node.targets: + if isinstance(target, ast.Name): + tables.setdefault(target.id, value.elts) + return tables + + +def function_params(tree): + """The parameter names of every function defined in this file.""" + params = {} + for node in ast.walk(tree): + if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + params[node.name] = [a.arg for a in node.args.args] + return params + + +def source_variables(tree): + """The variables that hold the source text a control writes into a shipped file. + + The signal is the call that rewrites the file, and the comparison a control makes between + two of its own anchors. Both are independent of how a case is judged, which is what makes + them usable here. + """ + source = set() + changed = True + while changed: + changed = False + for node in ast.walk(tree): + if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) \ + and node.func.attr in SOURCE_METHODS: + for arg in node.args: + for name in names_in(arg): + if name not in source: + source.add(name) + changed = True + elif isinstance(node, ast.Compare): + for index, op in enumerate(node.ops): + if type(op).__name__ not in ("Eq", "NotEq"): + continue + left = names_in(node.left) + right = names_in(node.comparators[index]) + if left & source and not (right & source): + grown = right - source + elif right & source and not (left & source): + grown = left - source + else: + continue + # Only a set that actually grows counts as a change. A comparison against a + # constant has no name on one side, and treating that as growth is a loop + # that never ends. + if grown: + source |= grown + changed = True + return source + + +def carries_case_value(node, values, containers): + """Whether an expression carries a value that came out of a case. + + A call is work, not a value: `read_exact(path)` reads the file the case is about and + `path.read_bytes()` reads it again. A conditional or a concatenation carries the value it + was built from, which is what `names = expect if isinstance(expect, list) else [expect]` + does. + """ + if isinstance(node, ast.Name): + return node.id in values or node.id in containers + if isinstance(node, ast.IfExp): + return (carries_case_value(node.body, values, containers) + or carries_case_value(node.orelse, values, containers)) + if isinstance(node, ast.BinOp): + return (carries_case_value(node.left, values, containers) + or carries_case_value(node.right, values, containers)) + if isinstance(node, (ast.List, ast.Tuple, ast.Set)): + return any(carries_case_value(e, values, containers) for e in node.elts) + if isinstance(node, ast.Subscript): + return carries_case_value(node.value, values, containers) + return False + + +def carries_source_value(node, source): + """Whether an expression carries the source text a control writes into a shipped file.""" + if isinstance(node, ast.Name): + return node.id in source + if isinstance(node, ast.IfExp): + return (carries_source_value(node.body, source) + or carries_source_value(node.orelse, source)) + if isinstance(node, ast.BinOp): + return (carries_source_value(node.left, source) + or carries_source_value(node.right, source)) + if isinstance(node, (ast.List, ast.Tuple, ast.Set)): + return any(carries_source_value(e, source) for e in node.elts) + if isinstance(node, ast.Subscript): + return carries_source_value(node.value, source) + return False + + +def _target_names(node): + """The names a target binds, whether it is a tuple of names or a single name.""" + if isinstance(node, ast.Tuple): + return [t.id for t in node.elts if isinstance(t, ast.Name)] + if isinstance(node, ast.Name): + return [node.id] + return [] + + +def case_values(tree, tables, params): + """The variables that hold a value that came out of a case. + + A case is a tuple in a table. A container is a name that holds a whole case. A case value is + a name that holds part of one -- the label, the expected rule name, the source snippet -- or + something computed from part of one. + + Two things are deliberately NOT case values. A value that came out of a CALL is work, not a + case: `data = path.read_bytes()` reads the file the case is about, and `text, newline = + read_exact(path)` reads it again. And a value computed only from the SOURCE snippet is the + file the control is about to write, not the case it is about. + """ + containers = set(tables) + values = set() + source = source_variables(tree) + calls = [] + for node in ast.walk(tree): + if isinstance(node, ast.Call) and isinstance(node.func, ast.Name): + calls.append(node) + + changed = True + while changed: + changed = False + + # A parameter that receives a case list is a case list, and a parameter that receives a + # case value is a case value. + for call in calls: + names = params.get(call.func.id) + if not names: + continue + for index, arg in enumerate(call.args): + if index >= len(names): + break + mentioned = names_in(arg) + if names[index] in containers or names[index] in values: + continue + if mentioned & containers: + containers.add(names[index]) + changed = True + elif mentioned & values and not (mentioned & source): + values.add(names[index]) + changed = True + + for node in ast.walk(tree): + # `for x in CASES:` binds a container; `for a, b in CASES:` binds two case values. + if isinstance(node, ast.For): + mentioned = names_in(node.iter) + if not (mentioned & (containers | values)): + continue + targets = _target_names(node.target) + whole_case = len(targets) == 1 and not isinstance(node.target, ast.Tuple) + if whole_case and (mentioned & containers): + if targets[0] not in containers: + containers.add(targets[0]) + changed = True + else: + for name in targets: + if name not in values: + values.add(name) + changed = True + + # `a, b, c = entry` and `x = entry[2]` take one case apart. + elif isinstance(node, ast.Assign): + builds_cases = any( + isinstance(comp, ast.ListComp) + and any(names_in(generator.iter) & containers + for generator in comp.generators) + for comp in ast.walk(node.value)) + mentioned = names_in(node.value) + for target in node.targets: + for name in _target_names(target): + if builds_cases: + if name not in containers: + containers.add(name) + changed = True + continue + if name in containers or name in values: + continue + if mentioned & containers: + values.add(name) + changed = True + elif (mentioned & values) and not (mentioned & source) \ + and carries_case_value(node.value, values, containers) \ + and not carries_source_value(node.value, source): + values.add(name) + changed = True + + # `for name in expects:` walks the list a case carries. + elif isinstance(node, ast.comprehension): + if names_in(node.iter) & (containers | values): + for name in _target_names(node.target): + if name not in values: + values.add(name) + changed = True + + return values, source + + +def single_shaped(node, bindings, depth=0): + """Whether an expression yields ONE reported name rather than a collection of them. + + The two ways a control can end up holding one name are taking the first element of a list + and taking the first match of an iterator. Both are what the contract forbids, so both are + recognised here. + """ + if depth > 6: + return False + if isinstance(node, ast.Call): + return isinstance(node.func, ast.Name) and node.func.id == "next" + if isinstance(node, ast.Subscript): + index = node.slice + if isinstance(index, ast.Slice): + return True + return isinstance(index, ast.Constant) and isinstance(index.value, int) + if isinstance(node, ast.IfExp): + return (single_shaped(node.body, bindings, depth + 1) + or single_shaped(node.orelse, bindings, depth + 1)) + if isinstance(node, ast.Name): + value = bindings.get(node.id) + if value is None: + return False + return single_shaped(value, bindings, depth + 1) + return False + + +def collection_literal(node, bindings, depth=0): + """Whether an expression is a collection of names rather than one name or one string.""" + if depth > 4: + return False + if isinstance(node, (ast.List, ast.Tuple, ast.Set)): + return True + if isinstance(node, (ast.ListComp, ast.SetComp, ast.GeneratorExp)): + return True + if isinstance(node, ast.Name): + value = bindings.get(node.id) + if value is None: + return False + return collection_literal(value, bindings, depth + 1) + return False + + +def parent_map(tree): + """Every node's parent, so a node can be read in the context it sits in.""" + parents = {} + for node in ast.walk(tree): + for child in ast.iter_child_nodes(node): + parents[child] = node + return parents + + +def name_tests(tree, values, bindings): + """The tests a control makes of a value that came out of a case. + + A test is a comparison the value takes part in, or a prefix match against it. The operand is + what the value is tested against, and that is the whole question: a collection of reported + names, or one reported name. + """ + parents = parent_map(tree) + tests = [] + for node in ast.walk(tree): + if isinstance(node, ast.Compare): + for index, op in enumerate(node.ops): + kind = type(op).__name__ + if kind not in ("In", "NotIn", "Eq", "NotEq"): + continue + left = names_in(node.left) + right = names_in(node.comparators[index]) + for side, operand in ((left, node.comparators[index]), + (right, node.left)): + hit = side & values + if not hit: + continue + # The side that holds the case's own value has to BE that value. A call on + # that side is work the control is doing, not the name it is judging by. + tested = node.left if side is left else node.comparators[index] + if has_call(tested): + continue + # What the name is tested against has to be something the guard reported. A + # literal is not: `"\r\n" in data` is a check on the file, not on a rule. + if isinstance(operand, ast.Constant): + continue + # A containment test asks whether the name is among the reported names. An + # equality test only counts when what it is compared against is a collection + # of names: comparing the name with one string is not a membership decision, + # and comparing it with the file the control is editing is not one either. + if kind in ("In", "NotIn"): + membership = True + else: + membership = collection_literal(operand, bindings) + tests.append({"kind": kind, "vars": sorted(hit), "operand": operand, + "membership": membership}) + elif isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) \ + and node.func.attr in ("startswith", "endswith"): + hit = set() + for arg in node.args: + hit |= names_in(arg) & values + if not hit: + continue + # The collection a prefix match walks is the iterable of the comprehension the + # match sits in, not the string being matched. + operand = node.func.value + walker = parents.get(node) + while walker is not None: + if isinstance(walker, (ast.GeneratorExp, ast.ListComp, ast.SetComp)): + operand = walker.generators[0].iter + break + walker = parents.get(walker) + tests.append({"kind": "Prefix", "vars": sorted(hit), + "operand": operand, "membership": True}) + return tests + + +def describes_unmarked(tree, values): + """What a control does instead of judging its case by the rule's name.""" + for node in ast.walk(tree): + if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) \ + and node.func.id == "next": + return ("takes the first name the guard reported and requires one to exist; " + "the case names no rule of its own") + for node in ast.walk(tree): + if isinstance(node, ast.Subscript) and isinstance(node.slice, ast.Constant) \ + and isinstance(node.slice.value, int): + return ("takes the first name the guard reported and carries it; " + "the case names no rule of its own") + for node in ast.walk(tree): + if not isinstance(node, ast.Compare): + continue + for index, op in enumerate(node.ops): + if type(op).__name__ not in ("In", "NotIn"): + continue + sides = ((node.left, node.comparators[index]), + (node.comparators[index], node.left)) + for literal, other in sides: + if isinstance(literal, ast.Constant) and isinstance(literal.value, str) \ + and not (names_in(other) & values): + return "decides by a fixed string appearing in the guard's output" + for node in ast.walk(tree): + if isinstance(node, ast.Name) and node.id.lower().endswith("code"): + return "decides by the guard's exit code alone" + return "decides without naming the rule the case broke" + + +# -------------------------------------------------------------------------------------- +# one control +# -------------------------------------------------------------------------------------- + +def analyse(path): + """Read one control and say how it decides a case.""" + tree = ast.parse(path.read_text(encoding="utf-8")) + tables = case_tables(tree) + params = function_params(tree) + values, source = case_values(tree, tables, params) + bindings = bindings_of(tree) + # A value that holds the source text a control writes into a shipped file is not a rule + # name. Testing one of those against the file the control is editing is anchor work. + judged = values - source + tests = name_tests(tree, judged, bindings) + + single = [] + membership = [] + for test in tests: + if single_shaped(test["operand"], bindings): + single.append(test) + elif test["membership"]: + membership.append(test) + + if single: + return { + "file": path.name, + "how": BY_SINGLE_NAME, + "detail": ("the case's own expected name is compared for equality with one " + "reported name: " + show(single[0]["operand"])), + "members": sorted(judged), + } + if membership: + return { + "file": path.name, + "how": BY_MEMBERSHIP, + "detail": "", + "members": sorted(judged), + } + return { + "file": path.name, + "how": BY_NO_NAME, + "detail": describes_unmarked(tree, judged), + "members": sorted(judged), + } + + +def bindings_of(tree): + """Every name and the expression it is bound to.""" + out = {} + for node in ast.walk(tree): + if isinstance(node, ast.Assign): + for target in node.targets: + if isinstance(target, ast.Name): + out[target.id] = node.value + elif isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name): + out[node.target.id] = node.value + return out + + +def derived_set(): + """The coverage set, derived from the package rather than written out.""" + return sorted(p for p in HERE.glob(COVERAGE_PATTERN) if p.is_file()) + + +# -------------------------------------------------------------------------------------- +# the guard +# -------------------------------------------------------------------------------------- + +def main(): + failures = [] + passed = 0 + + def check(rule, ok, detail=""): + nonlocal passed + if ok: + passed += 1 + print("ok " + rule + (": " + detail if detail else "")) + return + failures.append((rule, detail)) + print("FAIL " + rule + (": " + detail if detail else "")) + + controls = derived_set() + + # 1. The set is derived, and it is not empty. A guard that reads nothing and reports + # nothing is a guard that cannot fail. + check(RULE_DERIVED, bool(controls), + "%d control(s) from %s" % (len(controls), COVERAGE_PATTERN)) + + results = [] + unparsed = [] + for path in controls: + try: + results.append(analyse(path)) + except (SyntaxError, UnicodeDecodeError, ValueError) as error: + unparsed.append("%s: %s" % (path.name, error)) + + check(RULE_PARSE, not unparsed, + "%d parsed" % len(results) if not unparsed else "; ".join(unparsed)) + + by_file = {r["file"]: r for r in results} + + # 2. No control may decide a case by taking one reported name. + singles = [r for r in results if r["how"] == BY_SINGLE_NAME] + check(RULE_MEMBERSHIP, not singles, + "%d of %d decide a case by one reported name" % (len(singles), len(results)) + if not singles + else "; ".join("%s: %s" % (r["file"], r["detail"]) for r in singles)) + + # 3. The anchor. The two controls the reviewer found must carry the contract. + missing = [name for name in ANCHOR if name not in by_file + or by_file[name]["how"] != BY_MEMBERSHIP] + check(RULE_ANCHOR, not missing, + ", ".join(ANCHOR) if not missing + else "; ".join("%s: the control does not decide its cases by membership" % name + for name in missing)) + + # The classification, which is the report this guard owes the reader. + members = [r for r in results if r["how"] == BY_MEMBERSHIP] + others = [r for r in results if r["how"] != BY_MEMBERSHIP] + print("") + print(" by membership (%d):" % len(members)) + for record in members: + print(" " + record["file"]) + print(" not by membership (%d):" % len(others)) + for record in others: + print(" " + record["file"] + " " + record["detail"]) + + if failures: + print("") + print(" [mutation-contracts] %d failure(s):" % len(failures)) + for rule, detail in failures: + print(" - " + rule) + for line in detail.split("; "): + if line: + print(" " + line) + print("") + print("[mutation-contracts] %d of %d controls decide their cases by membership" + % (len(members), len(results))) + return 1 + + print("") + print("[mutation-contracts] %d of %d controls decide their cases by membership" + % (len(members), len(results))) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From e6e059d269d42b3a5a294ccb4401cad08d2832ee Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 06:36:13 +0800 Subject: [PATCH 60/95] Seven controls now judge a case by the NAME of the rule that case broke. The shape of the defect: a control applies a mutation to the shipped source, runs the guard that owns the rule, and decides whether the case was caught. Seven of them decided that by taking the FIRST rule name the guard happened to print, and they printed "all N mutations caught, each by name" -- so the sentence they printed was not true of what they did. A case could break one rule while the control printed another rule's name, and the control still counted the case as caught. The seven, and the shape each of them took: `tests/mutate-css.py`, `tests/mutate-dead-code.py`, `tests/mutate-file-index-rows.py` and `tests/mutate-freshness.py` took the first line the guard printed that begins with "FAIL"; `tests/mutate-handoff-identity.py`, `tests/mutate-record-detail.py` and `tests/mutate-search-client.py` took the first line that begins with "- ". Every case in all seven now names the rule it breaks, the runner collects every name the guard reported, and a case counts as caught only when its own name is among them. A case whose rule was not reported is reported as MISSED, together with the names that were, and the control exits non-zero. What did NOT change: the number of cases -- 3, 3, 4, 5, 2, 17 and 29 -- and the old and the new source snippet of every case. This is a change to how a case is judged, not to what is covered. The proof that the old contract was worth nothing, taken against the shipped controls. A decoy failure line inserted into a guard's failure report, so that the decoy is the first name the guard reports, makes the old shape attribute the DECOY to every case while still printing "all N mutations caught, each by name" -- the claim was false and the false claim was visible in the control's own output. With the decoy as the only name the guard reports, the old shape still counted every case as caught and exited 0 (shown on `tests/mutate-search-client.py`, whose old output was "all 29 mutations caught, each by name" with exit 0, while the guard had reported one rule that none of the 29 broke). The new shape names each case its own rule in the first variant, and reports every case as MISSED and exits non-zero in the second, on all seven. Each guard was restored byte for byte after its probe. One judgement call is stated rather than buried: a mutation can trip more than one assertion, and a case carries the name of the assertion that STATES the rule the mutation breaks. Where the probe reported several names, the one that carries the rule is the one the case names. --- .../mmc-trajectory/tests/mutate-css.py | 70 +++++++++++--- .../mmc-trajectory/tests/mutate-dead-code.py | 70 +++++++++++--- .../tests/mutate-file-index-rows.py | 71 +++++++++++--- .../mmc-trajectory/tests/mutate-freshness.py | 72 ++++++++++++++- .../tests/mutate-handoff-identity.py | 64 ++++++++++++- .../tests/mutate-record-detail.py | 84 ++++++++++++++--- .../tests/mutate-search-client.py | 92 +++++++++++++++++-- 7 files changed, 459 insertions(+), 64 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-css.py b/plugins/avatasia/mmc-trajectory/tests/mutate-css.py index f1b02ed6..da9d0e56 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-css.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-css.py @@ -19,7 +19,13 @@ Each mutation must be caught *by name*. A probe that dies on an exception names nothing, and a control that reports that as "caught" is reporting a crash as if it were evidence. + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the probe reported THAT rule: a probe that went red on some other assertion has not shown +that this guard owns this mutation. """ +import json +import re import subprocess import sys from pathlib import Path @@ -48,17 +54,20 @@ } .kpi-section {""", + "every class selector in the stylesheet has a consumer outside it", ), ( "the turn header loses its meta line and the rule stays behind", TURN_META_CONSUMER, "var meta = el('span');", + "every class selector in the stylesheet has a consumer outside it", ), ( "a custom property is declared in a theme block and never read", LIGHT_THEME_TAIL, """ --mcode-danger: #f73646; --mcode-never-read: #123456;""", + "--mcode-never-read is read somewhere", ), ] @@ -88,13 +97,43 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") -def named_failure(output): - """The first rule the probe named, or None when it failed without naming one.""" - return next( - (line.strip()[len("FAIL"):].strip() for line in output.splitlines() - if line.strip().startswith("FAIL")), - None, - ) +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + Two formats, as in `mutate-search-endpoint.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names def main(): @@ -108,7 +147,7 @@ def main(): original = CLIENT.read_bytes() missed = [] unnamed = [] - for label, old, new in MUTATIONS: + for label, old, new, expect in MUTATIONS: text, newline = read_exact(CLIENT) if old not in text: print(f"SKIP {label}: anchor not found") @@ -131,12 +170,19 @@ def main(): print(f"MISSED {label}: the probe still passed") missed.append(label) continue - name = named_failure(output) - if name is None: + names = reported_rules(output) + if not names: print(f"UNNAMED {label}: the probe failed without naming a rule") unnamed.append(label) - else: - print(f"caught {label}\n -> {name}") + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") if CLIENT.read_bytes() != original: print(f"NOT RESTORED {CLIENT}") diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-dead-code.py b/plugins/avatasia/mmc-trajectory/tests/mutate-dead-code.py index 1173dcd7..a58c0b0b 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-dead-code.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-dead-code.py @@ -21,7 +21,13 @@ Each must be caught *by name*. A probe that dies on an exception names nothing, and a control that reports that as caught is reporting a crash as if it were evidence. + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the probe reported THAT rule: a probe that went red on some other assertion has not shown +that this guard owns this mutation. """ +import json +import re import subprocess import sys from pathlib import Path @@ -47,16 +53,19 @@ } function axisIcon() {""", + "every function the client declares is used by something", ), ( "a caller is deleted and the helper is left behind", AXIS_ICON_CALLER, " axis.appendChild(document.createTextNode(''));", + "every function the client declares is used by something", ), ( "the only reference to a function is left in a comment", AXIS_ICON_CALLER, " // was: axis.appendChild(axisIcon())\n axis.appendChild(document.createTextNode(''));", + "every function the client declares is used by something", ), ] @@ -86,13 +95,43 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") -def named_failure(output): - """The first rule the probe named, or None when it failed without naming one.""" - return next( - (line.strip()[len("FAIL"):].strip() for line in output.splitlines() - if line.strip().startswith("FAIL")), - None, - ) +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + Two formats, as in `mutate-search-endpoint.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names def main(): @@ -106,7 +145,7 @@ def main(): original = CLIENT.read_bytes() missed = [] unnamed = [] - for label, old, new in MUTATIONS: + for label, old, new, expect in MUTATIONS: text, newline = read_exact(CLIENT) if old not in text: print(f"SKIP {label}: anchor not found") @@ -129,12 +168,19 @@ def main(): print(f"MISSED {label}: the probe still passed") missed.append(label) continue - name = named_failure(output) - if name is None: + names = reported_rules(output) + if not names: print(f"UNNAMED {label}: the probe failed without naming a rule") unnamed.append(label) - else: - print(f"caught {label}\n -> {name}") + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") if CLIENT.read_bytes() != original: print(f"NOT RESTORED {CLIENT}") diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-file-index-rows.py b/plugins/avatasia/mmc-trajectory/tests/mutate-file-index-rows.py index e8714df2..d6463110 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-file-index-rows.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-file-index-rows.py @@ -24,7 +24,13 @@ Each mutation must be caught *by name*. A probe that dies on an exception names nothing, and a control that reports that as caught is reporting a crash as if it were evidence. + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the probe reported THAT rule: a probe that went red on some other assertion has not shown +that this guard owns this mutation. """ +import json +import re import subprocess import sys from pathlib import Path @@ -56,21 +62,25 @@ "a record with no locator is repaired to the first file instead of -1", ROW_UNLOCATED, UNLOCATED_REPAIRED, + "a record with no locator says file -1", ), ( "a locator whose file moved is filed under the first file instead of -1", ROW, MOVED_REPAIRED, + "a locator whose file is not in the table says file -1, not file 0", ), ( "two generations holding the same file name are matched by basename", FULL_PATH, BASENAME, + "each record names the file its locator names", ), ( "the byte range is copied onto the row again, so it rides along on every segment", ROW_UNLOCATED, BYTES_BACK, + "the byte range the locator carries is dropped, not copied onto the row", ), ] @@ -100,13 +110,43 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") -def named_failure(output): - """The first rule the probe named, or None when it failed without naming one.""" - return next( - (line.strip()[len("FAIL"):].strip() for line in output.splitlines() - if line.strip().startswith("FAIL")), - None, - ) +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + Two formats, as in `mutate-search-endpoint.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names def main(): @@ -120,7 +160,7 @@ def main(): original = MODULE.read_bytes() missed = [] unnamed = [] - for label, old, new in MUTATIONS: + for label, old, new, expect in MUTATIONS: text, newline = read_exact(MODULE) if old not in text: print(f"SKIP {label}: anchor not found") @@ -143,12 +183,19 @@ def main(): print(f"MISSED {label}: the probe still passed") missed.append(label) continue - name = named_failure(output) - if name is None: + names = reported_rules(output) + if not names: print(f"UNNAMED {label}: the probe failed without naming a rule") unnamed.append(label) - else: - print(f"caught {label}\n -> {name.splitlines()[0]}") + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") if MODULE.read_bytes() != original: print(f"NOT RESTORED {MODULE}") diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-freshness.py b/plugins/avatasia/mmc-trajectory/tests/mutate-freshness.py index 666f4d15..168ca652 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-freshness.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-freshness.py @@ -8,7 +8,13 @@ The last case is the other direction: an edit that only ADDS A COMMENT must leave the probe green. A guard that cannot pass is as broken as one that cannot fail, and that is the only way to prove this probe is reading behaviour rather than prose. + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the probe reported THAT rule: a probe that went red on some other assertion has not shown +that this guard owns this mutation. """ +import json +import re import subprocess import sys from pathlib import Path @@ -24,6 +30,7 @@ "a 304 is taken as nothing having happened, so the warning outlives the fault", " var cleared = noteFresh('trajectory', true, '');\n if (state.loading) {", " var cleared = false;\n if (state.loading) {", + "a 304 takes the warning down — this is the defect the guard exists for", ), ( # The other half of the same layer: the list failure used to be written to a flag that @@ -31,24 +38,28 @@ "a failed session list is recorded and never shown", " noteFresh('sessions', false, freshnessReason(error));\n renderBars();", " renderBars();", + "a failed list is visible at all — it used to write a flag nothing read", ), # ---- the count ------------------------------------------------------------------------ ( "recovering does not reset the count, so a past outage is remembered forever", " if (ok) {\n slot.streak = 0;\n slot.reason = '';", " if (ok) {\n slot.reason = '';", + "and the streak is back to zero", ), # ---- which kind of failure ------------------------------------------------------------- ( "a dropped connection is reported as a bad answer, which sends the reader the wrong way", " if (error && typeof error.status === 'number') return 'bad_response';\n return 'offline';", " return 'bad_response';", + "a dropped connection says so", ), # ---- the cost of the fix --------------------------------------------------------------- ( "every 304 re-renders, trading a stuck warning for a ledger rebuilt every 8 seconds", " if (cleared) render();", " render();", + "five more 304s on a quiet session render nothing", ), ] @@ -67,6 +78,11 @@ def write_exact(path, text, newline): def run_probe(): + """The probe's verdict, stdout and stderr together. + + This probe writes its failures to stderr and its summary to stdout, so both are captured: + a name that arrived on stderr is a name the guard reported. + """ proc = subprocess.run( ["node", str(PROBE)], cwd=str(ROOT), @@ -78,6 +94,45 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + Two formats, as in `mutate-search-endpoint.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names + + def main(): code, output = run_probe() if code != 0: @@ -89,7 +144,7 @@ def main(): original = HTML.read_bytes() missed = [] unnamed = [] - for label, old, new in MUTATIONS: + for label, old, new, expect in MUTATIONS: text, newline = read_exact(HTML) if old not in text: print(f"SKIP {label}: anchor not found") @@ -108,12 +163,19 @@ def main(): print(f"MISSED {label}: the probe still passed") missed.append(label) continue - first = next((line.strip() for line in output.splitlines() if line.strip().startswith("- ")), None) - if first is None: + names = reported_rules(output) + if not names: print(f"UNNAMED {label}: the probe failed without naming a rule") unnamed.append(label) - else: - print(f"caught {label}\n -> {first}") + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") # ---- the other direction --------------------------------------------------------------- # A note in this file must not change a single verdict. If adding prose can turn the probe diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-handoff-identity.py b/plugins/avatasia/mmc-trajectory/tests/mutate-handoff-identity.py index c3d05878..0daba4f8 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-handoff-identity.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-handoff-identity.py @@ -16,7 +16,13 @@ Both have to be caught by the same named assertion in client-check.mjs, which runs the shipped `handoffSelected` over a deferred raw read and moves the page while that read is in flight. + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the probe reported THAT rule: a probe that went red on some other assertion has not shown +that this guard owns this mutation. """ +import json +import re import subprocess import sys from pathlib import Path @@ -31,11 +37,13 @@ "the handoff reads the session off the page after its reads, so a navigation names the wrong conversation", " var session = state.payload ? state.payload.session : null;", " var session = { get id() { return state.payload.session.id; }, get label() { return state.payload.session.label; } };", + 'the handoff names the session the selected record came from', ), ( "the handoff names no conversation at all", " var session = state.payload ? state.payload.session : null;", " var session = null;", + 'the handoff names the session the selected record came from', ), ] @@ -65,6 +73,45 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + The probe prints `FAILURES:` and one ` - ` line per failure, so a case is only caught + when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names + + def main(): code, output = run_probe() if code != 0: @@ -76,7 +123,7 @@ def main(): original = CLIENT.read_bytes() missed = [] unnamed = [] - for label, old, new in MUTATIONS: + for label, old, new, expect in MUTATIONS: text, newline = read_exact(CLIENT) if old not in text: print(f"SKIP {label}: anchor not found") @@ -95,12 +142,19 @@ def main(): print(f"MISSED {label}: the probe still passed") missed.append(label) continue - first = next((line.strip() for line in output.splitlines() if line.strip().startswith("- ")), None) - if first is None: + names = reported_rules(output) + if not names: print(f"UNNAMED {label}: the probe failed without naming a rule") unnamed.append(label) - else: - print(f"caught {label}\n -> {first}") + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") if CLIENT.read_bytes() != original: print(f"NOT RESTORED {CLIENT}") diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-record-detail.py b/plugins/avatasia/mmc-trajectory/tests/mutate-record-detail.py index d4141674..e2d8697c 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-record-detail.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-record-detail.py @@ -12,7 +12,13 @@ Each mutation below is one of those, and each has to be caught **by name** — a guard that fails for some other reason is not evidence that this one was caught. + +Every case names the rule it breaks in its `expect` field, and a case counts as caught only when +the probe reported THAT rule: a probe that went red on some other assertion has not shown that +this guard owns this mutation. """ +import json +import re import subprocess import sys from pathlib import Path @@ -30,18 +36,21 @@ SERVER, " hasText: !isNilField(patch.text),", " hasText: true,", + 'a body-less reply reports no text', ), ( "hasResult is taken on trust, so a tool result with no body still offers its tab", SERVER, " hasResult: !isNilField(patch.result),", " hasResult: true,", + 'hasResult matches "result" on #1 (user)', ), ( "hasArguments counts an empty string as absent, which is not the test the page used", SERVER, " hasArguments: patch.arguments !== null && patch.arguments !== undefined,", " hasArguments: !isNilField(patch.arguments),", + 'an empty string counts as arguments, because that is what the page tested', ), # --- the strip ------------------------------------------------------------------- ( @@ -49,18 +58,21 @@ SERVER, " delete record.text;\n delete record.thinking;", " delete record.thinking;", + '"text" is gone from the wire', ), ( "the strip runs after the body was serialized, so it changes nothing that was sent", SERVER, " slimPayloadForWire(payload);\n\n let body;", " let body;", + 'the trajectory route strips before serializing', ), ( "the strip also runs on the search path, which still reads those fields in this process", SERVER, " const needle = foldNeedle(asked);", " const needle = foldNeedle(asked);\n slimPayloadForWire({ turns: [] });", + 'the search path does not strip, because it still reads those fields in this process', ), # --- which record the endpoint answers for ---------------------------------------- ( @@ -68,30 +80,35 @@ SERVER, " rowPos: rowPos++,", " rowPos: 0,", + 'line 1 numbers its records 0..n-1 with no repeats', ), ( "the position is offered even for a record with no line to project", SERVER, " if (rowPos !== null && typeof pos === 'number' && Number.isInteger(pos) && pos >= 0) {", " if (rowPos !== null) {", + 'a record whose position is "x" gets no position', ), ( "the endpoint projects the row twice, so every position after the first is one off", SERVER, " rows: [found.row],", " rows: [found.row, found.row],", + 'it re-projects a single row rather than the session', ), ( "a position the one-row projection cannot produce is answered with whatever is nearest", SERVER, " if (record === undefined) {", " if (record === undefined) {\n record = one.turns.flatMap((turn) => turn.records ?? [])[0];\n }\n if (record === undefined) {", + 'and nothing replaces it afterwards', ), ( "the endpoint answers with a kind, which re-projecting one line cannot reproduce", SERVER, " sendJson(response, 200, {\n index: wanted,", " sendJson(response, 200, {\n index: wanted,\n kind: record.kind,", + 'and does NOT answer with a kind, which re-projecting one line cannot reproduce faithfully', ), # --- the summary the page used to assemble ---------------------------------------- ( @@ -99,6 +116,7 @@ SERVER, " if (!record.summary) record.summary = record.toolName || record.title || '';", " if (!record.summary) record.summary = '';", + 'while its summary is not empty, so hasText cannot be read off it', ), # --- the page -------------------------------------------------------------------- ( @@ -106,12 +124,14 @@ CLIENT, "return !!record && record.kind === 'reply' && record.hasText === false;", "return !!record && record.kind === 'reply' && isNil(record.text);", + 'the 空答复 rule reads the flag, not a field the page no longer receives', ), ( "the row falls back to the tool name itself, undoing the fold on the server", CLIENT, " var summaryText = record.summary || '';", " var summaryText = record.summary || record.toolName || '';", + 'the row summary is the one server-derived field, with no fallback chain under it', ), ( # The replacement text is written in live code on purpose. It used to call @@ -124,18 +144,21 @@ CLIENT, " if (tabId === 'result') return buildDetailBlock('工具结果', record, 'result', false);", " if (tabId === 'result') return el('div', 'note', String(record.result));", + 'the 工具结果 panel draws "result" from the detail, not off the record', ), ( "the tab is offered from a field the page no longer has, so it is always on", CLIENT, " return record.kind === 'toolResult' && record.hasResult === true;", " return record.kind === 'toolResult' && record.result !== null;", + 'and the 工具结果 tab is offered from hasResult', ), ( "failure is decided by the field that was a duplicate, which no longer exists", CLIENT, " var failed = record.isError === true;", " var failed = !!(record.isError || record.error);", + 'failure is decided by isError alone, which is what the old `|| error` always agreed with', ), ] @@ -160,13 +183,43 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") -def named_failure(output): - """The first assertion the probe named, or None when it failed without naming one.""" - return next( - (line.strip()[1:].strip() for line in output.splitlines() - if line.strip().startswith("- ")), - None, - ) +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + The probe prints one ` - ` line per failure, so a case is only caught when the rule + it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names def main(): @@ -180,7 +233,7 @@ def main(): originals = {path: path.read_bytes() for path in {m[1] for m in MUTATIONS}} missed, unnamed = [], [] try: - for label, path, old, new in MUTATIONS: + for label, path, old, new, expect in MUTATIONS: text, newline = read_exact(path) if old not in text: print(f"SKIP {label}: anchor not found") @@ -199,12 +252,19 @@ def main(): print(f"MISSED {label}: the probe still passed") missed.append(label) continue - name = named_failure(output) - if name is None: + names = reported_rules(output) + if not names: print(f"UNNAMED {label}: the probe failed without naming a rule") unnamed.append(label) - else: - print(f"caught {label}\n -> {name}") + continue + # The case names the rule it is about, so the probe has to name THAT rule and not + # some other one that happened to go red in the same run: a probe that went red + # elsewhere has not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") finally: for path, data in originals.items(): path.write_bytes(data) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py index 2af5251b..18597bec 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-search-client.py @@ -2,7 +2,7 @@ The filter used to be `recordSearchText(record).indexOf(needle) >= 0` — a scan of five fields the page happened to be holding. It is now a set of record indexes the server sent, and every mutation -here undoes one of the four things that move required and the page now owns: +here undoes one of the six things that move required and the page now owns: - asking a set at all, instead of scanning text; - telling 「no answer yet」 apart from 「matched nothing」; @@ -15,7 +15,13 @@ the pending answer of the record that took its place. Each is a change the page renders differently and none of them throws. + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the probe reported THAT rule: a probe that went red on some other assertion has not shown +that this guard owns this mutation. """ +import json +import re import subprocess import sys from pathlib import Path @@ -30,31 +36,37 @@ "the filter scans the record's own text again, ignoring the server's answer", " if (hitSet === null || hitSet === undefined) return true;\n return !!(record && hitSet.has(record.index));", " if (hitSet === null || hitSet === undefined) return true;\n return String((record && record.summary) || '').toLowerCase().indexOf(String(needle).toLowerCase()) >= 0;", + "a record whose text contains the needle but which the server did not name fails", ), ( "an unanswered search is rendered as matched nothing, so the table empties while it loads", " if (hitSet === null || hitSet === undefined) return true;", " if (hitSet === null || hitSet === undefined) return false;", + "an unanswered needle filters nothing, and says so elsewhere", ), ( "no answer in force is turned into an empty set, so it reads as 「matched nothing」", " if (!Array.isArray(hits)) return null;", " if (!Array.isArray(hits)) return new Set();", + "an unanswered needle is not an empty result", ), ( "a hit with no usable index is filed under a key that matches every record", " if (Number.isInteger(index)) set.add(index);", " set.add(Number.isInteger(index) ? index : NaN);", + "a hit without an index is dropped rather than becoming one", ), ( "a late answer is allowed to overwrite a newer one", " if (seq !== searchSeq) return;\n searchController = null;", " searchController = null;", + "policy: only the newest search answer is allowed to write", ), ( "a search in flight is not aborted, so every keystroke's request still runs to the end", " if (searchController) searchController.abort();", "", + "policy: the search request in flight is aborted when a newer one starts", ), ( "a changed payload keeps the old hit set, so the new table is filtered by the old answer", @@ -64,41 +76,49 @@ " if (state.query && (state.searchSet && displayedRecordSet() !== state.searchSet)) {\n" " runSearch(state.query);\n" " }", + "a payload that changed re-runs the search rather than filtering by the old answer", ), ( "a content-only payload change does not retire the previous search answer", " if (state.query && (changed || (state.searchSet && displayedRecordSet() !== state.searchSet))) {", " if (state.query && changed) {", + "a content-only payload change retires the previous search answer", ), ( "a read joins a newer record's pending read after reload, because the key does not carry the record's version", " return sid + '#' + gen + '#' + (state.etag || '') + '#' + index + '#' + (version || '');", " return sid + '#' + gen + '#' + (state.etag || '') + '#' + index;", + "a retried read cannot join a newer record's pending read after reload", ), ( "clearing the filters leaves the hit set behind, so the next query filters by a stale answer", " state.searchHits = null;\n state.searchPending = false;\n state.searchTotal = 0;\n state.searchTruncated = false;\n state.searchError = null;\n dropOverviewSelection();", " dropOverviewSelection();", + "policy: clearing the filters drops the answer with them", ), ( "the count line cannot tell 「searching」 from 「searched and found none」", " if (searchState.searchHits !== null && searchState.searchTotal === 0) return '未找到';", " if (searchState.searchHits !== null) return '命中 ' + formatInt(searchState.searchTotal) + ' 条';", + "a needle answered with nothing", ), ( "a search that failed is rendered as an empty table, which reads as an answer", " if (searchState.searchError) return '搜索失败，以下为上次结果';", "", + "a needle that failed says so rather than showing an empty table", ), ( "the needle is sent without the generation, so a slice search answers about the whole lineage", " + (generation === null ? '' : '&generation=' + encodeURIComponent(String(generation)))", " + ''", + "policy: a search is asked about the slice the body on screen is", ), ( "the hit set is built from the summaries rather than the indexes, so it can never match a row", " var index = entry && typeof entry.index === 'number' ? entry.index : NaN;", " var index = entry && typeof entry.summary === 'string' ? entry.summary.length : NaN;", + "the hit set holds the server's indexes", ), # The one an acceptance review caught, and the reason the mutation list above is not the end of # the story. Every entry there changes what the page COMPUTES. This one changes what it REMEMBERS @@ -116,6 +136,7 @@ "an answer that arrives leaves the failure that preceded it standing", " state.searchError = null;\n state.searchRefresh = 0;\n resetWindowMode();", " state.searchRefresh = 0;\n resetWindowMode();", + "and the retry that answers it clears the failure, not just the pending flag", ), # ---- the search is about the body on screen --------------------------------------- # @@ -135,6 +156,7 @@ "the search asks about the current session instead of the one on screen", " var session = displayedSessionParam();", " var session = sessionParam();", + "the search asks about the session on screen", ), ( "the search stops checking which session the answer is about", @@ -142,12 +164,14 @@ " || answeredSlice !== displayedSlice) {", " if (!answered || !displayed\n" " || answeredSlice !== displayedSlice) {", + "an answer about another session does not install its hit set", ), ( "the search stops checking which slice of the session the answer is about", " if (!answered || !displayed || answered.id !== displayed.id\n" " || answeredSlice !== displayedSlice) {", " if (!answered || !displayed || answered.id !== displayed.id) {", + "an answer about another slice of the same session does not install its hit set", ), # ---- the slice is the one the BODY is, not the one the reader picked --------------- # @@ -161,6 +185,7 @@ "the search asks about the generation the reader picked, not the slice on screen", " var session = displayedSessionParam();\n var generation = displayedGeneration();", " var session = displayedSessionParam();\n var generation = state.generation;", + "the search asks about the slice on screen, not the one the reader picked", ), ( "the raw read asks about the generation the reader picked, not the slice on screen", @@ -170,6 +195,7 @@ " var url = '/api/trajectory/raw?index=' + encodeURIComponent(String(index))\n" " + '&session=' + displayedSessionParam();\n" " var generation = state.generation;", + "a raw read asks for the slice on screen", ), ( "the record read asks about the generation the reader picked, not the slice on screen", @@ -179,11 +205,13 @@ " var url = '/api/trajectory/record?index=' + encodeURIComponent(String(index))\n" " + '&session=' + displayedSessionParam();\n" " var generation = state.generation;", + "a detail read asks for the slice on screen too", ), ( "the raw cache key names the generation the reader picked, not the slice on screen", " var gen = displayedGeneration() === null ? 'all' : String(displayedGeneration());", " var gen = state.generation === null ? 'all' : String(state.generation);", + "the raw cache key names the slice on screen", ), # ---- the version of the record a position names ------------------------------------ # @@ -196,12 +224,14 @@ "the search stops checking which version of the record a hit names", " if (typeof shown.v === 'string' && shown.v !== '' && shown.v !== hit.v) {", " if (false) {", + "a search answer for a newer body cannot filter the displayed older body", ), ( "the raw read stops naming the version of the record it asks for", " // the retry is a different question wearing the same question's clothes.\n" " if (version) url += '&v=' + encodeURIComponent(version);", " // the retry is a different question wearing the same question's clothes.", + "a raw read names the version of the record it is asking for", ), ( "the record read stops naming the version of the record it asks for", @@ -212,6 +242,7 @@ " // The same record, the same way: the version travels with the request so the service can\n" " // refuse when the position it would answer is a different record than the one opened — and\n" " // it is the record's own version, so a retry after a reload asks about the same record.", + "and a detail read names it too", ), # ---- the version travels with the RECORD, not with the screen ------------------------ # @@ -240,6 +271,7 @@ " }\n" " return '';\n" " }", + "and not about the record that happens to be at that index after the reload", ), # ---- the answer has to be about the SET the page is showing -------------------------- # @@ -254,12 +286,14 @@ " + '&n=' + versions.length\n" " + '&fp=' + encodeURIComponent(recordSetFingerprint(versions));", " + '';", + "the search names how many records the body on screen has", ), ( "the search says how many records it is about, but not which ones", " + '&n=' + versions.length\n" " + '&fp=' + encodeURIComponent(recordSetFingerprint(versions));", " + '&n=' + versions.length;", + "and a fingerprint of their versions", ), ( "a refusal from the service is treated as an ordinary failure instead of asking again", @@ -268,6 +302,7 @@ " return;\n" " }", "", + "and the page asks again against a body it fetches, a bounded number of times", ), ] @@ -297,6 +332,44 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") +def rule_name(text): + """The name of a rule the probe reported, without the evidence a comparison carries. + + The probe reports a comparison as `: got , want `. The values are the + evidence, not the rule, so they are cut off — and the cut is made at a `: got ` whose tail + really is `, want `, so a name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules the probe named as broken. + + Two formats, as in `mutate-search-endpoint.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names + + def main(): code, output = run_probe() if code != 0: @@ -308,7 +381,7 @@ def main(): original = CLIENT.read_bytes() missed = [] unnamed = [] - for label, old, new in MUTATIONS: + for label, old, new, expect in MUTATIONS: text, newline = read_exact(CLIENT) if old not in text: print(f"SKIP {label}: anchor not found") @@ -327,12 +400,19 @@ def main(): print(f"MISSED {label}: the probe still passed") missed.append(label) continue - first = next((line.strip() for line in output.splitlines() if line.strip().startswith("- ")), None) - if first is None: + names = reported_rules(output) + if not names: print(f"UNNAMED {label}: the probe failed without naming a rule") unnamed.append(label) - else: - print(f"caught {label}\n -> {first}") + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") if CLIENT.read_bytes() != original: print(f"NOT RESTORED {CLIENT}") From 8939b745f0dc0a2874a4756cc5e49c479e6297b2 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 06:36:13 +0800 Subject: [PATCH 61/95] A control that says it caught its cases by name has to mean it, and now something checks. The rule a previous review round found was a SHAPE: a control that takes the first name its guard reported and prints it as the rule the case broke. Two controls were fixed for it. The shape itself had no owner, so relaxing any of the others back to it would have failed nothing -- and the guard that classifies the controls could see that shape, but did not rule on it. This adds that owner. The guard reads the source of every mutation control in the package and classifies it, and one of the rules it now states is 'a control that claims it caught cases by name judges them by membership'. The claim is read from the strings the control PRINTS, not from anywhere in its source: a comment or a docstring that mentions the phrase is prose about the control, not a claim it makes to whoever reads its output. That is also why the rule is about text: the guards in this package print a failure in several shapes, and a rule that had to parse them all would be a rule about formats instead. What the rule found, and what was done about it: it named seven controls that printed exactly that sentence while taking the first name their guard reported. Those seven judge by membership now, in the commit before this one. The guard still cannot be silenced by the set it derives shrinking: it reads the set from its own directory, so a copy of it placed where no controls sit derives an empty set and fails. The control that owns the new rule has six cases now, each caught by name. The new one puts the claim back into a control that does not judge by membership -- in a copy, never in the package -- and the guard has to fail and name THAT rule: a guard that went red for another reason has not shown that this one bites. The control it puts the claim into is not written out here either; it is taken from the guard's own report, which names every control that does not judge by membership. The counts: the guard states five rules, and the control has six cases. The classification the guard prints on every run is the report it owes the reader -- how many controls judge by membership, how many do not, and which of them claim to catch cases by name. --- .../tests/mutate-mutation-contracts.py | 98 ++++++++++++++++++- .../tests/verify-mutation-contracts.py | 50 ++++++++++ 2 files changed, 147 insertions(+), 1 deletion(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-mutation-contracts.py b/plugins/avatasia/mmc-trajectory/tests/mutate-mutation-contracts.py index bafde2e6..eab7caa6 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-mutation-contracts.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-mutation-contracts.py @@ -39,6 +39,15 @@ RULE_MEMBERSHIP = "a mutation control decides a case by membership, not by one reported name" RULE_ANCHOR = "the two reviewed mutation controls decide their cases by membership" RULE_DERIVED = "the set of mutation controls is derived from the package and is not empty" +RULE_CLAIM = "a control that claims it caught cases by name judges them by membership" + +# The two controls the guard anchors on. They are copied into the shorter trees below so that the +# anchor holds there and the rule under test is the only one that can fail. +ANCHOR = ("mutate-search-endpoint.py", "mutate-search-segments.py") + +# The claim, as a control that does not honour it would print it. It goes at the END of a control's +# source, so it is a print of its own and nothing else in the file changes. +CLAIM_PRINT = 'print("all mutations caught, each by name")' # The decision that is correct, and the two that are not. The case table is not touched by the # first two: every case keeps its expected rule name, and only the decision changes. @@ -212,6 +221,88 @@ def run_in_a_shorter_tree(label, expect, keep): shutil.rmtree(workspace, ignore_errors=True) +def non_claiming_control(output): + """The first control the guard reports as not judging by membership AND not claiming to. + + Both lists come out of the guard's own report, so nothing here is written out by hand: a + control added tomorrow is picked up without this file being edited. A control that claims to + catch its cases by name and does not judge them by membership is already reported by the + rule under test, so it cannot serve as the subject of this case. + """ + not_members = [] + claimers = [] + section = None + for line in output.splitlines(): + text = line.strip() + if text.startswith("by membership ("): + section = "members" + continue + if text.startswith("not by membership ("): + section = "not" + continue + if text.startswith("claiming to catch cases by name ("): + section = "claim" + continue + if not text: + continue + if not text.startswith("mutate-"): + section = None + continue + name = text.split()[0] + if section == "not": + not_members.append(name) + elif section == "claim": + claimers.append(name) + for name in not_members: + if name not in claimers: + return name + return None + + +def run_with_a_claim_in_a_shorter_tree(): + """Run a COPY of the guard against a control that claims to catch cases by name. + + The control is one the guard itself reports as not judging by membership, and the claim is + put back into it -- in the copy, never in the package. The guard has to fail, and it has to + name the rule: a guard that went red for another reason has not shown that this one bites. + """ + code, output = run_guard() + if code != 0: + print("the guard does not pass on the package as it stands; fix that first") + return False + target = non_claiming_control(output) + if target is None: + print("no control that does not judge by membership and does not claim to: " + "nothing for this case to prove") + return False + + workspace = tempfile.mkdtemp(prefix="mutation-contracts-claim-") + try: + shutil.copy(GUARD, Path(workspace) / GUARD.name) + for name in list(ANCHOR) + [target]: + shutil.copy(HERE / name, Path(workspace) / name) + path = Path(workspace) / target + text = path.read_text(encoding="utf-8") + path.write_bytes((text + "\n\n" + CLAIM_PRINT + "\n").encode("utf-8")) + code, output = run_guard(Path(workspace) / GUARD.name, cwd=workspace) + names = reported_rules(output) + if code == 0: + print(f"MISSED the claim that the cases were caught by name is put back into a " + f"control that does not judge by membership: the guard passed with " + f"{target} claiming to catch cases by name") + return False + if RULE_CLAIM not in names: + print(f"MISSED the claim that the cases were caught by name is put back into a " + f"control that does not judge by membership: the guard named {names} " + f"instead of {RULE_CLAIM}") + return False + print(f"caught the claim that the cases were caught by name is put back into a " + f"control that does not judge by membership\n -> {RULE_CLAIM}") + return True + finally: + shutil.rmtree(workspace, ignore_errors=True) + + def main(): original_bytes = TARGET.read_bytes() original_digest = digest(TARGET) @@ -269,6 +360,11 @@ def main(): RULE_ANCHOR, ["mutate-search-endpoint.py"]): missed.append("the set of controls the guard derives has been shrunk to one") + # Nor by a control claiming to catch its cases by name while judging them by something else. + if not run_with_a_claim_in_a_shorter_tree(): + missed.append("the claim that the cases were caught by name is put back into a " + "control that does not judge by membership") + if TARGET.read_bytes() != original_bytes: print(f"NOT RESTORED {TARGET}") return 1 @@ -283,7 +379,7 @@ def main(): if unnamed: print(f"{len(unnamed)} mutation(s) failed without naming a rule") return 1 - print(f"\nall {len(RELAXATIONS) + 2} relaxations caught, each by name") + print(f"\nall {len(RELAXATIONS) + 3} relaxations caught, each by name") return 0 diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-mutation-contracts.py b/plugins/avatasia/mmc-trajectory/tests/verify-mutation-contracts.py index 24dcfebc..2b772afc 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-mutation-contracts.py +++ b/plugins/avatasia/mmc-trajectory/tests/verify-mutation-contracts.py @@ -54,6 +54,17 @@ RULE_ANCHOR = "the two reviewed mutation controls decide their cases by membership" RULE_DERIVED = "the set of mutation controls is derived from the package and is not empty" RULE_PARSE = "every mutation control in the package parses" +RULE_CLAIM = "a control that claims it caught cases by name judges them by membership" + +# The claim itself. A control that PRINTS this is saying that it caught its cases by name, and +# that has to be true of it. +# +# It is looked for in the strings a control prints, not anywhere in its source: a comment or a +# docstring that mentions the phrase is prose, not a claim, and a rule that scanned the whole +# file would fire on the documentation of the very controls that do honour it. This is a rule +# about text, and the text it is about is the claim the control makes to whoever reads its +# output. +CLAIM_PHRASE = "each by name" # How a control decides. BY_MEMBERSHIP = "membership" @@ -448,6 +459,27 @@ def describes_unmarked(tree, values): # one control # -------------------------------------------------------------------------------------- +def claims_by_name(tree): + """Whether the control PRINTS a claim that it caught its cases by name. + + Only the strings the control prints count. A comment or a docstring that mentions the + phrase is prose about the control, not a claim the control makes to whoever reads its + output, and the difference matters: a control that documents the phrase must not be + reported as claiming it. + """ + for node in ast.walk(tree): + if not isinstance(node, ast.Call): + continue + if not (isinstance(node.func, ast.Name) and node.func.id == "print"): + continue + for argument in node.args: + for inner in ast.walk(argument): + if isinstance(inner, ast.Constant) and isinstance(inner.value, str): + if CLAIM_PHRASE in inner.value: + return True + return False + + def analyse(path): """Read one control and say how it decides a case.""" tree = ast.parse(path.read_text(encoding="utf-8")) @@ -459,6 +491,7 @@ def analyse(path): # name. Testing one of those against the file the control is editing is anchor work. judged = values - source tests = name_tests(tree, judged, bindings) + claims = claims_by_name(tree) single = [] membership = [] @@ -475,6 +508,7 @@ def analyse(path): "detail": ("the case's own expected name is compared for equality with one " "reported name: " + show(single[0]["operand"])), "members": sorted(judged), + "claims": claims, } if membership: return { @@ -482,12 +516,14 @@ def analyse(path): "how": BY_MEMBERSHIP, "detail": "", "members": sorted(judged), + "claims": claims, } return { "file": path.name, "how": BY_NO_NAME, "detail": describes_unmarked(tree, judged), "members": sorted(judged), + "claims": claims, } @@ -561,6 +597,17 @@ def check(rule, ok, detail=""): else "; ".join("%s: the control does not decide its cases by membership" % name for name in missing)) + # 4. The claim. A control that prints that it caught its cases by name has to judge them by + # membership, or the sentence it prints is false. This is the rule the previous run's + # finding was the same shape as, applied to every control rather than to two of them. + claimers = sorted(r["file"] for r in results if r["claims"]) + unfounded = [r for r in results if r["claims"] and r["how"] != BY_MEMBERSHIP] + check(RULE_CLAIM, not unfounded, + "%d control(s) claim it caught cases by name" % len(claimers) + if not unfounded + else "; ".join("%s: the control claims it caught cases by name but does not decide " + "them by membership" % r["file"] for r in unfounded)) + # The classification, which is the report this guard owes the reader. members = [r for r in results if r["how"] == BY_MEMBERSHIP] others = [r for r in results if r["how"] != BY_MEMBERSHIP] @@ -571,6 +618,9 @@ def check(rule, ok, detail=""): print(" not by membership (%d):" % len(others)) for record in others: print(" " + record["file"] + " " + record["detail"]) + print(" claiming to catch cases by name (%d):" % len(claimers)) + for name in claimers: + print(" " + name) if failures: print("") From f996ff8f6949db4c3a4ad99584bbe5409a2a7f91 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 08:57:12 +0800 Subject: [PATCH 62/95] The rule about a build that stopped watching now has an owner, and the comment that said more than it had established says what it establishes. A build writes the groups of its first batch with the height they are really at, so a group laid out at 1,500 px reads back 1,500 px, and the refresh that a resize drives cannot tell "laid out at 1,500 px" from "sized at 1,500 px": it reads the number it was given and skips the group. The height the group remembers is then still the old one, and the height it keeps after it is released is the height of a layout that is gone. What keeps the rule true is the observer's visit, and the observer is re-registered by the build -- a section from the previous table is not in the ledger content any more, so an observer left pointing at the old table never sees the new one. That rule had no owner, so a build that stopped watching the groups it had just rebuilt could not be caught by anything. It has one now: - a case in the ledger lifecycle guard: a resize and a rebuilt observer keep the layout the group has now, through the release that takes its rows away; - a control in the ledger lifecycle mutations: the build does not observe the groups it rebuilt, caught by that name. The case needed a harness that could hold a layout change at all: the elements the shipped code builds now read a height the case can change, the fake DOM answers the build's own query for the groups of its first batch that are holding their rows, and the observer's visit happens only if the build called the watcher -- which is what a browser does with an observer that was never pointed at the new table. And the comment in the client that said a group reading back the size it was given "is being sized rather than laid out" now says what is true: it may be, and it is not proof, because the first batch's own size writer puts the group's real height on it. What keeps the rule true is the observer's visit, and the rebuild is what re-registers the observer. No behaviour changed. --- .../mmc-trajectory/miniapp/client/index.html | 15 ++- .../tests/mutate-ledger-lifecycle.py | 15 +++ .../tests/verify-ledger-lifecycle.mjs | 114 +++++++++++++++++- 3 files changed, 138 insertions(+), 6 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html index da6cc14f..a32485d8 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/client/index.html +++ b/plugins/avatasia/mmc-trajectory/miniapp/client/index.html @@ -7477,8 +7477,9 @@

检视

* never visited again, keeps the height it was last really measured at. Off screen the * browser answers with the group's own `contain-intrinsic-size`, and that number is a * prediction — a prediction written down as a measurement is a guess the next rebuild - * reads back as a fact. So a group whose height is the size it was given is left alone, - * and this is the same limit the build path has. + * reads back as a fact. So a group that reads back the size it was given is left alone + * here, and the observer's visit is what measures a group that happens to be laid out at + * exactly that height — the same limit the build path has. */ function refreshLedgerMeasurements() { if (ledgerMeasureFrame !== null) return; @@ -7493,8 +7494,14 @@

检视

var group = ledgerContent.get(section); if (!group) continue; // The size the browser answers with while a group is off screen is the size - // written on it, so a group that reads back exactly the number it was given is - // being sized rather than laid out — and what it reads is a prediction. + // written on it — so a group that reads back exactly the number it was given MAY + // be being sized rather than laid out, and what it reads is then a prediction. + // It is not proof of that: the first batch's own size writer puts the group's REAL + // height on it, so a group that is laid out reads back its own height too. What + // keeps the rule true is the observer: a group the reader is on is measured again + // by the observer's visit, and the rebuild is what re-registers that observer. A + // group that is neither visited nor observed keeps the height it was measured at, + // and that is the same limit the build path has. var sizes = String(section.style.containIntrinsicSize || '').match(/(-?[0-9.]+)px/g); if (sizes) { var predicted = Math.round(parseFloat(sizes[sizes.length - 1])); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py index b10aea63..d89c6134 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -409,6 +409,21 @@ " }\n" " ledgerState.forEach(function (state, key) {\n")], "the window's first group keeps its rows when the clock cannot break the tie"), + + # The observer is registered by the build, and the build is what re-registers it after + # every rebuild: a section from the previous table is not in `ledgerContent` any more, so + # an observer left pointing at the old table never sees the new one. A build that stops + # watching leaves the reader's next visit unobserved, and the height the group keeps + # after that is the height of a layout that is gone — the group stands 500 px shorter + # than the content it is holding. + # + # The refresh that a resize drives cannot cover for it: the build sized the groups of its + # first batch by the height they are really at, so the number the group reads back is the + # number it was given, and the refresh skips it as a group that is sized rather than laid + # out. The observer's visit is the only thing that measures it again. + ("the build does not observe the groups it rebuilt", + [(" watchLedgerGroups();\n", "")], + "resize and a rebuilt observer preserve the current layout through release"), ] original = io.open(CLIENT, encoding="utf-8", newline="").read() diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs index ebfd7ad4..dd2455ec 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -186,6 +186,16 @@ function makeEl(tag) { if (selector === ':scope > section.turn-group') { return el.children.filter((c) => c && c.tagName === 'SECTION' && c.classes.has('turn-group')); } + // The build's own query for the groups of its first batch that are holding their rows: + // it is the query that decides which groups get the height they are really at written + // on them. A fake DOM that answers it with an empty list leaves every group sized by a + // number nobody measured — and the case about a layout that changed cannot be built at + // all, because the number on the group and the number it reads back are then the same + // one for the wrong reason. + if (selector === '.turn-group[data-rows="live"]') { + return el.children.filter((c) => c && c.classes.has('turn-group') + && c.getAttribute('data-rows') === 'live'); + } return []; }, getBoundingClientRect() { return { height: 240, width: 900, top: 0, bottom: 240 }; } @@ -230,7 +240,11 @@ function entriesOf(count, bands, prefix) { * time and a test that has to wait fifteen seconds to check it is a test nobody runs. */ function harness() { - const box = { clock: 1000, built: [], sized: [], frames: [], listeners: {} }; + // layoutHeight is the height the browser answers with for the elements the shipped code + // builds. It is a variable rather than a fixed number because a height is a measurement of + // a layout, and a case about a layout that changed has to be able to change it and then ask + // again — a DOM that always answers 240 px cannot hold a layout change at all. + const box = { clock: 1000, built: [], sized: [], frames: [], listeners: {}, layoutHeight: 240 }; // The window the shipped code talks to. It records the handlers it is given and the frames // it is asked for, so a case can deliver a real event and then run the frame the handler // deferred its work to. @@ -264,6 +278,13 @@ function harness() { el: (tag, className) => { const e = makeEl(tag); if (className) String(className).split(/\s+/).forEach((c) => { if (c) e.classes.add(c); }); + // The height the browser answers with for this element. The fake DOM's own answer is a + // fixed 240 px, which is the right answer for a fixture and the wrong one for a case + // about a layout that changed — so the elements the shipped code builds read the + // harness's height instead, and a case can change it. + e.getBoundingClientRect = () => ({ + height: box.layoutHeight, width: 900, top: 0, bottom: box.layoutHeight + }); box.built.push(e); return e; }, @@ -342,7 +363,12 @@ function buildHarness(options) { s.setLedgerProgress = () => {}; s.restoreRowAnchor = () => {}; s.flushDeferredPoll = () => {}; - s.watchLedgerGroups = () => {}; + // The build re-registers the observer after every rebuild: a section from the previous table + // is not in `ledgerContent` any more, so an observer left pointing at the old table never + // sees the new one. The harness records that the call happened, and the observer's visit + // happens only if it did — which is what a browser does with an observer that was never + // pointed at the new table. + s.watchLedgerGroups = () => { h.observed = true; }; s.syncStickyOffsets = () => {}; // The same window the harness made, so a case that delivers a resize event reaches the // handler the client registered for it. @@ -384,6 +410,24 @@ function sectionOf(h, index) { return h.sandbox.ledger.children[index]; } +/** + * The observer's visit, the way the browser delivers it. + * + * The observer is what measures a group that is really laid out, and the build is what points + * it at the new table. So the visit happens only if the build called the watcher — and when it + * did not, the observer is still pointing at the previous table, the group the reader is on is + * never measured again, and the height it keeps is the height of a layout that is gone. + * + * @param {object} h the harness + * @param {object} section the group the reader is on + * @returns {boolean} whether the observer was there to deliver the visit + */ +function observeGroups(h, section) { + if (!h.observed) return false; + h.sandbox.fillLedgerGroup(section); + return true; +} + /* ---------- 1. the rows of a group the reader is not on are not built ---------- */ { @@ -1119,6 +1163,72 @@ function measureGroup(h, key, entries, bands, height) { s.ledgerState.get('k2').height === 1500); } +/* ---------- 3l. a resize and a rebuilt observer keep the layout the group has now ---------- */ + +/* + * The height a released group keeps is the height it was measured at, and a measurement is of + * a layout that was there then. A layout can change without the content changing — the window + * is resized — and then the height the group remembers describes a layout that is no longer + * there, and the document under the reader comes up short by exactly the difference. + * + * The refresh that a resize drives cannot fix that on its own, and the reason is the shape of + * the numbers. The build sizes the groups of its first batch by the height they are really + * at, so a group laid out at 1,500 px reads back 1,500 px — and the refresh cannot tell + * "laid out at 1,500 px" from "sized at 1,500 px". It reads the number it was given and skips + * the group, so the height the group remembers is still the old one. + * + * What keeps the rule true is the observer: the group the reader is on is measured again by + * the observer's visit, and the observer is re-registered by the build — a section from the + * previous table is not in `ledgerContent` any more. A build that stops watching leaves the + * reader's next visit unobserved, and the height the group keeps after that is the height of + * a layout that is gone. + * + * The case: eight groups of three rows; the second one laid out at 1,000 px and visited; the + * layout changed to 1,500 px; a rebuild, which is what re-registers the observer; the resize + * delivered the way the browser delivers it, with the measurement deferred to the next frame; + * the observer's visit; and then the reader walks through the rest of the window, so that the + * cap takes the group away. The height the released group keeps has to be the height its + * layout has now — 1,500 px, and not the 1,000 px it was measured at. + */ +{ + const h = buildHarness(); + const s = h.sandbox; + const n = 8; + const groups = []; + for (let i = 0; i < n; i += 1) { + groups.push({ turn: { index: i + 1 }, entries: entriesOf(3, 0, 'g' + i) }); + } + h.clock = 1000; + h.layoutHeight = 1000; + rebuild(h, groups); + // The reader is on the second group, and it is laid out for real at 1,000 px. + s.fillLedgerGroup(sectionOf(h, 1)); + // The layout changes and nothing about the content does: the same three rows and the same + // number of bands, in a box that is 500 px taller. + h.layoutHeight = 1500; + // The poll brings a rebuild, and the rebuild is what re-registers the observer. + h.clock += 10; + rebuild(h, groups); + // The window is resized, and the measurement it drives cannot see the change: the build + // sized the groups of its first batch by the height they are really at, so the number the + // group reads back is the number it was given. + for (const handler of (h.listeners.resize || [])) handler(); + const frames = h.frames.splice(0, h.frames.length); + for (const frame of frames) frame(); + // The observer's visit: the group is laid out for real, and it is measured again. + observeGroups(h, sectionOf(h, 1)); + // The reader walks through the rest of the window, so the cap releases the group. + for (let i = 2; i < n; i += 1) { + h.clock += 10; + s.fillLedgerGroup(sectionOf(h, i)); + } + const released = sectionOf(h, 1); + // The rule, by name: the height the released group keeps is the height its layout has now. + // No numbers in the name — a control catches a rule by name. + assert('resize and a rebuilt observer preserve the current layout through release', + released.style.minHeight === '1500px'); +} + /* ---------- 4. the rows come back when the reader reaches the group ---------- */ { From 17e77a51f638a023c4030c2aa7e2700e8af121ae Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 09:52:45 +0800 Subject: [PATCH 63/95] tests: make the rebuilt-observer owner discriminate re-registration The guard's case for the rule about a rebuild re-registering the observer accepted a build that registered the observer once and never again. Its fixture recorded only that the build had called the watcher, and a sticky flag cannot tell a first registration from a re-registration -- so the rule whose name says "rebuilt" was proving less than its name claimed. The harness now runs the shipped watcher, with an observer that keeps the set of sections it is pointed at and forgets them when it is disconnected, which is what the browser's observer does. The visit goes through the shipped callback, and it is delivered only to a section the observer is actually watching. Two assertions own the two halves of the rule: a rebuilt observer watches the group the reader is on, not the one it replaced resize and a rebuilt observer preserve the current layout through release and two controls break them: the build does not observe the groups it rebuilt the build observes the groups it rebuilt only once The second break replaces the watcher's disconnect() with a return, so the observer keeps watching the previous table and the group the reader is on is never measured again. Against that build the guard was green before this change. No product file changed. The shipped watcher already re-registered the observer; it was the owner that could not see it. --- .../tests/mutate-ledger-lifecycle.py | 17 +++- .../tests/verify-ledger-lifecycle.mjs | 78 +++++++++++++++---- 2 files changed, 78 insertions(+), 17 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py index d89c6134..d7c46397 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-ledger-lifecycle.py @@ -423,7 +423,22 @@ # out. The observer's visit is the only thing that measures it again. ("the build does not observe the groups it rebuilt", [(" watchLedgerGroups();\n", "")], - "resize and a rebuilt observer preserve the current layout through release"), + ["resize and a rebuilt observer preserve the current layout through release", + "a rebuilt observer watches the group the reader is on, not the one it replaced"]), + + # The other half of the same rule, and the one a deletion cannot reach: an observer that + # is registered once and never pointed at the new table is still watching the sections of + # the previous table. The build's watcher disconnects the observer before it observes the + # new sections, and that disconnect is the whole of what makes the second build's + # observer the current one. Without it the observer's visit never reaches the group the + # reader is on — the section that replaced it is not one the observer knows — so the group + # keeps the height of the layout before the resize, and the document under the reader comes + # up short by exactly the difference. + ("the build observes the groups it rebuilt only once", + [(" } else {\n ledgerObserver.disconnect();\n }\n", + " } else {\n return;\n }\n")], + ["resize and a rebuilt observer preserve the current layout through release", + "a rebuilt observer watches the group the reader is on, not the one it replaced"]), ] original = io.open(CLIENT, encoding="utf-8", newline="").read() diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs index dd2455ec..e34f05bd 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-ledger-lifecycle.mjs @@ -95,7 +95,12 @@ const LIFECYCLE_FNS = [ // The build itself, because the rule this file has to prove is about what the build // writes. A group the reader has left must not be kept alive by the rebuild — and the // source of a function cannot say whether it stamps the reader's lease or not. - 'buildStep' + 'buildStep', + // The watcher itself, and not a stub of it. The rule below is about which sections the + // observer is pointed at after a rebuild, and only the shipped watcher can say that: a + // stub that records "the build called the watcher" cannot tell a re-registration from a + // first one, and a rule whose name says "rebuilt" has to be able to. + 'watchLedgerGroups' ]; const fnSrc = LIFECYCLE_FNS.map((name) => functionSource(name)); @@ -145,6 +150,11 @@ const resizeRegistration = refreshSrc // and not a function, and it is read by the shipped handler. const measureFrameDecl = html.match(/^[ \t]*var ledgerMeasureFrame = [^;]+;/m); +// The observer the shipped watcher registers. It is a declaration and not a function, so it +// is lifted on its own — and it has to be the same binding the watcher assigns, or the +// watcher would be reading one observer and the case another. +const observerDecl = html.match(/^[ \t]*var ledgerObserver = [^;]+;/m); + // The two limits are declarations, not functions, so they are lifted on their own. They are // numbers the shipped code reads, and the guard below uses the shipped values rather than // numbers written here — a copy of the cap in this file would keep passing after the shipped @@ -244,7 +254,8 @@ function harness() { // builds. It is a variable rather than a fixed number because a height is a measurement of // a layout, and a case about a layout that changed has to be able to change it and then ask // again — a DOM that always answers 240 px cannot hold a layout change at all. - const box = { clock: 1000, built: [], sized: [], frames: [], listeners: {}, layoutHeight: 240 }; + const box = { clock: 1000, built: [], sized: [], frames: [], listeners: {}, layoutHeight: 240, + observers: [] }; // The window the shipped code talks to. It records the handlers it is given and the frames // it is asked for, so a case can deliver a real event and then run the frame the handler // deferred its work to. @@ -269,6 +280,23 @@ function harness() { // The row heights the last build calibrated. The visit path prices the difference a // released group has with them, so the build writes them and the visit path reads them. ledgerCalibration: null, + // The observer the shipped watcher registers, and the set of sections it is pointed at. + // That set is the whole of what a rebuild has to replace: a section from the previous + // table is not in `ledgerContent` any more, so an observer left pointing at the old table + // never sees the new one. A browser's observer forgets its targets when it is + // disconnected, so this double forgets them too — which is what makes "the build + // re-registered the observer" and "the build registered it once" two different things. + IntersectionObserver: function (callback) { + const observer = { + callback, + targets: new Set(), + observe(target) { observer.targets.add(target); }, + unobserve(target) { observer.targets.delete(target); }, + disconnect() { observer.targets.clear(); } + }; + box.observers.push(observer); + return observer; + }, // The ledger the table is in. The shipped code reads it when it has to take the rows out // of a group the reader is no longer on. dom: { ledger: makeEl('div') }, @@ -315,8 +343,10 @@ function harness() { const registration = resizeRegistration ? resizeRegistration[0] + '\n' : ''; const frameDecl = measureFrameDecl ? measureFrameDecl[0] + '\n' : (refreshSrc ? 'var ledgerMeasureFrame = null;\n' : ''); + // The observer's own binding, so the watcher assigns the very observer the case reads. + const observerDeclSrc = observerDecl ? observerDecl[0] + '\n' : 'var ledgerObserver = null;\n'; vm.runInContext( - idleDecl[0] + '\n' + capDecl[0] + '\n' + batchDecl[0] + '\n' + frameDecl + idleDecl[0] + '\n' + capDecl[0] + '\n' + batchDecl[0] + '\n' + frameDecl + observerDeclSrc + fnSrc.join('\n') + '\n' + optionalSrc.join('\n') + '\n' + registration, sandbox ); @@ -363,12 +393,12 @@ function buildHarness(options) { s.setLedgerProgress = () => {}; s.restoreRowAnchor = () => {}; s.flushDeferredPoll = () => {}; - // The build re-registers the observer after every rebuild: a section from the previous table - // is not in `ledgerContent` any more, so an observer left pointing at the old table never - // sees the new one. The harness records that the call happened, and the observer's visit - // happens only if it did — which is what a browser does with an observer that was never - // pointed at the new table. - s.watchLedgerGroups = () => { h.observed = true; }; + // The watcher here is the shipped one, not a stub of it. It is the shipped watcher that + // disconnects the observer and points it at the new table, and the rule under test is + // about exactly that: a section from the previous table is not in `ledgerContent` any more, + // so an observer left pointing at the old table never sees the new one. A stub that + // recorded "the build called the watcher" could not tell the two apart — and a rule whose + // name says "rebuilt" has to be able to. s.syncStickyOffsets = () => {}; // The same window the harness made, so a case that delivers a resize event reaches the // handler the client registered for it. @@ -414,17 +444,24 @@ function sectionOf(h, index) { * The observer's visit, the way the browser delivers it. * * The observer is what measures a group that is really laid out, and the build is what points - * it at the new table. So the visit happens only if the build called the watcher — and when it - * did not, the observer is still pointing at the previous table, the group the reader is on is - * never measured again, and the height it keeps is the height of a layout that is gone. + * it at the new table. So the visit is delivered only if the section is one the observer is + * pointed at — and after a rebuild the sections are new ones, so an observer that was not + * re-registered is still pointing at the previous table: the group the reader is on is never + * measured again, and the height it keeps is the height of a layout that is gone. + * + * The visit goes through the observer's own callback, and that callback is the shipped one: it + * is what fills the group, and a case that filled the group by hand would be testing the + * harness rather than the client. * * @param {object} h the harness * @param {object} section the group the reader is on * @returns {boolean} whether the observer was there to deliver the visit */ function observeGroups(h, section) { - if (!h.observed) return false; - h.sandbox.fillLedgerGroup(section); + const observer = h.sandbox.ledgerObserver; + if (!observer) return false; + if (!observer.targets.has(section)) return false; + observer.callback([{ target: section, isIntersecting: true }]); return true; } @@ -1202,13 +1239,18 @@ function measureGroup(h, key, entries, bands, height) { h.layoutHeight = 1000; rebuild(h, groups); // The reader is on the second group, and it is laid out for real at 1,000 px. - s.fillLedgerGroup(sectionOf(h, 1)); + const firstSection = sectionOf(h, 1); + s.fillLedgerGroup(firstSection); // The layout changes and nothing about the content does: the same three rows and the same // number of bands, in a box that is 500 px taller. h.layoutHeight = 1500; // The poll brings a rebuild, and the rebuild is what re-registers the observer. h.clock += 10; rebuild(h, groups); + // The table is a new one, so the group the reader is on is a new section — and the + // observer has to be pointed at it. This is the half the rule is named after: an observer + // that was registered once and never again is still pointing at the section it replaced. + const rebuiltSection = sectionOf(h, 1); // The window is resized, and the measurement it drives cannot see the change: the build // sized the groups of its first batch by the height they are really at, so the number the // group reads back is the number it was given. @@ -1216,7 +1258,11 @@ function measureGroup(h, key, entries, bands, height) { const frames = h.frames.splice(0, h.frames.length); for (const frame of frames) frame(); // The observer's visit: the group is laid out for real, and it is measured again. - observeGroups(h, sectionOf(h, 1)); + const visited = observeGroups(h, rebuiltSection); + const observer = h.sandbox.ledgerObserver; + assert('a rebuilt observer watches the group the reader is on, not the one it replaced', + visited === true && !!observer && observer.targets.has(rebuiltSection) + && !observer.targets.has(firstSection)); // The reader walks through the rest of the window, so the cap releases the group. for (let i = 2; i < n; i += 1) { h.clock += 10; From 1ca4413a827453fe423edb21e12b7ecb6e6ed4de Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 13:27:24 +0800 Subject: [PATCH 64/95] The service keeps the rendered body as bytes instead of as one string. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A large session's body used to be built by a single JSON.stringify call, so the service held all of it as one JavaScript string at once — and because a transcript contains characters outside Latin-1, V8 keeps such a string at two bytes per character. For the session measured while making this change, that string was 99.9 MB of heap for a 53.6 MB body. The body is now serialized in pieces: the array of turns is cut at element boundaries, each piece is serialized on its own and turned into bytes immediately, and the string is let go before the next piece is built. The service keeps the bytes. The bytes on the wire do not change. The pieces concatenate back into exactly what JSON.stringify would have produced, and the declared length is the sum of the pieces rather than the length of one string. The route serves the same body and the same validator, and a revalidated request is still answered with 304. Measured with one script, one machine, one session, and the code before this change beside the code after it: the process's high-water mark fell from 1,548.0 MB to 1,396.8 MB, the heap retained after the body had been read once and dropped fell from 302.7 MB to 202.4 MB, and the share of that heap held by the rendered body fell from 106.3 MB to 5.9 MB. The floor — the size of the process with both caches emptied and the machine idle — did not move (1,070.1 MB before, 1,069.7 MB after). That floor is memory the request touched and the platform did not give back; this change does not lower it, and nothing in the package asserts a number. One assertion in the record-detail guard named the call the route used to serialize the payload with. It now names the call the route uses, and the order it pins — strip before serializing — is unchanged. --- .../mmc-trajectory/miniapp/node/server.mjs | 156 +++++++++++++++++- .../tests/verify-record-detail.mjs | 2 +- 2 files changed, 153 insertions(+), 5 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index db59acd8..e4df9902 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -90,6 +90,22 @@ const LIST_CACHE_TTL_MS = 3000; const SESSION_CACHE_TTL_MS = 5000; const SESSION_CACHE_ENTRIES = 80; const TRAJECTORY_CACHE_ENTRIES = 4; +/** + * How many bytes one piece of the trajectory body may hold. + * + * The body of a large session is tens of megabytes, and it used to be built by ONE + * `JSON.stringify` call — so the process had to hold all of it as one string at once, and V8's + * string builder leaves its intermediate copies behind while it doubles its buffer. Measured on + * this machine, serializing a 52.4M-character body cost 745 MB of heap on top of the payload it + * came from, and the pages that peak left behind stayed resident long after the body itself was + * gone: the process kept 965 MB with an 8.4 MB live heap. + * + * A megabyte is a compromise. Small enough that the builder's doubling is noise next to the + * payload it is serializing; large enough that a 53 MB body is fifty-odd writes rather than + * fifty thousand. A body piece that cannot be made smaller than this is one element bigger than + * the budget, and the budget does not pretend otherwise: it bounds that element instead. + */ +const TRAJECTORY_BODY_CHUNK_BYTES = 1024 * 1024; /** * The most hits one search response carries. The count is still reported in full — a needle that * matches ten thousand records is a real answer — so this is a transport bound and not a claim @@ -3772,13 +3788,132 @@ export async function loadParsedRows(cache, filePath, limit, options) { return value; } +/** + * One piece of a JSON array, serialized on its own. + * + * The array is cut at element boundaries, and the pieces concatenate back into exactly the + * string `JSON.stringify` would have produced for the whole array — the opening bracket opens + * the first piece, the closing bracket closes the last one, and every element after the first + * carries its comma with it, so a piece boundary never lands between an element and its comma. + * + * An element that does not fit the budget on its own becomes a piece of its own. That is the one + * thing the budget cannot bound, and it is bounded by the element instead of by the budget. + * + * @param {Array} items + * @param {number} budgetBytes + * @returns {Array} + */ +function jsonArrayPieces(items, budgetBytes) { + const pieces = []; + let current = '['; + let currentBytes = 1; + let first = true; + for (const item of items) { + const encoded = JSON.stringify(item); + // JSON.stringify writes an undefined element as null, and so does this. + const text = encoded === undefined ? 'null' : encoded; + const textBytes = Buffer.byteLength(text, 'utf8'); + // The comma is counted but not concatenated into the piece: `',' + text` would be a + // cons string, and measuring its length would flatten it into a second copy of the + // element. On a body of fifty thousand records that is fifty thousand copies. + const pieceBytes = textBytes + (first ? 0 : 1); + if (currentBytes > 1 && currentBytes + pieceBytes > budgetBytes) { + pieces.push(Buffer.from(current, 'utf8')); + current = ''; + currentBytes = 0; + } + if (!first) { + current += ','; + currentBytes += 1; + } + current += text; + currentBytes += textBytes; + first = false; + } + current += ']'; + pieces.push(Buffer.from(current, 'utf8')); + return pieces; +} + +/** + * The trajectory body, serialized in pieces small enough that no single string ever holds it. + * + * THE BYTES ARE THE SAME BYTES. Concatenating the pieces gives exactly what + * `JSON.stringify(payload)` would have produced — same key order, same escaping, same numbers — + * so the page, the search, `/raw` and `?since=` cannot tell the difference. A guard pins that + * byte for byte, because a body that is *nearly* the same is a body the page cannot parse. + * + * WHAT CHANGES IS HOW THEY ARE PRODUCED. The payload's big array is cut into pieces at element + * boundaries, each piece is serialized on its own and turned into bytes immediately, and the + * string is let go before the next piece is built. So the process never holds the whole body as + * one string, and never holds the builder's intermediate copies either. + * + * WHY IT IS WORTH THE ROUTE'S ATTENTION. Measured on this machine, one cold read of a + * 57,192-record session left the process at 1094 MB resident with a 299 MB live heap, and the + * peak inside the route was 1341 MB: the payload was 361 MB, and the single `JSON.stringify` + * that followed it added 745 MB on top. Clearing every cache afterwards freed the live heap + * down to 8.4 MB and the process still held 965 MB — the pages that peak left behind. The + * floor follows the peak, not the cache, so the peak is the thing to change. + * + * @param {Record} payload + * @param {number} [budgetBytes] + * @returns {Array} + */ +export function serializeTrajectoryBody(payload, budgetBytes = TRAJECTORY_BODY_CHUNK_BYTES) { + const pieces = []; + let current = ''; + let currentBytes = 0; + let needsComma = false; + + const flush = () => { + if (currentBytes === 0) return; + pieces.push(Buffer.from(current, 'utf8')); + current = ''; + currentBytes = 0; + }; + const append = (text) => { + current += text; + currentBytes += Buffer.byteLength(text, 'utf8'); + }; + + append('{'); + for (const key of Object.keys(payload)) { + const value = payload[key]; + // What JSON.stringify leaves out, this leaves out: a key whose value is undefined, a + // function or a symbol is not in the JSON at all. + if (value === undefined || typeof value === 'function' || typeof value === 'symbol') continue; + if (Array.isArray(value)) { + if (needsComma) append(','); + append(JSON.stringify(key) + ':'); + // The array goes out in pieces, so the string it would have been never exists. + flush(); + for (const piece of jsonArrayPieces(value, budgetBytes)) pieces.push(piece); + needsComma = true; + continue; + } + const encoded = JSON.stringify(value); + // A value whose toJSON() returns undefined is left out by JSON.stringify as well. + if (encoded === undefined) continue; + if (needsComma) append(','); + append(JSON.stringify(key) + ':' + encoded); + needsComma = true; + } + append('}'); + flush(); + return pieces; +} + /** * Sends an already-serialized JSON body. Unlike {@link sendJson} it supports revalidation: * `no-store` would suppress the 304 path, so revalidated responses advertise `no-cache` * (storable, but must be revalidated) and carry the tag the page echoes back. + * + * The body is either one string or a list of byte pieces. The list is what the trajectory route + * sends now: the bytes are the same bytes, but they were never held as one string, so the + * declared length is the sum of the pieces rather than the length of a string. * @param {any} response * @param {number} status - * @param {string} body + * @param {string | Buffer | Array} body * @param {{ etag?: string, revalidate?: boolean }} [options] */ export function sendJsonBody(response, status, body, options) { @@ -3796,10 +3931,20 @@ export function sendJsonBody(response, status, body, options) { } return; } - headers['content-length'] = Buffer.byteLength(body, 'utf8'); + const pieces = Array.isArray(body) ? body : [body]; + let length = 0; + for (const piece of pieces) { + length += Buffer.isBuffer(piece) ? piece.length : Buffer.byteLength(piece, 'utf8'); + } + headers['content-length'] = length; try { response.writeHead(status, headers); - response.end(body); + if (pieces.length === 1 && typeof pieces[0] === 'string') { + response.end(pieces[0]); + return; + } + for (const piece of pieces) response.write(piece); + response.end(); } catch { // The client polls aggressively and aborts requests; a closed socket is not an error here. } @@ -4657,7 +4802,10 @@ export async function start(context) { let body; try { - body = JSON.stringify(payload); + // In pieces, not as one string. The bytes are the same bytes — see serializeTrajectoryBody + // for why that is a contract and not an accident — but the process never holds the whole + // body as a string, and the peak this route used to leave behind does not come back. + body = serializeTrajectoryBody(payload); } catch { sendJson(response, 500, { error: 'trajectory_unserializable', message: '会话数据暂时无法读取' }); return; diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-record-detail.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-record-detail.mjs index 4d70fb1b..79fb5d4f 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-record-detail.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-record-detail.mjs @@ -382,7 +382,7 @@ const routeEnd = route.indexOf('\n /**'); const trajectory = route.slice(0, routeEnd === -1 ? undefined : routeEnd); assert('the trajectory route strips before serializing', trajectory.indexOf('slimPayloadForWire(payload)') !== -1 - && trajectory.indexOf('slimPayloadForWire(payload)') < trajectory.indexOf('JSON.stringify(payload)')); + && trajectory.indexOf('slimPayloadForWire(payload)') < trajectory.indexOf('serializeTrajectoryBody(payload)')); check('the strip happens once', trajectory.split('slimPayloadForWire(payload)').length - 1, 1); assert('the delta is built from the stripped payload, so a poll carries no more bulk than the first load', trajectory.indexOf('buildDeltaPayload') > trajectory.indexOf('slimPayloadForWire(payload)')); From 2cbad75b2e038e6747f4a684b1bfc8ef0a012e80 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 13:27:25 +0800 Subject: [PATCH 65/95] Five ways to undo the new body serialization, and an owner for each of them. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A guard owns the rules the new body serialization introduces: that the pieces are the bytes JSON.stringify would have produced, that the declared length is the sum of the pieces, that the body is cut into more pieces as it grows so that no single string holds it, that the pieces are bytes and not strings, and that the route uses the pieces and the cache keeps them. It also asks a real service for a real body and checks the bytes on the wire on the first read and on the warm read that comes out of the cache. A mutation control breaks each of those rules in the shipped source and requires the guard to report it by name — seven cases, because the last rule can be broken in two ways. The case that drops the comma between two pieces is the one worth naming: it is invisible in the shape of any single piece and only shows up as a body the page cannot parse. --- .../tests/mutate-trajectory-body.py | 223 +++++++++++++ .../tests/verify-trajectory-body.mjs | 303 ++++++++++++++++++ 2 files changed, 526 insertions(+) create mode 100644 plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py create mode 100644 plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py new file mode 100644 index 00000000..d34c98bb --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py @@ -0,0 +1,223 @@ +"""Break the trajectory body serializer, one rule at a time, and require the guard to catch it +by name. + +The body of a large session used to be built by ONE `JSON.stringify(payload)` call. That cost +745 MB of heap on top of the payload on this machine, and the pages that peak left behind kept +the process at 965 MB resident with an 8.4 MB live heap. The body is now serialized in pieces and +kept as bytes, and each of the seven ways back below is a rule the shipped code has to keep. + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the rule it names is among the names the guard reported. A control that takes the FIRST +reported name and calls it the rule would report a case as caught whenever the guard went red for +any reason at all — a second assertion in the same file, a crash in an unrelated check, a stale +expectation — so the number it prints would be a number that means nothing. + +Anchors are multi-line and server.mjs is LF, so they are written with bare \\n and the file is +read and written as bytes, with its own line endings put back exactly. +""" +import json +import re +import subprocess +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parent +SERVER = ROOT / "miniapp" / "node" / "server.mjs" +PROBE = HERE / "verify-trajectory-body.mjs" + +# Each entry is (title, old, new, expect). +MUTATIONS = [ + # The rule the whole change rests on: the pieces have to be the bytes JSON.stringify would + # have produced. Dropping the comma between two elements does not change the shape of any + # one element, so nothing else about the body notices — but the page cannot parse it, and a + # body it cannot parse is a page that shows nothing. + ( + "the pieces lose the comma between two elements", + " if (!first) {", + " if (first) {", + "the chunked body is the same bytes JSON.stringify would have produced", + ), + + # The length the socket is told about. Taking the first piece's length is what a serializer + # that forgot to sum looks like, and the page waits for a body that ended early. + ( + "the declared length is the first piece's length", + " headers['content-length'] = length;", + " headers['content-length'] = pieces[0].length;", + "the declared length is the sum of the pieces, not the length of one string", + ), + + # The budget. Enlarging it until the body fits in one piece produces exactly the right bytes + # and exactly the peak this change was made to remove. + ( + "the budget grows until the whole body fits in one piece", + "const TRAJECTORY_BODY_CHUNK_BYTES = 1024 * 1024;", + "const TRAJECTORY_BODY_CHUNK_BYTES = 64 * 1024 * 1024;", + "the body is cut into more pieces as it grows, so no single string holds it", + ), + + # The pieces as strings. Every character outside Latin-1 makes V8 hold a string as two bytes + # per character, which is the whole reason the pieces are bytes: a Chinese transcript doubles. + ( + "the pieces are handed back as strings", + " pieces.push(Buffer.from(current, 'utf8'));", + " pieces.push(current);", + "the pieces are bytes, so the body is not held as a string", + ), + + # The budget stops bounding a piece. The bytes are still right — the body is one piece that + # holds everything, and the peak is back. + ( + "the budget stops bounding a piece", + " if (currentBytes > 1 && currentBytes + pieceBytes > budgetBytes) {", + " if (false) {", + "no piece is bigger than the budget unless it holds a single element", + ), + + # The route goes back to one string. The helper stays correct, so every unit assertion above + # still passes, and the memory is exactly as bad as it was before. + ( + "the route serializes the body as one string again", + " body = serializeTrajectoryBody(payload);", + " body = JSON.stringify(payload);", + "the route serializes the body in pieces", + ), + + # The cache is handed a string instead of the pieces. The body on the wire is still correct, + # because a string of the same bytes is the same bytes — and the process is holding the whole + # body as one two-byte string again, which is the half of the change that is about memory. + ( + "the cache is handed a string instead of the pieces", + " body = serializeTrajectoryBody(payload);", + " body = serializeTrajectoryBody(payload).map((piece) => piece.toString('utf8'));", + "the cache keeps the pieces the serializer produced, not a string of its own", + ), +] + + +def read_exact(path): + """Normalise to LF for matching, and report the file's own newline so it can be put back.""" + data = path.read_bytes() + text = data.decode("utf-8").replace("\r\n", "\n") + return text, ("\r\n" if b"\r\n" in data else "\n") + + +def write_exact(path, text, newline): + if newline == "\r\n": + text = text.replace("\n", "\r\n") + path.write_bytes(text.encode("utf-8")) + + +def run_probe(): + proc = subprocess.run( + ["node", str(PROBE)], + cwd=str(ROOT), + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + ) + return proc.returncode, (proc.stdout or "") + (proc.stderr or "") + + +def rule_name(text): + """The name of a rule the guard reported, without the evidence a comparison carries. + + The guard reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules the guard named as broken. + + Two formats: ` - ` for the failure list and `FAIL `. A case is only caught when + the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names + + +def main(): + code, output = run_probe() + if code != 0: + print("baseline guard does not pass; fix that first") + print(output) + return 1 + print("baseline clean") + + original = SERVER.read_bytes() + missed = [] + unnamed = [] + for label, old, new, expect in MUTATIONS: + text, newline = read_exact(SERVER) + if text.count(old) != 1: + print(f"SKIP {label}: anchor found {text.count(old)} times") + missed.append(label) + continue + if old == new: + print(f"SKIP {label}: the mutation changes nothing") + missed.append(label) + continue + write_exact(SERVER, text.replace(old, new, 1), newline) + try: + code, output = run_probe() + finally: + SERVER.write_bytes(original) + if code == 0: + print(f"MISSED {label}: the guard still passed") + missed.append(label) + continue + names = reported_rules(output) + if not names: + print(f"UNNAMED {label}: the guard failed without naming a rule") + unnamed.append(label) + continue + # The case names the rule it is about, so the guard has to name THAT rule and not some + # other one that happened to go red in the same run: a guard that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the guard named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") + + if SERVER.read_bytes() != original: + print(f"NOT RESTORED {SERVER}") + return 1 + print("file restored byte-for-byte") + + if missed or unnamed: + if missed: + print(f"\n{len(missed)} mutation(s) got through") + if unnamed: + print(f"\n{len(unnamed)} mutation(s) failed without naming a rule") + return 1 + print(f"\nall {len(MUTATIONS)} mutations caught, each by name") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs new file mode 100644 index 00000000..77667941 --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs @@ -0,0 +1,303 @@ +/** + * The trajectory body is built in pieces, and the pieces are the same bytes. + * + * WHAT CHANGED. The body used to be produced by ONE `JSON.stringify(payload)` call, so the + * process had to hold all of it as one string at once, and V8's string builder leaves its + * intermediate copies behind while it doubles its buffer. Measured on this machine, serializing + * a 52.4M-character body cost 745 MB of heap on top of the payload it came from; the peak that + * left behind kept the process at 965 MB resident with an 8.4 MB live heap, long after the body + * itself was gone. The body is now serialized in pieces and kept as bytes. + * + * WHY THIS FILE IS A GUARD AND NOT A COMMENT. Five things about that change can each be undone + * without anything else noticing, and each of them has a name here: + * + * 1. the pieces must be the same bytes `JSON.stringify` would have produced. A body that is + * *nearly* the same is a body the page cannot parse, and the page is the only consumer that + * would notice — the server would answer 200 the whole time. + * 2. the pieces must be BYTES. Strings are what the change was for: a piece kept as a string + * is a piece V8 holds as a two-byte string whenever it holds one character outside Latin-1, + * which is every Chinese transcript on this machine. + * 3. there must be MORE THAN ONE piece, and no piece may be bigger than the budget unless it + * holds a single element. A serializer that stopped splitting would still produce the right + * bytes and would bring the whole peak back. + * 4. the declared length must be the sum of the pieces. A length computed from one piece, or + * from the number of characters rather than the number of bytes, is a length the socket + * disagrees with, and the page hangs waiting for a body that ended early. + * 5. the route must USE the pieces, and the cache must KEEP them. A helper that is correct and + * a route that does not call it are both green, and the memory is exactly as bad as before. + * + * The live section at the end is the wiring: it asks a real service for a real body and checks + * that the bytes on the wire are the bytes the serializer produced, on the first read and on the + * warm read that comes out of the cache. + * + * `mutate-trajectory-body.py` breaks each of these in the shipped source and requires this guard + * to catch it by name. + */ +import { mkdtempSync, readFileSync } from 'node:fs'; +import { mkdir, writeFile, utimes } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { serializeTrajectoryBody, sendJsonBody, start } from '../miniapp/node/server.mjs'; + +const SERVER = readFileSync(new URL('../miniapp/node/server.mjs', import.meta.url), 'utf8'); +/** The package root, so the route serves the same page the reader gets. */ +const PLUGIN_ROOT = fileURLToPath(new URL('..', import.meta.url)); + +// `node:sqlite` is still flagged experimental, and the flag goes to stderr the moment the module +// is first loaded. The runner reads `stdout + stderr` and prints the LAST non-empty line, so the +// warning would land where this guard's own result line should be. +const emitWarning = process.emitWarning; +process.emitWarning = (warning, ...rest) => ( + String(warning).startsWith('SQLite is an experimental feature') + ? undefined + : emitWarning.call(process, warning, ...rest)); + +let pass = 0; +const failures = []; +function check(name, actual, expected) { + if (JSON.stringify(actual) === JSON.stringify(expected)) pass += 1; + else failures.push(`${name}: got ${JSON.stringify(actual)}, want ${JSON.stringify(expected)}`); +} +function assert(name, cond) { + if (cond) pass += 1; + else failures.push(name); +} + +/** + * The pieces as bytes, whatever they are. + * + * A piece that came back as a string is read as its own bytes rather than crashing the report: + * the assertion that owns "the pieces are bytes" has to be able to FAIL, and a crash takes the + * names of the other assertions with it. + */ +const asBytes = (pieces) => Buffer.concat(pieces.map((piece) => ( + Buffer.isBuffer(piece) ? piece : Buffer.from(piece, 'utf8') +))); + +/* ===================================================================== + * 1 — the bytes are the same bytes + * ===================================================================== */ + +const SHAPES = [ + { name: 'a plain object', payload: { a: 1, b: 'two', c: true, d: null } }, + { + name: 'the shape the route builds', + payload: { + session: { id: 'sess', createdAtMs: 1791600000000, lastActiveAt: 1791600000000 }, + stats: { messages: 2, turns: 1 }, + turns: [ + { id: 't1', index: 1, records: [{ index: 0, id: 'm1', kind: 'user' }, { index: 1, id: 'm2', kind: 'assistant' }] }, + { id: 't2', index: 2, records: [] }, + ], + }, + }, + { name: 'text outside Latin-1', payload: { turns: [{ summary: '会话轨迹', text: '你好，世界' }] } }, + { + name: 'characters that have to be escaped', + payload: { turns: [{ s: 'quote " backslash \\ newline \n tab \t control ' }] }, + }, + { name: 'an empty array', payload: { turns: [] } }, + { name: 'keys that have to be escaped', payload: { 'a"b': 1, 'c\\d': 2, 'e\nf': 3 } }, + { name: 'values JSON.stringify leaves out', payload: { a: 1, b: undefined, c: () => 1, d: 2 } }, + { name: 'numbers of every shape', payload: { turns: [{ n: [0, -0, 1.5, 1e21, 1e-7, 12345678901234567890] }] } }, + { name: 'nested objects and arrays', payload: { turns: [{ r: { deep: { deeper: [1, [2, [3]]] } } }] } }, + { name: 'an undefined element inside the array', payload: { turns: [undefined, 1] } }, +]; + +const mismatched = []; +for (const shape of SHAPES) { + const expected = JSON.stringify(shape.payload); + const pieces = serializeTrajectoryBody(shape.payload); + const actual = asBytes(pieces).toString('utf8'); + check(`the chunked body is the same bytes JSON.stringify would have produced (${shape.name})`, + actual, expected); + if (actual !== expected) mismatched.push(shape.name); +} +assert('the chunked body is the same bytes JSON.stringify would have produced', mismatched.length === 0); + +/* ===================================================================== + * 2 — the pieces are bytes, and there is more than one of them + * ===================================================================== */ + +const turnsFor = (count) => Array.from({ length: count }, (unused, index) => ({ + id: 't' + index, + index, + records: [{ index, id: 'm' + index, turnId: 't' + index, kind: 'assistant', role: 'user', timestamp: 1791600000000 + index }], +})); +const SMALL = { turns: turnsFor(10) }; +const LARGE = { turns: turnsFor(20000) }; +const smallPieces = serializeTrajectoryBody(SMALL); +const largePieces = serializeTrajectoryBody(LARGE); + +// The number of pieces a body is cut into has to GROW with the body. That is the whole +// invariant: a serializer that stops splitting answers a bigger session with the same number +// of pieces, and the piece that holds everything is the peak this change was made to remove. +// The count is not compared with a number — the body's own small case is the yardstick. +assert('the body is cut into more pieces as it grows, so no single string holds it', + largePieces.length > smallPieces.length); +assert('the pieces are bytes, so the body is not held as a string', + largePieces.every((piece) => Buffer.isBuffer(piece))); +check('the pieces add up to the body', + asBytes(largePieces).length, Buffer.byteLength(JSON.stringify(LARGE), 'utf8')); + +/* ===================================================================== + * 3 — the budget bounds a piece + * ===================================================================== */ + +const BUDGET = 4096; +const budgeted = serializeTrajectoryBody({ + turns: Array.from({ length: 50 }, (unused, index) => ({ id: 'm' + index, text: 'x'.repeat(200) })), +}, BUDGET); +assert('no piece is bigger than the budget unless it holds a single element', + budgeted.length > 1 && budgeted.every((piece) => piece.length <= BUDGET)); + +// The one thing the budget cannot bound: an element bigger than the budget on its own. It +// becomes a piece of its own rather than being cut in half, because half a record is not a +// record. +const HUGE = { turns: [{ id: 'a', text: 'x'.repeat(BUDGET * 2) }] }; +const huge = serializeTrajectoryBody(HUGE, BUDGET); +assert('an element bigger than the budget becomes a piece of its own', + huge.length === 3 && huge[1].length > BUDGET); +check('and it is still the same bytes', asBytes(huge).toString('utf8'), JSON.stringify(HUGE)); + +/* ===================================================================== + * 4 — the declared length is the sum of the pieces + * ===================================================================== */ + +const fakeResponse = () => ({ + writableEnded: false, + destroyed: false, + headers: null, + written: [], + writeHead(status, headers) { this.headers = headers; }, + write(chunk) { this.written.push(chunk); return true; }, + end(chunk) { if (chunk !== undefined) this.written.push(chunk); }, +}); + +const chunked = fakeResponse(); +sendJsonBody(chunked, 200, largePieces, {}); +const declared = Number(chunked.headers['content-length']); +const sent = asBytes(chunked.written).length; +assert('the declared length is the sum of the pieces, not the length of one string', + declared === sent && declared === asBytes(largePieces).length); +assert('and the pieces are what reached the socket', sent === asBytes(largePieces).length); + +const asString = fakeResponse(); +sendJsonBody(asString, 200, JSON.stringify(LARGE), {}); +assert('a body that is one string still declares its own length', + Number(asString.headers['content-length']) === Buffer.byteLength(JSON.stringify(LARGE), 'utf8')); + +/* ===================================================================== + * 5 — the route uses the pieces, and the cache keeps them + * ===================================================================== */ + +const routeStart = SERVER.indexOf('const handleTrajectory = async'); +const route = SERVER.slice(routeStart, SERVER.indexOf('\n /**', routeStart)); +// How many times the route assigns the body it is about to keep. One, and it is the +// serializer's own result: a second assignment is a copy, and a copy of a body this size is +// the thing that has to stop happening. +const bodyAssignments = route.match(/^\s*body = .*$/gm) || []; +assert('the route serializes the body in pieces', route.includes('body = serializeTrajectoryBody(payload);')); +assert('the route does not serialize the whole payload as one string', !route.includes('JSON.stringify(payload)')); +assert('the cache keeps the pieces the serializer produced, not a string of its own', + bodyAssignments.length === 1 + && bodyAssignments[0].trim() === 'body = serializeTrajectoryBody(payload);' + && /trajectoryCache\.set\([^;]*\{ at: Date\.now\(\), etag, body, raws/.test(route)); + +/* ===================================================================== + * 6 — the live route: the bytes on the wire are the bytes the serializer produced + * ===================================================================== */ + +const home = mkdtempSync(join(tmpdir(), 'mmc-trajectory-body-')); +const SID = 'mvs_body_pieces'; +const sessionDir = join(home, 'v2', 'sessions', '2026', '10', '22', '00-00-00-session_' + + Buffer.from(SID, 'utf8').toString('base64url')); +await mkdir(sessionDir, { recursive: true }); +const rows = []; +// Enough records that the body on the wire is bigger than one piece, so the live section +// exercises the path this change is about and not only the small-body case. +for (let index = 0; index < 12000; index += 1) { + rows.push(JSON.stringify({ + message_id: 'm' + index, + turn_id: 't' + (index >> 3), + message: { role: 'user', content: 'PIECE ' + index, timestamp: 1791600000000 + index }, + })); +} +await writeFile(join(sessionDir, 'messages.jsonl'), rows.join('\n') + '\n', 'utf8'); +await writeFile(join(sessionDir, 'manifest.json'), JSON.stringify({ sessionId: SID }), 'utf8'); +const stamp = new Date(1791600000000); +await utimes(join(sessionDir, 'messages.jsonl'), stamp, stamp); + +const previousRoot = process.env.MMC_TRAJECTORY_ROOT; +process.env.MMC_TRAJECTORY_ROOT = home; +const controller = new AbortController(); +const noop = () => {}; +let instance = null; +try { + let port = 0; + for (let attempt = 0; attempt < 12 && port === 0; attempt += 1) { + const candidate = 20900 + Math.floor(Math.random() * 900) + attempt; + try { + instance = await start({ + pluginId: 'mmc-trajectory', + pluginRoot: PLUGIN_ROOT, + dataDir: home, + generationId: 'guard', + listen: { host: '127.0.0.1', port: candidate }, + signal: controller.signal, + logger: { debug: noop, info: noop, warn: noop, error: noop }, + }); + port = candidate; + } catch (error) { + // The port was taken. Another one is tried; a guard that gave up here would be a guard + // that sometimes checks nothing. + } + } + assert('the service came up on a port of its own', port !== 0); + if (port !== 0) { + const base = 'http://127.0.0.1:' + port; + const cold = await fetch(base + '/api/trajectory?session=' + encodeURIComponent(SID)); + const coldBytes = Buffer.from(await cold.arrayBuffer()); + const coldLength = Number(cold.headers.get('content-length')); + assert('the cold body declares the length it really has', coldLength === coldBytes.length); + // A body that cannot be parsed is read as no body at all rather than as a thrown error: the + // throw would take this report with it, and the names above are what a mutation is caught by. + let parsed = null; + try { + parsed = JSON.parse(coldBytes.toString('utf8')); + } catch (error) { + parsed = null; + } + assert('and it is a body the page can parse', + parsed !== null && Array.isArray(parsed.turns) && parsed.turns.length > 0); + assert('and it holds the records the session has', + parsed !== null && Array.isArray(parsed.turns) + ? parsed.turns.flatMap((turn) => turn.records || []).length === 12000 + : false); + + // The warm read is the one that comes out of the cache, and the cache keeps PIECES now. + const warm = await fetch(base + '/api/trajectory?session=' + encodeURIComponent(SID)); + const warmBytes = Buffer.from(await warm.arrayBuffer()); + assert('the warm body declares the length it really has', + Number(warm.headers.get('content-length')) === warmBytes.length); + assert('the warm body is the same bytes as the cold one', warmBytes.equals(coldBytes)); + + // And the validator still answers, which is the one path that sends no body at all. + const revalidated = await fetch(base + '/api/trajectory?session=' + encodeURIComponent(SID), { + headers: { 'if-none-match': cold.headers.get('etag') || '' }, + }); + assert('a revalidated request is still answered with 304', revalidated.status === 304); + } +} finally { + if (instance) await instance.dispose(); + if (previousRoot === undefined) delete process.env.MMC_TRAJECTORY_ROOT; + else process.env.MMC_TRAJECTORY_ROOT = previousRoot; +} + +console.log('[trajectory-body] ' + pass + ' passed, ' + failures.length + ' failed'); +if (failures.length) { + console.log('FAILURES:\n - ' + failures.join('\n - ')); + process.exit(1); +} +console.log('[trajectory-body] OK'); From 01c5dcf2ba50e0d4595e595da9c1bedde14857c5 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 14:49:22 +0800 Subject: [PATCH 66/95] Give the parsed-rows cache a time limit, and say what its budget counts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The parsed-rows cache held the rows of a conversation's sealed generations until the runtime stopped. A reader who closed the tab stopped asking for them, and the service kept them anyway — on a conversation of 79 sealed generations that is about 187 MB of live heap held for nothing. The rows are now released ten minutes after the last request that wanted them. The release does not need a request to happen first: a sweep runs on its own, because a sweep that only ran on a request would release the rows exactly when the reader comes back. The limit is long enough that stepping away from the tab for a while does not cost a re-parse, and the page's own poll interval is the yardstick it is checked against. The comments around the two budgets said two things that were wrong: - the parsed-rows budget was described as an upper bound on the rows kept resident, in bytes. It is not: V8 cannot report the size of an object graph, so an entry is charged for the source bytes it was parsed from, times a proxy. That is what makes the coverage computable — the source bytes a request does not have to parse again are the conversation's source bytes minus the source bytes the cache holds. - the same comment claimed sealed generations are rare and the budget therefore generous. On this machine a long conversation had 79 of them and the budget held 49.2% of their source text, so the other half was parsed again on every request. The comment now says what the budget actually is: a trade, not a generosity. The measurements the comments cite are from one machine on 2026-10-23, each with its own session and hour, and each comment names its round. --- .../mmc-trajectory/miniapp/node/server.mjs | 173 +++++++++++++++--- 1 file changed, 148 insertions(+), 25 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index e4df9902..8d39c336 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -95,10 +95,13 @@ const TRAJECTORY_CACHE_ENTRIES = 4; * * The body of a large session is tens of megabytes, and it used to be built by ONE * `JSON.stringify` call — so the process had to hold all of it as one string at once, and V8's - * string builder leaves its intermediate copies behind while it doubles its buffer. Measured on - * this machine, serializing a 52.4M-character body cost 745 MB of heap on top of the payload it - * came from, and the pages that peak left behind stayed resident long after the body itself was - * gone: the process kept 965 MB with an 8.4 MB live heap. + * string builder leaves its intermediate copies behind while it doubles its buffer. + * + * The numbers here were measured on 2026-10-23 on this machine, on the code as it stood before + * the body was chunked, on one session of 57,436 records whose body was 53,581,503 bytes + * (52,648,733 characters): serializing it cost 749 MB of heap on top of the 366 MB payload it + * came from, and with every cache emptied the process still held 1,070 MB of resident memory on + * an 8.4 MB live heap. * * A megabyte is a compromise. Small enough that the builder's doubling is noise next to the * payload it is serializing; large enough that a 53 MB body is fifty-odd writes rather than @@ -113,35 +116,77 @@ const TRAJECTORY_BODY_CHUNK_BYTES = 1024 * 1024; */ const MAX_SEARCH_HITS = MAX_HITS; /** - * Upper bound on the parsed generation rows kept resident, in bytes. + * How many BYTES OF ACCOUNTING the parsed generation rows may cost. + * + * The unit is NOT resident bytes, and this comment used to say it was. V8 cannot report an object + * graph's size, so an entry is charged for the SOURCE bytes it was parsed from, times + * `PARSED_ROWS_HEAP_PER_SOURCE_BYTE`. What this budget bounds is therefore how much INPUT the + * cache holds, not how much memory the process ends up holding. The two are related by a measured + * factor — on this machine the retained rows cost about 1.4x their source bytes — and that factor + * moves with the shape of the messages, so it is a measurement and not an identity. * * This budget no longer covers the file that matters. A cache keyed on path+size+mtime for a file * that is being appended to has no hit rate to speak of: `handleTrajectory` only reaches the parse * after the trajectory key — which carries the same size and mtime — has already missed, so every * store here is followed by a miss and the entry is dead weight that still costs its rows' - * memory. Measured on this machine, retaining it was the largest single cost in the process: 378 MB - * of heap after one session and 304 polls, against 145 MB once it is not retained. + * memory. * * What is left is the sealed generations, whose stat never changes and which are therefore the - * only parses a second request can genuinely reuse. Those files are rare at a session root — the - * store keeps rotated generations under `snapshots/`, not beside `messages.jsonl` — so the budget - * is now generous rather than tuned. Pinned by tests/verify-parsed-rows.mjs. + * only parses a second request can genuinely reuse. A LONG session has a great many of them — the + * store keeps one rotated generation per compaction under `snapshots/` — and when they add up to + * more than this budget, the ones the budget cannot hold are parsed again on every request. So + * this is a trade and not a generosity: what the budget holds is paid for in memory, and what it + * does not hold is paid for again on every poll. Measured on 2026-10-23 on this machine, on one + * session of 79 sealed generations and 255 MB of source text, this budget retained 49.2% of that + * source (about 187 MB of live heap) and the other 50.6% was parsed again on every request, at + * 1.33 s of parsing per poll. + * + * `PARSED_ROWS_CACHE_TTL_MS` bounds the other end: how long the retained rows stay after nobody + * asks for them. Pinned by tests/verify-parsed-rows.mjs. */ const PARSED_ROWS_CACHE_BYTES = 256 * 1024 * 1024; /** * Heap charged to one cache entry per source byte read. V8 cannot report an object graph's * size, so a byte budget needs a proxy. * - * Measured on this machine's own lineage: 95.3 MB of messages.jsonl retained 116 MB of heap - * after two forced collections, i.e. 1.22x the source. 2x leaves headroom over that while - * still erring on the side of over-counting, because a proxy that under-counts would let the - * cache grow past its own budget without noticing. + * Measured on this machine: on 2026-10-15, 95.3 MB of messages.jsonl retained 116 MB of heap + * after two forced collections, i.e. 1.22x the source; on 2026-10-23, the rows a session of 255 + * MB of source left in the cache cost about 187 MB of live heap for the 131.5 MB of source the + * budget held, i.e. 1.42x. 2x leaves headroom over both while still erring on the side of + * over-counting, because a proxy that under-counts would let the cache grow past its own budget + * without noticing. * * It was 3x before, and that was wrong in the direction that matters: 95.3 MB x 3 charged * 286 MB against a 256 MB budget, so the LRU evicted entries it had room for in reality and * re-parsed them on the next poll. The budget is only as good as this number. */ const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2; +/** + * How long the parsed rows of a generation stay in the cache after the last request that wanted + * them. + * + * The rows are the largest thing this service holds, and they are only worth holding while + * somebody is asking for them. The page rebuilds the whole trajectory every 8 seconds while it is + * open, and it stops asking the moment the tab is hidden or closed — so a cache with no expiry + * keeps a conversation's rows for as long as the runtime lives, which is how the last session a + * reader opened stayed resident after they walked away. + * + * Ten minutes has two numbers behind it. The page polls every 8 seconds + * (`TRAJECTORY_POLL_INTERVAL` in the client), so an open tab asks again long before this expires. + * And the page stops polling the moment its tab is HIDDEN, so a reader who switches away for a + * while is the ordinary case and not an exception — a limit shorter than that would make every + * return to the tab cost a re-parse. A reader who is really gone gets their rows back within ten + * minutes. + * + * An expiry is a MISS and not a deletion of something still wanted: the next request parses the + * file again and the cache fills itself back up. The price of that is one re-parse, and only when + * the whole-session cache misses too — which happens only when the conversation grew while the + * reader was away, in which case the rebuild was going to happen anyway. + * + * The timer that does the releasing is unref'd, so it never keeps the process alive, and it is + * stopped when the cache empties. Pinned by tests/verify-parsed-rows.mjs. + */ +const PARSED_ROWS_CACHE_TTL_MS = 10 * 60 * 1000; /** * How many decoded messages one trajectory entry keeps for `/raw`, per open session. * @@ -3568,18 +3613,77 @@ export function parsedRowsKey(filePath, size, mtimeMs, limit) { } /** - * LRU over parsed generation rows, bounded by bytes. + * LRU over parsed generation rows, bounded by bytes and by how long they are wanted. * * Bounded by bytes rather than entries because entries say nothing about volume: one * generation file is 40 KB and the next is 10 MB. A single entry larger than the whole budget * is not stored at all — it would evict everything else and still miss every time. * + * The accounting unit is the SOURCE bytes the entry was parsed from, times the heap proxy — not + * resident bytes, which V8 cannot report. That is what makes the coverage computable: the source + * bytes a round does NOT have to parse again are the lineage's source bytes minus the source + * bytes this cache holds, and the cache can say the second number exactly. + * + * Bounded by time as well, because the rows are only worth holding while somebody asks for them: + * a reader who closes the tab stops asking, and a cache with no expiry would keep that + * conversation's rows for as long as the runtime lives. The expiry is a MISS and not a deletion + * of something still wanted — the next request parses the file again and the cache fills itself + * back up. + * * @param {number} budget + * @param {{ ttlMs?: number, now?: () => number, sweepIntervalMs?: number }} [options] */ -export function createParsedRowsCache(budget) { - /** @type {Map} */ +export function createParsedRowsCache(budget, options = {}) { + const ttlMs = options.ttlMs ?? PARSED_ROWS_CACHE_TTL_MS; + const now = options.now ?? (() => Date.now()); + const sweepIntervalMs = options.sweepIntervalMs ?? Math.max(1000, Math.floor(ttlMs / 4)); + /** @type {Map} */ const entries = new Map(); let bytes = 0; + /** @type {ReturnType | null} */ + let sweeper = null; + + /** @param {number} at */ + const expired = (at) => now() - at >= ttlMs; + + /** @param {string} key */ + const drop = (key) => { + const victim = entries.get(key); + if (victim === undefined) return 0; + entries.delete(key); + bytes -= victim.size; + return victim.size; + }; + + const stopSweeper = () => { + if (sweeper === null) return; + clearInterval(sweeper); + sweeper = null; + }; + + /** + * Release what nobody has wanted for a while. This is what makes the rows go away while the + * reader is gone: a sweep that only ran on a request would release them exactly when the + * reader comes back, which is the one moment they are wanted again. + * + * @returns {number} the bytes still held afterwards + */ + const sweep = () => { + for (const [key, entry] of entries) { + if (expired(entry.at)) drop(key); + } + if (entries.size === 0) stopSweeper(); + return bytes; + }; + + const startSweeper = () => { + if (sweeper !== null || ttlMs <= 0 || sweepIntervalMs <= 0) return; + sweeper = setInterval(sweep, sweepIntervalMs); + // An interval that is not unref'd keeps the event loop alive after the server closes, and the + // runtime would wait for it. It must never be the reason the process is still running. + if (typeof sweeper.unref === 'function') sweeper.unref(); + }; + return { get bytes() { return bytes; }, get size() { return entries.size; }, @@ -3590,9 +3694,16 @@ export function createParsedRowsCache(budget) { get(key) { const hit = entries.get(key); if (hit === undefined) return null; + if (expired(hit.at)) { + // The rows have been here longer than anybody asked for them. Releasing them here is the + // same release the sweeper would do; the only difference is that this request asked. + drop(key); + return null; + } // Re-reading is the only signal that a file is still hot, so it refreshes recency. entries.delete(key); entries.set(key, hit); + hit.at = now(); return hit.value; }, /** @@ -3608,7 +3719,7 @@ export function createParsedRowsCache(budget) { entries.delete(key); } if (size > budget) return bytes; - entries.set(key, { value, size }); + entries.set(key, { value, size, at: now() }); bytes += size; while (bytes > budget && entries.size > 1) { const oldest = entries.keys().next(); @@ -3617,11 +3728,15 @@ export function createParsedRowsCache(budget) { entries.delete(oldest.value); bytes -= victim ? victim.size : 0; } + startSweeper(); return bytes; }, + /** Release what expired. The timer calls this; a caller may call it too. */ + sweep, clear() { entries.clear(); bytes = 0; + stopSweeper(); }, }; } @@ -3848,12 +3963,18 @@ function jsonArrayPieces(items, budgetBytes) { * string is let go before the next piece is built. So the process never holds the whole body as * one string, and never holds the builder's intermediate copies either. * - * WHY IT IS WORTH THE ROUTE'S ATTENTION. Measured on this machine, one cold read of a - * 57,192-record session left the process at 1094 MB resident with a 299 MB live heap, and the - * peak inside the route was 1341 MB: the payload was 361 MB, and the single `JSON.stringify` - * that followed it added 745 MB on top. Clearing every cache afterwards freed the live heap - * down to 8.4 MB and the process still held 965 MB — the pages that peak left behind. The - * floor follows the peak, not the cache, so the peak is the thing to change. + * WHY IT IS WORTH THE ROUTE'S ATTENTION. Measured on 2026-10-23 on this machine, on the code as + * it stood before the body was chunked, on one session of 57,436 records: one cold read left the + * process at 1,199 MB resident with a 302.7 MB live heap, and the peak inside the route was + * 1,331 MB resident on 1,115 MB of live heap — the payload object graph was 366 MB, and the single + * `JSON.stringify` that followed it added 749 MB on top. Clearing every cache afterwards freed + * the live heap down to 8.4 MB and the process still held 1,070 MB: the pages this request + * touched are not given back. + * + * And the floor does NOT follow the peak. Chunking the body took 151 MB off the high-water mark + * and the floor did not move (1,070.1 MB before, 1,069.7 MB after). So the floor is neither what + * the cache holds nor what the peak left behind: it is what this request allocated, and cutting + * the allocation is the lever. * * @param {Record} payload * @param {number} [budgetBytes] @@ -5546,7 +5667,9 @@ export async function start(context) { trajectoryCache.clear(); // The parsed-row cache only ever holds sealed generations now, but it is still a cache that // has to be released here: a disposed runtime that kept it would still hold a rotated - // generation's rows. The line is here so the next cache added cannot repeat the omission. + // generation's rows. Clearing it also stops the sweeper that releases the rows nobody wants + // any more, so a disposed runtime leaves no timer behind either. The line is here so the next + // cache added cannot repeat the omission. parsedRows.clear(); clientEntry = null; disposal = close(server).then(() => undefined); From 28e3aec931e3acfb3e12e22680476c5a454418b8 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 14:49:25 +0800 Subject: [PATCH 67/95] Pin the parsed-rows cache's own contract, and name every mutation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cache had a guard for what it serves the page — a hit must be byte-identical to a cold read — and nothing for what it can say about itself. Two things were unpinned: - the accounting. The budget is charged in source bytes times the heap proxy, and the source bytes the cache holds are recoverable from its own accounting by dividing by that factor. That relationship is what makes the coverage a number instead of a guess, so it is now asserted: the cache is charged the source bytes times the proxy, and the charge the route makes is that one. - the lifetime. The rows are released after ten minutes without a request, the time limit is longer than one poll interval and long enough that stepping away does not cost a re-parse, and the timer that does the releasing never keeps the process alive. The control for this guard changed shape too. It used to decide a case by looking at the first failing line, which meant it could not say which rule it had caught; every case now names the rule it is about and counts as caught only when the probe reports THAT rule. It went from twelve cases with no names to twenty, all caught by name — and the new cases are the ones about the accounting and about the release. --- .../tests/mutate-parsed-rows.py | 234 ++++++++++++++---- .../mmc-trajectory/tests/mutate-review16.py | 4 +- .../tests/verify-parsed-rows.mjs | 101 ++++++++ .../mmc-trajectory/tests/verify-review16.mjs | 2 +- 4 files changed, 290 insertions(+), 51 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py index 1d49ae04..88689019 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py @@ -1,15 +1,28 @@ """Mutation control for verify-parsed-rows.mjs. -Each mutation reintroduces exactly one bug the probe is supposed to catch. The probe must FAIL -every time, and the file must come back byte-for-byte identical afterwards. A mutation that the -probe still passes means the assertion was tautological; a mutation whose anchor no longer -exists means the assertion went dead, which is worse — the runner reports those as misses too. - -Every mutation here is non-trivial: it changes behaviour under the condition the assertion is -about, not just text. In particular the four key mutations each break a *different* component, -because a key that drops two of them would still be caught by the wrong test and we would not -learn which one mattered. +Each mutation reintroduces exactly one bug the probe is supposed to catch, BY NAME. The ones that +matter most are the first three: each of them makes the endpoint answer something plausible and +wrong, and none of them throws. + + - searching the STORED rows instead of the projected view reports versions the ledger is not + showing, and hands back an index no row occupies; + - dropping one of the five searched fields narrows what a reader can find with no error at all; + - counting only what fits under the cap turns "how many matched" into "how many did we send". + +Every case names the rule it is about in its `expect` field, and a case counts as caught only +when the probe reported THAT rule: a probe that went red on some other assertion has not shown +that this guard owns this mutation. The rules the last six cases are about are the ones this +cache has to answer about ITSELF rather than about the page's bytes: + + - the cache is charged the SOURCE bytes it holds (times the heap proxy), so the part of a + lineage a round does not have to parse again is a number and not a guess; + - the rows are released when nobody has wanted them for a while, without a request, because a + release that only ran on a request would happen exactly when the reader comes back; + - that release is longer than one poll, or an open tab would lose its rows between polls; + - and the timer that does it never keeps the process alive. """ +import json +import re import subprocess import sys from pathlib import Path @@ -23,45 +36,52 @@ # ---- the key: each component removed on its own ------------------------------- ( "server: the key stops naming the file, so two sessions share their messages.jsonl", - SERVER, " return `${filePath}|${size}|${mtimeMs}|${limit}`;", " return `${filePath.split(/[\\\\/]/).pop()}|${size}|${mtimeMs}|${limit}`;", + 'the path is part of the key (two sessions, one basename)', ), ( "server: the key stops carrying the size, so a grown file serves a stale tail", - SERVER, " return `${filePath}|${size}|${mtimeMs}|${limit}`;", " return `${filePath}|${mtimeMs}|${limit}`;", + 'the size is part of the key (the active file grows)', ), ( "server: the key stops carrying the mtime, so a same-size rewrite serves stale rows", - SERVER, " return `${filePath}|${size}|${mtimeMs}|${limit}`;", " return `${filePath}|${size}|${limit}`;", + 'the mtime is part of the key (a rewrite at the same size)', ), ( "server: the key stops carrying the limit, so the truncation verdict is read at another width", - SERVER, " return `${filePath}|${size}|${mtimeMs}|${limit}`;", " return `${filePath}|${size}|${mtimeMs}|64 * 1024 * 1024`;", + 'the limit is part of the key (readHead truncates at it)', ), # ---- the cache value ---------------------------------------------------------- ( "server: `empty` is not carried on the cached value, so a hit stops reporting it", - SERVER, " empty: head.buffer.length === 0,\n", " empty: false,\n", + 'an empty file still reports empty on a hit', ), ( - "server: `skipped` is not carried, so a hit under-counts the lines it dropped", - SERVER, + "server: `skipped` is not carried, so a read under-counts the lines it dropped", " skipped: parsed.skipped,\n", " skipped: 0,\n", + 'two lines are skipped: one unparseable, one not an object', + ), + ( + # The value carries the count the parse reported, so the only way a HIT can disagree with + # a cold read is if the hit path stops handing back what it was given. + "a hit reports a skipped count of its own rather than the one it was given", + " if (hit !== null) return hit;\n", + " if (hit !== null) return { ...hit, skipped: 0 };\n", + 'a hit reports the same skipped count as a cold read', ), # ---- the budget --------------------------------------------------------------- ( "the budget stops evicting, so the cache grows without limit", - SERVER, """ while (bytes > budget && entries.size > 1) { const oldest = entries.keys().next(); if (oldest.done) break; @@ -71,26 +91,27 @@ } """, "", + 'the third evicts the oldest, keeping the budget', ), ( "reading stops refreshing recency, so the 22 frozen files get evicted by insertion order", - SERVER, """ // Re-reading is the only signal that a file is still hot, so it refreshes recency. entries.delete(key); entries.set(key, hit); + hit.at = now(); return hit.value; """, " return hit.value;\n", + 'a file that was just read survives the eviction that follows', ), ( "an entry larger than the whole budget is stored anyway, evicting everything for one hit", - SERVER, " if (size > budget) return bytes;\n", "", + 'a single entry larger than the budget is not stored', ), ( "re-storing a key double-counts it, so the budget shrinks over time", - SERVER, """ const previous = entries.get(key); if (previous !== undefined) { bytes -= previous.size; @@ -98,19 +119,82 @@ } """, "", + 're-storing the same key counts once', ), # ---- structure ---------------------------------------------------------------- ( "the cache becomes a module-level singleton, shared across every runtime instance", - SERVER, " const parsedRows = createParsedRowsCache(PARSED_ROWS_CACHE_BYTES);", " globalThis.__parsedRows = globalThis.__parsedRows || createParsedRowsCache(PARSED_ROWS_CACHE_BYTES);\n const parsedRows = globalThis.__parsedRows;", + 'the cache is created inside start(), not at module level', ), ( "the route parses the file inline again instead of going through the cache", - SERVER, " parsedFile = await loadParsedRows(parsedRows, filePath, limit, { cacheable: link.fileName !== activeFileName });", " parsedFile = await (async () => { const h = await readHead(filePath, limit); const p = parseMessagesJsonl(h.buffer); return { rows: p.rows, locators: p.spans.map(([s, e]) => ({ start: s, end: e, digest: lineDigest(h.buffer.subarray(s, e)) })), skipped: p.skipped, size: h.size, read: h.read, empty: h.buffer.length === 0, truncated: truncatedByCap(h.size, limit) }; })();", + 'the route reaches the file through loadParsedRows()', + ), + # ---- the coverage the cache can state about itself --------------------------------- + ( + # The budget is charged in SOURCE bytes times the heap proxy. That is what makes the + # coverage computable: the source bytes a round does not have to parse again are the + # lineage's source bytes minus the source bytes this cache holds. Charge the source + # bytes alone and the number a reader can compute no longer matches what the cache holds. + "the cache is charged the source bytes without the heap proxy, so its accounting drifts from its input", + " if (cacheable) cache.set(key, value, Math.ceil(head.size * PARSED_ROWS_HEAP_PER_SOURCE_BYTE));", + " if (cacheable) cache.set(key, value, head.size);", + 'the cache is charged the source bytes times the heap proxy', + ), + # ---- the rows are released when nobody wants them ------------------------------------ + ( + # The read path has to notice that the rows have been here longer than anybody asked + # for them. Without this the cache keeps a conversation's rows for as long as the + # runtime lives, which is how the last session a reader opened stayed resident. + "a read does not notice that the rows have expired", + " if (expired(hit.at)) {\n", + " if (false) {\n", + 'a read after the time limit is a miss', + ), + ( + # The sweep is what releases the rows while the reader is gone. A sweep that releases + # nothing is a sweep that does not exist. + "the sweep releases nothing", + " if (expired(entry.at)) drop(key);\n", + " if (false) drop(key);\n", + 'a sweep releases what expired', + ), + ( + # Without the timer, the rows are only released by the next request — which is exactly + # the moment the reader comes back and wants them again. + "the sweeper is never started, so the rows stay until the next request", + " startSweeper();\n", + "", + 'the rows are released without a request', + ), + ( + # An interval that is not unref'd keeps the event loop alive after the server closes, + # and the runtime would wait for it. + "the sweeper keeps the process alive", + " if (typeof sweeper.unref === 'function') sweeper.unref();\n", + "", + 'the sweeper does not keep the process alive', + ), + ( + # The page polls every 8 seconds. A limit shorter than one poll would make an open tab + # lose its rows between two polls and re-parse the whole lineage for nothing. + "the time limit is shorter than one poll", + "const PARSED_ROWS_CACHE_TTL_MS = 10 * 60 * 1000;", + "const PARSED_ROWS_CACHE_TTL_MS = 1000;", + 'the time limit is longer than one poll, so an open tab never loses its rows', + ), + ( + # The page stops polling while its tab is hidden, so a reader who switches away for a + # while is the ordinary case. A minute is short enough that every return to the tab + # costs a re-parse — which is the other half of what this limit buys. + "the time limit is short enough that switching away from the tab costs a re-parse", + "const PARSED_ROWS_CACHE_TTL_MS = 10 * 60 * 1000;", + "const PARSED_ROWS_CACHE_TTL_MS = 60 * 1000;", + 'the time limit is at least five minutes, so stepping away from the tab does not cost a re-parse', ), ] @@ -118,9 +202,9 @@ def read_exact(path): """Normalise to LF for matching, and report the file's own newline so it can be put back. - server.mjs is CRLF in the working tree. Matching against the raw bytes would make every - multi-line anchor miss — which the runner reports as a surviving mutation, so the failure - looks like a dead guard rather than a broken anchor. + Matching against the raw bytes would make every multi-line anchor miss — which the runner + reports as a surviving mutation, so the failure looks like a dead guard rather than a + broken anchor. """ data = path.read_bytes() text = data.decode("utf-8").replace("\r\n", "\n") @@ -145,6 +229,45 @@ def run_probe(): return proc.returncode, (proc.stdout or "") + (proc.stderr or "") +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off — and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + Two formats, as in `mutate-search-endpoint.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names + + def main(): code, output = run_probe() if code != 0: @@ -153,41 +276,56 @@ def main(): return 1 print("baseline clean") - originals = {path: path.read_bytes() for path in {m[1] for m in MUTATIONS}} + original = SERVER.read_bytes() missed = [] - for label, path, old, new in MUTATIONS: - text, newline = read_exact(path) + unnamed = [] + for label, old, new, expect in MUTATIONS: + text, newline = read_exact(SERVER) if old not in text: - print(f"SKIP {label}: anchor not found in {path.name}") + print(f"SKIP {label}: anchor not found") + missed.append(label) + continue + if old == new: + print(f"SKIP {label}: the mutation changes nothing") missed.append(label) continue - write_exact(path, text.replace(old, new, 1), newline) + write_exact(SERVER, text.replace(old, new, 1), newline) try: code, output = run_probe() finally: - path.write_bytes(originals[path]) + SERVER.write_bytes(original) if code == 0: print(f"MISSED {label}: the probe still passed") missed.append(label) - else: - first = next( - (line.strip() for line in output.splitlines() if line.strip().startswith("- ")), - "(no detail)", - ) - print(f"caught {label}\n -> {first}") - - for path, data in originals.items(): - if path.read_bytes() != data: - print(f"NOT RESTORED {path}") - return 1 - print("all files restored byte-for-byte") - - if missed: - print(f"\n{len(missed)} mutation(s) got through") + continue + names = reported_rules(output) + if not names: + print(f"UNNAMED {label}: the probe failed without naming a rule") + unnamed.append(label) + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") + + if SERVER.read_bytes() != original: + print(f"NOT RESTORED {SERVER}") + return 1 + print("file restored byte-for-byte") + + if missed or unnamed: + if missed: + print(f"\n{len(missed)} mutation(s) got through") + if unnamed: + print(f"{len(unnamed)} mutation(s) failed without naming a rule") return 1 - print(f"\nall {len(MUTATIONS)} mutations caught") + print(f"\nall {len(MUTATIONS)} mutations caught, each by name") return 0 if __name__ == "__main__": - sys.exit(main()) \ No newline at end of file + sys.exit(main()) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py b/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py index 53cd4e36..a1d896f6 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py @@ -247,8 +247,8 @@ # and this harness reports an ambiguous anchor as unresolved rather than guessing. The bytes # below are what make it specific to the parsed-row cache: its counter has to be reset too. ("the parsed-row cache loses its clear()", - " clear() {\n entries.clear();\n bytes = 0;\n },", - " reset() {\n entries.clear();\n bytes = 0;\n },", + " clear() {\n entries.clear();\n bytes = 0;\n stopSweeper();\n },", + " reset() {\n entries.clear();\n bytes = 0;\n stopSweeper();\n },", "the parsed-row cache really does expose clear()"), # ---- 6. the empty state points at inert controls again ---- diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index 06cbd632..9333473d 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -342,6 +342,107 @@ assert('the truncation verdict comes from truncatedByCap', assert('loadParsedRows is exported so a test can run the real step', SERVER.includes('export async function loadParsedRows(')); +/* ===================================================================== + * 9 — the coverage is computable, and the rows are released when nobody wants them + * + * Two questions this cache has to answer, and neither of them is "how many entries are in + * there" — that number moves with the order the files happened to be read in. + * + * COVERAGE. The budget is charged in SOURCE bytes times the heap proxy, not in resident bytes, + * which V8 cannot report. That is what makes the coverage computable: the source bytes a + * round does NOT have to parse again are the lineage's source bytes minus the source bytes this + * cache holds, and the cache can say the second number exactly. A cache whose accounting had + * drifted from its input could not answer that at all, and then the budget would only be a + * number in a source file. + * + * LIFETIME. The rows are the largest thing this service holds and they are only worth holding + * while somebody asks for them: the page stops asking the moment its tab is hidden, so a cache + * with no expiry keeps that conversation's rows for as long as the runtime lives. The release + * has to happen WITHOUT a request — a sweep that only ran on a request would release them + * exactly when the reader comes back, which is the one moment they are wanted again. + * ===================================================================== */ + +// A constant the source declares as an expression — `60 * 1000` — is read by arithmetic rather +// than by a digit pattern, so the check follows the constant instead of repeating it. +const declaredNumber = (source, name) => { + const match = source.match(new RegExp(`const ${name} = ([0-9_ *]+);`)); + if (match === null) return null; + const parts = match[1].replace(/_/g, '').split('*').map((part) => Number(part.trim())); + if (parts.some((part) => !Number.isFinite(part))) return null; + return parts.reduce((carry, part) => carry * part, 1); +}; + +// The proxy is read out of the shipped source rather than repeated here, so the relationship +// below is checked against the constant the service actually charges with. +const PROXY = declaredNumber(SERVER, 'PARSED_ROWS_HEAP_PER_SOURCE_BYTE'); +assert('the heap proxy is a number the service declares', PROXY !== null); + +// A file of a known size, loaded through the shipped step: the cache must be charged the source +// bytes times the proxy, so the source bytes it holds are recoverable from its own accounting. +const sized = join(root, 'sized.jsonl'); +const sizedText = ROWS.map((r) => JSON.stringify(r)).join('\n') + '\n'; +await writeFile(sized, sizedText); +const sizedBytes = Buffer.byteLength(sizedText, 'utf8'); +const coverageCache = createParsedRowsCache(64 * 1024 * 1024); +await loadParsedRows(coverageCache, sized, LIMIT); +check('the cache is charged the source bytes times the heap proxy', + coverageCache.bytes, Math.ceil(sizedBytes * PROXY)); +check('so the source bytes it holds are computable from its own accounting', + coverageCache.bytes / PROXY, sizedBytes); +assert('and the charge the route makes is that one', + SERVER.includes('cache.set(key, value, Math.ceil(head.size * PARSED_ROWS_HEAP_PER_SOURCE_BYTE));')); + +// A clock the test controls, so "expired" is a fact rather than a wait. +let clock = 1_700_000_000_000; +const ttlCache = createParsedRowsCache(1000, { ttlMs: 60_000, now: () => clock }); +ttlCache.set('k', val, 500); +assert('a stored entry is there', ttlCache.get('k') !== null); +clock += 59_000; +assert('a read inside the time limit is still a hit', ttlCache.get('k') !== null); +check('and it is still held', ttlCache.bytes, 500); +// The read above refreshed the entry, so the limit runs from THAT moment: the clock has to move +// a whole limit again, not a little more than the limit. +clock += 61_000; +assert('a read after the time limit is a miss', ttlCache.get('k') === null); +check('and the entry is not held any more', ttlCache.bytes, 0); + +// The sweep releases what expired and keeps what has not. +const sweepCache = createParsedRowsCache(1000, { ttlMs: 60_000, now: () => clock }); +sweepCache.set('k', val, 500); +sweepCache.sweep(); +check('a sweep inside the time limit keeps the entry', sweepCache.bytes, 500); +clock += 61_000; +sweepCache.sweep(); +check('a sweep releases what expired', sweepCache.bytes, 0); +check('and leaves nothing behind', sweepCache.size, 0); + +// And the release happens without a request, which is the whole point of the time limit. +const liveCache = createParsedRowsCache(1000, { ttlMs: 100, sweepIntervalMs: 25 }); +liveCache.set('k', val, 500); +assert('the rows are there to begin with', liveCache.bytes, 500); +await new Promise((resolve) => setTimeout(resolve, 1000)); +check('the rows are released without a request', liveCache.bytes, 0); +check('and the cache is empty', liveCache.size, 0); + +// The time limit has to outlive one poll, or an open tab would lose its rows between polls. +const clientSource = readFileSync(new URL('../miniapp/client/index.html', import.meta.url), 'utf8'); +const pollMatch = clientSource.match(/TRAJECTORY_POLL_INTERVAL = (\d+)/); +assert('the client says how often it polls the trajectory', pollMatch !== null); +const ttlMatch = SERVER.match(/const PARSED_ROWS_CACHE_TTL_MS = ([0-9_ *]+);/); +assert('the rows cache has a time limit', ttlMatch !== null); +const TTL = declaredNumber(SERVER, 'PARSED_ROWS_CACHE_TTL_MS'); +assert('the time limit is a number the service declares', TTL !== null); +assert('the time limit is longer than one poll, so an open tab never loses its rows', + TTL > Number(pollMatch[1])); +// The page stops polling while its tab is hidden, so a reader who switches away for a while is +// the ordinary case. A limit shorter than that would make every return to the tab cost a +// re-parse — which is the other half of what this limit buys, and it has to be paid for. +assert('the time limit is at least five minutes, so stepping away from the tab does not cost a re-parse', + TTL >= 5 * 60 * 1000); + +// And the sweeper must never be the reason the process is still running. +assert('the sweeper does not keep the process alive', /\.unref\(\)/.test(SERVER)); + await rm(root, { recursive: true, force: true }); /* ===================================================================== */ diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-review16.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-review16.mjs index 9e14962d..89442042 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-review16.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-review16.mjs @@ -268,7 +268,7 @@ assert('dispose() still clears the other two caches', !!disposeBody && /descriptorCache\.clear\(\)/.test(disposeBody[1]) && /trajectoryCache\.clear\(\)/.test(disposeBody[1])); assert('the parsed-row cache really does expose clear()', - /createParsedRowsCache[\s\S]*?clear\(\)\s*\{\s*entries\.clear\(\);\s*bytes = 0;\s*\}/.test(serverJs)); + /createParsedRowsCache[\s\S]*?clear\(\)\s*\{[^}]*entries\.clear\(\);[^}]*bytes = 0;[^}]*\}/.test(serverJs)); /* ---------- 6. the empty state names a control that still works ---------- */ From ff232b96bd114c269021a664a1fe3ff127802277 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 15:50:10 +0800 Subject: [PATCH 68/95] Say when the rows are really released, and mean one thing by a limit of zero A review of the committed bytes found these things in the parsed-rows cache, and the code did not say any of them: - the promise was tighter than the implementation. The limit is when the rows stop being wanted; the RELEASE happens on the next sweep, and the sweep runs every two and a half minutes, so the rows are gone no later than twelve and a half minutes after the last request -- not ten. The number is now the real one everywhere it is stated. - a limit of zero meant two contradictory things. The rows expired at once -- every read a miss, every request a re-parse -- while the sweeper never started, so nothing was ever released. Zero now means no limit and nothing else: the rows never expire and the sweeper never runs. - the assertion that the sweeper does not keep the process alive matched any unref in the file, so a later timer with an unref of its own would have kept the rule passing after the sweeper had lost it. It matches the sweeper's own call now. - the relation between the limit and the sweep interval held only for the numbers this package happens to ship. A caller who asked for a longer interval got one, and the promise to a reader was then wrong by that much; the two numbers were also only related in the source, not in the cache, so changing the place the interval is taken from left the assertion passing. The sweep interval is a named constant rather than a fraction written at the call site, because the promise to a reader is the limit plus that interval, and the two have to stay in that relation. And that relation is now enforced where the interval is TAKEN rather than only in a comment: the interval is clamped to a quarter of the limit the cache was given, so "never later than a quarter of the limit beyond it" holds for every limit -- and for the interval a caller asks for -- and not only for the one this package ships with. The guard reads the interval the running cache uses instead of two numbers in the source, and the control has a case on the line that takes it: reading the source would have kept the assertion passing after that line had changed. --- .../mmc-trajectory/miniapp/node/server.mjs | 44 +++++++++++++++++-- .../tests/mutate-parsed-rows.py | 25 +++++++++++ .../tests/verify-parsed-rows.mjs | 38 ++++++++++++++-- 3 files changed, 100 insertions(+), 7 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 8d39c336..9b124953 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -175,8 +175,13 @@ const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2; * (`TRAJECTORY_POLL_INTERVAL` in the client), so an open tab asks again long before this expires. * And the page stops polling the moment its tab is HIDDEN, so a reader who switches away for a * while is the ordinary case and not an exception — a limit shorter than that would make every - * return to the tab cost a re-parse. A reader who is really gone gets their rows back within ten - * minutes. + * return to the tab cost a re-parse. A reader who is really gone gets their rows back. + * + * **WHEN THEY COME BACK IS NOT THE SAME MOMENT AS WHEN THEY EXPIRE.** The limit is when the rows + * stop being wanted; the release happens on the next sweep, so the rows are gone no later than + * one sweep interval after the limit — twelve and a half minutes with the shipped values, and + * never longer than a quarter of the limit beyond it. The number to promise a reader is the + * second one, and it is asserted: the sweep interval is at most a quarter of the limit. * * An expiry is a MISS and not a deletion of something still wanted: the next request parses the * file again and the cache fills itself back up. The price of that is one re-parse, and only when @@ -187,6 +192,23 @@ const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2; * stopped when the cache empties. Pinned by tests/verify-parsed-rows.mjs. */ const PARSED_ROWS_CACHE_TTL_MS = 10 * 60 * 1000; +/** + * How often the cache sweeps for what has expired. + * + * The release lands on a tick, so this interval is also the gap between "the rows expired" and + * "the rows are gone" — which is why it is a named constant and not a fraction written at the + * call site: the promise to a reader is the limit PLUS this, and the two have to stay in that + * relation. + * + * The relation is enforced where the interval is TAKEN, not only here: the interval is clamped + * to a quarter of the limit the cache was given, so "never later than a quarter of the limit + * beyond it" holds for every limit and not only for the shipped one. A constant that could + * outgrow the limit would make the promise true on the day it was written and false after + * somebody changed the limit. + * + * Pinned by tests/verify-parsed-rows.mjs. + */ +const PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS = 150 * 1000; /** * How many decoded messages one trajectory entry keeps for `/raw`, per open session. * @@ -3630,13 +3652,24 @@ export function parsedRowsKey(filePath, size, mtimeMs, limit) { * of something still wanted — the next request parses the file again and the cache fills itself * back up. * + * A time limit of zero means NO time limit, and it means that in one place only: the rows never + * expire and the sweeper never runs. (The other reading — "everything expires at once" — would be + * a cache that never hits and never releases, which is worse than either.) + * * @param {number} budget * @param {{ ttlMs?: number, now?: () => number, sweepIntervalMs?: number }} [options] */ export function createParsedRowsCache(budget, options = {}) { const ttlMs = options.ttlMs ?? PARSED_ROWS_CACHE_TTL_MS; const now = options.now ?? (() => Date.now()); - const sweepIntervalMs = options.sweepIntervalMs ?? Math.max(1000, Math.floor(ttlMs / 4)); + // The interval the sweep actually uses, clamped to a quarter of the limit the cache was + // given. The clamp is the rule, not a convenience: without it the promise to a reader — the + // rows are gone no later than a quarter of the limit after they expire — would hold for the + // shipped limit and for nothing else, and the limit is an option a caller may set. + const sweepIntervalMs = Math.min( + options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS, + Math.max(1, Math.floor(ttlMs / 4)), + ); /** @type {Map} */ const entries = new Map(); let bytes = 0; @@ -3644,7 +3677,7 @@ export function createParsedRowsCache(budget, options = {}) { let sweeper = null; /** @param {number} at */ - const expired = (at) => now() - at >= ttlMs; + const expired = (at) => ttlMs > 0 && now() - at >= ttlMs; /** @param {string} key */ const drop = (key) => { @@ -3687,6 +3720,9 @@ export function createParsedRowsCache(budget, options = {}) { return { get bytes() { return bytes; }, get size() { return entries.size; }, + // The interval the sweep actually uses, so the relation between the limit and the release + // can be checked on the running cache rather than on the two numbers in the source. + get sweepIntervalMs() { return sweepIntervalMs; }, /** * @param {string} key * @returns {ParsedRows | null} diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py index 88689019..6435dd3c 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py @@ -196,6 +196,31 @@ "const PARSED_ROWS_CACHE_TTL_MS = 60 * 1000;", 'the time limit is at least five minutes, so stepping away from the tab does not cost a re-parse', ), + ( + # The relation between the limit and the sweep interval has to hold for the interval the + # cache ACTUALLY uses. Reading two numbers out of the source would keep passing after the + # line that TAKES the interval had changed, so the mutation is on that line. + # + # The case that used to change the constant's value is not a defect any more and is gone + # for that reason, not because its rule lost an owner: the interval is clamped to the + # limit where it is taken, so a longer constant is simply never used. The rule itself is + # still owned — by this case. + "the sweep interval is taken without being clamped to the limit", + """ const sweepIntervalMs = Math.min( + options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS, + Math.max(1, Math.floor(ttlMs / 4)), + );""", + " const sweepIntervalMs = options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS;", + 'the sweep interval the cache uses is at most a quarter of the limit', + ), + ( + # A time limit of zero means NO time limit. The other reading — everything expires at + # once — would be a cache that never hits and never releases. + "a time limit of zero expires everything anyway", + " const expired = (at) => ttlMs > 0 && now() - at >= ttlMs;", + " const expired = (at) => now() - at >= ttlMs;", + 'a time limit of zero means no time limit, not that everything expires at once', + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index 9333473d..54a0c47e 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -359,7 +359,10 @@ assert('loadParsedRows is exported so a test can run the real step', * while somebody asks for them: the page stops asking the moment its tab is hidden, so a cache * with no expiry keeps that conversation's rows for as long as the runtime lives. The release * has to happen WITHOUT a request — a sweep that only ran on a request would release them - * exactly when the reader comes back, which is the one moment they are wanted again. + * exactly when the reader comes back, which is the one moment they are wanted again. And the + * release lands on a sweep tick, so the rows are gone a sweep interval AFTER the limit, not at + * it: the number to promise a reader is the limit plus that interval, and the two have to stay + * in that relation. * ===================================================================== */ // A constant the source declares as an expression — `60 * 1000` — is read by arithmetic rather @@ -440,8 +443,37 @@ assert('the time limit is longer than one poll, so an open tab never loses its r assert('the time limit is at least five minutes, so stepping away from the tab does not cost a re-parse', TTL >= 5 * 60 * 1000); -// And the sweeper must never be the reason the process is still running. -assert('the sweeper does not keep the process alive', /\.unref\(\)/.test(SERVER)); +// And the sweeper must never be the reason the process is still running. The name is the +// sweeper's own, because a whole-file pattern would keep passing after some OTHER timer in the +// service grew an unref of its own — and the rule would quietly lose its owner. +assert('the sweeper does not keep the process alive', /sweeper\.unref\(\)/.test(SERVER)); + +// The release lands on a sweep tick, so the rows are gone a sweep interval AFTER the limit. The +// promise to a reader is that interval added on, and it only stays a promise while the interval +// is a fraction of the limit. This is checked on the interval the cache ACTUALLY uses, for +// several limits — not on two numbers read out of the source, which would keep passing after +// the line that takes the interval had changed. +const SHIPPED_SWEEP = declaredNumber(SERVER, 'PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS'); +assert('the sweep interval is a number the service declares', SHIPPED_SWEEP !== null); +for (const limit of [1_000, 60_000, 600_000, 3_600_000]) { + const bounded = createParsedRowsCache(1000, { ttlMs: limit }); + assert('the sweep interval the cache uses is at most a quarter of the limit', + bounded.sweepIntervalMs <= limit / 4); +} +const asked = createParsedRowsCache(1000, { ttlMs: 1_000, sweepIntervalMs: 500_000 }); +assert('an interval the caller asks for is clamped to the limit as well', + asked.sweepIntervalMs <= 1_000 / 4); +assert('the shipped default is the interval the service declares', + createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP); + +// A time limit of zero means NO time limit. The other reading — everything expires at once — +// would be a cache that never hits and never releases, which is worse than either. +const noLimit = createParsedRowsCache(1000, { ttlMs: 0, now: () => clock }); +noLimit.set('k', val, 500); +clock += 10 * 60 * 1000; +assert('a time limit of zero means no time limit, not that everything expires at once', + noLimit.get('k') !== null); +check('and the rows are still held', noLimit.bytes, 500); await rm(root, { recursive: true, force: true }); From 49df7ace65e87b08f7ea81899720d17775d2048a Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 16:53:52 +0800 Subject: [PATCH 69/95] The pieces reach the socket as pieces, and the closing bracket is counted against the budget A review of the committed bytes found two things in the body serialization, and both of them are about a comparison that was too weak: - the send path compared the LENGTHS of what reached the socket and of what the serializer produced. Two escapes passed it: one that joins the pieces into a single write, and one that changes the bytes on the way out without changing their length. The guard now compares the bytes against an expectation it derives itself, checks that one write per piece reached the socket, and checks that each of them is bytes. Each of the three has a mutation of its own, and the control for that guard went from seven cases to twelve, every one of them caught by name. - the closing bracket of the array was appended after the size of a piece had been decided, so a piece holding more than one element could come out one byte over the budget. The bracket is counted where the size is decided now, and the guard has the boundary fixture that sees it. Two wordings in the package were wrong as well, and they are corrected here: - three places said the process never holds the whole body as a string. A value in the payload that is not an array is serialized on its own and appended to the piece being built, so that one is still a string. The three places now say the payload's BIG ARRAY, and write the exception out. - the byte account said that turning a Chinese transcript into bytes makes it one byte per character instead of two. UTF-8 gives a Chinese character three bytes, so that is not half the size. V8 holds a string in ONE representation for the WHOLE string, so a single character outside Latin-1 raises the whole string to two bytes per character and the ASCII majority goes up with it; what the change removes is the STRING, and the bytes live outside the JS heap. The suite passes with every case of that control caught by name. --- .../mmc-trajectory/miniapp/node/server.mjs | 32 +++++++--- .../tests/mutate-trajectory-body.py | 61 ++++++++++++++++++- .../tests/verify-trajectory-body.mjs | 46 +++++++++++++- 3 files changed, 128 insertions(+), 11 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 9b124953..17b0e08f 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -3950,6 +3950,11 @@ export async function loadParsedRows(cache, filePath, limit, options) { * An element that does not fit the budget on its own becomes a piece of its own. That is the one * thing the budget cannot bound, and it is bounded by the element instead of by the budget. * + * THE CLOSING BRACKET IS COUNTED. It is one byte, it lands on the piece that holds the last + * element, and a piece that held two elements and ended one byte over the budget was a piece + * the budget did not bound. One byte is reserved for it on every piece, so no piece that holds + * more than one element is ever bigger than the budget. + * * @param {Array} items * @param {number} budgetBytes * @returns {Array} @@ -3968,7 +3973,12 @@ function jsonArrayPieces(items, budgetBytes) { // cons string, and measuring its length would flatten it into a second copy of the // element. On a body of fifty thousand records that is fifty thousand copies. const pieceBytes = textBytes + (first ? 0 : 1); - if (currentBytes > 1 && currentBytes + pieceBytes > budgetBytes) { + // The closing bracket is counted here, not appended at the end. It lands on the piece that + // holds the LAST element, and a piece that ended exactly at the budget came out one byte + // over it — so the budget did not bound the piece it was supposed to bound. One byte is + // reserved on every piece, which is one byte more than the pieces that do not carry the + // bracket need: the budget is a ceiling, not a target. + if (currentBytes > 1 && currentBytes + pieceBytes + 1 > budgetBytes) { pieces.push(Buffer.from(current, 'utf8')); current = ''; currentBytes = 0; @@ -3996,8 +4006,13 @@ function jsonArrayPieces(items, budgetBytes) { * * WHAT CHANGES IS HOW THEY ARE PRODUCED. The payload's big array is cut into pieces at element * boundaries, each piece is serialized on its own and turned into bytes immediately, and the - * string is let go before the next piece is built. So the process never holds the whole body as - * one string, and never holds the builder's intermediate copies either. + * string is let go before the next piece is built. So the process never holds THAT ARRAY as one + * string, and never holds the builder's intermediate copies either. + * + * THE EXCEPTION IS A VALUE THAT IS NOT AN ARRAY. It is serialized on its own, by + * `JSON.stringify(value)`, and appended to the piece being built — so a payload whose bulk is one + * large string still holds that string, and the piece that holds it is a piece like any other. + * What this change removes is the array. * * WHY IT IS WORTH THE ROUTE'S ATTENTION. Measured on 2026-10-23 on this machine, on the code as * it stood before the body was chunked, on one session of 57,436 records: one cold read left the @@ -4066,8 +4081,10 @@ export function serializeTrajectoryBody(payload, budgetBytes = TRAJECTORY_BODY_C * (storable, but must be revalidated) and carry the tag the page echoes back. * * The body is either one string or a list of byte pieces. The list is what the trajectory route - * sends now: the bytes are the same bytes, but they were never held as one string, so the - * declared length is the sum of the pieces rather than the length of a string. + * sends now: the bytes are the same bytes, but the payload's big array was never held as one + * string, so the declared length is the sum of the pieces rather than the length of a string. + * (A value inside the payload that is not an array is still one string — see + * `serializeTrajectoryBody`.) * @param {any} response * @param {number} status * @param {string | Buffer | Array} body @@ -4960,8 +4977,9 @@ export async function start(context) { let body; try { // In pieces, not as one string. The bytes are the same bytes — see serializeTrajectoryBody - // for why that is a contract and not an accident — but the process never holds the whole - // body as a string, and the peak this route used to leave behind does not come back. + // for why that is a contract and not an accident — but the process never holds the + // payload's big array as a string, and the peak this route used to leave behind does not + // come back. body = serializeTrajectoryBody(payload); } catch { sendJson(response, 500, { error: 'trajectory_unserializable', message: '会话数据暂时无法读取' }); diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py index d34c98bb..dc15f1bb 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py @@ -57,8 +57,11 @@ "the body is cut into more pieces as it grows, so no single string holds it", ), - # The pieces as strings. Every character outside Latin-1 makes V8 hold a string as two bytes - # per character, which is the whole reason the pieces are bytes: a Chinese transcript doubles. + # The pieces as strings. V8 holds a string in ONE representation for the whole string: one + # character outside Latin-1 and the whole string is two bytes per character, ASCII majority + # included. That is the whole reason the pieces are bytes: they are the UTF-8 the text already + # is — a Chinese character is three of those bytes, not one — and they live outside the JS + # heap instead of in a string that has to be copied. ( "the pieces are handed back as strings", " pieces.push(Buffer.from(current, 'utf8'));", @@ -70,11 +73,63 @@ # holds everything, and the peak is back. ( "the budget stops bounding a piece", - " if (currentBytes > 1 && currentBytes + pieceBytes > budgetBytes) {", + " if (currentBytes > 1 && currentBytes + pieceBytes + 1 > budgetBytes) {", " if (false) {", "no piece is bigger than the budget unless it holds a single element", ), + # The closing bracket is not counted against the budget. The bytes are still right, and the + # piece that holds two elements comes out ONE BYTE over the budget — the bracket lands on it + # after the size was decided. The boundary fixture is the one that sees it: the fixture above + # cannot, because its piece holds a single element. + ( + "the closing bracket is not counted against the budget", + " if (currentBytes > 1 && currentBytes + pieceBytes + 1 > budgetBytes) {", + " if (currentBytes > 1 && currentBytes + pieceBytes > budgetBytes) {", + "a piece never ends over the budget because of the bracket that closes it", + ), + + # The send path joins the pieces into one write. The bytes that reach the socket are the + # same bytes, and the length is the same length — but the body is handed over in one piece + # again, which is the half of the change that is about memory. + ( + "the send path joins the pieces into one write", + " for (const piece of pieces) response.write(piece);", + " response.write(Buffer.concat(pieces));", + "and the pieces are what reached the socket", + ), + + # The send path hands the socket a string instead of the bytes. The bytes that reach it are + # the same bytes, and the length is the same length — but the body is a string again. + ( + "the send path hands the socket a string instead of the bytes", + " for (const piece of pieces) response.write(piece);", + " for (const piece of pieces) response.write(piece.toString('utf8'));", + "the socket was handed the pieces as bytes, not as a string", + ), + + # The send path changes the bytes on the way out without changing their length. The body is + # still parseable, still holds the same number of records, and still declares the same + # length — and it is a different body. Only a byte-for-byte comparison sees it. + ( + "the send path changes the bytes without changing their length", + " for (const piece of pieces) response.write(piece);", + " for (const piece of pieces) response.write(Buffer.from(piece.toString('utf8').replaceAll('user', 'xxxx'), 'utf8'));", + "the bytes that reached the socket are the bytes of the body", + ), + + # The live body carries the wrong records. The count is the same, the body still parses, and + # the declared length is still the length of what was written — and the records in it are not + # the ones the file holds. This is what gives the live section's own assertion an owner: + # without a case that breaks it, that assertion could stop being true while the count and the + # parse stayed green. + ( + "the live body carries records that are not the ones the file holds", + " id: messageId,", + " id: messageId + 'x',", + "the bytes on the wire carry the records the file holds", + ), + # The route goes back to one string. The helper stays correct, so every unit assertion above # still passes, and the memory is exactly as bad as it was before. ( diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs index 77667941..d30e14b7 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs @@ -25,6 +25,14 @@ * disagrees with, and the page hangs waiting for a body that ended early. * 5. the route must USE the pieces, and the cache must KEEP them. A helper that is correct and * a route that does not call it are both green, and the memory is exactly as bad as before. + * 6. the socket has to be handed the PIECES, as bytes, and the bytes that arrive have to be + * the bytes of the body. A send path that joins the pieces into one write brings the peak + * this change was made to remove back, and one that changes the bytes without changing + * their length is a body the page cannot tell from the right one. Comparing the LENGTHS + * sees neither. + * 7. the closing bracket has to be counted against the budget. It is one byte and it lands on + * the piece that holds the last element, so a piece that held two elements and ended + * exactly at the budget came out one byte over it. * * The live section at the end is the wiring: it asks a real service for a real body and checks * that the bytes on the wire are the bytes the serializer produced, on the first read and on the @@ -161,6 +169,19 @@ assert('an element bigger than the budget becomes a piece of its own', huge.length === 3 && huge[1].length > BUDGET); check('and it is still the same bytes', asBytes(huge).toString('utf8'), JSON.stringify(HUGE)); +// The bracket that closes the array is counted against the budget. The element below is sized +// so that the array ENDS exactly at the budget: with the bracket left out of the count, the +// piece that holds both elements comes out one byte over it, and the budget did not bound the +// piece it was supposed to bound. This is the fixture that sees it, and the one above cannot: +// its piece holds a single element, and a single element is the one thing the budget cannot +// bound. +const BOUNDARY_PAYLOAD = { turns: ['x'.repeat(BUDGET - 5), 0] }; +const boundary = serializeTrajectoryBody(BOUNDARY_PAYLOAD, BUDGET); +assert('a piece never ends over the budget because of the bracket that closes it', + boundary.every((piece) => piece.length <= BUDGET)); +check('and the bytes are the same bytes', asBytes(boundary).toString('utf8'), + JSON.stringify(BOUNDARY_PAYLOAD)); + /* ===================================================================== * 4 — the declared length is the sum of the pieces * ===================================================================== */ @@ -181,7 +202,22 @@ const declared = Number(chunked.headers['content-length']); const sent = asBytes(chunked.written).length; assert('the declared length is the sum of the pieces, not the length of one string', declared === sent && declared === asBytes(largePieces).length); -assert('and the pieces are what reached the socket', sent === asBytes(largePieces).length); +// What reached the socket has to BE the pieces: one write per piece, each of them bytes. +// A send path that joins them into one write brings back the peak this change was made to +// remove, and one that hands over a string does it with the very representation the change +// was made to get rid of. The COUNT is what sees the first, and the TYPE is what sees the +// second -- and a length comparison sees neither. +assert('and the pieces are what reached the socket', + chunked.written.length === largePieces.length); +assert('the socket was handed the pieces as bytes, not as a string', + chunked.written.every((chunk) => Buffer.isBuffer(chunk))); +// And the bytes that arrived have to be the bytes of the body, compared against an +// expectation THIS FILE derives itself -- its own JSON.stringify of the payload -- rather +// than against whatever the product produced. A body changed on the way out without +// changing its length is the same length and a different body, and the page cannot tell it +// from the right one. +assert('the bytes that reached the socket are the bytes of the body', + asBytes(chunked.written).equals(Buffer.from(JSON.stringify(LARGE), 'utf8'))); const asString = fakeResponse(); sendJsonBody(asString, 200, JSON.stringify(LARGE), {}); @@ -275,6 +311,14 @@ try { parsed !== null && Array.isArray(parsed.turns) ? parsed.turns.flatMap((turn) => turn.records || []).length === 12000 : false); + // The bytes on the wire have to be the ones the FILE holds. The route builds its own + // payload, so the oracle here is the file this guard wrote: the ids are in it, and the + // body has to carry them. A count and a parse are not enough -- a body can hold the + // right number of records and be the wrong records. + const idsInBody = ['m0', 'm11999'].every((id) => ( + coldBytes.includes(Buffer.from('"id":"' + id + '"', 'utf8')) + )); + assert('the bytes on the wire carry the records the file holds', idsInBody); // The warm read is the one that comes out of the cache, and the cache keeps PIECES now. const warm = await fetch(base + '/api/trajectory?session=' + encodeURIComponent(SID)); From 63589d69ca04fa00f9294be7f194b6c5d099ac7f Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 17:52:41 +0800 Subject: [PATCH 70/95] Write N has to be piece N, and the records are compared one by one A read of the committed bytes left two owners weaker than the promises they carry, and both of them are fixed here. - the socket check asked whether the NUMBER of writes matched the number of pieces. A send path that keeps that number and cuts the pieces up differently passes it: the bytes are the same bytes, every write is still bytes, and the body still declares the same length, but the first write is not the first piece. The check compares the writes to the pieces one for one now, and a write has to BE the piece, not merely be the same number of them. - the live check compared two of the record ids, so a body can carry the right number of records, the right first and the right last record, and the wrong records in between, with the count and the parse both green. The whole sequence of ids is compared now, against the ids the file was written with. Each of the two has a mutation of its own, and both mutations are the reproductions that were run against the fixed code. The two owners are separate: under the break that belongs to one of them the other stays green. The control for that guard went from twelve cases to fourteen, and the suite passes with every one of them caught by name. --- .../tests/mutate-trajectory-body.py | 29 +++++++++++++++++++ .../tests/verify-trajectory-body.mjs | 28 ++++++++++++++---- 2 files changed, 52 insertions(+), 5 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py index dc15f1bb..98f15735 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py @@ -108,6 +108,24 @@ "the socket was handed the pieces as bytes, not as a string", ), + # The send path keeps the NUMBER of writes and cuts the pieces up differently. The bytes + # that reach the socket are the same bytes, every write is still a Buffer, and the body + # still declares the same length — but the first two writes are not the first two pieces. + # A count cannot see this, and neither can the total bytes. + ( + "the send path cuts the pieces up differently and writes the same number of times", + " for (const piece of pieces) response.write(piece);", + """ if (pieces.length > 1) { + const pair = Buffer.concat(pieces.slice(0, 2)); + response.write(pair.subarray(0, 1)); + response.write(pair.subarray(1)); + for (const piece of pieces.slice(2)) response.write(piece); + } else { + for (const piece of pieces) response.write(piece); + }""", + "and the pieces are what reached the socket", + ), + # The send path changes the bytes on the way out without changing their length. The body is # still parseable, still holds the same number of records, and still declares the same # length — and it is a different body. Only a byte-for-byte comparison sees it. @@ -130,6 +148,17 @@ "the bytes on the wire carry the records the file holds", ), + # The live body carries ONE wrong record, in the middle. The count is the same, the body + # still parses, the first record and the last record are the right ones — and everything + # in between is not. A check that looks at two of the ids cannot see this, and neither + # can the count. + ( + "the live body carries one record that is not the one the file holds", + " id: messageId,", + " id: messageId === 'm6000' ? 'x6000' : messageId,", + "the bytes on the wire carry the records the file holds", + ), + # The route goes back to one string. The helper stays correct, so every unit assertion above # still passes, and the memory is exactly as bad as it was before. ( diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs index d30e14b7..a1032eb5 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs @@ -207,8 +207,15 @@ assert('the declared length is the sum of the pieces, not the length of one stri // remove, and one that hands over a string does it with the very representation the change // was made to get rid of. The COUNT is what sees the first, and the TYPE is what sees the // second -- and a length comparison sees neither. +// +// The correspondence has to be one for one and not just the same NUMBER of writes: a send path +// that cut the pieces up differently and wrote the same number of times would keep the count, +// the type and the total bytes, and would still be handing the socket something else. So the +// check is the objects themselves -- write N has to BE piece N -- which pins the boundaries +// and the lengths as well, because the object carries them. assert('and the pieces are what reached the socket', - chunked.written.length === largePieces.length); + chunked.written.length === largePieces.length + && chunked.written.every((chunk, index) => chunk === largePieces[index])); assert('the socket was handed the pieces as bytes, not as a string', chunked.written.every((chunk) => Buffer.isBuffer(chunk))); // And the bytes that arrived have to be the bytes of the body, compared against an @@ -251,9 +258,13 @@ const sessionDir = join(home, 'v2', 'sessions', '2026', '10', '22', '00-00-00-se + Buffer.from(SID, 'utf8').toString('base64url')); await mkdir(sessionDir, { recursive: true }); const rows = []; +// The ids the file holds, in order. The live section compares the body's record ids against +// this list, so the oracle is the file this guard wrote rather than two names typed by hand. +const idsInTheFile = []; // Enough records that the body on the wire is bigger than one piece, so the live section // exercises the path this change is about and not only the small-body case. for (let index = 0; index < 12000; index += 1) { + idsInTheFile.push('m' + index); rows.push(JSON.stringify({ message_id: 'm' + index, turn_id: 't' + (index >> 3), @@ -315,10 +326,17 @@ try { // payload, so the oracle here is the file this guard wrote: the ids are in it, and the // body has to carry them. A count and a parse are not enough -- a body can hold the // right number of records and be the wrong records. - const idsInBody = ['m0', 'm11999'].every((id) => ( - coldBytes.includes(Buffer.from('"id":"' + id + '"', 'utf8')) - )); - assert('the bytes on the wire carry the records the file holds', idsInBody); + // + // ALL of the ids, in order, and not two of them: a body can hold the right number of + // records, the right FIRST and LAST record, and the wrong records in between, and the + // count and the parse would both stay green. So the whole sequence is compared, against + // the ids the file was written with. + const idsOnTheWire = parsed !== null && Array.isArray(parsed.turns) + ? parsed.turns.flatMap((turn) => turn.records || []).map((record) => record.id) + : []; + assert('the bytes on the wire carry the records the file holds', + idsOnTheWire.length === idsInTheFile.length + && idsOnTheWire.every((id, index) => id === idsInTheFile[index])); // The warm read is the one that comes out of the cache, and the cache keeps PIECES now. const warm = await fetch(base + '/api/trajectory?session=' + encodeURIComponent(SID)); From 7b28b14417c730d82668f738201953e56da0520e Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 18:46:42 +0800 Subject: [PATCH 71/95] The byte owner has an escape of its own, and the overlap is spelled out A read of the committed bytes asked for two things: keep the piece correspondence as an identity check, and give the byte comparison a break that only it catches. The control has that break now. The send path changes the bytes of the piece IN PLACE -- the same object, the same length -- so the correspondence stays green (the object did not change) and the type stays green (it is still bytes), and the byte comparison is the only thing that goes red. Every occurrence is replaced, and not only the first one: replacing the first is not idempotent, and the warm read sends the same pieces again, so it would replace the next one and the cold body and the warm body would stop being the same bytes. The guard says the rest of it in prose. The three owners are not independent of each other and they are not meant to be: the correspondence is an identity check, so a send path that hands the socket a different object -- a copy, a string, a modified buffer -- is not writing piece N and goes red there as well, whatever else it breaks. That is the strength of the check rather than a false alarm: a body the socket receives as copies is the memory shape this change was made to remove. Each of the three is judged on its own property, and each of the three has a break that only it catches. The control for that guard went from fourteen cases to fifteen, and the suite passes with every one of them caught by name. --- .../tests/mutate-trajectory-body.py | 27 +++++++++++++++++++ .../tests/verify-trajectory-body.mjs | 13 +++++++++ 2 files changed, 40 insertions(+) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py index 98f15735..f6de5873 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-trajectory-body.py @@ -136,6 +136,33 @@ "the bytes that reached the socket are the bytes of the body", ), + ( + # The send path changes the bytes of the piece IN PLACE: the same object, the same length, + # different bytes. The correspondence stays green (the object did not change), the type + # stays green (it is still bytes) -- and the byte comparison is the only thing that goes + # red. That is what makes this the byte owner's OWN escape: the other two owners have + # theirs, and this one is the one that was missing. + # + # Every occurrence is replaced, and not only the first one, because replacing the first is + # not idempotent: the warm read sends the SAME pieces again, so it would replace the next + # occurrence, and the cold body and the warm body would stop being the same bytes -- which + # is a different owner's failure and would hide this one. The replacement is the same + # length, so the declared length does not move either. + "the send path changes the bytes of the piece in place, without changing the object", + " for (const piece of pieces) response.write(piece);", + """ for (const piece of pieces) { + if (Buffer.isBuffer(piece)) { + let at = piece.indexOf('user'); + while (at >= 0) { + piece.write('xxxx', at, 'utf8'); + at = piece.indexOf('user', at + 4); + } + } + response.write(piece); + }""", + "the bytes that reached the socket are the bytes of the body", + ), + # The live body carries the wrong records. The count is the same, the body still parses, and # the declared length is still the length of what was written — and the records in it are not # the ones the file holds. This is what gives the live section's own assertion an owner: diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs index a1032eb5..a7728c73 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-trajectory-body.mjs @@ -226,6 +226,19 @@ assert('the socket was handed the pieces as bytes, not as a string', assert('the bytes that reached the socket are the bytes of the body', asBytes(chunked.written).equals(Buffer.from(JSON.stringify(LARGE), 'utf8'))); +// The three owners above are not independent of each other, and they are not meant to be. The +// correspondence check is an IDENTITY check, so a send path that hands the socket a different +// object -- a copy, a string, a modified buffer -- is not writing piece N, and it goes red there +// as well, whatever else it breaks. That is the strength of the check rather than a false alarm: +// a body the socket receives as copies is the memory shape this change was made to remove. +// +// So the three are judged one by one, each on its own property: the correspondence is green when +// every write IS its piece, the type is green when every write is bytes, and the bytes are green +// when what arrived is what the body says. Each of the three has a break that only it catches, +// and the control has one for each: a send path that cuts the pieces up differently (the +// correspondence), a serializer that hands the pieces back as strings (the type), and a send path +// that changes the bytes of the piece IN PLACE, without changing the object (the bytes). + const asString = fakeResponse(); sendJsonBody(asString, 200, JSON.stringify(LARGE), {}); assert('a body that is one string still declares its own length', From 0dc919029ad144370eeedd83b2a2ef9fe02a5575 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 18:46:43 +0800 Subject: [PATCH 72/95] The timer that releases the rows runs on the interval the cache reports Three things about the parsed-rows cache were open, and they are fixed here. - the promise to a reader -- the rows are gone no later than one sweep interval after the limit -- had half an owner. The number in the prose was compared with the limit and with the interval the cache reports, but nothing watched the call that starts the timer, so an interval multiplied by two left every check green while the real release moved to "the limit plus twice the interval" and the promise went on saying twelve and a half minutes. The timer is not reachable from outside the cache, so the check reads the call that starts it: it has to take the interval the cache itself reports, with nothing done to it on the way. It has a mutation of its own, and it is caught by name. - the assertion that owns that number said "no later than", which is an upper bound, while what it compares is an equality. The name says what it compares now, and the two mutations that break it name the new one. - the control did not check that its anchors occur exactly once, so a duplicated anchor would have been edited in the wrong place, and the case would have passed or failed for a reason that had nothing to do with the rule it is about. It counts them now, the same way the control for the body does, and a case whose anchor is not there exactly once is reported as missed instead of being run. The interval is written into the comment of the constant as well, so a reader meets the timer and the promise in the same place. The control for that guard went from twenty-six cases to twenty-seven, and the suite passes with every one of them caught by name. --- .../mmc-trajectory/miniapp/node/server.mjs | 63 ++++++++++++---- .../tests/mutate-parsed-rows.py | 75 +++++++++++++++++-- .../tests/verify-parsed-rows.mjs | 69 ++++++++++++++++- 3 files changed, 185 insertions(+), 22 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs index 17b0e08f..7abc0aa2 100644 --- a/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs +++ b/plugins/avatasia/mmc-trajectory/miniapp/node/server.mjs @@ -179,9 +179,18 @@ const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2; * * **WHEN THEY COME BACK IS NOT THE SAME MOMENT AS WHEN THEY EXPIRE.** The limit is when the rows * stop being wanted; the release happens on the next sweep, so the rows are gone no later than - * one sweep interval after the limit — twelve and a half minutes with the shipped values, and - * never longer than a quarter of the limit beyond it. The number to promise a reader is the - * second one, and it is asserted: the sweep interval is at most a quarter of the limit. + * one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped + * values. The number to promise a reader is that one, and it is the number in brackets: it is + * the limit plus the interval, and the two have to move together. The interval is at most a + * quarter of the limit, so the release is no later than a quarter of the limit after it; and + * when the limit is so small that a quarter of it is less than a millisecond, the interval is + * not finer than a millisecond and the release is at most a millisecond late. Both of those are + * asserted. + * + * The number in brackets is not a loose upper bound: it IS the limit plus the interval, to the + * millisecond. An assertion of its own compares it as an equality with the limit the service + * declares and the interval the running cache uses, so this number is watched: changing one of + * the three without the others is caught there, and not by a reader. * * An expiry is a MISS and not a deletion of something still wanted: the next request parses the * file again and the cache fills itself back up. The price of that is one re-parse, and only when @@ -202,9 +211,18 @@ const PARSED_ROWS_CACHE_TTL_MS = 10 * 60 * 1000; * * The relation is enforced where the interval is TAKEN, not only here: the interval is clamped * to a quarter of the limit the cache was given, so "never later than a quarter of the limit - * beyond it" holds for every limit and not only for the shipped one. A constant that could - * outgrow the limit would make the promise true on the day it was written and false after - * somebody changed the limit. + * beyond it" holds for every limit above four milliseconds and not only for the shipped one. A + * constant that could outgrow the limit would make the promise true on the day it was written + * and false after somebody changed the limit. + * + * The timer that runs the sweep is started with this interval, and the cache reports the interval + * it uses, so the promise above is checked against the running cache rather than against two + * numbers in the source. The call that starts the timer is read by a check of its own, so the + * promise cannot drift away from the timer that has to keep it. + * + * The clamp has a floor of one millisecond, because a timer cannot be finer than that. For a + * limit below four milliseconds the floor is the larger of the two, and the release is then at + * most a millisecond late rather than a quarter of the limit late. * * Pinned by tests/verify-parsed-rows.mjs. */ @@ -3656,20 +3674,33 @@ export function parsedRowsKey(filePath, size, mtimeMs, limit) { * expire and the sweeper never runs. (The other reading — "everything expires at once" — would be * a cache that never hits and never releases, which is worse than either.) * + * A sweep interval that is not positive is taken as one millisecond: a timer cannot be finer + * than that, and an interval of zero used to switch the sweeper off, which left the rows to be + * released only when somebody asked for them. + * * @param {number} budget * @param {{ ttlMs?: number, now?: () => number, sweepIntervalMs?: number }} [options] */ export function createParsedRowsCache(budget, options = {}) { const ttlMs = options.ttlMs ?? PARSED_ROWS_CACHE_TTL_MS; const now = options.now ?? (() => Date.now()); - // The interval the sweep actually uses, clamped to a quarter of the limit the cache was - // given. The clamp is the rule, not a convenience: without it the promise to a reader — the - // rows are gone no later than a quarter of the limit after they expire — would hold for the - // shipped limit and for nothing else, and the limit is an option a caller may set. - const sweepIntervalMs = Math.min( - options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS, - Math.max(1, Math.floor(ttlMs / 4)), - ); + // The interval the sweep actually uses, taken between two bounds. + // + // The limit bound is the rule the promise to a reader is made of: the interval is at most a + // quarter of the limit the cache was given, so the rows are gone no later than a quarter of + // the limit after they expire. The clamp is the rule and not a convenience: without it the + // promise would hold for the shipped limit and for nothing else, and the limit is an option a + // caller may set. Its floor is one millisecond, because a timer cannot be finer than that — + // so for a limit below four milliseconds the release is at most a millisecond late rather + // than a quarter of a tiny limit late. + // + // The asked bound is that same floor applied to what the caller asked for. A caller who asks + // for an interval that is not positive used to switch the sweeper off, and a cache whose + // sweeper is off only releases the rows when somebody asks for them — the one moment they + // are wanted again. The floor keeps the sweeper running. + const limitBoundMs = Math.max(1, Math.floor(ttlMs / 4)); + const askedBoundMs = Math.max(1, options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS); + const sweepIntervalMs = Math.min(limitBoundMs, askedBoundMs); /** @type {Map} */ const entries = new Map(); let bytes = 0; @@ -3710,6 +3741,10 @@ export function createParsedRowsCache(budget, options = {}) { }; const startSweeper = () => { + // The interval is floored at a millisecond where it is taken, so the third condition cannot + // fire today. It stays because it is the difference between the two ways a non-positive + // interval could hurt: a timer that runs as fast as the platform allows, or no timer at all + // — and no timer means the rows are released only when somebody asks for them. if (sweeper !== null || ttlMs <= 0 || sweepIntervalMs <= 0) return; sweeper = setInterval(sweep, sweepIntervalMs); // An interval that is not unref'd keeps the event loop alive after the server closes, and the diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py index 6435dd3c..a5ba5a53 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py @@ -20,6 +20,10 @@ release that only ran on a request would happen exactly when the reader comes back; - that release is longer than one poll, or an open tab would lose its rows between polls; - and the timer that does it never keeps the process alive. + - the sweep interval is floored at a millisecond on both sides: the quarter of a limit below + four milliseconds is finer than a timer can be, and an interval a caller asks for that is not + positive used to switch the sweeper off — which left the rows to be released only when + somebody asked for them. """ import json import re @@ -206,13 +210,33 @@ # limit where it is taken, so a longer constant is simply never used. The rule itself is # still owned — by this case. "the sweep interval is taken without being clamped to the limit", - """ const sweepIntervalMs = Math.min( - options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS, - Math.max(1, Math.floor(ttlMs / 4)), - );""", - " const sweepIntervalMs = options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS;", + """ const limitBoundMs = Math.max(1, Math.floor(ttlMs / 4)); + const askedBoundMs = Math.max(1, options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS); + const sweepIntervalMs = Math.min(limitBoundMs, askedBoundMs);""", + """ const askedBoundMs = Math.max(1, options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS); + const sweepIntervalMs = askedBoundMs;""", 'the sweep interval the cache uses is at most a quarter of the limit', ), + ( + # The floor of a millisecond is what makes the promise true for a limit below four + # milliseconds: a quarter of such a limit is finer than a timer can be, so the floor is + # the larger of the two. Without the floor the interval would be zero there, and a zero + # interval is not a fast timer — it is no timer at all. + "the limit bound loses its floor of a millisecond", + " const limitBoundMs = Math.max(1, Math.floor(ttlMs / 4));", + " const limitBoundMs = Math.floor(ttlMs / 4);", + 'a limit too small for a quarter of a millisecond still sweeps no finer than a millisecond', + ), + ( + # A caller who asks for an interval that is not positive used to switch the sweeper off, + # and a cache whose sweeper is off only releases the rows when somebody asks for them — + # the one moment they are wanted again. The floor on what the caller asked for is what + # keeps the sweeper running. + "the asked interval loses its floor of a millisecond", + " const askedBoundMs = Math.max(1, options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS);", + " const askedBoundMs = options.sweepIntervalMs ?? PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS;", + 'an interval the caller asks for that is not positive is taken as a millisecond', + ), ( # A time limit of zero means NO time limit. The other reading — everything expires at # once — would be a cache that never hits and never releases. @@ -221,6 +245,38 @@ " const expired = (at) => now() - at >= ttlMs;", 'a time limit of zero means no time limit, not that everything expires at once', ), + ( + # The number the package promises a reader is the limit plus the interval, and it is + # written down in prose where a reader meets it. Change the prose and the code keeps + # its promise while the promise itself says something else — so the prose has to be + # compared with the code, not trusted. + "the promise to a reader says a number the code does not have", + "(750000 ms) with the shipped", + "(600000 ms) with the shipped", + 'the number promised to a reader is the limit plus one sweep interval', + ), + ( + # The other half of the same tie: the limit moves and the promise stays where it was. + # The rows would still be released on a sweep, and the reader would still be promised + # twelve and a half minutes. + "the limit is longer than the promise to a reader", + "const PARSED_ROWS_CACHE_TTL_MS = 10 * 60 * 1000;", + "const PARSED_ROWS_CACHE_TTL_MS = 20 * 60 * 1000;", + 'the number promised to a reader is the limit plus one sweep interval', + ), + + ( + # The promise to a reader is that the rows are gone no later than one sweep interval after + # the limit -- and that holds only if the TIMER runs on that interval. Nothing watched the + # call that starts the timer, so doubling the interval there left every other case green: + # the cache still reported the interval it was asked for, the number in the prose still + # matched that reported interval, and the real release had moved to "the limit plus twice + # the interval" while the promise went on saying twelve and a half minutes. + "the timer that releases the rows runs on twice the interval the cache reports", + " sweeper = setInterval(sweep, sweepIntervalMs);", + " sweeper = setInterval(sweep, sweepIntervalMs * 2);", + 'the timer that releases the rows runs on the interval the cache reports', + ), ] @@ -306,8 +362,13 @@ def main(): unnamed = [] for label, old, new, expect in MUTATIONS: text, newline = read_exact(SERVER) - if old not in text: - print(f"SKIP {label}: anchor not found") + # The anchor has to be there EXACTLY once. A control that replaces the first occurrence of + # a string that occurs twice edits a place it did not mean to edit, and the case then + # passes or fails for a reason that has nothing to do with the rule it is about -- so the + # count is checked here, by the control itself, instead of being counted by somebody + # reading the source. + if text.count(old) != 1: + print(f"SKIP {label}: anchor found {text.count(old)} times") missed.append(label) continue if old == new: diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index 54a0c47e..68c5a4d1 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -362,7 +362,10 @@ assert('loadParsedRows is exported so a test can run the real step', * exactly when the reader comes back, which is the one moment they are wanted again. And the * release lands on a sweep tick, so the rows are gone a sweep interval AFTER the limit, not at * it: the number to promise a reader is the limit plus that interval, and the two have to stay - * in that relation. + * in that relation. Both ends of that interval are floored at a millisecond, because a timer + * cannot be finer than that: an interval a caller asks for that is not positive would + * otherwise switch the sweeper off, and a sweeper that is off releases the rows only when + * somebody asks for them. * ===================================================================== */ // A constant the source declares as an expression — `60 * 1000` — is read by arithmetic rather @@ -375,6 +378,19 @@ const declaredNumber = (source, name) => { return parts.reduce((carry, part) => carry * part, 1); }; +// The doc block above a constant, so a promise the package states in prose can be read and +// compared with what the code does. The block is taken around ONE declaration rather than the +// whole file, because a whole-file match keeps passing after the promise has moved somewhere +// else -- which is the way a rule quietly loses its owner. +const docBlockOf = (source, name) => { + const at = source.indexOf('const ' + name + ' ='); + if (at < 0) return null; + const before = source.slice(0, at); + const start = before.lastIndexOf('/**'); + const end = before.lastIndexOf('*/'); + return start >= 0 && end > start ? before.slice(start, end) : null; +}; + // The proxy is read out of the shipped source rather than repeated here, so the relationship // below is checked against the constant the service actually charges with. const PROXY = declaredNumber(SERVER, 'PARSED_ROWS_HEAP_PER_SOURCE_BYTE'); @@ -466,6 +482,57 @@ assert('an interval the caller asks for is clamped to the limit as well', assert('the shipped default is the interval the service declares', createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP); +// The number a reader is promised: the rows are gone no later than the limit plus one sweep +// interval. The bound is computed here from the limit the service DECLARES and the interval +// the RUNNING cache uses, and it is compared with the number the package states in prose -- so +// a change to one of them without the other is caught here and not by a reader. The promise is +// in prose because that is where a reader meets it, and a promise nobody checks is a promise +// that is true on the day it was written. +const ttlDoc = docBlockOf(SERVER, 'PARSED_ROWS_CACHE_TTL_MS'); +const promisedMs = ttlDoc === null ? null : /\((\d+) ms\)/.exec(ttlDoc); +assert('the number promised to a reader is the limit plus one sweep interval', + promisedMs !== null + && Number(promisedMs[1]) === TTL + createParsedRowsCache(1000).sweepIntervalMs); + +// The promise above holds only if the TIMER runs on that interval: the rows are released on the +// next tick, and the tick is the sweep. Nothing in the package watched the call that starts that +// timer, so an interval multiplied by two left every other check here green while the real +// release moved to "the limit plus twice the interval" -- and the promise to a reader went on +// saying twelve and a half minutes. +// +// The timer is not reachable from outside the cache, so what can be checked is the call that +// starts it: it has to take the interval the cache itself reports, with nothing done to it on the +// way. An interval that is not the reported one is not the interval the number in the prose is +// built from, and the promise would be true of a timer that does not exist. +const sweepTimerCalls = [...SERVER.matchAll(/setInterval\(\s*sweep\s*,\s*([^)]*)\)/g)]; +assert('the timer that releases the rows runs on the interval the cache reports', + sweepTimerCalls.length === 1 && sweepTimerCalls[0][1].trim() === 'sweepIntervalMs'); + +// The quarter has a floor of a millisecond, because a timer cannot be finer than that. For a +// limit below four milliseconds the floor is the larger of the two, so the release is then at +// most a millisecond late rather than a quarter of a tiny limit late. The limit below is +// deliberately tiny: with a limit of a second the floor never applies, and a case that cannot +// reach the floor would pass without anything having been decided. +const tinyLimit = createParsedRowsCache(1000, { ttlMs: 3 }); +assert('a limit too small for a quarter of a millisecond still sweeps no finer than a millisecond', + tinyLimit.sweepIntervalMs === 1); + +// An interval the caller asks for that is not positive is taken as a millisecond. It used to +// switch the sweeper off instead, and a cache whose sweeper is off only releases the rows when +// somebody asks for them -- the one moment they are wanted again. +const nonPositive = createParsedRowsCache(1000, { ttlMs: 1_000, sweepIntervalMs: 0 }); +const negative = createParsedRowsCache(1000, { ttlMs: 1_000, sweepIntervalMs: -5 }); +assert('an interval the caller asks for that is not positive is taken as a millisecond', + nonPositive.sweepIntervalMs === 1 && negative.sweepIntervalMs === 1); + +// And the consequence of that floor: the sweeper still runs, so the rows are still released +// without a request. This one is a real timer, because the sweeper is a timer. +const nonPositiveCache = createParsedRowsCache(1000, { ttlMs: 100, sweepIntervalMs: 0 }); +nonPositiveCache.set('k', val, 500); +await new Promise((resolve) => setTimeout(resolve, 500)); +check('a caller who asks for a non-positive interval still gets the rows released without a request', + nonPositiveCache.bytes, 0); + // A time limit of zero means NO time limit. The other reading — everything expires at once — // would be a cache that never hits and never releases, which is worse than either. const noLimit = createParsedRowsCache(1000, { ttlMs: 0, now: () => clock }); From 20272b4316bccdfc03b94a21303a9a03e0c22c96 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 19:52:44 +0800 Subject: [PATCH 73/95] The timer that releases the rows runs on the interval the cache reports Two owners of the parsed-rows cache did not hold the rule they name, and both are fixed here. The cache itself is not changed: the shipped product behaves as it is documented, and what was missing was an owner that could see a break. - the owner of the timer compared the way the call SPELLED the interval, not the interval the timer was given. It looked for one setInterval in the source and required the argument to read as the name the cache reports -- and a local binding that shadows that name keeps the spelling while changing the number. The cache kept reporting the interval it was asked for, the number in the prose kept matching that reported interval, and the real release moved out to nearly fifteen minutes while every check that read the source stayed green. The owner now observes the timer instead: it stands in for the clock's own timer call while the cache runs, records the interval the timer was actually given, and compares that with the interval the cache reports. The rule it names is the relation it now holds. - the owner of a limit of zero covered one half of its promise. The promise has two -- the rows never expire AND the sweeper never runs -- and the owner checked a hit and the retained bytes, which say nothing about whether a timer was started. Moving the boundary of the sweeper's start condition by one comparison let the sweeper run under a limit of zero; and since nothing in such a cache ever expires, that timer would scan it for ever. The absence of a sweeper now has its own owner, which observes the timer calls of a zero-limit cache and requires that there are none. Each of the two has a mutation of its own, and each is caught by name. Three cases were added: a local binding that shadows the reported name and doubles the interval the timer really gets, one that shadows it with a number of its own, and one that lets a limit of zero start the sweeper. The two owners were measured against each other, one break at a time: the interval breaks leave the zero-limit owner green, and the zero-limit break leaves the interval owner green. The break that replaces the interval with a number of its own also turns the live release assertions red, because that break really does stop the rows from being released without a request -- it is reported as breaking more than one rule, not as breaking the one it was measured for. The guard went from eighty-nine assertions to ninety. --- .../tests/mutate-parsed-rows.py | 31 ++++++++++++ .../tests/verify-parsed-rows.mjs | 49 ++++++++++++++++--- 2 files changed, 74 insertions(+), 6 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py index a5ba5a53..9d79aef4 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py @@ -277,6 +277,37 @@ " sweeper = setInterval(sweep, sweepIntervalMs * 2);", 'the timer that releases the rows runs on the interval the cache reports', ), + ( + # The rule is about the NUMBER the timer is given, not about the way the call spells it. + # A local binding that shadows the name keeps the spelling and doubles the number that is + # actually passed: the cache kept reporting the interval it was asked for, the number in + # the prose kept matching that reported interval, and the real release moved out to nearly + # fifteen minutes -- while every check that only read the source stayed green. + "a shadowed binding doubles the interval the timer really gets", + " sweeper = setInterval(sweep, sweepIntervalMs);", + " { const sweepIntervalMs = Math.min(limitBoundMs, askedBoundMs) * 2;\n" + " sweeper = setInterval(sweep, sweepIntervalMs); }", + 'the timer that releases the rows runs on the interval the cache reports', + ), + ( + # The same escape with a number that is not derived from the interval at all: the spelling + # is still the reported name, and the value the timer gets is not the reported interval. + "a shadowed binding replaces the interval with a number of its own", + " sweeper = setInterval(sweep, sweepIntervalMs);", + " { const sweepIntervalMs = 300000;\n" + " sweeper = setInterval(sweep, sweepIntervalMs); }", + 'the timer that releases the rows runs on the interval the cache reports', + ), + ( + # Zero means no limit AND no sweeper. The owner checked the hit and the retained bytes but + # never the timer, so moving the boundary of the start condition by one comparison let the + # sweeper run under a zero limit -- and nothing in such a cache ever expires, so that timer + # would scan it for ever. + "a zero limit starts the sweeper it promised not to start", + " if (sweeper !== null || ttlMs <= 0 || sweepIntervalMs <= 0) return;", + " if (sweeper !== null || ttlMs < 0 || sweepIntervalMs <= 0) return;", + 'a time limit of zero starts no sweeper, because nothing in it ever expires', + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index 68c5a4d1..0ef67054 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -500,13 +500,39 @@ assert('the number promised to a reader is the limit plus one sweep interval', // release moved to "the limit plus twice the interval" -- and the promise to a reader went on // saying twelve and a half minutes. // -// The timer is not reachable from outside the cache, so what can be checked is the call that -// starts it: it has to take the interval the cache itself reports, with nothing done to it on the -// way. An interval that is not the reported one is not the interval the number in the prose is -// built from, and the promise would be true of a timer that does not exist. -const sweepTimerCalls = [...SERVER.matchAll(/setInterval\(\s*sweep\s*,\s*([^)]*)\)/g)]; +// What has to be checked is the NUMBER the timer is given, not the way the call spells it. A +// pattern over the source can only see the spelling, and a local binding that shadows the name +// keeps that spelling while doubling the number that is actually passed: the cache kept reporting +// the interval it was asked for, the number in the prose kept matching that reported interval, and +// the real release moved out to nearly fifteen minutes. +// +// The timer is not reachable from outside the cache, so the only way to see the interval it really +// got is to stand in for the clock's own timer call while the cache runs. +function withRecordedIntervals(run) { + const recorded = []; + const realSetInterval = globalThis.setInterval; + const realClearInterval = globalThis.clearInterval; + globalThis.setInterval = (callback, delay) => { + recorded.push(delay); + return { unref() { return this; }, ref() { return this; }, hasRef() { return false; } }; + }; + globalThis.clearInterval = () => {}; + try { + run(); + } finally { + globalThis.setInterval = realSetInterval; + globalThis.clearInterval = realClearInterval; + } + return recorded; +} + +let timedCache = null; +const timerCalls = withRecordedIntervals(() => { + timedCache = createParsedRowsCache(1000, { ttlMs: TTL }); + timedCache.set('k', val, 500); +}); assert('the timer that releases the rows runs on the interval the cache reports', - sweepTimerCalls.length === 1 && sweepTimerCalls[0][1].trim() === 'sweepIntervalMs'); + timerCalls.length === 1 && timerCalls[0] === timedCache.sweepIntervalMs); // The quarter has a floor of a millisecond, because a timer cannot be finer than that. For a // limit below four milliseconds the floor is the larger of the two, so the release is then at @@ -542,6 +568,17 @@ assert('a time limit of zero means no time limit, not that everything expires at noLimit.get('k') !== null); check('and the rows are still held', noLimit.bytes, 500); +// The other half of that promise is that NO SWEEPER runs. In a cache in which nothing ever +// expires, a timer would scan it for ever, because the sweep only stops when the cache empties +// and this one never does. So that half has to be owned by watching the TIMER, not by watching +// the hit: the hit stays true and the bytes stay counted either way. +const zeroTimerCalls = withRecordedIntervals(() => { + const zeroCache = createParsedRowsCache(1000, { ttlMs: 0, now: () => clock }); + zeroCache.set('k', val, 500); +}); +assert('a time limit of zero starts no sweeper, because nothing in it ever expires', + zeroTimerCalls.length === 0); + await rm(root, { recursive: true, force: true }); /* ===================================================================== */ From 803e10f550a37d401f86c64d15cda5bba05560f2 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 20:48:22 +0800 Subject: [PATCH 74/95] The two owners that read the source now watch the thing they are about Two owners in this package decided the truth of a rule by matching text against the source, and a match on text can be satisfied by text that is not the thing the rule is about. Both are fixed here, and the product is not changed: what was missing was an owner that could see a break. - the owner that the sweeper's timer does not keep the process alive looked for the text of an unref call anywhere in the service. An unrelated function with a parameter of the same name satisfied it -- a function that is never called -- while the real timer stayed referenced and the process would have waited for it. The owner now watches the handle the cache was given: it stands in for the clock's own timer call while the cache runs, and the timer the cache created has to be one that does not hold the process open. That is the platform's own answer about the handle, not this file's reading of the text. The handle is cleared as soon as the window closes, so no timer is left behind and its callback can never run. - the owner that the parsed-row cache exposes clear() looked for the shape of a clear method after the name of the cache factory, and the search could run on past the cache's own body and find that shape in an unrelated object further down the file: the parsed cache's own clear() could be renamed -- and the dispose path would then throw when it reached the cache it meant to release -- while another clear() supplied the text. The owner now asks the cache: it creates one, puts rows in it, calls clear(), and checks that the method is there and that the cache is empty afterwards. Each of the two has a mutation of its own that reproduces the escape it was found by -- one that puts an unrelated unref where the real one was, one that renames the method and gives an unrelated object a clear() of its own -- and each is caught by name. The two controls went from thirty cases to thirty-one and from fifty-four to fifty-five, and every case in both is still caught by name. The two owners were measured against each other, one break at a time: each of the four breaks trips exactly one assertion, and the file is restored byte for byte after each one. The guards went from eighty-nine assertions to ninety and from fifty to fifty; no assertion was added or removed, only what each one looks at. --- .../tests/mutate-parsed-rows.py | 12 +++ .../mmc-trajectory/tests/mutate-review16.py | 15 ++++ .../tests/verify-parsed-rows.mjs | 79 +++++++++++++------ .../mmc-trajectory/tests/verify-review16.mjs | 14 +++- 4 files changed, 93 insertions(+), 27 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py index 9d79aef4..7ec10ac5 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py @@ -183,6 +183,18 @@ "", 'the sweeper does not keep the process alive', ), + ( + # The same rule, but with a decoy: the owner used to be a pattern over the source, and a + # pattern cannot tell the cache's own handle from an unrelated one's. A function that is + # never called, with a parameter of the same name, satisfied it -- while the real timer was + # left referenced and the process would have waited for it. The owner now watches the handle + # the cache was given, so the decoy has to go with the real unref. + "an unrelated unref stands in for the sweeper's own", + " if (typeof sweeper.unref === 'function') sweeper.unref();\n", + " const unrelatedTimer = (sweeper) => { sweeper.unref(); };\n" + " if (typeof sweeper.ref === 'function') sweeper.ref();\n", + 'the sweeper does not keep the process alive', + ), ( # The page polls every 8 seconds. A limit shorter than one poll would make an open tab # lose its rows between two polls and re-parse the whole lineage for nothing. diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py b/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py index a1d896f6..73d00222 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-review16.py @@ -251,6 +251,21 @@ " reset() {\n entries.clear();\n bytes = 0;\n stopSweeper();\n },", "the parsed-row cache really does expose clear()"), + # The same rename, with a decoy: the owner used to be a pattern over the source, and the + # pattern could be satisfied by an unrelated cache's method further down the file. The parsed + # cache's own clear() could be renamed -- and the dispose path would throw when it reached the + # cache it meant to release -- while another clear() supplied the text. The owner now asks the + # cache, so the decoy has to go with the rename. + ("an unrelated clear() stands in for the parsed-row cache's own", + " clear() {\n entries.clear();\n bytes = 0;\n stopSweeper();\n },", + " reset() {\n entries.clear();\n bytes = 0;\n stopSweeper();\n },\n" + " unrelatedCache() {\n" + " const entries = new Map();\n" + " let bytes = 0;\n" + " return { clear() { entries.clear(); bytes = 0; } };\n" + " },", + "the parsed-row cache really does expose clear()"), + # ---- 6. the empty state points at inert controls again ---- ("the empty state goes back to one hint for both modes", "var anomalyOn = state.anomaly.rules.length > 0;", diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index 0ef67054..5f9a0d6b 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -51,6 +51,41 @@ function assert(name, cond) { else failures.push(name); } +// The timer the cache starts is not reachable from outside it, so the only way to see the +// interval it really got -- and the only way to see whether it holds the process open -- is to +// stand in for the clock's own timer call while the cache runs. +// +// The handle handed to the cache is a REAL timer handle, so its reference state is the platform's +// own answer and not this file's opinion of it. The handle is cleared as soon as the window +// closes, so no timer is left behind, and its callback can never run: the window is synchronous +// and the event loop does not turn inside it. +// +// A pattern over the source cannot do this. It can see that the text says `sweeper.unref()`, but +// not that the handle the cache was given is the one that was unref'd -- and an unrelated +// function with a parameter of the same name used to satisfy it while the real timer stayed +// referenced. +function withRecordedTimers(run) { + const recorded = []; + const handles = []; + const realSetInterval = globalThis.setInterval; + const realClearInterval = globalThis.clearInterval; + globalThis.setInterval = (callback, delay, ...rest) => { + recorded.push(delay); + const handle = realSetInterval(callback, delay, ...rest); + handles.push(handle); + return handle; + }; + globalThis.clearInterval = () => {}; + try { + run(); + } finally { + globalThis.setInterval = realSetInterval; + globalThis.clearInterval = realClearInterval; + for (const handle of handles) realClearInterval(handle); + } + return { recorded, handles }; +} + const LIMIT = 64 * 1024 * 1024; const root = await mkdtemp(join(tmpdir(), 'parsed-rows-')); @@ -459,10 +494,19 @@ assert('the time limit is longer than one poll, so an open tab never loses its r assert('the time limit is at least five minutes, so stepping away from the tab does not cost a re-parse', TTL >= 5 * 60 * 1000); -// And the sweeper must never be the reason the process is still running. The name is the -// sweeper's own, because a whole-file pattern would keep passing after some OTHER timer in the -// service grew an unref of its own — and the rule would quietly lose its owner. -assert('the sweeper does not keep the process alive', /sweeper\.unref\(\)/.test(SERVER)); +// And the sweeper must never be the reason the process is still running. The owner of that rule +// used to be a pattern over the source, and a pattern cannot tell the cache's own handle from an +// unrelated one's: a function elsewhere with a parameter of the same name satisfied it while the +// real timer stayed referenced. So the owner watches the handle the cache was given: the timer it +// created has to be one that does not hold the process open. +const unrefWindow = withRecordedTimers(() => { + const unrefCache = createParsedRowsCache(1000, { ttlMs: 1_000 }); + unrefCache.set('k', val, 500); +}); +assert('the sweeper does not keep the process alive', + unrefWindow.handles.length === 1 + && typeof unrefWindow.handles[0].hasRef === 'function' + && unrefWindow.handles[0].hasRef() === false); // The release lands on a sweep tick, so the rows are gone a sweep interval AFTER the limit. The // promise to a reader is that interval added on, and it only stays a promise while the interval @@ -508,31 +552,14 @@ assert('the number promised to a reader is the limit plus one sweep interval', // // The timer is not reachable from outside the cache, so the only way to see the interval it really // got is to stand in for the clock's own timer call while the cache runs. -function withRecordedIntervals(run) { - const recorded = []; - const realSetInterval = globalThis.setInterval; - const realClearInterval = globalThis.clearInterval; - globalThis.setInterval = (callback, delay) => { - recorded.push(delay); - return { unref() { return this; }, ref() { return this; }, hasRef() { return false; } }; - }; - globalThis.clearInterval = () => {}; - try { - run(); - } finally { - globalThis.setInterval = realSetInterval; - globalThis.clearInterval = realClearInterval; - } - return recorded; -} - let timedCache = null; -const timerCalls = withRecordedIntervals(() => { +const timerWindow = withRecordedTimers(() => { timedCache = createParsedRowsCache(1000, { ttlMs: TTL }); timedCache.set('k', val, 500); }); assert('the timer that releases the rows runs on the interval the cache reports', - timerCalls.length === 1 && timerCalls[0] === timedCache.sweepIntervalMs); + timerWindow.recorded.length === 1 + && timerWindow.recorded[0] === timedCache.sweepIntervalMs); // The quarter has a floor of a millisecond, because a timer cannot be finer than that. For a // limit below four milliseconds the floor is the larger of the two, so the release is then at @@ -572,12 +599,12 @@ check('and the rows are still held', noLimit.bytes, 500); // expires, a timer would scan it for ever, because the sweep only stops when the cache empties // and this one never does. So that half has to be owned by watching the TIMER, not by watching // the hit: the hit stays true and the bytes stay counted either way. -const zeroTimerCalls = withRecordedIntervals(() => { +const zeroTimerCalls = withRecordedTimers(() => { const zeroCache = createParsedRowsCache(1000, { ttlMs: 0, now: () => clock }); zeroCache.set('k', val, 500); }); assert('a time limit of zero starts no sweeper, because nothing in it ever expires', - zeroTimerCalls.length === 0); + zeroTimerCalls.recorded.length === 0); await rm(root, { recursive: true, force: true }); diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-review16.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-review16.mjs index 89442042..4cd7e649 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-review16.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-review16.mjs @@ -18,6 +18,7 @@ import { tmpdir } from 'node:os'; import { spawnSync } from 'node:child_process'; import { fileURLToPath } from 'node:url'; import { join, basename, dirname, relative, isAbsolute } from 'node:path'; +import { createParsedRowsCache } from '../miniapp/node/server.mjs'; const HERE = fileURLToPath(new URL('.', import.meta.url)); const PKG = join(HERE, '..'); @@ -267,8 +268,19 @@ assert('dispose() clears the parsed-row cache too', assert('dispose() still clears the other two caches', !!disposeBody && /descriptorCache\.clear\(\)/.test(disposeBody[1]) && /trajectoryCache\.clear\(\)/.test(disposeBody[1])); +// The parsed-row cache has to expose clear(), and it has to be THAT cache's clear(). The pattern +// this replaces could be satisfied by an unrelated cache's method further down the file: the parsed +// cache's own clear() could be renamed while another one supplied the text the pattern was looking +// for -- and then the dispose path would throw when it reached the cache it meant to release. So +// the owner asks the cache instead of reading about it. +const parsedRowsCache = createParsedRowsCache(1000); +parsedRowsCache.set('k', { rows: [] }, 500); +const parsedRowsExposesClear = typeof parsedRowsCache.clear === 'function'; +if (parsedRowsExposesClear) parsedRowsCache.clear(); assert('the parsed-row cache really does expose clear()', - /createParsedRowsCache[\s\S]*?clear\(\)\s*\{[^}]*entries\.clear\(\);[^}]*bytes = 0;[^}]*\}/.test(serverJs)); + parsedRowsExposesClear + && parsedRowsCache.bytes === 0 + && parsedRowsCache.size === 0); /* ---------- 6. the empty state names a control that still works ---------- */ From adab5c79b0d9da936c94895b30c9cecaa17ef5d2 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 21:18:18 +0800 Subject: [PATCH 75/95] The two owners that accused each other now rule about their own rule Two owners in this package were standing in for each other: a break in one rule turned an owner of another rule red, so a guard could not tell which rule had actually been broken. Both are fixed here, and the product is not changed. - the owner of the interval the release timer runs on required that a timer had been started at all -- exactly one call, and the interval it was given. A cache whose sweeper was never started therefore reported that owner, even though the rule it names was not broken by that: with no timer there is no timer running on the wrong interval. The owner now rules about the interval and about nothing else. It holds for every timer the cache starts, and holds vacuously when it starts none. - the owner that the sweeper does not keep the process alive required that the cache had been given exactly one handle. The same break reported that owner too, and nothing about a cache that starts no timer can hold the process open. It now holds for every handle the cache was given, and holds vacuously when it was given none. The rule that a sweeper has to start at all is a separate rule, and it keeps an owner of its own: the rows have to be released without a request. That owner still fails when the sweeper is never started, and a mutation of its own now says so -- a case that takes the call that starts the sweeper away and expects the release owner to name itself. The control went from thirty-one cases to thirty-two, and every case in it is still caught by name. The two owners were measured against that break, one at a time: taking the call away trips exactly the three release owners and nothing else, so neither of the two owners is left accusing another rule. The file is restored byte for byte after each break. The guard stayed at ninety assertions; no assertion was added or removed, only what each one looks at. --- .../tests/mutate-parsed-rows.py | 13 +++++++++++ .../tests/verify-parsed-rows.mjs | 22 +++++++++++++------ 2 files changed, 28 insertions(+), 7 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py index 7ec10ac5..c86e1a37 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py @@ -320,6 +320,19 @@ " if (sweeper !== null || ttlMs < 0 || sweepIntervalMs <= 0) return;", 'a time limit of zero starts no sweeper, because nothing in it ever expires', ), + ( + # The sweeper is what releases the rows without a request. Take the call away and the rows + # are held until somebody asks for them -- the shape the whole time limit exists to avoid. + # + # This case belongs to the release owners, and the point of it is that the other two + # owners stay out of it: a cache that starts no timer has no timer running on the wrong + # interval, and nothing that could hold the process open, so neither of those rules is + # broken by this break. If they went red here, one rule would be standing in for another. + "the sweeper is never started, so the rows are held until somebody asks for them", + " startSweeper();\n", + "", + 'the rows are released without a request', + ), ] diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index 5f9a0d6b..fd5352c5 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -497,16 +497,19 @@ assert('the time limit is at least five minutes, so stepping away from the tab d // And the sweeper must never be the reason the process is still running. The owner of that rule // used to be a pattern over the source, and a pattern cannot tell the cache's own handle from an // unrelated one's: a function elsewhere with a parameter of the same name satisfied it while the -// real timer stayed referenced. So the owner watches the handle the cache was given: the timer it -// created has to be one that does not hold the process open. +// real timer stayed referenced. So the owner watches the handle the cache was given: a timer the +// cache created has to be one that does not hold the process open. +// +// A cache that starts no timer at all has nothing that could hold the process open, so that is +// green here: whether a sweeper starts at all is another owner's rule, and this one is not a second +// accusation of it. const unrefWindow = withRecordedTimers(() => { const unrefCache = createParsedRowsCache(1000, { ttlMs: 1_000 }); unrefCache.set('k', val, 500); }); assert('the sweeper does not keep the process alive', - unrefWindow.handles.length === 1 - && typeof unrefWindow.handles[0].hasRef === 'function' - && unrefWindow.handles[0].hasRef() === false); + unrefWindow.handles.every((handle) => typeof handle.hasRef !== 'function' + || handle.hasRef() === false)); // The release lands on a sweep tick, so the rows are gone a sweep interval AFTER the limit. The // promise to a reader is that interval added on, and it only stays a promise while the interval @@ -552,14 +555,19 @@ assert('the number promised to a reader is the limit plus one sweep interval', // // The timer is not reachable from outside the cache, so the only way to see the interval it really // got is to stand in for the clock's own timer call while the cache runs. +// +// What this owner rules about is the INTERVAL a timer runs on -- not whether a timer is started at +// all. A cache that starts no sweeper has no timer running on the wrong interval, and "a sweeper +// has to start" is a different rule with a different owner (the rows have to be released without a +// request). So the check below holds for every timer the cache starts, and holds vacuously when +// it starts none: that keeps the two rules apart instead of making one of them the other's echo. let timedCache = null; const timerWindow = withRecordedTimers(() => { timedCache = createParsedRowsCache(1000, { ttlMs: TTL }); timedCache.set('k', val, 500); }); assert('the timer that releases the rows runs on the interval the cache reports', - timerWindow.recorded.length === 1 - && timerWindow.recorded[0] === timedCache.sweepIntervalMs); + timerWindow.recorded.every((interval) => interval === timedCache.sweepIntervalMs)); // The quarter has a floor of a millisecond, because a timer cannot be finer than that. For a // limit below four milliseconds the floor is the larger of the two, so the release is then at From b6aba3c2090e5b544a165e538f85a6366b1e4091 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 22:08:42 +0800 Subject: [PATCH 76/95] The owners that answered from the text now answer from what the code does The five owners of this package decided the truth of a rule by reading the source, and a source says what the code intends, not what it does. All five are fixed here, and the product is not changed. - the owner of the lifetime the cache gives its rows when nobody tells it one read the declaration of the limit. A cache whose default came from somewhere else than the constant answered every declaration-based check with the documented number while the rows it actually held lived twice as long. The owner now moves a clock until the cache lets the rows go by itself, and the moment it does is the limit. - the promise to a reader is written twice -- as a number of minutes and as a number of milliseconds -- and the two forms have to be the same promise. The owner took the number out of the whole documentation block, so a number left in an annotation beside the promise answered the pattern while the sentence a reader reads said something else. It now takes the sentence that makes the promise, reads the number out of that sentence, and compares the two forms with each other and with the limit the cache really gives its rows. - the owner of the delay the poll waits read the number the page declares. A poll that multiplies the interval it declares loses its rows between polls while every check that reads the declaration stays green. The owner now finds every use of that name in the page, evaluates the expression the timer is really given, and requires the delay to be the declared one. - the owner of the charge the route makes looked for the text of the charge somewhere in the file, which a commented-out line satisfies, and the fixture it ran left the options out -- so it took a different branch of the same function from the route. It now makes the call the route makes and reads back the bytes the cache was charged. - the owner of the proxy constant took the first match of its pattern in the file, and a declaration that is commented out is not a declaration: an annotation left beside the real one answered the pattern while the binding was a string. The owner now requires the line the match is on to declare it -- nothing but whitespace before it, and no block comment open around it. Sixth, a handle that could not answer the question was accepted as one that did not hold the process open: the owner of that rule let a handle pass when it had no way of saying whether it keeps the process alive. It now requires every handle to answer, and to say that it does not hold the process open. Under the clock this package runs on, every recorded handle is a real timer and can answer, so no case in the control tells the two forms apart; the owner was tightened anyway, because the rule it names is about what the handle says. Six mutations were added and each is caught by name: the default doubled, the promise rewritten in one of its two forms, the promised number moved into an obsolete annotation, the poll waiting a thousand times the interval it declares, the route charging the source bytes while a commented-out line supplies the old text, and the proxy declaration commented out with the binding a string. The control went from thirty-two cases to thirty-eight, and every case in it is still caught by name. One of the new cases breaks the page rather than the service, so the control was extended to let a case name the file it breaks; each file is restored byte for byte after its own break. The guard went from ninety assertions to ninety-three. The owners of the lifetime, the promise and the poll now read numbers that were observed rather than declared, and three owners were added: that the limit the service declares is the limit the cache gives its rows, that the poll waits as long as the client declares, and that the promise to a reader is the same promise in both of the forms it is written in. --- .../tests/mutate-parsed-rows.py | 86 ++++++++- .../tests/verify-parsed-rows.mjs | 168 +++++++++++++++++- 2 files changed, 237 insertions(+), 17 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py index c86e1a37..3790b3eb 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py @@ -333,6 +333,68 @@ "", 'the rows are released without a request', ), + ( + # The owners of the lifetime read the DECLARATION of the limit, so a cache whose default + # comes from somewhere else answered them with the documented number while the rows it + # actually held lived twice as long. The owner now watches the limit the cache gives its + # rows when nobody tells it one. + "the default lifetime is doubled, so the declaration no longer says what the cache does", + " const ttlMs = options.ttlMs ?? PARSED_ROWS_CACHE_TTL_MS;", + " const ttlMs = options.ttlMs ?? (PARSED_ROWS_CACHE_TTL_MS * 2);", + 'the limit the service declares is the limit the cache gives its rows', + ), + ( + # The promise to a reader is written twice -- as minutes and as milliseconds -- and the two + # forms have to be the same promise. Change the words and the reader is told one thing + # while the service does another. + "the promise to a reader says ten minutes while the number still says 750000", + "one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped", + "one sweep interval after the limit — ten minutes (750000 ms) with the shipped", + 'the promise to a reader is the same promise in both of the forms it is written in', + ), + ( + # The same promise, read out of the whole block: a number left in an obsolete annotation + # answered the pattern while the sentence a reader reads no longer stated one. + "the number promised to a reader is moved into an obsolete annotation", + "one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped\n" + " * values. The number to promise a reader is that one, and it is the number in brackets: it is", + "one sweep interval after the limit — twelve and a half minutes with the shipped\n" + " * values. Internal obsolete annotation (750000 ms). The number to promise a reader is that one," + " and it is the number in brackets: it is", + 'the number promised to a reader is the limit plus one sweep interval', + ), + ( + # The page polls every TRAJECTORY_POLL_INTERVAL milliseconds -- unless the expression the + # timer is given says otherwise. A poll that waits a thousand times longer than the limit + # it has to survive loses its rows between polls, and the owners that read the declaration + # said nothing about it. This one breaks the PAGE, so it names the file it breaks. + "the poll waits a thousand times the interval it declares", + " }, TRAJECTORY_POLL_INTERVAL);", + " }, TRAJECTORY_POLL_INTERVAL * 1000);", + 'the poll waits as long as the client declares', + "miniapp/client/index.html", + ), + ( + # The route passes options, and a fixture that leaves them out takes a different branch of + # the same function -- so the charge the route really made was never the charge that was + # checked, and a commented-out line satisfied the check that only asked about the text. + "the route charges the source bytes while a commented-out line supplies the old text", + " if (cacheable) cache.set(key, value, Math.ceil(head.size * PARSED_ROWS_HEAP_PER_SOURCE_BYTE));", + " // cache.set(key, value, Math.ceil(head.size * PARSED_ROWS_HEAP_PER_SOURCE_BYTE));\n" + " if (cacheable) cache.set(key, value,\n" + " options === undefined\n" + " ? Math.ceil(head.size * PARSED_ROWS_HEAP_PER_SOURCE_BYTE)\n" + " : head.size);", + 'and the charge the route makes is that one', + ), + ( + # A declaration that is commented out is not a declaration. The pattern read the comment + # and answered with the number the annotation carried while the binding was a string. + "the proxy declaration is commented out and the binding is a string", + "const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;", + "// const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = '2';", + 'the heap proxy is a number the service declares', + ), ] @@ -413,11 +475,18 @@ def main(): return 1 print("baseline clean") - original = SERVER.read_bytes() + originals = {} missed = [] unnamed = [] - for label, old, new, expect in MUTATIONS: - text, newline = read_exact(SERVER) + for label, old, new, expect, *rest in MUTATIONS: + # A case may name the file it breaks. Most break the service; the ones about the PAGE break + # the page, because a control that could only break one file could not hold a case about + # the other half of the package. + target = SERVER if not rest else (ROOT / rest[0]) + if target not in originals: + originals[target] = target.read_bytes() + original = originals[target] + text, newline = read_exact(target) # The anchor has to be there EXACTLY once. A control that replaces the first occurrence of # a string that occurs twice edits a place it did not mean to edit, and the case then # passes or fails for a reason that has nothing to do with the rule it is about -- so the @@ -431,11 +500,11 @@ def main(): print(f"SKIP {label}: the mutation changes nothing") missed.append(label) continue - write_exact(SERVER, text.replace(old, new, 1), newline) + write_exact(target, text.replace(old, new, 1), newline) try: code, output = run_probe() finally: - SERVER.write_bytes(original) + target.write_bytes(original) if code == 0: print(f"MISSED {label}: the probe still passed") missed.append(label) @@ -454,9 +523,10 @@ def main(): continue print(f"caught {label}\n -> {expect}") - if SERVER.read_bytes() != original: - print(f"NOT RESTORED {SERVER}") - return 1 + for target, original in originals.items(): + if target.read_bytes() != original: + print(f"NOT RESTORED {target}") + return 1 print("file restored byte-for-byte") if missed or unnamed: diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index fd5352c5..f6471181 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -408,6 +408,16 @@ assert('loadParsedRows is exported so a test can run the real step', const declaredNumber = (source, name) => { const match = source.match(new RegExp(`const ${name} = ([0-9_ *]+);`)); if (match === null) return null; + // A declaration that is COMMENTED OUT is not a declaration. The first match in the file is + // otherwise enough to answer with a number the code does not have: an annotation left beside + // the real one satisfies the pattern while the binding is something else, and the real one + // turned into a string stops matching at all -- so the number read would be the annotation's + // and the binding's type would go unexamined. The line the match is on therefore has to + // DECLARE it: nothing but whitespace before it, and no block comment open around it. + const lineStart = source.lastIndexOf('\n', match.index) + 1; + if (source.slice(lineStart, match.index).trim() !== '') return null; + const before = source.slice(0, match.index); + if (before.split('/*').length !== before.split('*/').length) return null; const parts = match[1].replace(/_/g, '').split('*').map((part) => Number(part.trim())); if (parts.some((part) => !Number.isFinite(part))) return null; return parts.reduce((carry, part) => carry * part, 1); @@ -426,6 +436,104 @@ const docBlockOf = (source, name) => { return start >= 0 && end > start ? before.slice(start, end) : null; }; +// The sentence of a doc block that makes a promise. A block can hold several sentences, and only +// one of them is the promise to a reader -- so the number has to be read out of THAT sentence and +// not out of the block. A number that sat in some other sentence of the same block would answer +// the pattern while the promise itself said something else. +const promiseSentenceOf = (block) => { + if (block === null) return null; + // The sentences of the block, keeping their terminators so a sentence can be recognised. + const sentences = block.split(/(?<=\.)\s+/); + return sentences.find((sentence) => sentence.includes('no later than')) ?? null; +}; + +// The minutes a sentence promises, in either form it can be written in. A promise to a reader is +// written both as a number of milliseconds and as a number of minutes, and the two forms have to +// be the same promise -- otherwise the reader is told one thing and the service does another. +const minutesPromised = (sentence) => { + if (sentence === null) return null; + const words = { + one: 1, two: 2, three: 3, four: 4, five: 5, six: 6, seven: 7, eight: 8, nine: 9, ten: 10, + eleven: 11, twelve: 12, twenty: 20, thirty: 30, forty: 40, fifty: 50, sixty: 60, + }; + const match = /\b(\d+(?:\.\d+)?|one|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve|twenty|thirty|forty|fifty|sixty)(?:\s+and\s+a\s+half)?\s+minutes?\b/i + .exec(sentence); + if (match === null) return null; + const raw = match[1].toLowerCase(); + let value = /^\d/.test(raw) ? Number(raw) : words[raw]; + if (value === undefined || !Number.isFinite(value)) return null; + if (/\s+and\s+a\s+half/.test(match[0])) value += 0.5; + return value; +}; + +// The lifetime the cache gives its rows when nobody tells it one, observed rather than read. +// +// The declaration of the limit says what the service INTENDS to give its rows; this says what it +// gives them. A cache whose default is taken from somewhere else than the constant the prose +// documents answers every declaration-based check with the documented number while the rows it +// actually holds live twice as long -- so the clock is moved until the cache lets the rows go by +// itself, and the moment it does is the limit. +const defaultLimitOf = (budget) => { + const CAP = 2 ** 31; + const releasedAt = (delta) => { + const clock = { at: 1_700_000_000_000 }; + const cache = createParsedRowsCache(budget, { now: () => clock.at }); + cache.set('k', { id: 'k' }, 1); + clock.at += delta; + cache.sweep(); + const released = cache.bytes === 0; + cache.clear(); + return released; + }; + // A cache that never lets the rows go has no limit to find. + if (!releasedAt(CAP)) return null; + let low = 0; + let high = CAP; + while (low + 1 < high) { + const middle = Math.floor((low + high) / 2); + if (releasedAt(middle)) high = middle; + else low = middle; + } + return high; +}; + +// The delay the page's poll actually waits, observed by evaluating the expression the timer is +// given. The declaration of the interval is a number in the source; the delay is what the poll +// really waits, and a poll that waits longer than the limit it has to survive loses its rows +// between polls while every check that reads the declaration stays green. +const effectivePollDelay = (clientSource) => { + const declared = /TRAJECTORY_POLL_INTERVAL\s*=\s*(\d+)\s*;/.exec(clientSource); + if (declared === null) return null; + const value = Number(declared[1]); + // Every use of the name that is not its declaration, and the expression it is part of. + const uses = []; + const pattern = /TRAJECTORY_POLL_INTERVAL/g; + let match; + while ((match = pattern.exec(clientSource)) !== null) { + const after = clientSource.slice(match.index + 'TRAJECTORY_POLL_INTERVAL'.length); + if (/^\s*=\s*\d+/.test(after)) continue; // the declaration itself + let start = match.index; + while (start > 0 && /[A-Za-z0-9_.*+\-/<> ]/.test(clientSource[start - 1])) start -= 1; + let end = match.index + 'TRAJECTORY_POLL_INTERVAL'.length; + while (end < clientSource.length && /[A-Za-z0-9_.*+\-/<> ]/.test(clientSource[end])) end += 1; + uses.push(clientSource.slice(start, end).trim()); + } + if (uses.length === 0) return null; + const delays = []; + for (const expression of uses) { + let evaluated; + try { + // The expression is the page's own code, evaluated with the value the page declares. + evaluated = new Function('TRAJECTORY_POLL_INTERVAL', `return (${expression});`)(value); + } catch { + return null; + } + if (!Number.isFinite(evaluated)) return null; + delays.push(evaluated); + } + return { declared: value, delays }; +}; + // The proxy is read out of the shipped source rather than repeated here, so the relationship // below is checked against the constant the service actually charges with. const PROXY = declaredNumber(SERVER, 'PARSED_ROWS_HEAP_PER_SOURCE_BYTE'); @@ -443,8 +551,14 @@ check('the cache is charged the source bytes times the heap proxy', coverageCache.bytes, Math.ceil(sizedBytes * PROXY)); check('so the source bytes it holds are computable from its own accounting', coverageCache.bytes / PROXY, sizedBytes); +// The charge the ROUTE makes, observed by making the call the route makes. The route passes +// options, and a fixture that leaves them out takes a different branch of the same function, so +// a charge that was wrong on the route's path was invisible here -- and the old owner asked only +// whether the text of the charge was somewhere in the file, which a commented-out line satisfied. +const routeCache = createParsedRowsCache(64 * 1024 * 1024); +await loadParsedRows(routeCache, sized, LIMIT, { cacheable: true }); assert('and the charge the route makes is that one', - SERVER.includes('cache.set(key, value, Math.ceil(head.size * PARSED_ROWS_HEAP_PER_SOURCE_BYTE));')); + routeCache.bytes === Math.ceil(sizedBytes * PROXY)); // A clock the test controls, so "expired" is a fact rather than a wait. let clock = 1_700_000_000_000; @@ -479,6 +593,14 @@ check('the rows are released without a request', liveCache.bytes, 0); check('and the cache is empty', liveCache.size, 0); // The time limit has to outlive one poll, or an open tab would lose its rows between polls. +// +// Both numbers here are OBSERVED, and that is the point of this block. The delay the poll waits +// and the lifetime the cache gives its rows were both read out of the source before, and a source +// says what the code INTENDS: the number a constant is declared with, and the number the page +// writes next to its timer. A cache whose default lifetime comes from somewhere else, or a poll +// that multiplies the interval it declared, leaves every declaration exactly as it was while the +// service behaves differently -- and the promise to a reader is about the service, not about a +// declaration. const clientSource = readFileSync(new URL('../miniapp/client/index.html', import.meta.url), 'utf8'); const pollMatch = clientSource.match(/TRAJECTORY_POLL_INTERVAL = (\d+)/); assert('the client says how often it polls the trajectory', pollMatch !== null); @@ -486,13 +608,22 @@ const ttlMatch = SERVER.match(/const PARSED_ROWS_CACHE_TTL_MS = ([0-9_ *]+);/); assert('the rows cache has a time limit', ttlMatch !== null); const TTL = declaredNumber(SERVER, 'PARSED_ROWS_CACHE_TTL_MS'); assert('the time limit is a number the service declares', TTL !== null); + +// The delay the page's timer is really given, and the lifetime the cache really gives its rows. +const poll = effectivePollDelay(clientSource); +assert('the poll waits as long as the client declares', + poll !== null && poll.delays.every((delay) => delay === poll.declared)); +const POLL_DELAY = poll === null ? null : Math.max(...poll.delays); +const DEFAULT_LIMIT = defaultLimitOf(1024 * 1024); +assert('the limit the service declares is the limit the cache gives its rows', + DEFAULT_LIMIT !== null && DEFAULT_LIMIT === TTL); assert('the time limit is longer than one poll, so an open tab never loses its rows', - TTL > Number(pollMatch[1])); + POLL_DELAY !== null && DEFAULT_LIMIT !== null && DEFAULT_LIMIT > POLL_DELAY); // The page stops polling while its tab is hidden, so a reader who switches away for a while is // the ordinary case. A limit shorter than that would make every return to the tab cost a // re-parse — which is the other half of what this limit buys, and it has to be paid for. assert('the time limit is at least five minutes, so stepping away from the tab does not cost a re-parse', - TTL >= 5 * 60 * 1000); + DEFAULT_LIMIT !== null && DEFAULT_LIMIT >= 5 * 60 * 1000); // And the sweeper must never be the reason the process is still running. The owner of that rule // used to be a pattern over the source, and a pattern cannot tell the cache's own handle from an @@ -502,14 +633,16 @@ assert('the time limit is at least five minutes, so stepping away from the tab d // // A cache that starts no timer at all has nothing that could hold the process open, so that is // green here: whether a sweeper starts at all is another owner's rule, and this one is not a second -// accusation of it. +// accusation of it. A handle that exists but cannot answer the question is not green either -- the +// owner has to be able to say that the handle does not hold the process open, and a handle it +// cannot ask has not said it. const unrefWindow = withRecordedTimers(() => { const unrefCache = createParsedRowsCache(1000, { ttlMs: 1_000 }); unrefCache.set('k', val, 500); }); assert('the sweeper does not keep the process alive', - unrefWindow.handles.every((handle) => typeof handle.hasRef !== 'function' - || handle.hasRef() === false)); + unrefWindow.handles.every((handle) => typeof handle.hasRef === 'function' + && handle.hasRef() === false)); // The release lands on a sweep tick, so the rows are gone a sweep interval AFTER the limit. The // promise to a reader is that interval added on, and it only stays a promise while the interval @@ -535,11 +668,28 @@ assert('the shipped default is the interval the service declares', // a change to one of them without the other is caught here and not by a reader. The promise is // in prose because that is where a reader meets it, and a promise nobody checks is a promise // that is true on the day it was written. +// The number the package promises a reader, read out of the SENTENCE that makes the promise. +// +// That block states the promise twice -- as a number of minutes and as a number of milliseconds +// -- and the two forms have to be the same promise, or the reader is told one thing and the +// service does another. A block can also hold a number that is not part of the promise at all: an +// annotation left beside it would answer the pattern while the sentence a reader reads said +// something else. So the number is taken from the promise sentence, that sentence's own words +// are read as well, and the two are compared with each other and with the limit the cache really +// gives its rows. const ttlDoc = docBlockOf(SERVER, 'PARSED_ROWS_CACHE_TTL_MS'); -const promisedMs = ttlDoc === null ? null : /\((\d+) ms\)/.exec(ttlDoc); +const promiseSentence = promiseSentenceOf(ttlDoc); +const promisedNumbers = promiseSentence === null + ? [] + : [...promiseSentence.matchAll(/\((\d+) ms\)/g)].map((found) => Number(found[1])); +const promisedMinutes = minutesPromised(promiseSentence); +const DEFAULT_INTERVAL = createParsedRowsCache(1000).sweepIntervalMs; assert('the number promised to a reader is the limit plus one sweep interval', - promisedMs !== null - && Number(promisedMs[1]) === TTL + createParsedRowsCache(1000).sweepIntervalMs); + promisedNumbers.length === 1 && DEFAULT_LIMIT !== null + && promisedNumbers[0] === DEFAULT_LIMIT + DEFAULT_INTERVAL); +assert('the promise to a reader is the same promise in both of the forms it is written in', + promisedMinutes === null || promisedNumbers.length === 0 + || Math.abs(promisedMinutes * 60_000 - promisedNumbers[0]) < 1); // The promise above holds only if the TIMER runs on that interval: the rows are released on the // next tick, and the tick is the sweep. Nothing in the package watched the call that starts that From e86642806ee311ea0724af11137674d6ae29bb55 Mon Sep 17 00:00:00 2001 From: avatasia Date: Fri, 23 Oct 2026 22:53:56 +0800 Subject: [PATCH 77/95] An owner that cannot read what it needs stops accusing the rules it is not about Five owners in this package were reporting rules as broken when the break had not broken them, and the control could not tell the difference. All five are fixed here, and the product is not changed. - the three owners that compare the cache's accounting against the heap proxy read the proxy out of the source, and when it could not be read the expected charge became zero. A cache whose proxy is the string "2" is still charged twice the source bytes -- that is what the service's own arithmetic does with it -- so the rule those three name was not broken by the break that made them red: a break of one rule was reported by the owners of three other rules. They now hold vacuously when there is no proxy to charge with, and the rule that the proxy has to be a number the service declares keeps the whole of that report -- it is the only owner that goes red, and it names itself. A vacuous holding is counted and printed with the result, so a reader can tell an owner that was checked and found true from one that said nothing at all. - the probe that finds the lifetime the cache gives its rows returned nothing when the cache never let the rows go, and the owners that read it went red -- including two whose rules were not broken by that. A cache that never releases its rows has a lifetime longer than any poll, and that is what those two rules say. The probe now returns the ceiling it reached, which is a floor on the lifetime rather than a measurement of it, and every owner reads it as one. - three owners named a weaker rule than the one they checked. The owner of the rule that the cached rows are the same array and not a copy of it compared the whole value a read hands back, so a read that returned a fresh wrapper around the very same array was reported as a copy of the array. The owner of the rule that the second read of an empty file is a hit compared objects, and "is a hit" is not what an object comparison can tell you: a cache that served a copy on a hit would still have been a hit. And the owner of the rule that the floor of the quarter is a millisecond compared for exactly a millisecond while its name said the sweep is not finer than one, so a sweep that was far coarser than a millisecond -- and therefore late rather than early -- was reported as a rule broken. The first now compares the arrays, which is the rule it names; the second now says what it compares, that the hit hands back the same object the cold read gave; the third now says what it holds, that the sweep runs every millisecond so the release is at most a millisecond late. The control now says which rules each break is ALLOWED to report, and not only which rule it has to report. A control that asks "was the rule named among those that went red" is also satisfied by a guard that reported three further rules which that break did not break -- so it could not tell the two apart. Every case now names the exact set of rules its break may report, in one table that the control reads, and a case fails when the probe reports anything outside it. What the set pins is which assertions go red; whether each of those rules really was broken by that break is a judgement, and it is the reader's. Two cases were added, one for each of the two owners that were fixed by comparison: a hit that hands back a copy of the rows array, and a hit that hands back a copy of the value. The control went from thirty-eight cases to forty, and every case in it is still caught by name. The guard stayed at ninety-three assertions: the three owners that now hold vacuously are still three assertions, they are green rather than red when there is nothing to compare against, and the count of those is printed with the result. One note on how the new helper compares. The helper this commit introduces compares its two sides with ===, while the comparison the guard already used compares the serialized values. Every side this commit compares with the helper is a number, so the two ways give the same answer here. The difference is written down because both ways of comparing are now in the same file, and a reader who took them for the same one would be wrong about one of them. --- .../tests/mutate-parsed-rows.py | 272 +++++++++++++++++- .../tests/verify-parsed-rows.mjs | 80 +++++- 2 files changed, 332 insertions(+), 20 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py index 3790b3eb..53210835 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows.py @@ -174,6 +174,13 @@ " startSweeper();\n", "", 'the rows are released without a request', + None, + ( + 'the rows are released without a request', + 'and the cache is empty', + 'a caller who asks for a non-positive interval still gets the rows released ' + 'without a request', + ), ), ( # An interval that is not unref'd keeps the event loop alive after the server closes, @@ -237,7 +244,7 @@ "the limit bound loses its floor of a millisecond", " const limitBoundMs = Math.max(1, Math.floor(ttlMs / 4));", " const limitBoundMs = Math.floor(ttlMs / 4);", - 'a limit too small for a quarter of a millisecond still sweeps no finer than a millisecond', + 'a limit below four milliseconds is still swept every millisecond, so the release is at most a millisecond late', ), ( # A caller who asks for an interval that is not positive used to switch the sweeper off, @@ -332,6 +339,13 @@ " startSweeper();\n", "", 'the rows are released without a request', + None, + ( + 'the rows are released without a request', + 'and the cache is empty', + 'a caller who asks for a non-positive interval still gets the rows released ' + 'without a request', + ), ), ( # The owners of the lifetime read the DECLARATION of the limit, so a cache whose default @@ -342,6 +356,11 @@ " const ttlMs = options.ttlMs ?? PARSED_ROWS_CACHE_TTL_MS;", " const ttlMs = options.ttlMs ?? (PARSED_ROWS_CACHE_TTL_MS * 2);", 'the limit the service declares is the limit the cache gives its rows', + None, + ( + 'the limit the service declares is the limit the cache gives its rows', + 'the number promised to a reader is the limit plus one sweep interval', + ), ), ( # The promise to a reader is written twice -- as minutes and as milliseconds -- and the two @@ -351,6 +370,8 @@ "one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped", "one sweep interval after the limit — ten minutes (750000 ms) with the shipped", 'the promise to a reader is the same promise in both of the forms it is written in', + None, + ('the promise to a reader is the same promise in both of the forms it is written in',), ), ( # The same promise, read out of the whole block: a number left in an obsolete annotation @@ -362,6 +383,8 @@ " * values. Internal obsolete annotation (750000 ms). The number to promise a reader is that one," " and it is the number in brackets: it is", 'the number promised to a reader is the limit plus one sweep interval', + None, + ('the number promised to a reader is the limit plus one sweep interval',), ), ( # The page polls every TRAJECTORY_POLL_INTERVAL milliseconds -- unless the expression the @@ -373,6 +396,10 @@ " }, TRAJECTORY_POLL_INTERVAL * 1000);", 'the poll waits as long as the client declares', "miniapp/client/index.html", + ( + 'the poll waits as long as the client declares', + 'the time limit is longer than one poll, so an open tab never loses its rows', + ), ), ( # The route passes options, and a fixture that leaves them out takes a different branch of @@ -386,6 +413,8 @@ " ? Math.ceil(head.size * PARSED_ROWS_HEAP_PER_SOURCE_BYTE)\n" " : head.size);", 'and the charge the route makes is that one', + None, + ('and the charge the route makes is that one',), ), ( # A declaration that is commented out is not a declaration. The pattern read the comment @@ -394,10 +423,218 @@ "const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;", "// const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = '2';", 'the heap proxy is a number the service declares', + None, + ('the heap proxy is a number the service declares',), + ), + ( + # The owner of the rule that the cached rows are the same ARRAY and not a copy of it now + # compares the arrays -- it used to compare the whole value a read hands back, which is a + # stronger thing than the name claims, and a read that handed back a fresh wrapper around + # the same array was reported as a copy of the array. This break hands back a copy of the + # array, and it has to be caught by the owner of the array. + "the hit hands back a copy of the rows array", + " if (hit !== null) return hit;\n", + " if (hit !== null) return { ...hit, rows: [...hit.rows] };\n", + 'and it is literally the same array, not a copy', + None, + ), + ( + # The owner of the rule that the hit gives back the same object the cold read gave used to + # be named "the hit is a hit" while it compared objects -- and "is a hit" is not what an + # object comparison can tell you. It now says what it compares, and this break is what + # breaks THAT: the hit hands back a copy of the value. + "the hit hands back a copy of the value", + " if (hit !== null) return hit;\n", + " if (hit !== null) return { ...hit };\n", + 'and the hit on an empty file gives back the same object the cold read gave', + None, ), ] +# The EXACT set of rules each case's break is allowed to report. +# +# A control that only asks "was the rule named among those that went red" is also satisfied by a +# guard that reported three further rules which that break did not break -- and a guard like that +# cannot tell the reader which rule had actually been broken. So each case names the rules its +# break may report, and the control fails when the probe reports anything outside that set. +# +# The table below is the single source of truth: a case that names a set inline has to agree with +# it, and the control refuses to run when the two disagree. +# +# WHAT THIS PINS, and what it does not: it pins WHICH ASSERTIONS GO RED. It does not decide which +# rule was broken -- that is the reviewer's judgement, and the table is what lets the reviewer +# check it: every rule in a case's set has to be a rule that the break really does break. + +EXACT = { + 'server: the key stops naming the file, so two sessions share their messages.jsonl': ( + 'two sessions with identically-sized, identically-timed messages.jsonl do not share rows', + "and the first one was not served the second one's content", + 'the cache kept both', + 'the path is part of the key (two sessions, one basename)', + ), + 'server: the key stops carrying the size, so a grown file serves a stale tail': ( + 'the size is part of the key (the active file grows)', + ), + 'server: the key stops carrying the mtime, so a same-size rewrite serves stale rows': ( + 'a same-size rewrite is noticed, via mtime', + 'and it is a different object, not the cached one', + 'the mtime is part of the key (a rewrite at the same size)', + ), + 'server: the key stops carrying the limit, so the truncation verdict is read at another width': ( + 'the limit is part of the key (readHead truncates at it)', + ), + 'server: `empty` is not carried on the cached value, so a hit stops reporting it': ( + 'an empty file reports empty on a cold read', + 'an empty file still reports empty on a hit', + ), + 'server: `skipped` is not carried, so a read under-counts the lines it dropped': ( + 'two lines are skipped: one unparseable, one not an object', + ), + 'a hit reports a skipped count of its own rather than the one it was given': ( + 'a hit reports the same skipped count as a cold read', + 'an unchanged file is served from the cache (same object, not a re-read)', + 'and the hit on an empty file gives back the same object the cold read gave', + ), + 'the budget stops evicting, so the cache grows without limit': ( + 'the third evicts the oldest, keeping the budget', + 'and the total stays inside the budget', + 'the least recently used entry is the one that went', + 'the file that was not read is the one evicted', + ), + 'reading stops refreshing recency, so the 22 frozen files get evicted by insertion order': ( + 'a file that was just read survives the eviction that follows', + 'the file that was not read is the one evicted', + ), + 'an entry larger than the whole budget is stored anyway, evicting everything for one hit': ( + 'a single entry larger than the budget is not stored', + 'and it is not retrievable', + ), + 're-storing a key double-counts it, so the budget shrinks over time': ( + 're-storing the same key counts once', + ), + 'the cache becomes a module-level singleton, shared across every runtime instance': ( + 'the cache is created inside start(), not at module level', + ), + 'the route parses the file inline again instead of going through the cache': ( + 'the route reaches the file through loadParsedRows()', + ), + 'the cache is charged the source bytes without the heap proxy, so its accounting drifts from its input': ( + 'the cache is charged the source bytes times the heap proxy', + 'so the source bytes it holds are computable from its own accounting', + 'and the charge the route makes is that one', + ), + 'a read does not notice that the rows have expired': ( + 'a read after the time limit is a miss', + 'and the entry is not held any more', + ), + 'the sweep releases nothing': ( + 'a sweep releases what expired', + 'and leaves nothing behind', + 'the rows are released without a request', + 'and the cache is empty', + 'the limit the service declares is the limit the cache gives its rows', + 'the number promised to a reader is the limit plus one sweep interval', + 'a caller who asks for a non-positive interval still gets the rows released without a request', + ), + 'the sweeper is never started, so the rows stay until the next request': ( + 'the rows are released without a request', + 'and the cache is empty', + 'a caller who asks for a non-positive interval still gets the rows released without a request', + ), + 'the sweeper keeps the process alive': ( + 'the sweeper does not keep the process alive', + ), + "an unrelated unref stands in for the sweeper's own": ( + 'the sweeper does not keep the process alive', + ), + 'the time limit is shorter than one poll': ( + 'the time limit is longer than one poll, so an open tab never loses its rows', + 'the time limit is at least five minutes, so stepping away from the tab does not cost a re-parse', + 'the shipped default is the interval the service declares', + 'the number promised to a reader is the limit plus one sweep interval', + ), + 'the time limit is short enough that switching away from the tab costs a re-parse': ( + 'the time limit is at least five minutes, so stepping away from the tab does not cost a re-parse', + 'the shipped default is the interval the service declares', + 'the number promised to a reader is the limit plus one sweep interval', + ), + 'the sweep interval is taken without being clamped to the limit': ( + 'the sweep interval the cache uses is at most a quarter of the limit', + 'the sweep interval the cache uses is at most a quarter of the limit', + 'an interval the caller asks for is clamped to the limit as well', + 'a limit below four milliseconds is still swept every millisecond, so the release is at most a millisecond late', + ), + 'the limit bound loses its floor of a millisecond': ( + 'a limit below four milliseconds is still swept every millisecond, so the release is at most a millisecond late', + ), + 'the asked interval loses its floor of a millisecond': ( + 'an interval the caller asks for that is not positive is taken as a millisecond', + 'a caller who asks for a non-positive interval still gets the rows released without a request', + ), + 'a time limit of zero expires everything anyway': ( + 'a time limit of zero means no time limit, not that everything expires at once', + 'and the rows are still held', + ), + 'the promise to a reader says a number the code does not have': ( + 'the number promised to a reader is the limit plus one sweep interval', + 'the promise to a reader is the same promise in both of the forms it is written in', + ), + 'the limit is longer than the promise to a reader': ( + 'the number promised to a reader is the limit plus one sweep interval', + ), + 'the timer that releases the rows runs on twice the interval the cache reports': ( + 'the timer that releases the rows runs on the interval the cache reports', + ), + 'a shadowed binding doubles the interval the timer really gets': ( + 'the timer that releases the rows runs on the interval the cache reports', + ), + 'a shadowed binding replaces the interval with a number of its own': ( + 'the rows are released without a request', + 'and the cache is empty', + 'the timer that releases the rows runs on the interval the cache reports', + 'a caller who asks for a non-positive interval still gets the rows released without a request', + ), + 'a zero limit starts the sweeper it promised not to start': ( + 'a time limit of zero starts no sweeper, because nothing in it ever expires', + ), + 'the sweeper is never started, so the rows are held until somebody asks for them': ( + 'the rows are released without a request', + 'and the cache is empty', + 'a caller who asks for a non-positive interval still gets the rows released without a request', + ), + 'the default lifetime is doubled, so the declaration no longer says what the cache does': ( + 'the limit the service declares is the limit the cache gives its rows', + 'the number promised to a reader is the limit plus one sweep interval', + ), + 'the promise to a reader says ten minutes while the number still says 750000': ( + 'the promise to a reader is the same promise in both of the forms it is written in', + ), + 'the number promised to a reader is moved into an obsolete annotation': ( + 'the number promised to a reader is the limit plus one sweep interval', + ), + 'the poll waits a thousand times the interval it declares': ( + 'the poll waits as long as the client declares', + 'the time limit is longer than one poll, so an open tab never loses its rows', + ), + 'the route charges the source bytes while a commented-out line supplies the old text': ( + 'and the charge the route makes is that one', + ), + 'the proxy declaration is commented out and the binding is a string': ( + 'the heap proxy is a number the service declares', + ), + 'the hit hands back a copy of the rows array': ( + 'an unchanged file is served from the cache (same object, not a re-read)', + 'and it is literally the same array, not a copy', + 'and the hit on an empty file gives back the same object the cold read gave', + ), + 'the hit hands back a copy of the value': ( + 'an unchanged file is served from the cache (same object, not a re-read)', + 'and the hit on an empty file gives back the same object the cold read gave', + ), +} + + def read_exact(path): """Normalise to LF for matching, and report the file's own newline so it can be put back. @@ -479,10 +716,28 @@ def main(): missed = [] unnamed = [] for label, old, new, expect, *rest in MUTATIONS: - # A case may name the file it breaks. Most break the service; the ones about the PAGE break - # the page, because a control that could only break one file could not hold a case about - # the other half of the package. - target = SERVER if not rest else (ROOT / rest[0]) + # A case may name the file it breaks, and it may name the EXACT set of rules that are + # allowed to go red. + # + # Most cases break the service; the ones about the PAGE break the page, because a control + # that could only break one file could not hold a case about the other half of the + # package. A case with no file named breaks the service. + # + # The exact set is there for the cases whose POINT is the isolation: a break that belongs + # to one rule has to be reported by the owner of THAT rule and by nobody else. A control + # that only asked "was the rule named among those that went red" would also pass on a + # guard that reported three further rules which that break did not break -- and that is + # the shape the two owners that stood in for each other had, and the shape this round is + # about. So for those cases the set of reported rules has to be EXACTLY the set the case + # names, and a case that names none claims no isolation. + target = SERVER if not rest or rest[0] is None else (ROOT / rest[0]) + # The exact set of rules this break is allowed to report comes from the table below -- the + # one place that holds it. A case that also names a set inline has to agree with it, and + # the control refuses to run when the two disagree: two places that have to agree are a + # check, one place that has to be remembered is a hope. + exact = set(EXACT[label]) + if len(rest) > 1 and set(rest[1]) != exact: + raise SystemExit("FAIL the case %r names two different exact sets" % label) if target not in originals: originals[target] = target.read_bytes() original = originals[target] @@ -521,6 +776,13 @@ def main(): print(f"MISSED {label}: the probe named {names} instead of {expect}") missed.append(label) continue + # When the case names the exact set of rules that may go red, a guard that also reported + # a rule this break did not break has failed the case: it accused another owner. + if exact is not None and set(names) != exact: + print(f"OVERREACH {label}: the probe named {sorted(set(names))} " + f"instead of exactly {sorted(exact)}") + missed.append(label) + continue print(f"caught {label}\n -> {expect}") for target, original in originals.items(): diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index f6471181..1c97b057 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -42,6 +42,20 @@ const SERVER = readFileSync(new URL('../miniapp/node/server.mjs', import.meta.ur let pass = 0; const failures = []; +// How many of the assertions below held VACUOUSLY -- green because there was nothing to compare +// against, rather than because the thing they are about was checked and matched. +// +// A vacuous holding has to be VISIBLE. An owner that says nothing cannot be told apart from one +// that was checked and found true, and the reader of this guard's output has to be able to tell +// the two apart -- that reader is the one asking "did this break break the rule that owner +// names?", and an owner that went quiet is not an answer to it. +let vacuous = 0; + +// A comparison that holds vacuously when it cannot be made, and counts itself when it does. +const holds = (actual, expected, comparable) => { + if (!comparable) { vacuous += 1; return true; } + return actual === expected; +}; function check(name, actual, expected) { if (JSON.stringify(actual) === JSON.stringify(expected)) pass += 1; else failures.push(`${name}: got ${JSON.stringify(actual)}, want ${JSON.stringify(expected)}`); @@ -324,7 +338,12 @@ const beforeShape = JSON.stringify(one.rows); await buildBody([{ cache: shareRows, path: a, generation: 4 }], null); const two = await loadParsedRows(shareRows, a, LIMIT); assert('building a payload leaves the cached rows byte-identical', beforeShape === JSON.stringify(two.rows)); -assert('and it is literally the same array, not a copy', one === two); +// The ARRAY is what the name says: the same array, not a copy of it. The check used to compare +// the whole value the read returned, which is a STRONGER thing than the name claims -- a read +// that handed back a fresh wrapper around the same array would have been reported as a copy of +// the array, and the rule this owner names would have been reported as broken by a break that +// did not break it. The owner now compares the arrays. +assert('and it is literally the same array, not a copy', one.rows === two.rows); // ROW_GENERATION is the one write the route makes to a cached row, and one path maps to one // generation — so stamping twice leaves the same value. check('stamping the generation is idempotent', two.rows[0][ROW_GENERATION], 4); @@ -349,7 +368,12 @@ const e1 = await loadParsedRows(emptyCache, emptyPath, LIMIT); const e2 = await loadParsedRows(emptyCache, emptyPath, LIMIT); assert('an empty file reports empty on a cold read', e1.empty === true); assert('an empty file still reports empty on a hit', e2.empty === true); -assert('and the hit is a hit', e1 === e2); +// The second read of an empty file is served from the cache, and what it hands back is the very +// object the cold read gave. The name used to say "the hit is a hit" while the check compared +// objects -- and "is a hit" is not what an object comparison can tell you: a cache that served a +// copy on a hit would still have been a hit, and the owner would have reported the rule it names +// as broken by a break that did not break it. The owner now says what it compares. +assert('and the hit on an empty file gives back the same object the cold read gave', e1 === e2); /* ===================================================================== * 8 — structure: where the cache lives and what it is keyed on @@ -485,8 +509,13 @@ const defaultLimitOf = (budget) => { cache.clear(); return released; }; - // A cache that never lets the rows go has no limit to find. - if (!releasedAt(CAP)) return null; + // A cache that never lets the rows go has no limit to find -- but it does have a FLOOR on the + // limit it gives: the ceiling this probe reached. So the probe returns that ceiling rather + // than nothing, and the owners below read it as the lower bound it is. A limit of at least + // twenty-four days is longer than one poll and longer than five minutes, and it is not the + // limit the service declares -- so every owner below says the right thing about it, instead of + // going quiet and accusing the rules of a break that did not break them. + if (!releasedAt(CAP)) return CAP; let low = 0; let high = CAP; while (low + 1 < high) { @@ -541,16 +570,29 @@ assert('the heap proxy is a number the service declares', PROXY !== null); // A file of a known size, loaded through the shipped step: the cache must be charged the source // bytes times the proxy, so the source bytes it holds are recoverable from its own accounting. +// +// The three owners below are about the CHARGE, and the proxy they compare against is read out of +// the source. When it cannot be read as a number there is no proxy to charge with -- and the rule +// that the proxy has to be a number the service DECLARES has an owner of its own, the one above. +// So these three hold vacuously in that case, and they never accuse the rule about the +// declaration. +// +// That is not a technicality. A cache whose proxy is the string "2" is still charged twice the +// source bytes, because that is what the service's own arithmetic does with it, so the rule these +// three name is not broken by the break that made them red. An owner that reports "I expected 0 +// bytes" is not reporting a broken rule -- it is reporting that it could not read the number it +// wanted. A break of the declaration has to be reported by the owner of the declaration and by +// nobody else. const sized = join(root, 'sized.jsonl'); const sizedText = ROWS.map((r) => JSON.stringify(r)).join('\n') + '\n'; await writeFile(sized, sizedText); const sizedBytes = Buffer.byteLength(sizedText, 'utf8'); const coverageCache = createParsedRowsCache(64 * 1024 * 1024); await loadParsedRows(coverageCache, sized, LIMIT); -check('the cache is charged the source bytes times the heap proxy', - coverageCache.bytes, Math.ceil(sizedBytes * PROXY)); -check('so the source bytes it holds are computable from its own accounting', - coverageCache.bytes / PROXY, sizedBytes); +assert('the cache is charged the source bytes times the heap proxy', + holds(coverageCache.bytes, Math.ceil(sizedBytes * PROXY), PROXY !== null)); +assert('so the source bytes it holds are computable from its own accounting', + holds(coverageCache.bytes / PROXY, sizedBytes, PROXY !== null)); // The charge the ROUTE makes, observed by making the call the route makes. The route passes // options, and a fixture that leaves them out takes a different branch of the same function, so // a charge that was wrong on the route's path was invisible here -- and the old owner asked only @@ -558,7 +600,7 @@ check('so the source bytes it holds are computable from its own accounting', const routeCache = createParsedRowsCache(64 * 1024 * 1024); await loadParsedRows(routeCache, sized, LIMIT, { cacheable: true }); assert('and the charge the route makes is that one', - routeCache.bytes === Math.ceil(sizedBytes * PROXY)); + holds(routeCache.bytes, Math.ceil(sizedBytes * PROXY), PROXY !== null)); // A clock the test controls, so "expired" is a fact rather than a wait. let clock = 1_700_000_000_000; @@ -616,14 +658,14 @@ assert('the poll waits as long as the client declares', const POLL_DELAY = poll === null ? null : Math.max(...poll.delays); const DEFAULT_LIMIT = defaultLimitOf(1024 * 1024); assert('the limit the service declares is the limit the cache gives its rows', - DEFAULT_LIMIT !== null && DEFAULT_LIMIT === TTL); + DEFAULT_LIMIT === TTL); assert('the time limit is longer than one poll, so an open tab never loses its rows', - POLL_DELAY !== null && DEFAULT_LIMIT !== null && DEFAULT_LIMIT > POLL_DELAY); + POLL_DELAY !== null && DEFAULT_LIMIT > POLL_DELAY); // The page stops polling while its tab is hidden, so a reader who switches away for a while is // the ordinary case. A limit shorter than that would make every return to the tab cost a // re-parse — which is the other half of what this limit buys, and it has to be paid for. assert('the time limit is at least five minutes, so stepping away from the tab does not cost a re-parse', - DEFAULT_LIMIT !== null && DEFAULT_LIMIT >= 5 * 60 * 1000); + DEFAULT_LIMIT >= 5 * 60 * 1000); // And the sweeper must never be the reason the process is still running. The owner of that rule // used to be a pattern over the source, and a pattern cannot tell the cache's own handle from an @@ -685,7 +727,7 @@ const promisedNumbers = promiseSentence === null const promisedMinutes = minutesPromised(promiseSentence); const DEFAULT_INTERVAL = createParsedRowsCache(1000).sweepIntervalMs; assert('the number promised to a reader is the limit plus one sweep interval', - promisedNumbers.length === 1 && DEFAULT_LIMIT !== null + promisedNumbers.length === 1 && promisedNumbers[0] === DEFAULT_LIMIT + DEFAULT_INTERVAL); assert('the promise to a reader is the same promise in both of the forms it is written in', promisedMinutes === null || promisedNumbers.length === 0 @@ -725,7 +767,7 @@ assert('the timer that releases the rows runs on the interval the cache reports' // deliberately tiny: with a limit of a second the floor never applies, and a case that cannot // reach the floor would pass without anything having been decided. const tinyLimit = createParsedRowsCache(1000, { ttlMs: 3 }); -assert('a limit too small for a quarter of a millisecond still sweeps no finer than a millisecond', +assert('a limit below four milliseconds is still swept every millisecond, so the release is at most a millisecond late', tinyLimit.sweepIntervalMs === 1); // An interval the caller asks for that is not positive is taken as a millisecond. It used to @@ -771,6 +813,14 @@ await rm(root, { recursive: true, force: true }); if (failures.length) { console.error(`\n[parsed-rows] ${failures.length} FAILED, ${pass} passed`); for (const f of failures) console.error(' - ' + f); + if (vacuous > 0) { + console.error(`[parsed-rows] ${vacuous} of them held vacuously: there was nothing to ` + + `compare against, so they say nothing about the thing they are about`); + } process.exit(1); } -console.log(`\n[parsed-rows] ${pass} assertions passed`); \ No newline at end of file +console.log(`\n[parsed-rows] ${pass} assertions passed`); +if (vacuous > 0) { + console.log(`[parsed-rows] ${vacuous} of them held vacuously: there was nothing to compare ` + + `against, so they say nothing about the thing they are about`); +} \ No newline at end of file From 05942efe5dda0e2fffa3f7e1244075fc89ba58a6 Mon Sep 17 00:00:00 2001 From: avatasia Date: Sat, 24 Oct 2026 01:09:37 +0800 Subject: [PATCH 78/95] The three owners that read the text now read what the service and the page really do Three rules in this package were decided by reading the source, and a source says what the code intends, not what it does. All three are fixed here, and the product is not changed. - the owner of the rule that the promise to a reader is the same promise in both of the forms it is written in read the minutes out of the sentence with a parser that knew some of the number words and not the rest. A promise that said "thirteen minutes (750000 ms)" was therefore a promise nobody could check, and the owner accepted that answer as agreement: the reader was told one number and the service another, and the guard said the two forms agreed. Every number word a promise can be written in is understood now. The phrase is also taken from the word the millisecond form is attached to, so a sentence that mentions minutes elsewhere in passing cannot be read as the promise. - the owner of the rule that the poll waits as long as the client declares read the characters around the interval's name. A comma expression hides a second value behind the first, so "(TRAJECTORY_POLL_INTERVAL, 8000000)" gave the timer eight million milliseconds while the characters around the name still read "TRAJECTORY_POLL_INTERVAL". The page's own script is now run in a context that answers the DOM with stubs, the delay is what the page really hands to its timer, and the timer that polls the trajectory is found by the handle the page itself stores -- not by the order the timers were started in, and not by which delay is larger. - the owner of the rule that the heap proxy is a number the service declares read the proxy out of the source, so a template literal holding a declaration impersonated it while the binding the service really uses was the string "2". The numbers the service declares are now read out of the module's own bindings: a copy of the shipped module with one export line appended is imported next to a copy of the module it imports relatively, and the values are read out of that namespace. A copy that will not import is reported as a failure of the guard, because the guard imports the same module statically -- that is a fact about the guard and not about the service, and it must not become a quiet holding. The premise of the second one is worth stating, because it is a premise and not a fact: the page's inline script has to run in the sandbox. If the page ever uses a browser API the stubs do not provide at the top level, the script throws, the owner cannot read the delay, and the guard says so and stops. It does not go quiet, because a quiet owner is a guard that has stopped holding the rule it names. Two comments in the guard are brought into line with that in the same commit. The header said the file imports the pure helpers and nothing else, and the block above the poll owners said the delay the poll waits was read out of the source. Neither was false -- the file still never starts the runtime -- but a reader of it would have been told it never runs anything, and it now runs the page's own inline script in a sandbox. A comment that understates what the code does has the same shape as an owner that checks less than it claims. Three mutations were added, one for each of the three. They live in a control of their own: the existing control owns forty cases of the cache's own behaviour, and the table it reads is the single source of truth for which rules each of those breaks may report. Those forty cases and that table are not this change's to rewrite, and adding to them would have meant two owners for one table. So the three escapes get their own control, and it holds them to the same standard: each break has to be caught by name, and by the only rules that break really breaks. The suite runs one more file; the guard stayed at ninety-three assertions. The service and the page are not changed by this commit. --- .../tests/mutate-parsed-rows-owner-escapes.py | 223 ++++++++ .../tests/verify-parsed-rows.mjs | 483 ++++++++++++++---- 2 files changed, 601 insertions(+), 105 deletions(-) create mode 100644 plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py new file mode 100644 index 00000000..e50f3add --- /dev/null +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py @@ -0,0 +1,223 @@ +"""Mutation control for the three owner escapes the sixteenth review round reported. + + R29-F31 the promise sentence names minutes in words the parser did not know, and the owner of + "the two forms are the same promise" held VACUOUSLY: it could not read the words, and + a promise nobody can check is not a promise anybody checked. + R29-F32 the page hands its timer a comma expression -- `(TRAJECTORY_POLL_INTERVAL, 8000000)` + -- and the owner read the characters around the name instead of the argument the timer + is really given, so the timer was seen to wait 8000 while it waits 8000000. + R29-F33 a template literal holding a declaration impersonated the proxy's numeric declaration + while the binding the service really uses was the string "2". + +Each case breaks one file and one rule, and a case counts as caught only when the guard reported +THAT rule -- by name -- and nothing else. The guard has to go red by NAMING a rule: a guard that +threw, or that went red without naming anything, has not shown that it owns this break. + +WHY THIS IS A SEPARATE FILE. `tests/mutate-parsed-rows.py` owns the forty cases of the cache's +own behaviour, and its EXACT table is the single source of truth for which rules each of those +breaks may report. Those forty cases and that table are frozen: this round changed no product +behaviour, only the owners of three rules, so adding these three to that file would have meant +rewriting a table that is not this round's to rewrite. A control whose table has two owners is a +table with no owner. So the three escapes get their own control, and the suite runs one more +file. + +The rules are checked in the way the round that found them asked for: the owner has to be a +BEHAVIOUR, not a shape. The page's script is run in a sandbox and the delay comes from the timer +the page itself starts; the service's numbers come from the module's own bindings. +""" +import json +import re +import subprocess +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parent +SERVER = ROOT / "miniapp" / "node" / "server.mjs" +CLIENT = ROOT / "miniapp" / "client" / "index.html" +PROBE = HERE / "verify-parsed-rows.mjs" + +# Each case: what it breaks, the file it breaks, the text before and after, the rule that owns +# it, and the EXACT set of rules that break may report. +# +# The exact set is the isolation, and it is what the frozen control's table does for the other +# forty: a guard that also reported a rule this break did not break has accused another owner, +# and a reader cannot tell which rule had actually been broken. +MUTATIONS = [ + ( + "the promise sentence names minutes in words the parser did not know", + SERVER, + "one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped", + "one sweep interval after the limit — thirteen minutes (750000 ms) with the shipped", + "the promise to a reader is the same promise in both of the forms it is written in", + ("the promise to a reader is the same promise in both of the forms it is written in",), + ), + ( + "the page hands its timer a comma expression", + CLIENT, + " }, TRAJECTORY_POLL_INTERVAL);", + " }, (TRAJECTORY_POLL_INTERVAL, 8000000));", + "the poll waits as long as the client declares", + ( + "the poll waits as long as the client declares", + "the time limit is longer than one poll, so an open tab never loses its rows", + ), + ), + ( + "a template literal impersonates the proxy's declaration while the binding is a string", + SERVER, + "\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n", + "\nconst annotation = `\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n`;\n" + "const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = '2';\n", + "the heap proxy is a number the service declares", + ("the heap proxy is a number the service declares",), + ), +] + + +def read_exact(path): + """Normalise to LF for matching, and report the file's own newline so it can be put back. + + Matching against the raw bytes would make every multi-line anchor miss -- which the runner + reports as a surviving mutation, so the failure looks like a dead guard rather than a + broken anchor. + """ + data = path.read_bytes() + text = data.decode("utf-8").replace("\r\n", "\n") + return text, ("\r\n" if b"\r\n" in data else "\n") + + +def write_exact(path, text, newline): + if newline == "\r\n": + text = text.replace("\n", "\r\n") + path.write_bytes(text.encode("utf-8")) + + +def run_probe(): + proc = subprocess.run( + ["node", str(PROBE)], + cwd=str(ROOT), + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + ) + return proc.returncode, (proc.stdout or "") + (proc.stderr or "") + + +def rule_name(text): + """The name of a rule a probe reported, without the evidence a comparison carries. + + A probe reports an assertion as `` and a comparison as + `: got , want `. The values are the evidence, not the rule, so they are + cut off -- and the cut is made at a `: got ` whose tail really is `, want `, so a + name that itself contains those words is not truncated. + """ + for match in reversed(list(re.finditer(re.escape(": got "), text))): + head = text[:match.start()].strip() + tail = text[match.end():] + cut = tail.rfind(", want ") + if cut < 0: + continue + try: + json.loads(tail[:cut]) + json.loads(tail[cut + len(", want "):]) + except ValueError: + continue + return head + return text + + +def reported_rules(output): + """The rules a probe named as broken. + + Two formats, as in `mutate-parsed-rows.py`: ` - ` and `FAIL `. A case is + only caught when the rule it names is among them. + """ + names = [] + for line in output.splitlines(): + text = line.strip() + if text.startswith("- "): + names.append(rule_name(text[2:].strip())) + elif text.startswith("FAIL "): + names.append(rule_name(text[5:].strip())) + return names + + +def main(): + code, output = run_probe() + if code != 0: + print("baseline probe does not pass; fix that first") + print(output) + return 1 + print("baseline clean") + + originals = {} + missed = [] + unnamed = [] + for label, target, old, new, expect, exact in MUTATIONS: + if target not in originals: + originals[target] = target.read_bytes() + original = originals[target] + text, newline = read_exact(target) + # The anchor has to be there EXACTLY once. A control that replaces the first occurrence of + # a string that occurs twice edits a place it did not mean to edit, and the case then + # passes or fails for a reason that has nothing to do with the rule it is about -- so the + # count is checked here, by the control itself, instead of being counted by somebody + # reading the source. + if text.count(old) != 1: + print(f"SKIP {label}: anchor found {text.count(old)} times") + missed.append(label) + continue + if old == new: + print(f"SKIP {label}: the mutation changes nothing") + missed.append(label) + continue + write_exact(target, text.replace(old, new, 1), newline) + try: + code, output = run_probe() + finally: + target.write_bytes(original) + if code == 0: + print(f"MISSED {label}: the probe still passed") + missed.append(label) + continue + names = reported_rules(output) + if not names: + print(f"UNNAMED {label}: the probe failed without naming a rule") + unnamed.append(label) + continue + # The case names the rule it is about, so the probe has to name THAT rule and not some + # other one that happened to go red in the same run: a probe that went red elsewhere has + # not shown that this guard owns this mutation. + if expect not in names: + print(f"MISSED {label}: the probe named {names} instead of {expect}") + missed.append(label) + continue + # When the case names the exact set of rules that may go red, a guard that also reported + # a rule this break did not break has failed the case: it accused another owner. + if set(names) != set(exact): + print(f"OVERREACH {label}: the probe named {sorted(set(names))} " + f"instead of exactly {sorted(set(exact))}") + missed.append(label) + continue + print(f"caught {label}\n -> {expect}") + + for target, original in originals.items(): + if target.read_bytes() != original: + print(f"NOT RESTORED {target}") + return 1 + print("file restored byte-for-byte") + + if missed or unnamed: + if missed: + print(f"\n{len(missed)} mutation(s) got through") + if unnamed: + print(f"{len(unnamed)} mutation(s) failed without naming a rule") + return 1 + print(f"\nall {len(MUTATIONS)} mutations caught, each by name") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index 1c97b057..a042669c 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -23,13 +23,17 @@ * a path, because every session has a `messages.jsonl` and a basename key lets two of them * collide; a size, because the active file grows; a limit, because readHead truncates at it. * - * Read-only: imports the pure helpers, never starts the runtime. - * mutate-parsed-rows.py breaks each of these in the shipped file. + * Read-only: it imports the pure helpers, and it runs the PAGE's own inline script in a sandbox + * whose DOM is stubbed; it never starts the runtime. + * mutate-parsed-rows.py and mutate-parsed-rows-owner-escapes.py break each of these in the + * shipped file. */ import { mkdtemp, mkdir, writeFile, rm, utimes, stat } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { readFileSync } from 'node:fs'; +import { pathToFileURL } from 'node:url'; +import vm from 'node:vm'; import { ROW_GENERATION, buildTrajectoryPayload, @@ -427,32 +431,238 @@ assert('loadParsedRows is exported so a test can run the real step', * somebody asks for them. * ===================================================================== */ -// A constant the source declares as an expression — `60 * 1000` — is read by arithmetic rather -// than by a digit pattern, so the check follows the constant instead of repeating it. -const declaredNumber = (source, name) => { - const match = source.match(new RegExp(`const ${name} = ([0-9_ *]+);`)); - if (match === null) return null; - // A declaration that is COMMENTED OUT is not a declaration. The first match in the file is - // otherwise enough to answer with a number the code does not have: an annotation left beside - // the real one satisfies the pattern while the binding is something else, and the real one - // turned into a string stops matching at all -- so the number read would be the annotation's - // and the binding's type would go unexamined. The line the match is on therefore has to - // DECLARE it: nothing but whitespace before it, and no block comment open around it. - const lineStart = source.lastIndexOf('\n', match.index) + 1; - if (source.slice(lineStart, match.index).trim() !== '') return null; - const before = source.slice(0, match.index); - if (before.split('/*').length !== before.split('*/').length) return null; - const parts = match[1].replace(/_/g, '').split('*').map((part) => Number(part.trim())); - if (parts.some((part) => !Number.isFinite(part))) return null; - return parts.reduce((carry, part) => carry * part, 1); +// A lexical mask of a JavaScript source: which characters are CODE, and which are inside a +// string, a template literal or a comment. +// +// A scanner that cannot tell those apart stops at a bracket that was never one, and reads a +// declaration that is only text. A template literal can hold a whole declaration -- and the +// service's own arithmetic cannot tell a number from the string that looks like one -- so the +// helpers below talk about the code, not about the characters. +// +// WHAT THIS IS NOT, and it is worth being exact about it: this is not a complete lexical +// analysis. A REGULAR EXPRESSION LITERAL is not recognised. `/` is both the division operator +// and the start of one, and telling them apart needs the parser this package does not have -- +// there is no JavaScript parser in it, and adding one is a bigger change than this guard is. +// +// The shipped service has such literals: miniapp/node/server.mjs:340, :345 and :349, and each of +// them holds a quote inside a character class. The scan reads that quote as the start of a string +// and swallows the text up to the next one, so the mask steps out of step with the file from +// line 340 on. +// +// The direction of that step is the part that matters, and it is LOUD rather than quiet. Every +// declaration this mask is asked about -- the time limit, at :203 -- is declared ABOVE those +// lines, so the mask is right about all of them today. And a declaration the mask could not find +// leaves the owner that wants it with nothing to read, which FAILS: the promise in brackets is +// read out of the file, and a file whose mask is out of step does not yield one. A mask that is +// out of step can only cost this guard a red, never a green. +const codeMask = (source) => { + const mask = new Uint8Array(source.length); + let i = 0; + while (i < source.length) { + const ch = source[i]; + if (ch === '/' && source[i + 1] === '/') { + while (i < source.length && source[i] !== '\n') i += 1; + continue; + } + if (ch === '/' && source[i + 1] === '*') { + i += 2; + while (i < source.length && !(source[i] === '*' && source[i + 1] === '/')) i += 1; + i += 2; + continue; + } + if (ch === '"' || ch === "'") { + const quote = ch; + i += 1; + while (i < source.length) { + if (source[i] === '\\') { i += 2; continue; } + if (source[i] === quote) { i += 1; break; } + i += 1; + } + continue; + } + if (ch === '`') { + i += 1; + let depth = 0; + while (i < source.length) { + if (source[i] === '\\') { i += 2; continue; } + if (source[i] === '$' && source[i + 1] === '{') { depth += 1; i += 2; continue; } + if (depth > 0 && source[i] === '}') { depth -= 1; i += 1; continue; } + if (depth === 0 && source[i] === '`') { i += 1; break; } + i += 1; + } + continue; + } + mask[i] = 1; + i += 1; + } + return mask; +}; + +// The page's own script, and the scope it declares. +// +// The interval the trajectory is polled at is declared by the page, and a constant read out of +// the page's TEXT is a line of text: a copy of the declaration inside a template literal +// satisfies any pattern over the text while the binding the page's timer really uses is +// something else. So the script is RUN, in a context that answers the DOM with stubs, and the +// values are read out of the scope it created. A template literal creates no binding, so a +// declaration that is only text cannot be read here. +// +// The DELAY is not read out of the text either. The page is asked to start its timers, the stub +// records what it was handed, and the timer that polls the trajectory is found by the handle the +// page itself stored in `trajectoryPollTimer` -- not by the order the timers were started in, +// and not by the biggest delay among them. So both halves of the comparison -- what the page +// declares, and what the page's own timer is really given -- are facts about the running page +// rather than about its characters. +// +// The context never lets the page start by itself: `document.readyState` is 'loading', so the +// page's own entry point is not called, no DOM is touched and no request is made. The guard +// starts the page's polling itself, because that is the thing being observed. +const CLIENT_SOURCE = readFileSync(new URL('../miniapp/client/index.html', import.meta.url), 'utf8'); +const CLIENT_SCRIPT = (() => { + const match = /]*>([\s\S]*?)<\/script>/i.exec(CLIENT_SOURCE); + return match === null ? null : match[1]; +})(); + +// A stub for anything the page asks the DOM for: a function that answers everything and does +// nothing. The script is run to observe the timers it starts, not to exercise the page, so the +// DOM only has to not be there. +const domStub = (name) => new Proxy(function stub() {}, { + get: (target, property) => { + if (property === 'then') return undefined; // not a thenable + if (property === Symbol.toPrimitive) return () => name; + return domStub(`${name}.${String(property)}`); + }, + apply: () => domStub(`${name}()`), + set: () => true, +}); + +// The scope the page's script declares, and the timers it starts. +// +// A script that will not run is a fact about the PAGE and not about this file's harness: the +// page's own entry point is not called here, so the only thing that can throw is the script's +// top level. Either way the owner below cannot read what it is about, and that is reported as a +// failure of this guard -- never as a quiet holding, and never as an accusation of the service. +// A guard that says nothing about a rule is a guard that has stopped holding it. +const CLIENT_SCOPE = (() => { + if (CLIENT_SCRIPT === null) { + return { ok: false, error: new Error('the page has no inline script') }; + } + const timers = []; + let nextHandle = 0; + const document = new Proxy({ readyState: 'loading' }, { + get: (target, property) => (property in target + ? target[property] + : domStub(`document.${String(property)}`)), + }); + const sandbox = { + console, + document, + fetch: () => new Promise(() => {}), + setTimeout: () => 0, + clearTimeout: () => {}, + // The page's timers are recorded, not run. The handle is a FRESH OBJECT every time, so the + // page's own global says which timer is which -- the page stores the handle it was given in + // `trajectoryPollTimer`, and the identity of that object is what identifies the timer. + setInterval: (_callback, delay) => { + const handle = { id: (nextHandle += 1) }; + timers.push({ handle, delay }); + return handle; + }, + clearInterval: () => {}, + matchMedia: () => domStub('matchMedia'), + location: { href: 'http://localhost/', search: '', hash: '' }, + navigator: domStub('navigator'), + }; + const context = vm.createContext(sandbox); + sandbox.window = sandbox; + try { + vm.runInContext(CLIENT_SCRIPT, context, { filename: 'client-inline.js', timeout: 20000 }); + // The page does not start its timers by itself -- its entry point waits for the DOM -- so the + // guard starts the polling, because that is the thing being observed. + if (typeof context.startPolling !== 'function') { + throw new Error('the page does not expose the function that starts its polling'); + } + context.startPolling(); + } catch (error) { + return { ok: false, error }; + } + return { ok: true, context, timers }; +})(); +if (!CLIENT_SCOPE.ok) { + failures.push('the page\'s inline script did not run, so the owner cannot read the delay the ' + + `page's poll waits: ${CLIENT_SCOPE.error && CLIENT_SCOPE.error.message}`); +} + +// A value read out of the page's own scope. +const readInPage = (expression) => { + if (!CLIENT_SCOPE.ok) return { ok: false, value: null }; + try { + return { ok: true, value: vm.runInContext(expression, CLIENT_SCOPE.context, { timeout: 5000 }) }; + } catch { + return { ok: false, value: null }; + } +}; + +// The values the shipped module really binds. +// +// A constant read out of the source TEXT is a line of text, not a binding: a template literal +// holding the same line satisfies any pattern over the text while the binding the service uses is +// something else -- a string, for instance, which the service's multiplication still coerces to the +// number it looks like, so every owner that reads the text goes on saying the rule holds. +// +// So the numbers are read from the module itself. A copy of it -- the shipped bytes with one export +// line appended -- is imported next to a copy of the module it imports relatively, and the values +// are read out of that namespace. The copy is the shipped file: the export line adds a name, not a +// value. +// +// If the copy cannot be imported at all, that is a fact about THIS FILE and not about the +// service: this guard imports the same module statically at the top of it, so a copy that will +// not import means the copy is broken. It is reported as a failure of this guard -- and NOT as a +// quiet holding. An owner that cannot read what it needs must not accuse the rules it is not +// about, and it must not go quiet either: the guard says that it could not read, and it stops. +const readModuleBindings = async (names) => { + try { + const dir = join(root, 'module'); + await mkdir(dir, { recursive: true }); + const nodeDir = new URL('../miniapp/node/', import.meta.url); + for (const file of ['server.mjs', 'search-segments.mjs']) { + await writeFile(join(dir, file), readFileSync(new URL(file, nodeDir))); + } + const shipped = readFileSync(new URL('server.mjs', nodeDir), 'utf8'); + await writeFile(join(dir, 'server.mjs'), `${shipped}\nexport { ${names.join(', ')} };\n`); + const copy = await import(pathToFileURL(join(dir, 'server.mjs')).href); + const values = {}; + for (const name of names) values[name] = copy[name]; + return { ok: true, values }; + } catch (error) { + return { ok: false, error }; + } }; +// The three numbers the service declares, read from the module itself. +const BINDINGS_RESULT = await readModuleBindings([ + 'PARSED_ROWS_HEAP_PER_SOURCE_BYTE', + 'PARSED_ROWS_CACHE_TTL_MS', + 'PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS', +]); +if (!BINDINGS_RESULT.ok) { + failures.push('the shipped module could not be imported by the copy this guard reads the ' + + `service's declared numbers from, so they cannot be read: ${BINDINGS_RESULT.error && BINDINGS_RESULT.error.message}`); +} +const BINDINGS = BINDINGS_RESULT.ok ? BINDINGS_RESULT.values : null; +const PROXY = BINDINGS === null ? null : BINDINGS.PARSED_ROWS_HEAP_PER_SOURCE_BYTE; +const TTL = BINDINGS === null ? null : BINDINGS.PARSED_ROWS_CACHE_TTL_MS; +const SHIPPED_SWEEP = BINDINGS === null ? null : BINDINGS.PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS; + // The doc block above a constant, so a promise the package states in prose can be read and // compared with what the code does. The block is taken around ONE declaration rather than the // whole file, because a whole-file match keeps passing after the promise has moved somewhere // else -- which is the way a rule quietly loses its owner. -const docBlockOf = (source, name) => { - const at = source.indexOf('const ' + name + ' ='); +// +// The declaration is found in the CODE, not in the text: a copy of it inside a template literal is +// not a declaration, and the block above such a copy is not the block above the real one. +const docBlockOf = (source, name, mask) => { + const at = declarationIndexOf(source, name, mask); if (at < 0) return null; const before = source.slice(0, at); const start = before.lastIndexOf('/**'); @@ -460,6 +670,19 @@ const docBlockOf = (source, name) => { return start >= 0 && end > start ? before.slice(start, end) : null; }; +// Where a constant is declared in the code of a source, or -1 when it is not declared in code. +const declarationIndexOf = (source, name, mask) => { + const pattern = new RegExp(`\\b(?:const|let|var)\\s+${name}\\s*=`, 'g'); + let match; + while ((match = pattern.exec(source)) !== null) { + if (mask[match.index]) return match.index; + } + return -1; +}; + +// The mask of the shipped service: which of its characters are code. +const SERVER_MASK = codeMask(SERVER); + // The sentence of a doc block that makes a promise. A block can hold several sentences, and only // one of them is the promise to a reader -- so the number has to be read out of THAT sentence and // not out of the block. A number that sat in some other sentence of the same block would answer @@ -471,23 +694,70 @@ const promiseSentenceOf = (block) => { return sentences.find((sentence) => sentence.includes('no later than')) ?? null; }; -// The minutes a sentence promises, in either form it can be written in. A promise to a reader is -// written both as a number of milliseconds and as a number of minutes, and the two forms have to -// be the same promise -- otherwise the reader is told one thing and the service does another. -const minutesPromised = (sentence) => { - if (sentence === null) return null; - const words = { - one: 1, two: 2, three: 3, four: 4, five: 5, six: 6, seven: 7, eight: 8, nine: 9, ten: 10, - eleven: 11, twelve: 12, twenty: 20, thirty: 30, forty: 40, fifty: 50, sixty: 60, - }; - const match = /\b(\d+(?:\.\d+)?|one|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve|twenty|thirty|forty|fifty|sixty)(?:\s+and\s+a\s+half)?\s+minutes?\b/i - .exec(sentence); - if (match === null) return null; - const raw = match[1].toLowerCase(); - let value = /^\d/.test(raw) ? Number(raw) : words[raw]; - if (value === undefined || !Number.isFinite(value)) return null; - if (/\s+and\s+a\s+half/.test(match[0])) value += 0.5; - return value; +// The number words a promise may be written in. +const UNITS = { + zero: 0, one: 1, two: 2, three: 3, four: 4, five: 5, six: 6, seven: 7, eight: 8, nine: 9, + ten: 10, eleven: 11, twelve: 12, thirteen: 13, fourteen: 14, fifteen: 15, sixteen: 16, + seventeen: 17, eighteen: 18, nineteen: 19, +}; +const TENS = { + twenty: 20, thirty: 30, forty: 40, fifty: 50, sixty: 60, seventy: 70, eighty: 80, ninety: 90, +}; + +// The number a run of words states, read from its END: the words of a number are the last ones +// before "minutes", and the words before them belong to the sentence around it. +const englishNumber = (text) => { + const words = text.toLowerCase().replace(/-/g, ' ').split(/\s+/).filter(Boolean); + let half = false; + if (words.length >= 3 + && words[words.length - 3] === 'and' + && words[words.length - 2] === 'a' + && words[words.length - 1] === 'half') { + half = true; + words.splice(words.length - 3, 3); + } + const last = words[words.length - 1]; + let value = null; + if (/^\d+(?:\.\d+)?$/.test(last)) value = Number(last); + else if (last in UNITS) value = UNITS[last]; + else if (last in TENS) { + const unit = words.length >= 2 ? UNITS[words[words.length - 2]] : undefined; + value = unit === undefined ? TENS[last] : TENS[last] + unit; + } + if (value === null) return null; + return half ? value + 0.5 : value; +}; + +// The minutes a sentence promises, and whether it states them in the form the promise is written +// in. +// +// The number words are parsed COMPLETELY. A parser that knows only some of the numbers a promise +// may be written in answers "I did not understand" for the rest -- and an owner that accepts that +// answer as agreement lets a promise state two different numbers and still report success. So +// every number word a promise can use is understood here, and a phrase that is still not +// understood is not agreement: it is a promise nobody can check. +// +// The word "minutes" is the one the millisecond form FOLLOWS. The promise writes the two forms +// together, and a sentence can hold other uses of the word -- "the rows are gone within a few +// minutes, and no later than twelve and a half minutes (750000 ms)" -- whose words are not the +// promise. Reading the wrong one accuses the promise of a number it never stated, so the phrase +// is taken from the word the millisecond form is attached to and from nowhere else. +// +// TWO ANSWERS, and they are kept apart. `written` says the promise states the minutes at all; +// `minutes` is the number, and it is null when the words are there but cannot be read. "There are +// no minutes in this promise" and "the minutes in this promise are words nobody can read" are +// different facts about the promise, and an owner that treats them as one thing either accuses +// the promise of a number it never stated or lets a promise nobody can check pass. +const minutesForm = (sentence) => { + if (sentence === null) return { written: false, minutes: null }; + const anchor = /\bminutes?\s*\(/i.exec(sentence); + if (anchor === null) return { written: false, minutes: null }; + const at = anchor.index; + let start = at; + while (start > 0 && /[A-Za-z0-9\s.-]/.test(sentence[start - 1])) start -= 1; + const phrase = sentence.slice(start, at).trim(); + const digits = /^\s*(\d+(?:\.\d+)?)\s*$/.exec(phrase); + return { written: true, minutes: digits !== null ? Number(digits[1]) : englishNumber(phrase) }; }; // The lifetime the cache gives its rows when nobody tells it one, observed rather than read. @@ -526,47 +796,12 @@ const defaultLimitOf = (budget) => { return high; }; -// The delay the page's poll actually waits, observed by evaluating the expression the timer is -// given. The declaration of the interval is a number in the source; the delay is what the poll -// really waits, and a poll that waits longer than the limit it has to survive loses its rows -// between polls while every check that reads the declaration stays green. -const effectivePollDelay = (clientSource) => { - const declared = /TRAJECTORY_POLL_INTERVAL\s*=\s*(\d+)\s*;/.exec(clientSource); - if (declared === null) return null; - const value = Number(declared[1]); - // Every use of the name that is not its declaration, and the expression it is part of. - const uses = []; - const pattern = /TRAJECTORY_POLL_INTERVAL/g; - let match; - while ((match = pattern.exec(clientSource)) !== null) { - const after = clientSource.slice(match.index + 'TRAJECTORY_POLL_INTERVAL'.length); - if (/^\s*=\s*\d+/.test(after)) continue; // the declaration itself - let start = match.index; - while (start > 0 && /[A-Za-z0-9_.*+\-/<> ]/.test(clientSource[start - 1])) start -= 1; - let end = match.index + 'TRAJECTORY_POLL_INTERVAL'.length; - while (end < clientSource.length && /[A-Za-z0-9_.*+\-/<> ]/.test(clientSource[end])) end += 1; - uses.push(clientSource.slice(start, end).trim()); - } - if (uses.length === 0) return null; - const delays = []; - for (const expression of uses) { - let evaluated; - try { - // The expression is the page's own code, evaluated with the value the page declares. - evaluated = new Function('TRAJECTORY_POLL_INTERVAL', `return (${expression});`)(value); - } catch { - return null; - } - if (!Number.isFinite(evaluated)) return null; - delays.push(evaluated); - } - return { declared: value, delays }; -}; - -// The proxy is read out of the shipped source rather than repeated here, so the relationship -// below is checked against the constant the service actually charges with. -const PROXY = declaredNumber(SERVER, 'PARSED_ROWS_HEAP_PER_SOURCE_BYTE'); -assert('the heap proxy is a number the service declares', PROXY !== null); +// The proxy is read out of the shipped MODULE rather than repeated here, so the relationship +// below is checked against the constant the service actually charges with -- and against the type +// it really has: the service's multiplication coerces the string "2" to the number 2, so only the +// binding itself can tell whether the proxy is a number or the text of one. +assert('the heap proxy is a number the service declares', + holds(typeof PROXY, 'number', PROXY !== null)); // A file of a known size, loaded through the shipped step: the cache must be charged the source // bytes times the proxy, so the source bytes it holds are recoverable from its own accounting. @@ -637,28 +872,50 @@ check('and the cache is empty', liveCache.size, 0); // The time limit has to outlive one poll, or an open tab would lose its rows between polls. // // Both numbers here are OBSERVED, and that is the point of this block. The delay the poll waits -// and the lifetime the cache gives its rows were both read out of the source before, and a source -// says what the code INTENDS: the number a constant is declared with, and the number the page -// writes next to its timer. A cache whose default lifetime comes from somewhere else, or a poll -// that multiplies the interval it declared, leaves every declaration exactly as it was while the -// service behaves differently -- and the promise to a reader is about the service, not about a -// declaration. -const clientSource = readFileSync(new URL('../miniapp/client/index.html', import.meta.url), 'utf8'); -const pollMatch = clientSource.match(/TRAJECTORY_POLL_INTERVAL = (\d+)/); -assert('the client says how often it polls the trajectory', pollMatch !== null); -const ttlMatch = SERVER.match(/const PARSED_ROWS_CACHE_TTL_MS = ([0-9_ *]+);/); -assert('the rows cache has a time limit', ttlMatch !== null); -const TTL = declaredNumber(SERVER, 'PARSED_ROWS_CACHE_TTL_MS'); -assert('the time limit is a number the service declares', TTL !== null); - -// The delay the page's timer is really given, and the lifetime the cache really gives its rows. -const poll = effectivePollDelay(clientSource); +// is what the page really hands to its timer, and the lifetime the cache gives its rows is what +// the running cache gives them. A source says what the code INTENDS -- the number a constant is +// declared with, and the number the page writes next to its timer -- and a cache whose default +// lifetime comes from somewhere else, or a poll that multiplies the interval it declared, leaves +// every declaration exactly as it was while the service behaves differently -- and the promise to +// a reader is about the service, not about a declaration. +// The interval the page declares, read out of the scope the page's own script created. +// +// The declaration has to be a NUMBER: this value is what the page hands to a timer, and a +// string that looks like one would be handed over as a string. +const DECLARED = readInPage('TRAJECTORY_POLL_INTERVAL'); +const DECLARED_INTERVAL = DECLARED.ok ? DECLARED.value : null; +assert('the client declares the interval it polls the trajectory at', + DECLARED_INTERVAL !== null + && typeof DECLARED_INTERVAL === 'number' + && Number.isFinite(DECLARED_INTERVAL)); + +// The timer the page polls the trajectory with, and the delay it was really given. +// +// The timer is found by the HANDLE the page stored in `trajectoryPollTimer` -- the page starts a +// second timer for the session list, and the two are told apart by the object the page was +// given, not by the order they were started in and not by which delay is the larger one. +// +// The delay is what the page handed to the timer, which is not the same thing as the text +// around the interval's name: a comma expression hides a second value behind the first, so +// `(TRAJECTORY_POLL_INTERVAL, 8000000)` gives the timer eight million milliseconds while the +// characters around the name still read "TRAJECTORY_POLL_INTERVAL". +const TRAJECTORY_TIMER = CLIENT_SCOPE.ok + && CLIENT_SCOPE.context.trajectoryPollTimer !== null + && CLIENT_SCOPE.context.trajectoryPollTimer !== undefined + ? CLIENT_SCOPE.timers.find((timer) => timer.handle === CLIENT_SCOPE.context.trajectoryPollTimer) + : undefined; +const POLL_DELAY = TRAJECTORY_TIMER === undefined ? null : TRAJECTORY_TIMER.delay; assert('the poll waits as long as the client declares', - poll !== null && poll.delays.every((delay) => delay === poll.declared)); -const POLL_DELAY = poll === null ? null : Math.max(...poll.delays); + POLL_DELAY !== null && POLL_DELAY === DECLARED_INTERVAL); + +assert('the rows cache has a time limit', holds(TTL !== null, true, BINDINGS !== null)); +assert('the time limit is a number the service declares', + holds(typeof TTL, 'number', BINDINGS !== null)); + +// The lifetime the cache really gives its rows. const DEFAULT_LIMIT = defaultLimitOf(1024 * 1024); assert('the limit the service declares is the limit the cache gives its rows', - DEFAULT_LIMIT === TTL); + holds(DEFAULT_LIMIT, TTL, BINDINGS !== null)); assert('the time limit is longer than one poll, so an open tab never loses its rows', POLL_DELAY !== null && DEFAULT_LIMIT > POLL_DELAY); // The page stops polling while its tab is hidden, so a reader who switches away for a while is @@ -691,8 +948,8 @@ assert('the sweeper does not keep the process alive', // is a fraction of the limit. This is checked on the interval the cache ACTUALLY uses, for // several limits — not on two numbers read out of the source, which would keep passing after // the line that takes the interval had changed. -const SHIPPED_SWEEP = declaredNumber(SERVER, 'PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS'); -assert('the sweep interval is a number the service declares', SHIPPED_SWEEP !== null); +assert('the sweep interval is a number the service declares', + holds(typeof SHIPPED_SWEEP, 'number', BINDINGS !== null)); for (const limit of [1_000, 60_000, 600_000, 3_600_000]) { const bounded = createParsedRowsCache(1000, { ttlMs: limit }); assert('the sweep interval the cache uses is at most a quarter of the limit', @@ -719,19 +976,35 @@ assert('the shipped default is the interval the service declares', // something else. So the number is taken from the promise sentence, that sentence's own words // are read as well, and the two are compared with each other and with the limit the cache really // gives its rows. -const ttlDoc = docBlockOf(SERVER, 'PARSED_ROWS_CACHE_TTL_MS'); +const ttlDoc = docBlockOf(SERVER, 'PARSED_ROWS_CACHE_TTL_MS', SERVER_MASK); const promiseSentence = promiseSentenceOf(ttlDoc); const promisedNumbers = promiseSentence === null ? [] : [...promiseSentence.matchAll(/\((\d+) ms\)/g)].map((found) => Number(found[1])); -const promisedMinutes = minutesPromised(promiseSentence); +const MINUTES_FORM = minutesForm(promiseSentence); +const promisedMinutes = MINUTES_FORM.minutes; const DEFAULT_INTERVAL = createParsedRowsCache(1000).sweepIntervalMs; assert('the number promised to a reader is the limit plus one sweep interval', promisedNumbers.length === 1 && promisedNumbers[0] === DEFAULT_LIMIT + DEFAULT_INTERVAL); +// The two forms have to be the SAME promise. A promise whose words cannot be read is not +// agreement: the words are parsed completely above, so a phrase that is still not a number is a +// promise nobody can check, and this owner says so rather than passing it. +// +// And a promise that is written in only ONE of the two forms is nothing for this owner to +// compare -- the minutes are there and the milliseconds are not, or the other way round. That is +// counted with the result, so a reader can tell it from one that was checked and matched. +// +// So the two answers about the minutes are kept apart here, and they are what the two cases are. +// The minutes are stated next to the millisecond form, the promise writes both forms, and the two +// have to agree: a promise that states the minutes in words nobody can read is a promise nobody +// can check, and it is NOT the same thing as a promise that states only one form. +const samePromiseInBothForms = promisedMinutes !== null + && promisedNumbers.length === 1 + && Math.abs(promisedMinutes * 60_000 - promisedNumbers[0]) < 1; +const bothFormsWritten = MINUTES_FORM.written && promisedNumbers.length === 1; assert('the promise to a reader is the same promise in both of the forms it is written in', - promisedMinutes === null || promisedNumbers.length === 0 - || Math.abs(promisedMinutes * 60_000 - promisedNumbers[0]) < 1); + holds(samePromiseInBothForms, true, bothFormsWritten)); // The promise above holds only if the TIMER runs on that interval: the rows are released on the // next tick, and the tick is the sweep. Nothing in the package watched the call that starts that From aae3468b1d8c220ca1b1a5b553008e7dceebea67 Mon Sep 17 00:00:00 2001 From: avatasia Date: Sat, 24 Oct 2026 02:17:17 +0800 Subject: [PATCH 79/95] The promise is read as the whole phrase it is written in, and a number the guard could not read accuses nothing Two owners were still deciding a rule from a piece of a sentence. Both are fixed here, and the product is not changed. - the owner of the rule that the promise to a reader is the same promise in both of the forms it is written in read the minutes out of the END of the words. The words of a number are not only its last word: "one hundred twelve and a half minutes (750000 ms)" states 112.5 minutes, and the parser dropped "one hundred" and answered 12.5 -- which is what 750000 ms says. So the promise could state one number in words and another in milliseconds, and the owner called that agreement. The trailing run of number words is now collected by its boundary and parsed as a whole, "hundred" included, and a phrase that holds a compound this parser does not state is reported as unreadable rather than answered with the part that happened to be understood. The comment above the parser already claimed that the number words are parsed completely; it is now true, and the fix was in the parser rather than in the comment, because a comment that understates what the code does has the same shape as an owner that checks less than it claims. - the owner of the rule that the shipped default is the interval the service declares compared the interval the cache really uses with the number the service declares -- even when that number could not be read. A copy of the module that would not import left the declared value empty, and "150000 === nothing" was reported as a broken rule: the harness broke, and the guard accused a rule nobody had broken. The comparison is made only when the declaration could be observed, the failure to read has a rule of its own that is named, and the owners that depend on the observation are listed, so the guard can check that it accused none of them. That check has teeth of its own: a case that breaks the copy AND leaves the comparison without its gate has to make the guard name that rule. The premise is worth stating, because it is a premise and not a fact: the guard reads the service's declared numbers out of a copy of the module, and that copy can fail to import. When it does, the guard fails loudly and by name, the owners that could not be compared count themselves as vacuous instead of holding quietly, and no owner that depends on the observation accuses anything. An owner that goes quiet is a guard that has stopped holding the rule it names. Three mutations were added, one for each of the three shapes: the compound minutes, the copy that will not import, and the two together. They live in the control of the escapes, which holds them to the same standard as the three before them -- each break has to be caught by name, and by the only rules that break really breaks. The control of the cache's own forty cases and the table it reads are not this change's to rewrite. The suite runs the same number of files; the guard now reports ninety-five assertions, and the two it gained are the rules about its own observation. The service and the page are not changed by this commit. --- .../tests/mutate-parsed-rows-owner-escapes.py | 207 +++++++++++++----- .../tests/verify-parsed-rows.mjs | 110 ++++++++-- 2 files changed, 250 insertions(+), 67 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py index e50f3add..a710f27e 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py @@ -1,5 +1,6 @@ -"""Mutation control for the three owner escapes the sixteenth review round reported. +"""Mutation control for the owner escapes two review rounds reported. + Round sixteen: R29-F31 the promise sentence names minutes in words the parser did not know, and the owner of "the two forms are the same promise" held VACUOUSLY: it could not read the words, and a promise nobody can check is not a promise anybody checked. @@ -9,21 +10,41 @@ R29-F33 a template literal holding a declaration impersonated the proxy's numeric declaration while the binding the service really uses was the string "2". + Round seventeen: + R29-F34 the same parser read the END of the words and dropped the rest, so "one hundred twelve + and a half minutes (750000 ms)" was read as twelve and a half -- a promise that states + one number in words and another in milliseconds, and the owner called it agreement. + R29-F35 the guard's own copy could not be imported, and the owner of "the shipped default is + the interval the service declares" compared the interval the cache really uses with the + number it could not read -- so a broken harness accused a rule that was not broken. + Each case breaks one file and one rule, and a case counts as caught only when the guard reported THAT rule -- by name -- and nothing else. The guard has to go red by NAMING a rule: a guard that threw, or that went red without naming anything, has not shown that it owns this break. WHY THIS IS A SEPARATE FILE. `tests/mutate-parsed-rows.py` owns the forty cases of the cache's own behaviour, and its EXACT table is the single source of truth for which rules each of those -breaks may report. Those forty cases and that table are frozen: this round changed no product -behaviour, only the owners of three rules, so adding these three to that file would have meant -rewriting a table that is not this round's to rewrite. A control whose table has two owners is a -table with no owner. So the three escapes get their own control, and the suite runs one more -file. +breaks may report. Those forty cases and that table are frozen: neither round changed product +behaviour, only the owners of rules, so adding these to that file would have meant rewriting a +table that is not this round's to rewrite. A control whose table has two owners is a table with no +owner. So the escapes get their own control, and the suite runs one more file. -The rules are checked in the way the round that found them asked for: the owner has to be a +THE RULES ARE CHECKED THE WAY THE ROUNDS THAT FOUND THEM ASKED FOR: the owner has to be a BEHAVIOUR, not a shape. The page's script is run in a sandbox and the delay comes from the timer the page itself starts; the service's numbers come from the module's own bindings. + +THE LAST TWO CASES BREAK THE GUARD ITSELF, and they are the two shapes a control of this kind +has to be able to state: + + * a case can break the copy the guard reads the service's numbers from, so the guard cannot + observe them. The guard then has to FAIL LOUDLY -- by naming the rule it could not observe -- + and it must not accuse any other rule: a number the guard could not read is not a broken + rule. The exact set is what says so: a guard that also reported "the shipped default is the + interval the service declares" would have accused a rule the break did not break. + * a case can break TWO things at once, when the escape only exists in that combination. The + mis-accusation needs the observation to have failed AND the owner to compare anyway, so the + case removes both and the guard has to name the rule that says it accuses nothing it could + not observe. """ import json import re @@ -37,41 +58,105 @@ CLIENT = ROOT / "miniapp" / "client" / "index.html" PROBE = HERE / "verify-parsed-rows.mjs" -# Each case: what it breaks, the file it breaks, the text before and after, the rule that owns -# it, and the EXACT set of rules that break may report. +# Each case: what it breaks, the edits that break it, the rule that owns it, the EXACT set of +# rules that break may report, and anything the output has to say. # # The exact set is the isolation, and it is what the frozen control's table does for the other # forty: a guard that also reported a rule this break did not break has accused another owner, # and a reader cannot tell which rule had actually been broken. +# +# The anchor of every edit has to occur EXACTLY once in the file it edits. A control that replaces +# the first occurrence of a string that occurs twice edits a place it did not mean to edit, and +# the case then passes or fails for a reason that has nothing to do with the rule it is about. MUTATIONS = [ - ( - "the promise sentence names minutes in words the parser did not know", - SERVER, - "one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped", - "one sweep interval after the limit — thirteen minutes (750000 ms) with the shipped", - "the promise to a reader is the same promise in both of the forms it is written in", - ("the promise to a reader is the same promise in both of the forms it is written in",), - ), - ( - "the page hands its timer a comma expression", - CLIENT, - " }, TRAJECTORY_POLL_INTERVAL);", - " }, (TRAJECTORY_POLL_INTERVAL, 8000000));", - "the poll waits as long as the client declares", - ( + { + "label": "the promise sentence names minutes in words the parser did not know", + "edits": [ + ( + SERVER, + "one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped", + "one sweep interval after the limit — thirteen minutes (750000 ms) with the shipped", + ), + ], + "expect": "the promise to a reader is the same promise in both of the forms it is written in", + "exact": ("the promise to a reader is the same promise in both of the forms it is written in",), + }, + { + "label": "the page hands its timer a comma expression", + "edits": [ + ( + CLIENT, + " }, TRAJECTORY_POLL_INTERVAL);", + " }, (TRAJECTORY_POLL_INTERVAL, 8000000));", + ), + ], + "expect": "the poll waits as long as the client declares", + "exact": ( "the poll waits as long as the client declares", "the time limit is longer than one poll, so an open tab never loses its rows", ), - ), - ( - "a template literal impersonates the proxy's declaration while the binding is a string", - SERVER, - "\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n", - "\nconst annotation = `\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n`;\n" - "const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = '2';\n", - "the heap proxy is a number the service declares", - ("the heap proxy is a number the service declares",), - ), + }, + { + "label": "a template literal impersonates the proxy's declaration while the binding is a string", + "edits": [ + ( + SERVER, + "\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n", + "\nconst annotation = `\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n`;\n" + "const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = '2';\n", + ), + ], + "expect": "the heap proxy is a number the service declares", + "exact": ("the heap proxy is a number the service declares",), + }, + { + "label": "the minutes are written as a compound the parser reads as a smaller number", + "edits": [ + ( + SERVER, + "after the limit — twelve and a half minutes (750000 ms)", + "after the limit — one hundred twelve and a half minutes (750000 ms)", + ), + ], + "expect": "the promise to a reader is the same promise in both of the forms it is written in", + "exact": ("the promise to a reader is the same promise in both of the forms it is written in",), + }, + { + "label": "the copy the guard reads the service's numbers from will not import", + "edits": [ + ( + PROBE, + "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { ${names.join(', ')} };\\n`);", + "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { nonexistentBinding };\\n`);", + ), + ], + "expect": "the guard could not observe the numbers the service declares", + "exact": ("the guard could not observe the numbers the service declares",), + # The owners that could not be compared have to say so: an owner that held quietly is an + # owner nobody can tell apart from one that was checked and matched. + "must_contain": ("held vacuously",), + }, + { + "label": "the copy will not import and the owner compares the interval anyway", + "edits": [ + ( + PROBE, + "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { ${names.join(', ')} };\\n`);", + "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { nonexistentBinding };\\n`);", + ), + ( + PROBE, + "holds(createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP, true, BINDINGS !== null)", + "holds(createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP, true, true)", + ), + ], + "expect": "the guard accuses no rule it could not observe", + "exact": ( + "the guard accuses no rule it could not observe", + "the guard could not observe the numbers the service declares", + "the shipped default is the interval the service declares", + ), + }, ] @@ -132,7 +217,8 @@ def reported_rules(output): """The rules a probe named as broken. Two formats, as in `mutate-parsed-rows.py`: ` - ` and `FAIL `. A case is - only caught when the rule it names is among them. + only caught when the rule it names is among them. A NOTE the probe printed -- the reason a + copy would not import -- is not a rule and is not among them. """ names = [] for line in output.splitlines(): @@ -155,29 +241,37 @@ def main(): originals = {} missed = [] unnamed = [] - for label, target, old, new, expect, exact in MUTATIONS: - if target not in originals: - originals[target] = target.read_bytes() - original = originals[target] - text, newline = read_exact(target) - # The anchor has to be there EXACTLY once. A control that replaces the first occurrence of - # a string that occurs twice edits a place it did not mean to edit, and the case then - # passes or fails for a reason that has nothing to do with the rule it is about -- so the - # count is checked here, by the control itself, instead of being counted by somebody - # reading the source. - if text.count(old) != 1: - print(f"SKIP {label}: anchor found {text.count(old)} times") + for case in MUTATIONS: + label = case["label"] + expect = case["expect"] + exact = case["exact"] + must_contain = case.get("must_contain", ()) + # Every edit has to land, or the case does not run at all: a case that did not happen + # proves nothing, and reporting it as "got through" would blame the guard for an anchor + # that has gone stale. + stale = None + for target, old, new in case["edits"]: + if target not in originals: + originals[target] = target.read_bytes() + text, newline = read_exact(target) + if text.count(old) != 1: + stale = f"anchor found {text.count(old)} times" + break + if old == new: + stale = "the mutation changes nothing" + break + if stale is not None: + print(f"SKIP {label}: {stale}") missed.append(label) continue - if old == new: - print(f"SKIP {label}: the mutation changes nothing") - missed.append(label) - continue - write_exact(target, text.replace(old, new, 1), newline) + for target, old, new in case["edits"]: + text, newline = read_exact(target) + write_exact(target, text.replace(old, new, 1), newline) try: code, output = run_probe() finally: - target.write_bytes(original) + for target in {target for target, _, _ in case["edits"]}: + target.write_bytes(originals[target]) if code == 0: print(f"MISSED {label}: the probe still passed") missed.append(label) @@ -201,6 +295,13 @@ def main(): f"instead of exactly {sorted(set(exact))}") missed.append(label) continue + # Anything the case says the probe has to say -- the owners that could not be compared + # have to say that they could not be compared. + missing = [needle for needle in must_contain if needle not in output] + if missing: + print(f"MISSED {label}: the probe did not say {missing}") + missed.append(label) + continue print(f"caught {label}\n -> {expect}") for target, original in originals.items(): diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index a042669c..82b404f2 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -646,9 +646,13 @@ const BINDINGS_RESULT = await readModuleBindings([ 'PARSED_ROWS_CACHE_SWEEP_INTERVAL_MS', ]); if (!BINDINGS_RESULT.ok) { - failures.push('the shipped module could not be imported by the copy this guard reads the ' - + `service's declared numbers from, so they cannot be read: ${BINDINGS_RESULT.error && BINDINGS_RESULT.error.message}`); + // The reason, as a note and not as a failure of its own: the rule that could not be observed is + // named by the assertion below, and a second entry in the failure list would be the same fact + // told twice -- which is how a reader stops knowing which entry is the rule. + console.error(' note: the copy this guard reads the service\'s declared numbers from would not ' + + `import: ${BINDINGS_RESULT.error && BINDINGS_RESULT.error.message}`); } +assert('the guard could not observe the numbers the service declares', BINDINGS_RESULT.ok); const BINDINGS = BINDINGS_RESULT.ok ? BINDINGS_RESULT.values : null; const PROXY = BINDINGS === null ? null : BINDINGS.PARSED_ROWS_HEAP_PER_SOURCE_BYTE; const TTL = BINDINGS === null ? null : BINDINGS.PARSED_ROWS_CACHE_TTL_MS; @@ -704,8 +708,59 @@ const TENS = { twenty: 20, thirty: 30, forty: 40, fifty: 50, sixty: 60, seventy: 70, eighty: 80, ninety: 90, }; -// The number a run of words states, read from its END: the words of a number are the last ones -// before "minutes", and the words before them belong to the sentence around it. +// The words that can be part of a number. A word in here that is not understood below is NOT +// dropped: the phrase is then a number this parser cannot state, and it says so. +const NUMBER_PARTS = new Set([...Object.keys(UNITS), ...Object.keys(TENS), + 'hundred', 'thousand', 'million', 'billion', 'dozen', 'score', 'and']); +const DIGIT_PART = /^\d+(?:\.\d+)?$/; + +// The number a run of number words states, or null when the run does not state one. +// +// The WHOLE run is parsed, because the words of a number are not only its last word: "one hundred +// twelve and a half" states 112.5, and a parser that reads the end of the run alone answers 12.5 +// -- a number the phrase never states, so a promise can say one thing in words and another in +// milliseconds and still be accepted. +const parseNumberWords = (words) => { + let total = 0; + let current = 0; + let unitBefore = false; + let sawWords = false; + let digits = null; + for (const word of words) { + if (word === 'and') continue; + if (DIGIT_PART.test(word)) { + if (digits !== null) return null; + digits = Number(word); + continue; + } + if (word in UNITS) { current += UNITS[word]; unitBefore = true; sawWords = true; continue; } + if (word in TENS) { current += TENS[word]; unitBefore = false; sawWords = true; continue; } + if (word === 'hundred') { + // "hundred" multiplies the one to nine before it: "one hundred" is 100 and "one hundred + // twelve" is 112. Anything else is a compound this parser does not state. + if (!unitBefore || current < 1 || current > 9) return null; + total += current * 100; + current = 0; + unitBefore = false; + sawWords = true; + continue; + } + // A scale this parser does not state -- "thousand" -- or a word that is not part of a number. + return null; + } + if (sawWords && digits !== null) return null; // a phrase that states two numbers + if (digits !== null) return digits; + if (!sawWords) return null; + return total + current; +}; + +// The number a phrase of words states. +// +// The number is the TRAILING run of words that can be part of a number; the words before it belong +// to the sentence around it, so "no later than twelve and a half" states twelve and a half. The +// run is collected by its boundary rather than by the first word the parser fails to know, so a +// phrase that holds a compound the parser does not state is reported as unreadable instead of +// being answered with the part that happened to be understood. const englishNumber = (text) => { const words = text.toLowerCase().replace(/-/g, ' ').split(/\s+/).filter(Boolean); let half = false; @@ -716,14 +771,11 @@ const englishNumber = (text) => { half = true; words.splice(words.length - 3, 3); } - const last = words[words.length - 1]; - let value = null; - if (/^\d+(?:\.\d+)?$/.test(last)) value = Number(last); - else if (last in UNITS) value = UNITS[last]; - else if (last in TENS) { - const unit = words.length >= 2 ? UNITS[words[words.length - 2]] : undefined; - value = unit === undefined ? TENS[last] : TENS[last] + unit; - } + let start = words.length; + while (start > 0 + && (NUMBER_PARTS.has(words[start - 1]) || DIGIT_PART.test(words[start - 1]))) start -= 1; + if (start === words.length) return null; + const value = parseNumberWords(words.slice(start)); if (value === null) return null; return half ? value + 0.5 : value; }; @@ -958,8 +1010,12 @@ for (const limit of [1_000, 60_000, 600_000, 3_600_000]) { const asked = createParsedRowsCache(1000, { ttlMs: 1_000, sweepIntervalMs: 500_000 }); assert('an interval the caller asks for is clamped to the limit as well', asked.sweepIntervalMs <= 1_000 / 4); +// The interval the cache gives its rows by default is the one the service DECLARES. The +// comparison needs the declared value, so it is made only when the declaration could be observed: +// a guard that cannot read the number must not accuse the rule because the number it could not +// read is not the number the cache uses. The failure to read has its own rule, above. assert('the shipped default is the interval the service declares', - createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP); + holds(createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP, true, BINDINGS !== null)); // The number a reader is promised: the rows are gone no later than the limit plus one sweep // interval. The bound is computed here from the limit the service DECLARES and the interval @@ -1026,13 +1082,18 @@ assert('the promise to a reader is the same promise in both of the forms it is w // has to start" is a different rule with a different owner (the rows have to be released without a // request). So the check below holds for every timer the cache starts, and holds vacuously when // it starts none: that keeps the two rules apart instead of making one of them the other's echo. +// +// And when the limit the cache was given could not be observed, the check is vacuous and COUNTS +// itself rather than holding quietly: the interval the cache really got is not knowable then, and +// an owner that says nothing about that is how a premise breaks without anybody noticing. let timedCache = null; const timerWindow = withRecordedTimers(() => { timedCache = createParsedRowsCache(1000, { ttlMs: TTL }); timedCache.set('k', val, 500); }); assert('the timer that releases the rows runs on the interval the cache reports', - timerWindow.recorded.every((interval) => interval === timedCache.sweepIntervalMs)); + holds(timerWindow.recorded.every((interval) => interval === timedCache.sweepIntervalMs), + true, TTL !== null)); // The quarter has a floor of a millisecond, because a timer cannot be finer than that. For a // limit below four milliseconds the floor is the larger of the two, so the release is then at @@ -1079,6 +1140,27 @@ const zeroTimerCalls = withRecordedTimers(() => { assert('a time limit of zero starts no sweeper, because nothing in it ever expires', zeroTimerCalls.recorded.length === 0); +// The owners that depend on the observation of the service's declared numbers must not accuse +// anything when that observation failed: a number the guard could not read is not a broken rule. +// This check is here, at the end, because it looks at what the other owners reported. +// +// The list is the owners that depend on the observation. Adding an owner that depends on it is +// adding a line here -- a list that is not kept in step is a list that quietly stops covering. +const OBSERVED_OWNERS = [ + 'the heap proxy is a number the service declares', + 'the cache is charged the source bytes times the heap proxy', + 'so the source bytes it holds are computable from its own accounting', + 'and the charge the route makes is that one', + 'the rows cache has a time limit', + 'the time limit is a number the service declares', + 'the limit the service declares is the limit the cache gives its rows', + 'the sweep interval is a number the service declares', + 'the shipped default is the interval the service declares', + 'the timer that releases the rows runs on the interval the cache reports', +]; +assert('the guard accuses no rule it could not observe', + BINDINGS !== null || !OBSERVED_OWNERS.some((name) => failures.includes(name))); + await rm(root, { recursive: true, force: true }); /* ===================================================================== */ From adb74b94c68f82cdcd662cec239f7f9723cf651d Mon Sep 17 00:00:00 2001 From: avatasia Date: Sat, 24 Oct 2026 02:41:57 +0800 Subject: [PATCH 80/95] The control's cases are tuples again, so the guard that reads them can read them Two things, both in the test layer, and the product is not changed. - the control of the escapes wrote its cases as dictionaries. The guard that holds every mutation control to the name contract reads a control's SOURCE to decide whether it judges a case by membership -- whether the case's own expected rule name is tested against the names the guard reported -- and it reads a case table as a list of TUPPLES: a case is a tuple in a table, and that is what the guard's own documentation says. A table of dictionaries is invisible to it, so the control was reported as judging by the guard's exit code alone, and the suite went red on it -- that guard and its own control failed together. The cases are tuples again, and a case that needs more than one edit carries its edits in parallel lists, because the escape the last case is about exists only in that combination. - the rule that says the guard accuses no rule it could not observe now says it per observed VALUE rather than per observation. One value that could not be read is not a reason to silence every owner that depends on the service's numbers: an owner whose own value WAS read may still accuse -- a declared interval that is a string is a broken declaration, and that is a true accusation. The three groups are the owners of the three values, and the check asks of each group only whether its own value was missing. The first one is worth stating plainly, because it is the shape of a mistake that hides: the control did decide its cases by membership, the guard could not see that it did, and the report said the opposite. A control the guard cannot read is a control the guard cannot hold to the contract, and the fix belongs in the control rather than in the guard -- the guard's rule is about how a control judges its cases, and a control written in a shape the guard cannot read has not broken that rule, it has left it unchecked. The service and the page are not changed by this commit. --- .../tests/mutate-parsed-rows-owner-escapes.py | 171 +++++++++--------- .../tests/verify-parsed-rows.mjs | 25 ++- 2 files changed, 99 insertions(+), 97 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py index a710f27e..772feb6e 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py @@ -33,6 +33,14 @@ BEHAVIOUR, not a shape. The page's script is run in a sandbox and the delay comes from the timer the page itself starts; the service's numbers come from the module's own bindings. +A CASE IS A TUPLE, AND THAT IS NOT A MATTER OF STYLE. `tests/verify-mutation-contracts.py` reads +this file's source to decide whether it judges its cases by membership -- whether the case's own +expected rule name is tested against the collection of names the guard reported -- and it reads a +case table as a LIST OF TUPLES. A case written as a dictionary is invisible to it, and a control +the guard cannot read is a control the guard cannot hold to the contract: this file was written +with dictionaries once, and the guard reported it as judging by the exit code alone. So the cases +are tuples, and a case that needs more than one edit carries its edits in parallel lists. + THE LAST TWO CASES BREAK THE GUARD ITSELF, and they are the two shapes a control of this kind has to be able to state: @@ -43,8 +51,8 @@ interval the service declares" would have accused a rule the break did not break. * a case can break TWO things at once, when the escape only exists in that combination. The mis-accusation needs the observation to have failed AND the owner to compare anyway, so the - case removes both and the guard has to name the rule that says it accuses nothing it could - not observe. + case breaks the copy and removes the gate, and the guard has to name the rule that says it + accuses nothing it could not observe. """ import json import re @@ -58,8 +66,9 @@ CLIENT = ROOT / "miniapp" / "client" / "index.html" PROBE = HERE / "verify-parsed-rows.mjs" -# Each case: what it breaks, the edits that break it, the rule that owns it, the EXACT set of -# rules that break may report, and anything the output has to say. +# Each case: what it breaks, the files it breaks, the text before and after each of them, the +# rule that owns it, the EXACT set of rules that break may report, and anything the output has to +# say. # # The exact set is the isolation, and it is what the frozen control's table does for the other # forty: a guard that also reported a rule this break did not break has accused another owner, @@ -69,94 +78,79 @@ # the first occurrence of a string that occurs twice edits a place it did not mean to edit, and # the case then passes or fails for a reason that has nothing to do with the rule it is about. MUTATIONS = [ - { - "label": "the promise sentence names minutes in words the parser did not know", - "edits": [ - ( - SERVER, - "one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped", - "one sweep interval after the limit — thirteen minutes (750000 ms) with the shipped", - ), - ], - "expect": "the promise to a reader is the same promise in both of the forms it is written in", - "exact": ("the promise to a reader is the same promise in both of the forms it is written in",), - }, - { - "label": "the page hands its timer a comma expression", - "edits": [ - ( - CLIENT, - " }, TRAJECTORY_POLL_INTERVAL);", - " }, (TRAJECTORY_POLL_INTERVAL, 8000000));", - ), - ], - "expect": "the poll waits as long as the client declares", - "exact": ( + ( + "the promise sentence names minutes in words the parser did not know", + [SERVER], + ["one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped"], + ["one sweep interval after the limit — thirteen minutes (750000 ms) with the shipped"], + "the promise to a reader is the same promise in both of the forms it is written in", + ("the promise to a reader is the same promise in both of the forms it is written in",), + (), + ), + ( + "the page hands its timer a comma expression", + [CLIENT], + [" }, TRAJECTORY_POLL_INTERVAL);"], + [" }, (TRAJECTORY_POLL_INTERVAL, 8000000));"], + "the poll waits as long as the client declares", + ( "the poll waits as long as the client declares", "the time limit is longer than one poll, so an open tab never loses its rows", ), - }, - { - "label": "a template literal impersonates the proxy's declaration while the binding is a string", - "edits": [ - ( - SERVER, - "\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n", - "\nconst annotation = `\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n`;\n" - "const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = '2';\n", - ), - ], - "expect": "the heap proxy is a number the service declares", - "exact": ("the heap proxy is a number the service declares",), - }, - { - "label": "the minutes are written as a compound the parser reads as a smaller number", - "edits": [ - ( - SERVER, - "after the limit — twelve and a half minutes (750000 ms)", - "after the limit — one hundred twelve and a half minutes (750000 ms)", - ), - ], - "expect": "the promise to a reader is the same promise in both of the forms it is written in", - "exact": ("the promise to a reader is the same promise in both of the forms it is written in",), - }, - { - "label": "the copy the guard reads the service's numbers from will not import", - "edits": [ - ( - PROBE, - "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { ${names.join(', ')} };\\n`);", - "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { nonexistentBinding };\\n`);", - ), - ], - "expect": "the guard could not observe the numbers the service declares", - "exact": ("the guard could not observe the numbers the service declares",), + (), + ), + ( + "a template literal impersonates the proxy's declaration while the binding is a string", + [SERVER], + ["\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n"], + ["\nconst annotation = `\nconst PARSED_ROWS_HEAP_PER_SOURCE_BYTE = 2;\n`;\n" + "const PARSED_ROWS_HEAP_PER_SOURCE_BYTE = '2';\n"], + "the heap proxy is a number the service declares", + ("the heap proxy is a number the service declares",), + (), + ), + ( + "the minutes are written as a compound the parser reads as a smaller number", + [SERVER], + ["after the limit — twelve and a half minutes (750000 ms)"], + ["after the limit — one hundred twelve and a half minutes (750000 ms)"], + "the promise to a reader is the same promise in both of the forms it is written in", + ("the promise to a reader is the same promise in both of the forms it is written in",), + (), + ), + ( + "the copy the guard reads the service's numbers from will not import", + [PROBE], + ["await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { ${names.join(', ')} };\\n`);"], + ["await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { nonexistentBinding };\\n`);"], + "the guard could not observe the numbers the service declares", + ("the guard could not observe the numbers the service declares",), # The owners that could not be compared have to say so: an owner that held quietly is an # owner nobody can tell apart from one that was checked and matched. - "must_contain": ("held vacuously",), - }, - { - "label": "the copy will not import and the owner compares the interval anyway", - "edits": [ - ( - PROBE, - "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { ${names.join(', ')} };\\n`);", - "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { nonexistentBinding };\\n`);", - ), - ( - PROBE, - "holds(createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP, true, BINDINGS !== null)", - "holds(createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP, true, true)", - ), + ("held vacuously",), + ), + ( + "the copy will not import and the owner compares the interval anyway", + [ + PROBE, + PROBE, + ], + [ + "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { ${names.join(', ')} };\\n`);", + "holds(createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP, true, BINDINGS !== null)", + ], + [ + "await writeFile(join(dir, 'server.mjs'), `${shipped}\\nexport { nonexistentBinding };\\n`);", + "holds(createParsedRowsCache(1000).sweepIntervalMs === SHIPPED_SWEEP, true, true)", ], - "expect": "the guard accuses no rule it could not observe", - "exact": ( + "the guard accuses no rule it could not observe", + ( "the guard accuses no rule it could not observe", "the guard could not observe the numbers the service declares", "the shipped default is the interval the service declares", ), - }, + (), + ), ] @@ -241,16 +235,13 @@ def main(): originals = {} missed = [] unnamed = [] - for case in MUTATIONS: - label = case["label"] - expect = case["expect"] - exact = case["exact"] - must_contain = case.get("must_contain", ()) + for label, targets, olds, news, expect, exact, must_contain in MUTATIONS: + edits = list(zip(targets, olds, news)) # Every edit has to land, or the case does not run at all: a case that did not happen # proves nothing, and reporting it as "got through" would blame the guard for an anchor # that has gone stale. stale = None - for target, old, new in case["edits"]: + for target, old, new in edits: if target not in originals: originals[target] = target.read_bytes() text, newline = read_exact(target) @@ -264,13 +255,13 @@ def main(): print(f"SKIP {label}: {stale}") missed.append(label) continue - for target, old, new in case["edits"]: + for target, old, new in edits: text, newline = read_exact(target) write_exact(target, text.replace(old, new, 1), newline) try: code, output = run_probe() finally: - for target in {target for target, _, _ in case["edits"]}: + for target in {target for target, _, _ in edits}: target.write_bytes(originals[target]) if code == 0: print(f"MISSED {label}: the probe still passed") diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index 82b404f2..5425130a 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -1141,25 +1141,36 @@ assert('a time limit of zero starts no sweeper, because nothing in it ever expir zeroTimerCalls.recorded.length === 0); // The owners that depend on the observation of the service's declared numbers must not accuse -// anything when that observation failed: a number the guard could not read is not a broken rule. -// This check is here, at the end, because it looks at what the other owners reported. +// anything when the value they depend on could not be read: a number the guard could not read is +// not a broken rule. This check is here, at the end, because it looks at what the other owners +// reported. // -// The list is the owners that depend on the observation. Adding an owner that depends on it is -// adding a line here -- a list that is not kept in step is a list that quietly stops covering. -const OBSERVED_OWNERS = [ +// The three groups are the owners of each observed value. Adding an owner that depends on one of +// them is adding a line to that group -- a list that is not kept in step is a list that quietly +// stops covering. +const PROXY_OWNERS = [ 'the heap proxy is a number the service declares', 'the cache is charged the source bytes times the heap proxy', 'so the source bytes it holds are computable from its own accounting', 'and the charge the route makes is that one', +]; +const TTL_OWNERS = [ 'the rows cache has a time limit', 'the time limit is a number the service declares', 'the limit the service declares is the limit the cache gives its rows', + 'the timer that releases the rows runs on the interval the cache reports', +]; +const SWEEP_OWNERS = [ 'the sweep interval is a number the service declares', 'the shipped default is the interval the service declares', - 'the timer that releases the rows runs on the interval the cache reports', +]; +const UNOBSERVABLE = [ + ...(PROXY === null ? PROXY_OWNERS : []), + ...(TTL === null ? TTL_OWNERS : []), + ...(SHIPPED_SWEEP === null ? SWEEP_OWNERS : []), ]; assert('the guard accuses no rule it could not observe', - BINDINGS !== null || !OBSERVED_OWNERS.some((name) => failures.includes(name))); + !UNOBSERVABLE.some((name) => failures.includes(name))); await rm(root, { recursive: true, force: true }); From dfc0db86b4c63532a70980e1bf8ff9e25084fb1d Mon Sep 17 00:00:00 2001 From: avatasia Date: Sat, 24 Oct 2026 03:53:34 +0800 Subject: [PATCH 81/95] Read the whole phrase, locate the promise itself, and accuse nothing unobserved Three owners of the parsed-rows rules decided the truth of a rule by reading something other than the thing they were about. The minutes of the promise to a reader were read out of the TAIL of the phrase. The walk back from the millisecond form stopped at a thousands separator and at an ampersand, and both of those can appear inside a number, so "one thousand, twelve and a half minutes (750000 ms)" was read as twelve and a half -- the number the promise states in milliseconds -- and the two forms were reported as the same promise. The walk now covers the characters a number can be written with, and the parser understands them: the separator is dropped as a separator, the ampersand reads as "and", and the scales are stated, so "one thousand, twelve and a half" is 1012.5 and "1,012.5" is 1012.5. A phrase that was cut inside a number is reported as unreadable rather than answered with its tail. The promise was taken from the first sentence of the doc block that contained some words, so a sentence written before it could own the promise while the sentence a reader reads said something else. It is now found by the promise's own form -- the sentence that carries the minutes and the number in brackets -- and a block that states that form zero times or more than once cannot say which sentence was meant, so the promise is then unreadable, which a rule of its own now says by name. The owners that depend on it hold vacuously instead of accusing a rule that was not broken. When the page's inline script did not run, three owners compared a delay they had never seen, so the premise failed loudly and three rules were accused anyway. The failure now has a rule of its own that is named, the three owners hold vacuously and count themselves when the page could not be observed, and the list of the owners that depend on an observation covers the page's half as well as the service's. The product is not changed by this commit: the service and the page are byte-for-byte the same. The guard has three new assertions, and the two that say that a phrase or a promise can be read are held by the assertion names themselves. --- .../tests/verify-parsed-rows.mjs | 171 +++++++++++++++--- 1 file changed, 142 insertions(+), 29 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs index 5425130a..bdba565f 100644 --- a/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs +++ b/plugins/avatasia/mmc-trajectory/tests/verify-parsed-rows.mjs @@ -589,9 +589,13 @@ const CLIENT_SCOPE = (() => { return { ok: true, context, timers }; })(); if (!CLIENT_SCOPE.ok) { - failures.push('the page\'s inline script did not run, so the owner cannot read the delay the ' - + `page's poll waits: ${CLIENT_SCOPE.error && CLIENT_SCOPE.error.message}`); + // The reason, as a note and not as a failure of its own: the rule that could not be observed is + // named by the assertion below, and a second entry in the failure list would be the same fact + // told twice -- which is how a reader stops knowing which entry is the rule. + console.error(' note: the page\'s inline script did not run, so the owners below cannot read the ' + + `delay the page's poll waits: ${CLIENT_SCOPE.error && CLIENT_SCOPE.error.message}`); } +assert('the guard could not observe the page\'s polling interval', CLIENT_SCOPE.ok); // A value read out of the page's own scope. const readInPage = (expression) => { @@ -687,15 +691,28 @@ const declarationIndexOf = (source, name, mask) => { // The mask of the shipped service: which of its characters are code. const SERVER_MASK = codeMask(SERVER); -// The sentence of a doc block that makes a promise. A block can hold several sentences, and only -// one of them is the promise to a reader -- so the number has to be read out of THAT sentence and -// not out of the block. A number that sat in some other sentence of the same block would answer -// the pattern while the promise itself said something else. -const promiseSentenceOf = (block) => { - if (block === null) return null; +// The sentence of a doc block that makes a promise, and HOW MANY of them there are. +// +// A block can hold several sentences, and only one of them is the promise to a reader -- so the +// number has to be read out of THAT sentence and not out of the block. A number that sat in some +// other sentence of the same block would answer the pattern while the promise itself said something +// else. +// +// WHICH sentence is the promise is not something a first match can answer: an annotation left +// beside the promise says the same words, and the first sentence that happens to contain them wins +// -- which is not the promise but whatever was written before it. So the sentence is found by the +// promise's OWN thing: the two forms of the same promise are written together, and the sentence +// that carries the number in brackets next to the minutes is the sentence that states it. +// +// And a block that states the promise TWICE leaves a reader with two sentences that both claim to +// be it. Which one the service meant is not something the block can say, so the promise is then +// UNREADABLE -- reported by name, and not resolved by picking one of the candidates. +const PROMISE_FORM = /\bminutes?\s*\(\s*[\d,]*\s*ms\s*\)/i; +const promiseSentencesOf = (block) => { + if (block === null) return []; // The sentences of the block, keeping their terminators so a sentence can be recognised. const sentences = block.split(/(?<=\.)\s+/); - return sentences.find((sentence) => sentence.includes('no later than')) ?? null; + return sentences.filter((sentence) => PROMISE_FORM.test(sentence)); }; // The number words a promise may be written in. @@ -708,10 +725,20 @@ const TENS = { twenty: 20, thirty: 30, forty: 40, fifty: 50, sixty: 60, seventy: 70, eighty: 80, ninety: 90, }; +// The scales a number may be written in. "One hundred" is a hundred; "one thousand, twelve and a +// half" is one thousand AND twelve and a half -- the words after the scale count on from it, and a +// reader that drops the scale reads a tail of the number instead of the number. +const SCALES = { + hundred: 100, + thousand: 1000, + million: 1000000, + billion: 1000000000, +}; + // The words that can be part of a number. A word in here that is not understood below is NOT // dropped: the phrase is then a number this parser cannot state, and it says so. -const NUMBER_PARTS = new Set([...Object.keys(UNITS), ...Object.keys(TENS), - 'hundred', 'thousand', 'million', 'billion', 'dozen', 'score', 'and']); +const NUMBER_PARTS = new Set([...Object.keys(UNITS), ...Object.keys(TENS), ...Object.keys(SCALES), + 'dozen', 'score', 'and']); const DIGIT_PART = /^\d+(?:\.\d+)?$/; // The number a run of number words states, or null when the run does not state one. @@ -735,17 +762,29 @@ const parseNumberWords = (words) => { } if (word in UNITS) { current += UNITS[word]; unitBefore = true; sawWords = true; continue; } if (word in TENS) { current += TENS[word]; unitBefore = false; sawWords = true; continue; } - if (word === 'hundred') { - // "hundred" multiplies the one to nine before it: "one hundred" is 100 and "one hundred - // twelve" is 112. Anything else is a compound this parser does not state. - if (!unitBefore || current < 1 || current > 9) return null; - total += current * 100; + if (word in SCALES) { + if (word === 'hundred') { + // "hundred" multiplies the one to nine before it: "one hundred" is 100 and "one hundred + // twelve" is 112. Anything else is a compound this parser does not state. + if (!unitBefore || current < 1 || current > 9) return null; + total += current * 100; + current = 0; + unitBefore = false; + sawWords = true; + continue; + } + // A bigger scale multiplies what came before it, and the rest of the phrase counts on FROM + // it: "one thousand, twelve and a half" is one thousand and twelve and a half. A scale with + // nothing in front of it states no number this parser knows. + if (current < 1) return null; + total += current * SCALES[word]; current = 0; unitBefore = false; sawWords = true; continue; } - // A scale this parser does not state -- "thousand" -- or a word that is not part of a number. + // A word that is not part of a number at all -- "dozen", "score" -- is not dropped either: the + // phrase is then a number this parser cannot state, and it says so. return null; } if (sawWords && digits !== null) return null; // a phrase that states two numbers @@ -762,7 +801,15 @@ const parseNumberWords = (words) => { // phrase that holds a compound the parser does not state is reported as unreadable instead of // being answered with the part that happened to be understood. const englishNumber = (text) => { - const words = text.toLowerCase().replace(/-/g, ' ').split(/\s+/).filter(Boolean); + // A thousands separator and an ampersand are INSIDE a number phrase, not around it: "one + // thousand, twelve and a half" states ONE number, and "one hundred & twelve and a half" states + // another. Dropping them leaves the tail of the phrase -- and a tail that happens to be a number + // of its own is worse than no number at all, because it agrees with something. + const words = text.toLowerCase() + .replace(/,/g, '') + .replace(/&/g, ' and ') + .replace(/-/g, ' ') + .split(/\s+/).filter(Boolean); let half = false; if (words.length >= 3 && words[words.length - 3] === 'and' @@ -780,6 +827,18 @@ const englishNumber = (text) => { return half ? value + 0.5 : value; }; +// The characters a number phrase is written with, and the characters that can appear INSIDE one. +// +// A number can be written with a thousands separator and with an ampersand: "one thousand, twelve +// and a half" and "one hundred & twelve and a half" are one number each. So a scan that stops at +// those leaves the TAIL of the phrase -- and the tail of a number is a number of its own, which is +// worse than reading no number at all: the tail can agree with the promise while the phrase does +// not. +const PHRASE_CLASS = /[A-Za-z0-9\s.,&'-]/; +// A character that can continue a number and that neither the scan nor the parser reads: the +// phrase was cut INSIDE a number, and a cut phrase is not a number -- it is the tail of one. +const NUMBER_CONTINUATION = /[\p{Nd}‘’½¼¾⅓⅔]/u; + // The minutes a sentence promises, and whether it states them in the form the promise is written // in. // @@ -806,12 +865,35 @@ const minutesForm = (sentence) => { if (anchor === null) return { written: false, minutes: null }; const at = anchor.index; let start = at; - while (start > 0 && /[A-Za-z0-9\s.-]/.test(sentence[start - 1])) start -= 1; + while (start > 0 && PHRASE_CLASS.test(sentence[start - 1])) start -= 1; + // The scan has to stop where the PHRASE ends, not inside the number. A character that can carry a + // number and that this scan cannot read is a sign that the phrase was cut in the middle of one, + // and a phrase cut in the middle of a number is not a number: it is the tail of one. The minutes + // are then unreadable, which is a different fact from "this promise states no minutes" -- and the + // owner of that difference says so. + if (start > 0 && NUMBER_CONTINUATION.test(sentence[start - 1])) { + return { written: true, minutes: null }; + } const phrase = sentence.slice(start, at).trim(); const digits = /^\s*(\d+(?:\.\d+)?)\s*$/.exec(phrase); return { written: true, minutes: digits !== null ? Number(digits[1]) : englishNumber(phrase) }; }; +// The reader itself is held to reading the WHOLE phrase, and not a tail of it. A tail of a number +// is a number of its own, and a tail that happens to agree with the promise is the worst case there +// is: the promise states one number in words and another in milliseconds, and the reader agrees +// with the tail. +// +// The phrases here are the ones a promise can be written in, and the numbers are the ones they +// state: "one thousand, twelve and a half" is one thousand AND twelve and a half, because a +// thousands separator is not a boundary, and neither is an ampersand. A phrase that was cut inside +// a number is not a number at all -- and it is not a tail of one either. +assert('the promised minutes are read from the whole phrase, not from a tail of it', + minutesForm('one thousand, twelve and a half minutes (750000 ms)').minutes === 1012.5 + && minutesForm('one hundred & twelve and a half minutes (750000 ms)').minutes === 112.5 + && minutesForm('1,012.5 minutes (750000 ms)').minutes === 1012.5 + && minutesForm('one hundred and twenty ½ minutes (750000 ms)').minutes === null); + // The lifetime the cache gives its rows when nobody tells it one, observed rather than read. // // The declaration of the limit says what the service INTENDS to give its rows; this says what it @@ -936,10 +1018,15 @@ check('and the cache is empty', liveCache.size, 0); // string that looks like one would be handed over as a string. const DECLARED = readInPage('TRAJECTORY_POLL_INTERVAL'); const DECLARED_INTERVAL = DECLARED.ok ? DECLARED.value : null; +// The owners that depend on the PAGE hold vacuously when the page could not be observed, and they +// count themselves when they do: a delay nobody could read does not contradict what the client +// declares, and an owner that accused it anyway would be accusing a rule it never looked at. The +// failure to observe has its own rule, above. assert('the client declares the interval it polls the trajectory at', - DECLARED_INTERVAL !== null - && typeof DECLARED_INTERVAL === 'number' - && Number.isFinite(DECLARED_INTERVAL)); + holds(DECLARED_INTERVAL !== null + && typeof DECLARED_INTERVAL === 'number' + && Number.isFinite(DECLARED_INTERVAL), + true, CLIENT_SCOPE.ok)); // The timer the page polls the trajectory with, and the delay it was really given. // @@ -958,7 +1045,7 @@ const TRAJECTORY_TIMER = CLIENT_SCOPE.ok : undefined; const POLL_DELAY = TRAJECTORY_TIMER === undefined ? null : TRAJECTORY_TIMER.delay; assert('the poll waits as long as the client declares', - POLL_DELAY !== null && POLL_DELAY === DECLARED_INTERVAL); + holds(POLL_DELAY !== null && POLL_DELAY === DECLARED_INTERVAL, true, CLIENT_SCOPE.ok)); assert('the rows cache has a time limit', holds(TTL !== null, true, BINDINGS !== null)); assert('the time limit is a number the service declares', @@ -969,7 +1056,7 @@ const DEFAULT_LIMIT = defaultLimitOf(1024 * 1024); assert('the limit the service declares is the limit the cache gives its rows', holds(DEFAULT_LIMIT, TTL, BINDINGS !== null)); assert('the time limit is longer than one poll, so an open tab never loses its rows', - POLL_DELAY !== null && DEFAULT_LIMIT > POLL_DELAY); + holds(POLL_DELAY !== null && DEFAULT_LIMIT > POLL_DELAY, true, CLIENT_SCOPE.ok)); // The page stops polling while its tab is hidden, so a reader who switches away for a while is // the ordinary case. A limit shorter than that would make every return to the tab cost a // re-parse — which is the other half of what this limit buys, and it has to be paid for. @@ -1033,16 +1120,27 @@ assert('the shipped default is the interval the service declares', // are read as well, and the two are compared with each other and with the limit the cache really // gives its rows. const ttlDoc = docBlockOf(SERVER, 'PARSED_ROWS_CACHE_TTL_MS', SERVER_MASK); -const promiseSentence = promiseSentenceOf(ttlDoc); +const promiseSentences = promiseSentencesOf(ttlDoc); +// The promise has to be READABLE, and it is readable only when the block states it once. A block +// that states it twice is not a block with two promises to choose from: it is a block where the +// promise cannot be told from the annotation beside it, and an owner that picks one of them accuses +// the service of a number it never promised. +assert('the promise to a reader is stated exactly once, in the sentence that carries the number in brackets', + promiseSentences.length === 1); +const promiseSentence = promiseSentences.length === 1 ? promiseSentences[0] : null; const promisedNumbers = promiseSentence === null ? [] - : [...promiseSentence.matchAll(/\((\d+) ms\)/g)].map((found) => Number(found[1])); + : [...promiseSentence.matchAll(/\((\d[\d,]*) ms\)/g)].map((found) => Number(found[1].replace(/,/g, ''))); const MINUTES_FORM = minutesForm(promiseSentence); const promisedMinutes = MINUTES_FORM.minutes; const DEFAULT_INTERVAL = createParsedRowsCache(1000).sweepIntervalMs; +// The owner of the number holds vacuously while the promise cannot be read: a number nobody could +// locate is not a number the service stated wrongly. The reason it could not be located has its own +// rule, above. assert('the number promised to a reader is the limit plus one sweep interval', - promisedNumbers.length === 1 - && promisedNumbers[0] === DEFAULT_LIMIT + DEFAULT_INTERVAL); + holds(promisedNumbers.length === 1 + && promisedNumbers[0] === DEFAULT_LIMIT + DEFAULT_INTERVAL, + true, promiseSentences.length === 1)); // The two forms have to be the SAME promise. A promise whose words cannot be read is not // agreement: the words are parsed completely above, so a phrase that is still not a number is a // promise nobody can check, and this owner says so rather than passing it. @@ -1145,7 +1243,7 @@ assert('a time limit of zero starts no sweeper, because nothing in it ever expir // not a broken rule. This check is here, at the end, because it looks at what the other owners // reported. // -// The three groups are the owners of each observed value. Adding an owner that depends on one of +// The groups are the owners of each observed value. Adding an owner that depends on one of // them is adding a line to that group -- a list that is not kept in step is a list that quietly // stops covering. const PROXY_OWNERS = [ @@ -1164,10 +1262,25 @@ const SWEEP_OWNERS = [ 'the sweep interval is a number the service declares', 'the shipped default is the interval the service declares', ]; +// The owners that read the PAGE. The page's script is run in a sandbox, and a script that will not +// run leaves the delay it waits unreadable -- so these owners must not accuse anything then. +const PAGE_OWNERS = [ + 'the client declares the interval it polls the trajectory at', + 'the poll waits as long as the client declares', + 'the time limit is longer than one poll, so an open tab never loses its rows', +]; +// The owners that read the PROMISE. A promise the block states more than once cannot be located, +// and a number nobody could locate is not a number the service stated wrongly. +const PROMISE_OWNERS = [ + 'the number promised to a reader is the limit plus one sweep interval', + 'the promise to a reader is the same promise in both of the forms it is written in', +]; const UNOBSERVABLE = [ ...(PROXY === null ? PROXY_OWNERS : []), ...(TTL === null ? TTL_OWNERS : []), ...(SHIPPED_SWEEP === null ? SWEEP_OWNERS : []), + ...(CLIENT_SCOPE.ok ? [] : PAGE_OWNERS), + ...(promiseSentences.length === 1 ? [] : PROMISE_OWNERS), ]; assert('the guard accuses no rule it could not observe', !UNOBSERVABLE.some((name) => failures.includes(name))); From 3f9d45abb86fcc4021a05bf0f74ad2f9fe6041e2 Mon Sep 17 00:00:00 2001 From: avatasia Date: Sat, 24 Oct 2026 03:53:34 +0800 Subject: [PATCH 82/95] Hold the three new owner escapes by name, and sync the one frozen entry The control of the owner escapes holds a case per escape, each caught BY NAME and by the only rules that break really breaks. It now holds eleven cases: the three the sixteenth run found, the three the seventeenth run found, and the five the eighteenth run found -- a compound number written with a thousands separator in the words, with an ampersand and in the digits, a sentence written before the promise that owns it, and a page whose inline script does not run. The last of those five is the shape of a premise: the page's script is run in a sandbox, and the failure to run has to be loud, named, and followed by silence from the owners that depend on what the page says. The case holds the guard to exactly that -- it must report the premise and NOTHING else, and the count of vacuous holdings has to say so. One entry of the frozen table changed, and it is the entry that says which rule owns the break "the number promised to a reader is moved into an obsolete annotation". That rule is now the one that says the promise can be read: the promise is located by its own form, so a block that no longer states that form states no promise, and the owner that used to compare a number the guard could not locate would have accused a rule that was not broken -- the block still promises 750000, in the annotation. The other thirty-nine cases and the rest of the table are unchanged. --- .../tests/mutate-parsed-rows-owner-escapes.py | 69 +++++++++++++++++++ .../tests/mutate-parsed-rows.py | 18 ++++- 2 files changed, 84 insertions(+), 3 deletions(-) diff --git a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py index 772feb6e..ee79886c 100644 --- a/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py +++ b/plugins/avatasia/mmc-trajectory/tests/mutate-parsed-rows-owner-escapes.py @@ -18,6 +18,18 @@ the interval the service declares" compared the interval the cache really uses with the number it could not read -- so a broken harness accused a rule that was not broken. + Round eighteen: + R29-F36 the PHRASE was cut, not the words: the scan that walks back from the millisecond form + stopped at a thousands separator and at an ampersand -- both of which can appear INSIDE a + number -- so "one thousand, twelve and a half minutes (750000 ms)" was read as twelve and + a half, and the tail agreed with the promise while the phrase did not. + R29-F37 a sentence written BEFORE the promise said the same words, and the owner took the first + sentence that contained them: the promise to a reader was read out of an annotation left + beside it, and the annotation agreed with the cache. + R29-F38 the page's inline script did not run, and the three owners that read the page's delay + compared a value they never saw: the premise failed loudly, and three rules were + accused anyway. + Each case breaks one file and one rule, and a case counts as caught only when the guard reported THAT rule -- by name -- and nothing else. The guard has to go red by NAMING a rule: a guard that threw, or that went red without naming anything, has not shown that it owns this break. @@ -151,6 +163,63 @@ ), (), ), + ( + "the promise states a compound number whose thousands separator hides the head of it", + [SERVER], + ["after the limit — twelve and a half minutes (750000 ms)"], + ["after the limit — one thousand, twelve and a half minutes (750000 ms)"], + "the promise to a reader is the same promise in both of the forms it is written in", + ("the promise to a reader is the same promise in both of the forms it is written in",), + (), + ), + ( + "the promise states a compound number with an ampersand in it", + [SERVER], + ["after the limit — twelve and a half minutes (750000 ms)"], + ["after the limit — one hundred & twelve and a half minutes (750000 ms)"], + "the promise to a reader is the same promise in both of the forms it is written in", + ("the promise to a reader is the same promise in both of the forms it is written in",), + (), + ), + ( + "the promise states the same number with a thousands separator in it", + [SERVER], + ["after the limit — twelve and a half minutes (750000 ms)"], + ["after the limit — 1,012.5 minutes (750000 ms)"], + "the promise to a reader is the same promise in both of the forms it is written in", + ("the promise to a reader is the same promise in both of the forms it is written in",), + (), + ), + ( + "a sentence written before the promise says the same words, and the promise says another number", + [ + SERVER, + SERVER, + ], + [ + " * **WHEN THEY COME BACK IS NOT THE SAME MOMENT AS WHEN THEY EXPIRE.**", + "one sweep interval after the limit — twelve and a half minutes (750000 ms) with the shipped", + ], + [ + " * Historical annotation, no later than twelve and a half minutes (750000 ms).\n" + " * **WHEN THEY COME BACK IS NOT THE SAME MOMENT AS WHEN THEY EXPIRE.**", + "one sweep interval after the limit — fourteen minutes (840000 ms) with the shipped", + ], + "the promise to a reader is stated exactly once, in the sentence that carries the number in brackets", + ("the promise to a reader is stated exactly once, in the sentence that carries the number in brackets",), + (), + ), + ( + "the page's inline script does not run, so the owners below cannot read what they are about", + [CLIENT], + ["