diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index c7dae42e..ae0cd4e1 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -36,6 +36,7 @@ - [ ] If dependencies or a lockfile changed: `npm run check:lock` passes, the root `package-lock.json` is synchronized, and `npm audit --omit=dev --audit-level=high` reports no vulnerabilities - [ ] If a subpackage changed (such as `dsh/dsh-nmg`): `npm run verify:packages` passes (frozen-lockfile install + build), and `pnpm-lock.yaml`/the lockfile matches `package.json` - [ ] For documentation changes, `npm run docs:check` was run; decision/design changes follow the doc-maintenance rules +- [ ] **Every doc owner this change touches is named here**, with the file that was added or edited: decision record (`docs/decisions/`), design (`docs/design/`), experiment (`docs/experiments/`), postmortem (`docs/postmortem/`), guide (`docs/guides/`) — or `none`, with the reason. A behaviour change with no owner named is a defect, not an omission: the next reader has nowhere to look. "Updated docs" without a path does not satisfy this item. - [ ] `## Not verified` is filled in (write `None.` if nothing remains); non-trivial changes include a decision record - [ ] New code methods stay within the cyclomatic-complexity threshold (CodeFactor / `npm run complexity:gate`) - [ ] No reproducible build artifacts are committed: do not check in `dist/`, `dsh/dsh-nmg/lib/`, `src/prompts/nmg-prompts.generated.ts`, or `.nmg-search-scope` (see `docs/decisions/rejected/2026-09-02-track-build-artifacts-in-git.md`) diff --git a/docs/decisions/implemented/2026-10-29-accept-without-cas.md b/docs/decisions/implemented/2026-10-29-accept-without-cas.md new file mode 100644 index 00000000..74b5bb27 --- /dev/null +++ b/docs/decisions/implemented/2026-10-29-accept-without-cas.md @@ -0,0 +1,94 @@ +# An accept takes no compare-and-swap + +[简体中文](2026-10-29-accept-without-cas.zh-CN.md) · [Terms](2026-10-29-accept-without-cas.terms.yaml) + +**Status:** implemented +**Approved:** explicit + +## Problem + +Measured from a clean network against the deployed build, one user's write path answers in 2.6 s at +concurrency 1 and in **51-106 s** at concurrency 4, where 7 of 24 requests were killed at the +120 s invocation ceiling; at concurrency 16, 5 of 32 connections never completed. A killed request +does not cost a score, it voids the attempt: the harness probes `/health` first and stops. + +The cause was structural, not a slow dependency. Every accept for one user read the manifest, +computed a new one and wrote it back under an ETag compare-and-swap; a loser re-read and recomputed, +up to twelve times, and each attempt re-read and re-exported a partition of up to 8 MiB. The only +thing the accept wrote there was the pending count. + +## Decision + +**The write path appends; it does not contend.** + +1. **An accept writes one thing**: its log entry, under a key derived from its own request hash, + which no rival can collide with. It reads the manifest once, for the two facts the log cannot + answer - the generation, and the receipt of a request that was already folded. +2. **The pending count is derived, not written.** Its only consumer is an operator reading + `/capacity`, which now lists a bounded prefix and reports `pending_capped` when the bound was + reached. Nothing on the read path trusts a zero any more: two gates (`pendingForSearch` and + `fusePendingRecords`) returned early on `pendingCount === 0`, and both are gone. +3. **The fold is scheduled, not counted.** Roughly one accept in `FOLD_TRIGGER` (overridable with + `NMG_FOLD_EVERY`) attempts a fold; the fold lists the log first and does nothing when fewer than + a batch is waiting, so a wasted attempt costs one listing while a missed backlog would hide data. +4. **An accept embeds its own messages, and the fold reuses those vectors.** The document text for + this path is `[day ]statement` - the service never supplies `previous_turn` - so it is known + before any partition is opened. Accept embeds under `NMG_ACCEPT_EMBED_MS` (8 s) and stores the + vectors in the entry with the index identity that produced them; the fold seeds its client from + them and applies storage only. Losing the budget is not an error: the entry is written without + vectors and the fold embeds as before. +5. **Every long operation is bounded.** The fold has `NMG_FOLD_BUDGET_MS` (20 s) and is skipped, not + failed, past it; the search loop stops at `NMG_SEARCH_DEADLINE_MS` (60 s) and answers with + `partial: true`; the fold's embedding backfill takes one batch of 32 (`maxBatches: 1`), matching + the search path, where it used to embed a whole partition's backlog in a single request. + +## Evidence + +| measurement | value | +| ----------------------------------------- | ---------------------------------------------------------------------------------------------- | +| Add p50, concurrency 1 | 2.6 s (max 21.9 s: one fold) | +| Add p50, concurrency 2 | 3.6 s (max 41.2 s: one fold) | +| Add p50, concurrency 4 | 51.6-106 s, 7 of 24 killed at 120 s | +| connection failures at 16 | 5 of 32 | +| provider calls per Add, after this change | at most **1**, even when the Add triggers a fold (24 sequential Adds, fold proven to have run) | +| provider retry budget | 3 attempts, only for 429/5xx; a timeout is not retried | + +## What this does and does not establish + +- Not verified against a deployed build. Every number above is the before-state; the after-state + needs one real run. +- It does not remove contention from the fold itself: two concurrent folds on one user still meet at + the manifest CAS and are retried. It removes it from the path that runs on **every** accept, which + is where the amplification was. +- A count that lags can only cause an extra fold attempt, never a missed one: a stale-high count + costs a listing, and the read path no longer trusts a zero. + +## Alternatives considered + +- **Keep the count and retry harder.** That is the measured failure: twelve attempts each redoing + megabyte-scale work. +- **Shard the counter (AWS's "write sharding")**, or a **G-Counter / PN-Counter**. Sharding divides + contention by the shard count and keeps it; a grow-only counter cannot express a backlog that + shrinks, and a PN-Counter's decrement cannot be reconciled with a log without ceasing to be + mergeable. Both were assessed as strictly worse than deriving the number. +- **Put the counter in KV** (1 GB, unused on this platform) - the industry-shaped answer, since + object storage is not a queue. Not taken because the only consumer is one operator reading + `/capacity`, and a listing answers it on demand. +- **Alibaba Batch API for the embedding.** Supported by `text-embedding-v4` at half price, but it is + asynchronous with a poll to collect results and this runtime has no timer; the poll would have to + be carried by later requests. Rejected for the serving path, retained as the right tool for an + offline re-embed. +- **Fire the embedding after the response (waitUntil).** Our entry never receives an execution + context - `edgeone-entry.mjs` documents that deliberately - so there is no continuation to + register. + +## Consequences + +- An accept is one PUT of a few KB. A burst no longer amplifies into retries. +- `/capacity` costs a listing, which is why the operator path pays it and the request path does not. +- **Rolling the code back now requires draining the log first.** An older deployment short-circuits + on `pendingCount === 0`, and this deployment no longer maintains that count, so it could read a + log with entries as empty. `/admin/fold` answers 409 until it is empty, which is the guard. +- A record whose inline embedding exceeded its budget is embedded twice (once abandoned, once in the + fold). That is the only duplicate spend this introduces, and it happens only when the provider is + slower than 8 s. diff --git a/docs/decisions/implemented/2026-10-29-accept-without-cas.terms.yaml b/docs/decisions/implemented/2026-10-29-accept-without-cas.terms.yaml new file mode 100644 index 00000000..c9c1a952 --- /dev/null +++ b/docs/decisions/implemented/2026-10-29-accept-without-cas.terms.yaml @@ -0,0 +1,3 @@ +version: 2 +terms: [] +references: [] diff --git a/docs/decisions/implemented/2026-10-29-accept-without-cas.zh-CN.md b/docs/decisions/implemented/2026-10-29-accept-without-cas.zh-CN.md new file mode 100644 index 00000000..8b4ab28b --- /dev/null +++ b/docs/decisions/implemented/2026-10-29-accept-without-cas.zh-CN.md @@ -0,0 +1,54 @@ +# accept 不参与比较交换 + +[English](2026-10-29-accept-without-cas.md) · [Terms](2026-10-29-accept-without-cas.terms.yaml) + +**Status:** implemented +**Approved:** explicit + +## 问题 + +从干净网络对已部署构建实测:单用户在并发 1 时写入 2.6 秒,并发 4 时 **51-106 秒**,24 次请求里 7 次被 120 秒的调用上限杀掉;并发 16 时 32 次连接里 5 次根本没建立。被杀不是丢一分,而是整次评测作废——比赛方先探 `/health`,失败就停。 + +成因是结构性的,不是某个依赖慢。同一个用户的每次 accept 都要读 manifest、算新版本、带 ETag 写回;输家重读重算,最多十二次,而每次都要重新读写一个最多 8 MiB 的分区。而 accept 在 manifest 上写的**只有那个计数**。 + +## 决策 + +**写路径只追加,不争用。** + +1. **accept 只写一样东西**:它自己的日志条目,键由请求哈希派生,没有对手能撞上。manifest 只读一次,为了拿日志答不了的两件事——代际,以及已折叠请求的回执。 +2. **计数是派生的,不是写出来的。** 它唯一的消费者是看 `/capacity` 的运维,现在按有界前缀列举,并在触到上限时标 `pending_capped`。读路径不再信任零:`pendingForSearch` 与 `fusePendingRecords` 两处"计数为 0 就返回"的短路都已删除。 +3. **折叠按排程触发,不再依赖计数。** 约每 `FOLD_TRIGGER` 次 accept 试一次(`NMG_FOLD_EVERY` 可覆盖);折叠先列举,不足一批就什么都不做——浪费一次尝试只花一次列举,漏掉积压才会藏数据。 +4. **accept 自己嵌自己这批消息,折叠复用这些向量。** 这条路径的文档文本是 `[日期 ]statement`(服务从不提供 `previous_turn`),所以在打开任何分区之前就已确定。accept 在 `NMG_ACCEPT_EMBED_MS`(8 秒)内嵌入并把向量与产生它的索引身份一起写进条目;折叠用它们播种客户端,只做存储。预算用尽不算错误:条目照写,折叠照旧嵌入。 +5. **每个长操作都有界。** 折叠有 `NMG_FOLD_BUDGET_MS`(20 秒),超了跳过而不是失败;检索循环在 `NMG_SEARCH_DEADLINE_MS`(60 秒)停止并返回 `partial: true`;折叠的嵌入回填只做一批 32 条(`maxBatches: 1`),与检索路径一致——它原本会在一次请求里嵌完整个分区的积压。 + +## 证据 + +| 测量 | 值 | +| ------------------------------- | --------------------------------------------------------------------------- | +| 并发 1 Add p50 | 2.6 s(max 21.9 s,一次折叠) | +| 并发 2 Add p50 | 3.6 s(max 41.2 s,一次折叠) | +| 并发 4 Add p50 | 51.6-106 s,24 次里 7 次被杀 | +| 并发 16 连接失败 | 32 次里 5 次 | +| 本次改动后每次 Add 的提供商调用 | **至多 1 次**,即使这次 Add 触发了折叠(24 次顺序 Add,且证明折叠确实跑过) | +| 提供商重试预算 | 3 次尝试,只对 429/5xx;超时不重试 | + +## 这确立了什么、没有确立什么 + +- **没有在已部署构建上验证。** 上面每个数字都是改动前;改动后需要一次真实运行。 +- 它没有消除**折叠自身**的争用:同一用户两个并发折叠仍会在 manifest 上相遇并被重试。它消除的是**每次 accept 都会走的那条路**上的争用,而放大正是发生在那里。 +- 计数滞后只会导致多折一次,绝不会导致漏折:偏高只花一次列举,而读路径不再信任零。 + +## 考虑过的替代方案 + +- **保留计数、加大重试**:这就是被测量的失败形态,十二次尝试每次都重做 MB 级工作。 +- **分片计数(AWS 的 write sharding)或 G-Counter / PN-Counter**:分片只是把争用除以片数,仍然存在;只增计数器表达不了会缩小的积压,而 PN 的减法无法与日志核对共存,一旦核对就不再可合并。两者都劣于"派生"。 +- **把计数放 KV**(本平台 1 GB、当前未用)——与业界同形的答案,因为对象存储本来不是队列。没有采纳的理由:唯一消费者是一个人偶尔看 `/capacity`,一次列举就能答。 +- **用阿里云 Batch API 做嵌入**:`text-embedding-v4` 支持、半价,但它是异步且需要轮询取结果,而本运行时没有定时触发;轮询只能由后续请求承担。服务路径上否决,离线全库重嵌仍然适用。 +- **响应之后再嵌(waitUntil)**:我们的入口从不接收执行上下文(`edgeone-entry.mjs` 里明确不传),因此没有可注册的续体。 + +## 后果 + +- 一次 accept 就是几 KB 的一次 PUT。突发不再放大成重试。 +- `/capacity` 要花一次列举——所以由运维路径付,请求路径不付。 +- **现在回滚代码必须先排空日志。** 旧部署用 `pendingCount === 0` 短路,而本部署不再维护该计数,于是它可能把有内容的日志读成空。守卫是 `/admin/fold`:未清空就返回 409。 +- 一次 accept 的内联嵌入若超预算,那条记录会被嵌两次(一次被放弃、一次在折叠里)。这是本次引入的唯一重复花费,且只在提供商慢于 8 秒时发生。