From 196557655c04cd79a64a3529fc0ed5521029cfde Mon Sep 17 00:00:00 2001 From: wefio <48851810+wefio@users.noreply.github.com> Date: Fri, 30 Oct 2026 00:06:29 +0800 Subject: [PATCH 1/2] docs(decision): an accept takes no compare-and-swap Records the round that removed the write path contention: no CAS on accept, the pending count derived instead of written, the fold scheduled rather than counted, the accept vectors carried into the fold, and every long operation bounded. It states the measured before-state (Add p50 2.6 s at concurrency 1, 51-106 s at 4 with 7 of 24 killed), what was rejected and why (keep-and-retry, sharded and PN counters, a KV counter, the Batch API, waitUntil), and the operational constraint this creates: rolling the code back now requires draining the log first, because an older deployment short-circuits on a count this one no longer maintains. --- .../2026-10-29-accept-without-cas.md | 94 +++++++++++++++++++ .../2026-10-29-accept-without-cas.terms.yaml | 3 + .../2026-10-29-accept-without-cas.zh-CN.md | 54 +++++++++++ 3 files changed, 151 insertions(+) create mode 100644 docs/decisions/implemented/2026-10-29-accept-without-cas.md create mode 100644 docs/decisions/implemented/2026-10-29-accept-without-cas.terms.yaml create mode 100644 docs/decisions/implemented/2026-10-29-accept-without-cas.zh-CN.md diff --git a/docs/decisions/implemented/2026-10-29-accept-without-cas.md b/docs/decisions/implemented/2026-10-29-accept-without-cas.md new file mode 100644 index 00000000..74b5bb27 --- /dev/null +++ b/docs/decisions/implemented/2026-10-29-accept-without-cas.md @@ -0,0 +1,94 @@ +# An accept takes no compare-and-swap + +[简体中文](2026-10-29-accept-without-cas.zh-CN.md) · [Terms](2026-10-29-accept-without-cas.terms.yaml) + +**Status:** implemented +**Approved:** explicit + +## Problem + +Measured from a clean network against the deployed build, one user's write path answers in 2.6 s at +concurrency 1 and in **51-106 s** at concurrency 4, where 7 of 24 requests were killed at the +120 s invocation ceiling; at concurrency 16, 5 of 32 connections never completed. A killed request +does not cost a score, it voids the attempt: the harness probes `/health` first and stops. + +The cause was structural, not a slow dependency. Every accept for one user read the manifest, +computed a new one and wrote it back under an ETag compare-and-swap; a loser re-read and recomputed, +up to twelve times, and each attempt re-read and re-exported a partition of up to 8 MiB. The only +thing the accept wrote there was the pending count. + +## Decision + +**The write path appends; it does not contend.** + +1. **An accept writes one thing**: its log entry, under a key derived from its own request hash, + which no rival can collide with. It reads the manifest once, for the two facts the log cannot + answer - the generation, and the receipt of a request that was already folded. +2. **The pending count is derived, not written.** Its only consumer is an operator reading + `/capacity`, which now lists a bounded prefix and reports `pending_capped` when the bound was + reached. Nothing on the read path trusts a zero any more: two gates (`pendingForSearch` and + `fusePendingRecords`) returned early on `pendingCount === 0`, and both are gone. +3. **The fold is scheduled, not counted.** Roughly one accept in `FOLD_TRIGGER` (overridable with + `NMG_FOLD_EVERY`) attempts a fold; the fold lists the log first and does nothing when fewer than + a batch is waiting, so a wasted attempt costs one listing while a missed backlog would hide data. +4. **An accept embeds its own messages, and the fold reuses those vectors.** The document text for + this path is `[day ]statement` - the service never supplies `previous_turn` - so it is known + before any partition is opened. Accept embeds under `NMG_ACCEPT_EMBED_MS` (8 s) and stores the + vectors in the entry with the index identity that produced them; the fold seeds its client from + them and applies storage only. Losing the budget is not an error: the entry is written without + vectors and the fold embeds as before. +5. **Every long operation is bounded.** The fold has `NMG_FOLD_BUDGET_MS` (20 s) and is skipped, not + failed, past it; the search loop stops at `NMG_SEARCH_DEADLINE_MS` (60 s) and answers with + `partial: true`; the fold's embedding backfill takes one batch of 32 (`maxBatches: 1`), matching + the search path, where it used to embed a whole partition's backlog in a single request. + +## Evidence + +| measurement | value | +| ----------------------------------------- | ---------------------------------------------------------------------------------------------- | +| Add p50, concurrency 1 | 2.6 s (max 21.9 s: one fold) | +| Add p50, concurrency 2 | 3.6 s (max 41.2 s: one fold) | +| Add p50, concurrency 4 | 51.6-106 s, 7 of 24 killed at 120 s | +| connection failures at 16 | 5 of 32 | +| provider calls per Add, after this change | at most **1**, even when the Add triggers a fold (24 sequential Adds, fold proven to have run) | +| provider retry budget | 3 attempts, only for 429/5xx; a timeout is not retried | + +## What this does and does not establish + +- Not verified against a deployed build. Every number above is the before-state; the after-state + needs one real run. +- It does not remove contention from the fold itself: two concurrent folds on one user still meet at + the manifest CAS and are retried. It removes it from the path that runs on **every** accept, which + is where the amplification was. +- A count that lags can only cause an extra fold attempt, never a missed one: a stale-high count + costs a listing, and the read path no longer trusts a zero. + +## Alternatives considered + +- **Keep the count and retry harder.** That is the measured failure: twelve attempts each redoing + megabyte-scale work. +- **Shard the counter (AWS's "write sharding")**, or a **G-Counter / PN-Counter**. Sharding divides + contention by the shard count and keeps it; a grow-only counter cannot express a backlog that + shrinks, and a PN-Counter's decrement cannot be reconciled with a log without ceasing to be + mergeable. Both were assessed as strictly worse than deriving the number. +- **Put the counter in KV** (1 GB, unused on this platform) - the industry-shaped answer, since + object storage is not a queue. Not taken because the only consumer is one operator reading + `/capacity`, and a listing answers it on demand. +- **Alibaba Batch API for the embedding.** Supported by `text-embedding-v4` at half price, but it is + asynchronous with a poll to collect results and this runtime has no timer; the poll would have to + be carried by later requests. Rejected for the serving path, retained as the right tool for an + offline re-embed. +- **Fire the embedding after the response (waitUntil).** Our entry never receives an execution + context - `edgeone-entry.mjs` documents that deliberately - so there is no continuation to + register. + +## Consequences + +- An accept is one PUT of a few KB. A burst no longer amplifies into retries. +- `/capacity` costs a listing, which is why the operator path pays it and the request path does not. +- **Rolling the code back now requires draining the log first.** An older deployment short-circuits + on `pendingCount === 0`, and this deployment no longer maintains that count, so it could read a + log with entries as empty. `/admin/fold` answers 409 until it is empty, which is the guard. +- A record whose inline embedding exceeded its budget is embedded twice (once abandoned, once in the + fold). That is the only duplicate spend this introduces, and it happens only when the provider is + slower than 8 s. diff --git a/docs/decisions/implemented/2026-10-29-accept-without-cas.terms.yaml b/docs/decisions/implemented/2026-10-29-accept-without-cas.terms.yaml new file mode 100644 index 00000000..c9c1a952 --- /dev/null +++ b/docs/decisions/implemented/2026-10-29-accept-without-cas.terms.yaml @@ -0,0 +1,3 @@ +version: 2 +terms: [] +references: [] diff --git a/docs/decisions/implemented/2026-10-29-accept-without-cas.zh-CN.md b/docs/decisions/implemented/2026-10-29-accept-without-cas.zh-CN.md new file mode 100644 index 00000000..8b4ab28b --- /dev/null +++ b/docs/decisions/implemented/2026-10-29-accept-without-cas.zh-CN.md @@ -0,0 +1,54 @@ +# accept 不参与比较交换 + +[English](2026-10-29-accept-without-cas.md) · [Terms](2026-10-29-accept-without-cas.terms.yaml) + +**Status:** implemented +**Approved:** explicit + +## 问题 + +从干净网络对已部署构建实测：单用户在并发 1 时写入 2.6 秒，并发 4 时 **51-106 秒**，24 次请求里 7 次被 120 秒的调用上限杀掉；并发 16 时 32 次连接里 5 次根本没建立。被杀不是丢一分，而是整次评测作废——比赛方先探 `/health`，失败就停。 + +成因是结构性的，不是某个依赖慢。同一个用户的每次 accept 都要读 manifest、算新版本、带 ETag 写回；输家重读重算，最多十二次，而每次都要重新读写一个最多 8 MiB 的分区。而 accept 在 manifest 上写的**只有那个计数**。 + +## 决策 + +**写路径只追加，不争用。** + +1. **accept 只写一样东西**：它自己的日志条目，键由请求哈希派生，没有对手能撞上。manifest 只读一次，为了拿日志答不了的两件事——代际，以及已折叠请求的回执。 +2. **计数是派生的，不是写出来的。** 它唯一的消费者是看 `/capacity` 的运维，现在按有界前缀列举，并在触到上限时标 `pending_capped`。读路径不再信任零：`pendingForSearch` 与 `fusePendingRecords` 两处"计数为 0 就返回"的短路都已删除。 +3. **折叠按排程触发，不再依赖计数。** 约每 `FOLD_TRIGGER` 次 accept 试一次（`NMG_FOLD_EVERY` 可覆盖）；折叠先列举，不足一批就什么都不做——浪费一次尝试只花一次列举，漏掉积压才会藏数据。 +4. **accept 自己嵌自己这批消息，折叠复用这些向量。** 这条路径的文档文本是 `[日期 ]statement`（服务从不提供 `previous_turn`），所以在打开任何分区之前就已确定。accept 在 `NMG_ACCEPT_EMBED_MS`（8 秒）内嵌入并把向量与产生它的索引身份一起写进条目；折叠用它们播种客户端，只做存储。预算用尽不算错误：条目照写，折叠照旧嵌入。 +5. **每个长操作都有界。** 折叠有 `NMG_FOLD_BUDGET_MS`（20 秒），超了跳过而不是失败；检索循环在 `NMG_SEARCH_DEADLINE_MS`（60 秒）停止并返回 `partial: true`；折叠的嵌入回填只做一批 32 条（`maxBatches: 1`），与检索路径一致——它原本会在一次请求里嵌完整个分区的积压。 + +## 证据 + +| 测量 | 值 | +| ------------------------------- | --------------------------------------------------------------------------- | +| 并发 1 Add p50 | 2.6 s（max 21.9 s，一次折叠） | +| 并发 2 Add p50 | 3.6 s（max 41.2 s，一次折叠） | +| 并发 4 Add p50 | 51.6-106 s，24 次里 7 次被杀 | +| 并发 16 连接失败 | 32 次里 5 次 | +| 本次改动后每次 Add 的提供商调用 | **至多 1 次**，即使这次 Add 触发了折叠（24 次顺序 Add，且证明折叠确实跑过） | +| 提供商重试预算 | 3 次尝试，只对 429/5xx；超时不重试 | + +## 这确立了什么、没有确立什么 + +- **没有在已部署构建上验证。** 上面每个数字都是改动前；改动后需要一次真实运行。 +- 它没有消除**折叠自身**的争用：同一用户两个并发折叠仍会在 manifest 上相遇并被重试。它消除的是**每次 accept 都会走的那条路**上的争用，而放大正是发生在那里。 +- 计数滞后只会导致多折一次，绝不会导致漏折：偏高只花一次列举，而读路径不再信任零。 + +## 考虑过的替代方案 + +- **保留计数、加大重试**：这就是被测量的失败形态，十二次尝试每次都重做 MB 级工作。 +- **分片计数（AWS 的 write sharding）或 G-Counter / PN-Counter**：分片只是把争用除以片数，仍然存在；只增计数器表达不了会缩小的积压，而 PN 的减法无法与日志核对共存，一旦核对就不再可合并。两者都劣于"派生"。 +- **把计数放 KV**（本平台 1 GB、当前未用）——与业界同形的答案，因为对象存储本来不是队列。没有采纳的理由：唯一消费者是一个人偶尔看 `/capacity`，一次列举就能答。 +- **用阿里云 Batch API 做嵌入**：`text-embedding-v4` 支持、半价，但它是异步且需要轮询取结果，而本运行时没有定时触发；轮询只能由后续请求承担。服务路径上否决，离线全库重嵌仍然适用。 +- **响应之后再嵌（waitUntil）**：我们的入口从不接收执行上下文（`edgeone-entry.mjs` 里明确不传），因此没有可注册的续体。 + +## 后果 + +- 一次 accept 就是几 KB 的一次 PUT。突发不再放大成重试。 +- `/capacity` 要花一次列举——所以由运维路径付，请求路径不付。 +- **现在回滚代码必须先排空日志。** 旧部署用 `pendingCount === 0` 短路，而本部署不再维护该计数，于是它可能把有内容的日志读成空。守卫是 `/admin/fold`：未清空就返回 409。 +- 一次 accept 的内联嵌入若超预算，那条记录会被嵌两次（一次被放弃、一次在折叠里）。这是本次引入的唯一重复花费，且只在提供商慢于 8 秒时发生。 From c3a81f488e9ca080329415df97bc21949381d9a5 Mon Sep 17 00:00:00 2001 From: wefio <48851810+wefio@users.noreply.github.com> Date: Fri, 30 Oct 2026 00:12:03 +0800 Subject: [PATCH 2/2] docs(template): make the doc-owner check name a path The checklist said "decision/design changes follow the doc-maintenance rules", which a submitter can tick without naming anything. The new item requires the owner and the file - decision record, design, experiment, postmortem or guide - or an explicit "none" with the reason, because a behaviour change with no owner named leaves the next reader nowhere to look, and "updated docs" is not a path. Also lands the decision record for the round that removed the write path's compare-and-swap. --- .github/pull_request_template.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index c7dae42e..ae0cd4e1 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -36,6 +36,7 @@ - [ ] If dependencies or a lockfile changed: `npm run check:lock` passes, the root `package-lock.json` is synchronized, and `npm audit --omit=dev --audit-level=high` reports no vulnerabilities - [ ] If a subpackage changed (such as `dsh/dsh-nmg`): `npm run verify:packages` passes (frozen-lockfile install + build), and `pnpm-lock.yaml`/the lockfile matches `package.json` - [ ] For documentation changes, `npm run docs:check` was run; decision/design changes follow the doc-maintenance rules +- [ ] **Every doc owner this change touches is named here**, with the file that was added or edited: decision record (`docs/decisions/`), design (`docs/design/`), experiment (`docs/experiments/`), postmortem (`docs/postmortem/`), guide (`docs/guides/`) — or `none`, with the reason. A behaviour change with no owner named is a defect, not an omission: the next reader has nowhere to look. "Updated docs" without a path does not satisfy this item. - [ ] `## Not verified` is filled in (write `None.` if nothing remains); non-trivial changes include a decision record - [ ] New code methods stay within the cyclomatic-complexity threshold (CodeFactor / `npm run complexity:gate`) - [ ] No reproducible build artifacts are committed: do not check in `dist/`, `dsh/dsh-nmg/lib/`, `src/prompts/nmg-prompts.generated.ts`, or `.nmg-search-scope` (see `docs/decisions/rejected/2026-09-02-track-build-artifacts-in-git.md`)