perf: batch remote filesystem walks by 7Sageer · Pull Request #4539 · MoonshotAI/kimi-code

7Sageer · 2026-10-30T03:37:48Z

Requirement or Bug

在 #4062 的远程执行环境上，文件搜索、补全和无 ripgrep 时的 grep 不应按目录数量放大 RPC 往返时间。

Bug Reproduction Steps

N/A

Root Cause

N/A

Code Changes

WorkspaceFsService.search / suggestWithNode / grepWithNode
-  readdir(root) -> readdir(child) -> ...    每个目录一次 RPC
+  hostFs.walk(root, ignoreRules)             新 executor 每个 root 一次 RPC
+  group entries by parent -> DFS visit      保持原有访问、匹配与去重顺序
+  collect errors + truncated               保留超大目录错误，报告不完整遍历

IHostFileSystem.walk
  HostFileSystem / AcpHostFileSystem    共用本地有界 walk；ACP 沿用原目录后端
  RemoteFileSystem                      fs/walk（data lane）
    ├─ MethodNotFound                   缓存旧 executor 能力，逐目录回退
    └─ JSON ignore > 1 MiB              不发送大请求，逐目录回退且保留全部规则
  executor FsHandler.walk                校验参数并在目标侧剪枝 ignore 子树

walk 保留 .gitignore、始终排除 .git、隐藏文件、glob、symlink 和去重语义；目录 symlink 不递归，默认 POSIX 路径规则与本机平台解耦。
限制深度 64、目录 10,000、条目 100,000、单目录 50,000、响应约 8 MiB；walk 默认 55 秒、上限 60 秒，给 RPC 返回预留余量，suggest / grep 保持各自 10 / 30 秒预算。
多 root 共用整体 suggest / grep deadline，每次 walk 只使用剩余预算；调用前或 RPC 等待期间中止时立即返回截断结果，旧 executor 回退同样生效。
目录枚举期间的普通读取错误继续跳过，超大目录仍映射为 FS_TOO_MANY_RESULTS。
协议 README 同步 fs/walk 与兼容回退。

Behavior Changes and Affected Users

Behavior	Before	After	Who relies on the old behavior	Escape hatch
远程 Node 文件枚举	每个目录一次 `fs/readDirectory`	新 executor 每个 workspace root 一次 `fs/walk`	使用 SSH / Docker / command 环境的 TUI、print、desktop / web、VS Code、ACP 与 SDK 用户	无开关；旧 executor 自动兼容
旧 executor	原有逐目录枚举	首次 walk 探测失败后缓存能力并逐目录回退，保留原行为	尚未升级远端 `kimi exec-server` 的远程会话	不需要升级；升级后启用批量 walk
大 gitignore	本地 matcher 逐目录筛选	不超过 1 MiB 的规则在 executor 侧剪枝；超限完整保留规则并回退逐目录	使用大 `.gitignore` 的远程 workspace	自动回退，无配置
遍历范围与结果完整性	深度限制 64；无全局目录 / 条目 / 响应预算	新增上述有界预算；达到目录、条目、响应或超时预算时 `truncated=true`	极大 workspace 中的搜索、补全及无 ripgrep 的 grep	缩小路径 / glob / 查询范围；ripgrep 路径不变
本地与 ACP 目录访问	本机逐目录递归	共用同一有界 walk 后恢复 DFS 顺序	默认本地会话及使用 ACP 文件能力的客户端	无需配置
文件读写与 rg 快路径	既有实现	不改变	所有文件读写用户；已安装 ripgrep 的搜索路径	N/A

旧路径功能对应：根 .gitignore 读取与缓存保留在 ignoreFor，.git/ 在无 gitignore 模式也作为默认剪枝规则；隐藏文件 / glob 继续在 Workspace visitor 筛选；symlink 仍只作为条目访问、不递归；real-root 去重及多 root 展示路径不变；DFS 访问顺序由 parent 分组恢复；普通 readdir 失败继续跳过，超大目录错误仍抛出。

测试覆盖：

agent-core-v2/test/remote/fs.test.ts：单 RPC、ignore 剪枝、旧 executor 能力缓存、超限规则回退、服务端参数拒绝。
agent-core-v2/test/os/backends/node-local/hostFsService.test.ts：symlink、深度、条目 / 目录 / 响应预算、取消与超时、错误收集。
agent-core-v2/test/workspace/workspaceFs/fsService.test.ts：忽略规则传递、truncated 传播、DFS 顺序、深层目录边界。
agent-core-v2 与 acp-server typecheck；ACP fs / terminal 11 项；远程 / Workspace / 本地 walk 130 项；type-aware lint 0 errors（既有 warnings）。
干净 Linux 隔离环境全量测试：18,225 通过，20 skipped。
import-boundary 仅剩基线已有的 loop.ts、loopService.ts、origin.ts 3 条；本 diff 无新增违规。

Checklist

I have read the CONTRIBUTING document.
I have linked a related issue（延续 feat: remote environments for SSH and Docker execution targets (phase 1: creation-time binding) #4062 的远程环境实现）。
I have added tests that prove my feature works.
The behavior-change table above is complete, and every removed behavior or flipped default is named in the changeset and either has an escape hatch or was explicitly approved by a maintainer in this PR.
Ran gen-changesets skill：patch，已写入。
Ran gen-docs skill：无需新增用户文档；远程协议 README 已更新。

changeset-bot · 2026-10-30T03:37:52Z

🦋 Changeset detected

Latest commit: 76f441c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@moonshot-ai/kimi-code	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

chatgpt-codex-connector · 2026-10-30T03:37:59Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-30T04:09:54.976311Z	`76f441c`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

7Hanrui · 2026-10-30T03:38:00Z

@codex review

pkg-pr-new · 2026-10-30T03:39:08Z

pnpm dlx https://pkg.pr.new/@moonshot-ai/kimi-code@76f441c

npx https://pkg.pr.new/@moonshot-ai/kimi-code@76f441c

commit: 76f441c

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cf68d7caf2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-10-30T03:46:01Z

+        `ignore must not exceed ${HOST_WALK_MAX_IGNORE_BYTES} bytes when encoded as JSON`,
+      );
+    }
+    const timeoutMs = optionalInteger(params, 'timeoutMs', 1, HOST_WALK_MAX_TIMEOUT_MS);


Keep walks inside the RPC deadline

WorkspaceFsService.search() does not pass timeoutMs, so this leaves executor-side fs/walk unbounded. For a remote workspace whose local enumeration takes more than 60 seconds in total, even when each directory read is fast, RemoteExecConnection.call() expires the request at 60 seconds; the user receives a timeout and the executor keeps the walk and its in-flight slot running instead of returning a partial result with truncated: true. Apply a default walk budget strictly below the transport deadline, and use the same bound for local searches.

AGENTS.md reference: AGENTS.md:L111-L113

Useful? React with 👍 / 👎.

7Hanrui · 2026-10-30T03:52:59Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4beb548315

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-10-30T03:59:34Z

+        ignore: options.ignore,
+        timeoutMs: options.timeoutMs,
+      }) as FsWalkResult;


Keep remote walks on the caller's deadline

When no-rg completion runs against multiple independent remote workspace roots, the first root can consume most of the global 10-second budget and the next root still gets a fresh 10-second fs/walk: this payload forwards timeoutMs but never checks or races options.signal. A large second root can therefore keep suggestWithNode pending for nearly 20 seconds; grepWithNode has the same per-root reset with its 30-second budget. Short-circuit aborted signals and carry the remaining overall deadline into each RPC.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-10-30T03:59:34Z

+"@moonshot-ai/kimi-code": patch
+---
+
+Speed up file completion in remote environments without ripgrep.


Disclose the new walk truncation limits

When a nonempty search traverses more than 10,000 directories, 100,000 entries, about 8 MiB of inventory, or 55 seconds, this commit now returns partial results even for local and ACP workspaces, whereas the previous DFS had no corresponding global bounds. The changeset mentions only faster remote completion, so release notes omit the completeness users lose and the lack of an escape hatch; name that truncation explicitly.

AGENTS.md reference: AGENTS.md:L115-L117

Useful? React with 👍 / 👎.

7Hanrui · 2026-10-30T04:04:01Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76f441c10d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-10-30T04:09:51Z

          matched += 1;
          top.push(this.displayCandidate(root, candidate));
-        });
+        }, { signal, timeoutMs: Math.max(1, remainingMs) }) || inventoryTruncated;


Return partial walk results before the caller aborts

When a no-rg suggestion spends its 10-second budget enumerating a large root, this gives hostFs.walk the entire remaining deadline even though the outer abort timer is already counting down. For remote walks the caller signal therefore wins raceWalkAbort before the executor's equal-or-later timeout and substitutes EMPTY_HOST_WALK_RESULT; for local walks the signal is already aborted before the newly buffered visitor runs. suggestWithNode consequently drops every candidate gathered before the deadline instead of returning the partial suggestions retained by the old DFS, and the identical full-budget call in grepWithNode can silently return zero scanned files rather than its zero-progress timeout. Give the walk an earlier internal deadline, or otherwise preserve its partial inventory, so it can be processed before the caller aborts.

AGENTS.md reference: AGENTS.md:L111-L113

Useful? React with 👍 / 👎.

7Sageer · 2026-10-30T04:12:03Z

Superseded by a redesign of walk deadline and cancellation semantics.

perf: batch remote filesystem walks

cf68d7c

Copilot AI balanced review requested due to automatic review settings October 30, 2026 03:37

Copilot AI reviewed Oct 30, 2026

test: add walk to mapped filesystem stub

7d995fa

chatgpt-codex-connector Bot reviewed Oct 30, 2026

View reviewed changes

fix: bound walks below the RPC deadline

4beb548

chatgpt-codex-connector Bot reviewed Oct 30, 2026

View reviewed changes

fix: keep remote walks on caller deadlines

76f441c

chatgpt-codex-connector Bot reviewed Oct 30, 2026

View reviewed changes

7Sageer closed this Oct 30, 2026

Conversation

7Sageer commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Requirement or Bug

Bug Reproduction Steps

Root Cause

Code Changes

Behavior Changes and Affected Users

Checklist

Uh oh!

changeset-bot Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

7Hanrui commented Oct 30, 2026

Uh oh!

pkg-pr-new Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

7Hanrui commented Oct 30, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

7Hanrui commented Oct 30, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

7Sageer commented Oct 30, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

7Sageer commented Oct 30, 2026 •

edited

Loading

changeset-bot Bot commented Oct 30, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 30, 2026 •

edited

Loading

pkg-pr-new Bot commented Oct 30, 2026 •

edited

Loading