Skip to content

feat(desktop): unify WorkHub conversation and floating controls - #4979

Open
M4n5ter wants to merge 24 commits into
apache:mainfrom
M4n5ter:feat/desktop-assistant
Open

feat(desktop): unify WorkHub conversation and floating controls#4979
M4n5ter wants to merge 24 commits into
apache:mainfrom
M4n5ter:feat/desktop-assistant

Conversation

@M4n5ter

@M4n5ter M4n5ter commented Sep 7, 2026

Copy link
Copy Markdown
Member
English

Manual recordings

Cap.2026-09-08.at.17.18.47.mp4
Cap.2026-09-08.at.16.53.45.mp4

Recorded by M4n5ter; predates the final shortcut adjustment below.

Summary

Make WorkHub a persistent conversation for coordinating tasks and controlling Maka. The main and floating windows share one renderer and Host-owned Session, preserving drafts, attachments and running work across window changes, and conversation history after restart.

  • Reuse the shared composer, model selection, transcript and task tools. Show submitted messages and the waiting timer immediately, including before Host admission.
  • Operate observed application controls with target revalidation, verified preference changes and guarded undo. User clicks, scrolling and typing interrupt control; pointer movement does not. Terminal, embedded browser, password controls and explicitly excluded elements remain outside observation. Other visible content may reach the model without secret scanning or rewriting.
  • Cmd/Ctrl+Shift+K shows or hides floating WorkHub without opening the main window. The return button docks it into Maka. Compact mode provides a draggable model wheel and bounded multiline input; animated expansion preserves the window's bottom edge. Both windows share theme and typography.
  • Keep task navigation draggable and main-window menus accessible over docked WorkHub. Preserve the macOS Dock icon and handle window destruction and subscription cancellation cleanly.
  • Remove the separate assistant state and obsolete action/record protocol. Admit WorkHub actions only through the active coordination turn, retaining task authority.

Protocol epoch 134 requires Desktop and Host to update together. New conversations persist after restart; pre-cutover beta history migration is out of scope.

Verification

  • Local workspace build, typecheck, lint, formatting, architecture and Desktop/UI unused-code checks pass. All workspace test suites pass after rebasing onto main.
  • Three WorkHub Electron scenarios pass, covering sending, model preview/commit, renderer/draft preservation across window changes, and crash recovery into the same running Session. Wheel focus and light/dark appearance are verified. Native macOS checks cover shortcut visibility, docking, expansion and the Dock icon.
  • Remote Hosts and native Windows/Linux interactions were not manually tested.

AI use

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Codex implemented and validated the change; GPT-5.6 Luna and DeepSeek ran model-driven scenarios.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally
  • Changes behavior as described above
中文

手动录屏

Cap.2026-09-08.at.17.18.47.mp4
Cap.2026-09-08.at.16.53.45.mp4

M4n5ter 录制,早于下述快捷键最终调整。

变更

将 WorkHub 统一为协调任务、操作 Maka 的持久对话。主窗口与悬浮窗共享渲染实例及 Host 会话;窗口切换保留草稿、附件和执行状态,重启恢复对话历史。

  • 复用公共输入框、模型选择、对话记录及任务工具。发送后立即显示消息和等待计时,覆盖 Host 确认前的空档。
  • 根据界面观察操作应用,复核目标、验证设置变更并提供受保护的撤销。用户点击、滚动和输入会接管,鼠标移动不中断;终端、内嵌浏览器、密码控件和显式排除的元素不在观察范围内;其他可见内容可能进入模型,不扫描或改写其中的秘密。
  • ⌘⇧K / Ctrl+Shift+K 显示或隐藏悬浮窗,不打开主窗口;返回按钮将 WorkHub 收回 Maka。紧凑模式支持拖动模型滚轮和限高输入,收放动画保持底边位置;两个窗口共用主题和字体设置。
  • 任务导航支持拖动,主窗口菜单可覆盖停靠的 WorkHub。保留 macOS Dock 图标,正常处理窗口销毁与订阅取消。
  • 删除独立助手状态及废弃的 action/record 协议,仅允许当前协调回合提交 WorkHub 动作,保留任务权限边界。

协议 epoch 134,Desktop 与 Host 需同步更新。新对话重启后仍保留;不承诺迁移切换前的 beta 历史。

验证

  • 本地工作区构建、类型、lint、格式、架构及 Desktop/UI 未使用代码检查通过。rebase 最新 main 后,各工作区测试全部通过。
  • 三项 WorkHub Electron 场景通过,覆盖发送、模型预览与确认、窗口切换保留渲染实例和草稿,以及崩溃后恢复原运行会话;滚轮焦点和明暗主题已验证;原生 macOS 验证了快捷键显隐、停靠、收放和 Dock 图标。
  • 未手动验证远程 Host 和 Windows/Linux 原生交互。

AI 使用

Codex 实现并验证;GPT-5.6 Luna 与 DeepSeek 执行真实模型场景。

@github-actions github-actions Bot added the effort/XXL Over 2500 readable lines label Sep 7, 2026
@M4n5ter
M4n5ter changed the base branch from main to stack/4956-session-experience-foundation September 7, 2026 10:49
@M4n5ter
M4n5ter force-pushed the feat/desktop-assistant branch from 38e10ea to 16cd57b Compare September 7, 2026 12:25
@M4n5ter M4n5ter changed the title feat(desktop): add a floating assistant with visible UI control feat(desktop): add a floating UI assistant / 新增悬浮界面助手 Sep 7, 2026
@M4n5ter
M4n5ter changed the base branch from stack/4956-session-experience-foundation to main September 7, 2026 12:27
@M4n5ter
M4n5ter force-pushed the feat/desktop-assistant branch 2 times, most recently from 0931e67 to ae57ce4 Compare September 8, 2026 03:50
@M4n5ter M4n5ter changed the title feat(desktop): add a floating UI assistant / 新增悬浮界面助手 feat(desktop): unify WorkHub conversation and floating controls / 统一工作台对话与悬浮操作 Sep 8, 2026
@M4n5ter
M4n5ter force-pushed the feat/desktop-assistant branch 2 times, most recently from fb8b920 to f399734 Compare September 8, 2026 05:27
@M4n5ter M4n5ter changed the title feat(desktop): unify WorkHub conversation and floating controls / 统一工作台对话与悬浮操作 feat(desktop): unify WorkHub conversation and floating controls Sep 8, 2026
@M4n5ter
M4n5ter force-pushed the feat/desktop-assistant branch from 8ad9f1c to a456645 Compare September 8, 2026 07:44
@M4n5ter
M4n5ter marked this pull request as ready for review September 8, 2026 10:19
Keep one durable coordination Session and one live renderer across docking, floating and hiding. Route input through its model, preserve delegated task authority, and reuse the shared composer, attachments and transcript ledger.

Generated-by: Codex
…lity

Use active coordination Turns for task actions, share document appearance, and keep shortcut visibility separate from explicit docking.

Generated-by: Codex
Account for legacy act and record grants during credential migration without granting active-turn authority. Refresh the renderer inventory after rebasing onto main.

Generated-by: Codex
@M4n5ter
M4n5ter force-pushed the feat/desktop-assistant branch from c72de7f to 238f01e Compare September 8, 2026 10:38

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for bringing the main and floating WorkHub into one conversation. The shared renderer and Host-owned execution direction looks sound, and the model wheel is worth keeping. Reviewed 238f01ead6ed25fcb1114149dfece8edb2363752 with six focused reviewer passes and primary-agent source/probe reconciliation.

[P2] Remove heuristic secret detection and state the observation contract accurately. Following the direction in #4119, please remove the sensitiveField label/name regex and its heuristic-only tests rather than extending its vocabulary. Our Chromium probe confirmed that ordinary MCP JSON, environment and HTTP-header editors can contribute their values to the observation. Under the intended arbitrary-content contract, this is not a request for another scrubber or a general DLP guarantee.

Keep explicit observation exclusions, password-control boundaries and credential-store/read-API protections. State that visible arbitrary content may reach the model and is not scanned or rewritten for secrets; the current blanket claim that secret inputs remain outside the control surface is too broad. Scope this cleanup to the WorkHub observation code, without pulling the repository-wide redaction removal into this PR.

Legacy WorkHub history migration is not a requirement for this beta cutover. Please adjust the persisted-history compatibility wording to distinguish pre-cutover beta history from restart persistence of new conversations; no legacy reader needs to be restored for this review.

Two local P2s are inline: recover a crashed WorkHub renderer, and keep the wheel's accessibility selection consistent with the committed model. Preserve the wheel interaction while checking the existing Astryx component/extension seam for semantics, focus and theme reuse.

Current-head CI is green. Validation included a Chromium observation probe with fake credentials, an Electron renderer-crash probe and source-level reconciliation; this is not full application UI acceptance or a fresh full-suite run. After the fixes, verify crash/reopen recovery and wheel preview/commit behavior, including light/dark appearance. AI-assisted review.

简体中文

感谢把主窗口与悬浮 WorkHub 收敛为同一个对话。共享 renderer、执行归 Host 的方向合理,模型滚轮也值得保留。本轮基于 238f01ea,由六位 reviewer 分工,再由主审核对源码与探针。

P2:删除启发式秘密检测,准确说明观察契约。#4119 的方向,请删除 sensitiveField 字段标签/名称正则及仅服务于该启发式的测试,不继续补词。Chromium 探针确认普通 MCP JSON、环境变量、HTTP headers 编辑器的值可以进入观察结果;在任意内容不承诺脱敏的契约下,这不要求新增 scrubber 或通用 DLP 保证。

保留明确的观察排除、密码控件边界和 credential store/读取 API 的保护。说明可见的任意内容可能进入模型,系统不扫描或改写其中的秘密;目前“秘密输入均在控制范围之外”的承诺过宽。本次只清理 WorkHub 观察代码,不把全仓脱敏删除塞进这个 PR。

这次 beta 切换不要求迁移旧 WorkHub 历史。请区分旧 beta 历史兼容与新对话重启后持久化的承诺;不要求恢复旧 reader。

两条局部 P2 已放行内:WorkHub renderer 崩溃恢复,以及滚轮无障碍选中状态与实际模型一致。保留滚轮交互,同时核对 Astryx 现有组件或扩展接缝,复用语义、焦点与主题机制。

当前 CI 全绿。验证包括假凭据 Chromium 观察探针、Electron renderer 崩溃探针和源码交叉核对,不代表完整应用界面验收或重新跑过全套测试。修复后请验证崩溃重开恢复、滚轮预览与确认,并检查 light/dark 外观。AI 辅助评审。

Comment thread apps/desktop/src/main/workhub-presentation.ts
Comment thread packages/ui/src/chat-model-switcher.tsx Outdated
…on contracts

Recreate a crashed renderer through the existing presentation lifecycle and revalidate queued IPC ownership. Keep wheel preview separate from committed accessibility selection, reuse focus tokens, and remove heuristic secret detection in favor of explicit observation boundaries.

Generated-by: Codex
@M4n5ter

M4n5ter commented Sep 8, 2026

Copy link
Copy Markdown
Member Author

Addressed in f546908, with details in both inline threads.

  • Removed the WorkHub label/name secret-detection regex. The existing test covers structural exclusions, so it was retained and renamed. Explicit exclusions and password boundaries remain; credential storage/read APIs were not changed. A real Chromium probe with fake credentials verifies that arbitrary editor/JSON content remains observable while explicitly excluded and password content stays outside observation.
  • Updated the tool description, PR body and domain documentation: visible arbitrary content may reach the model without secret scanning or rewriting. This cleanup is limited to WorkHub observation.
  • Corrected the history contract: new conversations persist across restarts; pre-cutover beta history migration is out of scope.

Validation: 422 UI tests, 19 Desktop control/presentation tests, all three WorkHub Electron scenarios, and Desktop typecheck, lint, formatting and the E2E budget pass. Crash recovery and wheel preview/commit were exercised in Electron; light/dark screenshots were inspected. The full workspace suite passed on the parent commit and was not rerun for this correction.

@M4n5ter
M4n5ter requested a review from Astro-Han September 8, 2026 13:13

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes. Rechecked f5469080a79036194d4497ba5240199b6ad46997 against 238f01ea with four focused reviewer passes and primary-agent reconciliation.

One P2 remains in the renderer-recovery thread: a docked renderer crash leaves an empty dock, and selecting the already-active WorkHub navigation does not recreate it. The existing owner should provide recovery through the actual visible dock entry. Details and the invariant are in the existing inline thread.

The wheel's preview/committed-selection distinction is fixed. Heuristic secret detection is removed while explicit exclusions and password boundaries remain, consistent with #4119; the beta-history migration exclusion is clear. This revision does not add a second execution authority or a parallel recovery implementation.

I ran 21 focused tests against the current source snapshot, all passing. Replacing the two affected implementations with the previous revision made both new regression checks fail, but those tests do not cover the remaining docked navigation path. I inspected the added Electron scenarios but did not rerun the full application scenarios locally. Current-head CI is green.

Human UI acceptance remains: docking/floating and shortcuts, conversation continuity, wheel preview/confirmation, and focus/light/dark appearance. Approval and merge remain pending that check. AI-assisted review; full manual UI acceptance is not claimed.

简体中文

感谢修复。本轮由四位 reviewer 分工、主审核对,以 f5469080 对照 238f01ea

仍有一项 P2,已补充在原崩溃恢复串中: 停靠态 renderer 崩溃后为空白,再点已经激活的 WorkHub 导航不会重建。应由现有 owner 通过实际可见 dock 入口提供恢复;具体路径与不变量见原行内串。

滚轮预览与实际选中已分开;启发式秘密检测已删除,显式排除和密码边界保留,符合 #4119;旧 beta 历史不迁移的范围也已明确。本轮未新增第二执行权威或并行恢复实现。

主审基于当前源码快照运行 21 项定向测试,全部通过;两处实现换回上一版后,两项新增回归检查均失败,但未覆盖剩余的停靠导航路径。已检查新增 Electron 场景,本轮未重新运行完整应用场景。当前 CI 全绿。

仍待人工确认停靠/悬浮、快捷键、对话连续性、滚轮预览/确认、焦点和明暗主题,再批准与合并。AI 辅助评审,不代表完整人工界面验收。

Publish unexpected renderer exits through the existing presentation owner and restore through its explicit dock action. Keep geometry notifications from triggering automatic reloads; verify recovery with an active session and unchanged dock bounds.

Generated-by: Codex

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing the docked crash path. Rechecked abbf4c18581d2a12bb23f23ad060c37a2ce7e5b2 with eight fresh reviewers, followed by primary-agent source checks, counterexamples and adjudication.

Two P2s remain, detailed inline: Stop can be lost before admission, and the native WorkHub entry points bypass the retained enable setting. Both already exist at f5469080; neither was introduced by the latest renderer-recovery fix. Please correct them at the existing pending-send/admission and client-settings boundaries, respectively. They do not need a new recovery framework, stop authority or settings store.

The prior docked-renderer finding is fixed. The actual dock now exposes Retry without a geometry change and uses the existing dock/create/ready owner. Its consumer probe passes here and fails with the previous component; the presentation regression also fails with the previous owner implementation. Normal window changes retain one renderer, and crash recovery retains the Host Session.

34 targeted checks passed, including presentation, control, runtime task calls, send visibility, tray, model picker and the actual React dock consumer. The separate delayed-admission Stop probe failed as described inline. These were source-based checks, with mocked DOM/native dependencies and an extracted existing stop function where stated; I did not rerun the complete Electron application scenarios. The requesting maintainer has confirmed manual UI acceptance.

The cumulative diff is 172 files, +8,531/-17,883. I found no demonstrated second execution authority or need to restore the removed renderer routing/action mechanism. Candidates based on old-reader assumptions, unsupported DOM mutation, or coverage alone were excluded. Preserve shared attachment handling and the existing wheel rather than adding parallel implementations or more speculative tests.

Converge the next pass on state transitions and event interleavings. The useful review model here is state-machine-based invariant checking: follow what happens when the same event arrives in different states. This is a reasoning/test model, not a request to introduce a state-machine library or another lifecycle manager. The two findings have different owners and should remain two small corrections.

  • Send/Stop: follow the existing send attempt through pending admission, admitted/running, and terminal/rejected states, with response loss treated as uncertainty rather than proof of failure. Check Stop before admission, after admission, and before a late admission observation when the response was lost. Keep it bound to the original Session/Turn. A Stop must not be reported as settled merely because that Turn is not visible yet; reconcile the intent when its outcome becomes known. Failed admission must clear the pending intent, and it must never stop a later or different Turn. Reuse the existing deferred-admission fixture and vary these event orders; do not build a separate cancellation store or a general race-test framework.
  • Enable setting/presentation: check disabled → shortcut/tray/sidebar open attempts, enabled → open, and open → disabled, including disabling while a window-open operation is pending. All entries should converge on the existing client setting. Disabling should leave no active WorkHub presentation while preserving drafts and Host-owned work. Reuse the presentation/settings seam; do not make another copy of the setting or add an execution-cancellation policy.

Before changing code, briefly name the violated invariant, its current owner, and the minimum correction for each group. Afterward, show the decisive checks fail without their correction, inspect the cumulative diff, and remove superseded branches or duplicate tests. The stopping condition is these two contracts plus the existing crash/wheel regressions passing—not an exhaustive cross-product of every event, platform and failure. Add another case only when a concrete uncovered transition justifies it. We have enough broad review coverage for this round; the next pass should close these demonstrated gaps without expanding the PR.

Current-head CI is green and mergeability is MERGEABLE. This remains a COMMENT pending the two corrections; no approval or merge was performed. AI-assisted review.

简体中文

感谢修复停靠态崩溃路径。本轮以 abbf4c18 为准,由八位 fresh reviewer 分工,再由主审核对源码、反例与归因。

仍有两项 P2,详见行内: 准入前的 Stop 意图丢失;原生 WorkHub 入口绕过保留的启用设置。两项在上个 f5469080 已存在,都不是最新 renderer 恢复修复引入。分别在现有 pending-send/准入收敛和 client setting 边界修复即可,不需要新增恢复框架、停止权威或设置存储。

此前停靠态崩溃问题已解决。实际 dock 无需几何变化即可显示 Retry,并复用原 dock/create/ready owner。消费者探针当前通过、换回旧组件失败;presentation 回归换回旧 owner 也失败。正常窗口切换保留同一 renderer,崩溃恢复保留 Host Session。

34 项定向检查通过,覆盖 presentation、control、任务调用、发送可见性、托盘、模型选择和实际 React dock。另一个延迟准入 Stop 探针按行内描述失败。这是源码级检查,相关 DOM/native 依赖使用 mock,停止函数按说明从现有源码提取;本轮未重跑完整 Electron 应用场景。发起评审的维护者已确认人工 UI 验收通过。

累计 diff 为 172 文件,+8,531/-17,883。未发现已证实的第二执行权威,也没有必要恢复已删除的 renderer routing/action 机制。基于旧 reader 假设、无生产来源的 DOM 改写或仅覆盖率缺口的候选均未采纳。保留共享附件处理和滚轮,不新增并行实现或推测性测试。

下一轮围绕“状态转换与事件交错”收敛。 这里适合用“基于状态机的不变量验证”:检查同一事件在不同状态下发生时,结果是否仍符合契约。这是推理与测试方法,不要求引入状态机库或生命周期管理器。两项问题的 owner 不同,保持两处小修正,不强行统一抽象。

  • 发送/停止: 沿现有发送尝试检查等待准入、已准入/运行、终态/拒绝;响应丢失表示结果未知,不等于失败。检查准入前 Stop、准入后 Stop,以及响应丢失后迟到准入观察之前的 Stop。始终绑定原 Session/Turn,不能因为暂时看不到 Turn 就把 Stop 当作已处理;结果确认后应完成收敛。准入失败要清理意图,不能误停后续或其他 Turn。复用现有延迟准入夹具改变事件顺序,不新增取消存储或通用竞态测试框架。
  • 启用设置/展示: 检查关闭 → 从快捷键/托盘/侧栏尝试打开,开启 → 打开,以及打开 → 关闭;包含窗口打开操作尚未完成时关闭设置。所有入口向既有 client setting 收敛。关闭后不应继续展示 WorkHub,同时保留草稿与 Host 执行。复用 presentation/settings 接缝,不复制设置,不另加执行取消策略。

改代码前,分别写明不变量、现有 owner 和最小修法。修后给出撤掉对应修复就失败的关键验证,再检查累计 diff,删除被替代分支和重复测试。停止条件是这两组契约及已有崩溃/滚轮回归通过,不是穷举所有事件、平台和故障组合;只有具体未覆盖的转换才值得加用例。本轮广泛扫描已经足够,下一轮应闭合已证实缺口,不再扩张 PR。

当前 CI 全绿,MERGEABLE。本次仍为 COMMENT,等待这两项修正;未批准或合并。AI 辅助评审。

Comment thread apps/desktop/src/main/runtime-host-boot.ts Outdated
Retain exact Session/Turn Stop intent across delayed or uncertain admission and retire it on interruption, rejection or terminal evidence. Gate native presentation through the existing client setting and hide on disable without discarding drafts or cancelling Host work.

Generated-by: Codex
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XXL Over 2500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants