Skip to content

Never drop the first prompt when the Smart Routing model switch fails - #863

Merged
lilly-luo merged 2 commits into
databricks:mainfrom
michaelp-db:first-prompt-replay-fail-open
Oct 1, 2026
Merged

lilly-luo merged 2 commits into
databricks:mainfrom
michaelp-db:first-prompt-replay-fail-open

Conversation

@michaelp-db

Copy link
Copy Markdown
Collaborator

What

Smart Routing's first-prompt replay can no longer drop the user's prompt. If the /model switch can't be confirmed, ucode now submits the prompt on the default model. Before this change it left the prompt typed but unsubmitted, or SIGTERMed Claude.

Why

The UserPromptSubmit hook blocks the first prompt, and run_claude_pty switches the model and replays it. Every non-happy path lost the prompt:

  • Switch timeout (6 s): ESC, then the prompt is re-pasted with submit=False ("restored but not submitted").
  • Persist timeout (2 s): os.kill(pid, SIGTERM).
  • waiting_to_switch: no bound. If output never goes quiet for 0.75 s, the prompt is never replayed.
  • Happy path: inject_prompt wrote ESC[200~…ESC[201~\r in one write. A freshly booted TUI intermittently absorbs a \r bundled with the paste close.

In unattended launches (isaac/OpenUI threads started with a positional prompt) nobody presses Enter, so the agent sits idle. Full write-up with diagrams: OpenUI first-prompt drop: ucode Smart Routing replay race

How

  • Paste with submit=False, then send Enter after PASTE_SUBMIT_DELAY_S (0.3 s).
  • Verify submission: the per-launch hook fires again when the replayed prompt is submitted (serve_first_prompt_socket(on_submitted_prompt=…)). If it hasn't fired within SUBMIT_VERIFY_S (4 s, which covers the hook CLI's cold start of about 1.4 s), re-send Enter, up to MAX_SUBMIT_ATTEMPTS (3). Extra Enters into an empty box are no-ops.
  • Fail open:
    • Switch timeout: ESC, restore the model setting, wait ESC_GAP_S so the paste isn't parsed as an Alt sequence, then submit.
    • Persist timeout: restore the setting, print a note to check /model, then submit. Claude is no longer killed.
    • READY_TIMEOUT_S (10 s): skip the switch and submit.

Tests

tests/test_claude_pty_replay.py covers the paste/Enter gap, Enter retry and stop-on-signal, all three fail-open paths, and the socket callback. just lint passes. Full suite (hermetic HOME, excluding test_codex_smart_routing_v2.py, whose test_owns_app_server_interposer_and_tui_lifecycle kills the pytest process on main too): 2744 passed, 41 failed. The same 41 fail on main; they are environment-dependent and unrelated.

This pull request and its description were written by Isaac.

The first-prompt replay lost the user's prompt on every non-happy path:
switch timeout left it pasted but unsubmitted, persist timeout SIGTERMed
Claude, an output stream that never went quiet stalled forever, and the
success path bundled Enter with the bracketed-paste close, which a freshly
booted TUI can absorb. Unattended launches (isaac/OpenUI) have nobody to
press Enter, so the agent sat idle.

Paste and Enter are now separate writes; submission is confirmed by the
UserPromptSubmit hook re-firing, with bounded Enter retries; and every
switch failure falls back to submitting on the default model.

Co-authored-by: Isaac <no-reply@databricks.com>
@michaelp-db

Copy link
Copy Markdown
Collaborator Author

🤖 @andy-xu-db @lilly-luo could you review? This hardens the first-prompt replay in claude_pty.py so a failed Smart Routing model switch submits the prompt on the default model instead of dropping it (details + diagrams in the linked doc). I can't request reviewers from a fork, hence the mention.

@lilly-luo

Copy link
Copy Markdown
Collaborator

the ci isn't working properly due to it being opened on a fork. confirmed first prompt works locally
Screenshot 2026-10-01 at 12 09 41 AM

@lilly-luo
lilly-luo merged commit 5fcfce7 into databricks:main Oct 1, 2026
41 of 43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants