feat(cli): really stop a job that uipath server is running [PC-4873]

[robert-ursu]

Draft. Stacked on #1835 (base feat/async-job-dispatch-and-result-push), so this diff is only the stop work. Re-cut on main @ 7bd1425: the queue, registry and result-push layer from the earlier version are gone because RunJob spans the job over uipath-ipc. What carries over is JobControl / run_job_loop and the stop sequence.

Paired with hdens#7868, which makes the handler send the stop and act on the answer, and hdens#7708, which kills and restarts the server when the answer is "not stopped". PC-4873

Why

uipath server could not stop a job:

• IPC StopJob was a no-op (fix(cli): report a job's real exit code and stop claiming stops [PC-4873] #1835 makes it return false in the meantime).
• HTTP had no stop route at all.
• Cancelling the awaiting task was worse than doing nothing. When the handler's IPC connection drops, uipath-ipc cancels every in-flight handler (uipath_ipc/client/connection.py, _teardown). await asyncio.to_thread(...) raised, the lock was released and env/cwd were restored while the job thread kept running. The next job then ran concurrently in a mutated process.

How a job is stopped

run, debug and eval each drive their own event loop on the worker thread, so a job is an event loop. They now call run_job_loop instead of asyncio.run. It publishes the loop and its root task to the JobControl carried on a ContextVar. Outside the server (uipath run on a terminal) it is asyncio.run unchanged.

stop_job(job_key, resume_version, force) is shared by IPC StopJob and the new POST /jobs/{key}/stop (200 {"stopped": bool}). It works in steps:

1. Cancel the root task. The runtime unwinds cooperatively, and UiPathRuntimeContext.__exit__ still writes the result. Wait 30 s (5 s with forceStop).
2. Cancel every task on the job's loop. This gives up on a clean cleanup. Wait 10 s (5 s with forceStop).
3. Answer False if the job is still running. It is stuck in a call that cannot be interrupted, and only ending the process stops it. Whether to do that is the handler's call: the process and the execution instance belong to it.

Cases that answer True:

• A queued job (waiting for the lock) is dropped before it runs.
• An unknown job, or a live run with a different resume version, is not running. Answering False there would invite the handler to kill a resumed run for a stop aimed at the previous one.

Within 60 s, which is the handler's StopJobTimeout.

The lock waits for the thread

The job core now returns or raises only once the worker thread has exited. Cancelling the caller, for example on a dropped connection, stops the job. The CancelledError is re-raised only after that, so the lock, env and cwd are never handed on while the job still runs.

The job-scope teardown (#1890) also completes before the env is restored. That removes the xfail in test_server_job_core.py.

Outcomes

How the job ended	ExitCode	Error
Stopped on request	143	Job stopped on request
Its own code let a CancelledError escape	1 (unexpected)	Job cancelled itself
Otherwise	unchanged from #1835	unchanged

A stopped job's result document still says Faulted/ERROR_CancelledError. That comes from uipath-runtime and is a later PR. The handler reports Stopped for an Orchestrator stop regardless of what the runtime wrote.

Worth a careful look

• The worker is a plain run_in_executor future, not a task. A task re-raises the job's SystemExit into the server loop. The job core has always turned SystemExit into an exit code.
• A job waiting on its own to_thread work (sync LangGraph nodes, sync HTTP clients) runs its finally at once. But the job's asyncio.Runner does not close until that call returns. run_job_loop withdraws the loop before the runner closes, so the sweep in step 2 cannot cancel the runner's wait on those threads, which would leak them. Such a job answers False if the call outlives the grace.
• JobControl.cancel() delivers at most once. A stop followed by a force stop is ordinary, and a second delivery would land inside the cleanup that writes output.json.

Testing

• uv run pytest tests/cli: 1519 passed, 1 skipped. The suite was run three times to check for flakiness.
• ruff check, ruff format --check and mypy are clean.
• New tests are in tests/cli/test_server_cancellation.py (16). They use click commands shaped like the real ones and cover:
  • a running job stops, its cleanup runs and the lock is freed;
  • a stop that arrives before the job has a loop is applied when it gets one;
  • a repeated or forced stop does not abort a slow cleanup;
  • a job waiting on a nested thread stops once that call returns;
  • an uninterruptible job answers False and keeps the lock until its thread exits;
  • a self-inflicted cancel is a fault;
  • a queued job is dropped;
  • a stop for another resume version leaves the live run alone;
  • a cancelled caller stops the job and waits for its thread, keeping the lock and the job's env meanwhile;
  • IPC StopJob stops a running RunJob;
  • the HTTP stop route stops a running /start and validates its body.
• Version 2.14.26 → 2.14.27.

Jira

PC-4873

🤖 Generated with Claude Code

https://claude.ai/code/session_01CFMKm7zHGcptS49z4cnnad

[github-actions]

🚨 Heads up: uipath-langchain cross-tests are FAILING 🚨

Your changes may break the uipath-langchain-python integration.

⚠️ These checks are NOT enforced by branch protection rules. Please review the failures before merging.

🔍 Inspect the failed run →