Skip to content

fix: preserve sandboxed background jobs on EPERM - #80

Open
behruznassre wants to merge 1 commit into
sendbird:mainfrom
behruznassre:fix/tracked-job-auto-reap-race
Open

fix: preserve sandboxed background jobs on EPERM#80
behruznassre wants to merge 1 commit into
sendbird:mainfrom
behruznassre:fix/tracked-job-auto-reap-race

Conversation

@behruznassre

@behruznassre behruznassre commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • preserve live background jobs when Codex's sandbox denies a zero-signal process probe with EPERM
  • keep ESRCH and other probe failures classified as not alive
  • add deterministic regression coverage for both branches

Root cause

isProcessAlive() treated every process.kill(pid, 0) exception as proof that the process was dead. reapStaleJobs() then moved the job to failed; the later successful completion CAS respected that terminal state, leaving the complete review only in the log.

The denial is scoped to non-descendant processes. Seatbelt denies signalling processes outside the sandbox, and reapStaleJobs() runs in a different invocation from the one that spawned the job — so the PID it probes is never its own child, and gets EPERM while the job is still running. A process probing its own child never reproduces this.

The zero-signal probe already sits at the correct shared boundary. This change preserves its current boolean API and recognizes the POSIX distinction: EPERM means the process exists but cannot be signaled, while ESRCH means it does not exist.

Validation

Probed inside a real Codex workspace-write sandbox (macOS 25.5.0, codex-cli 0.145.0), with an unsandboxed baseline on the same machine:

Case Unsandboxed In sandbox
Live own descendant alive alive
Live non-descendant, same uid alive EPERM
Dead own descendant ESRCH ESRCH
Dead non-descendant ESRCH ESRCH
Nonexistent PID ESRCH ESRCH
spawnSync ps ok EPERM (always)

Row 2 is the bug. Row 4 is why this fix is safe: a dead PID still returns ESRCH inside the sandbox, because the kernel resolves process existence before the sandbox policy denies the signal. EPERM → alive therefore cannot mask a genuinely dead process, and stale jobs are still reaped normally — no unbounded running state is introduced. REAP_GRACE_MS is 2s with no max-duration bound, so that distinction is load-bearing.

Also:

  • Regression test failed before the production change and passed afterward.
  • Real reapStaleJobs() path with an EPERM PID retained status: running, phase: running, and no error.
  • npm run check
    • 507 unit tests passed
    • 41 integration tests passed
    • 12 E2E tests passed
    • lint, typecheck, version sync, and changelog checks passed

An earlier revision of this description cited process.kill(1, 0) returning EPERM as sandbox evidence. That was wrong and has been replaced: kill(1, 0) returns EPERM on plain macOS with no sandbox, because a non-root user cannot signal root's init. The conclusion was right, the evidence was not.

Not addressed here

validateProcessIdentity() has the same defect — a bare catch returning false, wrapping getProcessIdentity()spawnSync ps, which the sandbox denies unconditionally (last row above). reapStaleJobs() prefers that branch whenever job.pidIdentity is set. It is unreachable today only because claude-cli.mjs swallows the identical ps failure at spawn and leaves pidIdentity: null, so the reaper falls back to isProcessAlive() and this fix applies. Tracked separately rather than widened into this PR.

Closes #79

Treat permission-denied zero-signal probes as evidence that a process exists so the stale-job reaper does not fail live Claude jobs. Add deterministic EPERM and ESRCH coverage.\n\nRefs sendbird#79
@upwind-code-us

Copy link
Copy Markdown

Upwind Upwind IaC Scan - 🔍 Scan in progress…

Upwind is scanning this PR. Results will appear here when the scan completes.

@upwind-code-us

Copy link
Copy Markdown

Upwind Upwind Code Scan - 🔍 Scan in progress…

Upwind is scanning this PR. Results will appear here when the scan completes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Background jobs are auto-reaped when sandbox denies process liveness probe

1 participant