Skip to content

feat(container-runner): drain engine sleeps unless the idle timer fired - #5594

Open
abcxff wants to merge 1 commit into
stack/feat-container-runner-jitter-idle-timeout-to-avoid-teardown-waves-wxwlxowmfrom
stack/feat-container-runner-drain-engine-sleeps-unless-the-idle-timer-fired-ktlrlsuw
Open

feat(container-runner): drain engine sleeps unless the idle timer fired#5594
abcxff wants to merge 1 commit into
stack/feat-container-runner-jitter-idle-timeout-to-avoid-teardown-waves-wxwlxowmfrom
stack/feat-container-runner-drain-engine-sleeps-unless-the-idle-timer-fired-ktlrlsuw

Conversation

@abcxff

@abcxff abcxff commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@claude

claude Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Review

Small, focused change to container-runner/src/actor.rs: it fixes the prior bug where on_sleep skipped the drain for any sleep as long as idle-timeout mode was configured (dashboard/crash-policy/eviction sleeps included), not just idle-timer-triggered sleeps. Adding IDLE_SLEEPING as an explicit third state and gating the drain-skip on that specific state is the right fix, and the updated doc comments accurately describe the new behavior.

Correctness

  • The CAS-based state transition is a genuine improvement over the old plain load()-then-sleep() check — the old code had a TOCTOU gap between reading idle_state and calling ctx.sleep(); the new compare_exchange(IDLE_ARMED, IDLE_SLEEPING, ...) makes the "did a request race the timer" decision atomic.
  • Residual race worth being aware of (not introduced by this PR, but not fully closed either): if a request arrives after the idle timer's CAS has already flipped ARMED → SLEEPING and called ctx.sleep(), but before the engine actually delivers on_sleep, note_request's CAS (ARMED → REQUESTED) will fail silently and the state stays SLEEPING. When on_sleep eventually fires, it will still skip the drain even though a request came in during that window. This is a narrow timing window and the one-shot atomic can't represent "committed to sleep, but now there's also a pending request," so there's no cheap fix within this design — just flagging it as a known edge case rather than a blocker, since it's strictly better than the previous behavior (which skipped the drain unconditionally).
  • Ordering::SeqCst is used for both CAS's, which is safe/conservative for a rarely-hit lifecycle transition; no performance concern here.

Test coverage

  • No unit/integration test exercises the new IDLE_SLEEPING transition or on_sleep's branch on it (container-runner/tests/ only has boot_id.rs and input.rs). Given this is a state-machine correctness fix for a real production bug (dropping in-flight drains), a test that arms the idle timer, lets it fire, and asserts on_sleep skips the drain — plus a companion test asserting a non-idle sleep still drains — would give good regression coverage. Understand container-runner may not have much existing test scaffolding for actor lifecycle, so this may be consistent with the surrounding code, but worth considering.

Style

  • Comments follow the repo's "complete sentences, no dash fragments" convention. No _ => fallthrough issues (no enum match involved). Doc comments were updated in sync with the code, which is appreciated.

Overall this is a solid, minimal bug fix. No blocking issues found; the residual race and missing test coverage are the two things worth a second look before merging.

@abcxff
abcxff force-pushed the stack/feat-container-runner-jitter-idle-timeout-to-avoid-teardown-waves-wxwlxowm branch from c320933 to ddf4aad Compare August 25, 2026 19:36
@abcxff
abcxff force-pushed the stack/feat-container-runner-drain-engine-sleeps-unless-the-idle-timer-fired-ktlrlsuw branch from c95961f to 6a5be12 Compare August 25, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant