Skip to content

fix(destroy): freeze a group whose member ends between two reads before destroy gives up on it - #333

Merged
ThomasK33 merged 5 commits into
mainfrom
fix/destroy-freeze-unreadable-member
Nov 5, 2026
Merged

ThomasK33 merged 5 commits into
mainfrom
fix/destroy-freeze-unreadable-member

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Why

The CI of #331 (head 5636b14, job 126789316550) failed "gc keeps an exited session while a fork chain that the program left runs" in all 3 attempts. Two attempts timed out with HOST_TIMEOUT. In the third, destroy returned ok and recorded destroyed, but the fork chain still ran 1 s later. In the same job, "destroy ends a fork chain that no scan of /proc shows" failed once before its retry passed. A plain destroy that reports success while the program's processes run is broken core behavior.

Root cause: stopProgramGroup sent SIGSTOP only after a judgement of gone. A fork-chain member lives about 2 ms, and a scan can show it while it ends:

  1. The member ends between its stat read and its autogroup read. readAutogroup gives null, so the judgement reads unknown, and stopProgramGroup returned with no freeze and no kill.
  2. The judgement reads owned, but the witness ends before it is read again. The loop then judged again without a freeze, and five such judgements in a row ended with no kill.

The wait after it then read the group ended: the probe found the group, but no scan showed a member (groupHasLiveProcess gives null), so destroy recorded destroyed. The model assumes that the freeze always brings the witness into view, and the code did not freeze in these two cases.

What changes

stopProgramGroup now freezes the group after every judgement that gets no kill, except unproven: gone as before, unknown while a member that shows no autogroup still runs, and owned whose witness ended before it was read again. The unknown verdict carries running for that. A stopped member cannot end, so the next judgement reads its autogroup and its witness, and the group gets the kill. The loop keeps its 5 attempts.

Safety argument

  • A freeze after unknown or after a failed witness read targets only a group id that a gone freeze already targets today. Such a judgement saw a member with the program's session id, or one that carries the token, so the group is at least as likely the program's as one that reads gone.
  • A group that reads unproven (another boot id, or members with another session id or autogroup) still gets no SIGSTOP. It returns before the freeze, as before.
  • No SIGSTOP goes out where /proc/self does not name this process (another pid namespace), as before.
  • The finally block still sends SIGCONT to every frozen group that got no kill, also when a scan throws.
  • A group that job control stopped stays stopped: a stopped member cannot end between two reads, so an unknown whose members without an autogroup are all stopped sends no signal (Codex round 1). A refused boot id reads unknown with running false too, so it sends no signal, as before.
  • A running member whose autogroup /proc refuses gets one SIGSTOP and one SIGCONT, and still no kill: after the SIGSTOP its member is stopped and reads the same way.

A SIGSTOP takes effect only when the kernel next runs the member: about 30 us on an idle machine (max 405 us here), longer under load. stopProgramGroup waits STOP_SETTLE_MS (1 ms, a synchronous Atomics.wait) after each SIGSTOP, so the next judgement mostly reads the frozen members stopped. A member that still runs then gets another SIGSTOP, within the 5 judgements. (The merge-group CI of 738daee's parent saw a real group get a second SIGSTOP. With one CPU shared with four busy loops, the second SIGSTOP came in 6 of 24 runs without the wait and in 1 of 24 with it.)

No CLI JSON or exit codes change.

Validation

  • Red first, in test/unit/host/programGroup.test.ts with a faked /proc:

    • "freezes a group whose member ends before its autogroup read, and kills it once a stopped member carries the token"
    • "freezes a group whose witness ends before it is read again, and kills it once a stopped member carries the token"
    • "sends no signal to a stopped group whose member shows no autogroup" (fails on fd2bb66, the first head of this PR).
    • The first two fail on main (no signal at all). The two tests "gets no kill and reads alive when the autogroup of its member names no autogroup / cannot be read" (a real group) now expect one SIGSTOP and one SIGCONT, and still no SIGKILL. On main they assert that the group gets no signal.
  • facts.mjs reads the new loop statement by statement:

    • Main's loop, where only gone freezes, turns ProvesGroupOwnership and FreezesUnseenGroup FALSE.
    • A return on unknown turns both FALSE.
    • A missing freeze after a failed witness read turns FreezesUnseenGroup FALSE.
    • The model is unchanged, and every guard constant still reads TRUE.
  • "gets no kill and reads alive when the autogroup of its member names no autogroup / cannot be read" accepts 1 to 5 SIGSTOPs before the SIGCONT, because the stop is asynchronous.

  • Real CLI, the scenario of the failed test (an exited session whose fork chain runs, then gc, then a plain destroy). I used a debug copy of the code that is not committed here. It adds a 5 ms delay inside the destroy CLI between the stat read and the autogroup read of each process (DBG_AUTOGROUP_DELAY_MS), or before the witness read (DBG_WITNESS_DELAY_MS), as a preemption on a loaded runner does. Results, 20 runs each:

    Delay Code Runs destroy recorded destroyed while the chain ran Runs with a judgement of unknown Runs with a witness that ended
    autogroup read, 5 ms main (11c0f17) 40 8 8 (no kill in any of them) 0
    autogroup read, 5 ms this PR (756e381) 40 0 8 (each got SIGSTOP, then SIGKILL) 0
    witness read, 5 ms main 20 0 0 2 (a later judgement read gone and froze)
    witness read, 5 ms this PR (756e381) 20 0 1 (killed) 7 (frozen, then killed)

    On main, every false destroyed follows a judgement of unknown with no freeze, and the wait's probe then found the group with no member in view. This matches CI attempt 3. Without the delays, under a CPU load of 2 cores and 10 busy loops, the traces on main showed owned judgements whose witness had ended by its read (2 in 16 runs).

  • tsc --noEmit, oxlint, oxfmt --check, check.sh --drift-only, test/unit (2,070 tests), the lifecycle fork-chain and thread integration tests, non-reaping-init, and other-user-group (with a setcap'd setpriv) pass.

Follow-up

A second PR changes the wait: when the probe finds the group but no scan shows a member, the group reads alive, as gc's GcProbesUnseenGroup already counts it. Then any future miss gives an honest HOST_TIMEOUT instead of a false destroyed. That PR changes the model (FreezesUnseenGroup, R11 DestroyedProgramStopped) and lands after this one.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh

…re destroy gives up on it

stopProgramGroup sent SIGSTOP only after a judgement of gone. A member of a
fork chain that a scan shows can end before its autogroup read, so the
judgement reads unknown, and stopProgramGroup returned with no freeze and no
kill. A witness that ended before its read again made the loop judge again
without a freeze. The wait after it then read the unseen chain ended, and
destroy recorded destroyed while the chain ran. CI of #331 hit this in "gc
keeps an exited session while a fork chain that the program left runs".

Now every judgement that gets no kill and is not unproven sends SIGSTOP and
judges the frozen group again: gone as before, unknown, and owned with a
witness that ended. A stopped member cannot end, so the next judgement reads
its autogroup and the witness, and the group gets the kill. The loop keeps
its 5 attempts, and a frozen group that gets no kill still gets SIGCONT in
the finally block.

New unit tests: a member that ends before its autogroup read, and a witness
that ends before its read again, both get SIGSTOP and then SIGKILL (both fail
on main). A member whose autogroup /proc refuses now gets 5 SIGSTOP and a
SIGCONT, and still no kill. facts.mjs reads the new loop. The model is
unchanged: it already assumes that the freeze brings the witness into view.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-05T00:40:32.144669Z 738daee Manual request
🔒 Security Review ✅ Completed 2026-11-05T00:42:23.855496Z 738daee Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fd2bb66074

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/host/programGroup.ts Resolved
@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: fd2bb66074

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…group

Codex round 1 on #333: a group that job control stopped, whose autogroup or
boot id /proc refuses, reads unknown at every judgement. The new freeze sent
it SIGSTOP and then SIGCONT, which resumed a program that its user stopped.

The unknown verdict now says whether a member without an autogroup still
runs. A stopped member cannot end between two reads, so /proc refused its
autogroup, and stopProgramGroup sends no signal then. A refused boot id is
unknown without a running member too, because no later judgement can prove
the group without it. A running member without an autogroup still gets the
freeze, once: its stopped member reads the same way, and the group gets
SIGCONT.

New unit test: a stopped group whose member shows no autogroup gets no
signal (fails on the previous head).
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 756e381418

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 756e381418

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…readable-member

# Conflicts:
#	.agents/skills/verify-session-lifecycle/coverage.json
@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 4, 2026
…readable-member

# Conflicts:
#	.agents/skills/verify-session-lifecycle/coverage.json
@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 5, 2026
…the group

The merge-group CI of #333 failed "gets no kill and reads alive when the
autogroup of its member names no autogroup": the real group got two
SIGSTOPs before its SIGCONT. A SIGSTOP takes effect only when the kernel
next runs the member. A read of /proc/<pid>/stat right after kill() showed
the member not yet stopped in 1,984 of 2,000 tries here, and the stop took
about 30 us (max 405 us) on an idle machine. A full /proc scan usually gives
it that time, but on a loaded runner the next judgement read the member
running and froze the group again. The code is safe either way: a member
that still runs gets another SIGSTOP, within the 5 judgements, and the
group still gets its SIGCONT.

stopProgramGroup now waits STOP_SETTLE_MS (1 ms) after each SIGSTOP, so the
next judgement mostly reads the frozen members stopped and fewer of the 5
judgements go to a stop that has not taken effect. On one CPU shared with
four busy loops, the test saw a second SIGSTOP in 6 of 24 runs without the
wait and in 1 of 24 with it. The test now accepts 1 to 5 SIGSTOPs before
the SIGCONT. facts.mjs reads the wait as part of the freeze.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 738daeeef2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 738daeeef2

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 5, 2026
Merged via the queue into main with commit 3b504b3 Nov 5, 2026
26 checks passed
@ThomasK33
ThomasK33 deleted the fix/destroy-freeze-unreadable-member branch November 5, 2026 01:50
@github-actions github-actions Bot mentioned this pull request Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant