Skip to content

[Bug]: required test-shard is red on main - the interrupted-native guard rejects the default-off context_delivery command #7402

Closed

Description

@JasonBuildAI

Preflight

  • I searched existing issues and discussions for this behavior.
  • This report is not an unpatched security vulnerability.
  • The report and attachments contain no credentials or private/internal material.

Issue origin

Observed or reproduced in a real environment.

LoopX version or commit

cd819fd206cd9de46a789df0669e13bb2ed763e5 (current main).

Host or runtime surface

Not host-specific. The failing test is a Python test collected by the required test-shard jobs.

LoopX area

Build, packaging, installer, or CI

Problem

The required test-shard jobs of Python Tests are red on current main: benchmark/tests/test_native_checkpoint.py::test_interrupted_cursor_requires_exact_unreleased_final_guard fails for every parameter. One failing test fails its shard, pytest requires every shard, and merge-gate therefore cannot pass for any backend PR. At 0e47330fe (the last completed main Python Tests run, workflow run 40281869323) test-shard (2) and test-shard (6) both failed on this test while the rest of those shards passed; the same red lane is inherited by every open PR.

Why this is especially important. This is a required merge-gate lane, not an advisory smoke: every contributor's Python Tests run inherits an unrelated red check, and maintainers cannot separate an inherited failure from a real regression in a PR. The defect is deterministic (9 of 9 parameter combinations), it is on the default path, and it reproduces on unmodified main.

Root cause. loopx/hosts/codex_checkpoint.py::_verify_interrupted_guard binds a captured native command to the held packet by comparing every parsed option. It expects context_delivery == "off" (guard.get("context_delivery", "off")), but commit d893f7dd1 removed the --context-delivery CLI default so that an omitted option parses as None (loopx/cli_commands/quota_registration.py now deliberately inherits the machine setting). The heartbeat renderer only spells --context-delivery for an enabled mode (loopx/control_plane/heartbeat/builder.py), so a default-off interrupted capture always arrives with the option omitted and is rejected with native pending guard differs from the held packet routes or identity.

The same mismatch is on the shipped benchmark wake path: benchmark/runtime/checkpoint_wake.py records guard["context_delivery"] = env.get("LOOPX_CONTEXT_DELIVERY", "off"), then the captured default-off command parses None, so an interrupted native checkpoint cannot be confirmed/recovered. The guard is added by 8cd5d5853; the CLI default was removed by d893f7dd1 without updating the guard.

Minimal reproduction

On current main (Linux, because the test module imports the POSIX-only benchmark controller):

uv sync --extra test
uv run --extra test python -m pytest -q \
  "benchmark/tests/test_native_checkpoint.py::test_interrupted_cursor_requires_exact_unreleased_final_guard"

The underlying comparison is platform-independent and reproduces directly:

uv run --extra test python -c "
import shlex
from loopx.hosts.codex_checkpoint import _verify_interrupted_guard
guard = {'cli': '/fixture/loopx', 'workspace': '/tmp/fixture', 'registry': '/fixture/registry.json',
         'runtime_root': '/fixture/runtime', 'goal_id': 'research', 'agent_id': 'worker',
         'turn_instance_id': 'turn-1', 'capture_root': '/fixture/decisions',
         'socket_path': '/tmp/fixture/socket', 'decision_sha256': 'a'*64}
argv = [guard['cli'], '--registry', guard['registry'], '--runtime-root', guard['runtime_root'],
        '--format', 'json', 'quota', 'should-run', '--goal-id', 'research', '--agent-id', 'worker',
        '--turn-instance-id', 'turn-1', '--turn-envelope', '--decision-output-root', guard['capture_root'],
        '--decision-handoff-socket', guard['socket_path']]
item = {'type': 'commandExecution', 'id': 'guard-call', 'cwd': '/tmp/fixture', 'status': 'failed',
        'aggregatedOutput': None, 'command': shlex.join(['/bin/bash', '-lc', shlex.join(argv)])}
turns = [{'id': 'retained-compaction', 'status': 'completed', 'items': [{'type': 'contextCompaction'}]},
         {'id': 'pending', 'status': 'interrupted', 'items': [item]}]
_verify_interrupted_guard(turns, guard)
"

Expected behavior

A rendered default-off command (option omitted) is the off mode and binds the held packet. Explicit --context-delivery off and --context-delivery on continue to bind only their matching modes.

Actual behavior

ValueError: native pending guard differs from the held packet routes or identity

raised from loopx/hosts/codex_checkpoint.py:124 for every parameter.

Sanitized diagnostics

FAILED benchmark/tests/test_native_checkpoint.py::test_interrupted_cursor_requires_exact_unreleased_final_guard[None-ordinary]
FAILED benchmark/tests/test_native_checkpoint.py::test_interrupted_cursor_requires_exact_unreleased_final_guard[codex_context_root-ordinary]
...
E ValueError: native pending guard differs from the held packet routes or identity
loopx/hosts/codex_checkpoint.py:124: ValueError

CI evidence at 0e47330fe:

Additional context

Activity

  1. JasonBuildAI commented on Oct 26, 2026

    @JasonBuildAI
    ContributorAuthor

    Closing as already carried by open PR #7334 (opened 2026-10-25T17:52Z, still open), which includes the same normalization in loopx/hosts/codex_checkpoint.py: an omitted --context-delivery is mapped to off before the held-packet comparison. No separate change is needed from this report.

  2. JasonBuildAI commented on Oct 26, 2026

    @JasonBuildAI
    ContributorAuthor

    AI disclosure: this report was drafted with AI assistance at JasonBuildAI's request, under their direction and review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions