Skip to content

tests: remove the subprocess time limits of CI tests and checks (EN-116) - #201

Draft
luisleo526 wants to merge 2 commits into
mainfrom
lv/en116-codegen-no-time-limits
Draft

luisleo526 wants to merge 2 commits into
mainfrom
lv/en116-codegen-no-time-limits

Conversation

@luisleo526

Copy link
Copy Markdown
Contributor

tests: a slow but correct compiler must pass the compile helper (red first, EN-116)

Owner rule (EN-105 G9, EN-116): slowness never fails a required check. tests/_compile.py
compile_cpp() kills the compiler after 60 seconds and the test fails with
subprocess.TimeoutExpired. This test runs the helper in a fresh interpreter with CXX set to
a wrapper that sleeps 62 seconds and then runs the real compiler with the same arguments,
and requires exit 0, the success marker and no timeout text. It takes about 65 seconds on
purpose. The test adds no time bound of its own and changes no production or helper code.

On this commit it FAILS (the helper raises TimeoutExpired at 60 seconds); the next commit
removes the bounds and it passes.

Remove every time limit that can fail a slow but correct codegen check (EN-116)

Owner rule (EN-105 G9, EN-116): slowness never fails a required check. The only hang catcher
is the job envelope at the platform maximum, and an expiry there is a HANG (re-run), not a
test failure. Exit codes, results and diagnostics are still checked.

  • 45 subprocess/spawn time limits removed (the timeout argument only; the call, its
    arguments and every result check stay): tests/_compile.py (4), tests/_e2e.py (4),
    tests/_requests.py (1), 19 test files (31 sites: build 60-180 s, run 10-30 s), scripts/gen_host_members.py (3),
    .github/pineforge/src/codegen-docs-rule.mjs (the 10 s git timeout).
  • 6 job-level timeout-minutes in .github/workflows/gate.yml (4) and test.yml (2) set from 10/30
    to 360, the platform maximum.
  • .github/workflows/promote-baseline.yml: the Cloud SQL proxy readiness loop keeps its
    one-second poll interval and waits until the proxy answers; the 30-poll deadline and its
    failure exit are removed (the job envelope is the only hang catcher). It fails at once, with a
    plain message, if the proxy process has exited before it became ready (a liveness check, not a
    time limit).

Not touched: the advertised 120 s transpile guard (pineforge_codegen/limits.py, README.md:483),
the tests that assert a product time budget or speed (tests/test_input_safety.py,
test_array_history.py:752, test_e2e_security_nested_helper_call.py:79, test_diagnostic_codes.py:257)
and the timeoutMs recorded in fixture metadata.

The previous commit adds tests/test_no_time_limit_on_slow_compiler.py, which fails without
this change (a 62 second compiler raises TimeoutExpired in tests/_compile.py) and passes with it.

Notes for review:

  • Untouched on purpose: the advertised 120 s transpile guard (pineforge_codegen/limits.py:134, a product guard), the measurement timings (class C), and fixture metadata.
  • .github/workflows/promote-baseline.yml: the Cloud SQL proxy readiness loop no longer gives up after 30 polls, but it fails at once if the proxy process has exited.
  • Job-level timeout-minutes are set to the platform maximum (360); a job stopped there is a hang to re-run.
  • Proof: red rj-20261012t173016-e42ea3 on the test-only commit (exactly the new test fails with TimeoutExpired); green rj-20261012t173226-63e80c on the removal (pytest 9906 passed / 36 skipped, compile-corpus 314, gate and pyodide parity OK over 279 fixtures). The final head differs from the green commit only by the promote-baseline.yml hunk, which hosted CI covers.

🤖 Generated with Claude Code

luisleo526 and others added 2 commits October 13, 2026 07:00
…first, EN-116)

Owner rule (EN-105 G9, EN-116): slowness never fails a required check. tests/_compile.py
compile_cpp() kills the compiler after 60 seconds and the test fails with
subprocess.TimeoutExpired. This test runs the helper in a fresh interpreter with CXX set to
a wrapper that sleeps 62 seconds and then runs the real compiler with the same arguments,
and requires exit 0, the success marker and no timeout text. It takes about 65 seconds on
purpose. The test adds no time bound of its own and changes no production or helper code.

On this commit it FAILS (the helper raises TimeoutExpired at 60 seconds); the next commit
removes the bounds and it passes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…k (EN-116)

Owner rule (EN-105 G9, EN-116): slowness never fails a required check. The only hang catcher
is the job envelope at the platform maximum, and an expiry there is a HANG (re-run), not a
test failure. Exit codes, results and diagnostics are still checked.

* 45 subprocess/spawn time limits removed (the `timeout` argument only; the call, its
  arguments and every result check stay): tests/_compile.py (4), tests/_e2e.py (4),
  tests/_requests.py (1), 19 test files (31 sites: build 60-180 s, run 10-30 s), scripts/gen_host_members.py (3),
  .github/pineforge/src/codegen-docs-rule.mjs (the 10 s git timeout).
* 6 job-level `timeout-minutes` in .github/workflows/gate.yml (4) and test.yml (2) set from 10/30
  to 360, the platform maximum.
* .github/workflows/promote-baseline.yml: the Cloud SQL proxy readiness loop keeps its
  one-second poll interval and waits until the proxy answers; the 30-poll deadline and its
  failure exit are removed (the job envelope is the only hang catcher). It fails at once, with a
  plain message, if the proxy process has exited before it became ready (a liveness check, not a
  time limit).

Not touched: the advertised 120 s transpile guard (pineforge_codegen/limits.py, README.md:483),
the tests that assert a product time budget or speed (tests/test_input_safety.py,
test_array_history.py:752, test_e2e_security_nested_helper_call.py:79, test_diagnostic_codes.py:257)
and the timeoutMs recorded in fixture metadata.

The previous commit adds tests/test_no_time_limit_on_slow_compiler.py, which fails without
this change (a 62 second compiler raises TimeoutExpired in tests/_compile.py) and passes with it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@luisleo526
luisleo526 force-pushed the lv/en116-codegen-no-time-limits branch from 341a67c to b7495e7 Compare October 12, 2026 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant