Skip to content

feat(health): add richer health checks payload - #99

Open
Prateep07 wants to merge 3 commits into
nitrocloudofficial:developfrom
Prateep07:feature/health-only-71
Open

Prateep07 wants to merge 3 commits into
nitrocloudofficial:developfrom
Prateep07:feature/health-only-71

Conversation

@Prateep07

Copy link
Copy Markdown
Collaborator

Summary
Adds a detailed health://checks resource with overall status, per-check results and latency, uptime, SDK version, timestamp, and transform telemetry. Checks run concurrently with configurable timeouts. The existing health://status response shape remains compatible.

closes #71

Verification

  • Health-focused tests: 14 passed
  • Full suite: 832 passed, 3 environment-related failures (missing bdist_wheel; regex worker permission errors)

@Prateep07
Prateep07 requested a review from ROHIT-JR October 29, 2026 05:34
@AadarshM07 AadarshM07 linked an issue Oct 29, 2026 that may be closed by this pull request
3 tasks

@ROHIT-JR ROHIT-JR left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the concurrent checks, per-check timeouts and the unchanged health://status shape are good. Two things need to change before merge.

Changes needed

  1. A hung sync check blocks shutdown. Sync checks now run through asyncio.to_thread. I tried a check that does time.sleep(3) with timeout=0.1. The payload correctly reports "Timed out after 0.1s", but asyncio.run(...) still took 3.0s to finish because it waits for the worker thread. A check that stays hung will also use up the default thread pool as it is polled. Running sync checks in a thread is also a behaviour change for checks that need the main thread (for example a thread-bound DB connection), and the README does not mention it. Please either keep sync checks on the loop (documenting that only async checks are timed out) or run them in a dedicated bounded executor and document the limit.
  2. Timeouts are found by string match. The aggregate uses message.startswith("Timed out") to tell a timeout from a false result. Use a distinct per-check status (for example timeout) so the message text can change safely.

Smaller points

  • ServerHealthPayload declares transformsPipeline, sandboxWorkerPool and spillover, and nothing fills them in. Remove them or populate them.
  • transforms[].details is always {}, so the "transform telemetry" is only a name list for now. Say so in the README or add real details.
  • importlib.metadata.version runs on every request, and the fallback is server_config.version (the app's version, not the SDK's). Compute it once and fall back to the SDK's __version__.
  • error checks return str(exc) to any client that can read the resource. This is already true for health://status, but consider a generic message here.
  • assert elapsed < 0.3 in the timeout test may be flaky on a loaded CI runner.

@Prateep07
Prateep07 requested a review from ROHIT-JR October 29, 2026 12:17

@ROHIT-JR ROHIT-JR left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, both blockers are fixed. Approving.

I re-ran my hung-check script (time.sleep(3), timeout=0.1) on the new head. asyncio.run now returns in 0.1s (it was 3.0s), the check reports status timeout, the other checks are unaffected, and tests/test_health_async_checks.py passes (7). The distinct timeout status, the generic error message on health://checks, the dead payload fields removed, the SDK version resolved once, and the README note about thread-bound resources all look good. CI is green on 3.10 to 3.12.

Two small, non-blocking notes:

  • The pool threads are non-daemon, so a check that never returns still delays process exit until it finishes (my script took 3.7s in total). That is much better than before and is a reasonable trade-off, but it is worth one line in the README.
  • The __version__ fallback is hard-coded to "0.3.2" and will drift. Reading it from pyproject.toml, or reporting "unknown", would be safer.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[health] Richer health payload, aligned with health://checks

2 participants