Skip to content

test(bigtable): add accelerator daemon startup and error handling system tests - #18790

Open
mutianf wants to merge 1 commit into
accel14from
accel15
Open

mutianf wants to merge 1 commit into
accel14from
accel15

Conversation

@mutianf

@mutianf mutianf commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Add Cloud Bigtable accelerator daemon startup fault-injection (test_daemon_startup.py) and end-to-end error handling / mid-flight crash fallback system tests (test_error_handling_async.py, test_error_handling_autogen.py).

Part of Stack #18691 (stacked on #18789).

@mutianf
mutianf requested a review from a team as a code owner October 20, 2026 20:02
@mutianf
mutianf added this pull request to stack #18691 October 20, 2026 20:02

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces system tests for the Bigtable Accelerator daemon, covering daemon startup behaviors and error handling scenarios (including fallback to native clients, midflight daemon termination, timeout enforcement, and error type matching) for both asynchronous and synchronous autogenerated paths. The feedback recommends wrapping daemon startup and assertions in try...finally blocks within test_daemon_startup.py to guarantee proper cleanup of daemon processes and prevent resource leaks in case of test failures.

Comment on lines +29 to +32
daemon = AcceleratorDaemon(_FLAGS, binary_path=path)
with pytest.raises(RuntimeError, match="spawn"):
_harness.call_with_timeout(daemon.start, timeout=5.0)
_harness.force_kill_daemon(daemon)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If an unexpected exception is raised or if the assertion fails, _harness.force_kill_daemon(daemon) will not be executed, potentially leaking the daemon process. Wrapping the startup and assertion in a try...finally block ensures that the daemon is always cleaned up properly, preventing resource leaks in the test environment.

Suggested change
daemon = AcceleratorDaemon(_FLAGS, binary_path=path)
with pytest.raises(RuntimeError, match="spawn"):
_harness.call_with_timeout(daemon.start, timeout=5.0)
_harness.force_kill_daemon(daemon)
daemon = AcceleratorDaemon(_FLAGS, binary_path=path)
try:
with pytest.raises(RuntimeError, match="spawn"):
_harness.call_with_timeout(daemon.start, timeout=5.0)
finally:
_harness.force_kill_daemon(daemon)

Comment on lines +38 to +44
daemon = AcceleratorDaemon(_FLAGS, binary_path=path)
started = time.monotonic()
with pytest.raises(RuntimeError, match="exited during startup"):
_harness.call_with_timeout(daemon.start, timeout=5.0)
elapsed = time.monotonic() - started
assert elapsed < 3.0, f"exit-during-startup took {elapsed:.2f}s; not prompt"
_harness.force_kill_daemon(daemon)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the previous test, if any assertion fails or an unexpected exception occurs, the daemon process will not be killed. Wrapping this block in a try...finally ensures reliable cleanup of the daemon process.

Suggested change
daemon = AcceleratorDaemon(_FLAGS, binary_path=path)
started = time.monotonic()
with pytest.raises(RuntimeError, match="exited during startup"):
_harness.call_with_timeout(daemon.start, timeout=5.0)
elapsed = time.monotonic() - started
assert elapsed < 3.0, f"exit-during-startup took {elapsed:.2f}s; not prompt"
_harness.force_kill_daemon(daemon)
daemon = AcceleratorDaemon(_FLAGS, binary_path=path)
started = time.monotonic()
try:
with pytest.raises(RuntimeError, match="exited during startup"):
_harness.call_with_timeout(daemon.start, timeout=5.0)
elapsed = time.monotonic() - started
assert elapsed < 3.0, f"exit-during-startup took {elapsed:.2f}s; not prompt"
finally:
_harness.force_kill_daemon(daemon)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant