Repository navigation
test: fix flaky run() compaction test and hang-on-failure - #4
Merged
Merged
Conversation
The time-bucketed scheduling test polled restore() while run() was still performing its first sync. On a slow runner the replica was still empty, so restore() threw NoMatchingBackupError and the helper rethrew it instead of retrying. Because ac.abort() only ran on the success path, the failed test left run() looping forever and the process never exited, which hung the release workflow. - waitForRestoredRows retries transient restore failures until its deadline and reports the last error on timeout - always abort and await run() in finally (compaction and leasing tests) - release workflow: skip tests (CI covers them) and add a job timeout Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0162xGMZU6wtdADEwqhqNqnY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Release run 38384169229 hung in the Test step for ~18 minutes until it was cancelled. The log shows:
Two bugs combined:
replica.run(), writes for ~150 ms, then pollsrestore(). On a slow runner the first sync hadn't uploaded anything yet, sorestore()correctly threwNoMatchingBackupErrorandwaitForRestoredRowsrethrew it instead of retrying.ac.abort()only ran on the success path, so after the failure therun()loop kept going and the test process never exited. The release job had no timeout.Reproduced locally by delaying the first
writeLtxFileby 300 ms: sameNoMatchingBackupError, then the process hangs.Changes
waitForRestoredRows(compaction-scheduling and compaction-retention tests) retries transient restore failures until its deadline and includes the last error in the timeout message. The retention test also needs this because retention can delete files while a restore is reading them.run()tests always abort and await the run loop infinally, so a failed assertion fails fast instead of hanging.release.yml: drop the Test step (CI already runs the tests on every push tomain) and addtimeout-minutes: 15.Verification
pnpm lint,pnpm typecheck:allandpnpm test:allpass (119 + 18 tests).🤖 Generated with Claude Code
https://claude.ai/code/session_0162xGMZU6wtdADEwqhqNqnY