ROX-35909: Test_OutputQueue_ExpiringMessage fails in PubSub#21903
Conversation
📝 WalkthroughWalkthroughOutput queue tests replace timeout-based asynchronous waits with ChangesOutput queue test synchronization
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…eue_ForwardMessages The tests had a race condition in pubsub mode where async event processing might not complete before test cleanup, causing gomock to detect missing ReprocessDeployments calls. Add synchronization using done channel and .Do() callback on mock expectation, matching the pattern in Test_OutputQueue_DetectorCalls. Partially generated with AI assistance.
5410249 to
b371bb9
Compare
|
@coderabbitai full review |
✅ Action performedFull review finished. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #21903 +/- ##
==========================================
- Coverage 51.40% 51.36% -0.04%
==========================================
Files 2857 2857
Lines 178776 178776
==========================================
- Hits 91891 91837 -54
- Misses 78865 78898 +33
- Partials 8020 8041 +21
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
🚀 Build Images ReadyImages are ready for commit 4d2381e. To use with deploy scripts: export MAIN_IMAGE_TAG=4.12.x-583-g4d2381ef8f |
|
/retest |
vikin91
left a comment
There was a problem hiding this comment.
Good idea for the fix! I am proposing to go one step further (sorry, I am a big synctest fan 😉 ).
|
Okay, I've switched them to use synctest 👍 |
vikin91
left a comment
There was a problem hiding this comment.
Looks good! Thanks for improving!
You may want to update the PR description before merging as it still suggests that a solution with a channel is used.
Description
Fixes test flakes in
Test_OutputQueue_ExpiringMessagesandTest_OutputQueue_ForwardMessagesthat caused intermittent CI failures.Root Cause:
In pubsub mode, event processing is asynchronous (occurs in a separate goroutine via the lane's
run()method). The tests set a mock expectation, sendan event, validate the assertion, then immediately finish without waiting for async processing to complete. This creates a race: if the test completes
before the lane's goroutine processes the event, gomock cleanup detects the missing call and fails the test.
Fix:
Rewrite all output queue tests to use
testing/synctest, matching the pattern already established in other pubsub async tests (e.g.detector_serializer_test.go).synctest.Testprovides deterministic goroutine scheduling, andsynctest.Wait()ensures all spawned goroutines have settled before assertions run — eliminating both the race condition and wall-clock timeouts.Validation:
go test -count=200 -parallel=8(failed immediately)go test -count=20with zero flakes)Introduced in PR #20898 (ROX-34880: Refactor resolver to use PubSub)
🤖 This PR was created with AI assistance.
User-facing documentation
Testing and quality
Automated testing
How I validated my change
go test -count=200 -parallel=8failed withmissing call(s) to *mocks.MockDetector.ReprocessDeploymentsgo test -count=20passes with zero flakes