test: make the iOS simulator suite survive a slow openurl - #361
Open
rexxars wants to merge 1 commit into
Open
Conversation
`simctl openurl` has to hand the URL to Safari and wait for it to accept it, and gives up after about ten seconds. On a CI runner - where booting the simulator alone takes two minutes - Safari's first launch does not fit in that window, and the run fails with NSPOSIXErrorDomain code 60 on a device that is perfectly healthy. It passed on the previous run, so it is a race, not a break. Launch Safari as part of booting, so the first launch is paid where nothing is racing a timeout, and retry `openurl` if it still loses. The retry restarts Safari first: a timed-out `openurl` may still have navigated, and a second tester page would otherwise connect under the same session id.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The iOS job went red on
mainwithFailed to open ... Operation timed out(NSPOSIXErrorDomain code 60) out ofxcrun simctl openurl, on a simulator that had booted fine a minute earlier. The run before it, same code, passed - so this is a race rather than something being broken.openurlhands the URL to Safari and waits for it to accept it, and it gives up after roughly ten seconds. On a runner that needs two minutes just to boot the simulator, Safari cold-starting does not fit in that window. Locallyopenurlreturns in well under a second, which is why this only turns up in CI.So two changes:
bootSimulator()now launches Safari once the device has finished booting. That pays the first-launch cost inside the boot step, where nothing is timing it, and it is the reason the pre-boot step exists in the first place.openPageretriesopenurla couple of times if it still loses the race, terminating Safari before each retry. Restarting rather than just asking again is deliberate: a timed-outopenurlmay still have navigated, and a second tester page would then connect under the same session id.What to review
The retry is the part I would look at. I could not reproduce the timeout locally, so I exercised the retry path by injecting a failure on the first attempt and confirmed the suite still ran, then reverted that. The three runs on this branch all passed without needing the retry at all, meaning the warm launch was enough each time - which is good news for the fix but does leave the retry itself only covered by the injected failure.
If it ever comes back, the log now says
Opening it failed (attempt N of 3), so we will know whether the retry saved it or whether ten seconds is simply not the real constraint.