Skip to content

[E2E][Bug] fix flaky E2E order confirmation detection - #764

Open
kieran-osgood-shopify wants to merge 1 commit into
mainfrom
e2e-confirmation-wait
Open

kieran-osgood-shopify wants to merge 1 commit into
mainfrom
e2e-confirmation-wait

Conversation

@kieran-osgood-shopify

Copy link
Copy Markdown
Contributor

Why

The required E2E gate intermittently closed checkout before order completion and then failed because the host cart had not been cleared.

BrowserStack command logs showed the existing broad completion regex returning only 250 ms after Pay now, while the payment spinner was still visible. The test then tapped the checkout close button. Later passing runs using the exact same APK and journey waited 976 ms and 4.8 seconds for the real confirmation state before closing.

Fix

Match complete order-confirmation headings instead of arbitrary text containing Thank you, Your order, Order confirmed, or confirmation.

Both shared order journeys now accept only:

Thank you[, name][!]
Your order is confirmed
Order confirmed
Order confirmation

How to test

GIVEN the PR's Kotlin Android checkout-guest BrowserStack journey has submitted payment

WHEN the reviewer watches the journey recording from the Pay now action through checkout closing

THEN checkout should remain open while payment is processing and close only after the order-confirmation heading appears

AND the host app should show an empty cart after checkout closes


GIVEN the PR's hardcoded-buyer-identity BrowserStack journey has submitted payment

WHEN the reviewer watches the journey recording from the final submit action through checkout closing

THEN checkout should remain open until a complete order-confirmation heading appears

AND the host app should show an empty cart after checkout closes

Evidence

Assisted-By: devx/c44b22ed-e92b-422a-8245-d34957f21f33
@kieran-osgood-shopify
kieran-osgood-shopify requested a review from a team as a code owner September 11, 2026 14:53
@github-actions github-actions Bot added the #gsd:50662 Rebase Checkout Kit on UCP label Sep 11, 2026
@bitrise

bitrise Bot commented Sep 11, 2026

Copy link
Copy Markdown

Install this build

Open Tophat, select your target device, then click Install. Links open on the Mac running Tophat.

SDK Install
React Native Install with Tophat
Swift Install with Tophat
Kotlin Install with Tophat

Checkout Kit E2E results

Status Tags Target Platform OS version tag Device
launch, checkout-presentation, checkout-completion, buyer-identity react-native ios latest iPhone 15
iOS 27 Beta
launch, checkout-presentation, checkout-completion, buyer-identity react-native android latest Google Pixel 9
Android 17.0
launch, checkout-presentation, checkout-completion, buyer-identity, preload kotlin android latest Google Pixel 9
Android 17.0
launch, checkout-presentation, checkout-completion, buyer-identity, preload swift ios latest iPhone 15
iOS 27 Beta

@kieran-osgood-shopify kieran-osgood-shopify changed the title Fix flaky E2E order confirmation detection [E2E]: fix flaky E2E order confirmation detection Sep 14, 2026
@kieran-osgood-shopify kieran-osgood-shopify changed the title [E2E]: fix flaky E2E order confirmation detection [E2E][Bug] fix flaky E2E order confirmation detection Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

#gsd:50662 Rebase Checkout Kit on UCP

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant