test: tolerate transient gateway 504 on live premium lane - #133
Merged
Conversation
Live-keyed lane went red on push to main: TestPremium.test_premium_target_market got HTTP 504 from prod (gateway timeout, not a client ReadTimeout — the 30s DATAMAXI_TIMEOUT from #130 can't help). The premium endpoint intermittently 504s under load; infra flakiness, not an SDK bug. - conftest: add live_call(fn) helper — retries transient 5xx (502/503/504) with linear backoff, pytest.skip on persistent transient; real 5xx/4xx and assertions propagate unchanged. - test_integration: route all TestPremium data calls through live_call. keyless CI lane: 134 passed, 11 skipped (unchanged). helper logic unit-verified.
martinkersner
added a commit
that referenced
this pull request
Jul 2, 2026
Live keyed lane went red again on a transient prod 504 — this time test_cex_candle (ServerError 504 "upstream request timeout"). #133's per-endpoint live_call wrapping only covered TestPremium, so the same infra flakiness resurfaced on the next unwrapped call. The run still showed green: job-level continue-on-error masked the failed job. - conftest: replace per-call live_call helper with autouse fixture _tolerate_transient_gateway — monkeypatches API.send_request (the one method every endpoint inherits) to retry transient 502/503/504 then pytest.skip. Covers test_call.py + test_integration.py + future live tests. No-op without a key (keyless/mocked lanes untouched). - test_integration: revert the 19 live_call(lambda: ...) premium wraps to direct calls; drop now-dead import. - live-tests.yml: move continue-on-error from job to pytest step so the job conclusion stays honest (setup failures still red) while the push stays non-blocking; add a step that annotates + writes a run-summary warning when the live suite fails, so failures are visible not silent. keyless offline suite unchanged: 134 passed, 11 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The non-blocking Live keyed tests lane went red on the last push to
main(run 28571765425):TestPremium.test_premium_target_marketfailed with an HTTP 504 Gateway Timeout from prod.A 504 is a gateway timeout — the edge gave up on a slow upstream. It is not a client
ReadTimeout, so theDATAMAXI_TIMEOUT10s→30s bump from #130 cannot prevent it. The premium endpoint intermittently 504s under load; this is prod infra flakiness, not an SDK bug.Change
tests/conftest.py: addlive_call(fn, retries=3, backoff=1.0)— retries transient 5xx (502/503/504) with linear backoff, andpytest.skips if every attempt still hits a transient status (the non-blocking live lane must not go red on infra flakiness). Real 5xx (e.g. 500), 4xx, and assertion failures propagate unchanged.tests/test_integration.py: route allTestPremiumpremium data calls throughlive_call. The fastpremium.exchanges()list call is left as-is.Tests
-m "not integration"): 134 passed, 11 skipped, 107 deselected — unchanged.live_calllogic unit-verified out of band: retry-then-succeed, persistent-transient→skip, non-transient 500→propagate, 4xx→propagate.Known failures / notes
main; it cannot run on this PR branch. If prod is healthy at merge time the premium tests pass; if it 504s they now skip instead of failing.