What
Add a CI lane that runs the live keyed test suite (DATAMAXI_API_KEY=... python -m pytest tests/) on push to main only.
Why
.github/workflows/python-package.yml currently runs on every push but executes only the offline mocked lane (python -m pytest tests/ -m "not integration", keyless). The live lanes — tests/test_integration.py (integration marker) and tests/test_call.py (smoke lane) — hit prod endpoints and are deliberately skipped in CI (smoke skips cleanly without a key; integration is deselected). So real API/SDK regressions against prod are never caught in CI.
Scope / approach
- New job (or gated step) that sets
DATAMAXI_API_KEY from a repo secret and runs python -m pytest tests/ (full suite, incl. integration + smoke).
- Gate to
main only, e.g. if: github.ref == 'refs/heads/main' (or a separate workflow with on: push: branches: [main]) — do not run on every branch push (avoids leaking the key to fork/branch runs and burning prod quota).
- Requires adding a
DATAMAXI_API_KEY GitHub Actions secret (repo settings → Secrets).
Caveat — expect intermittent red
PR #130 removed the flaky-prod xfail markers, so the live lane is no longer masked and can go red intermittently through no code fault:
- Cold API pods →
funding_rate.* and naver.* return 500 "no data found" (NATS-warmed caches empty on cold start).
premium() on an empty page raises ValueError('no data found') instead of returning empty (client-side SDK gap; limit=10 reduces but doesn't eliminate the empty-window risk).
Because of this, make the live lane non-blocking for now — a scheduled/nightly run, or continue-on-error: true, rather than a required check — until the empty-result handling is normalized in the SDK. Note the DATAMAXI_TIMEOUT env knob (conftest, default 30s) for slow prod endpoints.
Done when
- A CI run on push to
main executes the full keyed suite against prod using the DATAMAXI_API_KEY secret.
- The lane is non-blocking (does not fail the required build on transient prod-data flakiness).
- Branch/fork pushes do not run the keyed lane (secret not exposed).
Related: #130
What
Add a CI lane that runs the live keyed test suite (
DATAMAXI_API_KEY=... python -m pytest tests/) on push tomainonly.Why
.github/workflows/python-package.ymlcurrently runs on every push but executes only the offline mocked lane (python -m pytest tests/ -m "not integration", keyless). The live lanes —tests/test_integration.py(integration marker) andtests/test_call.py(smoke lane) — hit prod endpoints and are deliberately skipped in CI (smoke skips cleanly without a key; integration is deselected). So real API/SDK regressions against prod are never caught in CI.Scope / approach
DATAMAXI_API_KEYfrom a repo secret and runspython -m pytest tests/(full suite, incl. integration + smoke).mainonly, e.g.if: github.ref == 'refs/heads/main'(or a separate workflow withon: push: branches: [main]) — do not run on every branch push (avoids leaking the key to fork/branch runs and burning prod quota).DATAMAXI_API_KEYGitHub Actions secret (repo settings → Secrets).Caveat — expect intermittent red
PR #130 removed the flaky-prod
xfailmarkers, so the live lane is no longer masked and can go red intermittently through no code fault:funding_rate.*andnaver.*return500 "no data found"(NATS-warmed caches empty on cold start).premium()on an empty page raisesValueError('no data found')instead of returning empty (client-side SDK gap;limit=10reduces but doesn't eliminate the empty-window risk).Because of this, make the live lane non-blocking for now — a scheduled/nightly run, or
continue-on-error: true, rather than a required check — until the empty-result handling is normalized in the SDK. Note theDATAMAXI_TIMEOUTenv knob (conftest, default 30s) for slow prod endpoints.Done when
mainexecutes the full keyed suite against prod using theDATAMAXI_API_KEYsecret.Related: #130