fix(ferndesk): bound + honour Retry-After on article write 429s (COR-444)

[echobt]

What

COR-444 residual, write side. #13 (b589269) fixed the infinite /collections pagination; this PR fixes the other half of the same ticket: the article write path.

POST /articles returns 429 / {"code":"rate_limited"} under load. The retry loop existed but:

• ignored Retry-After entirely,
• used one fixed ladder capped at 90s with no wall-clock bound,
• logged a generic rate/limit 429 with no article identity,
• and aborted the entire run on the first exhausted write, losing the SUMMARY.

Changes

scripts/ferndesk_sync.py

• Retry-After is honoured when the server sends it and it exceeds the exponential backoff. Both formats parse: delta-seconds and HTTP-date. Values over 300s are treated as unusable and fall back to the backoff ladder, so a hostile header cannot park CI.
• Backoff: 3s doubling, capped at 120s (was a fixed ladder capped at 90s).
• Bounded three ways so a rate-limited run always finishes and reports:

  Env	Default	Scope
  FERNDESK_WRITE_RETRIES	8	attempts per write
  FERNDESK_WRITE_DEADLINE	900	seconds per write
  FERNDESK_WRITE_BUDGET	3600	seconds of retry time per run (0 disables)

• Reads keep the lighter 6-attempt budget. Hard 4xx (400/401/404) still fails fast without retrying.
• Clear logs: each retry names the article slug, the status (HTTP 429 / cf1010/403 / transport), and why it waited (Retry-After vs backoff).
• One stuck page no longer aborts the run. The sync continues, logs a FAILURES line, records failed + failed_slugs in the SUMMARY, and exits 1. Nothing is cached for a failed write, so the next run retries that slug cleanly.

scripts/tests/ferndesk-sync-retry.test.py (new)

42 offline checks: faked transport, virtual clock, no network and no real waits. Covers Retry-After parsing (absent/blank/garbage/seconds/negative/HTTP-date), the backoff ladder and cap, the over-cap fallback, 429-then-success, exhausted retries, the per-write deadline, the run-wide budget, fail-fast on 400, the read budget, and an end-to-end run where a page stays rate-limited.

.github/workflows/ci.yml runs that test in the Docs site job. scripts/FERNDESK.md documents the knobs.

Test plan

node scripts/check-docs-site.mjs                      # ok (24 problem pages)
bash scripts/tests/check-docs-site.test.sh            # ok
python3 scripts/tests/ferndesk-sync-retry.test.py     # ok, 42 checks
npm exec --yes --package=mint@4.2.876 -- mint validate # build validation passed
FERNDESK_DRY_LOCAL=1 python3 scripts/ferndesk_sync.py  # 99 pages discovered

No live API write was attempted from this session: the fix is verified against a faked transport, and the real SUMMARY-green run needs the FERNDESK_API_KEY secret in Actions. This PR does not claim Comp Done.

Separate Comp / ops items (not blocking this PR)

1. Connect domain for docs.cortex.foundation in the FernDesk UI (HTTPS). Unrelated to the CF 1010 API path.
2. FERNDESK_DOCS_DISPATCH_TOKEN — the backend production-deploy hook that fires repository_dispatch: ferndesk-sync.

Both remain open Comp/ops work; the sync cannot report SUMMARY-green end to end until the secret is present and the rate limit is not saturating.

@greptileai

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
cortex-foundation-add13747	🟢 Ready	View Preview	Sep 14, 2026, 10:02 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

[echobt]

@greptileai review — rebased onto main (f2e9f08) and reconciled with #14: Retry-After handling, per-write deadline, run-wide write budget, and honest FAILURES reporting layered on top of #14's 5s→180s 429 ladder and create-conflict slug lookup + PATCH recovery.