Coalesce content exclusion fetches to stop exhausting the GitHub API rate limit

[lramos15]

Fixes #322275

The problem

A user with many git repos in one workspace (an AOSP checkout) hit API rate limit exceeded every few prompts. Their log shows 463 completed content exclusion refreshes and 16,374 failed batch requests in ~30 minutes.

RemoteContentExclusion.makeContentExclusionRequest let every waiting caller start its own refresh:

if (this._contentExclusionFetchPromise) {
    await this._contentExclusionFetchPromise;
}
// every waiting caller then continues here and starts another refresh

...and each refresh re-fetched every cached repo, because _contentExclusionRequest snapshotted the whole cache. Request volume therefore grew quadratically with repo count.

Changes

Request coalescing. Per-repo DeferredPromises, a 50ms batching window (measured from the first enqueue so a steady trickle can't starve it), and a Limiter(5) so batches run concurrently rather than serially. Each caller waits only on its own repos, so a large background load can't stall a single file check.

Measured by stashing the new implementation and running the new test against the original code:

	requests	repo-slots sent	unique repos
before	75	650	26
after	3	26	26

That's 26 repos; it's quadratic, so this is the ~16,000 → ~35 difference at the reporter's scale.

Two fail-open bugs. Both caused exclusions to silently stop applying during a rate-limit window:

• Empty placeholder rules were written on discovery and left in place on failure, so a failed fetch was indistinguishable from "no rules" — and shouldFetchContentExclusionRules then returned false, blocking any retry for 30 minutes. Rules are now cached only on a successful response.
• Even after that, a negative verdict was memoised into _ignoreGlobResultCache, so a file checked during an outage stayed allowed permanently. ensureRulesLoaded now reports whether rules actually loaded, and the verdict is only memoised when they did.

Shared rate-limit middleware. shared-fetch-utils already had authBlockedMiddleware (401/403), serverErrorBackoffMiddleware (5xx) and etagMiddleware, but nothing for 429 / Retry-After — a gap that had been filled by hand in three places. New rateLimitBackoffMiddleware covers 429 and quota-exhausted 403, honours Retry-After and x-ratelimit-remaining/x-ratelimit-reset, and caps any wait (including a server hint) at maxDelayMs so a bogus header can't stall the client.

RemoteContentExclusion now composes:

composeFetchMiddleware(
    authBlockedMiddleware(),              // bare 401/403 -> block token
    rateLimitBackoffMiddleware({ now }),  // 429 / quota 403 -> server-hinted wait
    serverErrorBackoffMiddleware(),       // 5xx -> exponential
)

Order is load-bearing and pinned by a test: rate-limit must sit inside auth-blocked, or authBlockedMiddleware throws on the 403 first and a rate limit gets misread as an auth failure, blocking the token for an hour instead of waiting for the actual reset.

CloudSessionApiClient migrated onto the same middleware. Removes _handleRateLimit, the dead write-only _rateLimitCount, and the isRateLimited() + _buildRequest + fetch + 429 preamble repeated across six methods, replaced by one _fetch helper. isRateLimited() and onRateLimited are preserved — the exporter polls the former synchronously to skip a flush cycle, so it stays as a projection of the middleware's window rather than a second implementation.

Deliberately not the full stack there: 403 means policy_blocked for that API (so no authBlockedMiddleware), and 5xx backoff is already owned by the exporter's CircuitBreaker (so no serverErrorBackoffMiddleware).

Performance. Glob patterns precompile into Minimatch instances behind a flat list instead of recompiling every pattern on every isIgnored call; isRegexContextExclusionsEnabled is a counter rather than a full cache scan; memo caches are only invalidated when rules that could change an outcome arrive; per-entry TTL replaces the blanket 30-minute refresh; malformed server patterns are skipped rather than throwing on every lookup.

Testing

288 tests pass across the touched areas; typecheck and eslint clean.

• 9 new RemoteContentExclusion tests: coalescing, incremental fetch, empty-ruleset caching, TTL expiry, failed-fetch retry, backoff, rate-limit reset, glob matching, malformed patterns.
• 7 new middleware tests, including that a plain 403 is not treated as a rate limit.
• 3 new CloudSessionApiClient tests. The pre-existing ones (429 -> rate_limited, 403 -> policy_blocked, 500 -> error) pass unmodified, which is the main evidence that migration is behavior-preserving.

Clocks are injected via optional constructor/option params defaulting to Date.now, so TTL and backoff are tested without stubbing globals.

Notes for reviewers

• Fail-open is preserved, not fixed. When rules can't be fetched, files are still treated as not excluded — this change makes that state transient and retryable rather than permanent. Fail-closed would be safer for content exclusion but would break Copilot entirely during a GitHub outage. Worth a separate discussion.
• etagMiddleware is intentionally omitted. GitHub 304s don't count against the rate limit, so it looks like a free win, but it holds a single etag slot per instance while our URL varies per batch (?repos=a,b,c) — it could serve batch A's rules for batch B. It needs to be keyed by request first.
• FetchedValue was not used. It models one endpoint → one value; content exclusion is a keyed, batched cache (N repos → ⌈N/10⌉ requests), and one FetchedValue per repo would make batching impossible.
• Follow-up: capiClientFetchedValue composes etag + authBlocked + serverErrorBackoff but not rate-limit, so every FetchedValue-backed CAPI endpoint still lacks 429 handling. One-line fix, but it affects every consumer, so I left it out of this PR.
• _ignoreGlobResultCache / _ignoreRegexResultCache remain unbounded. Real concern on a million-file workspace, but eviction changes hit-rate characteristics and deserves its own change.

[Copilot]

🟡 Not ready to approve

Cache freshness and in-flight coalescing defects can still bypass exclusions or exhaust the API quota.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Reduces GitHub API usage from content-exclusion refreshes and centralizes rate-limit handling.

Changes:

• Coalesces and concurrently limits content-exclusion requests.
• Adds shared rate-limit backoff middleware.
• Migrates cloud-session requests to the middleware.

File summaries

File	Description
rateLimitBackoffMiddleware.ts	Implements shared rate-limit backoff.
rateLimitBackoffMiddleware.spec.ts	Tests backoff behavior.
remoteContentExclusion.ts	Adds batching, caching, and compiled rules.
remoteContentExclusion.spec.ts	Tests coalescing, recovery, and matching.
mockCAPIClientService.ts	Expands content-exclusion test support.
cloudSessionApiClient.ts	Centralizes cloud request handling.
cloudSessionApiClient.spec.ts	Tests cloud rate-limit behavior.

Review details

Comments suppressed due to low confidence (1)

extensions/copilot/src/platform/ignore/node/remoteContentExclusion.ts:304

• After an expired entry fails to refresh, the stale cache entry still makes this return true. isIgnored then memoizes a negative verdict as though fresh rules loaded, causing future checks of that URI to skip retries and potentially bypass newly added exclusions. Recheck freshness after the waits, not just key presence.

		for (const url of required) {
			if (!this._contentExclusionCache.has(url)) {
				return false;
			}

• Files reviewed: 7/7 changed files
• Comments generated: 5
• Review effort level: Medium

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.