Skip to content

feat: split oversized IN filters into multiple requests - #26

Open
ivasilov wants to merge 2 commits into
mainfrom
ivasilov/split-large-in-filter-urls
Open

ivasilov wants to merge 2 commits into
mainfrom
ivasilov/split-large-in-filter-urls

Conversation

@ivasilov

@ivasilov ivasilov commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

Large `inArray` filters (e.g. from lazy joins on on-demand collections) produced URLs past the Supabase gateway's ~8 KB limit and failed with 414 URI Too Long; reads now split the largest top-level `IN` list into chunks whose rendered URL fits a fixed 8000-character budget (`MAX_URL_LENGTH`, matching postgrest-js's `urlLengthLimit` default), fetch them with up to 6 in parallel (aborting siblings on the first failure), and concatenate the results under the same query key. Subsets that already fit send a byte-identical URL, `not in`/OR-nested `IN` are never split, limit/offset windows fetch a safe superset, and two huge `IN` lists are balanced by halving the next-largest one. The behaviour is documented in the README. Covered by unit tests for the splitter and the fan-out plus e2e tests showing the raw request 414s on the local stack while the same filter loads fully through a collection.

🤖 Generated with Claude Code

ivasilov and others added 2 commits September 26, 2026 21:43
Reads whose URL would exceed the gateway's ~8 KB request-line limit (414 URI
Too Long) now split their largest top-level IN list into chunks that each fit,
fetched with bounded concurrency and concatenated. Adds a `maxUrlLength`
collection option (defaults to the client's `urlLengthLimit`, 8000).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Replace the `maxUrlLength` collection option with a fixed
`MAX_URL_LENGTH = 8000`, matching postgrest-js's `urlLengthLimit` default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant