Limit proxies list to 100 proxies by default - #1
Conversation
`proxies list` walked every page of the proxy list and buffered the whole thing before printing a single byte. On a residential backbone plan, where the list runs to millions of entries at ~25-60 per request, that is thousands of sequential round trips with no output and no progress — the command is indistinguishable from a hang. Default --limit to 100, matching `proxies replaced`. --limit 0 restores the previous fetch-everything behavior, and the command now points at `proxies download`, which renders a full list server-side in one request. When the limit cuts the list short, say so on stderr so the cap is never silent — stderr keeps the notice out of piped proxy lists.
The root command's examples redirect the proxy list to a file, which now needs --limit 0 to stay a complete list.
CI pins staticcheck@2026.1 (v0.7.0) but installs go-version: stable, which is now Go 1.27.1. staticcheck v0.7.0 cannot read Go 1.27's export data and fails on stdlib packages before it reaches any repo code: internal error in importing "math/bits" (cannot decode "math/bits", export data version 4 is greater than maximum supported version 2) This is unrelated to any code change — it reproduces on an unmodified main. 2026.2.1 supports the current toolchain and reports nothing on this repo.
Heads-up: this PR also unblocks CIThe first run here failed lint for a reason unrelated to the change: CI pins Verified pre-existing: running that exact staticcheck version against an unmodified Bumped the pin to Worth considering separately: the pinned linter against a floating |
The problem
webshare proxies list --mode=backbonelooks like it hangs. It doesn't — it is paginating the entire proxy list.Three things compound:
--limitdefaulted to0= "all", so the loop droveProxies.ListAllthrough every page via the envelope'snextURL, sequentially.Backbone makes it worse: only
modeandcountry_code__inwork as filters there (spec/notes/proxy.md), so most ways a user would try to narrow the list are silently ignored while still paying for the full walk.The change
--limitnow defaults to 100, matching the existing default onproxies replaced.--limit 0restores the old fetch-everything behavior.address:port:username:passwordstream.proxies downloadfor full exports: the server renders the whole list in a single request, which is the right tool for that job.--limit 0, so they keep producing complete lists.Behavior
Against a fake API serving a residential-sized list (6M entries, 60/page):
proxies list --mode=backboneproxies list --mode=backbone --limit 0Tests
Three new tests drive a multi-page fake API and assert on the number of pages actually fetched, not just the output:
--limit 0still fetches every pageVerified the cap is what's doing the work: with
defaultProxyListLimitset back to0,TestProxiesListStopsAtDefaultLimitnever completes.Breaking change
Scripts relying on
proxies listreturning the complete list need--limit 0(or, better,proxies download). This is worth a minor version bump — proposing v0.2.0.