Skip to content

Release load_regulation slots of in_use conns when a pool stops#892

Merged
benoitc merged 1 commit into
masterfrom
fix/pool-stop-releases-inuse-slots
Jun 18, 2026
Merged

Release load_regulation slots of in_use conns when a pool stops#892
benoitc merged 1 commit into
masterfrom
fix/pool-stop-releases-inuse-slots

Conversation

@benoitc

@benoitc benoitc commented Jun 18, 2026

Copy link
Copy Markdown
Owner

Stopping a pool while requests were in flight leaked the global per-host load_regulation slots of the checked-out connections. Those slots are the hard per-host concurrency cap, shared across pools by {host, port}, so the leak starved that host node-wide and later requests timed out at checkout.

Two causes:

  • hackney_pool:init/2 did not trap exits, so stop_poolsupervisor:terminate_child (shutdown=10000) killed the pool directly and terminate/2 never ran.
  • terminate/2 only stopped available/h2/h3 connections, never the in_use ones, so even when it did run it never released their slots.

Fix: trap exits in init/2 so terminate/2 runs on shutdown (the catch-all handle_info already tolerates stray EXIT messages), and in terminate/2 release each in_use connection's slot and stop it.

This was also the cause of the intermittent test_checkout_timeout CI failure: the prior test closes its conns asynchronously then calls stop_pool on the same host, and when stop_pool won the race the leaked slot made the next max_per_host=1 request time out. Added test_stop_pool_releases_inuse_slots, a delta-based regression test asserting the per-host count returns to baseline after a pool is stopped with a request still checked out.

Stopping a pool while requests were in flight leaked the global per-host
load_regulation slots of the checked-out connections, starving that host's
concurrency cap node-wide so later requests timed out at checkout.

Two causes: the pool did not trap exits, so stop_pool (supervisor:terminate_child)
killed it without running terminate/2; and terminate/2 only stopped available/h2/h3
connections, never the in_use ones. Trap exits so terminate/2 runs on shutdown, and
release each in_use connection's slot (and stop it) there.

Add a regression test that checks the per-host count returns to baseline after a
pool is stopped with a request still checked out.
@benoitc benoitc merged commit 9d5c302 into master Jun 18, 2026
6 checks passed
benoitc added a commit that referenced this pull request Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant