Repository navigation
Conversation
FakeServer listens on a free port and records every client message per connection. It speaks the Massive protocol (connected, auth_success or auth_failed, per-topic success acks) and can be scripted: replace the auth reply, reject topics or handshakes, send status and data messages, and close or drop connections with a chosen code. Tests cover connect, auth, subscribe, reconnect, and independent clients in parallel threads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
16 threads make 200 requests each through one client against a local HTTP server and check every response matches its request and per-request options don't leak into the client's default headers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
d35a158 to
639baa1
Compare
Replace the deprecated [tool.poetry.dev-dependencies] section with a `test` group (pook, all the unit tests need) and a `dev` group (linters, type stubs, Sphinx, orjson for the examples). `poetry install` still installs both, so local and CI workflows are unchanged, while `poetry install --only main,test` installs just what `make test` needs. Versions are unchanged. Re-locking also merged a stale duplicate entry for typing-extensions (4.12.2 below Python 3.15, 4.16.0 from 3.15) into 4.16.0, a transitive dependency of the dev tools. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a job that runs `make test` on 3.13t and 3.14t with PYTHON_GIL=0, and extend the regular matrix to 3.12, 3.13 and 3.14. The free-threaded job installs only the main and test dependency groups: the dev group is not needed to run the tests, and orjson in it does not support free-threaded Python. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
67febb0 to
f5d4b4b
Compare
chipkent
left a comment
There was a problem hiding this comment.
Assessment: not ready to merge — one likely blocker and a few medium-risk edge concerns.
PR summary
- The design direction is strong: reconnect on clean close, preserve desired subscriptions, transient/fatal retry split, thread-safe shutdown path, and substantial test expansion.
- Test coverage is materially improved (especially reconnect and fake-server behavior).
- Main risk is in reconnect-loop mechanics and close/cancel races.
Blocking issue
_open()likely reuses a single-shot awaitable across reconnect attempts.
Inconnect(),connector = connect(...)is created once and reused in the loop via_open(connector)._open()wraps it withasyncio.ensure_future(connector). Ifconnectoris coroutine-like/single-use, later attempts can fail or become undefined.
Why this is blocking: reconnect reliability depends on this path; failure here breaks core semantics in production outages.
What to change: create a fresh connect awaitable each attempt (factory per loop iteration), rather than storing one reusable awaitable.
Other adversarial findings (non-blocking but important)
-
Class-level mutable
_late_closescan cross-contaminate instances.
Make it instance state (self._late_closes) to avoid shared global task bookkeeping across multiple clients/loops. -
Exception taxonomy may be brittle across websockets versions.
_is_transient()treatsInvalidMessageas transient only when__cause__isEOFError. Equivalent transient handshake failures can surface differently by version/platform. Consider broadening classification or pinning/version-gating behavior with tests. -
close()/request_close()contract needs one explicit invariant in docs/tests:
“After close request, no further reconnect attempts occur, even if handshake completes concurrently.”
You test many races already; add a direct assertion on attempt-count stability after close request.
Design correctness check
- Subscription persistence logic: looks correct (
scheduled_subspreserved,subsreset). - Reconnect after clean close: correctly implemented.
- Backoff bounds and defaults: coherent and well-tested.
max_reconnectsas consecutive failures: semantics are improved and match long-lived process expectations.max_connectionssurfacing: good improvement and tested.
Conventions / project fit
- Overall style and test structure match repo conventions (unittest + isolated async tests).
- Docs updated for
request_close(): good. - Minor cleanup nit: duplicated
import loggingin websocket module area.
Merge readiness
- Status: Changes requested
- Risk: Medium-high until reconnect awaitable lifecycle is fixed
- Decision: Fix blocker, rerun websocket suite + matrix, then re-review
connect() returned on ConnectionClosedOK, so a clean close from the server, which it sends e.g. after a slow_consumer or max_connections status, ended the client for good. Treat it like any other disconnect and reconnect, unless close() was called. On any disconnect, keep scheduled_subs, the desired set: it used to be replaced with the topics sent so far, so a subscribe() or unsubscribe() made just before the drop was lost. Reset only subs, so the next connection subscribes exactly the desired set. When connect() gives up after max_reconnects, it raises the error that ended the last connection, ConnectionClosedOK included. The status test that relied on connect() returning now sets max_reconnects=0. BREAKING CHANGE: connect() no longer returns when the server closes the connection cleanly; it reconnects until close() is called or max_reconnects is exceeded.
Reconnects were immediate after a disconnect: the websockets connect() iterator resets its delay after every successful connection, and its own delay for refused handshakes starts below 5 s. Massive asks clients to wait at least 15 s. The iterator also retried every error, including ones that cannot pass, such as HTTP 401 or an unverifiable certificate. connect() now makes each attempt itself, with a new websockets connection object per attempt; one is built up front so that invalid arguments still raise at once. Only transient failures are retried: OSError (refused, unreachable, timed out, a DNS failure, or a TLS error other than certificate verification), the connection closing before a response (EOFError in websockets 14, InvalidMessage caused by EOFError or OSError in websockets 15), and HTTP 408, 429 and 5xx. Anything else is raised at once. A process_exception argument, which only the iterator used, is rejected with TypeError. After the n-th consecutive failure the client logs a warning and waits uniform(b, min(1.2 * b, max_reconnect_backoff)) seconds, where b = min(reconnect_backoff * 2**n, max_reconnect_backoff). Jitter only lengthens a wait and every wait lies in [reconnect_backoff, max_reconnect_backoff]. The count resets once a connection authenticates. close() during the wait returns at once, without the "no websocket open to close" warning. New constructor parameters: reconnect_backoff (default 15.0) and max_reconnect_backoff (default 120.0). BREAKING CHANGE: reconnects wait at least 15 s by default (they used to be immediate); pass reconnect_backoff to change it. Errors that cannot pass, e.g. HTTP 4xx other than 408 and 429, are raised instead of retried.
max_reconnects limited the reconnects over the client's lifetime, so with the default of 5 a long-running client raised on its sixth outage even if every reconnect had succeeded. Count consecutive failed attempts instead (refused connections included) and start over whenever a connection authenticates. The default becomes None, retrying forever. Only transient failures are retried, with a growing wait, and other errors are raised at once, so run() is safe in a long-lived process. BREAKING CHANGE: max_reconnects now defaults to None (retry forever) instead of 5, and counts consecutive failed attempts rather than all reconnects.
close() is a coroutine that uses the websocket from whatever thread calls it, and while connect() was between connections it only logged "no websocket open to close". request_close() can be called from any thread: it sets the close flag and, through loop.call_soon_threadsafe, wakes connect() and closes the socket, so connect() returns while connected, while connecting, or while waiting to reconnect. close() is kept. A request made while no connect() runs stays pending until a connect() returns because of it, and a stale wake-up queued for an earlier connect() is ignored. If connect() is cancelled while an attempt is in flight, the attempt is cancelled and awaited (websockets bounds it by open_timeout), and a connection it opened meanwhile is closed.
When the account has too many connections open, the server answers auth with a max_connections status and then closes the connection. The client only logged it, so the caller could not tell it from a network drop. Now connect() sets max_connections_reached until a later handshake succeeds and, in raw mode, passes the message to the processor, as it does for statuses after the handshake. Such a handshake does not reset the failure count, so the backoff keeps growing while the limit is hit.
f5d4b4b to
b2bc14d
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Initial handshake rejection handling and overlapping shutdown cancellation still have correctness gaps.
2 open issues
What changed in this PR
This PR revises the Massive Python client’s WebSocket reconnect behavior for long-running connections, with supporting tests, documentation, and CI updates.
Changes:
- Reconnects after clean closes, adds bounded backoff, and counts consecutive failures.
- Adds thread-safe
request_close()and exposes handshake connection-limit statuses. - Expands local-server tests and Python coverage, including free-threaded builds.
| File | Description |
|---|---|
test_websocket/test_status.py |
Updates clean-close expectations. |
test_websocket/test_reconnect.py |
Tests retries, backoff, shutdown, and handshake statuses. |
test_websocket/test_fake_server.py |
Tests client behavior against the fake server. |
test_websocket/fake_server.py |
Adds a scriptable local WebSocket server. |
test_rest/test_concurrency.py |
Tests a REST client shared across threads. |
pyproject.toml |
Separates test and development dependencies. |
poetry.lock |
Synchronizes dependency groups and resolution. |
massive/websocket/__init__.py |
Implements reconnect and shutdown semantics. |
docs/source/WebSocket.rst |
Documents request_close(). |
.github/workflows/test.yml |
Expands Python and free-threaded test coverage. |
Review effort: Balanced
| for frame, parsed in ((msg, msg_parsed), (auth_msg, auth_msg_parsed)): | ||
| if any(m.get("status") == "max_connections" for m in parsed): | ||
| self.max_connections_reached = True | ||
| if self.raw: | ||
| await processor(frame) # type: ignore |
| closing.cancel() | ||
| if not stop: | ||
| return attempt.result() | ||
| await self._abandon(attempt) |

Summary
Five changes to how
WebSocketClient.connect()handles disconnects, one commit each. The lastcommit is separable.
connect()returned onConnectionClosedOK. The server closes cleanly afterstatuses such as
slow_consumerormax_connections, so the client ended for good.close()orrequest_close()was called. The desiredsubscriptions are kept across the reconnect. If
connect()gives up (see 3), it re-raisesthe error that ended the last connection.
15 seconds before reconnecting
(knowledge base).
The
websocketsiterator also retried every error, including ones that can never succeed,such as HTTP 401/403/404 or an unverifiable certificate.
connect()makes each attempt itself, with a newwebsocketsconnection object perattempt; one is built up front so that invalid arguments raise immediately. Only transient
failures are retried:
OSError(including DNS and TLS errors other than certificateverification), the connection closing before a response (as reported by websockets 14 and
15), and HTTP 408, 429 and 5xx. Each retry logs a WARNING with the delay. The wait after the
n-th consecutive failure is
uniform(b, min(1.2·b, max))withb = min(reconnect_backoff·2ⁿ, max_reconnect_backoff), so every wait lies within theconfigured bounds.
process_exception, which only the iterator used, is rejected withTypeError.reconnect_backoff=15.0andmax_reconnect_backoff=120.0.max_reconnectscounts consecutive failures.raised on its sixth outage, even if every reconnect had succeeded.
default becomes
None, retrying forever. Because non-transient errors raise at once (2),run()is safe for long-lived processes.request_close().close()is a coroutine and is not thread-safe.request_close()can be called from any thread. It stopsconnect()promptly,including while it is connecting or waiting to reconnect, and no further attempt starts.
Cancelling
connect()also closes a connection opened in the meantime.max_connectionsreceived during the handshake (separable).max_connections_reachedis set, and in raw mode the frame is passed to theprocessor, so callers can tell a full connection slot from a network drop.
Breaking changes
Commits 1–3 are marked
fix!:withBREAKING CHANGE:footers:connect()no longer returns when the server closes the connection cleanly;it reconnects until
close()/request_close()is called ormax_reconnectsis exceeded.that cannot pass (HTTP 4xx other than 408/429, certificate failures, invalid arguments) are
raised at once instead of being retried forever.
max_reconnects: defaults toNone(retry forever) instead of 5, and counts consecutivefailed attempts rather than all reconnects.
Test plan
test_websocket/test_reconnect.py, covering:random.uniformpatched, plus measuredwaits against the fake server with tolerances for Windows timer resolution);
request_close()from another thread, while waiting to reconnect, during a stuck handshake,before
connect(), and as a handshake completes (no further attempt);max_connections.make testpasses on 3.10 to 3.14 (ubuntu and windows), and on 3.13t and 3.14t withPYTHON_GIL=0; the suite also passes with websockets 14.0.make lintpasses.