Skip to content

build: replace httpretty with responses and drop it - #39271

Draft
feanil wants to merge 9 commits into
masterfrom
feanil/remove-httpretty
Draft

feanil wants to merge 9 commits into
masterfrom
feanil/remove-httpretty

Conversation

@feanil

@feanil feanil commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #39246. Replaces httpretty with responses in every test that used it and drops httpretty from the testing group. httpretty's last release was in 2021, it patches sockets globally, which is how it broke MongoDB connections under pymongo 4.12+, and it does not support Python 3.14.

One commit per app so each is reviewable on its own. The first adds the dependency, the last removes httpretty; everything in between is green at each step.

A few things that are not mechanical and worth a look:

  • Discussion: CommentsServiceMockMixin and the request-parsing helpers only it used are deleted rather than migrated. Since the move to openedx-forum every discussion test resolves the same register_* names to ForumMockUtilsMixin, so the HTTP mixin was dead code.
  • Commerce mocks: the mock_* context managers in lms/djangoapps/commerce/tests/mocks.py used to enable and disable httpretty themselves. They now register on the responses default registry and expect the caller to have responses active, and expect_called counts requests to the mocked endpoint rather than inspecting the global last request. reset_on_exit is gone because it only existed to work around the old nesting behaviour.
  • Class-level @httpretty.activate: httpretty decorated every test method; responses.activate on a class wraps the class object instead and pytest silently stops collecting its tests. Those became responses.start() in setUp with stop and reset cleanups. The cache_programs tests had been silently dropped by an early version of this change, which is how I found out.
  • Query strings: responses matches a registration on scheme, host and path only. Where tests register several mocks on the same path with different query strings (pathway pages in cache_programs) they now use matchers.query_param_matcher. Where a test re-registers a path to override a setUp mock (two TestShib SAP SuccessFactors tests), it removes the setUp mock first, since responses serves the oldest registration and httpretty served the newest.
  • Request bodies: responses hands back the prepared request body as str or bytes depending on how the client sent it, so the tests pass it to json.loads directly or go through a small bytes helper instead of calling .decode().
  • _mock_catalog_api in the edx_api_utils test was being handed a url inside httpretty.Response, where it was ignored. It now goes to the helper, where it was meant to go.

After this there are no httpretty references left in the repo outside the lockfile history.

feanil and others added 9 commits October 21, 2026 09:44
httpretty has had no release since 2021 and patches sockets globally,
which is how it broke MongoDB connections under pymongo 4.12+ in #39246.
`responses` mocks at the `requests` adapter level instead, which covers
every HTTP call the httpretty-based tests exercise, and it is already
used by the user retirement scripts in this repo.  This adds it so the
httpretty tests can be migrated one app at a time; httpretty itself is
removed once the last use is gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ests

`CommentsServiceMockMixin` registered httpretty responses for the old
cs_comments_service endpoints, but since the move to openedx-forum every
discussion test resolves the same `register_*` names to
`ForumMockUtilsMixin`, which mocks the forum API's Python functions.  The
only remaining references were an unused import, a base class that was
shadowed in the MRO, and `httpretty.enable()` calls in setUp that never
had a URL registered against them.

Remove the mixin, the request-parsing helpers and callbacks only it used,
and the vestigial httpretty setup in the serializer tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The commerce mock context managers used to enable and disable httpretty
themselves, which is why nested uses needed `reset_on_exit=False` and the
module docstring warned against mixing them with direct httpretty use.
They now register on the `responses` default registry and the caller
keeps `responses` active, via `@responses.activate` or `responses.start()`
in setUp.  `expect_called` compares the number of requests to the mocked
endpoint before and after the block instead of inspecting the global
last request, so it no longer depends on what other mocks did.

The verify_student, refunds and course_modes tests that depend on these
mocks are converted in the same commit.  Request bodies are passed to
`json.loads` directly, which accepts both `str` and `bytes`, since
`responses` hands back the prepared request body as either depending on
how the client sent it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mechanical replacement of `httpretty.register_uri` with `responses.add`
and `@httpretty.activate` with `@responses.activate`.  Request counts
come from `responses.calls` instead of `httpretty.latest_requests`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mostly mechanical, with a few things worth knowing:

- `@httpretty.activate` on a class decorated every test method; the same
  decorator from `responses` wraps the class object itself, so pytest
  stops collecting the class and its tests silently disappear.  The
  class-level uses in the cache_programs and edx_api_utils tests become
  `responses.start()` in setUp with stop and reset as cleanups.
- `responses` matches a registration on scheme, host and path only, so
  the per-page pathway mocks in cache_programs match on their query
  parameters explicitly with `matchers.query_param_matcher`.
- Callback bodies take the prepared request alone and return the
  response headers themselves, and query strings are read from the
  request URL since there is no `querystring` attribute.
- The edx_api_utils test passed a `url` keyword into `httpretty.Response`
  where it was ignored; it now goes to `_mock_catalog_api`, which is
  where it was meant to go.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`ThirdPartyOAuthTestMixin` carried a class-level `@httpretty.activate`
that every OAuth test class in auth_exchange, oauth_dispatch and
user_authn repeated.  The mixin's setUp now starts `responses` and
registers stop and reset as cleanups, and the per-class decorators go
away, since `responses.activate` on a class would hide its tests from
pytest.

The TestShib SAML spec used httpretty callbacks for the SAP SuccessFactors
endpoints.  Those become `responses.add_callback` callbacks that take the
prepared request.  Two tests re-register the OData user endpoint to
override the default mock from setUp; httpretty served the newest
registration for a path but `responses` serves the oldest, so those tests
remove the setUp mock first.  Request bodies are compared as bytes
through a small helper because `requests` prepares form data as `str`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
httpretty's last release was 1.1.4 in 2021.  It patches the socket module
globally, which broke MongoDB connections under pymongo 4.12+ in #39246,
and it does not support Python 3.14.  Every test that used it has been
migrated to `responses`, so remove it from the testing group and the
lock.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant