fix: only cache the internals pp after a successful lookup

[henryiii]

🤖 AI text below 🤖

Description

internals_pp_manager::get_pp() now updates the per-thread cache only after get_or_create_pp_in_state_dict() succeeds. Previously, last_istate_tls() was updated before the lookup. If that lookup threw, later calls on the same thread could skip the refresh and return either nullptr or a pointer cached from another interpreter.

The deterministic regression test covers both an initially empty cache and a cached pointer from another interpreter. It uses a dedicated manager whose on_fetch callback throws std::bad_alloc once, then checks that retrying returns the current interpreter's pointer. Both sections fail without the fix and pass with it.

This is a probable explanation for the intermittent MinGW failure in test_import_in_subinterpreter_concurrently (get_internals: get_pp() returned nullptr), reported on mingw64 in #6189 and on mingw32 in dependabot run 36804041471, attempt 1. The mingw64 log does not show the initiating lookup exception, so the cause of that failure remains unconfirmed.

Local validation passed the embedded-interpreter suite and the applicable Python sub-interpreter tests on CPython 3.12, 3.14, and free-threaded 3.14, with C++11 and warnings treated as errors. The free-threaded runs used PYTHON_GIL=0. prek -a --quiet also passed.

Suggested changelog entry:

• Fixed internals cache corruption after a failed lookup when using sub-interpreters. Subsequent calls could return nullptr or use another interpreter's internals.

[rwgk]

I'll add the test as soon as I get a chance.

---

codex GPT-6.1-Sol ultra:

The change in PR 6190 looks correct. I found no introduced correctness defect, but I would request a deterministic regression test before merging.

The subtle invariant is that the cached interpreter and pointer must describe the same successful lookup. The old ordering breaks that invariant in two ways:

Lookup throws when…	Subsequent call before fix	With this PR
Cache is initially empty	Returns null	Retries successfully
Switching from interpreter A to B	Returns A’s pointer while running in B	Fetches B’s pointer

That second case is more serious than the null-pointer symptom described in the PR.

I reproduced both failures by deliberately failing one Python allocation during the lookup. Both failed on the base commit and passed on the PR head, using CPython 3.12 and 3.14, including free-threaded 3.14. The standalone probe demonstrates that regression coverage is feasible without reproducing the intermittent race.

The two cache assignments after lookup cannot throw and have no callback between them. I found no new reentrancy hazard or ABI change.

One qualification: the original MinGW log confirms the null-pointer symptom but shows no initiating lookup exception. The proposed cause remains plausible, rather than established. I would describe the changelog entry as fixing cache corruption after a failed lookup; calling it a race implies more than the evidence supports.

Validation also passed the existing before-main, after-main, and concurrent-import cases on CPython 3.14.4 with and without the GIL. All 82 non-skipped PR checks passed.