`PyThreadState_GetFrame()` thread-safety is unclear (and inconsistent with `sys._current_frames()`)

[devdanzin]

Documentation

The documentation for PyThreadState_GetFrame() says:

Get the current frame of the Python thread state tstate.

Return a strong reference. Return NULL if no frame is currently executing.

See also PyEval_GetFrame().

tstate must not be NULL, and must be attached.

Added in version 3.9.

The "must be attached" requirement doesn't say whether tstate must be the caller's attached thread state, or may be any thread state attached to any OS thread. In the GIL build it didn't matter — any thread state could only be attached when the GIL was held, so cross-thread reads were implicitly serialized. In free-threaded builds the distinction is load-bearing: multiple thread states can be attached concurrently, and calling PyThreadState_GetFrame(foreign_tstate) races with the owning thread's own frame operations.

Meanwhile, the Python-level equivalent sys._current_frames() is internally synchronized against exactly this hazard. From Python/pystate.c:_PyThread_CurrentFrames:

PyObject *
_PyThread_CurrentFrames(void)
{
    // ...
    _PyEval_StopTheWorldAll(runtime);
    HEAD_LOCK(runtime);
    for (i = runtime->interpreters.head; i != NULL; i = i->next) {
        _Py_FOR_EACH_TSTATE_UNLOCKED(i, t) {
            _PyInterpreterFrame *frame = t->current_frame;
            // ... _PyFrame_GetFrameObject(frame) ...
        }
    }
    HEAD_UNLOCK(runtime);
    _PyEval_StartTheWorldAll(runtime);
    // ...
}

Note the _PyEval_StopTheWorldAll and HEAD_LOCK. So pure-Python code using sys._current_frames() gets a safe snapshot of every thread's frame under free-threading.

The public C API PyThreadState_GetFrame() does neither:

PyFrameObject*
PyThreadState_GetFrame(PyThreadState *tstate)
{
    assert(tstate != NULL);
    _PyInterpreterFrame *f = _PyThreadState_GetFrame(tstate);   // racy read of tstate->current_frame
    if (f == NULL) {
        return NULL;
    }
    PyFrameObject *frame = _PyFrame_GetFrameObject(f);           // may allocate + write f->frame_obj
    if (frame == NULL) {
        PyErr_Clear();
    }
    return (PyFrameObject*)Py_XNewRef(frame);
}

Both operations are problematic across threads:

1. _PyThreadState_GetFrame(tstate) reads tstate->current_frame — but this pointer is written on every Python frame push/pop by the owning thread, so concurrent reads race with bytecode execution.
2. _PyFrame_GetFrameObject(f), if f->frame_obj == NULL, calls _PyFrame_MakeAndSetFrameObject(f) which allocates a fresh PyFrameObject and stores it into f->frame_obj — i.e., it writes to memory owned by another thread's interpreter frame. If the owning thread is simultaneously in clear_thread_frame → take_ownership → _PyFrame_Copy (its normal frame-cleanup path), these operations race.

Under ThreadSanitizer I can see the race reliably. (I haven't managed to produce a pure-Python reproducer because both sys._current_frames() and sys._current_exceptions() are already properly synchronized internally. Reproduction requires a C extension that calls PyThreadState_GetFrame directly on a foreign thread state — for example, any heap profiler or introspection library that walks all threads.)

Suggested docs clarification

Whichever of the following is the intended answer, the docs should say so:

Option A — "caller must own the tstate": Add a line stating that tstate must be the caller's own thread state (i.e., tstate == PyThreadState_Get(), or equivalent). Document that reading a foreign thread's frame from a C extension requires stopping the world first, and point at sys._current_frames() as the supported pure-Python alternative.

Option B — "safe on any tstate": Add _PyEval_StopTheWorldAll + HEAD_LOCK to PyThreadState_GetFrame, matching _PyThread_CurrentFrames. Document that the call is safe on any attached thread state, at the cost of a brief stop-the-world.

Option C — "racy but benign": Add _Py_NO_SANITIZE_THREAD to PyThreadState_GetFrame and the helpers it calls (_PyThreadState_GetFrame, _PyFrame_GetFrameObject, _PyFrame_MakeAndSetFrameObject), matching gh-131548's treatment of PyUnstable_InterpreterFrame_GetLine and _PyFrame_IsIncomplete. Document the race as benign and explain the caller's expectations.

I don't have enough context to know which of these is the right call. Option A is the most conservative; Option B matches sys._current_frames() behavior at the C level; Option C is consistent with what gh-128421 and gh-131548 did for sibling APIs.

How I got here

I was running ThreadSanitizer on the guppy3 heap profiler (which installs its own ref tracer and walks all threads to find roots) on a free-threaded 3.14 debug + TSan CPython build. The TSan reports pointed at _PyFrame_Copy, take_ownership, clear_thread_frame, _PyErr_GetRaisedException, and _PyEval_EvalFrameDefault. Tracing all of them back to their triggering callers, every one was reached from guppy3's rootstate_traverse_unlocked → PyThreadState_GetFrame(foreign_tstate) → _PyFrame_MakeAndSetFrameObject.

In guppy3's case, the right fix is to switch to sys._current_frames() or wrap the traversal in _PyEval_StopTheWorld. But the fact that a stable-ABI C API documented as returning "the current frame of the Python thread state tstate" is silently unsafe on foreign tstate under free-threading feels like something the docs should at least warn about.

Investigation and issue draft done with assistance from Claude Code, reviewed by a human.