Skip to content

Commit a81eeec

Browse files
committed
Docs: add a free-threading section on testing with ThreadSanitizer
There was no documentation on using ThreadSanitizer with the free-threaded build beyond the configure option. Describe how to build, how extensions are instrumented, how to read a report, and how to run CPython's TSan tests, and note that TSan does not detect logic races in Python code.
1 parent 7b12a41 commit a81eeec

2 files changed

Lines changed: 71 additions & 1 deletion

File tree

‎Doc/howto/free-threading-extensions.rst‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -411,6 +411,75 @@ field (or another synchronization primitive) to your object struct.
411411
an additional one.
412412

413413

414+
.. _freethreading-tsan:
415+
416+
Testing with ThreadSanitizer
417+
============================
418+
419+
Data races in C code are easy to introduce and hard to reproduce.
420+
`ThreadSanitizer <https://clang.llvm.org/docs/ThreadSanitizer.html>`_ (TSan) is
421+
a data race detector supported by Clang and GCC that reports conflicting
422+
accesses to the same memory location from different threads that are not
423+
ordered by a lock or an atomic operation. It is a good way to find missing
424+
locks, critical sections or atomic operations in an extension.
425+
426+
.. note::
427+
428+
TSan reports data races in C code, both in CPython and in extensions. It
429+
does not report races in the logic of Python code, such as two threads
430+
running ``counter = counter + 1`` on a shared object. Each of those
431+
operations is synchronized internally, so the program is free of data races
432+
even though it loses updates.
433+
434+
To use it, build a free-threaded interpreter with TSan enabled::
435+
436+
./configure --disable-gil --with-thread-sanitizer --with-pydebug
437+
make
438+
439+
``--with-pydebug`` is optional, but enables assertions that catch some
440+
additional errors. See :option:`--with-thread-sanitizer`. TSan can also be
441+
used with a build that has the GIL enabled, but the GIL orders most
442+
accesses, so a free-threaded build is much more informative.
443+
444+
The build records ``-fsanitize=thread`` in its compiler and linker flags
445+
(see :func:`sysconfig.get_config_var`), so extensions built with that
446+
interpreter by build tools that use these flags, such as ``setuptools``, are
447+
instrumented automatically. If you build an extension by hand, pass
448+
``-fsanitize=thread`` when compiling and linking. Code that is not
449+
instrumented, such as third-party libraries built without TSan, is not
450+
checked.
451+
452+
Then run your tests with the TSan interpreter, using several threads that
453+
exercise the same objects. When a race is found, TSan prints the conflicting
454+
accesses with stack traces and a summary line. For example, for an extension
455+
function that increments a global variable without synchronization::
456+
457+
WARNING: ThreadSanitizer: data race (pid=36446)
458+
Write of size 8 at 0x00010f1f8108 by thread T2:
459+
#0 bump racy.c:8 (racy.cpython-316td-darwin.so:arm64+0x7a8)
460+
...
461+
Previous write of size 8 at 0x00010f1f8108 by thread T1:
462+
#0 bump racy.c:8 (racy.cpython-316td-darwin.so:arm64+0x7a8)
463+
...
464+
Location is global 'counter' at 0x00010f1f8108 (racy.cpython-316td-darwin.so+0x8108)
465+
466+
SUMMARY: ThreadSanitizer: data race racy.c:8 in bump
467+
468+
The TSan runtime can be configured with the ``TSAN_OPTIONS`` environment
469+
variable, for example ``TSAN_OPTIONS="suppressions=path/to/suppressions.txt"``
470+
to ignore known issues. The CPython source tree has a suppressions file for
471+
the free-threaded build in :source:`Tools/tsan/suppressions_free_threading.txt`.
472+
On Linux, TSan may fail to start on kernels with high address space layout
473+
randomization; if that happens, lowering it with
474+
``sudo sysctl -w vm.mmap_rnd_bits=28`` is a common fix.
475+
476+
CPython's own test suite can be run under TSan with
477+
``./python -m test --tsan``, which runs a subset of tests that use threads.
478+
Tests that do not use threads themselves can be run in several threads at once
479+
with ``./python -m test --tsan-parallel --parallel-threads=N``. This is
480+
useful if you want to check a standard library or CPython change that touches
481+
code shared between threads.
482+
414483
Building Extensions for the Free-Threaded Build
415484
===============================================
416485

‎Doc/using/configure.rst‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1084,7 +1084,8 @@ Debug options
10841084
.. option:: --with-thread-sanitizer
10851085

10861086
Enable ThreadSanitizer data race detector, ``tsan``
1087-
(default is no).
1087+
(default is no). It is most useful together with :option:`--disable-gil`.
1088+
See :ref:`freethreading-tsan`.
10881089

10891090
.. versionadded:: 3.13
10901091

0 commit comments

Comments
 (0)