Skip to content

Commit 2d5b380

Browse files
Add a test.support.isolated() decorator
Run a test in a fresh interpreter subprocess, so that it does not share global or interpreter state with the rest of the test run. It can decorate a test method (only that method runs in a subprocess) or a TestCase subclass (the whole class runs in one subprocess, with its setUpClass()/setUp()/tearDown()/ tearDownClass() running once there). Failures, errors and skips, including those of individual subtests, are reported for the test and show the original subprocess traceback. The subprocess inherits the parent's resource, memory and verbosity configuration, so that requires_resource(), bigmemtest() and similar behave the same in both processes. The test.support.running_isolated flag is true in the subprocess, so that fixtures can choose what to run there. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 2670cb0 commit 2d5b380

5 files changed

Lines changed: 365 additions & 0 deletions

File tree

Doc/library/test.rst

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -961,6 +961,53 @@ The :mod:`!test.support` module defines the following functions:
961961
:mod:`tracemalloc` is enabled.
962962

963963

964+
.. decorator:: isolated()
965+
966+
Decorator that runs the decorated test in isolation, in a fresh interpreter
967+
subprocess, so that it does not share global or interpreter state with the
968+
rest of the test run. It can decorate a test method or a whole
969+
:class:`~unittest.TestCase` subclass. Decorated methods must take no extra
970+
arguments. A failure, error or skip in the subprocess is reported for the
971+
corresponding test, and individual :meth:`subtests
972+
<unittest.TestCase.subTest>` that fail or are skipped are reported
973+
individually. A reported failure or error shows the original subprocess
974+
traceback as the cause of the exception.
975+
976+
When a **method** is decorated, only that method runs in a subprocess;
977+
:meth:`~unittest.TestCase.setUp` and :meth:`~unittest.TestCase.tearDown`
978+
run both in the parent process (as usual) and in the subprocess around the
979+
method.
980+
981+
When a **class** is decorated, the whole class runs in a single subprocess,
982+
and :meth:`~unittest.TestCase.setUpClass`,
983+
:meth:`~unittest.TestCase.tearDownClass`, :meth:`~unittest.TestCase.setUp`
984+
and :meth:`~unittest.TestCase.tearDown` run once each in the subprocess and
985+
are skipped in the parent process. A failure or skip of
986+
:meth:`~unittest.TestCase.setUpClass` in the subprocess is reported for the
987+
whole class. ``setUpModule()`` cannot be controlled by a class decorator,
988+
so it still runs in the parent process too; test it with
989+
:data:`running_isolated` if needed.
990+
991+
Fixtures can test :data:`running_isolated` to decide what to run in each
992+
process.
993+
994+
The subprocess inherits the enabled resources (``-u``), memory limit
995+
(``-M``) and verbosity (``-v``) of the parent test run, so that
996+
:func:`requires_resource`, :func:`requires`, :func:`bigmemtest` and the like
997+
behave consistently in both processes.
998+
999+
1000+
.. data:: running_isolated
1001+
1002+
``True`` while the code runs in the isolated subprocess spawned by
1003+
:func:`isolated`, and ``False`` otherwise (including in the parent process
1004+
and in a normal, non-isolated test run). Fixtures such as
1005+
:meth:`~unittest.TestCase.setUp`, :meth:`~unittest.TestCase.tearDown`,
1006+
:meth:`~unittest.TestCase.setUpClass`, :meth:`~unittest.TestCase.tearDownClass`,
1007+
``setUpModule()`` and ``tearDownModule()`` can test it to choose which code
1008+
to run in the subprocess.
1009+
1010+
9641011
.. function:: check_free_after_iterating(test, iter, cls, args=())
9651012

9661013
Assert instances of *cls* are deallocated after iterating.

Lib/test/support/__init__.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1089,6 +1089,12 @@ def wrapper(self, /, *args, **kwargs):
10891089
return wrapper
10901090
return decorator
10911091

1092+
# Run a test method or class in an isolated subprocess. Implemented in a
1093+
# dedicated module so that its frames carry the __unittest marker and are
1094+
# stripped from reported tracebacks.
1095+
from test.support._isolation import isolated, running_isolated
1096+
1097+
10921098
#=======================================================================
10931099
# Decorator/context manager for running a code in a different locale,
10941100
# correctly resetting it afterwards.

Lib/test/support/_isolation.py

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
"""Run tests in isolated subprocesses (the test.support.isolated decorator).
2+
3+
A failure, error or skip that happens in the subprocess is replayed in the
4+
parent process so that the test runner records it. The original (subprocess)
5+
traceback is attached as the cause of the replayed exception, the same way
6+
:mod:`concurrent.futures` surfaces tracebacks from worker processes.
7+
"""
8+
9+
import functools
10+
import os
11+
import sys
12+
import unittest
13+
14+
# Mark this module's frames as belonging to the test machinery, so that
15+
# unittest strips them from reported tracebacks (see TestResult._clean_tracebacks
16+
# in Lib/unittest/result.py). Only the original subprocess traceback, attached
17+
# as the cause, is then shown -- not the parent-side replay frames.
18+
__unittest = True
19+
20+
# Environment variable set in the child process so that the decorated test
21+
# method runs its real body instead of spawning yet another subprocess.
22+
_RUN_IN_SUBPROCESS_ENV = '_PYTHON_RUN_IN_SUBPROCESS'
23+
24+
# Environment variable carrying (as JSON) the regrtest-configured test.support
25+
# state, so that the bare subprocess honors -u, -M, -v, etc. like the parent.
26+
_CONFIG_ENV = '_PYTHON_ISOLATED_CONFIG'
27+
28+
# test.support globals set by regrtest (libregrtest/setup.py) that affect how
29+
# tests run and which are skipped at runtime in the subprocess.
30+
_PROPAGATED_CONFIG = (
31+
'use_resources', # -u (is_resource_enabled/requires)
32+
'max_memuse', 'real_max_memuse', # -M (bigmemtest)
33+
'verbose', # -v
34+
'failfast', # -f
35+
)
36+
37+
def _child_config():
38+
import test.support as support
39+
return {name: getattr(support, name) for name in _PROPAGATED_CONFIG}
40+
41+
def _apply_child_config():
42+
"""Mirror the parent's test.support configuration in the subprocess.
43+
44+
Called by subprocess_runner before loading the test, so that import-time
45+
decorators (e.g. requires_resource) and runtime checks see the same -u/-M/-v
46+
configuration as the parent process.
47+
"""
48+
import json
49+
import test.support as support
50+
data = os.environ.get(_CONFIG_ENV)
51+
if data:
52+
for name, value in json.loads(data).items():
53+
setattr(support, name, value)
54+
55+
# True while running inside the isolated subprocess spawned by @isolated().
56+
# setUp()/tearDown() and the class- and module-level fixtures can test it to
57+
# decide which code to run in the subprocess as opposed to the parent process.
58+
running_isolated = bool(os.environ.get(_RUN_IN_SUBPROCESS_ENV))
59+
60+
61+
class _RemoteTraceback(Exception):
62+
"""Carry a formatted traceback string from the subprocess for display.
63+
64+
Attached as the ``__cause__`` of the replayed failure/error, so that the
65+
original traceback is shown by the traceback machinery.
66+
"""
67+
def __init__(self, tb):
68+
self.tb = tb
69+
70+
def __str__(self):
71+
return self.tb
72+
73+
74+
class _SubprocessTestError(Exception):
75+
"""Replay a subprocess error (as opposed to a failure) in the parent."""
76+
77+
78+
def _remote(detail):
79+
# Wrap the subprocess traceback the way concurrent.futures does, so it is
80+
# clearly delimited when shown as the cause.
81+
return _RemoteTraceback('\n"""\n%s"""' % detail)
82+
83+
84+
def _run_in_subprocess(module, qualname):
85+
"""Run module.qualname (a test method or class) in a fresh subprocess.
86+
87+
Return ``(outcomes, output, returncode)``, where *outcomes* is the list of
88+
test outcomes decoded from the subprocess, or ``None`` if it did not run to
89+
completion (crash, import error, ...).
90+
"""
91+
import json
92+
import subprocess
93+
import tempfile
94+
env = dict(os.environ)
95+
env[_RUN_IN_SUBPROCESS_ENV] = '1'
96+
env[_CONFIG_ENV] = json.dumps(_child_config())
97+
fd, result_path = tempfile.mkstemp(suffix='.json')
98+
os.close(fd)
99+
try:
100+
cmd = [sys.executable, '-m', 'test.support.subprocess_runner',
101+
module, qualname, result_path]
102+
proc = subprocess.run(cmd, capture_output=True, text=True, env=env)
103+
try:
104+
with open(result_path, encoding='utf-8') as f:
105+
outcomes = json.load(f)
106+
except (OSError, ValueError):
107+
outcomes = None
108+
finally:
109+
try:
110+
os.unlink(result_path)
111+
except OSError:
112+
pass
113+
return outcomes, (proc.stdout or '') + (proc.stderr or ''), proc.returncode
114+
115+
116+
def _replay_outcome(test, outcome):
117+
kind = outcome['kind']
118+
detail = outcome['detail']
119+
if kind == 'skipped':
120+
test.skipTest(detail) # the detail is the skip reason, not a traceback
121+
elif kind == 'failure':
122+
raise test.failureException('test failed in the subprocess') \
123+
from _remote(detail)
124+
else: # 'error'
125+
raise _SubprocessTestError('test failed in the subprocess') \
126+
from _remote(detail)
127+
128+
129+
def _replay_outcomes(test, outcomes):
130+
# Replay each subtest outcome in its own subTest() context so that they are
131+
# reported individually, then replay the whole-test outcome (if any).
132+
main = []
133+
for outcome in outcomes:
134+
if outcome['subtest']:
135+
with test.subTest(outcome['desc']):
136+
_replay_outcome(test, outcome)
137+
else:
138+
main.append(outcome)
139+
for outcome in main:
140+
_replay_outcome(test, outcome)
141+
142+
143+
def _raise_fixture_outcome(outcome):
144+
# Reproduce a setUpClass()/setUpModule() failure or skip from the
145+
# subprocess in a parent-process fixture, so it applies to every test.
146+
if outcome['kind'] == 'skipped':
147+
raise unittest.SkipTest(outcome['detail'])
148+
raise _SubprocessTestError('class failed in the subprocess') \
149+
from _remote(outcome['detail'])
150+
151+
152+
def _isolate_method(func):
153+
@functools.wraps(func)
154+
def wrapper(self, /, *args, **kwargs):
155+
if running_isolated:
156+
# Already running in the subprocess: run the real test.
157+
return func(self, *args, **kwargs)
158+
cls = type(self)
159+
qualname = f'{cls.__qualname__}.{func.__name__}'
160+
outcomes, output, returncode = _run_in_subprocess(cls.__module__,
161+
qualname)
162+
if outcomes is None:
163+
raise _SubprocessTestError(
164+
f'test did not complete in a subprocess (exit code '
165+
f'{returncode})') from _remote(output)
166+
_replay_outcomes(self, outcomes)
167+
return wrapper
168+
169+
170+
def _isolate_class(cls):
171+
orig_setUpClass = cls.setUpClass.__func__
172+
orig_tearDownClass = cls.tearDownClass.__func__
173+
orig_setUp = cls.setUp
174+
orig_tearDown = cls.tearDown
175+
176+
def setUpClass(cls):
177+
if running_isolated:
178+
orig_setUpClass(cls)
179+
return
180+
# Run the whole class in a single subprocess and stash the outcomes
181+
# for the wrapped test methods to replay.
182+
outcomes, output, returncode = _run_in_subprocess(cls.__module__,
183+
cls.__qualname__)
184+
if outcomes is None:
185+
raise _SubprocessTestError(
186+
f'class did not complete in a subprocess (exit code '
187+
f'{returncode})') from _remote(output)
188+
by_id = {}
189+
for outcome in outcomes:
190+
if outcome['fixture']:
191+
# A setUpClass()/setUpModule() failure or skip: apply it to the
192+
# whole class by raising it here, in the parent's setUpClass().
193+
_raise_fixture_outcome(outcome)
194+
by_id.setdefault(outcome['id'], []).append(outcome)
195+
cls._isolated_outcomes = by_id
196+
197+
def tearDownClass(cls):
198+
if running_isolated:
199+
orig_tearDownClass(cls)
200+
else:
201+
cls._isolated_outcomes = None
202+
203+
def setUp(self):
204+
# In the parent the real test does not run, so neither should setUp().
205+
if running_isolated:
206+
orig_setUp(self)
207+
208+
def tearDown(self):
209+
if running_isolated:
210+
orig_tearDown(self)
211+
212+
def replay(self):
213+
by_id = getattr(type(self), '_isolated_outcomes', None) or {}
214+
_replay_outcomes(self, by_id.get(self.id(), []))
215+
216+
cls.setUpClass = classmethod(setUpClass)
217+
cls.tearDownClass = classmethod(tearDownClass)
218+
cls.setUp = setUp
219+
cls.tearDown = tearDown
220+
for name in unittest.TestLoader().getTestCaseNames(cls):
221+
method = getattr(cls, name)
222+
@functools.wraps(method)
223+
def wrapper(self, /, *args, __func=method, **kwargs):
224+
if running_isolated:
225+
return __func(self, *args, **kwargs)
226+
replay(self)
227+
setattr(cls, name, wrapper)
228+
return cls
229+
230+
231+
def isolated():
232+
"""Decorator to run a test method or class in isolation from the rest.
233+
234+
The decorated test runs in a separate, fresh Python process, so it does not
235+
share global or interpreter state with the rest of the test run. When a
236+
:class:`~unittest.TestCase` subclass is decorated, the whole class runs in a
237+
single subprocess and its ``setUpClass()``/``setUpModule()`` fixtures run
238+
once there; when a method is decorated, only that method runs in a
239+
subprocess. Decorated methods must take no extra arguments.
240+
241+
A failure, error or skip of the whole test is reported for the test, and
242+
individual subtests (:meth:`~unittest.TestCase.subTest`) that fail or are
243+
skipped are reported individually. The original subprocess traceback is
244+
shown as the cause of a reported failure or error. Use
245+
:data:`running_isolated` in fixtures to choose what to run in the subprocess.
246+
"""
247+
def decorator(obj):
248+
if isinstance(obj, type):
249+
return _isolate_class(obj)
250+
return _isolate_method(obj)
251+
return decorator
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
"""Run a single test method in this (sub)process and report the result.
2+
3+
Invoked as ``python -m test.support.subprocess_runner MODULE QUALNAME OUTFILE``
4+
by :func:`test.support.isolated`. The outcome of the test (including
5+
that of each individual subtest) is written as JSON to OUTFILE. This module is
6+
not meant to be imported.
7+
"""
8+
9+
import json
10+
import sys
11+
import unittest
12+
from unittest.case import _SubTest
13+
14+
if __name__ != '__main__':
15+
raise ImportError('this module cannot be directly imported')
16+
17+
if len(sys.argv) != 4:
18+
print('usage: python -m test.support.subprocess_runner '
19+
'MODULE QUALNAME OUTFILE', file=sys.stderr)
20+
sys.exit(2)
21+
22+
module = sys.argv[1]
23+
qualname = sys.argv[2]
24+
outfile = sys.argv[3]
25+
26+
# Mirror the parent's regrtest configuration (-u, -M, -v, ...) before importing
27+
# the test, so resource gating and bigmem sizing match the parent process.
28+
from test.support._isolation import _apply_child_config
29+
_apply_child_config()
30+
31+
suite = unittest.TestLoader().loadTestsFromName(f'{module}.{qualname}')
32+
result = unittest.TestResult()
33+
suite.run(result)
34+
35+
36+
def _outcome(kind, test, detail):
37+
subtest = isinstance(test, _SubTest)
38+
real = test.test_case if subtest else test
39+
return {
40+
'kind': kind,
41+
'subtest': subtest,
42+
'desc': test._subDescription() if subtest else '',
43+
# id() groups outcomes by test method; a non-TestCase (e.g. an
44+
# _ErrorHolder) marks a setUpClass()/setUpModule() fixture failure.
45+
'id': real.id(),
46+
'fixture': not isinstance(real, unittest.TestCase),
47+
'detail': detail,
48+
}
49+
50+
51+
outcomes = [_outcome('failure', t, tb) for t, tb in result.failures]
52+
outcomes += [_outcome('error', t, tb) for t, tb in result.errors]
53+
outcomes += [_outcome('skipped', t, reason) for t, reason in result.skipped]
54+
55+
with open(outfile, 'w', encoding='utf-8') as f:
56+
json.dump(outcomes, f)
57+
58+
sys.exit(0)
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Add the :func:`test.support.isolated` decorator to run a test method or
2+
``TestCase`` subclass in a fresh interpreter subprocess, isolated from the rest
3+
of the test run.

0 commit comments

Comments
 (0)