Skip to content

Commit 4b844c6

Browse files
authored
Merge branch 'main' into gh-158197-split-dict-iteration
2 parents 703df80 + 6f439ee commit 4b844c6

261 files changed

Lines changed: 4851 additions & 1847 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/CODEOWNERS‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -283,11 +283,10 @@ Programs/_bootstrap_python.c @ericsnowcurrently @FFY00
283283
Programs/python.c @ericsnowcurrently @FFY00
284284

285285
# JIT
286-
.github/workflows/jit.yml @savannahostrowski
287-
Include/internal/pycore_jit.h @brandtbucher @savannahostrowski @diegorusso
288-
Python/jit.c @brandtbucher @savannahostrowski @diegorusso
289-
Tools/jit/ @brandtbucher @savannahostrowski @diegorusso
290-
InternalDocs/jit.md @brandtbucher @savannahostrowski @diegorusso @AA-Turner
286+
Include/internal/pycore_jit.h @brandtbucher @diegorusso
287+
Python/jit.c @brandtbucher @diegorusso
288+
Tools/jit/ @brandtbucher @diegorusso
289+
InternalDocs/jit.md @brandtbucher @diegorusso @AA-Turner
291290

292291
# Lazy imports (PEP 810)
293292
.github/workflows/reusable-test-lazy-imports-all.yml @yhg1s @DinoV @pablogsal
@@ -297,9 +296,9 @@ Lib/test/test_lazy_import @yhg1s @DinoV @pablogsal
297296

298297
# Micro-op / μop / Tier 2 Optimiser
299298
Python/optimizer.c @markshannon @Fidget-Spinner
300-
Python/optimizer_analysis.c @markshannon @tomasr8 @Fidget-Spinner @savannahostrowski
301-
Python/optimizer_bytecodes.c @markshannon @tomasr8 @Fidget-Spinner @savannahostrowski
302-
Python/optimizer_symbols.c @markshannon @tomasr8 @Fidget-Spinner @savannahostrowski
299+
Python/optimizer_analysis.c @markshannon @tomasr8 @Fidget-Spinner
300+
Python/optimizer_bytecodes.c @markshannon @tomasr8 @Fidget-Spinner
301+
Python/optimizer_symbols.c @markshannon @tomasr8 @Fidget-Spinner
303302

304303
# Parser, Lexer, and Grammar
305304
Grammar/python.gram @pablogsal @lysnikolaou

‎.github/workflows/build.yml‎

Lines changed: 3 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -491,53 +491,6 @@ jobs:
491491
if: fromJSON(needs.build-context.outputs.run-tests)
492492
uses: ./.github/workflows/reusable-test-lazy-imports-all.yml
493493

494-
build-asan:
495-
name: 'Address sanitizer'
496-
runs-on: ${{ matrix.os }}
497-
timeout-minutes: 60
498-
needs: build-context
499-
if: needs.build-context.outputs.run-ubuntu == 'true'
500-
strategy:
501-
fail-fast: false
502-
matrix:
503-
os: [ubuntu-26.04]
504-
env:
505-
OPENSSL_VER: 3.5.9
506-
PYTHONSTRICTEXTENSIONBUILD: 1
507-
ASAN_OPTIONS: detect_leaks=0:allocator_may_return_null=1:handle_segv=0
508-
steps:
509-
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
510-
with:
511-
persist-credentials: false
512-
- name: Runner image version
513-
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
514-
- name: Register gcc problem matcher
515-
run: echo "::add-matcher::.github/problem-matchers/gcc.json"
516-
- name: Install dependencies
517-
run: sudo ./.github/workflows/posix-deps-apt.sh
518-
- name: Configure OpenSSL env vars
519-
run: |
520-
echo "MULTISSL_DIR=${GITHUB_WORKSPACE}/multissl" >> "$GITHUB_ENV"
521-
echo "OPENSSL_DIR=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}" >> "$GITHUB_ENV"
522-
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> "$GITHUB_ENV"
523-
- name: 'Restore OpenSSL build'
524-
id: cache-openssl
525-
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
526-
with:
527-
path: ./multissl/openssl/${{ env.OPENSSL_VER }}
528-
key: ${{ matrix.os }}-multissl-openssl-${{ env.OPENSSL_VER }}
529-
- name: Install OpenSSL
530-
if: steps.cache-openssl.outputs.cache-hit != 'true'
531-
run: python3 Tools/ssl/multissltests.py --steps=library --base-directory "$MULTISSL_DIR" --openssl "$OPENSSL_VER" --system Linux
532-
- name: Configure CPython
533-
run: ./configure --config-cache --with-address-sanitizer --without-pymalloc --with-openssl="$OPENSSL_DIR"
534-
- name: Build CPython
535-
run: make -j4
536-
- name: Display build info
537-
run: make pythoninfo
538-
- name: Tests
539-
run: xvfb-run make ci
540-
541494
build-san:
542495
# ${{ '' } is a hack to nest jobs under the same sidebar category.
543496
name: Sanitizers${{ '' }} # zizmor: ignore[obfuscation]
@@ -554,6 +507,9 @@ jobs:
554507
sanitizer:
555508
- TSan
556509
include:
510+
- check-name: Address
511+
sanitizer: ASan
512+
free-threading: false
557513
- check-name: Undefined behavior
558514
sanitizer: UBSan
559515
free-threading: false
@@ -667,7 +623,6 @@ jobs:
667623
- build-wasi
668624
- test-hypothesis
669625
- test-lazy-imports-all
670-
- build-asan
671626
- build-san
672627
- cross-build-linux
673628
- cifuzz
@@ -714,7 +669,6 @@ jobs:
714669
build-ubuntu-installed,
715670
build-ubuntu-ssltests,
716671
test-hypothesis,
717-
build-asan,
718672
build-san,
719673
cross-build-linux,
720674
'

‎.github/workflows/reusable-san.yml‎

Lines changed: 26 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,15 @@ jobs:
4343
# please update the versions in `Tools/pixi-packages/variants.yaml` too.
4444
echo "CC=clang" >> "$GITHUB_ENV"
4545
echo "CXX=clang++" >> "$GITHUB_ENV"
46+
- name: ASan option setup
47+
if: inputs.sanitizer == 'ASan'
48+
run: |
49+
echo "MULTISSL_DIR=${GITHUB_WORKSPACE}/multissl" >> "$GITHUB_ENV"
50+
echo "OPENSSL_DIR=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}" >> "$GITHUB_ENV"
51+
echo "ASAN_OPTIONS=${SAN_LOG_OPTION} detect_leaks=0 allocator_may_return_null=1 handle_segv=0" >> "$GITHUB_ENV"
52+
echo "PYTHONSTRICTEXTENSIONBUILD=1" >> "$GITHUB_ENV"
53+
env:
54+
SAN_LOG_OPTION: log_path=${{ github.workspace }}/san_log
4655
- name: TSan option setup
4756
if: inputs.sanitizer == 'TSan'
4857
run: |
@@ -88,24 +97,27 @@ jobs:
8897
- name: Add ccache to PATH
8998
run: |
9099
echo "PATH=/usr/lib/ccache:$PATH" >> "$GITHUB_ENV"
91-
- name: 'Restore OpenSSL build (TSan)'
100+
- name: 'Restore OpenSSL build'
92101
id: cache-openssl
93-
if: inputs.sanitizer == 'TSan'
102+
if: inputs.sanitizer == 'TSan' || inputs.sanitizer == 'ASan'
94103
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
95104
with:
96105
path: ./multissl/openssl/${{ env.OPENSSL_VER }}
97-
key: ${{ env.IMAGE_OS_VERSION }}-multissl-openssl-tsan-${{ env.OPENSSL_VER }}
98-
- name: Install OpenSSL (TSan)
106+
key: >-
107+
${{ env.IMAGE_OS_VERSION }}-multissl-openssl-${{
108+
inputs.sanitizer == 'TSan' && 'tsan-' || ''
109+
}}${{ env.OPENSSL_VER }}
110+
- name: Install OpenSSL
99111
if: >-
100-
inputs.sanitizer == 'TSan'
112+
(inputs.sanitizer == 'TSan' || inputs.sanitizer == 'ASan')
101113
&& steps.cache-openssl.outputs.cache-hit != 'true'
102114
run: >-
103115
python3 Tools/ssl/multissltests.py
104116
--steps=library
105117
--base-directory="${MULTISSL_DIR}"
106118
--openssl="${OPENSSL_VER}"
107119
--system=Linux
108-
--tsan
120+
${{ inputs.sanitizer == 'TSan' && '--tsan' || '' }}
109121
# gh-157958: -O2 instead of the pydebug default -Og to avoid a clang 21
110122
# compile-time blowup on some interpreter files.
111123
# (https://github.com/llvm/llvm-project/issues/179695)
@@ -117,14 +129,20 @@ jobs:
117129
--config-cache
118130
OPT="-O2 -g"
119131
${{
120-
inputs.sanitizer == 'TSan'
132+
inputs.sanitizer == 'ASan'
133+
&& '--with-address-sanitizer --without-pymalloc'
134+
|| inputs.sanitizer == 'TSan'
121135
&& '--with-thread-sanitizer'
122136
|| inputs.sanitizer == 'MSan'
123137
&& '--with-memory-sanitizer'
124138
|| '--with-undefined-behavior-sanitizer --with-strict-overflow'
125139
}}
126140
${{ inputs.sanitizer == 'MSan' && '--with-assertions' || '--with-pydebug' }}
127-
${{ inputs.sanitizer == 'TSan' && '--with-openssl="$OPENSSL_DIR" --with-openssl-rpath=auto' || '' }}
141+
${{
142+
(inputs.sanitizer == 'TSan' || inputs.sanitizer == 'ASan')
143+
&& '--with-openssl="$OPENSSL_DIR" --with-openssl-rpath=auto'
144+
|| ''
145+
}}
128146
${{ inputs.free-threading && '--disable-gil' || '' }}
129147
- name: Build CPython
130148
run: make -j4

‎Doc/c-api/extension-modules.rst‎

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -313,16 +313,32 @@ This is typically done using :c:func:`PyModule_Create` and functions like
313313
Single-phase initialization differs from the :ref:`default <multi-phase-initialization>`
314314
in the following ways:
315315
316-
* Single-phase modules are, or rather *contain*, “singletons”.
316+
* Single-phase modules with :c:member:`PyModuleDef.m_size` set to -1 are,
317+
or rather *contain*, “singletons”.
317318
318-
When the module is first initialized, Python saves the contents of
319+
When such a module is first initialized, Python saves the contents of
319320
the module's ``__dict__`` (that is, typically, the module's functions and
320321
types).
321322
322323
For subsequent imports, Python does not call the initialization function
323324
again.
324325
Instead, it creates a new module object with a new ``__dict__``, and copies
325326
the saved contents to it.
327+
Modules are matched to the saved contents by a (*module_name*, *file_name*)
328+
tuple where:
329+
330+
- *module_name* is the module's full dotted name, taken from
331+
:py:attr:`ModuleSpec.name <importlib.machinery.ModuleSpec.name>`
332+
(or the equivalent argument of the importing function).
333+
Note that names that the initialization function uses to create the module
334+
(for example, :c:member:`PyModuleDef.m_name`) are not taken into account.
335+
- *file_name* is the extension's file name.
336+
It is unset for modules not loaded from a file, such as built-ins and
337+
modules added with :c:func:`PyImport_CreateModuleFromInitfunc` or
338+
:c:func:`PyImport_AppendInittab`.
339+
340+
As part of the first initialization, Python also adds the module to
341+
:data:`sys.modules` under its name.
326342
For example, given a single-phase module ``_testsinglephase``
327343
[#testsinglephase]_ that defines a function ``sum`` and an exception class
328344
``error``:
@@ -344,6 +360,16 @@ in the following ways:
344360
345361
The exact behavior should be considered a CPython implementation detail.
346362

363+
* For single-phase modules with non-negative :c:member:`PyModuleDef.m_size`,
364+
Python may cache a pointer to the initialization function under the
365+
(*module_name*, *file_name*) key as for ``m_size == -1``.
366+
367+
Subsequent imports that specify a different initialization function
368+
(for example, using :c:func:`PyImport_CreateModuleFromInitfunc`)
369+
may call either the cached function or the provided "new" one.
370+
371+
The exact behavior should again be considered a CPython implementation detail.
372+
347373
* To work around the fact that ``PyInit_modulename`` does not take a *spec*
348374
argument, some state of the import machinery is saved and applied to the
349375
first suitable module created during the ``PyInit_modulename`` call.

‎Doc/c-api/file.rst‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,29 @@ the :mod:`io` APIs instead.
6060
raised if the end of the file is reached immediately.
6161
6262
63+
.. c:function:: char* Py_UniversalNewlineFgets(char *buf, int n, FILE *stream, PyObject *fobj)
64+
65+
Read a line from the C stream *stream* into the buffer *buf*, like the
66+
C standard library function :c:func:`!fgets`, but treating all of ``\n``,
67+
``\r`` and ``\r\n`` as line endings (see :term:`universal newlines`).
68+
Any ``\r`` or ``\r\n`` read from the stream is stored in *buf* as a
69+
single ``\n``.
70+
71+
At most ``n - 1`` characters are stored in *buf*; reading stops after a
72+
newline has been stored or when the end of the file is reached.
73+
The result is always null-terminated.
74+
*stream* should be opened in binary mode, so that the C library does not
75+
perform its own newline translation.
76+
77+
Return *buf* on success.
78+
Return ``NULL`` if no characters could be read, because *stream* is at
79+
end of file, a read error occurred, or *n* is less than 2.
80+
Like :c:func:`!fgets`, this function does not distinguish between end of
81+
file and a read error, and it does not set an exception.
82+
83+
*fobj* exists only for backward compatibility and must be ``NULL``.
84+
85+
6386
.. c:function:: int PyFile_SetOpenCodeHook(Py_OpenCodeHookFunction handler)
6487
6588
Overrides the normal behavior of :func:`io.open_code` to pass its parameter

‎Doc/c-api/float.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,7 @@ Pack functions
214214
The pack routines write 2, 4 or 8 bytes, starting at *p*. *le* is an
215215
:c:expr:`int` argument, non-zero if you want the bytes string in little-endian
216216
format (exponent last, at ``p+1``, ``p+3``, or ``p+6`` and ``p+7``), zero if you
217-
want big-endian format (exponent first, at *p*). Use the :c:macro:`!PY_LITTLE_ENDIAN`
217+
want big-endian format (exponent first, at *p*). Use the :c:macro:`PY_LITTLE_ENDIAN`
218218
constant to select the native endian: it is equal to ``0`` on big
219219
endian processor, or ``1`` on little endian processor.
220220
@@ -240,7 +240,7 @@ Unpack functions
240240
The unpack routines read 2, 4 or 8 bytes, starting at *p*. *le* is an
241241
:c:expr:`int` argument, non-zero if the bytes string is in little-endian format
242242
(exponent last, at ``p+1``, ``p+3`` or ``p+6`` and ``p+7``), zero if big-endian
243-
(exponent first, at *p*). Use the :c:macro:`!PY_LITTLE_ENDIAN` constant to
243+
(exponent first, at *p*). Use the :c:macro:`PY_LITTLE_ENDIAN` constant to
244244
select the native endian: it is equal to ``0`` on big endian processor, or ``1``
245245
on little endian processor.
246246

‎Doc/c-api/import.rst‎

Lines changed: 22 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -408,7 +408,7 @@ Importing Modules
408408
409409
*spec* must be a :class:`~importlib.machinery.ModuleSpec` object.
410410
411-
*initfunc* must be an :ref:`initialization function <extension-export-hook>`,
411+
*initfunc* must be an :ref:`initialization function <extension-pyinit>`,
412412
the same as for :c:func:`PyImport_AppendInittab`.
413413
414414
On success, create and return a module object.
@@ -417,10 +417,27 @@ Importing Modules
417417
(Custom importers should do this in their
418418
:py:meth:`~importlib.abc.Loader.exec_module` method.)
419419
420-
If the spec's name is dotted and *initfunc* creates the module using only
421-
the last component of the name, the module's :attr:`~module.__name__`
422-
is set to the full name from the spec.
423-
424420
On error, return NULL with an exception set.
425421
422+
For modules using :ref:`legacy single-phase initialization <single-phase-initialization>`,
423+
most of the import process is done in this function, rather than in
424+
extra steps that importers typically take after
425+
:py:meth:`~importlib.abc.Loader.create_module`.
426+
Specifically:
427+
428+
- The module is added to :data:`sys.modules` under the *spec*'s
429+
:py:attr:`~importlib.machinery.ModuleSpec.name` as part of initialization.
430+
- If a single-phase module was previously imported under the
431+
same :py:attr:`~importlib.machinery.ModuleSpec.name` by this function
432+
or as a built-in module (for example, one registered with
433+
:c:func:`PyImport_AppendInittab`), Python may reuse the previous module's
434+
contents instead of calling *initfunc* again.
435+
- The module will be fully initialized by *initfunc*.
436+
Calling :c:func:`PyModule_Exec` on it will have no effect, though it is
437+
still recommended, in case the module later switches to
438+
:ref:`multi-phase initialization <multi-phase-initialization>`.
439+
- Also, if the *spec*'s :py:attr:`!name` contains dots, this name is used
440+
for prepending a parent package to the new module's
441+
:attr:`~module.__name__`.
442+
426443
.. versionadded:: 3.15

‎Doc/c-api/interp-lifecycle.rst‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -265,6 +265,48 @@ Initializing and finalizing the interpreter
265265
first initialized).
266266
267267
268+
.. c:function:: int Py_FrozenMain(int argc, char **argv)
269+
270+
.. index:: single: freeze utility
271+
272+
The main program for a "frozen" Python program (an executable that embeds
273+
the interpreter together with the bytecode of the program's modules, as
274+
produced by the :program:`freeze` utility; see :source:`Tools/freeze`).
275+
276+
This function initializes the interpreter, imports the frozen module named
277+
``__main__``, and finalizes the interpreter.
278+
The frozen modules must have been registered in
279+
:c:var:`PyImport_FrozenModules` before calling this function;
280+
it is a fatal error if ``__main__`` is not among them.
281+
282+
*argc* and *argv* are byte strings like the arguments of a C program's
283+
``main`` function, and are passed to the interpreter with
284+
:c:func:`PyConfig_SetBytesArgv`.
285+
Unlike :c:func:`Py_Main`, the arguments are not parsed as Python command
286+
line options (:c:member:`PyConfig.parse_argv` is ``0``), so all of them
287+
are made available to the program in :data:`sys.argv`.
288+
Apart from that, the interpreter uses the
289+
:ref:`Python Configuration <init-python-config>`, so environment variables
290+
such as :envvar:`PYTHONPATH` are honored.
291+
Warnings about the path configuration
292+
(:c:member:`PyConfig.pathconfig_warnings`) are suppressed.
293+
If :envvar:`PYTHONINSPECT` is set and standard input is a terminal, an
294+
interactive prompt is run after ``__main__`` has finished, as with the
295+
:option:`-i` option.
296+
297+
Return ``0`` if ``__main__`` ran without an unhandled exception, ``1``
298+
otherwise, and ``120`` if finalizing the interpreter failed (see
299+
:c:func:`Py_FinalizeEx`).
300+
An unhandled :exc:`SystemExit` terminates the process.
301+
If the interpreter cannot be initialized, the process exits via
302+
:c:func:`Py_ExitStatusException`.
303+
304+
On Windows, this function is not exported by the Python DLL.
305+
The :program:`freeze` utility compiles :file:`Python/frozenmain.c` into the
306+
frozen program instead, together with the Windows-specific hooks that the
307+
function requires.
308+
309+
268310
.. c:function:: int Py_RunMain(void)
269311
270312
Executes the main module in a fully configured CPython runtime.

0 commit comments

Comments
 (0)