Skip to content

Commit cf3e769

Browse files
Merge branch 'main' into gh-159434-func-annotate-stale
2 parents f6ce5f0 + 9c0cfb1 commit cf3e769

75 files changed

Lines changed: 1803 additions & 402 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Doc/howto/annotations.rst‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -243,10 +243,22 @@ If you use a class with a custom metaclass and access ``__annotations__``
243243
on the class, you may observe unexpected behavior; see
244244
:pep:`749 <749#pep749-metaclasses>` for some examples. You can avoid these
245245
quirks by using :func:`annotationlib.get_annotations` on Python 3.14+ or
246-
:func:`inspect.get_annotations` on Python 3.10+. On earlier versions of
247-
Python, you can avoid these bugs by accessing the annotations from the
248-
class's :attr:`~type.__dict__`
249-
(for example, ``cls.__dict__.get('__annotations__', None)``).
246+
:func:`inspect.get_annotations` on Python 3.10+.
247+
248+
.. note::
249+
250+
On Python 3.14 and later, annotations are evaluated lazily (see
251+
:pep:`649` and :pep:`749`), and how the annotations dict is stored in
252+
the class's :attr:`~type.__dict__` is a private implementation detail.
253+
Reading ``cls.__dict__.get('__annotations__', None)`` may therefore
254+
return ``None`` even when the class has annotations. On these versions,
255+
always use :func:`annotationlib.get_annotations` (or
256+
:func:`inspect.get_annotations`) rather than accessing
257+
:attr:`~type.__dict__` directly.
258+
259+
Only on Python 3.9 and earlier, where neither function is available, can you
260+
avoid these bugs by reading the annotations from the class's
261+
:attr:`~type.__dict__` (for example, ``cls.__dict__.get('__annotations__', None)``).
250262

251263
In some versions of Python, instances of classes may have an ``__annotations__``
252264
attribute. However, this is not supported functionality. If you need the

‎Doc/library/functools.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -339,6 +339,9 @@ The :mod:`!functools` module defines the following functions:
339339

340340
.. versionadded:: 3.14
341341

342+
.. versionchanged:: next
343+
``Placeholder`` is now a :class:`sentinel`.
344+
342345
.. function:: partial(func, /, *args, **keywords)
343346

344347
Return a new :ref:`partial object<partial-objects>` which when called

‎Doc/library/numbers.rst‎

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ them to the same values. This may be subtle if there are two different
9393
extensions of the real numbers. See also :ref:`numeric-hash`.
9494

9595

96-
Adding More Numeric ABCs
96+
Adding more numeric ABCs
9797
~~~~~~~~~~~~~~~~~~~~~~~~
9898

9999
There are, of course, more possible ABCs for numbers, and this would
@@ -143,11 +143,12 @@ defined as::
143143

144144

145145
There are 5 different cases for a mixed-type operation on subclasses
146-
of :class:`Complex`. I'll refer to all of the above code that doesn't
147-
refer to ``MyIntegral`` and ``OtherTypeIKnowAbout`` as
148-
"boilerplate". ``a`` will be an instance of ``A``, which is a subtype
149-
of :class:`Complex` (``a : A <: Complex``), and ``b : B <:
150-
Complex``. I'll consider ``a + b``:
146+
of :class:`Complex`. All of the above code that doesn't refer to
147+
``MyIntegral`` and ``OtherTypeIKnowAbout`` is referred to as "boilerplate".
148+
149+
In this description, ``a`` is an instance of ``A`` and ``b`` is an
150+
instance of ``B``, where ``A`` and ``B`` are both subtypes of :class:`Complex`.
151+
Consider ``a + b``:
151152

152153
1. If ``A`` defines an :meth:`~object.__add__` which accepts ``b``, all is
153154
well.
@@ -162,13 +163,14 @@ Complex``. I'll consider ``a + b``:
162163
4. If it falls back to the boilerplate, there are no more possible
163164
methods to try, so this is where the default implementation
164165
should live.
165-
5. If ``B <: A``, Python tries ``B.__radd__`` before
166+
5. If ``B`` is a subtype of ``A``, Python tries ``B.__radd__`` before
166167
``A.__add__``. This is ok, because it was implemented with
167168
knowledge of ``A``, so it can handle those instances before
168169
delegating to :class:`Complex`.
169170

170-
If ``A <: Complex`` and ``B <: Real`` without sharing any other knowledge,
171-
then the appropriate shared operation is the one involving the built
171+
If ``A`` is a subtype of :class:`Complex` and ``B`` is a subtype of
172+
:class:`Real` without sharing any other knowledge, then the appropriate
173+
shared operation is the one involving the built
172174
in :class:`complex`, and both :meth:`~object.__radd__` s land there, so ``a+b
173175
== b+a``.
174176

‎Doc/library/urllib.error.rst‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@ The following exceptions are raised by :mod:`!urllib.error` as appropriate:
2727
:exc:`URLError` used to be a subtype of :exc:`IOError`, which is now an
2828
alias of :exc:`OSError`.
2929

30+
.. versionchanged:: 3.16
31+
The :attr:`~OSError.errno` and :attr:`~OSError.strerror` attributes are
32+
now set from :attr:`reason` when it is an :exc:`OSError`.
33+
3034

3135
.. exception:: HTTPError(url, code, msg, hdrs, fp)
3236

‎Lib/bz2.py‎

Lines changed: 10 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@
2525
# Value 2 no longer used
2626
_MODE_WRITE = 3
2727

28+
# Every bzip2 stream starts with one of these, "BZh" and the block size.
29+
_STREAM_MAGIC = (b'BZh1', b'BZh2', b'BZh3', b'BZh4', b'BZh5', b'BZh6', b'BZh7', b'BZh8', b'BZh9')
30+
2831

2932
class BZ2File(_streams.BaseStream):
3033

@@ -92,7 +95,8 @@ def __init__(self, filename, mode="r", *, compresslevel=9):
9295

9396
if self._mode == _MODE_READ:
9497
raw = _streams.DecompressReader(self._fp,
95-
BZ2Decompressor, trailing_error=OSError)
98+
BZ2Decompressor, trailing_error=OSError,
99+
stream_magic=_STREAM_MAGIC)
96100
self._buffer = io.BufferedReader(raw)
97101
else:
98102
self._pos = 0
@@ -337,22 +341,11 @@ def decompress(data):
337341
338342
For incremental decompression, use a BZ2Decompressor object instead.
339343
"""
340-
results = []
341-
while data:
342-
decomp = BZ2Decompressor()
343-
try:
344-
res = decomp.decompress(data)
345-
except OSError:
346-
if results:
347-
break # Leftover data is not a valid bzip2 stream; ignore it.
348-
else:
349-
raise # Error on the first iteration; bail out.
350-
results.append(res)
351-
if not decomp.eof:
352-
raise ValueError("Compressed data ended before the "
353-
"end-of-stream marker was reached")
354-
data = decomp.unused_data
355-
return b"".join(results)
344+
if not data:
345+
return b""
346+
return _streams._decompress_all(data, BZ2Decompressor, OSError,
347+
stream_magic=_STREAM_MAGIC,
348+
truncated_error=ValueError)
356349

357350

358351
def __getattr__(name):

‎Lib/codecs.py‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -402,9 +402,10 @@ def reset(self):
402402
pass
403403

404404
def seek(self, offset, whence=0):
405-
self.stream.seek(offset, whence)
405+
# reset() may write to the stream: call it at the old position.
406406
if whence == 0 and offset == 0:
407407
self.reset()
408+
self.stream.seek(offset, whence)
408409

409410
def __getattr__(self, name,
410411
getattr=getattr):
@@ -741,10 +742,11 @@ def reset(self):
741742
self.writer.reset()
742743

743744
def seek(self, offset, whence=0):
744-
self.stream.seek(offset, whence)
745-
self.reader.reset()
745+
# writer.reset() may write to the stream: call it at the old position.
746746
if whence == 0 and offset == 0:
747747
self.writer.reset()
748+
self.stream.seek(offset, whence)
749+
self.reader.reset()
748750

749751
def __getattr__(self, name,
750752
getattr=getattr):
@@ -864,8 +866,8 @@ def reset(self):
864866
def seek(self, offset, whence=0):
865867
# Seeks must be propagated to both the readers and writers
866868
# as they might need to reset their internal buffers.
867-
self.reader.seek(offset, whence)
868869
self.writer.seek(offset, whence)
870+
self.reader.seek(offset, whence)
869871

870872
def __getattr__(self, name,
871873
getattr=getattr):

‎Lib/collections/__init__.py‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1163,8 +1163,7 @@ def __ror__(self, other):
11631163
if not isinstance(other, _collections_abc.Mapping):
11641164
return NotImplemented
11651165
m = dict(other)
1166-
for child in reversed(self.maps):
1167-
m.update(child)
1166+
m.update(self)
11681167
return self.__class__(m)
11691168

11701169

‎Lib/compression/_common/_streams.py‎

Lines changed: 67 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,8 @@ class DecompressReader(io.RawIOBase):
3636
def readable(self):
3737
return True
3838

39-
def __init__(self, fp, decomp_factory, trailing_error=(), **decomp_args):
39+
def __init__(self, fp, decomp_factory, trailing_error=(), *,
40+
stream_magic=(), **decomp_args):
4041
self._fp = fp
4142
self._eof = False
4243
self._pos = 0 # Current offset in decompressed stream
@@ -56,6 +57,11 @@ def __init__(self, fp, decomp_factory, trailing_error=(), **decomp_args):
5657
# trailing data to ignore
5758
self._trailing_error = trailing_error
5859

60+
# Prefix (or tuple of prefixes) that every compressed stream starts
61+
# with, if the format has one. Data starting with it is a damaged
62+
# stream rather than trailing data, so its error is not ignored.
63+
self._stream_magic = stream_magic
64+
5965
def close(self):
6066
self._decompressor = None
6167
return super().close()
@@ -90,6 +96,9 @@ def read(self, size=-1):
9096
try:
9197
data = self._decompressor.decompress(rawblock, size)
9298
except self._trailing_error:
99+
if (self._stream_magic
100+
and rawblock.startswith(self._stream_magic)):
101+
raise # Damaged stream, not trailing data.
93102
# Trailing data isn't a valid compressed stream; ignore it.
94103
break
95104
else:
@@ -160,3 +169,60 @@ def seek(self, offset, whence=io.SEEK_SET):
160169
def tell(self):
161170
"""Return the current file position."""
162171
return self._pos
172+
173+
174+
def _decompress_all(data, decomp_factory, trailing_error=(), *,
175+
stream_magic=(), truncated_error, **decomp_args):
176+
"""Decompress all of the concatenated streams in *data*.
177+
178+
Each stream is decompressed by a new decompressor, made by calling
179+
*decomp_factory* with *decomp_args*. As in DecompressReader, an error
180+
of class *trailing_error* in a stream after the first one ends the
181+
data, unless that stream starts with *stream_magic*. A stream without
182+
an end-of-stream marker raises *truncated_error*.
183+
"""
184+
results = []
185+
# The views are released on exit, so that the caller can resize a
186+
# bytearray argument again, also while handling an exception.
187+
with memoryview(data) as view, view.cast("B") as data:
188+
pos = 0
189+
# There is usually a single stream: the first chunk is all of the
190+
# data, to decompress it in one go.
191+
chunk_size = len(data)
192+
while True:
193+
decomp = decomp_factory(**decomp_args)
194+
start = pos
195+
res = []
196+
try:
197+
# Feed the stream in chunks of increasing size, so that what
198+
# is read past its end (and copied into unused_data) is
199+
# bounded by the size of the stream, not by the size of all
200+
# the streams that follow it.
201+
while not decomp.eof and pos < len(data):
202+
with data[pos:pos + chunk_size] as chunk:
203+
res.append(decomp.decompress(chunk))
204+
pos += len(chunk)
205+
chunk_size *= 2
206+
except trailing_error:
207+
if not results:
208+
raise # Error in the first stream; bail out.
209+
if stream_magic:
210+
if isinstance(stream_magic, bytes):
211+
stream_magic = (stream_magic,)
212+
size = max(map(len, stream_magic))
213+
with data[start:start + size] as head:
214+
if head.tobytes().startswith(stream_magic):
215+
raise # Damaged stream, not trailing data.
216+
# Trailing data isn't a valid compressed stream; ignore it.
217+
break
218+
results.extend(res)
219+
if not decomp.eof:
220+
raise truncated_error("Compressed data ended before the "
221+
"end-of-stream marker was reached")
222+
pos -= len(decomp.unused_data)
223+
if pos == len(data):
224+
break
225+
# Later streams start small: a larger first chunk reads further
226+
# past the end of a short stream, a smaller one needs more calls.
227+
chunk_size = 1024
228+
return b"".join(results)

‎Lib/compression/zstd/__init__.py‎

Lines changed: 4 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
from _zstd import (ZstdCompressor, ZstdDecompressor, ZstdDict, ZstdError,
3434
get_frame_size, zstd_version, ZSTD_VERSION,
3535
zstd_version_info, ZSTD_VERSION_INFO)
36+
from compression._common import _streams
3637
from compression.zstd._zstdfile import ZstdFile, open, _nbytes
3738

3839

@@ -160,17 +161,9 @@ def decompress(data, zstd_dict=None, options=None):
160161
161162
For incremental decompression, use a ZstdDecompressor instead.
162163
"""
163-
results = []
164-
while True:
165-
decomp = ZstdDecompressor(options=options, zstd_dict=zstd_dict)
166-
results.append(decomp.decompress(data))
167-
if not decomp.eof:
168-
raise ZstdError('Compressed data ended before the '
169-
'end-of-stream marker was reached')
170-
data = decomp.unused_data
171-
if not data:
172-
break
173-
return b''.join(results)
164+
return _streams._decompress_all(data, ZstdDecompressor,
165+
truncated_error=ZstdError,
166+
options=options, zstd_dict=zstd_dict)
174167

175168

176169
class CompressionParameter(enum.IntEnum):

‎Lib/functools.py‎

Lines changed: 2 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -274,30 +274,7 @@ def reduce(function, sequence, /, initial=_initial_missing):
274274
### partial() argument application
275275
################################################################################
276276

277-
278-
class _PlaceholderType:
279-
"""The type of the Placeholder singleton.
280-
281-
Used as a placeholder for partial arguments.
282-
"""
283-
__instance = None
284-
__slots__ = ()
285-
286-
def __init_subclass__(cls, *args, **kwargs):
287-
raise TypeError(f"type '{cls.__name__}' is not an acceptable base type")
288-
289-
def __new__(cls):
290-
if cls.__instance is None:
291-
cls.__instance = object.__new__(cls)
292-
return cls.__instance
293-
294-
def __repr__(self):
295-
return 'Placeholder'
296-
297-
def __reduce__(self):
298-
return 'Placeholder'
299-
300-
Placeholder = _PlaceholderType()
277+
Placeholder = sentinel('Placeholder')
301278

302279
def _partial_prepare_merger(args):
303280
if not args:
@@ -440,7 +417,7 @@ def __setstate__(self, state):
440417

441418

442419
try:
443-
from _functools import partial, Placeholder, _PlaceholderType
420+
from _functools import partial, Placeholder
444421
except ImportError:
445422
pass
446423

0 commit comments

Comments
 (0)