Skip to content

gh-161172: Add PyUnicodeWriter to the limited C API #161175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 2 commits into from
Nov 4, 2026
Merged

gh-161172: Add PyUnicodeWriter to the limited C API #161175

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
924cd95
gh-161172: Add PyUnicodeWriter to the limited C API
vstinner Nov 2, 2026
cddac40
Add pyunicodewriter ref in the doc
vstinner Nov 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions Doc/c-api/unicode.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1697,10 +1697,12 @@ They all return ``NULL`` or ``-1`` if an exception occurs.
intern-related details, may change between CPython versions.


.. _pyunicodewriter:

PyUnicodeWriter
^^^^^^^^^^^^^^^

The :c:type:`PyUnicodeWriter` API can be used to create a Python :class:`str`
The ``PyUnicodeWriter`` API can be used to create a Python :class:`str`
object.

.. versionadded:: 3.14
Expand Down Expand Up @@ -1916,7 +1918,7 @@ then filling in its contents.
They are :term:`soft deprecated`: they break the assumption that
strings are immutable, making them hard to use correctly.

Prefer using the :c:type:`PyUnicodeWriter` API,
Prefer using the :ref:`PyUnicodeWriter API <pyunicodewriter>`,
or one of the ``PyUnicode_From*``
functions such as :c:func:`PyUnicode_FromStringAndSize`.

Expand Down Expand Up @@ -1971,7 +1973,7 @@ Python does not always check these requirements.
.. versionadded:: 3.3

.. soft-deprecated:: 3.16
Use the :c:type:`PyUnicodeWriter` API instead.
Use the :ref:`PyUnicodeWriter API <pyunicodewriter>` instead.


.. c:function:: Py_ssize_t PyUnicode_CopyCharacters(PyObject *to, \
Expand All @@ -1991,7 +1993,7 @@ Python does not always check these requirements.
.. versionadded:: 3.3

.. soft-deprecated:: 3.16
Use the :c:type:`PyUnicodeWriter` API instead.
Use the :ref:`PyUnicodeWriter API <pyunicodewriter>` instead.


.. c:function:: int PyUnicode_Resize(PyObject **unicode, Py_ssize_t length);
Expand All @@ -2012,7 +2014,7 @@ Python does not always check these requirements.
See :ref:`pyunicode-new-mutating` for details.

.. soft-deprecated:: 3.16
Use the :c:type:`PyUnicodeWriter` API instead.
Use the :ref:`PyUnicodeWriter API <pyunicodewriter>` instead.


.. c:function:: Py_ssize_t PyUnicode_Fill(PyObject *unicode, Py_ssize_t start, \
Expand All @@ -2033,7 +2035,7 @@ Python does not always check these requirements.
.. versionadded:: 3.3

.. soft-deprecated:: 3.16
Use the :c:type:`PyUnicodeWriter` API instead.
Use the :ref:`PyUnicodeWriter API <pyunicodewriter>` instead.


.. c:function:: int PyUnicode_WriteChar(PyObject *unicode, Py_ssize_t index, \
Expand All @@ -2053,4 +2055,4 @@ Python does not always check these requirements.
.. versionadded:: 3.3

.. soft-deprecated:: 3.16
Use the :c:type:`PyUnicodeWriter` API instead.
Use the :ref:`PyUnicodeWriter API <pyunicodewriter>` instead.
13 changes: 13 additions & 0 deletions Doc/data/stable_abi.dat
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Doc/whatsnew/3.14.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2990,7 +2990,7 @@ New features in the C API
The function is also added to the Limited C API.
(Contributed by Victor Stinner in :gh:`124502`.)

* Add a new :c:type:`PyUnicodeWriter` API to create a Python :class:`str`
* Add a new :ref:`PyUnicodeWriter API <pyunicodewriter>` to create a Python :class:`str`
object, with the following functions:

* :c:func:`PyUnicodeWriter_Create`
Expand Down
5 changes: 4 additions & 1 deletion Doc/whatsnew/3.16.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1384,6 +1384,9 @@ New features
API.
(Contributed by Nathan Goldbaum and Victor Stinner in :gh:`158842`.)

* Add the :ref:`PyUnicodeWriter API <pyunicodewriter>` to the limited C API.
(Contributed by Victor Stinner in :gh:`161172`.)

Porting to Python 3.16
----------------------

Expand Down Expand Up @@ -1431,7 +1434,7 @@ Deprecated C APIs
:c:func:`PyUnicode_New`, :c:func:`PyUnicode_CopyCharacters`,
:c:func:`PyUnicode_Fill`, :c:func:`PyUnicode_Resize`,
:c:func:`PyUnicode_WRITE` and :c:func:`PyUnicode_WriteChar`.
Use the safer :c:type:`PyUnicodeWriter` API instead.
Use the safer :ref:`PyUnicodeWriter API <pyunicodewriter>` instead.
(Contributed by Victor Stinner in :gh:`157710`.)

.. Add C API deprecations above alphabetically, not here at the end.
Expand Down
51 changes: 0 additions & 51 deletions Include/cpython/unicodeobject.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -471,57 +471,6 @@ PyAPI_FUNC(PyObject*) PyUnicode_FromKindAndData(
Py_ssize_t size);


/* --- Public PyUnicodeWriter API ----------------------------------------- */

typedef struct PyUnicodeWriter PyUnicodeWriter;

PyAPI_FUNC(PyUnicodeWriter*) PyUnicodeWriter_Create(Py_ssize_t length);
PyAPI_FUNC(void) PyUnicodeWriter_Discard(PyUnicodeWriter *writer);
PyAPI_FUNC(PyObject*) PyUnicodeWriter_Finish(PyUnicodeWriter *writer);

PyAPI_FUNC(int) PyUnicodeWriter_WriteChar(
PyUnicodeWriter *writer,
Py_UCS4 ch);
PyAPI_FUNC(int) PyUnicodeWriter_WriteUTF8(
PyUnicodeWriter *writer,
const char *str,
Py_ssize_t size);
PyAPI_FUNC(int) PyUnicodeWriter_WriteASCII(
PyUnicodeWriter *writer,
const char *str,
Py_ssize_t size);
PyAPI_FUNC(int) PyUnicodeWriter_WriteWideChar(
PyUnicodeWriter *writer,
const wchar_t *str,
Py_ssize_t size);
PyAPI_FUNC(int) PyUnicodeWriter_WriteUCS4(
PyUnicodeWriter *writer,
const Py_UCS4 *str,
Py_ssize_t size);

PyAPI_FUNC(int) PyUnicodeWriter_WriteStr(
PyUnicodeWriter *writer,
PyObject *obj);
PyAPI_FUNC(int) PyUnicodeWriter_WriteRepr(
PyUnicodeWriter *writer,
PyObject *obj);
PyAPI_FUNC(int) PyUnicodeWriter_WriteSubstring(
PyUnicodeWriter *writer,
PyObject *str,
Py_ssize_t start,
Py_ssize_t end);
PyAPI_FUNC(int) PyUnicodeWriter_Format(
PyUnicodeWriter *writer,
const char *format,
...);
PyAPI_FUNC(int) PyUnicodeWriter_DecodeUTF8Stateful(
PyUnicodeWriter *writer,
const char *str, /* UTF-8 encoded string */
Py_ssize_t size, /* size of string */
const char *errors, /* error handling */
Py_ssize_t *consumed); /* bytes consumed */


/* --- Private _PyUnicodeWriter API --------------------------------------- */

typedef struct {
Expand Down
54 changes: 53 additions & 1 deletion Include/unicodeobject.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -954,7 +954,59 @@ PyAPI_FUNC(int) PyUnicode_Contains(

PyAPI_FUNC(int) PyUnicode_IsIdentifier(PyObject *s);

/* === Characters Type APIs =============================================== */

/* === Public PyUnicodeWriter API ========================================= */

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= _Py_PACK_VERSION(3, 16)
typedef struct PyUnicodeWriter PyUnicodeWriter;

PyAPI_FUNC(PyUnicodeWriter*) PyUnicodeWriter_Create(Py_ssize_t length);
PyAPI_FUNC(void) PyUnicodeWriter_Discard(PyUnicodeWriter *writer);
PyAPI_FUNC(PyObject*) PyUnicodeWriter_Finish(PyUnicodeWriter *writer);

PyAPI_FUNC(int) PyUnicodeWriter_WriteChar(
PyUnicodeWriter *writer,
Py_UCS4 ch);
PyAPI_FUNC(int) PyUnicodeWriter_WriteUTF8(
PyUnicodeWriter *writer,
const char *str,
Py_ssize_t size);
PyAPI_FUNC(int) PyUnicodeWriter_WriteASCII(
PyUnicodeWriter *writer,
const char *str,
Py_ssize_t size);
PyAPI_FUNC(int) PyUnicodeWriter_WriteWideChar(
PyUnicodeWriter *writer,
const wchar_t *str,
Py_ssize_t size);
PyAPI_FUNC(int) PyUnicodeWriter_WriteUCS4(
PyUnicodeWriter *writer,
const Py_UCS4 *str,
Py_ssize_t size);

PyAPI_FUNC(int) PyUnicodeWriter_WriteStr(
PyUnicodeWriter *writer,
PyObject *obj);
PyAPI_FUNC(int) PyUnicodeWriter_WriteRepr(
PyUnicodeWriter *writer,
PyObject *obj);
PyAPI_FUNC(int) PyUnicodeWriter_WriteSubstring(
PyUnicodeWriter *writer,
PyObject *str,
Py_ssize_t start,
Py_ssize_t end);
PyAPI_FUNC(int) PyUnicodeWriter_Format(
PyUnicodeWriter *writer,
const char *format,
...);
PyAPI_FUNC(int) PyUnicodeWriter_DecodeUTF8Stateful(
PyUnicodeWriter *writer,
const char *str, /* UTF-8 encoded string */
Py_ssize_t size, /* size of string */
const char *errors, /* error handling */
Py_ssize_t *consumed); /* bytes consumed */
#endif // !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= _Py_PACK_VERSION(3, 16)


#ifndef Py_LIMITED_API
# define Py_CPYTHON_UNICODEOBJECT_H
Expand Down
13 changes: 13 additions & 0 deletions Lib/test/test_stable_abi_ctypes.py
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/C_API/2026-11-02-13-15-22.gh-issue-161172.U6kpuN.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add the :ref:`PyUnicodeWriter API <pyunicodewriter>` to the limited C API.
Patch by Victor Stinner.
26 changes: 26 additions & 0 deletions Misc/stable_abi.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2915,3 +2915,29 @@
added = '3.16'
[macro.Py_XSETREF]
added = '3.16'
[function.PyUnicodeWriter_Create]
added = '3.16'
[function.PyUnicodeWriter_Discard]
added = '3.16'
[function.PyUnicodeWriter_Finish]
added = '3.16'
[function.PyUnicodeWriter_WriteChar]
added = '3.16'
[function.PyUnicodeWriter_WriteUTF8]
added = '3.16'
[function.PyUnicodeWriter_WriteASCII]
added = '3.16'
[function.PyUnicodeWriter_WriteWideChar]
added = '3.16'
[function.PyUnicodeWriter_WriteUCS4]
added = '3.16'
[function.PyUnicodeWriter_WriteStr]
added = '3.16'
[function.PyUnicodeWriter_WriteRepr]
added = '3.16'
[function.PyUnicodeWriter_WriteSubstring]
added = '3.16'
[function.PyUnicodeWriter_Format]
added = '3.16'
[function.PyUnicodeWriter_DecodeUTF8Stateful]
added = '3.16'
13 changes: 13 additions & 0 deletions PC/python3dll.c
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading