PEP 813: Address feedback so far in the DPO thread (python#4827)

* Address feedback so far in the DPO thread
* Rich treats any false-y name in the 2-tuple form as a positional argument
* Describe the new PyObject_Pretty() function
* Clarify `name` must be a `str`
* For f-strings only (not ``str.format()``) the ``!p`` conversion field takes an optional "format spec".
* The PEP no longer proposes a ``pretty`` argument to the ``print()`` built-in function.  With the
    addition of ``!p:callable`` syntax for f-strings, the new argument is unnecessary.
* Specify that to pretty print tuples as positional arguments, use the 2-tuple value format, passing
    a "false-y" value as the argument name.
* Clarify that a truth-y ``name`` must be a ``str``.
* Specify that the ``!p`` conversion in f-strings and ``str.format()`` implicitly perform an
    import of the ``pprint`` module.
* Describe the new Limited C API function ``PyObject_Pretty()``, and add the optional argument.
* Clean up the format-specs-for-f-strings-only language and add a cross reference

Author: warsaw

peps/pep-0813.rst

@@ -37,12 +37,13 @@ content.  However, the existing :mod:`pprint` module can only format builtin obj
 By providing a way for classes to customize how their instances participate in pretty printing,
 users have more options for visually improving the display of their complex data, especially for debugging.
 
-By extending the built-in :func:`print` function to automatically pretty print its output, debugging with
-user-friendly display is made even more convenient.  Since no extra imports are required, users can easily
-just piggyback on well-worn "print debugging" patterns, at least for the most common use cases.
+By adding a ``!p`` conversion specifier to f-strings and ``str.format()``, debugging with user-friendly
+display is made even more convenient.  Since no extra imports are required, users can easily just piggyback
+on well-worn "print debugging" patterns, at least for the most common use cases.
 
 These extensions work both independently and complimentary, to provide powerful new use cases.
 
+.. _specification:
 
 Specification
 =============
@@ -70,41 +71,56 @@ class name.  The printed representation will usually look like a class construct
 keyword, and default arguments.  The values can be any of the following formats:
 
 * A single value, representing a positional argument.  The value itself is used.
-* A 2-tuple of ``(name, value)`` representing a keyword argument.  A representation of
-  ``name=value`` is used.
+* A 2-tuple of ``(name, value)`` representing a keyword argument.  A
+  representation of ``name=value`` is used.  If ``name`` is "false-y", then
+  ``value`` is treated as a positional argument.  This is how you would print
+  a positional argument with a tuple value.  See :ref:`examples`.  Otherwise,
+  ``name`` **MUST** exactly be a ``str``.
 * A 3-tuple of ``(name, value, default_value)`` representing a keyword argument with a default
   value.  If ``value`` equals ``default_value``, then this tuple is skipped, otherwise
-  ``name=value`` is used.
+  ``name=value`` is used.  ``name`` **MUST** exactly be a ``str``.
 
 .. note::
 
    This protocol is compatible with the `Rich library's pretty printing protocol
    <https://rich.readthedocs.io/en/latest/pretty.html#rich-repr-protocol>`_.
 
 
-A new argument to built-in ``print``
-------------------------------------
+Additions to ``f-strings`` and ``str.format()``
+-----------------------------------------------
 
-Built-in :func:`print` takes a new optional argument, appended to the end of the argument list, called
-``pretty``, which can take one of the following values:
+In addition to the existing ``!s``, ``!r``, and ``!a`` conversion specifiers, a new ``!p``
+conversion specifier will be added.  The effect of this specifier with an expression ``value`` will
+be to call :py:func:`python:pprint.pformat` (importing the ``pprint`` module as needed), passing
+``value`` as the only argument.
 
-* ``None`` - the default. No pretty printing is invoked.  Fully backward compatible.
-* ``True`` - use a temporary instance of the :py:class:`python:pprint.PrettyPrinter` class to get a
-  pretty representation of the object.
-* An instance with a ``pformat()`` method, which has the same signature as
-  :py:meth:`python:pprint.PrettyPrinter.pformat`.  When given, this will usually be an instance of a
-  subclass of ``PrettyPrinter`` with its ``pformat()`` method overridden.  Note that this form
-  requires **an instance** of a pretty printer, not a class, as only ``print(..., pretty=True)``
-  performs implicit instantiation.
+For f-strings only, the ``!p`` conversion specifier accepts an optional "format spec" expression, after
+the normal separating ``:``, for example: ``f'{obj!p:expression}'``.  Formally, the expression can
+be anything that evaluates to a callable accepting a single argument (the object to format), and
+returns a string which is used as the f-string substitution value.  Also for f-strings, the ``!p``
+specifier is fully compatible with the ``obj=`` form, e.g. ``f'{obj=!p:expression}'``.  If no format
+spec is given, as above :py:func:`python:pprint.pformat` will be used.
 
+Note that format specs are *not* allowed in ``str.format()`` calls, at least for the :ref:`initial
+implementation <deferred>` of this PEP.
 
-Additions to ``f-strings`` and ``str.format()``
------------------------------------------------
 
-In addition to the existing ``!s``, ``!r``, and ``!a`` conversion specifiers, an additional ``!p`` conversion
-will be added.  The effect of this specifier with an expression ``value`` will be to call
-:py:func:`python:pprint.pformat`, passing ``value`` as the only argument. In this initial specification, it
-will be an error to provide any format specifier if ``!p`` is used.
+Additions to the C-API
+----------------------
+
+To support ``!p``, a new function, ``PyObject_Pretty()`` is added to the
+`Limited C API <https://docs.python.org/3/c-api/stable.html#limited-c-api>`_.
+This function takes two arguments: a ``PyObject *`` for the object to pretty
+print, and an optional ``PyObject *`` for the formatter callable (which may be
+``NULL``).  When the formatter is ``NULL``, this function imports the ``pprint``
+module and calls :func:`pprint.pformat` with the object as its argument,
+returning the results.  When the formatter is not ``NULL``, it must be a
+callable that accepts the object as its single argument and returns a string;
+this is used to support the already-evaluated ``:expression`` in
+``f'{obj!p:expression}'``.
+
+
+.. _examples:
 
 Examples
 ========
@@ -125,7 +141,7 @@ class:
             yield 'pickups', self._pickups
             yield 'active', self._active, False
 
-Now let's create a couple of instances, and pretty print them:
+Now let's create a couple of instances and pretty print them:
 
 .. code-block:: pycon
 
@@ -137,16 +153,28 @@ Now let's create a couple of instances, and pretty print them:
     >>> pprint.pprint(stingray)
     Bass(5, pickups='humbucker', active=True)
 
-Here's an example of using the ``pretty`` argument to built-in ``print()``:
+The ``!p`` conversion specifier can be used in f-strings and ``str.format()`` to pretty print values:
 
 .. code-block:: pycon
+   :force:
+
+    >>> print(f'{precision=!p}')
+    precision=Bass(4, pickups='split coil P')
+
+    >>> print('{!p}'.format(precision))
+    Bass(4, pickups='split coil P')
+
+For more complex objects, ``!p`` can help make debugging output more readable:
+
+.. code-block:: pycon
+   :force:
 
     >>> import os
     >>> print(os.pathconf_names)
     {'PC_ASYNC_IO': 17, 'PC_CHOWN_RESTRICTED': 7, 'PC_FILESIZEBITS': 18, 'PC_LINK_MAX': 1, 'PC_MAX_CANON': 2, 'PC_MAX_INPUT': 3, 'PC_NAME_MAX': 4, 'PC_NO_TRUNC': 8, 'PC_PATH_MAX': 5, 'PC_PIPE_BUF': 6, 'PC_PRIO_IO': 19, 'PC_SYNC_IO': 25, 'PC_VDISABLE': 9, 'PC_MIN_HOLE_SIZE': 27, 'PC_ALLOC_SIZE_MIN': 16, 'PC_REC_INCR_XFER_SIZE': 20, 'PC_REC_MAX_XFER_SIZE': 21, 'PC_REC_MIN_XFER_SIZE': 22, 'PC_REC_XFER_ALIGN': 23, 'PC_SYMLINK_MAX': 24}
 
-    >>> print(os.pathconf_names, pretty=True)
-    {'PC_ALLOC_SIZE_MIN': 16,
+    >>> print(f'{os.pathconf_names = !p}')
+    os.pathconf_names = {'PC_ALLOC_SIZE_MIN': 16,
      'PC_ASYNC_IO': 17,
      'PC_CHOWN_RESTRICTED': 7,
      'PC_FILESIZEBITS': 18,
@@ -167,12 +195,38 @@ Here's an example of using the ``pretty`` argument to built-in ``print()``:
      'PC_SYNC_IO': 25,
      'PC_VDISABLE': 9}
 
+For f-strings only, the ``!p`` conversion specifier also accepts a format spec expression, which must
+evaluate to a callable taking a single argument and returning a string:
+
+.. code-block:: pycon
+   :force:
+
+    >>> def slappa(da: Bass) -> str:
+    ...     return 'All about that bass'
+
+    >>> print(f'{precision=!p:slappa}')
+    precision=All about that bass
+
+Here's an example where a positional argument has a tuple value.  In this case, you use the 2-tuple format,
+with the ``name`` being "false-y".
+
+.. code-block:: pycon
+
+    >>> class Things:
+    ...     def __pprint__(self):
+    ...         yield (None, (1, 2))
+    ...         yield ('', (3, 4))
+    ...         yield ('arg', (5, 6))
+    ...
+    >>> from rich.pretty import pprint
+    >>> pprint(Things())
+    Things((1, 2), (3, 4), arg=(5, 6))
+
 
 Backwards Compatibility
 =======================
 
-When none of the new features are used, this PEP is fully backward compatible, both for built-in
-``print()`` and the ``pprint`` module.
+When none of the new features are used, this PEP is fully backward compatible.
 
 
 Security Implications
@@ -184,7 +238,7 @@ There are no known security implications for this proposal.
 How to Teach This
 =================
 
-Documentation and examples are added to the ``pprint`` module and the ``print()`` function.
+Documentation and examples are added to the ``pprint`` module, f-strings, and ``str.format()``.
 Beginners don't need to be taught these new features until they want prettier representations of
 their objects.
 
@@ -199,16 +253,30 @@ branch <https://github.com/warsaw/cpython/tree/pprint>`__.
 Rejected Ideas
 ==============
 
-None at this time.
+We considered an alternative :ref:`specification <specification>` of the ``__pprint__()`` return
+values, where either :func:`~collections.namedtuple`\s, :mod:`dataclasses`, or a duck-typed instance
+were used as the return types.  Ultimately we rejected this because we don't want to force folks to
+define a new class or add any imports just to return values from this function.
+
+
+.. _deferred:
+
+Deferred Ideas
+==============
+
+In the future, we could add support for ``!p`` conversions to t-strings.  Addition of the ``:expression``
+format for ``!p`` conversions on ``str.format()`` is also deferred.
 
 
 Open Issues
 ===========
 
-The output format and APIs are heavily inspired by `Rich
-<rich-repr-protocol_>`_. The idea is that Rich could
-implement an API compatible with ``print(..., pretty=RichPrinter)`` fairly easily.  Rich's API is designed to
-print constructor-like representations of instances, which means that it's not possible to control much of the
+Rich compatibility
+------------------
+
+The output format and APIs are heavily inspired by `Rich <rich-repr-protocol_>`_. The idea is that Rich could
+implement a callable compatible with ``!p:callable`` fairly easily.  Rich's API is designed to print
+constructor-like representations of instances, which means that it's not possible to control much of the
 "class chrome" around the arguments.  Rich does support using angle brackets (i.e. ``<...>``) instead of
 parentheses by setting the attribute ``.angular=True`` on the rich repr method.  This PEP does not support
 that feature, although it likely could in the future.
@@ -217,32 +285,34 @@ This also means that there's no way to control the pretty printed format of buil
 dicts, lists, etc.  This seems fine as ``pprint`` is not intended to be as feature-rich (pun intended!) as
 Rich.  This PEP purposefully deems such fancy features as out-of-scope.
 
-One consequence of ``print(..., pretty=True)`` is that it can be more less obvious if you wanted to print
-multiple objects with, say a newline between the object representations.  Compare these two outputs:
-
-.. code-block:: pycon
-
-    >>> print(precision, '\n', stingray, pretty=True)
-    Bass(4, pickups='split coil P') '\n' Bass(5, pickups='humbucker', active=True)
-
-    >>> print(precision, stingray, sep='\n', pretty=True)
-    Bass(4, pickups='split coil P')
-    Bass(5, pickups='humbucker', active=True)
-
-It's likely you'll want the second output, but more complicated multi-object displays could get even less
-convenient and/or more verbose.
-
 
 Acknowledgments
 ===============
 
-TBD
+Pablo Galindo Salgado for helping the PEP authors prototype the use of and prove the feasibility of
+``!p:callable`` for f-strings.
 
 
 Footnotes
 =========
 
-TBD
+None at this time.
+
+
+Change History
+==============
+
+* `TBD <TBD>`__
+
+  * For f-strings only (not ``str.format()``) the ``!p`` conversion specifier takes an optional "format spec".
+  * The PEP no longer proposes a ``pretty`` argument to the ``print()`` built-in function.  With the
+    addition of ``!p:callable`` syntax for f-strings, the new argument is unnecessary.
+  * Specify that to pretty print tuples as positional arguments, use the 2-tuple value format, passing
+    a "false-y" value as the argument name.
+  * Clarify that a truth-y ``name`` must be a ``str``.
+  * Specify that the ``!p`` conversion in f-strings and ``str.format()`` implicitly perform an
+    import of the ``pprint`` module.
+  * Describe the new Limited C API function ``PyObject_Pretty()``, and add the optional argument.
 
 
 Copyright