Integrate cffi-buildtool into CFFI itself

.github/workflows/ci.yaml

@@ -187,7 +187,7 @@ jobs:
         CIBW_MUSLLINUX_X86_64_IMAGE: ${{ matrix.musllinux_img || 'musllinux_1_2' }}
         CIBW_MUSLLINUX_I686_IMAGE: ${{ matrix.musllinux_img || 'musllinux_1_2' }}
         CIBW_MUSLLINUX_AARCH64_IMAGE: ${{ matrix.musllinux_img || 'musllinux_1_2' }}
-        CIBW_TEST_REQUIRES: pytest setuptools  # 3.12+ no longer includes distutils, just always ensure setuptools is present
+        CIBW_TEST_REQUIRES: pytest setuptools meson-python ninja # 3.12+ no longer includes distutils, just always ensure setuptools is present
         CIBW_TEST_COMMAND: PYTHONUNBUFFERED=1 python -m pytest ${{ matrix.test_args || '{project}' }}  # default to test all
       run: |
         set -eux
@@ -276,7 +276,7 @@ jobs:
       env:
         CIBW_BUILD: ${{ matrix.spec }}
         CIBW_ENABLE: cpython-prerelease
-        CIBW_TEST_REQUIRES: pytest setuptools
+        CIBW_TEST_REQUIRES: pytest setuptools meson-python ninja
         CIBW_TEST_COMMAND: pip install pip --upgrade; cd {project}; PYTHONUNBUFFERED=1 pytest
         MACOSX_DEPLOYMENT_TARGET: ${{ matrix.deployment_target || '10.13' }}
         SDKROOT: ${{ matrix.sdkroot || 'macosx' }}
@@ -366,7 +366,7 @@ jobs:
       env:
         CIBW_BUILD: ${{ matrix.spec }}
         CIBW_ENABLE: cpython-prerelease
-        CIBW_TEST_REQUIRES: pytest setuptools
+        CIBW_TEST_REQUIRES: pytest setuptools meson-python ninja
         CIBW_TEST_COMMAND: ${{ matrix.test_cmd || 'python -m pytest {package}/src/c' }}
         # FIXME: /testing takes ~45min on Windows and has some failures...
         # CIBW_TEST_COMMAND='python -m pytest {package}/src/c {package}/testing'
@@ -544,7 +544,7 @@ jobs:
 
     - name: build and install
       run: |
-        python -m pip install pytest setuptools pytest-run-parallel
+        python -m pip install pytest setuptools meson-python ninja pytest-run-parallel
         python -m pip install .
 
     - name: run tests under pytest-run-parallel
@@ -559,13 +559,7 @@ jobs:
 
     - name: build and install
       run: |
-        # these two steps can be removed when
-        # https://github.com/nascheme/cpython_sanity/issues/12 and the images
-        # are rebuilt
-        apt update
-        apt install -y llvm
-
-        python -m pip install setuptools pytest pytest-run-parallel
+        python -m pip install setuptools meson-python ninja pytest pytest-run-parallel
         CFLAGS="-g -O3 -fsanitize=thread" python -m pip install -v .
 
     - name: run tests under pytest-run-parallel

MANIFEST.in

@@ -1,6 +1,6 @@
 recursive-include src/cffi *.py *.h
 recursive-include src/c *.c *.h *.asm *.py win64.obj ffi.lib
-recursive-include testing *.py *.c *.h
+recursive-include testing *.py *.c *.h meson.build pyproject.toml *.txt
 recursive-include doc *.py *.rst Makefile *.bat
 recursive-include demo py.cleanup *.py embedding_test.c manual.c
 include AUTHORS LICENSE setup.py setup_base.py

doc/source/cdef.rst

@@ -1,9 +1,18 @@
-======================================
-Preparing and Distributing modules
-======================================
+=========================
+Preparing Wrapper Modules
+=========================
 
 .. contents::
 
+.. note::
+
+   A *wrapper module* is a Python module that uses CFFI to expose
+   functions and data from a C library, so that the rest of your
+   program can import it and call the library through the ``ffi`` and
+   ``lib`` objects. This page covers how to create wrapper modules.
+   See :ref:`cffi_gen_src_docs` for instructions on generating C source
+   code for wrapper modules.
+
 There are three or four different ways to use CFFI in a project.
 In order of complexity:
 
@@ -76,92 +85,6 @@ In order of complexity:
 
     # use ffi and lib here
 
-.. _distutils-setuptools:
-
-* Finally, you can (but don't have to) use CFFI's **Distutils** or
-  **Setuptools integration** when writing a ``setup.py``.  For
-  Distutils (only in out-of-line API mode; deprecated since
-  Python 3.10):
-
-  .. code-block:: python
-
-    # setup.py (requires CFFI to be installed first)
-    from distutils.core import setup
-
-    import foo_build   # possibly with sys.path tricks to find it
-
-    setup(
-        ...,
-        ext_modules=[foo_build.ffibuilder.distutils_extension()],
-    )
-
-  For Setuptools (out-of-line only, but works in ABI or API mode;
-  recommended):
-
-  .. code-block:: python
-
-    # setup.py (with automatic dependency tracking)
-    from setuptools import setup
-
-    setup(
-        ...,
-        setup_requires=["cffi>=1.0.0"],
-        cffi_modules=["package/foo_build.py:ffibuilder"],
-        install_requires=["cffi>=1.0.0"],
-    )
-
-  Note again that the ``foo_build.py`` example contains the following
-  lines, which mean that the ``ffibuilder`` is not actually compiled
-  when ``package.foo_build`` is merely imported---it will be compiled
-  independently by the Setuptools logic, using compilation parameters
-  provided by Setuptools:
-
-  .. code-block:: python
-
-    if __name__ == "__main__":    # not when running with setuptools
-        ffibuilder.compile(verbose=True)
-
-* Note that some bundler tools that try to find all modules used by a
-  project, like PyInstaller, will miss ``_cffi_backend`` in the
-  out-of-line mode because your program contains no explicit ``import
-  cffi`` or ``import _cffi_backend``.  You need to add
-  ``_cffi_backend`` explicitly (as a "hidden import" in PyInstaller,
-  but it can also be done more generally by adding the line ``import
-  _cffi_backend`` in your main program).
-
-Note that CFFI actually contains two different ``FFI`` classes.  The
-page `Using the ffi/lib objects`_ describes the common functionality.
-It is what you get in the ``from package._foo import ffi`` lines above.
-On the other hand, the extended ``FFI`` class is the one you get from
-``import cffi; ffi_or_ffibuilder = cffi.FFI()``.  It has the same
-functionality (for in-line use), but also the extra methods described
-below (to prepare the FFI).  NOTE: We use the name ``ffibuilder``
-instead of ``ffi`` in the out-of-line context, when the code is about
-producing a ``_foo.so`` file; this is an attempt to distinguish it
-from the different ``ffi`` object that you get by later saying
-``from _foo import ffi``.
-
-.. _`Using the ffi/lib objects`: using.html
-
-The reason for this split of functionality is that a regular program
-using CFFI out-of-line does not need to import the ``cffi`` pure
-Python package at all.  (Internally it still needs ``_cffi_backend``,
-a C extension module that comes with CFFI; this is why CFFI is also
-listed in ``install_requires=..`` above.  In the future this might be
-split into a different PyPI package that only installs
-``_cffi_backend``.)
-
-Note that a few small differences do exist: notably, ``from _foo import
-ffi`` returns an object of a type written in C, which does not let you
-add random attributes to it (nor does it have all the
-underscore-prefixed internal attributes of the Python version).
-Similarly, the ``lib`` objects returned by the C version are read-only,
-apart from writes to global variables.  Also, ``lib.__dict__`` does
-not work before version 1.2 or if ``lib`` happens to declare a name
-called ``__dict__`` (use instead ``dir(lib)``).  The same is true
-for ``lib.__class__``, ``lib.__all__`` and ``lib.__name__`` added
-in successive versions.
-
 
 .. _cdef:
 
@@ -935,10 +858,10 @@ steps.
 
 and *if* the "stuff" part is big enough that import time is a concern,
 then rewrite it as described in `the out-of-line but still ABI mode`__
-above.  Optionally, see also the `setuptools integration`__ paragraph.
+above.  Optionally, see also the :ref:`C source generation <cffi_gen_src_docs>`
+documentation.
 
 .. __: out-of-line-abi_
-.. __: distutils-setuptools_
 
 
 **API mode** if your CFFI project uses ``ffi.verify()``:
@@ -951,16 +874,15 @@ above.  Optionally, see also the `setuptools integration`__ paragraph.
     ffi.cdef("stuff")
     lib = ffi.verify("real C code")
 
-then you should really rewrite it as described in `the out-of-line,
-API mode`__ above.  It avoids a number of issues that have caused
+then you should really rewrite it as described in `the out-of-line, API
+mode`__ above.  It avoids a number of issues that have caused
 ``ffi.verify()`` to grow a number of extra arguments over time.  Then
-see the `distutils or setuptools`__ paragraph.  Also, remember to
-remove the ``ext_package=".."`` from your ``setup.py``, which was
-sometimes needed with ``verify()`` but is just creating confusion with
-``set_source()``.
+see the :ref:`C source generation <cffi_gen_src_docs>`
+documentation.  Also, remember to remove the ``ext_package=".."`` from
+your ``setup.py``, which was sometimes needed with ``verify()`` but is
+just creating confusion with ``set_source()``.
 
 .. __: out-of-line-api_
-.. __: distutils-setuptools_
 
 The following example should work both with old (pre-1.0) and new
 versions of CFFI---supporting both is important to run on old