How to build a C extension in keeping with PEP 517, i.e. with pyproject.toml instead of setup.py?

Question

I want to build a C extension for CPython. I could do it traditionally with a setup.py file. However, for the reasons mentioned in PEP 517, I would prefer a declarative approach using a pyproject.toml. I understand that setuptools is the only build backend that can build C extensions on all relevant platforms. In fact, I am unaware of any backend capable of building C extensions at all alongside the outdated distutils.

Against this background, a common setup.py would look like this:

from setuptools import setup, Extension
kwargs = dict(
    name='mypackage',
    # more metadata
    ext_modules=[
        Extension('mypackage.mymodule', ['lib/mymodule.c',
                                         'lib/mypackage.c',
                                         'lib/myalloc.c'],
                  include_dirs=['lib'],
                  py_limited_api=True)])

setup(**kwargs)

Now, the challenge is to put the above into a pyproject.toml plus a setup.cfg.

The setuptools docs suggest a pyproject.toml like this:

[build-system]
requires = [
    "setuptools >=52.0",
        'wheel >= 0.36']
build-backend = "setuptools.build_meta"

See

• https://setuptools.readthedocs.io/en/latest/build_meta.html and
• https://setuptools.readthedocs.io/en/latest/userguide/declarative_config.html#declarative-config

Further, the actual metadata should go into setup.cfg. However, I haven't found any explanation on how to translate the ext_modules kwarg, in particular the Extension() call, into setup.cfg syntax.

Answer 1

pyproject.toml is not strictly meant to replace setup.py, but rather to ensure its correct execution if it's still needed (see PEP 517 and my answer here):

Where the build-backend key exists, this takes precedence and the source tree follows the format and conventions of the specified backend (as such no setup.py is needed unless the backend requires it). Projects may still wish to include a setup.py for compatibility with tools that do not use this spec.

While setuptools plans to move everything from a script into a config file, it's not always possible:

There are two types of metadata: static and dynamic.

• Static metadata (setup.cfg): guaranteed to be the same every time. This is simpler, easier to read, and avoids many common errors, like encoding errors.
• Dynamic metadata (setup.py): possibly non-deterministic. Any items that are dynamic or determined at install-time, as well as extension modules or extensions to setuptools, need to go into setup.py.

Static metadata should be preferred and dynamic metadata should be used only as an escape hatch when absolutely necessary.

In fact, setuptools will still use an imaginary setup.py if it doesn't exist.

Note: Since version 61.0.0 setuptools allows to specify project metadata and other configuration options in pyproject.toml file. Its use looks more attractive as this file has another, more useful function and allows to specify most of the metadata in standardized, tool-agnostic way.

---

With that being said, if you want to stick as much as possible to a static way, you could move everything you can into pyproject.toml file and leave the rest in setup.py for now:

pyproject.toml

[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"

[project]
name = "mypackage"
# more metadata

setup.py

from setuptools import setup, Extension

setup_args = dict(
    ext_modules = [
        Extension(
            'mypackage.mymodule',
            ['lib/mymodule.c', 'lib/mypackage.c', 'lib/myalloc.c'],
            include_dirs = ['lib'],
            py_limited_api = True
        )
    ]
)
setup(**setup_args)

Answer 2

Below is a hack which can be used to build ext with pyproject.toml without depending on setup.py

pyproject.toml

[tool.setuptools]
py-modules = ["_custom_build"]

[tool.setuptools.cmdclass]
build_py = "_custom_build.build_py"

_custom_build.py

from setuptools import Extension
from setuptools.command.build_py import build_py as _build_py

class build_py(_build_py):
    def run(self):
        self.run_command("build_ext")
        return super().run()

    def initialize_options(self):
        super().initialize_options()
        if self.distribution.ext_modules == None:
            self.distribution.ext_modules = []

        self.distribution.ext_modules.append(
            Extension(
                "termial_random.random",
                sources=["termial_random/random.c"],
                extra_compile_args=["-std=c17", "-lm"],
            )
        )