19

The importlib_resources backport for Python < 3.7 of the importlib.resources standard library module has the following section in the setup.cfg file:

[options]
python_requires = >=2.7,!=3.0,!=3.1,!=3.2,!=3.3
setup_requires =
    setuptools
    wheel
install_requires =
    pathlib2; python_version < '3'
    typing; python_version < '3.5'
packages = find:

Why does setup_requires include setuptools? This does not seem to make sense since:

  • the first line of the setup.py file imports setuptools, so by the time the setup function is called and reads the setup.cfg file that instructs to install setuptools it is already too late to install setuptools:

    from setuptools import setup
setup()

  • setuptools is already installed on any fresh Python installation (well, only tested on Windows 10 and MacOS 10.15 with Python 3.8.0):

    $ python -V
Python 3.8.0
$ pip list
Package    Version
---------- -------
pip        19.2.3
setuptools 41.2.0
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
Géry Ogam
  • 6,336
  • 4
  • 38
  • 67
  • Python is included in every fresh python install as well, and yet there it is... – Mad Physicist Oct 23 '19 at 11:29
  • 3
    @MadPhysicist `python_requires` has a different semantics: it is not for installing Python, it is for preventing `pip` from installing a distribution for the wrong Python version (see https://packaging.python.org/guides/distributing-packages-using-setuptools/#python-requires). – Géry Ogam Oct 23 '19 at 12:21
  • 1
    It doesn't make any sense at all. Neither `setup_requires` nor `install_requires` are standard keywords and won't be recognized by `distutils`; they have any meaning for `setuptools` only. A simple test: `python -c "from distutils.core import setup; setup(install_requires=['foo'])" --version` will spit a `UserWarning: Unknown distribution option: 'install_requires'`. This means that if `setuptools` is not installed, `setup_requires` won't have any effect; thus listing it in dependencies is useless. – hoefling Oct 23 '19 at 23:31
  • `setuptools` is not installed on Python by default. It may be installed in some distributions such as you've found on Windows, but in the general case, it is not, and the packaging community is working to make setuptools fully optional. – Jason R. Coombs Jan 15 '20 at 23:14

3 Answers3

10

No, setuptools should not be included in setup_requires, according to PEP 518 (bold emphasis mine):

Setuptools tried to solve this with a setup_requires argument to its setup() function [3]. This solution has a number of issues, such as:

  • No tooling (besides setuptools itself) can access this information without executing the setup.py, but setup.py can't be executed without having these items installed.
  • While setuptools itself will install anything listed in this, they won't be installed until during the execution of the setup() function, which means that the only way to actually use anything added here is through increasingly complex machinations that delay the import and usage of these modules until later on in the execution of the setup() function.
  • This cannot include setuptools itself nor can it include a replacement to setuptools, which means that projects such as numpy.distutils are largely incapable of utilizing it and projects cannot take advantage of newer setuptools features until their users naturally upgrade the version of setuptools to a newer one.
  • The items listed in setup_requires get implicitly installed whenever you execute the setup.py but one of the common ways that the setup.py is executed is via another tool, such as pip, who is already managing dependencies. This means that a command like pip install spam might end up having both pip and setuptools downloading and installing packages and end users needing to configure both tools (and for setuptools without being in control of the invocation) to change settings like which repository it installs from. It also means that users need to be aware of the discovery rules for both tools, as one may support different package formats or determine the latest version differently.
Géry Ogam
  • 6,336
  • 4
  • 38
  • 67
9

The accepted answer is mostly correct, but where PEP 518 says.

[The setup_requires mechanism] cannot include setuptools itself...

It's technically incorrect, and as importlib_resources demonstrates, it can actually include setuptools. The problem is that including setuptools in setup_requires serves mostly as documentation. It declares that setuptools is a build requirement (required to run setup.py), but it won't be capable of satisfying that requirement if it's not already satisfied.

But, the presence of setuptools in setup_requires is technically correct and does serve the purpose of declaring the requirement and asking setuptools to verify that the requirement is in fact installed (alongside other setup-time requirements).

It is, however, just a legacy artifact and doesn't provide that much value, and as can be seen in the question and answers, it does lead to confusion. The recommended, proper, approach is to use PEP 517 and 518 declarations and builders, but that part of the ecosystem hasn't matured yet, so setuptools vestiges will remain. Try not to let them bother you.

Jason R. Coombs
  • 41,115
  • 10
  • 83
  • 93
2

Why does setup_requires includes setuptools? This does not seem to make sense

Does not make sense at all. On the other hand it doesn't hamper anything so why not?

phd
  • 82,685
  • 13
  • 120
  • 165
  • Thanks for the feedback. But `importlib_resources` maintainers are core Python developers so it is unlikely that they did that accidentally. Maybe I just don't understand how `setup_requires` work. I have read somewhere that `setup_requires` is incompatible with `pip`. Is that true? – Géry Ogam Oct 23 '19 at 12:04
  • Nop, not true. See https://setuptools.readthedocs.io/en/latest/setuptools.html#new-and-changed-setup-keywords – phd Oct 23 '19 at 12:21
  • 2
    I don't believe it is true. What is actually happening is that _pip_ is slowly moving towards being _build system agnostic_. See [PEP 517](https://www.python.org/dev/peps/pep-0517/) for some info. Basically, pip will delegate all the build actions to _setuptools_, _flit_, _poetry_, or any other alternative. Interface between _pip_ and the build system will be the `pyproject.toml` file, see [PEP 518](https://www.python.org/dev/peps/pep-0518/). – sinoroc Oct 23 '19 at 12:26
  • 1
    @sinoroc Yes, since PEP 517-518 decouple `pip` from `setuptools`, you now have to specify the build tool in the pyproject.toml file. Which the `importlib_resources` module does by the way. But the module also specifies `setuptools` in the setup.cfg file, which does not appear to make sense since at that time it is too late. – Géry Ogam Oct 23 '19 at 16:35
  • 1
    Yes, I think we all agree, that it was probably a mistake, since there's not much point in having setuptools in the setup requirements. – sinoroc Oct 23 '19 at 17:05