1

I'm writing a source bundle (not a fully packaged module, but some scripts with dependencies) to be installed and executed inside a framework application (Specifically, Amazon SageMaker's TensorFlow serving container - running Python 3.5).

One of my dependencies is matplotlib, which in turn needs kiwisolver, which has C++ components.

It seems like my target container doesn't have wheel installed by default, because when I supply just a requirements.txt file I get the error described in "Why is python setup.py saying invalid command 'bdist_wheel' on Travis CI?".

I think I got it working by supplying a setup.py instead, with setup_requires=["wheel"] as advised in the answers to that Travis CI question.

My Python packaging-fu is weak, so my question is: Who should be specifying this dependency, because it seems like it shouldn't be me?

  • Should kiwisolver be advertising that it needs wheel?
  • Does a framework application/environment installing user code modules via requirements.txt have an implicit contract to make wheel available in the environment, for some reason in Python's packaging ethos?
  • Maybe it really is on me to know that, since I'm indirectly consuming a module like kiwisolver, my package requires wheel for setup and a straight pip install -r requirements.txt won't work?

Even better if somebody can explain whether this answer is changing with PEP 518 and the deprecation of setup_requires :S

dingus
  • 655
  • 1
  • 7
  • 18

1 Answers1

1

Usually wheel could be considered a build-time dependency and not an install-time dependency. But actually, wheel is just a way of distributing Python projects (libraries or applications), so it usually isn't a mandatory dependency.

The one system building the library (kiwisolver) might have a need to have the wheel tool installed. But if I am not mistaken recent versions of pip have wheel already bundled in, so nowadays there is often no need to install it explicitly.

In many cases there are wheels already built available on PyPI. But sometimes there are no wheels compatible with the target system (Python interpreter version, operating system, CPU bitness). In your case here, kiwisolver has a wide range of wheels available but not for Python 3.5.

So it seems like the system you want to install kiwisolver on, is not compatible with any of the wheels available on PyPI. So pip has to build it locally. Usually pip tries to build a wheel first, but as far as I know it's not a deal-breaker if a wheel cannot be built then pip usually just continues and installs the project without going to the wheel intermediary step.

But still pip has to be able to build the library, which might require some C/C++ compilers or that other unusual conditions are met on the local system. Which is why distributing libraries as wheel is very comfortable, since the build step is already done.

So to sum it up, from my point of view, no one really has to declare wheel as a dependency or install wheel unless they actually want to build wheels. But wheel really is just an intermediary optional step. It's a way of distributing Python projects (libraries or applications). I don't see the absolute need for adding wheel to setuptools' setup_requires (which is deprecated or on close to it) nor to pyproject.toml's build-system.requires, it's more of a (very common, and quasi standard) convenience.

Now what would I do in your situation?

  • Before installing from the requirements.txt file that contains kiwisolver (directly or indirectly) either make sure that pip is up-to-date or explicitly install wheel, and:
    • Use a version of Python for which wheels are already available on PyPI.
    • If you want to stay on Python 3.5:
      • Make sure the target system is able to build kiwisolver itself (maybe it requires a C/C++ compiler plus some other native libraries).
sinoroc
  • 18,409
  • 2
  • 39
  • 70