Passing command line arguments to pip install

Question

I'm currently working on a Python project that is importing a Fortran module. The setup.py looks similar to that

from numpy.distutils.core import Extension
from numpy.distutils.core import setup

ext = Extension(
    name = "fortran_module",
    sources = ["fortran_module.f90"],
    extra_f90_compile_args = ["-some -compile -arguments"]
)

setup(
    ...,
    ...,
    ...,
    ext_modules = [ext],
    ...,
    ...,
    ...
)

I'm currently installing it with pip install -e ., which works fine. But sometimes I need to change the extra_f90_compile_args and I would like to do give them as command line argument during installation with pip rather than changing the setup.py file. Something like this for example:

pip install -e --compile_args="-O3 -fbacktrace -fbounds-check -fopenmp" .

Is it possible to pass command line arguments via pip into the setup.py file?

Related to that, I would also like to pass different setup options into setup.py. For example

pip install -e --setup=setup1 .

or

pip install -e --setup=setup2 .

And depending on whether setup1 or setup2 or none of them is passed, I would like to include different Fortran source files and compile them with different extra_f90_compile_args.

Is this possible?

Edit: Let's consider the following example: The Fortran module is parallelized with OpenMP. Now I want the user to decide, whether to compile parallel or not. Maybe the OpenMP libraries are not available and the user needs to compile the serial version.

My setup.py now looks as follows

from numpy.distutils.core import Extension
from numpy.distutils.core import setup
import os
from setuptools.command.install import install as _install

extra_f90_compile_args = ""
extra_link_args = ""

class install(_install):
    user_options = _install.user_options + [('build=', None, None)]

    def initialize_options(self):
        _install.initialize_options(self)
        self.build = None

    def finalize_options(self):
        _install.finalize_options(self)

    def run(self):
        global extra_f90_compile_args
        global extra_link_args
        if(self.build == "parallel"):
            extra_f90_compile_args += "-fopenmp"
            extra_link_args += "-lgomp"
            os.makedirs("~/test/")
        _install.run(self)

ext = Extension(
    name="test_module.fortran_module",
    sources=["test_module/fortran_module.f90"],
    extra_f90_compile_args=[extra_f90_compile_args],
    extra_link_args=[extra_link_args]
)

setup(
    name="test_module",
    packages=["test_module"],
    ext_modules=[ext],
    cmdclass={'install': install}
)

If I install this with

pip install . --install-option="--build=parallel"

it is executing code in the if-block. I created the test/ directory just to check this. If build is not given or different from parallel, the test/ directory is not created.

However, the code is not compiled with OpenMP. I think this is, because the Extension object is created before the call of setup(), where the arguments get evaluated. I would like to first evaluate the arguments, then create the Extension object depending on the arguments, and then call setup().

How can I do that?

I'm aware of this and several other questions. However, I cannot get it to work in my case. See edit. — Sebastian, Nov 10 '17 at 16:41
In the added code there is indentation problem — methods are not properly indented. Please fix. — phd, Nov 10 '17 at 17:00
Fixed. My problem is that I would like to evaluate the arguments in the `setup.py` file **before** `setup()` is called. — Sebastian, Nov 10 '17 at 17:29
Aha, I see — you need to evaluate `ext` from the global vars. — phd, Nov 10 '17 at 18:54

NegatioN · Answer 1 · 2019-10-23T11:22:29.073

Dealing with both --install-option and --global-option seems very messy if there's more than one option in my opinion. I would like to add this real life solution

Take in environment variables instead, and set them at install time.

In setup.py:

import os
config_var = os.environ.get("MY_PARAM", None)
if config_var:
    do_something()
setup()

and call pip like this: MY_PARAM="Some_value" pip install my_module

score 3 · Accepted Answer · answered Nov 20 '17 at 13:27

After digging through the source code, I can answer this now.

The setup.py file:

from numpy.distutils.command.install import install as _install
from numpy.distutils.core import Extension
from numpy.distutils.core import setup

extF = Extension(name="test_module.fortran_module",
                 sources=["test_module/fortran_module.f90"])
compile_args_parallel = "-fopenmp"
link_args_parallel = "-lgomp"

class install(_install):
    user_options = _install.user_options + [('build=', None, None)]

    def initialize_options(self):
        _install.initialize_options(self)
        self.build = None

    def finalize_options(self):
        _install.finalize_options(self)
        if(self.build == "parallel"):
            for ext in self.distribution.ext_modules:
                if(ext.name == "test_module.fortran_module"):
                    ext.extra_f90_compile_args = compile_args_parallel
                    ext.extra_link_args = link_args_parallel

    def run(self):
        _install.run(self)

setup(
    name="test_module",
    packages=["test_module"],
    ext_modules=[extF],
    cmdclass={"install": install}
)

Installing this with

pip install .

installs the serial version and

pip install . --install-option="--build=parallel"

installs the parallel version of the Fortran module.

I am also creating a package with some command line args. Can you tell me how to make it ? — babygame0ver, Oct 07 '18 at 07:20
`--install-option` was deprecated in [pip 22.3](https://pip.pypa.io/en/stable/news/#v22-3) and removed in [pip 23.1](https://pip.pypa.io/en/stable/news/#v23-1). — wim, Jul 28 '23 at 19:19

Passing command line arguments to pip install

2 Answers2

Linked