0

I'm trying to launch a command in different Python virtualenvs using subprocess.check_call().

To activate the virtualenv (in a Python 2/3 agnostic manner), I simply append the path to my virtualenv bin (or Scripts under Windows) to the PATH and then call subprocess.check_call() with this modified environment. Like so:

environment = os.environ.copy()
environment['PATH'] = os.pathsep.join([bin_path, environment['PATH']])

subprocess.check_call("pip install -r dev_requirements.txt", env=environment)

However, what I notice is that pip installs everything in the system Python site-packages. If I change the check_call() for:

subprocess.check_call("pip install -r dev_requirements.txt", env=environment, shell=True)

Then suddenly pip operates in the virtualenv as expected.

What bothers me is that in both cases running where pip gives me the path to the virtualenv's pip first.

What could explain this odd behavior ?

jmunsch
  • 22,771
  • 11
  • 93
  • 114
ereOn
  • 53,676
  • 39
  • 161
  • 238

1 Answers1

2

PATH is not the first place where CreateProcess() used by Popen() on Windows looks for the executable. It may use pip.exe from the same directory as the parent python.exe process. The shell (cmd.exe) uses different rules. See Popen with conflicting executable/path.

To avoid the dependency; use the explicit full path to pip. You don't need to change the environment in this case:

import os
from subprocess import check_call

check_call([os.path.join(bin_path, 'pip.exe')] + 
           'install -r dev_requirements.txt'.split())
Community
  • 1
  • 1
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • Well, in my case its the user that specifies the command line to execute (and my `pip` example just happens to be one of the commands that fails without `shell=True`) so it's not my call to prepend the full path to pip. I guess using `shell=True` makes sense given that I'm trying to emulate the behavior a user would get by running the command in the terminal. Thanks for the explanation though, that was driving me crazy. – ereOn Feb 18 '15 at 14:51
  • instead of using the 'pip.exe' can't os.system('which pip') will be the best? – Mohit Sep 21 '15 at 05:57
  • @ahujamoh: `which pip` might return a wrong `pip` with the unmodified environment otherwise `call([sys.executable, '-m', 'pip', ..])` could be used. If you have an explicit path to the virtualenv; there is no reason to use `which` (even ignoring other reasons to avoid `which`). – jfs Sep 21 '15 at 06:33