ImportError: No module named requests in 'pytest'

Question

root
| +-- demo
      |-->__init__.py
      |-->conftest.py
      |-->test.py

conftest.py

import pytest
def tear_down():
    print "\nTEARDOWN after all tests"

@pytest.fixture(autouse=True)
def set_up(request):
    print "\nSETUP before all tests"
    if request.cls.__name__ == 'TestClassA':
        return ["username", "password"]
    request.addfinalizer(tear_down)

test.py

#import requests # this is commented

class TestClassA:
    def test_1(self,set_up):
        print "test A1 called"
        print("username :-- %s and password is %s" % (set_up[0], set_up[1]))
    def test_2(self):
        print "test A2 called"

class TestClassB:
    def test_1(self):
        print "test B1 called"

pytest -s -v demo/test.py::TestClassA

This code works fine. observe the first line of test.py, it's commented. Now, if I run the same script with uncommenting import requests , getting below error

ImportError while importing test module 'some_path/../demo/test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python2.7/site-packages/six-1.11.0-py2.7.egg/six.py:709: in exec_
    exec("""exec _code_ in _globs_, _locs_""")
demo/test.py:1: in <module>
    import requests
E   ImportError: No module named requests

Executing without pytest, works fine (no import error) And also, if test.py calls the function of other module (which has import requests) throws same error. Is it conflict of request of pytest ? I really don't understand this, can you please help me ?

which python : /Library/Frameworks/Python.framework/Versions/2.7/bin/python pytest --version : 'imported from /usr/local/lib/python2.7/site-packages/pytest-3.8.2-py2.7.egg/pytest.pyc' , is this a reason for failure ?

pytest --version

This is pytest version 3.8.2, imported from /usr/local/lib/python2.7/site-packages/pytest-3.8.2-py2.7.egg/pytest.pyc
setuptools registered plugins:
  celery-4.0.2 at /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/celery/contrib/pytest.py
  hypothesis-3.8.2 at /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/hypothesis/extra/pytestplugin.pyc
  pytest-cov-2.4.0 at /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pytest_cov/plugin.py

requests is not installed for the interpreter/venv you are using. — Klaus D., Oct 09 '18 at 04:03
@KlausD. How does that impact here ? because, even from python interactive i could able to import requests successfully. — StackGuru, Oct 09 '18 at 04:06
nope, it's one and same /Library/Frameworks/Python.framework/Versions/2.7/bin/python — StackGuru, Oct 09 '18 at 04:09
So when you run `/Library/Frameworks/Python.framework/Versions/2.7/bin/python -c "import requests"`, do you get any output? — hoefling, Oct 09 '18 at 07:12
which python >> `/Library/Frameworks/Python.framework/Versions/2.7/bin/python` | pytest --version >> 'imported from /usr/local/lib/python2.7/site-packages/pytest-3.8.2-py2.7.egg/pytest.pyc' , is this a reason for failure ? — StackGuru, Oct 09 '18 at 07:34
@hoefling executing /Library/Frameworks/Python.framework/Versions/2.7/bin/python -c "import requests" , doesn't give any output. it's success. — StackGuru, Oct 09 '18 at 07:47
I think, you need to install the requests module. you can install it by pip install requests — JON, Oct 09 '18 at 08:00
If you are sure you are using the same interpreter on running the program and tests, it must be a `sys.path` difference. Running `/Library/Frameworks/Python.framework/Versions/2.7/bin/python -c "import requests; print(requests.__file__)"` should give you the path to where `requests` are imported from, probably some `/usr/local/lib/python2.7/site-packages` dir. Now add the lines to the test script: `import sys` and `print(sys.path)`, run the test script via `pytest -s testscript.py` and check whether the dir above is present in `sys.path` list. — hoefling, Oct 09 '18 at 08:39
btw, if you're unsure you're using the same interpreter, running tests via `pytest -v` will output the path to the used interpreter along some other info, so you may check that easily. — hoefling, Oct 09 '18 at 08:45
@hoefling which python : /Library/Frameworks/Python.framework/Versions/2.7/bin/python pytest --version : 'imported from /usr/local/lib/python2.7/site-packages/pytest-3.8.2-py2.7.egg/pytest.pyc' , is this a reason for failure ? — StackGuru, Oct 09 '18 at 08:49
/Library/Frameworks/Python.framework/Versions/2.7/bin/pip install pytest Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.7/bin/pip", line 7, in from pip import main ImportError: cannot import name main — StackGuru, Oct 09 '18 at 08:51
Why should this be an error? It only means you've installed `pytest` via `easy_install` and not `pip`, but this is not an error per se. It still looks to me that you have installed two different interpreters, one in `/Library/Frameworks/Python.framework/Versions/2.7` (presumably by downloading a `pkg` installer from the python.org website), and a brewed one (installed via `brew install python2`). You need to sort that out first. — hoefling, Oct 09 '18 at 08:54
i just uninstalled the one from brew unlink python@2 , but still same — StackGuru, Oct 09 '18 at 09:37
earlier i had verified that, installing requests module from /usr/local/bin/pip install requests. This worked. but not the root cause i guess. because, when i run the scripts again from pytest it throws different error for different packages. whereas same script without pytest works totally fine — StackGuru, Oct 09 '18 at 09:38
No, it is the root error. You have installed your program's dependencies for one interpreter and are running tests with another interpreter that does not have any of the dependencies installed. Run the script with `/usr/local/bin/python` (or whatever interpreter Homebrew installs) and you will get the same errors as in the test mode. Again, run the tests via `/Library/Frameworks/Python.framework/Versions/2.7 -m pytest ...` (assuming this interpreter version has `pytest` available), and the tests will run just fine. — hoefling, Oct 09 '18 at 10:03
echo $PYTHONPATH :/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python , i'm really confused where another interpreter has been set for this. How can i avoid this conflict ? — StackGuru, Oct 09 '18 at 14:12

score 1 · Answer 1 · answered Oct 09 '18 at 08:05

1

Add the following lines of code to your script and run it :

from distutils.sysconfig import get_python_lib
print(get_python_lib())

Now check the output, you will get some path printed which points to the exact location of the packages the python interpreter is currently using

e.g. "/usr/lib/python2.7/dist-packages"

cd(change directory) to the above path and ls(list directory) to check if package exists ; if not :

sudo pip3 install requests -t . # dot indicates current directory

or else if you have a requirements.txt file then you could try:

sudo pip3 install -r requirements.txt -t "/usr/lib/python2.7/dist-packages" 
#try this from the directory where  "requirements.txt" file exists

Now run your scripts and please let me know if it worked

answered Oct 09 '18 at 08:05

DeWil

362
3
10

earlier i had verified that, installing requests module from /usr/local/bin/pip install requests. This worked. but not the root cause i guess. because, when i run the scripts again from pytest it throws different error for different packages. whereas same script without pytest works totally fine. – StackGuru Oct 09 '18 at 08:47
as i said earlier, which python : /Library/Frameworks/Python.framework/Versions/2.7/bin/python pytest --version : 'imported from /usr/local/lib/python2.7/site-packages/pytest-3.8.2-py2.7.egg/pytest.pyc' , is this a reason for failure ? – StackGuru Oct 09 '18 at 08:47
/usr/local/bin/pip install requests ---> Don't run this. Add the lines "from distutils.sysconfig import get_python_lib print(get_python_lib())" to your script and run it, because you want to know the path to the specified packages that the python interpreter is using. May be your pip is pointing to site-packages but while running the code the interpreter points to dist-packages which may not contain "requests" – DeWil Oct 09 '18 at 14:06
when i add get_python_lib to the code and run the with pytest, it outputs '/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages'. the same code, when i run normally python test.py (i.e., without pytest), it outputs '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages' – StackGuru Oct 09 '18 at 14:27
This was the problem i mentioned, now follow the steps i have written in my answer and install requests to the folder you mentioned and you are good to go – DeWil Oct 09 '18 at 14:33
there are so many such errors, if it's fulfilled for 'requests' it errors for other modules. I do get these errors only when i run with pytest. and most importantly it errors out different path – StackGuru Oct 09 '18 at 14:45
import paramiko /usr/local/lib/python2.7/site-packages/paramiko/__init__.py:31: in from paramiko.transport import SecurityOptions, Transport /usr/local/lib/python2.7/site-packages/paramiko/transport.py:56: in from paramiko.dsskey import DSSKey /usr/local/lib/python2.7/site-packages/paramiko/dsskey.py:27: in from cryptography.hazmat.primitives.asymmetric.utils import (/usr/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/asymmetric/utils.py:9: in from asn1crypto.algos import DSASignature E ImportError: No module named asn1crypto.algos – StackGuru Oct 09 '18 at 14:46
why can't i change it to respective path, instead of installing modules at different locations ? – StackGuru Oct 09 '18 at 14:47
I am not aware as to how to change paths, however you could create a requirements text file with all the required packages mentioned in it and use pip as mentioned in my answer and download all packages in 1 go. – DeWil Oct 09 '18 at 18:14
check this link https://stackoverflow.com/questions/38428735/importerror-no-module-named-selenium?rq=1 – DeWil Oct 09 '18 at 18:28

ImportError: No module named requests in 'pytest'

1 Answers1