90

I have a project directory structure as follows (which I think is pretty standard):

my_project
    setup.py
    mypkg
        __init__.py
        foo.py
    tests
        functional
            test_f1.py
        unit
            test_u1.py

I'm using py.test for my testing framework, and I'd expect to be able to run py.test tests when in the my_project directory to run my tests. This does indeed work, until I try to import my application code using (for example) import mypkg in a test. At that point, I get the error "No module named mypkg". On doing a bit of investigation, it appears that py.test runs the tests with the directory of the test file in sys.path, but not the directory that py.test was run from.

In order to work around this, I have added a conftest.py file to my tests directory, containing the following code:

import sys, os

# Make sure that the application source directory (this directory's parent) is
# on sys.path.

here = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, here)

This seems to work, but is it a good way of making sure that the tests see the application code? Is there a better way of achieving this, or am I doing something wrong in how I have my project structured?

I've looked at some other projects that use py.test (for example, pip) but I can't see code that does anything like this, and yet running py.test tests seems to work there. I don't know quite why, but I'm worried that they may have achieved the same result in a simpler way.

I've looked in the py.test documentation, but I can't see an explanation of this problem or what the recommended approach is to deal with it.

Paul Moore
  • 6,569
  • 6
  • 40
  • 47

4 Answers4

72

As you say yourself py.test basically assumes you have the PYTHONPATH setup up correctly. There are several ways of achieving this:

  • Give your project a setup.py and use pip install -e . in a virtualenv for this project. This is probably the standard method.

  • As a variation on this if you have a virtualenv but no setup.py use your venv's facility to add the projects directory on sys.path, e.g. pew add . if you use pew, or add2virtualenv . if you use virtualenv and the extensions of virtualenvwrapper.

  • If you always like the current working directory on sys.path you can simply always export PYTHONPATH='' in your shell. That is ensure the empty string on on sys.path which python will interpret as the current working direcotry. This is potentially a security hazard though.

  • My own favourite hack, abuse how py.test loads conftest files: put an empty conftest.py in the project's top-level directory.

The reason for py.test to behave this way is to make it easy to run the tests in a tests/ directory of a checkout against an installed package. If it would unconditionally add the project directory to the PYTHONPATH then this would not be possible anymore.

Brian Peterson
  • 2,800
  • 6
  • 29
  • 36
flub
  • 5,953
  • 27
  • 24
  • 3
    Just adding to the first point: if you don't have virtualenv, you can use "python setup.py develop" to the same effect. – Bruno Oliveira Jan 07 '14 at 14:47
  • Thanks, I was aware of the various options for PYTHONPATH hacking. Generally, though, I don't tend to use them, I just run Python from the project root directory and it works. IIRC, unittest also works without any PYTHONPATH hacking (at least, I've never done anything complex enough to break it yet :-)) I guess with py.test I just have to bite the bullet and get used to setting my PYTHONPATH properly. – Paul Moore Jan 07 '14 at 15:12
  • 9
    the conftest.py hack is a great idea!!! Too bad it's an abuse, IOW it might go away :( – lab419 Feb 17 '14 at 14:17
  • 6
    Install something to run tests??? Using more tools just to get a test runner working??? Manipulating Python path, which is never recommended and has to be changed again whenever something is moved in the project??? -- No! Adding an empty file seems like the least hacky solution! I don't know why it works, but I am glad it does. The other options seem just wrong, until someone explains to me why I'd need to install something to test it, when I have all the code in place already. – Zelphir Kaltstahl Nov 27 '16 at 13:00
  • 1
    @PascalVKooten It was discussed a little more recently over [here](https://github.com/pytest-dev/pytest/issues/4699) about why this `conftest.py` hack "works", and also why `pytest` doesn't "undo" this `sys.path` modification after importing. – wim Apr 04 '19 at 05:55
  • 2
    @Zelphir As for why you want to install something first to test it, well, you can think of it as a way to also test the *installer* itself. Without this step, it's completely possible to have a passing test suite and yet have a broken installer (and you end up to accidentally publish a broken package). – wim Apr 04 '19 at 05:56
  • 1
    @wim Hmmm, OK, I can live with that explanation, thank you. I guess it depends on how one runs ones code. If always running from source, there would be no need for such test. If running after installing as package, it seems justified to have it installed for testing. – Zelphir Kaltstahl Apr 04 '19 at 06:06
  • 1
    This is one mess of an issue...After endless reading it was @wim who managed to boil down the whole rational. – bad_coder Mar 07 '20 at 23:58
  • 2
    Just providing some confirmation for others in the same situation as me and going slowly insane: for some projects, neither of these hacks (`__init__.py` or `conftest.py`) actually work, the only thing that makes pytest happy is what's shown in the question. You can adjust to relative imports to make pytest run, but then when running the program *outside* of pytest it doesn't work. Why can't pytest simply use the same PYTHONPATH as python? What on **earth** is the point of making it different, except causing situations like this?! – durka42 May 07 '21 at 23:47
  • Addendum: if still can't resolve modules, after installing the package into an virtual env, check that the `pytest` executable is coming from that same virtual env and not from somewhere else in the system; to be sure invoke via `python -m pytest` instead of `pytest`. – alexei Nov 17 '22 at 20:33
33

The easy way of doing it is, in terminal/cmd change directory to where the parent directory is, (e.g. in this case cd C:/.../my_project).

Then run: python -m pytest --cov=mypkg tests

No need to mess with the PYTHONPATH environment variable. By running with python -m pytest, it automatically adds the current directory to sys.path.

wim
  • 338,267
  • 99
  • 616
  • 750
A H
  • 2,164
  • 1
  • 21
  • 36
  • 11
    +1 This is a legit and documented feature, not just an accident. See https://docs.pytest.org/en/latest/pythonpath.html#invoking-pytest-versus-python-m-pytest – wim Apr 04 '19 at 05:41
  • 1
    Worth following that link, which explains that `python -m pytest` departs from the behavior of plain `pytest` in this one regard. – Martin Dorey Feb 04 '23 at 00:59
10

The answer is actually much easier, as seen here.

All you need to do is add an __init__.py to your test directory and each of its sub directories, like so;

tests/__init__.py
tests/functional/__init__.py
tests/unit/__init__.py
Community
  • 1
  • 1
SleepyCal
  • 5,739
  • 5
  • 33
  • 47
  • 1
    This actually adds tests to the package. That may not always be desired. – Brian Bruggeman Mar 14 '16 at 18:08
  • 25
    –1 Because doing this is **explicitly discouraged** in the ["good practices" section of the pytest docs](http://doc.pytest.org/en/latest/goodpractices.html). – wim Feb 24 '17 at 21:56
  • 1
    @BrianBruggeman, I thought that `packages=find_packages(exclude=['contrib', 'docs', 'tests'])` in setup.py takes care of that? @wim, that doc now offers two scenarios - having tests **inside** or **outside** the app package, so this is now a valid option, I assume? – Alen Siljak Dec 28 '18 at 13:00
  • 1
    @AlenSiljak In practice, `/tests` is still encouraged and having a `tests` folder within the structure of the code is discouraged. There are still inconsistent behaviors with pytest's some of plugins that I've run into (I primarily use coverage, isort, flake8). That said, I've begun to put unit tests in `tests` folders within the code base and integration/system level tests within the top /tests folder and that has worked out okay for me. – Brian Bruggeman Dec 28 '18 at 19:41
6

Run pytest referencing a single/multiple packages

For 1 package

PYTHONPATH=$(pwd)/mypkg/ python3 -m pytest 
# or
PYTHONPATH=$(pwd)/mypkg/ python3 -m pytest  path/to/tests

For 2 packages (pkg1 pkg2)

# Use ; as a separator in windows
PYTHONPATH=/path/to/pkg1/:/path/to/pkg2/ python3 -m pytest tests
A H
  • 2,164
  • 1
  • 21
  • 36