28

I'm having a weird problem with tox, py.test, coverage and pytest-cov: when py.test with the --cov option is launched from tox, it seems to require an __init__.py file in the tests folder which is not immediately obvious.

While writing this post, I have kind of solved the initial problem by adding the aforesaid tests/__init__.py, but to this moment I don't fully understand why exactly it works or doesn't work, so I'm still asking for help. Please see below for details.

I've found a related question on SO but it only makes it more confusing because the answer seems to be opposite to what I've figured out so far: `py.test` and `__init__.py` files

See also the official docs here: py.test - Good Integration Practices (the very bottom of the page).

Simplified project structure:

setup.py
tox.ini
.coveragerc
project/
    __init__.py
    module1.py
    module2.py
    tests/
        __init__.py (optional, an empty file)
        test_module1.py
        test_module2.py

Relevant part of tox.ini:

[testenv:check]
commands = py.test --cov=project --cov-report=term
deps =
    pytest
    coverage
    pytest-cov

[pytest]
python_files = test_*.py
norecursedirs = .tox

Relevant part of .coveragerc:

[run]
branch = True
omit = project/tests/*

Now, the results:

  • py.test --cov=project --cov-report=term run from project root => correct coverage whether tests/__init__.py file is present or not.
  • tox -e check without tests/__init__.py => the tests are discovered and run, but I get a warning "Coverage.py warning: No data was collected." and the coverage is 0% for all modules
  • tox -e check with tests/__init__.py => correct coverage again.

It's not immediately obvious to me why the tests/__init__.py file has to be there (adding this empty file solved the initial problem) for the tox run, but it doesn't matter when you run the tests/coverage manually. Any ideas?

vvvvv
  • 25,404
  • 19
  • 49
  • 81
aldanor
  • 3,371
  • 2
  • 26
  • 26
  • 1
    See http://lists.idyll.org/pipermail/testing-in-python/2014-November/006224.html on how you should check coverage with tox. – schlamar Nov 24 '14 at 17:57

2 Answers2

53

Use --cov {envsitepackagesdir}/<your-package-name> in tox.ini.

Matt W
  • 6,078
  • 3
  • 32
  • 40
4

See: Using py.test with coverage doesn't include imports

I got rid of using pytest-cov and run coverage outright instead..

Also noticed with pytest, I did need the blank __init__.py in my test directory to function correctly. There is probably a reason for it somewhere.

I realize this is a couple of years old, but in case someone else comes across this..

reynoldsnlp
  • 1,072
  • 1
  • 18
  • 45
Brian Kruger
  • 41
  • 1