Why does import work with Python nose when test directory contains init.py?

Question

Consider the following project structure:

a.py
test/
  test_a.py

with test_a.py importing module a:

import a

As expected, running nosetests in the test directory results in import error:

ERROR: Failure: ImportError (No module named a)

However, I noticed that adding an empty __init__.py file to the test directory makes import work with nosetests (but not when you run test_a.py with Python). Could you explain why?

I understand that adding __init__.py makes test a package. But does it mean that import includes the directory containing the package in the lookup?

http://stackoverflow.com/questions/448271/what-is-init-py-for — Mir, Jun 05 '15 at 17:35
@Mir I understand what `__init__.py` for. I don't understand why it affects import of other modules. — vitaut, Jun 05 '15 at 17:36
Because if it wasn't in there, then you could import any directory by accident, which would be next to useless. Its a programmatic check of sorts. — Mmm Donuts, Jun 05 '15 at 17:41
@SomeDeveloper But I'm not importing a package. I'm importing module `a`. — vitaut, Jun 05 '15 at 17:42
People need to read OPs question a bit more carefully. Why would making `test` a package suddenly make `import a` look up `a` in the scope of the *parent* module? — Lukas Graf, Jun 05 '15 at 17:43
Its still in a directory though, is it not? If you want to just... import a, it still needs to be in a python package. It can't stand alone. Ah, you're wondering how it finds it in parent directory. Import has a search functionality built in. Im assuming it would start in the parent directory if the file isn't in the current directory. — Mmm Donuts, Jun 05 '15 at 17:44
@vitaut the Python version is very relevant here (regarding import mechanics). Are you on Python 2 or 3? — Lukas Graf, Jun 05 '15 at 17:46
@LukasGraf This is on Python 2.7. This is probably more of a question about Nose than Python, because running `test_a.py` with Python give import error in both cases. Probably some `nosetests` does some magic with the import paths but I couldn't find anything about it in the docs. — vitaut, Jun 05 '15 at 17:50
@LukasGraf I don't think the behaviour is different between python 2 and 3 here. — wim, Jun 05 '15 at 18:07
@vitaut Checked in the source code and found out why. Sorry I didn't read your questions closely the first time. — Mir, Jun 05 '15 at 18:13
@wim I'm not positive it is (or whether this is even a Python issue rather than a side effect of some test discovery magic). But things did change regarding explicit vs. implicit relative imports between 2 and 3, so I just thought it to be helpful to get the question of the Python version out of the way early on. — Lukas Graf, Jun 05 '15 at 18:17

wim · Accepted Answer · 2015-06-05T18:11:54.140

5

The presence of an __init__.py file in the directory transforms test from just a plain old directory into a python package. This has an effect on sys.path.

Modify your test_a.py module like this:

import sys

def test_thing():
    for i, p in enumerate(sys.path):
        print i, p

try:
    import a
except ImportError:
    print('caught import error')

Then try running nosetests -s from the test directory, with and without an __init__.py in there.

Note: it is the test runner that munges sys.path. And that is documented in the second "Note" of this section here (thanks @davidism). You won't see any change there just by running python test_a.py with and without the package structure.

edited Jun 05 '15 at 18:11

answered Jun 05 '15 at 17:36

wim

338,267
99
616
750

3

I understand that, but how does it affect import lookup? – vitaut Jun 05 '15 at 17:40
Right. I've added more detail. – wim Jun 05 '15 at 17:47
Yes, and they explicitly recommend to *not* put the `__init__.py` in the tests directory. You should set up `PYTHONPATH` in your virtualenv. – wim Jun 05 '15 at 17:51
1

Thanks for the suggestion, @wim. The first entry in sys.path turned out to be the project directory which explains why import works. I wonder why nosetests puts it there. – vitaut Jun 05 '15 at 17:55

score 2 · Answer 2 · edited Aug 30 '15 at 13:26

I looked into the souce code of nose module and here's why.

def importFromPath(self, path, fqname):
    """Import a dotted-name package whose tail is at path. In other words,
    given foo.bar and path/to/foo/bar.py, import foo from path/to/foo then
    bar from path/to/foo/bar, returning bar.
    """
    # find the base dir of the package
    path_parts = os.path.normpath(os.path.abspath(path)).split(os.sep)
    name_parts = fqname.split('.')
    if path_parts[-1] == '__init__.py':
        path_parts.pop()
    path_parts = path_parts[:-(len(name_parts))]
    dir_path = os.sep.join(path_parts)
    # then import fqname starting from that dir
    return self.importFromDir(dir_path, fqname)

def importFromDir(self, dir, fqname):
    """Import a module *only* from path, ignoring sys.path and
    reloading if the version in sys.modules is not the one we want.
    """
    dir = os.path.normpath(os.path.abspath(dir))

In your case when importFromDir is called from importFromPath, 'dir' is the directory a level above from the __init__.py directory. So that's why adding __init__.py to your test makes 'import a' work

Why does import work with Python nose when test directory contains __init__.py?

2 Answers2

Why does import work with Python nose when test directory contains init.py?