Python imports structure

Question

I want to have this structure for my project:

requirements.txt
README.md
.gitignore
project/
    __init__.py
    project.py
    core/
        __init__.py
        base.py
    engines/
        __init__.py
        engine1.py
        engine2.py
    utils/
        __init__.py
        refine_data.py
        whatever.py

The application is run from project/project.py. However, I constantly get import errors when using relative or absolute imports.

Both engines need to import from project.core.base, the utils need to import from project.core.base as well, and project.py (the main file ran) needs to be able to import from engines.

Absolute imports don't work:

# engines/engine1.py
from project.core.base import MyBaseClass

which gives the error:

ImportError: No module named project.core.base

But if I try a relative import instead

# engines/engine1.py
from ..core.base import MyBaseClass

I get:

ValueError: Attempted relative import beyond toplevel package

I've seen other projects on Github structured similarly, but this seems to cause all sorts of problems. How do I get this to work?

sibling packages should be able to just import from each other. so engines can just import core. — pvg, Dec 14 '15 at 18:17

Patrick Carroll · Answer 1 · 2015-12-14T18:27:01.690

4

Take a look at your sys.path. It's likely that the top project directory is in the python path, and it sees your sub-packages (ie. utils, engines, etc.) as separate packages, which is why it's giving you an error that you're trying to import from outside your package when doing relative imports, and absolute imports don't work because it can't find the top project directory because it's not under any of the python paths.

The directory above the top project directory is what needs to be added to the python path.

Ex.

/path/is/here/project/core/...

# Add this to the PYTHONPATH
/path/is/here

edited Dec 14 '15 at 18:27

answered Dec 14 '15 at 18:21

Patrick Carroll

71
3

Should I just do a `sys.path.append()` within the script itself? – Bob Dylan Dec 14 '15 at 18:34
You certainly can do that. Is your package meant to be distributed and installed? If so, you should look into creating a `setup.py` script for your package and installing it, which make sure it is in the pythonpath – Patrick Carroll Dec 14 '15 at 18:55
I'm just trying to run `project/project.py` from the command line. But I'll be sure to make a `setup.py` if I distribute it (I'll have to Google how, I don't know how to do it). – Bob Dylan Dec 14 '15 at 19:24
If you're running `project.py` directory, realize that you cannot do relative imports from there. Also, currently, whenever you run a python file, it automatically adds the directory it's in to the python path. – Patrick Carroll Dec 14 '15 at 19:26
1

@BobDylan See this answer for a better explanation of using a script to run your project and where to place it -- http://stackoverflow.com/a/22183108/5679120 – Patrick Carroll Dec 14 '15 at 21:23
Thanks! I appreciate the link – Bob Dylan Dec 14 '15 at 21:50
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/97905/discussion-between-patrick-carroll-and-bob-dylan). – Patrick Carroll Dec 14 '15 at 22:31
Unfortunately my employer blocks chat. – Bob Dylan Dec 14 '15 at 22:46
ha, I just clicked on the link to see what it would do. Didn't realize it would add a comment – Patrick Carroll Dec 15 '15 at 17:55

score 2 · Accepted Answer · answered Dec 14 '15 at 18:28

Try to use these imports:

engine1.py:

from core import base

refine_data.py:

from core import base

project.py

from engines import engine1

if you use pycharm mark project directory as sources root and then try to run project.py. If you don't use pycharm you can run project.py by going to project directory and running command:

python project.py

Python imports structure

2 Answers2