Ambiguity of import modules in python

Question

I have a Python library that looks like this (contains packages):

|-- python_lib/
    |-- plotting.py
    |-- verification.py
    |-- io.py
    |
    |-- ensemble/
        | -- plotting.py
        | -- verification.py

However, I am having problems with ambiguous module imports. In ensemble.plotting.py I want to import the verification.py module from the top level (python_lib):

# ensemble/plotting.py
import verification.obs as verobs

However, I get an import error as this tries to import the verification.py module from the ensemble directory, rather than the top level of the library.

I thought that the "abslute imports" feature would solve this:

from ..verification import obs

but I get this error:

ValueError: Attempted relative import beyond toplevel package

How do I target python_lib/verification.py as an import from python_lib/ensemble/*.py without trying to import the local package version?

I am using Python version 2.7.

@leovp python_lib is not a package, it just happens to the the base directory that the source is in. — ccbunney, Mar 20 '17 at 13:19
Try to add each sub-directory empty file called __init__.py. look at this https://stackoverflow.com/questions/11536764/how-to-fix-attempted-relative-import-in-non-package-even-with-init-py?rq=1 also — nivhanin, Mar 20 '17 at 13:25
how about make the directories pagckages by adding empty __init__.py — hukaixuan, Mar 20 '17 at 13:25
@nivhanin Each sub-directory does have an __init__.py file. The top level doesn't as it is not a package (just a directory). — ccbunney, Mar 20 '17 at 13:26
sounds good than, I'm pretty sure that the post I add can really help you. Have a look. — nivhanin, Mar 20 '17 at 13:28
Is there a `python_lib/__init__.py` file? Else, you cannot import from it. Please update your file tree with these files for clarity. — Pierre de Buyl, Mar 20 '17 at 13:28
@PierredeBuyl My PYTHONPATH environment variable has a path to `python_lib`. The point is that `python_lib` might be called something else... — ccbunney, Mar 20 '17 at 13:30
Well then `..` is above the top-level for the internal import for the package and you cannot import from it. — Pierre de Buyl, Mar 20 '17 at 13:35

score 1 · Answer 1 · answered Mar 20 '17 at 13:36

All your directories should contain files __init__.py, or they won't be recognized as packages.
Your directory python_lib must be a known path. You said in a comment that your PYTHONPATH contains a path to python_lib, so that seems good.
If you start with e.g. python_lib/ensemble/verification.py as your main entry point, then the directory python_lib/ensemble shadows your directory python_lib, and you have to manually tell Python that your script belongs to that package.

Like this:

 if __name__ == "__main__" and __package__ is None:
     __package__ = "pythonlib.ensemble"
     import pythonlib

After that is done, this should work:

from ..verification import obs

ddnomad · Answer 2 · 2017-03-20T13:46:49.000

It's a hacky solution that would work but is generally discouraged.

You can modify python path in place when you have to omit conflicts. Like that:

import sys  # needed to make it work

tmp = sys.path[0]
sys.path = sys.path[1:]  # ugly and hacky thingy

# import stuff you need (would work with __init__.py in a base dir)
from python_lib import verification
from ..verification.obs import obs

# fix path back
sys.path.insert(0, tmp)

I just tested this pattern and it seems to work (Python 3.6.0)

score 0 · Answer 3 · edited Mar 20 '17 at 18:40

There are some rules that has to be followed while creating packages and modules.

Make sure there is __init__.py file as a part of your package. Every python package is expected to have this file
While the module is looked it happens in 3 ways.

A very good documentation is available Module-search-path

Usually while building projects it a practice to create a lib package and add sub directories based on different categories and add the main directory to the PYTHONPATH variable. This make the module available for other programs.

Ambiguity of import modules in python

3 Answers3