71

I was playing the the Python's import system in order to understand better how it works, and I encountered another problem. I have the following structure 

pkg/
    __init__.py
    c.py
    d.py

    subpkg/
        __init__.py
        a.py
        b.py

Inside a.py I have the following code:

from . import b
from .. import d

And inside c.py I have the following:

import subpkg.a

Now I receive the following error:

ValueError: attempted relative import beyond top-level package

But why? How can I solve it? I am running c.py from the IDLE, and pkg should be considered a package, since it has the __init__.py file.

The first import works fine, but it's the following that doesn't work:

from .. import d

Because I am attempting to import something from a parent package, but apparently I cannot, for some weird reason.

nbro
  • 15,395
  • 32
  • 113
  • 196

6 Answers6

51

This had me question my insanity.

The problem stems from the confusion that people mistakenly take the relative import as path relative which is not.

Relative imports depend on the location of the file that is run.

This answer goes deeper into explaining how the python modules actually work, but to summarize.

  1. When a file is loaded, it is given a name:
    • If it was loaded as the top-level script (run directly), its name is __main__.
    • If it was loaded as a module (with import), its name is the filename, preceded by the names of any packages/subpackages of which it is a part, separated by dots - pkg.subpkg.a
  2. If you do a from .. there must be at least 2 dots in the file name. from ... - 3 dots.

Now comes the funny part.

If you run c.py directly, then it is given the name __main__ and a.py has subpkg.a.

As per the 2nd statement, you must have at least 2 dots in the name of subpkg.a to run from .. inside it.

The fix

Create a new file outside the pkg, say main.py

pkg/
    __init__.py
    c.py
    d.py

    subpkg/
        __init__.py
        a.py
        b.py
main.py

Inside main.py

import pkg.c

If we run main.py, it get's the name __main__, and a.py get's pkg.subpkg.a. As per the 2nd statement it now has 2 dots in the name and we can do the from ..

One more thing. Now that c.py is loaded as a module, we have to use from to load a.py.

from .subpkg import a
typhon04
  • 2,350
  • 25
  • 22
5

Python 3 changed the import system so every time you want a module that is around the one you are working, you need relative imports (unless you mess with PYTHONPATH or sys.path).

The correct usage here should be

from .subpkg import a

When you are working with IDLE, you have a totally different environment. Therefore, you could add the current location to your path so imports work again.

try:

sys.path.insert(0, '')

It might be weird, but it is for a greater good

PS: If this last thing do not work -- I don't have an IDLE environment right now -- it is probably because the work directory is set wrong.

Try this answer instead: https://stackoverflow.com/a/17361545/754991

dfrankow
  • 20,191
  • 41
  • 152
  • 214
JBernardo
  • 32,262
  • 10
  • 90
  • 115
  • If I do `from .subpkg import a` I receive: `SystemError: Parent module '' not loaded, cannot perform relative import` from the IDLE and from the terminal... – nbro Feb 03 '16 at 02:27
  • 1
    @nbro that's because you are inside IDLE, but when you run it inside a module, the import should be that way. IDLE is a terrible IDE and bad for python development in general. The option that works best on both IDLE and inside modules is to use the full package name: `from pkg import subpkg.a` after adding the directory for `pkg` on `sys.path` – JBernardo Feb 03 '16 at 11:22
  • What do you mean: "when you run it inside a module"? I said, I run the file `c.py` (as main) both from the terminal and the IDLE as you suggested, and it gives me the error I am mentioning above. IDLE is a terrible IDE but it comes with Python? What would be the purpose of carrying a terrible IDE? You can say that IDLE is a simplistic IDE that for sure is not one of the best, but it should theoretically work well with Python, otherwise nothing makes sense. – nbro Feb 03 '16 at 14:59
  • I meant. When you run inside a package. It occurs when someone from outside your package does `from pkg import c`. This is what you should care about. Being able to run it inside a package... Your problem and many other people is that they want to test that single file to see if it works and by doing that, the relative imports do not work – JBernardo Feb 03 '16 at 16:48
  • 2
    I have played a little bit with what you said and the Python's import system in general, and I think I understood more how it works. Basically, there's the distinction between running files as "main" and as modules or packages imported by other modules or by a "main" file. If you have just one file that runs the whole application, you can do relative imports between the submodules and subpackages. On the other hand, as you are saying, relative imports do not work in a a file run as "main". This is unfortunately a bad limitation... – nbro Feb 03 '16 at 17:46
  • Also, `from .subpkg import a` only works if the file that I run as main is outside the package where the module that does that import is. Apparently, if you run a file as main, and you want to import modules in its current directory, you should do `from this_package import something` (i.e. without relative imports). Actually, the problem has nothing to do with the IDLE, but is related to the fact that people do not explain all the important details that make you understand how this import system really works...I understood more by trying different things then by reading whole pages... – nbro Feb 03 '16 at 17:49
  • 46
    `from .subpkg import a` doesn't answer the question. The question is asking how to `from .. import d`. – endolith Jun 18 '17 at 14:06
  • Why `from .subpkg import a` should start with a dot? – mrgloom Sep 11 '19 at 13:58
  • It may be weird but it's for the greater good, is a sign of an overengineered solution. Python has very little to offer beyond intuitiveness. The relative import semantics kill any benefit it may have had over C++ or C. – Alex Petrosyan Apr 05 '20 at 16:02
4

I found this solution:

#! /usr/bin/env python
import os
import sys
sys.path.append(os.path.realpath('.'))
from d import *
Alex44
  • 3,597
  • 7
  • 39
  • 56
  • 1
    Doesn't this make your code depend on the working directory of the process? If I run it from somewhere else or `chdir` into a differnt directory, it will fail, I believe. – Ulrich Eckhardt Jan 13 '22 at 12:44
  • @UlrichEckhardt That's true, but this can still be useful for cases when that doesn't matter (in my case, I'm just writing an internal program for the purpose of reproducing a bug). – Newbyte Jul 30 '23 at 19:36
2

simply adding/creating init.py file in all folders solved the problem.

folder1 -folder2 -file1.py -folder3 -folder4 - file2.py

I wanted to use a method present in file1.py inside file2.py [ basically two levels up]. so I added empty init.py in all folders & subfolders above and used below in file2.py:-

from folder2.file1 import <method_name>

Anil_rockstar
  • 21
  • 2
1

typhon04 has an excellent description that helped me understand the problem, but I disagree with his conclusion of creating a dummy main.py outside everything. Given we're relative to c.py the answer would seem to be we simply no longer need "from .." and simply "import d" is sufficient.

Mark Z.
  • 11
  • 1
0

I am thankful for the top answer, but I find the proposed fix a bit unsatisfying. Here is my suggestion : simply add sys.path.append(".") to your main file. This allows to import first level packages without any change to the project architecture. Just drop the previous .. which are not required anymore.

Anaël Leinert
  • 1