Problems when importing in Python 3.x

Question

my problem can be summarized as follows.

I have the following files structure:

<path>/
- a.py
<path>/subdir/
- b.py
- c.py

The relation is that a imports b and b imports c (a->b->c).

More precisely, the content of each file is as follows:

c.py

def foo():
    print("Hello World!")

b.py

import c
def fie():
    c.foo()
    print("Second Hello World!")
if __name__=="__main__":
    fie()

a.py

from subdir import b
b.fie()

What I want is that invoking a.py will print those strings, namely:

$ python3 a.py
Hello World!
Second Hello World!
$

and invoking b.py from /subdir will print the same thing:

$ cd subdir
$ python3 b.py
Hello World!
Second Hello World!
$

Instead if I invoke a.py from <path> I have this:

$ python3 a.py
Traceback (most recent call last):
  File "a.py", line 1, in <module>
    from subdir import b
  File "<path>/subdir/b.py", line 1, in <module>
    import c
ModuleNotFoundError: No module named 'c'
$

if I, instead, invoke b.py from <path>/subdir I have the intended behavior:

$ cd subdir
$ python3 b.py
Hello World!
Second Hello World!
$

If I change b.py the following way:

b.py

from subdir import c
def fie():
    c.foo()
    print("Second Hello World!")
if __name__=="__main__":
    fie()

the invocation of a.py works as intended, but this time is the invocation of b.py that gives ModuleNotFoundError:

$ cd subdir
$ python3 b.py
Traceback (most recent call last):
  File "b.py", line 1, in <module>
    from subdir import c
ModuleNotFoundError: No module named 'subdir'
$

Strangely enough even this invocation from <path> gives the same error:

$ python3 subdir/b.py
Traceback (most recent call last):
  File "b.py", line 1, in <module>
    from subdir import c
ModuleNotFoundError: No module named 'subdir'
$

If I replace the following string in b.py:

from subdir import c

in

from . import c

I have exactly the same effect of the lastest test.

In the real case scenario both b and c are part of a library self-contained in subdir that is meant to be used both directly (with the if __name__ trick) and imported from anywhere else in order to use their functions. Said library can not be installed in a fixed space of the file system at this moment (or at least I shouldn't rely on that).

I never had such problems when working with Python 2.6 and I'm recently migrating to Python 3.x. Reading the python 3.x guide to importing was helpful but I didn't find any useful information to solve this case which should be pretty basic (which is why I'm thinking that I've omitted something trivial there).

Can you suggest a way to change any of [a,b,c].py files to cover this use case?

G1Rao · Answer 1 · 2019-06-29T20:32:50.320

0

I kepy my directory structure like this: Directory of C:\Users\G1\Desktop\Test

30-06-2019  01:17                   a.py
30-06-2019  01:17    <DIR>          path/

 Directory of C:\Users\G1\Desktop\Test\path

30-06-2019  01:18                b.py
30-06-2019  01:15                c.py

a.py

import sys
**sys.path.append("path")**
from path import b
b.fie()

b.py

import c
def fie():
    c.foo()
    print("Second Hello World!")
if __name__=="__main__":
    fie()

c.py

def foo():
    print("Hello World!")

outputs:

C:\Users\G1\Desktop\Test>python a.py
Hello World!
Second Hello World!

C:\Users\G1\Desktop\Test>python path\b.py
Hello World!
Second Hello World!

You have to append the path of module c.py

Above code is tested in python3 and python2 also.

edited Jun 29 '19 at 20:32

answered Jun 29 '19 at 19:58

G1Rao

424
5
11

This is what I used to do in Python 2.6. Is not working anymore in Python 3.x. Can you confirm that you are using Python 3.x in your tests? – joe345wa Jun 29 '19 at 20:03
I don't believe adding `__init__.py` is necessary for python 3+ https://stackoverflow.com/a/37140173/7585554 – probat Jun 29 '19 at 20:29
yes, there is no need of __init__.py, but you need to append the path of the module c, in a.py because a.py doesn't know the path of c.py – G1Rao Jun 29 '19 at 20:34
Yes, the import mechanic changed in Python3.x, that's the problem. I was expecting that it worked exactly the same removing the __init__.py but it is not. That's why I was wondering what's wrong with that code and what's the best way to cover that use case in Python 3.x. Either way, with or without __init__.py, I still have the same error using Python 3.x. – joe345wa Jun 29 '19 at 20:36
1

did you try appending the path as mentioned in above code? Try that, that's working fine, I tested it. – G1Rao Jun 29 '19 at 20:38
@JeevanRao yes, I've read elsewhere that this is a workaround. To me it doesn't make much sense, since a doesn't import c, b does and b knows where c is. Following this reasoning c might import something else and so on, while a should be responsible to know the position of each and every imported module starting from b. I was looking for a better solution honestly, this is a very basic use case. – joe345wa Jun 29 '19 at 20:41
@joe345wa, it is user defined module then either we need to specify complete path or append the path of the module to sys.path, so that interpreter search in that path, else it will through no module error , as per python3 documentation. – G1Rao Jun 29 '19 at 20:46
@joe345 you should be able to get it working with absolute or relative imports like you were trying. When I put `from subdir import c` in the top of `b.py` and I run `b.py` it works for me and I do not get a `ModuleNotFoundError`. Running Python 3.7 Windows 10. I'm not sure whats going on... – probat Jun 29 '19 at 20:51
@JeevanRao if you are referring to a.py it has the complete path of b.py. Otherwise, if you are referring to b.py it also has the path of c.py. In fact every module mentioned have the information about every module it explicitly import. What is missing here is the information about c from *a* even if *a* doesn't import c. – joe345wa Jun 29 '19 at 20:57
Now, ofc a solution would be to add the path of each and every user-defined modules, but what I want to know here is if there's a way to replicate the Python 2.6 __init__.py trick, which allowed me to import any module from anywhere in the file system without adding a path specifically (since each python file was responsible to having the path of the modules directly imported, and nothing more). – joe345wa Jun 29 '19 at 20:58
@probat, joe345wa said, both b.py, c.py should be in same folder, then I added from path import c, it thrown an exception ModuleNotFoundError: No module named 'path' – G1Rao Jun 29 '19 at 20:58
@probat ok that's strange, are you sure that you don't have the subdir among your paths? I've tried and with `from subdir import c` in b.py if I run a.py it works, if I run b.py it doesn't – joe345wa Jun 29 '19 at 21:02
@joe345, printing `sys.path` in `a.py` did not yield path to the `sub` directory. Just shows the path for `a.py` of course. – probat Jun 29 '19 at 21:13
I'm also using pycharm and I'm not sure if it's doing some magic behind the scenes. – probat Jun 29 '19 at 21:19

score 0 · Answer 2 · answered Jun 29 '19 at 21:17

The following works for me (under Windows at least).

Start your main program by calling addpath giving as parameter the directory (as string) or directories (as list of strings) that you want your app to have access to:

def addpath(dirs):
  import sys,os
  my_path = '/'.join(os.path.abspath(sys.argv[0]).replace('\\','/').split('/')[:-1])+'/'
  if type(dirs) is str:
    sys.path.append(my_path+dirs)
  elif type(dirs) is list:
    for dir in dirs:
      sys.path.append(my_path+dir)

################################################################################

addpath('subdir')  #or addpath(['subdir1',...])

from subdir import b
b.fie()

Problems when importing in Python 3.x

2 Answers2