Module name binding by relative import in __init__.py

Question

I wrote a package containing two submodules:

pkg/__init__.py
pkg/foo.py
pkg/bar.py

I put the following code in __init__.py and also in bar.py.

from . import foo as f
foo
print("Hello!")

While importing pkg suceeds, pkg.bar doesn't:

>>> import pkg
Hello!

>>> import pkg.bar
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\fumito\Documents\pkg\bar.py", line 2, in <module>
    foo
NameError: name 'foo' is not defined

Why is foo defined in the pkg namespace but not in the pkg.bar?

Answer 1

By importing the foo module, it implicitly becomes an attribute of pkg, because its fully qualified location is pkg.foo. This makes the name foo available in __init__.py, because it became an attribute of pkg/__init__.py.

But obviously, pkg.bar.foo would make no sense, since that's not the fully qualified name of the foo module, so importing foo in bar does not create a name foo inside bar.

(This is a rather handwavey explanation, but summarises the core mechanism at play here.)

Answer 2

EDIT : This answer is incorrect. Please read the explanation in the next section.

It is because when you try to import the modules when you are in the same directory as pkg, the relative import happens from where pkg is. That means that the following statement in bar.py

from . import foo as f

attempts to find foo in the parent directory of pkg.

You could choose to write the following import statement in pkg/bar.py instead (I personally prefer this method because it doesn't modify the path)

import pkg.foo as f

Or you could choose to modify the path variable within your __init__.py file

import os, sys

script_dir = os.path.dirname(os.path.abspath(__file__))
sys.path.append(script_dir)

And then, add the following import statement in bar.py

import foo as f

=============================================================== ATTEMPT 2

So deceze has provided a really nice answer which I will try to expand upon.

What you are doing is quite similar to what is stated in the Python docs in 5.4.2. Submodules:

When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in __import__()) a binding is placed in the parent module’s namespace to the submodule object. For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule. Let’s say you have the following directory structure:

spam/
    __init__.py
    foo.py

and spam/__init__.py has the following line in it:

from .foo import Foo

then executing the following puts name bindings for foo and Foo in the spam module:

>>>import spam
>>>spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>>spam.Foo
<class 'spam.foo.Foo'>

Given Python’s familiar name binding rules this might seem surprising, but it’s actually a fundamental feature of the import system. The invariant holding is that if you have sys.modules['spam'] and sys.modules['spam.foo'] (as you would after the above import), the latter must appear as the foo attribute of the former.

Similar to what is mentioned in the docs, when you add the statement

# pkg/__init__.py

from . import foo as f
foo
print("Hello!")

in __init__.py and then import pkg, pkg.foo exists because of the import statement, which is why NameError is not raised. Therefore, importing pkg will be successful. You can check that pkg.foo is referring to foo.py if you check the value of pkg.foo within the terminal - you will see a similar return value to

<module 'pkg.foo' from 'absolute/path/to/pkg/foo.py'>

Now adding the following code in bar.py

# pkg/bar.py

from . import foo as f
foo
print("Hello!")

and attempting to execute

import pkg.bar

does raise the NameError exception because an object foo does not exist within bar.py; foo does not exist within the namespace of bar.py.

There is something interesting that is also happening when you used an alias to import foo within __init__.py. It is actually possible to overwrite the reference pkg.foo while still maintaining access to the contents of foo.py. If you added the following block within your __init__.py file after foo

class foo:
    pass

then checking the reference pkg.foo will return

<class 'pkg.foo'>

but contents of foo.py can still be accessed by using pkg.f. If you check the value of pkg.f, you will see someting similar to

<module 'pkg.foo' from 'absolute/path/to/pkg/foo.py'>