python import different subpackages with the same root packge name and different locations

Question

I was wondering if anyone could shed light on this. We have multiple package libraries with the same root package e.g. a. I also have package a.b located in X and package a.c located in Y. Both X and Y are in my PYTHONPATH and When I do:

import a.c
import a.b

I get an error: "No module named b". After reading around it seems to me that once a.c is loaded python writes info about a as well, and when I come to do a.b because it has information about a already it never bothers to look in location X for a.b and throws an error that no module named b can be found.

Moreover, I found that order with which X and Y are specified in the PYTHONPATH seem to affect the importing. For example, when I do

PYTHONPATH=$PYTHONPATH:X:Y python
>>> import a.b    # works
>>> import a.c    # fails

But if I do

PYTHONPATH=$PYTHONPATH:Y:X python
>>> import a.b    # fails
>>> import a.c    # works

Is that correct and if so, how can I work around this? It's convenient to have a common module root name and different sub packages reside in different projects etc. Of course, I am coming from Java point of view where you could do this kind of overlap.

Answer 1

I have found related question but lost link to it.

The solution is to include:

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

in the root __init__.py in all projects. In this case in a/__init__.py BOTH at location X and Y. If you have multiple levels of subpackages you still only need to include it once.

This helped me and documentation for extend_path, and info What is __path__ useful for?