Proper way to dynamically import a module with relative imports?

Question

I need to dynamically import modules into my project from another package.

The structure is like:

project_folder/
    project/
        __init__.py
        __main__.py
    plugins/
        __init__.py
        plugin1/
            __init__.py
            ...
        plugin2/
            __init__.py
            ...

I made this function to load a module:

import os

from importlib.util import spec_from_file_location, module_from_spec


def load_module(path, name=""):
    """ loads a module by path """
    try:
        name = name if name != "" else path.split(os.sep)[-1]  # take the module name by default
        spec = spec_from_file_location(name, os.path.join(path, "__init__.py"))
        plugin_module = module_from_spec(spec)
        spec.loader.exec_module(plugin_module)
        return plugin_module
    except Exception as e:
        print("failed to load module", path, "-->", e)

It works, unless the module uses relative imports:

failed to load module /path/to/plugins/plugin1 --> Parent module 'plugin1' not loaded, cannot perform relative import

What am I doing wrong?

The solution is given in [this answer](https://stackoverflow.com/a/50395128/1088938): you need to also add the imported module to `sys.modules` so that the relative import works. I.e. include `sys.modules[spec.name] = plugin_module` just before you call `spec.loader.exec_module(plugin_module)`. — mforbes, Jul 14 '18 at 07:29

score 15 · Accepted Answer · answered Jun 20 '16 at 15:01

15

I managed to solve my own issue after a LOT of googling. Turns out I needed to import using relative paths:

>>> from importlib import import_module
>>> config = import_module("plugins.config")
>>> config
<module 'plugins.config' from '/path/to/plugins/config/__init__.py'>
>>>

answered Jun 20 '16 at 15:01

user3638162

421
1
3
12

1

How to do `from plugins.config import *`, using **import_module** or **__import__** ? – Rajesh Jai Nov 15 '17 at 06:45
Been two years since i used importing in python, this saved me a bit of time messing about.. (yes i had absolute paths... grrr) .. cheers + 1 – Angry 84 Nov 18 '18 at 22:53

Gábor Fekete · Answer 2 · 2016-06-14T11:14:23.783

5

I had a similar problem not long ago. I added the path of the project folder to the sys.path using the module's absolute path like this:

import sys
import os
sys.path.append(os.path.dirname(os.path.realpath(__file__))+'/..')

This adds the project_folder to the sys.path thus allowing the import statement to find the plugin modules.

edited Jun 14 '16 at 11:14

answered Jun 14 '16 at 10:44

Gábor Fekete

1,343
8
16

Sadly this won't work for me. I added it to the beginning of my `project/__main__.py` file before I call `load_module`. Is that the correct place to put it? – user3638162 Jun 14 '16 at 11:58
I forgot to mention that this uses Python3. You could try to add an __init__.py to your project folder too. – Gábor Fekete Jun 15 '16 at 08:39
Sorry, markdown screwed up my comment. It should be another `__init__.py` file in the project_folder. – Gábor Fekete Jun 15 '16 at 08:55

Proper way to dynamically import a module with relative imports?

2 Answers2

Linked