0

How do you deep reload a package without importing modules from any other outside packages?

For example, for reloading the following:

# example_pkg.py
import logging  # do not reload this stdlib package
import example_pkg.ex_mod  # reload this module

IPython's deepreload module cannot specify a whitelist for imports and uses module level variables during reloads which makes it unreliable for threaded environments.

A question similar to this has been asked before, but it focused on discovering dependencies (as was mentioned in a comment) not exclusively within a single package.

Tankobot
  • 1,492
  • 14
  • 21

1 Answers1

2

Using the sys and importlib modules, a function can be written to remove the package and its modules from Python's import cache. This allows for the package to reload its child modules when it is re-imported.

import sys
import importlib
from types import ModuleType

def deep_reload(m: ModuleType):
    name = m.__name__  # get the name that is used in sys.modules
    name_ext = name + '.'  # support finding sub modules or packages
    del m
    
    def compare(loaded: str):
        return (loaded == name) or loaded.startswith(name_ext)
    
    all_mods = tuple(sys.modules)  # prevent changing iterable while iterating over it
    sub_mods = filter(compare, all_mods)
    for pkg in sub_mods:
        del sys.modules[pkg]  # remove sub modules and packages from import cache
    
    return importlib.import_module(name)

This code can be extended with a Lock to make it thread-safe as well:

from threading import Lock

sys_mod_lock = Lock()  # all accesses to sys.modules must be programmed to acquire this lock first
# this means do not use any builtin import mechanism such as the 'import' statement once the following function is being used
# instead use importlib's import_module function while sys_mod_lock is acquired

def tsafe_reload(m: ModuleType):
    with sys_mod_lock:
        return deep_reload(m)

Note: these functions come with one of the caveats from the standard library's reload. Any references elsewhere in the program leading to the old package will be maintained and will not be replaced automatically. For that, you can look at globalsub, which can replace all references to an object in the interpreter with a different object (usually for debugging purposes).

Tankobot
  • 1,492
  • 14
  • 21