Reload submodule imported in other modules

Question

I have the following issue and I'll share four different .py files to better explain myself. I'm running the code from spyder (not jupyter), python 3.4. I have a master script "master001.py" from which I execute the code. it looks like this:

import sys
before = [str(m) for m in sys.modules]

from importlib import reload
import time
#from child001 import calculation as calc
import child001 
from child002 import calculation_two
from child003 import calculation_three

after = [str(m) for m in sys.modules]
print("########################")   
print([m for m in after if not m in before])
print("########################\n")




stop = False
while stop == False:
    print("\n\n\n\n\n\n\n")
    reload_child_one = input("reload child 1 function? Enter Y or N\n")
    reload_child_one = reload_child_one.lower()

    if reload_child_one == "y":
        print("Script will try to reload the calculation 1 / child 1 module.")
        time.sleep(1)
        reload(child001)



    reload_child_two = input("reload child 2 function? Enter Y or N\n")
    reload_child_two = reload_child_two.lower()

    if reload_child_two == "y":
        print("Script will try to reload the calculation 2 / child 2 module.")
        time.sleep(1)
        #reload(sys.modules[calculation_two.__module__])
        #del calculation_two
        #from child002 import calculation_two
        #__import__("child002", fromlist='calculation_two')
        calculation_two = reload(sys.modules["child002"]).calculation_two



    print("\n####################################################")
    a = input("Enter number that will be saved in variable 'a' or enter Q to quit prorgam\n")

    if a.lower() == "q" :
        stop = True
        print("\nFunction complted. Script will quit.")
        print("####################################################\n")
        time.sleep(2)

    else:
        try:
            a = int(a)

            print("Master - Launching Child function 'calculation'")
            b = child001.calculation(a)

            print("\nMaster - Inside Master file. Result = b = {}".format(b))
            print("####################################################\n")

            print("Master - Launching Child 2 function 'calculation_two' on input variable")
            c = calculation_two(a)     

            print("\nMaster - Inside Master file. Result = c = {}".format(c))            
            print("####################################################\n")

            print("Master - Launching child 3")
            calculation_three()
            time.sleep(2)

        except:
            print("input value was not a valid number. Please, try again.\n")
            print("####################################################\n")
            time.sleep(2)

master001.py calls child001.py to perform a simple calculation:

print("wassupp from child 1 !!!")

def calculation(a):

    print("\n----------------------------------------")
    print("Child 1 - function 'calculation' started.")
    print("Child 1 - Operation that will be executed is: input variable + 20")

    result = a + 20

    print("Child 1 - Returning result =  {}".format(result))
    print("----------------------------------------\n")
    return result

Then, master001.py calls child002.py in which another simple calculation is performed:

print("wassupp from child 2 !!!")

def calculation_two(a):

    print("\n----------------------------------------")
    print("Child 2 - function  'calculation_two' started.")
    print("Child 2 - Operation that will be executed is: input variable + 200")

    result = a + 200

    print("Child 2 - Returning result =  {}".format(result))
    print("----------------------------------------\n")
    return result

So far so good. Finally, I have child003.py. in this module I perform a calculation that is actually imported from child002.py

from child002 import calculation_two

print("wassupp from child 3 !!!")

def calculation_three():

    print("\n----------------------------------------")
    print("Child 3 function - Calculation will use the one in child 2 applied to value '3'.!\n")

    result = calculation_two(3)

    print("Child 3 - result =  {}".format(result))
    print("----------------------------------------\n")
    return

as you can see from running master001.py, when I reload calculation_two using

calculation_two = reload(sys.modules["child002"]).calculation_two

that works for calculation_two run from child002.py, however it doesn't reload calculation_two called by child003.py.

More specifically, if you run master001.py and before manually inputting anything change the content of calculation_two, then when you get asked

reload child 1 function? Enter Y or N

you enter N, and when you get asked

reload child 2 function? Enter Y or N

you enter Y, you will see the value returned by child003.py not reflecting the new updated code.

I read How do I unload (reload) a Python module? and How to reload python module imported using `from module import *` they are very helpful but I can't find there a solution to this specific problem.

Answer 1

Your problem is with how you imported the function from child002:

from child002 import calculation_two

This creates a reference to the function object in child003, and that reference is not replaced. Python names are like labels on strings, tied to objects. You can tie multiple labels to an object, and if you want to replace that object with another, then you must make sure to re-tie all those labels.

You start with this:

sys.modules['child002']
    -> module object created from child002.py
        -> module.__dict__ (the module globals)
            -> module.__dict__['calculation_two']
                    |
                    |
                    +--> function object named calculation_two
                    |
                    |
            -> module.__dict__['calculation_two']
        -> module.__dict__ (the module globals)
    -> module object for child003.py
sys.modules['child003']

and when you then reload the child002 module, Python replaces all the existing globals with new objects, so now you have:

sys.modules['child002']
    -> module object created from child002.py
        -> module.__dict__ (the module globals)
            -> module.__dict__['calculation_two']
                    |
                    |
                    +--> *new* function object named calculation_two


                    +--> *old* function object named calculation_two
                    |
                    |
            -> module.__dict__['calculation_two']
        -> module.__dict__ (the module globals)
    -> module object for child003.py
sys.modules['child003']

because the calculation_two reference in the child003 module object is an independent label.

You either have to replace that label manually:

calculation_two = reload(sys.modules["child002"]).calculation_two
child003.calculation_two = calculation_two

or you could just not reference calculation_two directly, and instead reference only the child002 module:

import child002

# ...

def calculation_three():
    # ...
    result = child002.calculation_two(3)

at which point you have the following relationship:

sys.modules['child002']
    -> module object created from child002.py
       ^ -> module.__dict__ (the module globals)
       |    -> module.__dict__['calculation_two']
       |            |
       |            |
       |            +--> function object named calculation_two
       |
       |
       +------------+
                    |
                    |
            -> module.__dict__['child002']
        -> module.__dict__ (the module globals)
    -> module object for child003.py
sys.modules['child003']

I can recommend reading Ned Batchelder's explanation of Python names and values for another perspective on this.