How to reconstruct a local variable environment in python?

Question

I'm reformatting a bunch of data processing code. The original code first declare several functions, they have certain topological dependency(which means some function rely on other function's result), then calling them sequentially(using a correct topo sort):

def func_1(df):
    return df.apply(...)

def func_2(df):
    return pd.concat([df, ...])

def func_3(df_1, df_2):
    return pd.merge(df_1, df_2)

if __name__ == "__main__":
    df=...
    df_1 = func_1(df)
    df_2 = func_2(df)
    result = func_3(df_1, df_2)# the func_3 rely on the result of func_1 & func_2

The problem is that I'm not able to retrieve intermediate data. Say I just want to apply func_1 & func_2, I need to change some code. And it gets complicated when topological dependency gets complicated.

So I want to change into kind of like makefiles's recursive recipe:

def recipe_1(df):
    return df.apply(...)

def recipe_2(df):
    return pd.concat([df, ...])

def recipe_3(df):
    df_1 = recipe_1(df)
    df_2 = recipe_2(df)
    #some process here.
    return 

if __name__ == '__main__':
    df = ...
    recipe_3(df) #Just call the intermediate node I need.

The problem of this approach is I need to collect a lot of variable from recipe_1andrecipe_2 in recipe_3, so I think it would be nice if I am able to retrieve the variables from locals(), which will leave the other code in #some process here. unchanged.

Now I'm thinking something like this but it looks ugly:

def func_to_be_reconstructed():
    a = 3
    return locals()

local_variables = func_to_be_reconstructed()
for key in local_variables.keys():
    exec(str(key) + '= local_variables[\'' + str(key) + '\']')

better solution?

What exactly are you trying to accomplish? What would be your use case for this? — Thierry Lathuille, Jan 03 '21 at 10:17
Does this answer your question? [How do I create variable variables?](https://stackoverflow.com/questions/1373164/how-do-i-create-variable-variables) TL;DR - don't use `exec` to create new variables, just use `loacals()` as a dict... — Tomerikoo, Jan 03 '21 at 10:25

score 3 · Accepted Answer · answered Jan 03 '21 at 10:29

3

globals() and locals() are just dicts...

So, instead of using exec in such a fishy way, just update the dict:

def func_to_be_reconstructed():
    a = 3
    return locals()

globals().update(func_to_be_reconstructed())

answered Jan 03 '21 at 10:29

Tomerikoo

18,379
16
47
61

So if I want to use them in another function, I can use locals().update(func_to_be_reconstructed()) in that function? – 古今中 Jan 03 '21 at 13:18
Yes, although I'm pretty sure a better way to do this is just `return` the values you need from the function and re-use them elsewhere... – Tomerikoo Jan 03 '21 at 13:20

DannyDannyDanny · Answer 2 · 2021-02-24T14:47:09.777

0

This is probably bad practice but you could use the global keyword on variables you wish to refer to outside the function.

def func_to_be_reconstructed():
    global a
    a = 3

print(a)

This is especially useful if you just need several variables. Take a look at this related question.

EDIT: A single global list can be filled with multiple variables:

def write_to_msv(*vars):
    global my_special_var
    my_special_var = []
    for k in vars:
        my_special_var.append(k)

myvar1 = 123
myvar2 = "hello"
myvar3 = {'no shoes': 'no service'}

write_to_msv(myvar1, myvar2, myvar3)

print(my_special_var)

Again: "You can use global variables but you shouldn't"

edited Feb 24 '21 at 14:47

answered Jan 03 '21 at 10:17

DannyDannyDanny

838
9
26

1

What if they have 10 variables? What if there are 50? – Tomerikoo Jan 03 '21 at 10:18
In that case I'd probably have the function return all the variables in a list or some kind of container. Check the edit above, the described method be applicable. – DannyDannyDanny Feb 24 '21 at 14:50

score 0 · Answer 3 · answered Jan 03 '21 at 10:29

0

This looks a bit cleaner but does mostly the same.

for var,value in func_to_be_reconstructed().items(): f"{var} = {value}")

answered Jan 03 '21 at 10:29

wuerfelfreak

2,363
1
14
29

How to reconstruct a local variable environment in python?

3 Answers3