0

I have a wrapper function that runs multiple subroutines. Instead defining the wrapper function to take hundreds of inputs that would pass to each subroutine, I would like the wrapper function to read an input file containing all the arguments to be passed to each subroutine.

For example, given an input file called eg_input.py:

## example of argument file

a = [1]
b = [2]
c = [3]
d = [4]
e = [5]
f = [6]

I'd like to have a wrapper function that looks something like this:

def wrap_fun(argument_file):
    
    ## read in the argument file
    exec(open(argument_file).read())


    ## run first sub routine
    sub_fun1(fun1_parma1=a, fun1_param2=b, fun1_param3=c)

    ## run second sub routine
    sub_fun2(fun2_param1=d, fun2_param2=e, fun2_param3=f)

    ## run third sub routine
    ## note some arguments are shared between subroutines 
    sub_fun2(fun3_param1=a, fun3_param2=f)
    
    ## etc...

    return()

such that I could run a script like:

wrap_fun(eg_input.py)

When I run code similar to the above, however, the arguments are not available to the subroutines.

You'll note I used the exec() function in the above example but I am not adamant about using it, i.e. if there is a pythonic way to accomplish this I would prefer that solution.

My motivation for this question is the ability to easily produce a variety of input files (again with each input file containing 100s of arguments) and observing how changing each argument affects the results of wrap_fun().

Thanks!

Alex Witsil
  • 855
  • 8
  • 22
  • 1
    This is a pretty odd request. Usually data in files is either code that is loaded as a Python module and interpreted, or it's raw data in an interchange format like CSV or JSON that can be serialized and loaded. You can also pickle Python objects and structures. But what you're asking for here is pretty much never the right way to do it, so if you provide a bit more context for what you're trying to achieve in your project, someone can help point you to a better solution than this. – ggorlen Jan 08 '21 at 18:29
  • 2
    A major problem with `exec` is it's essentially executing user input, which is a major security concern. It would be trivial for someone to replace a line with a call to `rm` or something and nuke your computer or start a `nc` backdoor. It would be safer, and cleaner to have the file as a JSON, or serialized dictionaries. Then you'd read it in, deserialize, and use it. What's the relationship between the order the arguments are needed, and how they are in the file? Is it always every three arguments in order are used for the same call? – Carcigenicate Jan 08 '21 at 18:30
  • 1
    (Those security concerns are quite paranoid though. I just had an instructor that had us do things like remove write permissions on scripts and data when we were done editing them, "just in case"). – Carcigenicate Jan 08 '21 at 18:32
  • I don't use `exec` but I believe you can pass the local vars dictionary: `exec(open(argument_file).read(), None, locals())` – 001 Jan 08 '21 at 18:34
  • If you can `exec` the contents of the file, you can simply import it. You would need to use `importlib` if the name of the file isn't hard-coded. – chepner Jan 08 '21 at 18:35
  • @ggorlen I added more context at the end of the question as you requested but please let me know if you need further clarification. – Alex Witsil Jan 08 '21 at 18:42
  • 1
    If you want to read input from a file, read input from a file of raw values instead of trying to park some of the source code into the file. Or make the whole file source code and import it as a module. Once it's in memory, call the function with the data. Is the data bunch of numbers or strings? A bit more concrete and less pseudocode-y would be helpful, probably. The issue isn't so much security as doing it in a non-awkward way, although the obvious security holes aren't great--can't be promoting SolarWinds-style coding here. – ggorlen Jan 08 '21 at 18:45
  • thanks @Carcigenicate I tried to clarify the relation between argument order and how they are used in the file in the definition of the wrapper function. – Alex Witsil Jan 08 '21 at 18:47
  • 2
    @AlexWitsil So there is no straightforward relationship between order in the file and usage order? I think storing it as a JSON, parsing the JSON, then doing something like `sub_fun1(json['a'], json['b'], json['c'])` (where `json` is the parsed JSON dictionary object) would be the more straightforward option. In the file you'd have something like `{"a": [1], "b": [2], "c": [3]}`. – Carcigenicate Jan 08 '21 at 18:52
  • 1
    I think [how do I create variable variables](https://stackoverflow.com/questions/1373164/how-do-i-create-variable-variables) may also be relevant here. – ggorlen Jan 08 '21 at 19:29

1 Answers1

1

You can supply a dictionary for global variables in exec. Any globals defined in the executed code end up there. Now, when you call your functions, read the values from that dict. Since class instances already have a dict, you could write the wrapper as a class that updates class instances from the file and then you can write any number of instance methods to run your code.

# functions to call
def sub_fun1(fun1_param1="not"):
    print("fun", fun1_param1)

def sub_fun2(fun2_param1="not"):
    print("too", fun2_param1)

# option 1: use a dict
def wrap_fun(argument_file):
    ns = {}
    exec(open(argument_file).read(), ns)
    print(ns.keys())
    sub_fun1(fun1_param1=ns["a"])

# option 2: use instance dict
class FunRunner:

    def __init__(self, argument_file):
        exec(open(argument_file).read(), self.__dict__)

    def have_some_fun(self):
        sub_fun1(fun1_param1=self.a)

    def have_more_fun(self):
        sub_fun2(fun2_param1=self.b)

# write test file
fn = "my_test_args"
open(fn,"w").write("""a = [1]
b = [2]
c = [3]
d = [4]
e = [5]
f = [6]""")

# option1 
wrap_fun(fn)

# option2 one shot
FunRunner(fn).have_some_fun()

# option2 multiple
fun = FunRunner(fn)
fun.have_some_fun()
fun.have_more_fun()
tdelaney
  • 73,364
  • 6
  • 83
  • 116