Replace runfile with exec/subprocess.call while passing a namespace

Question

I have a python script that accepts an argument, say test.py 1 and generates some data. A second script experiment.py launches test.py repeatedly. I achieve this using runfile in Spyder IDE:

So, experiment.py reads something like:

experiments = {}
for i in range(10):
    experiments[str(i)] = {}
    runfile(`test.py`, args=str(i), wdir=`/path/to/script/`, namespace=experiments[str(i)])

## some post-processing using data from the dict experiments

This allows me to store variables of test.py for each call in a separate namespace. Everything works well, but (here comes my question):

How can I replace runfile with exec or subprocess.call and be able to pass a namespace and an argument list?

The objective is to run the script from command line rather than Spyder. Just to clarify, I am aware that the best way to perform such a task is to convert test.py into a function. However, I am trying to make-do with the current script test.py which is quite long.

Namespaces don't make sense outside of Spyder, do they? I'm not familiar with it but quick Duck Duck Going suggests that the meaning of this parameter is to separate the variables in the subscript from the ones in Spyder itself, which of course you are not running when you aren't. — tripleee, Apr 19 '23 at 10:07
`subprocess.run(['test.py', str(i)], check=True, cwd='/path/to/script')`, possibly add `text=True` and `capture_output=True` if you want that. Perhaps see also https://stackoverflow.com/questions/4256107/running-bash-commands-in-python/51950538#51950538 — tripleee, Apr 19 '23 at 10:08

score 0 · Accepted Answer · answered Apr 19 '23 at 10:09

Modify test.py to save its variables to a file and then loading the file in experiment.py instead of using exec use subprocess.run so that you can run the script from the command line and avoid using the Spyder-specific runfile function.

test.py will look like this

import sys
import json

arg = int(sys.argv[1])

# Your original code here

variables_to_save = {'variable1': variable1, 'variable2': variable2}
with open(f"variables_{arg}.json", "w") as f:
    json.dump(variables_to_save, f)

and experiment.py

import subprocess
import json

experiments = {}
for i in range(10):
    experiments[str(i)] = {}
    
    subprocess.run(["python", "test.py", str(i)], cwd="/path/to/script/")
    
    with open(f"variables_{i}.json", "r") as f:
        experiments[str(i)] = json.load(f)

Replace runfile with exec/subprocess.call while passing a namespace

1 Answers1