If I have a list of subprocess.Popen objects, is there any way to tell what command was initially used when generating them?
Python 2.7
Context: I have a list of various commands that fire off tests. If one of the tests fails, a script cleans up the environment. I would like to then retry those failed commands only.
Note: Commands below are for demonstration purposes only; those called in my prod code are more complex, but that is beside the point. If I can this to work, it will work with my prod cmds.
commands = [['nosetests', '-V'],
['nosetests', '--collect-only'],
['nosetests', '--with-id']
]
def cleanup_env():
...do things...
def run_in_parallel(cmds, retry):
retry_tasks = []
if not cmds:
return
def done(p):
return p.poll() is not None
def success(p):
return p.returncode == 0
def fail(p):
if not retry:
retry_tasks.append(p)
print("{} failed, will need to retry.".format(retry_tasks))
else:
pass # if this is already a retry, we don't care, not going to retry again
MAX_PARALLEL = 4
processes = []
while True:
while cmds and len(processes) < MAX_PARALLEL:
task = cmds.pop() # pop last cmd off the stack
processes.append(subprocess.Popen(task))
for p in processes:
if done(p):
if success(p):
processes.remove(p)
else:
fail(p)
processes.remove(p)
if not processes and not cmds:
break
else:
time.sleep(0.05)
return retry_tasks
Calling the above:
retry_list=run_in_parallel(commands, False)
if retry_list:
cleanup_env()
run_in_parallel(retry_list, True)
The first part works, but calling the retry doesn't because I'm passing a list of subprocess.Popen objects, rather than their initial input.
Hence the question, how do I get at the input of a subprocess.Popoen object?
