How not to run a batch file and python in parallel?

Question

I have below scripts

1)

  os.chdir("C:\\RTM3Run_Full\\Python Codes")
  os.startfile("2015_RUNcvm_64_Crash.bat")

2)

 for i in range(settings.loop):
    ....

I want to start the "for i in range(settings.loop):" section (i.e. part 2)after the batch file is completely finished. But it only runs part 1) and part 2) in parallel. I know I should be able to set up a pause function to wait. But I don't know how long part 1) takes. Anyone know there is other smarter way to run 1) and 2) in sequence?

Many thanks.

Answer 1

os.startfile simulates a click in the explorer. It means that your batch file will run in background.

Replace this:

os.chdir("C:\\RTM3Run_Full\\Python Codes")
os.startfile("2015_RUNcvm_64_Crash.bat")

by a proper blocking subprocess call, and since it's a batch file and you want to avoid shell=True, prefix by cmd /c, as an argument list. Also don't chdir, just use current working directory argument so the current directory isn't changed:

rc = subprocess.call(["cmd","/c","2015_RUNcvm_64_Crash.bat"],cwd=r"C:\RTM3Run_Full\Python Codes")

you may want to check the return code, or use check_call to stop with an exception if the batch file returns a non-zero exit code.

Answer 2

My answer is based on this link

Suppose that you have this commands in separates scripts, you can do this:

import multiprocessing
import os                                              

# Creating the tuple of all the processes which can be run in parallel
all_parallel_processes = ('script_A.py', 'script_B.py', 'script_C.py')     

next_run = ('script_D.py')

# This block of code enables us to call the script from command line.                                                                                
def execute(process):                                                             
    os.system(f'python {process}')                                       


process_pool = Pool(processes = 4)                                                        
process_pool.map(execute, all_parallel_processes)
process_pool.map(execute, next_run)

Where script.d only run after a, b and c script finish (they run in parallel).

Answer 3

Finally I found an easy solution

1)

os.chdir("C:\RTM3Run_Full\Python Codes")

os.system("2015_RUNcvm_64_Crash.bat")

2)

for i in range(settings.loop): ....

This assured to run part 2) after part 1) is completed.

Thanks everyone for the helps~