running two scripts simultaneously from a master script when each script has multiple threads within it in python

Question

I want to run two or more python scripts simultaneously from a master script. Each of these scripts already have threads within them which are running in parallel. For example I run

script1.py

if __name__ == '__main__':
    pid_vav = PID_VAV('B2')
    t1 = threading.Thread(target=pid_vav.Controls)
    t1.daemon = False
    t1.start()
    t2 = threading.Thread(target=pid_vav.mqttConnection)
    t2.daemon = False
    t2.start()

script2.py

if __name__ == '__main__':
    pid_vav = PID_VAV('B4')
    t1 = threading.Thread(target=pid_vav.Controls)
    t1.daemon = False
    t1.start()
    t2 = threading.Thread(target=pid_vav.mqttConnection)
    t2.daemon = False
    t2.start()

I am running this script1.py and script2.py separately. Only difference is the parameter which I am passing to the class. Is it possible to have a master script such that if I just run that, both these scripts will run ?

Thanks

First off the above code cannot run from another script because `__name__` will be `__main__` only if the file is invoked if it is imported that piece of code won't work. If you want to make it work you need to move that code to a function and import that function in the master script and call both the functions simultaneously — jaswanth, Jan 31 '20 at 10:26
@jaswanth the `runpy` module works around that, but of course that's not pretty. — o11c, Feb 03 '20 at 04:11
I am not realizing why the question is being downvoted.If anyone understands why, please help me understand — chink, Feb 07 '20 at 05:28

score 4 · Answer 1 · answered Feb 09 '20 at 19:52

Assuming you want the output of both scripts to be shown when you run the master script.
You can make use of the subprocess module to call the python file, and you can use the threading module to start separate threads

from threading import Thread
import subprocess

t1 = Thread(target=subprocess.run, args=(["python", "script1.py"],))
t2 = Thread(target=subprocess.run, args=(["python", "script2.py"],))

t1.start()
t2.start()

t1.join()
t2.join()

score 2 · Answer 2 · answered Jan 31 '20 at 13:37

2

If u want to trigger 2 scripts from a master script u can use the below method. It will help you trigger both scripts as thread and the thread can also produce different threads based on the callable scripts. You can even make Scripts run independently.

import subprocess

pid1 = subprocess.Popen([sys.executable, "script1.py"]) 
pid2 = subprocess.Popen([sys.executable, "script2.py"])

answered Jan 31 '20 at 13:37

DeenaDeepak

111
1
1
9

When I run this in a separate script, the script ran without error, but nothing was happening? – chink Jan 31 '20 at 15:22
was the separate script in the same directory than `script1.py` and `script2.py`? – gelonida Feb 07 '20 at 02:04
You can save this snippet as a separate script and run the script !!! keep your scripts parallel to this script. then this script will help you to execute both your script. – DeenaDeepak Feb 26 '20 at 18:12

n1tr0xs · Answer 3 · 2020-02-09T19:20:58.980

1

Yes, ofc.

script_master.py:

from os import system
system('start script1.py && start script2.py')

But I think you could to use this code:

script_together.py:

if __name__ == '__main__':
    todo=[]

    todo.append(threading.Thread(target=lambda: PID_VAV('B2').Controls, daemon=False))
    todo.append(threading.Thread(target=lambda: PID_VAV('B4').mqttConnection, daemon=False))
    for th in todo:
        th.start()
    for th in todo:
        th.join()

edited Feb 09 '20 at 19:20

answered Jan 31 '20 at 10:24

n1tr0xs

408
2
9

@cvg I think yes, but I`m cant test. – n1tr0xs Jan 31 '20 at 10:37
can you help me understand how it runs in background. 2 threads for each argument ? – chink Jan 31 '20 at 12:36
thread1 is doing operations related to B2 only, how many threads will be running? – chink Jan 31 '20 at 14:32
@cvg but in question you wrote `pid_vav = PID_VAV('B4')` – n1tr0xs Jan 31 '20 at 14:37
operations of thread 1 are performed twice, once for each parameter , but both the operations are done for paramter B2 only. – chink Jan 31 '20 at 14:44
i want it to happen for both B2 and B4 – chink Jan 31 '20 at 14:44
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/206977/discussion-between-cvg-and-n1tr0xs). – chink Jan 31 '20 at 14:45

score 0 · Answer 4 · answered Feb 09 '20 at 20:37

If you're happy to have the code for both live in one file, you can use multiprocessing to run them concurrently on different CPU cores.

import multiprocessing as mp
from threading import Thread

def start_process(pid_vav_label):
    pid_vav, threads = PID_VAV(pid_vav_label), []
    threads.append(Thread(target=pid_vav.Controls))
    threads.append(Thread(target=pid_vav.mqttConnection))
    for thread in threads:
        thread.start()
    # Join if necessary
    for thread in threads:
        thread.join()

if __name__ == '__main__':
    processes = []
    for label in ['B2', 'B4']:
        processes.append(mp.Process(target=start_process, args=(label,)))
        processes[-1].start()
    # Again, can join if necessary
    for process in processes:
        process.join()

running two scripts simultaneously from a master script when each script has multiple threads within it in python

4 Answers4

Linked