threads not running parallel in python script

Question

I am new to python and threading. I am trying to run multiple threads at a time. Here is my basic code :

  import threading
  import time
  threads = []

  print "hello"

  class myThread(threading.Thread):
          def __init__(self,i):
                  threading.Thread.__init__(self)
                  print "i = ",i
                  for j in range(0,i):
                          print "j = ",j
                          time.sleep(5)

  for i in range(1,4):
          thread = myThread(i)
          thread.start()

While 1 thread is waiting for time.sleep(5) i want another thread to start. In short, all the threads should run parallel.

Afaik, threads in python all run on a **single core**. So the only thing threads can do is increase the cpu burst so to speak. You need the `multiprocessing` library that starts new processes to get more than a single core. — Willem Van Onsem, Feb 02 '17 at 12:00
when the thread thread is constructed, it is on the same thread. If you want your code to be run concurently, you should put your `time.sleep' in the `run` method, not the constructor. — n00dl3, Feb 02 '17 at 12:04
You may want to look at [this answer](http://stackoverflow.com/a/18963931/4124317) on how to use `threading`. You basically don't need to subclass `Thread` at all, but if you do, don't use the `__init__()` constructor to perform tasks that should be threaded. — ImportanceOfBeingErnest, Feb 02 '17 at 12:07
No, in the code you show, everything is sequential, because you perform your work in the constructor of the Thread, even before the thread is started. — ImportanceOfBeingErnest, Feb 02 '17 at 12:11

score 7 · Accepted Answer · answered Feb 02 '17 at 12:24

You might have some misunderstandings on how to subclass threading.Thread, first of all __init__() method is roughly what represents a constructor in Python, basically it'll get executed every time you create an instance, so in your case when thread = myThread(i) executes, it'll block till the end of __init__().

Then you should move your activity into run(), so that when start() is called, the thread will start to run. For example:

import threading
import time
threads = []

print "hello"

class myThread(threading.Thread):
    def __init__(self, i):
        threading.Thread.__init__(self)
        self.i = i

    def run(self):
        print "i = ", self.i
        for j in range(0, self.i):
            print "j = ",j
            time.sleep(5)

for i in range(1,4):
    thread = myThread(i)
    thread.start()

P.S. Because of the existence of GIL in CPython, you might not be able to fully take advantages of all your processors if the task is CPU-bound.

It's working and i am finally understanding how it works. Thanks a ton. — Utkarsh Patel, Feb 02 '17 at 12:31

score 3 · Answer 2 · answered Feb 02 '17 at 12:22

Here is an example on how you could use threading based on your code:

import threading
import time
threads = []

print "hello"

def doWork(i):
    print "i = ",i
    for j in range(0,i):
        print "j = ",j
        time.sleep(5)

for i in range(1,4):
    thread = threading.Thread(target=doWork, args=(i,))
    threads.append(thread)
    thread.start()

# you need to wait for the threads to finish
for thread in threads:
    thread.join()

print "Finished"

LouisXW · Answer 3 · 2019-03-17T15:48:56.453

2

import threading
import subprocess


def obj_func(simid):
    simid = simid
    workingdir = './' +str (simid) # the working directory for the simulation
    cmd = './run_delwaq.sh' # cmd is a bash commend to launch the external execution
    subprocess.Popen(cmd, cwd=workingdir).wait()


def example_subprocess_files():
    num_threads = 4
    jobs = []

    # Launch the threads and give them access to the objective function
    for i in range(num_threads):
        workertask = threading.Thread(target=obj_func(i))
        jobs.append(workertask)

    for j in jobs:
        j.start()

    for j in jobs:
        j.join()

    print('All the work finished!')


if __name__ == '__main__':
    example_subprocess_files()

This one not works for my case that the task is not print but CPU-Intensive task. The thread are excluded in serial.

edited Mar 17 '19 at 15:48

answered Mar 17 '19 at 02:58

LouisXW

41
4

for better understanding provide an explanation to your solution – Anantha Raju C Mar 17 '19 at 03:13
I have updated the code above to the one that I used in my case. For each thread, the worker need to run a external program. I create separate folder for each external program so they are not interact with each other. However, when I check the result I found the external program in each thread is running in serial instead of in parallel. – LouisXW Mar 17 '19 at 15:53
I tested the solution posted by @Shane for my problem(that overrides the [run()](https://docs.python.org/2/library/threading.html#threading.Thread.run) method in a subclass of Threading.Thread class) . It works. However, not quite sure why the alternative [method](https://stackoverflow.com/a/55203456/11207606) I posted above not work. Since according to the document in [Threading.Thread](https://docs.python.org/2/library/threading.html#thread-objects), both method should work. – LouisXW Mar 17 '19 at 16:46

threads not running parallel in python script

3 Answers3