0

Update: Putting f outside the class seems to make the example work as well.

My goal is:

  • Create a class that processes tasks in a pool
  • Create an instance, start the processes and monitor the progress

However I do not understand why the following code only works when calling time.sleep() after pool.close().

Minimum example:

import multiprocessing as mp
import time


class MultProc:
    def __init__(self, n: int):
        self.n = n
 
    # Also works with putting function outside this class
    def f(self, i: int, return_list):
        time.sleep(5)
        return_list.append(i)

    def multiproc(self):
        manager = mp.Manager()
        self.return_list = manager.list()
        pool = mp.Pool()
        results = {}
        for i in range(self.n):
            results[i] = pool.apply_async(self.f, args=(i, self.return_list))
        pool.close()

        time.sleep(0.01) # Only works with this line

        self.pool = pool

def main():
    mult = MultProc(n=10)
    mult.multiproc()
    n = 0
    while True:
        n+=1
        print(mult.return_list)
        time.sleep(1)        
        if n > 20:
            break


if __name__ == "__main__":
    main()
user3673486
  • 39
  • 1
  • 4
  • 1
    You need to call *get()* on the AsyncResultset. Having said that, it's unclear (to me anyway) exactly what you're trying to achieve here – DarkKnight Jul 08 '22 at 15:41
  • I'd like to have a class: DoSomething().start_processes() Then I'd like to monitor the processes status after instantiating the class, like Pseudo-Code: ``` do_something_instance = DoSomething() do_something_instance.start_processes() while processes_unfinished: progress = do_something_instance.get_progress() print(progress) if progress == 100: processes_unfinished = False break ``` Apart from that I'm curious why the code works with the time.sleep() but not without it. – user3673486 Jul 08 '22 at 15:58
  • Check this out https://stackoverflow.com/a/72794232/16310741 – Charchit Agarwal Jul 08 '22 at 16:01
  • if you don't explicitly wait on the results by calling `results[i].get()`, the pool will be closed and terminated before it has a chance to actually perform any work. It doesn't matter that you're passing the results back a different way (via a managed list) – Aaron Jul 08 '22 at 23:10
  • Thanks for your replies. Still: Why does the pool doesn’t close when a) including the time.sleep or b) putting the function outside the class? – user3673486 Jul 09 '22 at 10:03

0 Answers0