2

I want to send tasks to the POOL inside the shared class based on some conditions. But I got some unexpected result, Which are shown below.

• Why the len(self.map) is 0, not 100.

• Do I have to reconstruct my code to achieve this goal.

from multiprocessing import Pool
from multiprocessing.managers import BaseManager

pool = None

def doSomething(obj, *args):
    obj.doSomething(*args)

class SharedClass:
    def __init__(self):
        global pool
        self.map = set()
        pool = Pool(4)

    def someCondition(self):
        # the condition is rely on the instance, here is just an example
        return True

    def go(self, n):
        global pool
        for i in xrange(n):
            if self.someCondition():
                # pass the shared class to other process
                pool.apply_async(doSomething, (self, i))

        pool.close()
        pool.join()

        # got AssertionError here
        # why the len of self.map is 0
        assert len(self.map) == 100

    def doSomething(self, n):
        # this should change the same SharedClass instance?
        self.map.add(n)


class MyManager(BaseManager):
    pass

MyManager.register("SharedClass", SharedClass)

def main():
    manager = MyManager()
    manager.start()
    obj = manager.SharedClass()
    obj.go(100)

if __name__ == "__main__":
    main()
Jiaming Lin
  • 57
  • 5
  • can you share the error? Is it about pickling? – jwillis0720 Mar 16 '21 at 06:40
  • I'm sorry, it's not error. It's an unexpected result. – Jiaming Lin Mar 16 '21 at 06:50
  • I think this post will help you solve your problem : https://stackoverflow.com/questions/10415028/how-can-i-recover-the-return-value-of-a-function-passed-to-multiprocessing-proce You need to use shared memory between your main and subrocess. When I have to deal with multiprocessing I generaly use Pipes to share data between my processes. I think you can do the same with a bit of refactoring. – CDulouard Mar 16 '21 at 10:11
  • `self.map` is not a proxy object, so modifications won't be propagated. (this will be true with all mutable attributes) – Aaron Mar 16 '21 at 17:08

0 Answers0