0

I have a script which is subscribed to RMQ channel and it should spawn a new process with long-playing IO task if it gets some special kind of messages. And after this it should be terminated.

I've very simple function which tries to spawn worker:

 async def spawn_worker(self):
     p = multiprocessing.Process(target=long_plaing_task)
     p.start()
     p.join()
     result = await p.get_a_result_somehow()
     p.kill()

I have never worked with multiprocessing so my question is simple and maybe stupid: how should a get that result from the spawned process? I found this question and answer to use Queues. But is this the only way to get a result or there are some more convinient ways?

Community
  • 1
  • 1
Paul
  • 6,641
  • 8
  • 41
  • 56
  • 1
    http://stackoverflow.com/questions/10415028/how-can-i-recover-the-return-value-of-a-function-passed-to-multiprocessing-proce does this help? I have always just used queues but apparently you have other means as well. – Hannu Dec 01 '16 at 14:36
  • In your example, `spawn_worker` is a coroutine, and you're using `await` to get a result from the `multiprocessing.Process`. I don't think the `multiprocessing` module supports asynchronous behavior like that. – dano Dec 01 '16 at 14:47
  • There isn't a more convenient way to get a return value from a `multiprocessing.Process` directly, but there is if you use `multiprocessing.Pool` instead. The question @Hannu links to has an answer that covers that. – dano Dec 01 '16 at 14:48
  • FWIW, if you need `multiprocessing` integration with `asyncio`, there is the third-party [`aioprocessing`](https://github.com/dano/aioprocessing) library (full disclosure: I am the author). – dano Dec 01 '16 at 14:49
  • 1
    Consider using [`loop.run_in_executor`](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.AbstractEventLoop.run_in_executor) with a [`ProcessPoolExecutor`](https://docs.python.org/dev/library/concurrent.futures.html#concurrent.futures.ProcessPoolExecutor): `result = await loop.run_in_executor(executor, target, *args)` – Vincent Dec 01 '16 at 14:54
  • Actually, assuming you need the `asyncio` integration, @Vincent's suggestion is probably the best choice. – dano Dec 01 '16 at 22:46

0 Answers0