2

I need to create a Runner class that can asynchronously runs multiple jobs and return the results one all jobs are completed. Here is the dummy code snippet which I already have

import asyncio

class Runner:
  def __init__(self):
    self.jobs_loop = asyncio.new_event_loop()

  async def slow_process(self, *args):
    ...  # run slow proces with args
    await asyncio.sleep(5)
    return "result"

  def batch_run(self, count, *args):
    results = self.jobs_loop.run_until_complete(asyncio.gather(
      *[self.slow_process(*args) for _ in range(count)],
      loop=self.jobs_loop,
      return_exceptions=True
    ))
    return results

runner = Runner()
print(runner.batch_run(10))

When the batch_run is called, it schedules count jobs and blocks until finished. This, however, doesn't work when called from a running event loop raising RuntimeError: Cannot run the event loop while another loop is running

How can I make batch_run work when called from existing event loop (e.g. Jupyter Console) as well as being callable from outside of the event loop.

EDIT

Ideally the function would look like this

def batch_run(self, count, *args):
  loop = asyncio.get_event_loop()
  jobs = asyncio.gather(...)
  if not loop.is_running():
    return loop.run_until_complete(jobs)
  else:
    future = asyncio.ensure_future(jobs, loop=loop)
    # loop.wait_until_completed(future)
    return future.result()

if only loop.wait_until_completed existed.

warownia1
  • 2,771
  • 1
  • 22
  • 30
  • Possibly duplicate of https://stackoverflow.com/questions/47518874/how-do-i-run-python-asyncio-code-in-a-jupyter-notebook – logicOnAbstractions Jul 28 '19 at 14:44
  • 1
    Not really a duplicate. This problem is not strictly related to the Jupyter console but is more generic question about how to make a function callable from both within and without asyncio loop. – warownia1 Jul 28 '19 at 14:54
  • What about passing the loop at the class constructor? There is a reason that `batch_run` is not async as well? – Amiram Dec 30 '19 at 12:23

0 Answers0