1

How to get the someone task's return immediately not until all task completed in [asyncio] ? 

import asyncio
import time

print(f"now: {time.strftime('%X')}")

async def test1():
    print(f"test1 started at {time.strftime('%X')}")
    await asyncio.sleep(5)
    with open('test.txt', 'w') as f:
        f.write('...')
    return 'end....'

async def test2(num):
    print(f"test2 started at {time.strftime('%X')}")
    return num * num

async def main(num):
    res = await asyncio.gather(test1(), test2(num))
    return res

def my(num):
    return asyncio.run(main(num))[1]

print(my(5))
print(f"all end at {time.strftime('%X')}")

From the above code(python 3.7+), I can only get test2 return after test1 and test2 are all completed.

How can I let the main function get test2 return after test2 is completed, instead of waiting until test1 is completed?, because test2 is executed more quickly. And test1 must be executed(generate test.txt file.)

Than means return (test2's return) to main function or my function as soon as possible when test1 and test2 is asynchronous.

stackoverflow26
  • 57
  • 6

1 Answers1

2

To run a set of awaitables/coroutines until any Future/Task finishes or is canceled - you need asyncio.wait coroutine:

...
async def main(num):
    done, pending = await asyncio.wait([test1(), test2(num)], 
                                       return_when=asyncio.FIRST_COMPLETED)
    for coro in done:
        return await coro


def my(num):
    return asyncio.run(main(num))


print(my(5))
print(f"all end at {time.strftime('%X')}")

return_when indicates when this function should return

The output:

now: 18:50:16
test1 started at 18:50:16
test2 started at 18:50:16
25
all end at 18:50:16

But since you need all coroutines to be completed - use asyncio.as_completed approach OR print results from done set, then - await from pending set and change print(my(5)) to just my(5):

...
async def main(num):
    done, pending = await asyncio.wait([test1(), test2(num)],
                                       return_when=asyncio.FIRST_COMPLETED)
    for coro in done:
        print(await coro)
    for coro in pending:
        await coro


def my(num):
    return asyncio.run(main(num))


my(5)
print(f"all end at {time.strftime('%X')}")
RomanPerekhrest
  • 88,541
  • 4
  • 65
  • 105
  • I've tried this method, but the disadvantage of `asyncio.FIRST_COMPLETED` is the rest coroutines will be canceled. It is not what I want. And I have edited the question. – stackoverflow26 Nov 23 '19 at 17:15
  • @stackoverflow26, no need to over-complicate things: since you need all coroutines to be completed - print results from `done` set, then - await from `pending` set – RomanPerekhrest Nov 23 '19 at 17:35
  • I wrongly confirmed the answer, the last sentence in my question, I need return(not just print) **test2's return** to `main` function or `my` function, if I use pending in front of done, like `for coro in pending: await coro for coro in done: return coro.result()`, the coro in `done` will wait `coro` in pending completed, but if exchage the order, the `RETURN` keywork will end the main funciton. so, how to return **test2's return** to main function before test1 completed? – stackoverflow26 Nov 24 '19 at 03:49
  • @stackoverflow26, changing your mind **11** hours later was really insidious. Now I feel that I've upvoted the question prematurely. – RomanPerekhrest Nov 24 '19 at 06:43
  • @ RomanPerekhrest Sorry, Since yesterday was too late, I did not verify it carefully. it’s my mistake, I cancel the reception, I just hope that you can continue to really help me. I will accept this answer. If you have time and good mood, I hope that you can continue to help me with the problems mentioned later. I am sorry for the disappointment caused to you. – stackoverflow26 Nov 24 '19 at 06:55
  • @stackoverflow26, Ok, it turns out that besides of initiating a set of coroutines and immediately **return** the first completed result, your expectation is to continue running *pending* coroutines in background. But that's **out** of scope of `asyncio` processing as it's running coroutines **concurrently** within a **single** thread. You could try a workaround with `loop.run_in_executor` like here https://stackoverflow.com/a/46075571/3185459, but that's likely to throw `ValueError: loop argument must agree with Future`. – RomanPerekhrest Nov 24 '19 at 12:10
  • @stackoverflow26, you would need to reconsider how do your coroutines conceptually relate to each other , as well as their order, and try to run a functions like `test1` (with blocking I/O operations) in a separate `Thread` – RomanPerekhrest Nov 24 '19 at 12:12
  • 1
    Yes, it can be solved very easily with a separate thread. I just want to know if I can solve it with async. In addition, thank you again for the my rude cancellation of the answer. Your answer solved my doubts. – stackoverflow26 Nov 25 '19 at 02:47