Is `await` in Python3 Cooperative Multitasking?

Question

I am trying to understand the new asyncio coroutines (introduced in Python 3.5).

In 1997 I attended a course at university which roughly covered the content of the book Modern Operating Systems by Andrew Tanenbaum.

Somehow the await in Python3 reminds me at Cooperative Multitasking.

From Wikipedia:

Cooperative multitasking, also known as non-preemptive multitasking, is a style of computer multitasking in which the operating system never initiates a context switch from a running process to another process. Instead, processes voluntarily yield control periodically or when idle in order to enable multiple applications to be run simultaneously. This type of multitasking is called "cooperative" because all programs must cooperate for the entire scheduling scheme to work.

If you look at the Python interpreter like an operating system, does the term "Cooperative Multitasking" apply to await?

But may be I am missing something.

Answer 1

Inside a coroutine function, the await expression can be used to suspend coroutine execution until the result is available. Any object can be awaited, as long as it implements the awaitable protocol by defining the await() method.

A coroutine can pause execution using the await keyword with another coroutine. While it is paused, the coroutine’s state is maintained, allowing it to resume where it left off the next time it is awakened. That sounds quite like Cooperative multitasking to me. See this example

Answer 2

It is cooperative multitasking indeed.

What about a small program to prove it. Let's first sleep with cooperative asyncio.sleep for a second and then let's sleep with blocking time.sleep for a second. Let's print a thread id, time spent in the coroutine and id of a task.

import threading
import asyncio
import time

async def async_function(i):
    started = time.time()
    print("Id:", i, "ThreadId:", threading.get_ident())
    await asyncio.sleep(1)
    time.sleep(1)
    print("Id:", i, "ThreadId:", threading.get_ident(), "Time:", time.time() - started)

async def async_main():
    await asyncio.gather(
        async_function(1),
        async_function(2),
        async_function(3)
    )

loop = asyncio.get_event_loop()
loop.run_until_complete(async_main())

Now let's try and see:

Id: 3 ThreadId: 140027884312320
Id: 2 ThreadId: 140027884312320
Id: 1 ThreadId: 140027884312320
Id: 3 ThreadId: 140027884312320 Time: 2.002575397491455
Id: 2 ThreadId: 140027884312320 Time: 3.0038201808929443
Id: 1 ThreadId: 140027884312320 Time: 4.00504469871521

As expected. Execution was only in one thread. asyncio.sleep(1) is nonblocking, so it took 1 second to process all of them concurrently. time.sleep(1) is blocking (it does not cooperate), so it blocks the rest. Id 1 waits for id 2 to finish while id 2 waits for id 3 to finish.

C# has async/await too, does it have cooperative multitasking as well?

Let's try the same thing in C#:

using System;
using System.Threading;
using System.Threading.Tasks;

namespace AsyncTest
{
    class MainClass {
        private static async Task AsyncMethod(int id) {
            var started = DateTime.Now;
            Console.WriteLine("Id: {0} ThreadId: {1}", id, Thread.CurrentThread.ManagedThreadId);
            await Task.Delay(1000);
            Thread.Sleep(1000);
            Console.WriteLine("Id: {0} ThreadId: {1} Time: {2}", id, Thread.CurrentThread.ManagedThreadId, DateTime.Now - started);
        }

        private static async Task MainAsync()
        {
            await Task.WhenAll(AsyncMethod(1), AsyncMethod(2), AsyncMethod(3));
        }

        public static void Main (string[] args) {
            MainAsync().Wait();
        }
    }
}

Run it and...

Id: 1 ThreadId: 1
Id: 2 ThreadId: 1
Id: 3 ThreadId: 1
Id: 2 ThreadId: 7 Time: 00:00:02.0147000
Id: 3 ThreadId: 8 Time: 00:00:02.0144560
Id: 1 ThreadId: 6 Time: 00:00:02.0878160

Damn. The threads are different after await. And it tooks just 2 seconds for each of the coroutine! What's wrong?

Nothing is wrong. Unlike Python, async/await in C# has a combination of cooperative multitasking and multithreading. Task.Delay(1000) is indeed nonblocking but when a coroutine resumes, it can resume in a totally different thread as it did in the example. Since the coroutines continued in three different threads, Thread.Sleep(1000) blocked them in parallel.

Note there are more things in C# which can influence this behavior (like SynchronizationContext), but this is a different topic.

Answer 3

Yes. According to Wikipedia:

Coroutines are computer program components that generalize subroutines for nonpreemptive multitasking, by allowing multiple entry points for suspending and resuming execution at certain locations.