Asyncio Loop and Lock

Question

i was tinkering around with asyncio loops and locks and found something that I thought was strange

import asyncio
import time
lock = asyncio.Lock()

async def side_func():
    print("trying to acquire Side")
    await lock.acquire()
    print("Acquired side")
    lock.release()

async def func2():
    print("Trying to aacquire 2nd")
    await lock.acquire()
    print("Acquired second")
    lock.release()
    
async def func1():
    print("Acquiring 1st")
    await lock.acquire()
    await asyncio.sleep(2)
    print("Acquired 1st")
    lock.release()
    await side_func()


async def main():
    await asyncio.wait([func1(), func2()])
    
asyncio.run(main())

given this code runes func1 first, it should first be able to acquire the lock for func1, jump to func2 on the asyncio.sleep(2) and wait for the lock there, when the sleep finishes it should execute side func.

But when trying to acquire the lock for side_func, the event loop instead jumps to func2 and acquires the lock there. From my understanding, it should acquire the lock in side_func first and then be able to jump to other functions

For some reason it is jumping to the abandoned func2 function first, is this a behaviour globally applicable to asynchronous functions or is this specific to lock.acquire? I guess it would make sense for event loops to jump to abandoned yet completed states first but I'm not really sure if this globally applicable to async functions

Sami Tahri · Accepted Answer · 2022-03-15T16:04:37.403

1

The internal behavior of the lock is this code :

def release(self):
    """Release a lock.

    When the lock is locked, reset it to unlocked, and return.
    If any other coroutines are blocked waiting for the lock to become
    unlocked, allow exactly one of them to proceed.

    When invoked on an unlocked lock, a RuntimeError is raised.

    There is no return value.
    """
    if self._locked:
        self._locked = False
        self._wake_up_first()
    else:
        raise RuntimeError('Lock is not acquired.')

So what it does is that every time you release your lock, it tries to wake the next waiter (the last coroutine that called this lock), which is :

fut = next(iter(self._waiters))

because the lock implement a queue for storing waiters

def __init__(self, *, loop=None):
    self._waiters = collections.deque()
    self._locked = False

the next waiter is the first that called lock.

When executing your code :

Acquiring 1st
Trying to aacquire 2nd
Acquired 1st
trying to acquire Side
Acquired second
Acquired side

we clearly see that 2nd is waiting before side. So, when 1st is releasing it, it is the func2 that is going to wake up, not Side.

edited Mar 15 '22 at 16:04

answered Mar 15 '22 at 10:01

Sami Tahri

1,077
9
19

consider the following code, https://pastebin.com/jLWv3Dm2, so from my understanding here, func2 does not wake up because it's in a queue, rather because the event loop is idle and that function is still unfinished and the queue's are only applicable to the locks – Kryptic Coconut Mar 15 '22 at 10:22
I ran your code which gave the results I mentionned in my answer ^^. What part don't you understand so I can be more specific/detailed ? Note: I changed your code so it runs without errors : asyncio.get_event_loop().run_until_complete(main()) for the last line – Sami Tahri Mar 15 '22 at 10:27
according to your answer, the waiter is awakened by lock.release, but the loop continues onto side_function as `trying to acquire Side` is printed before `Acquired second` – Kryptic Coconut Mar 16 '22 at 00:57
1

Awakened doesn't mean that it switch context instantly, it just notify that the code is ready to be executed, and the event loop will manages how/when to change the execution of code, in the same manner a single-core CPU scheduler works. You can check https://stackoverflow.com/a/51116910/10034177 which explains a lot of how event loops works :). – Sami Tahri Mar 16 '22 at 11:43
Ah, I see, so it just takes a higher priority than the other tasks, thanks! – Kryptic Coconut Mar 16 '22 at 16:22
In fact its more complex than that. The switching context is done by asyncio eventloop. What happens is that when the release() method is called, a Future, that tries to perform the awakening of the lock for func2 is created, but this code is performed only when asyncio wants to treat this event in the event loop. – Sami Tahri Mar 16 '22 at 20:04

Asyncio Loop and Lock

1 Answers1