classification
Title: "loop argument must agree with lock" instantiating asyncio.Condition
Type: Stage: resolved
Components: asyncio Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Joongi Kim, asvetlov, lukasz.langa, miss-islington, serhiy.storchaka, simonw, uriyyo, yselivanov
Priority: normal Keywords: patch

Created on 2021-10-08 20:30 by simonw, last changed 2021-10-12 09:03 by asvetlov. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 28850 merged Joongi Kim, 2021-10-10 09:26
PR 28856 merged miss-islington, 2021-10-10 16:01
Messages (7)
msg403500 - (view) Author: Simon Willison (simonw) * Date: 2021-10-08 20:30
In Python 3.10 it is not possible to instantiate an asyncio.Condition that wraps an asyncio.Lock without raising a "loop argument must agree with lock" exception.

This code raises that exception:

    asyncio.Condition(asyncio.Lock())

This worked in previous Python versions.

Note that the error only occurs if an event loop is running. Here's a simple script that replicates the problem:

    import asyncio

    # This runs without an exception:
    print(asyncio.Condition(asyncio.Lock()))

    # This does not work:
    async def example():
        print(asyncio.Condition(asyncio.Lock()))

    # This raises "ValueError: loop argument must agree with lock":
    asyncio.run(example())
msg403501 - (view) Author: Simon Willison (simonw) * Date: 2021-10-08 20:31
I ran across this issue while trying to use the https://pypi.org/project/janus/ locking library with Python 3.10 - see my issue on their tracker here: https://github.com/aio-libs/janus/issues/358
msg403502 - (view) Author: Simon Willison (simonw) * Date: 2021-10-08 20:38
It looks like the relevant test is here: https://github.com/python/cpython/blob/a1092f62492a3fcd6195bea94eccf8d5a300acb1/Lib/test/test_asyncio/test_locks.py#L722-L727

    def test_explicit_lock(self):
        lock = asyncio.Lock()
        cond = asyncio.Condition(lock)

        self.assertIs(cond._lock, lock)
        self.assertIs(cond._loop, lock._loop)

But... that test doesn't appear to run inside an event loop, so it's not covering the behaviour described in this issue.
msg403506 - (view) Author: Ɓukasz Langa (lukasz.langa) * (Python committer) Date: 2021-10-08 21:16
Issue confirmed. The problem is that `Condition.__init__` compares its own loop with Lock's internal `_loop` attribute. That attribute is set to None unless the lock waited to be acquired.

uriyyo, Andrew, Yury, since it's pretty likely that Lock objects will now have `_loop` set to None, I think the check for "loop agreement" in Condition is kind of useless. So I propose to simply remove it. It's the only such check in all of asyncio.

If you insist on keeping the loop check, it should special-case `_loop is None`.

Simon, thanks for your detailed report! As a backwards-compatible workaround till we get a fix in, your user code can do the following:

>>> l = asyncio.Lock()
>>> getattr(l, '_get_loop', lambda: None)()
<_UnixSelectorEventLoop running=True closed=False debug=False>

You can use such lock without issues now:

>>> asyncio.Condition(l)
<asyncio.locks.Condition object at 0x10c05bee0 [unlocked]>

Alternatively, if the above disgusts you and you only want to trigger public APIs, you can do this dance:

>>> l = asyncio.Lock()
>>> await l.acquire()  # first acquire will just work
True
>>> try:
...   # second acquire will block so we time it out
...   await asyncio.wait_for(l.acquire(), 0.1)
... except asyncio.TimeoutError:
...   pass
...
>>> l.release()

Now the lock is fully initialized and we can use it:

>>> c = asyncio.Condition(l)

Both workarounds should be compatible with Python 3.7+ asyncio.
msg403599 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-10-10 16:01
New changeset 1a7892414e654aa5c99efa31db767baba7f4a424 by Joongi Kim in branch 'main':
bpo-45416: Fix use of asyncio.Condition() with explicit Lock objects (GH-28850)
https://github.com/python/cpython/commit/1a7892414e654aa5c99efa31db767baba7f4a424
msg403601 - (view) Author: miss-islington (miss-islington) Date: 2021-10-10 16:25
New changeset 164dddf5f8c9c6b93f32c9f79b4301fc804576e9 by Miss Islington (bot) in branch '3.10':
bpo-45416: Fix use of asyncio.Condition() with explicit Lock objects (GH-28850)
https://github.com/python/cpython/commit/164dddf5f8c9c6b93f32c9f79b4301fc804576e9
msg403719 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2021-10-12 09:03
Thanks, guys!
History
Date User Action Args
2021-10-12 09:03:49asvetlovsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-10-12 09:03:39asvetlovsetmessages: + msg403719
2021-10-10 16:25:21miss-islingtonsetmessages: + msg403601
2021-10-10 16:01:49miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request27163
2021-10-10 16:01:48serhiy.storchakasetmessages: + msg403599
2021-10-10 09:26:07Joongi Kimsetkeywords: + patch
nosy: + Joongi Kim

pull_requests: + pull_request27160
stage: patch review
2021-10-10 08:12:38corona10setversions: + Python 3.11
2021-10-09 10:42:28serhiy.storchakasetnosy: + serhiy.storchaka
2021-10-08 21:16:42lukasz.langasetnosy: + uriyyo, lukasz.langa
messages: + msg403506
2021-10-08 20:38:06simonwsetmessages: + msg403502
2021-10-08 20:31:57simonwsetmessages: + msg403501
2021-10-08 20:30:58simonwcreate