This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_asyncio: test_get_cancelled() fails randomly on x86-64 macOS 3.x
Type: Stage: resolved
Components: asyncio, Tests Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, kumaraditya, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2022-03-21 03:14 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 32020 closed kumaraditya, 2022-03-21 06:16
PR 32040 merged asvetlov, 2022-03-21 23:55
Messages (9)
msg415653 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-21 03:14
x86-64 macOS 3.x failures:

* https://buildbot.python.org/all/#/builders/366/builds/2085
* https://buildbot.python.org/all/#/builders/366/builds/2078
* https://buildbot.python.org/all/#/builders/366/builds/2077
* https://buildbot.python.org/all/#/builders/366/builds/2073
* https://buildbot.python.org/all/#/builders/366/builds/2072
* https://buildbot.python.org/all/#/builders/366/builds/2069

Logs:
---
0:18:33 load avg: 2.21 Re-running test_asyncio in verbose mode (matching: test_get_cancelled)
test_get_cancelled (test.test_asyncio.test_queues.QueueGetTests) ... ERROR

======================================================================
ERROR: test_get_cancelled (test.test_asyncio.test_queues.QueueGetTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/asyncio/tasks.py", line 490, in wait_for
    return fut.result()
           ^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/asyncio/queues.py", line 158, in get
    await getter
    ^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/unittest/async_case.py", line 66, in _callTestMethod
    if self._callMaybeAsync(method) is not None:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/unittest/async_case.py", line 93, in _callMaybeAsync
    return self._asyncioTestLoop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/asyncio/base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/test/test_asyncio/test_queues.py", line 179, in test_get_cancelled
    self.assertEqual(1, await get_task)
                        ^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/test/test_asyncio/test_queues.py", line 174, in queue_get
    return await asyncio.wait_for(q.get(), 0.051)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/asyncio/tasks.py", line 492, in wait_for
    raise exceptions.TimeoutError() from exc
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError
---


Recent change related to this test:

commit e707ceb6e464b24ddc9fdbdac7bfe15f7eeb43fd
Author: Andrew Svetlov <andrew.svetlov@gmail.com>
Date:   Wed Mar 16 16:59:12 2022 +0200

    Rewrite asyncio.Queue tests with IsolatedAsyncioTestCace (#31935)
msg415655 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-21 03:15
> Recent change related to this test: commit e707ceb6e464b24ddc9fdbdac7bfe15f7eeb43fd

This change comes from bpo-47035 "Rewrite asyncio queue tests with IsolatedAsyncioTestCase".
msg415680 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-21 11:36
Again a relative short timeouts and super slow test boxes.
I₩'ll relax numbers.
Thanks forbthe report!
msg415682 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-21 12:57
> Again a relative short timeouts and super slow test boxes.

The problem is more that a sleep is not a reliable synchronization primitive:
https://pythondev.readthedocs.io/unstable_tests.html#don-t-use-sleep-as-synchronization
msg415729 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-22 00:04
> The problem is more that a sleep is not a reliable synchronization primitive

Yes, sure! I'm trying to avoid 'sleep for synchronization' when I'm writing new tests or fixing existing ones.

The problem with this particular queue tests is: I rewrote old-styled tests that used a loop time shift generator with IsolatedAsyncioTestCase keeping the minimal invasive changes.  It doesn't work well, now tiny shifts are removed; only 'await sleep(0)' are left when a bare context switch is needed without a delay.
msg415770 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-22 14:00
New changeset 673755bfbac46b3cd2c84d7e0d68c2c488e039c3 by Andrew Svetlov in branch 'main':
bpo-47076: Make asyncio.Queue stable on slow test boxes (GH-32040)
https://github.com/python/cpython/commit/673755bfbac46b3cd2c84d7e0d68c2c488e039c3
msg415771 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-22 14:01
Fixed.
msg415774 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-22 14:16
Thank you Andrew!
msg415778 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-22 14:47
Welcome!
History
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91232
2022-03-22 14:47:03asvetlovsetmessages: + msg415778
2022-03-22 14:16:21vstinnersetmessages: + msg415774
2022-03-22 14:01:59asvetlovsetstatus: open -> closed
resolution: fixed
messages: + msg415771

stage: patch review -> resolved
2022-03-22 14:00:33asvetlovsetmessages: + msg415770
2022-03-22 00:04:25asvetlovsetmessages: + msg415729
2022-03-21 23:55:45asvetlovsetpull_requests: + pull_request30130
2022-03-21 12:57:42vstinnersetmessages: + msg415682
2022-03-21 11:36:07asvetlovsetmessages: + msg415680
2022-03-21 06:16:29kumaradityasetkeywords: + patch
nosy: + kumaraditya

pull_requests: + pull_request30108
stage: patch review
2022-03-21 03:15:52vstinnersetmessages: + msg415655
2022-03-21 03:14:47vstinnercreate