This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_urllib: testInterruptCaught() has a race condition and fails randomly
Type: Stage: resolved
Components: Tests Versions: Python 3.11
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Regression caused by CALL_FUNCTION specialization for C function calls (test_urllib fails when run multiple times)
View: 46465
Assigned To: Nosy List: Mark.Shannon, kumaraditya, sobolevn, vstinner, xtreak
Priority: normal Keywords: patch

Created on 2022-02-10 12:04 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 31273 closed sobolevn, 2022-02-11 09:09
PR 31404 merged sobolevn, 2022-02-18 10:58
Messages (7)
msg412993 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-02-10 12:04
test_urllib failed and then passed when re-run on s390x RHEL7 Refleaks 3.x:
https://buildbot.python.org/all/#builders/129/builds/300

I can reproduce the issue on my Linux laptop:

$ ./python -m test -m unittest.test.test_break.TestBreakDefaultIntHandler.testInterruptCaught test_unittest -F
0:00:00 load avg: 1.52 Run tests sequentially
0:00:00 load avg: 1.52 [  1] test_unittest
0:00:00 load avg: 1.52 [  2] test_unittest
0:00:00 load avg: 1.52 [  3] test_unittest
0:00:00 load avg: 1.52 [  4] test_unittest
0:00:00 load avg: 1.52 [  5] test_unittest
0:00:01 load avg: 1.52 [  6] test_unittest
0:00:01 load avg: 1.52 [  7] test_unittest
0:00:01 load avg: 1.52 [  8] test_unittest
test test_unittest failed -- Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/unittest/test/test_break.py", line 66, in testInterruptCaught
    test(result)
    ^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/unittest/test/test_break.py", line 63, in test
    self.assertTrue(result.shouldStop)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: False is not true

test_unittest failed (1 failure)

== Tests result: FAILURE ==

7 tests OK.

1 test failed:
    test_unittest

Total duration: 1.7 sec
Tests result: FAILURE
msg413040 - (view) Author: Nikita Sobolev (sobolevn) * (Python triager) Date: 2022-02-11 07:43
I am trying to debug this.

Several intersting notes:
1. `time.sleep()` does not help
2. It always fails on `8`th turn
3. Changing `self.assertTrue(result.shouldStop)` to

```
msg = f'{type(result)} {vars(result)}'
self.assertTrue(result.shouldStop, msg)
```

fixes the problem.
msg413041 - (view) Author: Nikita Sobolev (sobolevn) * (Python triager) Date: 2022-02-11 07:46
I think this might be a side effect of https://docs.python.org/3/library/signal.html#execution-of-python-signal-handlers

> A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the virtual machine to execute the corresponding Python signal handler at a later point(for example at the next bytecode instruction). This has consequences
msg413043 - (view) Author: Nikita Sobolev (sobolevn) * (Python triager) Date: 2022-02-11 07:52
Other tests are also affected:
- `./python.exe -m test -m unittest.test.test_break.TestBreakDefaultIntHandler.testSecondInterrupt test_unittest -F`
- `./python.exe -m test -m unittest.test.test_break.TestBreakDefaultIntHandler.testTwoResults test_unittest -F`
- `./python.exe -m test -m unittest.test.test_break.TestBreakDefaultIntHandler.testHandlerReplacedButCalled test_unittest -F`
- etc

I think that we need a universal solution, I am going to write a helper method and add it to all tests there.
msg413057 - (view) Author: Kumar Aditya (kumaraditya) * (Python triager) Date: 2022-02-11 11:03
Same issue as https://bugs.python.org/issue46465, PR https://github.com/python/cpython/pull/30826 fixes this.
msg413068 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-02-11 13:20
Oops, I forgot about my own bpo-46465. I failed to find it when I searched for "test_urllib".
msg413479 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2022-02-18 12:56
New changeset e2c28616ce6c3cdb1013c415125220a0b86b86a1 by Nikita Sobolev in branch 'main':
bpo-46709: check eval breaker in specialized `CALL` opcodes (GH-31404)
https://github.com/python/cpython/commit/e2c28616ce6c3cdb1013c415125220a0b86b86a1
History
Date User Action Args
2022-04-11 14:59:56adminsetgithub: 90865
2022-02-18 12:56:30Mark.Shannonsetnosy: + Mark.Shannon
messages: + msg413479
2022-02-18 10:58:28sobolevnsetpull_requests: + pull_request29544
2022-02-11 13:20:16vstinnersetstatus: open -> closed
superseder: Regression caused by CALL_FUNCTION specialization for C function calls (test_urllib fails when run multiple times)
messages: + msg413068

resolution: duplicate
stage: patch review -> resolved
2022-02-11 11:03:34kumaradityasetnosy: + kumaraditya
messages: + msg413057
2022-02-11 09:09:23sobolevnsetkeywords: + patch
stage: patch review
pull_requests: + pull_request29435
2022-02-11 07:52:35sobolevnsetmessages: + msg413043
2022-02-11 07:46:08sobolevnsetmessages: + msg413041
2022-02-11 07:43:20sobolevnsetnosy: + sobolevn
messages: + msg413040
2022-02-10 13:58:21xtreaksetnosy: + xtreak
2022-02-10 12:04:16vstinnercreate