classification
Title: test_ctypes hangs inside Portage build env since 'subprocess: Use vfork() instead of fork() [...]'
Type: Stage: resolved
Components: Tests Versions: Python 3.10
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: izbyshev, mgorny
Priority: normal Keywords:

Created on 2021-03-03 11:34 by mgorny, last changed 2021-03-03 12:30 by mgorny. This issue is now closed.

Messages (2)
msg388014 - (view) Author: Michał Górny (mgorny) * Date: 2021-03-03 11:33
So I've finally found time to bisect this.

Long story short, test_ctypes started hanging on Gentoo package builds since 3.10.0a2.  Previously, the test took less than a second.  Now, it just keeps running for minutes until I kill it.

The weird thing is that I can't reproduce it when running it manually.  I've tried hard to rebuild Portage-like environment to make it hang, to no avail.

I've finally gotten around to bisecting it, and established that the problem is caused by the following change:

```
commit 976da903a746a5455998e9ca45fbc4d3ad3479d8
Author: Alexey Izbyshev <izbyshev@ispras.ru>
Date:   2020-10-24 02:47:01 +0200

    bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe (GH-11671)
[...]
```

After running the test with a timeout, I get the following backtrace:

```
test_issue_8959_a (ctypes.test.test_callbacks.SampleCallbacksTestCase) ... Timeout (0:00:30)!
Thread 0x00007f72f2507740 (most recent call first):
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/subprocess.py", line 1773 in _execute_child
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/subprocess.py", line 962 in __init__
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/ctypes/util.py", line 289 in _findSoname_ldconfig
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/ctypes/util.py", line 329 in find_library
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/ctypes/test/test_callbacks.py", line 183 in test_issue_8959_a
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/case.py", line 549 in _callTestMethod
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/case.py", line 592 in run
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/case.py", line 652 in __call__
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 122 in run
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 84 in __call__
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 122 in run
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 84 in __call__
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 122 in run
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 84 in __call__
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 122 in run
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 84 in __call__
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 122 in run
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/suite.py", line 84 in __call__
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/unittest/runner.py", line 176 in run
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/support/__init__.py", line 959 in _run_suite
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/support/__init__.py", line 1082 in run_unittest
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/libregrtest/runtest.py", line 211 in _test_module
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/libregrtest/runtest.py", line 236 in _runtest_inner2
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/libregrtest/runtest.py", line 272 in _runtest_inner
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/libregrtest/runtest.py", line 155 in _runtest
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/libregrtest/runtest.py", line 195 in runtest
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/libregrtest/main.py", line 319 in rerun_failed_tests
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/libregrtest/main.py", line 696 in _main
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/libregrtest/main.py", line 639 in main
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/libregrtest/main.py", line 717 in main
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/test/__main__.py", line 2 in <module>
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/runpy.py", line 87 in _run_code
  File "/var/tmp/portage/dev-lang/python-3.10.0_alpha6/work/Python-3.10.0a6/Lib/runpy.py", line 197 in _run_module_as_main
make: *** [Makefile:1204: test] Error 1
```

I'd appreciate any help in debugging this further.
msg388016 - (view) Author: Michał Górny (mgorny) * Date: 2021-03-03 12:30
Nevermind, I've been testing wrong and this is most likely our fault.  I'm sorry about the noise.  I am going to investigate further and reopen if it turns out the problem's on CPython end.
History
Date User Action Args
2021-03-03 12:30:56mgornysetstatus: open -> closed

messages: + msg388016
stage: resolved
2021-03-03 11:34:00mgornycreate