This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Alexander Kanavin
Recipients Alexander Kanavin
Date 2021-12-01.08:58:16
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1638349096.96.0.35958886089.issue45945@roundup.psfhosted.org>
In-reply-to
Content
Hello, Yocto project has had to disable -j0 for compileall, so that it runs serially. If it doesn't, then 'make install' locks up sporadically, with hanging processes:
```
3837320 ?        SN     0:00                  \_ make -j 16 -l 52 STAGING_LIBDIR=/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280438/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/recipe-sysroot/usr/lib STAGING_INCDIR=/
 157523 ?        SNl    0:02                      \_ python3.10 -Wi -OO /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280438/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/compileall.py -j0 -d /u
 160673 ?        SN     0:00                          \_ python3.10 -Wi -OO /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280438/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/compileall.py -j0 -
 160677 ?        SN     0:00                          \_ python3.10 -Wi -OO /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280438/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/compileall.py -j0 -
 160682 ?        SN     0:00                          \_ python3.10 -Wi -OO /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280438/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/compileall.py -j0 -
 160697 ?        SN     0:00                          \_ python3.10 -Wi -OO /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280438/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/compileall.py -j0 -
 160759 ?        SN     0:00                          \_ python3.10 -Wi -OO /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280438/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/compileall.py -j0 -
 160816 ?        SN     0:00                          \_ python3.10 -Wi -OO /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280438/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/compileall.py -j0 -
...
```

and installation log reveals:
```
poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/compileall.py \
        -j0 -d /usr/lib/python3.10 -f \
        -x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
        /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10
Listing '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10'...
Listing '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/asyncio'...
Listing '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/collections'...
Listing '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/concurrent'...
Listing '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/concurrent/futures'...
Listing '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/config-3.10-x86_64-linux-gnu'...
Listing '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/ctypes'...
Listing '/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/ctypes/macholib'...
...
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/concurrent/futures/process.py", line 317, in run
    result_item, is_broken, cause = self.wait_result_broken_or_wakeup()
  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/concurrent/futures/process.py", line 376, in wait_result_broken_or_wakeup
    worker_sentinels = [p.sentinel for p in self.processes.values()]
  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1280388/tmp/work/core2-64-poky-linux/python3/3.10.0-r0/image/usr/lib/python3.10/concurrent/futures/process.py", line 376, in <listcomp>
    worker_sentinels = [p.sentinel for p in self.processes.values()]
RuntimeError: dictionary changed size during iteration
```
History
Date User Action Args
2021-12-01 08:58:17Alexander Kanavinsetrecipients: + Alexander Kanavin
2021-12-01 08:58:16Alexander Kanavinsetmessageid: <1638349096.96.0.35958886089.issue45945@roundup.psfhosted.org>
2021-12-01 08:58:16Alexander Kanavinlinkissue45945 messages
2021-12-01 08:58:16Alexander Kanavincreate