This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Thread safety : disable intel’s compiler autopar where it’s being relevant.
Type: crash Stage:
Components: Build, C API, Interpreter Core Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Laël Cellier
Priority: normal Keywords:

Created on 2020-06-09 00:11 by Laël Cellier, last changed 2022-04-11 14:59 by admin.

Messages (9)
msg371066 - (view) Author: Laël Cellier (Laël Cellier) Date: 2020-06-09 00:11
As the bug tracker constantly crash over a continuation byte error while using latest Edgeʜᴛᴍʟ’s Edge browser the description is posted here : https://pastebin.com/5AU9HuQk
msg371067 - (view) Author: Laël Cellier (Laël Cellier) Date: 2020-06-09 00:12
(about the bug tracker I was meaning server‑side error message)
msg371068 - (view) Author: Laël Cellier (Laël Cellier) Date: 2020-06-09 00:16
It also turns out the problematic binary can’t be posted here because of a ʜᴛᴛᴘ 413 Entity too large error despite the attachment being only 4MB

https://filebin.net/4lp4nb61dav9qamo
msg371085 - (view) Author: Laël Cellier (Laël Cellier) Date: 2020-06-09 09:20
Unlike gcc, icc can pretty much parallelize everything while detecting cases like the mutexes used by the ɢɪʟ for deciding not to parallelize.

The requirement is to use a less conservative approach using -par-threshold97 while using -par-schedule-auto -qoverride-limits -parallel -ipo1 -O3 ‑ip inside icc.cfg and CFLAGS at the same time.

The build does produce the main python C library, but the python interpreter using it doesn’t even start :
Program received signal SIGSEGV, Segmentation fault.
0x00007fffff6128b0 in _PyEval_EvalFrameDefault () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
(gdb) bt
#0  0x00007fffff6128b0 in _PyEval_EvalFrameDefault () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#1  0x00007fffff3d55bc in _PyEval_EvalCodeWithName () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#2  0x00007fffff3d474b in _PyFunction_Vectorcall () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#3  0x00007fffff60da57 in call_function () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#4  0x00007fffff613fb4 in _PyEval_EvalFrameDefault () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#5  0x00007fffff3d55bc in _PyEval_EvalCodeWithName () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#6  0x00007fffff4bbbb2 in builtin___build_class__ () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#7  0x00007fffff42d72a in cfunction_vectorcall_FASTCALL_KEYWORDS () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#8  0x00007fffff60da57 in call_function () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#9  0x00007fffff613fb4 in _PyEval_EvalFrameDefault () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#10 0x00007fffff3d55bc in _PyEval_EvalCodeWithName () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#11 0x00007fffff4ea2ff in exec_code_in_module () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#12 0x00007fffff4e9fdf in PyImport_ImportFrozenModuleObject () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#13 0x00007fffff4e9d8f in PyImport_ImportFrozenModule () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#14 0x00007fffff5e3643 in init_importlib () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#15 0x00007fffff5e442c in pycore_init_import_warnings () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#16 0x00007fffff5e45ac in pyinit_config () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#17 0x00007fffff5e4922 in pyinit_core () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#18 0x00007fffff5eb67c in Py_InitializeFromConfig () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#19 0x00007fffff5d8fd0 in pymain_init () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#20 0x00007fffff50e099 in pymain_main () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#21 0x00007fffff50e071 in Py_BytesMain () from /home/example/rpmbuild/BUILD/Python-3.8.3/libpython3.8.so.1.0
#22 0x00000000004011b5 in main ()

The point of doing this is about significant speedups at some intensive workloads like those handling large strings.

The configure script is already detecting case where icc is used in order to append ‑fp‑model fast=1. It looks like the same should be done where (on the specific files) it’s relevant (but I’m not having enough time to determines where not even to debug this).

A better approach in order to fix this would be to ensure thread safety at the relevant places if we consider gcc might one day be able to automatically parallelize at that level which means disabling the compiler option for icc is only a temporary fix at best.
msg371089 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-06-09 10:46
It would be easier to understand this issue if you provide the exact
command line.

icc also segfaults without -fwrapv, see #40223, so that may be the
cause here, too.
msg371091 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-06-09 10:47
s/icc/Python compiled with icc/
msg371099 - (view) Author: Laël Cellier (Laël Cellier) Date: 2020-06-09 12:29
I’m using parallel studio xe 2020 and I didn’t receive any crash from icc itself. The problem is the resulting binary which can be downloaded here https://filebin.net/4lp4nb61dav9qamo but that can’t be posted on this issue that is crashing.

The only thing I put in CFLAGS is −O3 and the remaining options are inside icc.cfg
msg371108 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-06-09 14:25
Yes, the generated Python crashes (as the link I gave shows).  I'm using the command line, so I can't be of any assistance with Parallel Studio XE.
msg371109 - (view) Author: Laël Cellier (Laël Cellier) Date: 2020-06-09 14:30
I’m talking about the latest icc installed along with Parallel studio xe.
History
Date User Action Args
2022-04-11 14:59:32adminsetgithub: 85095
2020-06-09 14:30:51Laël Celliersetmessages: + msg371109
2020-06-09 14:25:24skrahsetnosy: - skrah
2020-06-09 14:25:17skrahsetmessages: + msg371108
2020-06-09 12:29:29Laël Celliersetmessages: + msg371099
2020-06-09 10:47:31skrahsetmessages: + msg371091
2020-06-09 10:46:35skrahsetnosy: + skrah
messages: + msg371089
2020-06-09 09:20:15Laël Celliersetmessages: + msg371085
2020-06-09 00:16:12Laël Celliersetmessages: + msg371068
2020-06-09 00:12:42Laël Celliersetmessages: + msg371067
2020-06-09 00:11:34Laël Celliercreate