Issue44877
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2021-08-09 23:37 by quellyn, last changed 2022-04-11 14:59 by admin.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
cpython.tar.gz | quellyn, 2021-08-09 23:37 | Tarball of output logs from configure and make |
Messages (9) | |||
---|---|---|---|
msg399296 - (view) | Author: Quellyn Snead (quellyn) * | Date: 2021-08-09 23:37 | |
Python 3.8 and above fails to build with the IBM XL compiler on Power9 platforms. ## System and Environment Information ``` $ arch ppc64le ``` ``` $ echo $CC xlc -F/projects/opt/ppc64le/ibm/xlc-16.1.1.7/xlC/16.1.1/etc/xlc.cfg.rhel.7.8.gcc.8.3.0.cuda.10.1 ``` ``` $ xlc --version IBM XL C/C++ for Linux, V16.1.1 (5725-C73, 5765-J13) Version: 16.01.0001.0007 ``` ## Test Procedure ``` $ git clone git@github.com:quellyn/cpython.git $ cd cpython $ checkout 3.8 $ ./configure --with-pydebug $ make -j2 | tee -a make.out ``` I tested for 3.7, 3.8, 3.9, 3.10, and master. In all cases the `make` failed for versions 3.8+. I've attached both the `config.log` and the make output logs for all. - Power9 architecture (ppc64le) - IBM XL C/C++ 16.1.1.7 Python 3.7 builds without issue. |
|||
msg399307 - (view) | Author: Eric V. Smith (eric.smith) * | Date: 2021-08-10 09:12 | |
Rather than have us wading through all of the output, can you summarize the problem? After a brief look I don't see any actual compiler errors. In make.out_3.9, at least, it looks like "generate-posix-vars failed", but I can't see why. What does ./python -E -S -m sysconfig --generate-posix-vars produce? Also, I'm not sure what the ".9" and ".10" files are supposed to be. Maybe accidentally included? |
|||
msg399335 - (view) | Author: Quellyn Snead (quellyn) * | Date: 2021-08-10 15:57 | |
Hi Eric, Looks like I didn't get the full output captured before. Sorry about that. The error appears when I run the make command: ``` [quellyn@cn2020 cpython]$ make -j ./python -E -S -m sysconfig --generate-posix-vars ;\ if test $? -ne 0 ; then \ echo "generate-posix-vars failed" ; \ rm -f ./pybuilddir.txt ; \ exit 1 ; \ fi Fatal Python error: take_gil: NULL tstate Python runtime state: preinitialized Current thread 0x0000200000045fe0 (most recent call first): /bin/sh: line 5: 79067 Segmentation fault (core dumped) ./python -E -S -m sysconfig --generate-posix-vars generate-posix-vars failed make: *** [pybuilddir.txt] Error 1 ``` P.S. The tarball I attached before contains the configure and make output for tests on multiple python version branches. I differentiated them by adding a version identifier suffix (_3.7, _3.8, _3.9, _3.10, _3.11) |
|||
msg399349 - (view) | Author: Jack DeVries (jack__d) * | Date: 2021-08-10 18:48 | |
There is a related failure message in the file name ".9" in the tarball (line 175): ./python -E -S -m sysconfig --generate-posix-vars ;\ if test $? -ne 0 ; then \ echo "generate-posix-vars failed" ; \ rm -f ./pybuilddir.txt ; \ exit 1 ; \ fi Fatal Python error: _PyMem_DebugMalloc: Python memory allocator called without holding the GIL Python runtime state: preinitialized Current thread 0x0000200000045fe0 (most recent call first): <no Python frame> /bin/sh: line 5: 122035 Aborted (core dumped) ./python -E -S -m sysconfig --generate-posix-vars generate-posix-vars failed |
|||
msg399350 - (view) | Author: Eric V. Smith (eric.smith) * | Date: 2021-08-10 18:50 | |
> P.S. The tarball I attached before contains the configure and make output for tests on multiple python version branches. I differentiated them by adding a version identifier suffix (_3.7, _3.8, _3.9, _3.10, _3.11) I was referring to the two files named exactly ".9" and ".10", which are hidden by default by the shell, since they begin with dots. |
|||
msg399351 - (view) | Author: Eric V. Smith (eric.smith) * | Date: 2021-08-10 18:52 | |
As to the actual problem, I think you're going to need to get out a debugger and at least get a stack trace. |
|||
msg399352 - (view) | Author: Jack DeVries (jack__d) * | Date: 2021-08-10 19:16 | |
I'm sure you are aware of this, but also note that the issue could be in pandas or ibm-db, which include C extensions. I'm pretty sure those are the only two dependencies you listed there that have C dependencies. |
|||
msg399353 - (view) | Author: Jack DeVries (jack__d) * | Date: 2021-08-10 19:17 | |
Woah, oops, nevermind! I was confusing this with a different bpo in my head. Sorry for the noise! |
|||
msg399355 - (view) | Author: Quellyn Snead (quellyn) * | Date: 2021-08-10 20:42 | |
> As to the actual problem, I think you're going to need to get out a debugger and at least get a stack trace. Here's my attempt: $ gdb ./python GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "ppc64le-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /vast/home/quellyn/GIT/cpython/python...done. (gdb) set args -E -S -m sysconfig --generate-posix-vars (gdb) run Starting program: /vast/home/quellyn/GIT/cpython/./python -E -S -m sysconfig --generate-posix-vars [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Could not find platform dependent libraries <exec_prefix> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>] Fatal Python error: take_gil: NULL tstate Python runtime state: preinitialized Current thread 0x0000200000045fe0 (most recent call first): Program received signal SIGSEGV, Segmentation fault. 0x00000000105a3870 in ?? () Missing separate debuginfos, use: debuginfo-install nss-softokn-freebl-3.53.1-6.el7_9.ppc64le (gdb) bt #0 0x00000000105a3870 in ?? () #1 0x000000001021c3a0 in dump_traceback (fd=2, tstate=0x105a3870, write_header=0) at Python/traceback.c:805 #2 0x000000001021c620 in _Py_DumpTracebackThreads (fd=2, interp=0x105a2b90, current_tstate=0x105a3870) at Python/traceback.c:915 #3 0x00000000101ee2ac in _Py_FatalError_DumpTracebacks (fd=2, interp=0x0, tstate=0x105a3870) at Python/pylifecycle.c:1981 #4 0x00000000101ee800 in fatal_error (prefix=0x0, msg=0x1041c7b4 "take_gil: NULL tstate", status=-1) at Python/pylifecycle.c:2159 #5 0x00000000101ee8b0 in Py_FatalError (msg=0x1041c7b4 "take_gil: NULL tstate") at Python/pylifecycle.c:2193 #6 0x000000001017ff04 in take_gil (ceval=0x1055c178 <_PyRuntime+584>, tstate=0x0) at Python/ceval_gil.h:187 #7 0x0000000010191db4 in PyEval_InitThreads () at Python/ceval.c:213 #8 0x00000000101f3490 in pycore_create_interpreter (runtime=0x1055bf30 <_PyRuntime>, config=0x105a2c40, interp_p=0x7fffffffd3c0) at Python/pylifecycle.c:550 #9 0x00000000101f35e0 in pyinit_config (runtime=0x1055bf30 <_PyRuntime>, interp_p=0x7fffffffd918, config=0x7fffffffd5f0) at Python/pylifecycle.c:674 #10 0x00000000101f3b38 in pyinit_core (runtime=0x1055bf30 <_PyRuntime>, src_config=0x7fffffffdac0, interp_p=0x7fffffffd918) at Python/pylifecycle.c:858 #11 0x00000000101f3dc4 in Py_InitializeFromConfig (config=0x7fffffffdac0) at Python/pylifecycle.c:1031 #12 0x000000001002409c in pymain_init (args=0x7fffffffde50) at Modules/main.c:78 ---Type <return> to continue, or q <return> to quit--- #13 0x000000001002428c in pymain_main (args=0x7fffffffde50) at Modules/main.c:710 #14 0x0000000010024394 in Py_BytesMain (argc=6, argv=0x7fffffffe2d8) at Modules/main.c:743 #15 0x0000000010021e18 in main (argc=6, argv=0x7fffffffe2d8) at ./Programs/python.c:16 (gdb) |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:48 | admin | set | github: 89040 |
2021-08-10 20:42:01 | quellyn | set | messages: + msg399355 |
2021-08-10 19:17:34 | jack__d | set | messages: + msg399353 |
2021-08-10 19:16:09 | jack__d | set | messages: + msg399352 |
2021-08-10 18:52:58 | eric.smith | set | messages: + msg399351 |
2021-08-10 18:50:28 | eric.smith | set | messages: + msg399350 |
2021-08-10 18:48:16 | jack__d | set | nosy:
+ jack__d messages: + msg399349 |
2021-08-10 15:57:13 | quellyn | set | messages: + msg399335 |
2021-08-10 09:12:41 | eric.smith | set | nosy:
+ eric.smith messages: + msg399307 |
2021-08-09 23:37:13 | quellyn | create |