Issue39367
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2020-01-17 11:17 by fhstoica, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Messages (7) | |||
---|---|---|---|
msg360170 - (view) | Author: Horace Stoica (fhstoica) | Date: 2020-01-17 11:17 | |
Built Python 3.8.1 from source on Fedora 30: kernel: 5.1.8-300.fc30.x86_64 Python 3.8.1 (default, Jan 15 2020, 08:49:34) [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux Installed the readline module as the arrows do not work when running Python interactively. ~/local/bin/pip3.8 install readline Collecting readline Using cached https://files.pythonhosted.org/packages/f4/01/2cf081af8d880b44939a5f1b446551a7f8d59eae414277fd0c303757ff1b/readline-6.2.4.1.tar.gz However, after installing readline Python core-dumps on Crtl+D or exit(): >>> exit() munmap_chunk(): invalid pointer Aborted (core dumped) I uninstalled the readline module and now it no longer core dumps. |
|||
msg360171 - (view) | Author: Ammar Askar (ammar2) * | Date: 2020-01-17 11:29 | |
> Installed the readline module as the arrows do not work when running Python interactively. This is not a suggested way to fix the issue, especially given that the module you installed is marked as deprecated: https://pypi.org/project/readline/ > Built Python 3.8.1 from source on Fedora 30: Instead, when you compile, look for a line like this Failed to find the necessary bits to build these modules: ... readline or Failed to build these modules: ... readline and then scan through the build log to either find the compiler error or you're most probably missing a dependency like readline-devel |
|||
msg360173 - (view) | Author: STINNER Victor (vstinner) * | Date: 2020-01-17 12:01 | |
I'm able to reproduce the issue on Fedora 31, but only if the 3rd party readline module is loaded, instead of the stdlib "readline" module. The problem is that https://pypi.org/project/readline/ misuses the Python memory allocator API. The GIL must be held to call PyMem_Malloc(). This project should be modified to use PyMem_RawMalloc(). https://docs.python.org/dev/c-api/memory.html#raw-memory-interface vs https://docs.python.org/dev/c-api/memory.html#memory-interface It's a bug in a 3rd party module, I close the issue. == packaged Python 3.8 == vstinner@apu$ python3.8 -m pip install --user readline Python still uses the stdlib module by default (sys.path order): vstinner@apu$ python3.8 Python 3.8.1 (default, Dec 19 2019, 00:00:00) [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import readline >>> readline <module 'readline' from '/usr/lib64/python3.8/lib-dynload/readline.cpython-38-x86_64-linux-gnu.so'> I'm using uninstall just to know the path, but then cancel: vstinner@apu$ python3.8 -m pip uninstall readline Uninstalling readline-6.2.4.1: Would remove: /home/vstinner/.local/lib/python3.8/site-packages/readline-6.2.4.1-py3.8.egg-info /home/vstinner/.local/lib/python3.8/site-packages/readline.cpython-38-x86_64-linux-gnu.so Proceed (y/n)? n I reproduce the issue if I force to prefer the 3rd party readline: vstinner@apu$ PYTHONPATH=/home/vstinner/.local/lib/python3.8/site-packages/ python3.8 Python 3.8.1 (default, Dec 19 2019, 00:00:00) [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import readline munmap_chunk(): invalid pointer Abandon (core dumped) == Python 3.9 compiled manually in debug mode == (...) vstinner@apu$ PYTHONPATH=/home/vstinner/python/master/env/lib/python3.9/site-packages/ gdb -args env/bin/python GNU gdb (GDB) Fedora 8.3.50.20190824-26.fc31 (gdb) run >>> import readline Fatal Python error: Python memory allocator called without holding the GIL Python runtime state: initialized Current thread 0x00007ffff7c1f740 (most recent call first): <no Python frame> Program received signal SIGABRT, Aborted. 0x00007ffff7c60625 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: dnf debuginfo-install libxcrypt-4.4.10-2.fc31.x86_64 ncurses-libs-6.1-12.20190803.fc31.x86_64 (gdb) where #0 0x00007ffff7c60625 in raise () from /lib64/libc.so.6 #1 0x00007ffff7c498d9 in abort () from /lib64/libc.so.6 #2 0x0000000000560b43 in fatal_error (prefix=0x0, msg=0x6b4c40 "Python memory allocator called without holding the GIL", status=-1) at Python/pylifecycle.c:2249 #3 0x0000000000560b6f in Py_FatalError (msg=0x6b4c40 "Python memory allocator called without holding the GIL") at Python/pylifecycle.c:2259 #4 0x000000000047d988 in _PyMem_DebugCheckGIL () at Objects/obmalloc.c:2320 #5 0x000000000047d9a0 in _PyMem_DebugMalloc (ctx=0x7cd950 <_PyMem_Debug+48>, nbytes=17) at Objects/obmalloc.c:2328 #6 0x000000000047b958 in PyMem_Malloc (size=17) at Objects/obmalloc.c:605 #7 0x00007fffeaa48dc0 in call_readline (sys_stdin=0x7ffff7de67e0 <_IO_2_1_stdin_>, sys_stdout=0x7ffff7de7500 <_IO_2_1_stdout_>, prompt=0x7fffeaa9c480 ">>> ") at Modules/3.x/readline.c:1122 #8 0x000000000069e8a4 in PyOS_Readline (sys_stdin=0x7ffff7de67e0 <_IO_2_1_stdin_>, sys_stdout=0x7ffff7de7500 <_IO_2_1_stdout_>, prompt=0x7fffeaa9c480 ">>> ") at Parser/myreadline.c:358 #9 0x0000000000615155 in tok_nextc (tok=0x988f00) at Parser/tokenizer.c:848 #10 0x0000000000615f7d in tok_get (tok=0x988f00, p_start=0x7fffffffd118, p_end=0x7fffffffd110) at Parser/tokenizer.c:1128 #11 0x0000000000617876 in PyTokenizer_Get (tok=0x988f00, p_start=0x7fffffffd118, p_end=0x7fffffffd110) at Parser/tokenizer.c:1774 #12 0x0000000000612862 in parsetok (tok=0x988f00, g=0x7f1ce0 <_PyParser_Grammar>, start=256, err_ret=0x7fffffffd280, flags=0x7fffffffd27c) at Parser/parsetok.c:251 #13 0x0000000000612672 in PyParser_ParseFileObject (fp=0x7ffff7de67e0 <_IO_2_1_stdin_>, filename='<stdin>', enc=0x7fffeab24340 "utf-8", g=0x7f1ce0 <_PyParser_Grammar>, start=256, ps1=0x7fffeaa9c480 ">>> ", ps2=0x7fffeaa9c520 "... ", err_ret=0x7fffffffd280, flags=0x7fffffffd27c) at Parser/parsetok.c:186 #14 0x0000000000567716 in PyParser_ASTFromFileObject (fp=0x7ffff7de67e0 <_IO_2_1_stdin_>, filename='<stdin>', enc=0x7fffeab24340 "utf-8", start=256, ps1=0x7fffeaa9c480 ">>> ", ps2=0x7fffeaa9c520 "... ", flags=0x7fffffffd470, errcode=0x7fffffffd314, arena=0x7fffeab2d7c0) at Python/pythonrun.c:1387 #15 0x0000000000564fc1 in PyRun_InteractiveOneObjectEx (fp=0x7ffff7de67e0 <_IO_2_1_stdin_>, filename='<stdin>', flags=0x7fffffffd470) at Python/pythonrun.c:239 #16 0x0000000000564cbd in PyRun_InteractiveLoopFlags (fp=0x7ffff7de67e0 <_IO_2_1_stdin_>, filename_str=0x69f43a "<stdin>", flags=0x7fffffffd470) at Python/pythonrun.c:121 #17 0x0000000000564b80 in PyRun_AnyFileExFlags (fp=0x7ffff7de67e0 <_IO_2_1_stdin_>, filename=0x69f43a "<stdin>", closeit=0, flags=0x7fffffffd470) at Python/pythonrun.c:80 #18 0x000000000041e910 in pymain_run_stdin (config=0x819240, cf=0x7fffffffd470) at Modules/main.c:467 #19 0x000000000041ec10 in pymain_run_python (exitcode=0x7fffffffd4bc) at Modules/main.c:556 #20 0x000000000041ece6 in Py_RunMain () at Modules/main.c:632 #21 0x000000000041ed60 in pymain_main (args=0x7fffffffd520) at Modules/main.c:662 #22 0x000000000041edda in Py_BytesMain (argc=1, argv=0x7fffffffd648) at Modules/main.c:686 #23 0x000000000041d796 in main (argc=1, argv=0x7fffffffd648) at ./Programs/python.c:16 |
|||
msg360175 - (view) | Author: STINNER Victor (vstinner) * | Date: 2020-01-17 12:12 | |
Oh right, Ammar is right, you miss a dependency. Try maybe: sudo dnf install -y dnf-plugins-core sudo dnf builddep -y python3 https://cpython-core-tutorial.readthedocs.io/en/latest/build_cpython_linux.html ... but why are you building Python from source? Fedora provides python3.8 ;-) At least, on Fedora 31, Python 3.8 and 3.9 are packaged ;-) If you built it to contribute to Python, that's great! |
|||
msg360176 - (view) | Author: Horace Stoica (fhstoica) | Date: 2020-01-17 12:33 | |
A big "thank you" to everyone, I installed readline-develop and rebuilt from source, and now the problem is fixed. The reason I am building from source is that Fedora 30 provides Python 3.7.3 and I wanted to try out the new features (like the walrus operator) that were introduced in version 3.8. I hope to get to the point of contributing to Python itself, for now I am looking mainly at extending Python with C++ and embedding Python in C++ applications. Thanks again, Horace |
|||
msg360178 - (view) | Author: STINNER Victor (vstinner) * | Date: 2020-01-17 12:36 | |
> The reason I am building from source is that Fedora 30 provides Python 3.7.3 and I wanted to try out the new features (like the walrus operator) that were introduced in version 3.8. Did you try "dnf install python38"? It's available on Fedora 30. FYI I'm part of the package maintainers of Python packages on Fedora and RHEL ;-) |
|||
msg360179 - (view) | Author: Horace Stoica (fhstoica) | Date: 2020-01-17 12:42 | |
Tried dnf install python38 and it works. Thank you! |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:25 | admin | set | github: 83548 |
2020-01-17 12:42:52 | fhstoica | set | messages: + msg360179 |
2020-01-17 12:36:11 | vstinner | set | messages: + msg360178 |
2020-01-17 12:33:41 | fhstoica | set | messages: + msg360176 |
2020-01-17 12:12:51 | vstinner | set | messages: + msg360175 |
2020-01-17 12:01:14 | vstinner | set | status: open -> closed nosy: + vstinner messages: + msg360173 resolution: third party stage: resolved |
2020-01-17 11:29:00 | ammar2 | set | nosy:
+ ammar2 messages: + msg360171 |
2020-01-17 11:17:46 | fhstoica | create |