This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Symbol resolution conflict when embeding python in an application using libedit
Type: crash Stage: resolved
Components: Library (Lib) Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: cjwatson, gregory.p.smith, miss-islington, ned.deily, pmpp, serge-sans-paille, tnir
Priority: normal Keywords: patch

Created on 2019-10-29 15:17 by serge-sans-paille, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 16986 merged serge-sans-paille, 2019-10-29 15:17
PR 17466 merged miss-islington, 2019-12-04 16:03
PR 25420 closed tnir, 2021-04-15 06:36
Messages (9)
msg355656 - (view) Author: (serge-sans-paille) * Date: 2019-10-29 15:17
See https://bugs.llvm.org/show_bug.cgi?id=43830, but basically the follwing code:

```
// a.c
#include <Python.h>

int main() {
  Py_Initialize();
  PyRun_SimpleString("import readline; print(readline.__doc__)");
  return 0;
}
```

compiled like this:

```
% gcc a.c `./install/bin/python3-config --cflags --ldflags --libs` -lpython3.9
```

runs fine:

```
% ./a.out                                          
Importing this module enables command line editing using GNU readline.
```

However the following:

```
% gcc a.c `./install/bin/python3-config --cflags --ldflags --libs` -ledit -lpython3.9
```

compiles but segfaults at runtime.
msg355672 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-10-29 19:42
This appears to be essentially a duplicate of Issue13631 which has long proposed full support of libedit.
msg355931 - (view) Author: (serge-sans-paille) * Date: 2019-11-04 08:27
@ned I(d rather see this as an evolution of Issue13631, as this solves a problem when libreadline and libedit are both loaded in the same executable. As such, using libedit instead of readline wouldn't solve the issue: what if the program Python is embeded in is linked to readline?

I find python approach relatively elegant: detect the linked library at runtime and use the ad-hoc implementation based on this.

An other option would be to dlopen readline using the RTLD_LOCAL flag, so that we get a better, non intrusive symbol resolution.

What do you think?
msg357808 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-12-04 16:03
New changeset 7105319ada2e663659020cbe9fdf7ff38f421ab2 by Victor Stinner (serge-sans-paille) in branch 'master':
 bpo-38634: Allow non-apple build to cope with libedit (GH-16986)
https://github.com/python/cpython/commit/7105319ada2e663659020cbe9fdf7ff38f421ab2
msg357810 - (view) Author: miss-islington (miss-islington) Date: 2019-12-04 16:21
New changeset 68669ef7883ea6338ca441e50f4f9d975f54d017 by Miss Islington (bot) in branch '3.8':
bpo-38634: Allow non-apple build to cope with libedit (GH-16986)
https://github.com/python/cpython/commit/68669ef7883ea6338ca441e50f4f9d975f54d017
msg357813 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-12-04 17:08
Thanks serge-sans-paille for the bug report and the fix!

I backported it to 3.8 since it could be automated, but I don't think that it's worth it to backport it to 3.7:
https://github.com/python/cpython/pull/16986#issuecomment-561711232
msg378261 - (view) Author: Colin Watson (cjwatson) * Date: 2020-10-08 17:33
FWIW I just ran into what I believe to be this bug with the Launchpad test suite on Python 3.6.9.  The output shows some normal test output followed by:

No entry for terminal type "unknown";
using dumb terminal settings.
bind: Invalid command `enable-meta-key'.
No entry for terminal type "unknown";
using dumb terminal settings.
No entry for terminal type "unknown";
using dumb terminal settings.
Segmentation fault

The test suite imports a lot of stuff, but it includes something which links against libLLVM which links against libedit (I haven't worked out exactly what, but probably GTK-related).  Then something else much later imports readline - again, I haven't worked out what yet, as it's taken me a day just to set up the exact right environment in which to reproduce this at all and an strace doesn't make the cause of the import especially clear.

So I understand why you see this as a rare use case, but it's *extremely* confusing and a massive time sink when you do run across it, as the cause is a long way distant from the effect and it can arise in situations where it is in no way deliberate to have this sort of setup, but rather an emergent effect of several other things.
msg378285 - (view) Author: Colin Watson (cjwatson) * Date: 2020-10-08 19:45
Here's a reasonably minimal reproduction recipe reduced from real code in the Launchpad test suite that doesn't require compiling a separate C extension.  It fails on Ubuntu 18.04 with the gir1.2-gtk-3.0, python3-gi, and xvfb packages installed.  (The xvfb-run part is just so that it works on a headless system; you can omit it if you have a working $DISPLAY.)

  xvfb-run python3 -c 'from gi.repository import Gtk; import readline'
msg386676 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2021-02-09 01:06
New changeset e1f77695132e814728cda982f11342a2e3c7272c by Roland Hieber in branch 'master':
bpo-13501: allow choosing between readline and libedit (GH-24189)
https://github.com/python/cpython/commit/e1f77695132e814728cda982f11342a2e3c7272c
History
Date User Action Args
2022-04-11 14:59:22adminsetgithub: 82815
2021-04-15 09:38:01vstinnersetnosy: - vstinner
2021-04-15 06:36:36tnirsetnosy: + tnir

pull_requests: + pull_request24152
2021-02-09 01:06:01gregory.p.smithsetmessages: + msg386676
2020-10-08 19:45:44cjwatsonsetmessages: + msg378285
2020-10-08 17:33:46cjwatsonsetnosy: + cjwatson
messages: + msg378261
2019-12-04 17:08:27vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg357813

stage: patch review -> resolved
2019-12-04 16:21:21miss-islingtonsetnosy: + miss-islington
messages: + msg357810
2019-12-04 16:03:58vstinnersetcomponents: + Library (Lib)
versions: + Python 3.8, Python 3.9
2019-12-04 16:03:22miss-islingtonsetpull_requests: + pull_request16945
2019-12-04 16:03:04vstinnersetnosy: + vstinner
messages: + msg357808
2019-11-04 08:27:47serge-sans-paillesetmessages: + msg355931
2019-10-29 19:42:04ned.deilysetassignee: gregory.p.smith

messages: + msg355672
nosy: + gregory.p.smith, ned.deily
2019-10-29 15:28:37pmppsetnosy: + pmpp
2019-10-29 15:17:41serge-sans-paillesetkeywords: + patch
stage: patch review
pull_requests: + pull_request16512
2019-10-29 15:17:05serge-sans-paillecreate