Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Symbol resolution conflict when embeding python in an application using libedit #82815

Closed
serge-sans-paille mannequin opened this issue Oct 29, 2019 · 9 comments
Closed
Assignees
Labels
3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@serge-sans-paille
Copy link
Mannequin

serge-sans-paille mannequin commented Oct 29, 2019

BPO 38634
Nosy @gpshead, @cjwatson, @ned-deily, @pmp-p, @serge-sans-paille, @miss-islington, @tnir
PRs
  • bpo-38634: Allow non-apple build to cope with libedit #16986
  • [3.8] bpo-38634: Allow non-apple build to cope with libedit (GH-16986) #17466
  • [3.9] bpo-13501: allow choosing between readline and libedit #25420
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/gpshead'
    closed_at = <Date 2019-12-04.17:08:27.246>
    created_at = <Date 2019-10-29.15:17:05.169>
    labels = ['3.8', 'library', '3.9', 'type-crash']
    title = 'Symbol resolution conflict when embeding python in an application using libedit'
    updated_at = <Date 2021-04-15.09:38:01.674>
    user = 'https://github.com/serge-sans-paille'

    bugs.python.org fields:

    activity = <Date 2021-04-15.09:38:01.674>
    actor = 'vstinner'
    assignee = 'gregory.p.smith'
    closed = True
    closed_date = <Date 2019-12-04.17:08:27.246>
    closer = 'vstinner'
    components = ['Library (Lib)']
    creation = <Date 2019-10-29.15:17:05.169>
    creator = 'serge-sans-paille'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 38634
    keywords = ['patch']
    message_count = 9.0
    messages = ['355656', '355672', '355931', '357808', '357810', '357813', '378261', '378285', '386676']
    nosy_count = 7.0
    nosy_names = ['gregory.p.smith', 'cjwatson', 'ned.deily', 'pmpp', 'serge-sans-paille', 'miss-islington', 'tnir']
    pr_nums = ['16986', '17466', '25420']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue38634'
    versions = ['Python 3.8', 'Python 3.9']

    @serge-sans-paille
    Copy link
    Mannequin Author

    serge-sans-paille mannequin commented Oct 29, 2019

    See https://bugs.llvm.org/show_bug.cgi?id=43830, but basically the follwing code:

    // a.c
    #include <Python.h>
    
    int main() {
      Py_Initialize();
      PyRun_SimpleString("import readline; print(readline.__doc__)");
      return 0;
    }
    

    compiled like this:

    % gcc a.c `./install/bin/python3-config --cflags --ldflags --libs` -lpython3.9
    

    runs fine:

    % ./a.out                                          
    Importing this module enables command line editing using GNU readline.
    

    However the following:

    % gcc a.c `./install/bin/python3-config --cflags --ldflags --libs` -ledit -lpython3.9
    

    compiles but segfaults at runtime.

    @serge-sans-paille serge-sans-paille mannequin added the type-crash A hard crash of the interpreter, possibly with a core dump label Oct 29, 2019
    @ned-deily
    Copy link
    Member

    This appears to be essentially a duplicate of bpo-13631 which has long proposed full support of libedit.

    @serge-sans-paille
    Copy link
    Mannequin Author

    serge-sans-paille mannequin commented Nov 4, 2019

    @ned I(d rather see this as an evolution of bpo-13631, as this solves a problem when libreadline and libedit are both loaded in the same executable. As such, using libedit instead of readline wouldn't solve the issue: what if the program Python is embeded in is linked to readline?

    I find python approach relatively elegant: detect the linked library at runtime and use the ad-hoc implementation based on this.

    An other option would be to dlopen readline using the RTLD_LOCAL flag, so that we get a better, non intrusive symbol resolution.

    What do you think?

    @vstinner
    Copy link
    Member

    vstinner commented Dec 4, 2019

    New changeset 7105319 by Victor Stinner (serge-sans-paille) in branch 'master':
    bpo-38634: Allow non-apple build to cope with libedit (GH-16986)
    7105319

    @vstinner vstinner added stdlib Python modules in the Lib dir 3.8 only security fixes 3.9 only security fixes labels Dec 4, 2019
    @miss-islington
    Copy link
    Contributor

    New changeset 68669ef by Miss Islington (bot) in branch '3.8':
    bpo-38634: Allow non-apple build to cope with libedit (GH-16986)
    68669ef

    @vstinner
    Copy link
    Member

    vstinner commented Dec 4, 2019

    Thanks serge-sans-paille for the bug report and the fix!

    I backported it to 3.8 since it could be automated, but I don't think that it's worth it to backport it to 3.7:
    #16986 (comment)

    @vstinner vstinner closed this as completed Dec 4, 2019
    @cjwatson
    Copy link
    Mannequin

    cjwatson mannequin commented Oct 8, 2020

    FWIW I just ran into what I believe to be this bug with the Launchpad test suite on Python 3.6.9. The output shows some normal test output followed by:

    No entry for terminal type "unknown";
    using dumb terminal settings.
    bind: Invalid command `enable-meta-key'.
    No entry for terminal type "unknown";
    using dumb terminal settings.
    No entry for terminal type "unknown";
    using dumb terminal settings.
    Segmentation fault

    The test suite imports a lot of stuff, but it includes something which links against libLLVM which links against libedit (I haven't worked out exactly what, but probably GTK-related). Then something else much later imports readline - again, I haven't worked out what yet, as it's taken me a day just to set up the exact right environment in which to reproduce this at all and an strace doesn't make the cause of the import especially clear.

    So I understand why you see this as a rare use case, but it's *extremely* confusing and a massive time sink when you do run across it, as the cause is a long way distant from the effect and it can arise in situations where it is in no way deliberate to have this sort of setup, but rather an emergent effect of several other things.

    @cjwatson
    Copy link
    Mannequin

    cjwatson mannequin commented Oct 8, 2020

    Here's a reasonably minimal reproduction recipe reduced from real code in the Launchpad test suite that doesn't require compiling a separate C extension. It fails on Ubuntu 18.04 with the gir1.2-gtk-3.0, python3-gi, and xvfb packages installed. (The xvfb-run part is just so that it works on a headless system; you can omit it if you have a working $DISPLAY.)

    xvfb-run python3 -c 'from gi.repository import Gtk; import readline'

    @gpshead
    Copy link
    Member

    gpshead commented Feb 9, 2021

    New changeset e1f7769 by Roland Hieber in branch 'master':
    bpo-13501: allow choosing between readline and libedit (GH-24189)
    e1f7769

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants