Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

eval frame rename in pep 0523 broke gdb's python extension #75166

Closed
bcap mannequin opened this issue Jul 21, 2017 · 15 comments
Closed

eval frame rename in pep 0523 broke gdb's python extension #75166

bcap mannequin opened this issue Jul 21, 2017 · 15 comments
Labels
3.7 (EOL) end of life

Comments

@bcap
Copy link
Mannequin

bcap mannequin commented Jul 21, 2017

BPO 30983
Nosy @vstinner, @ambv, @cooperlees, @bcap
PRs
  • bpo-30983: eval frame rename in pep 0523 broke gdb's python extension #2803
  • [3.6] bpo-30983: eval frame rename in pep 0523 broke gdb's python extension (GH-2803) #3090
  • bpo-30983: Add Bruno Penteado to ACKS #3091
  • bpo-30983: Revert changes which broke most buildbots #3100
  • [3.6] Revert "[3.6] bpo-30983: eval frame rename in pep 0523 broke gdb's py… #3101
  • bpo-30983: [gdb] Fix py-bt, etc. for non-debug shared builds #3153
  • bpo-30983: [gdb] Fix py-bt, etc. for non-debug shared builds #3153
  • [3.6] bpo-30983: [gdb] Fix py-bt, etc. for non-debug shared builds (GH-3153) #3192
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2017-08-22.20:23:45.657>
created_at = <Date 2017-07-21.13:02:35.557>
labels = ['3.7']
title = "eval frame rename in pep 0523 broke gdb's python extension"
updated_at = <Date 2017-08-22.20:23:45.656>
user = 'https://github.com/bcap'

    bugs.python.org fields:

    activity = <Date 2017-08-22.20:23:45.656>
actor = 'lukasz.langa'
assignee = 'none'
closed = True
closed_date = <Date 2017-08-22.20:23:45.657>
closer = 'lukasz.langa'
components = []
creation = <Date 2017-07-21.13:02:35.557>
creator = 'bcap'
dependencies = []
files = []
hgrepos = []
issue_num = 30983
keywords = []
message_count = 15.0
messages = ['298804', '300267', '300269', '300332', '300333', '300340', '300341', '300457', '300461', '300462', '300528', '300529', '300552', '300662', '300720']
nosy_count = 4.0
nosy_names = ['vstinner', 'lukasz.langa', 'cooperlees', 'bcap']
pr_nums = ['2803', '3090', '3091', '3100', '3101', '3153', '3153', '3192']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue30983'
versions = ['Python 3.6', 'Python 3.7']
    @bcap
    Copy link
    Mannequin Author

    bcap mannequin commented Jul 21, 2017

    PEP-0523 renames PyEval_EvalFrameEx to _PyEval_EvalFrameDefault while the gdb python extension only looks for PyEval_EvalFrameEx to understand if it is dealing with a frame.

    Final effect is that attaching gdb to a python3.6 process doesnt resolve python objects. Eg. py-list and py-bt dont work properly

    @bcap bcap mannequin added the 3.7 (EOL) end of life label Jul 21, 2017
    @bcap bcap mannequin changed the title eval frame rename in pep 0523 broke gdp's python extension eval frame rename in pep 0523 broke gdb's python extension Jul 21, 2017
    @ambv
    Copy link
    Contributor

    ambv commented Aug 14, 2017

    New changeset 2e0f4db by Łukasz Langa (Bruno "Polaco" Penteado) in branch 'master':
    bpo-30983: eval frame rename in PEP-0523 broke gdb's python extension (bpo-2803)
    2e0f4db

    @ambv
    Copy link
    Contributor

    ambv commented Aug 14, 2017

    New changeset 09b7716 by Łukasz Langa in branch '3.6':
    [3.6] bpo-30983: eval frame rename in PEP-0523 broke gdb's python extension (GH-2803) (bpo-3090)
    09b7716

    @ambv ambv closed this as completed Aug 14, 2017
    @vstinner
    Copy link
    Member

    This change broke test_gdb on most buildbots :-(

    http://buildbot.python.org/all/builders/x86%20Ubuntu%20Shared%203.x/builds/1159/steps/test/logs/stdio

    ======================================================================
    FAIL: test_pyup_command (test.test_gdb.StackNavigationTests)
    Verify that the "py-up" command works
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_gdb.py", line 688, in test_pyup_command
    $''')
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_gdb.py", line 259, in assertMultilineMatches
    self.fail(msg='%r did not match %r' % (actual, pattern))
AssertionError: 'Breakpoint 1 (builtin_id) pending.\n[Thread debugging using libthread_db enabled]\nUsing host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".\n\nBreakpoint 1, builtin_id (self=self@entry=, v=42) at Python/bltinmodule.c:1103\n1103\t{\n#4 Frame 0x4086b4cc, for file /srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/gdb_sample.py, line 10, in baz (args=(1, 2, 3))\n    id(42)\n#5 Frame 0x4086b4cc, for file /srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/gdb_sample.py, line 10, in baz (args=(1, 2, 3))\n    id(42)\n' did not match '^.*\n#[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 7, in bar \\(a=1, b=2, c=3\\)\n    baz\\(a, b, c\\)\n$'

    @vstinner vstinner reopened this Aug 16, 2017
    @vstinner
    Copy link
    Member

    I chose to revert the change because I don't have the bandwidth right now to investigate why the change broke test_gdb.

    I'm surprised that a change affecting python-gdb.py wasn't properly tested manually using test_gdb.py :-( I understand that Travis CI doesn't have gdb and/or that the test pass in some cases?

    The revert only gives us more time to design the proper solution.

    @vstinner
    Copy link
    Member

    New changeset 7cc3399 by Victor Stinner in branch 'master':
    bpo-30983: Revert changes which broke most buildbots (bpo-3100)
    7cc3399

    @vstinner
    Copy link
    Member

    New changeset 98e2697 by Victor Stinner in branch '3.6':
    Revert "[3.6] bpo-30983: eval frame rename in PEP-0523 broke gdb's python extension (GH-2803) (bpo-3090)" (bpo-3101)
    98e2697

    @ambv
    Copy link
    Contributor

    ambv commented Aug 17, 2017

    In this case it would be great if the buildbots commented on the relevant pull request that it broke the build. Otherwise this creates this dissonance that it was all green during merging of the PR when in fact it wasn't.

    I think we need to make test_gdb run on Travis, this is an unfortunate omission.

    @vstinner
    Copy link
    Member

    While our CI is getting better everyday, it still has a few flaws.

    There is no automated buildbot failure notification to the author and
    committer yet because random failures were very common 6 months ago. I
    fixed something like 90% of the most common race conditions (in tests but
    also in the code), and now the remaining 10% are the least funny ones :-)
    My latest report:
    https://haypo.github.io/python-buildbots-2017q2.html

    In the meanwhile I check manually buildbots output and report bugs and
    regressions.

    For gdb, don't worry, the buildbots were fixed quickly, it's fine if the
    pre-commit CI missed the issue since gdb changes are rare. I am confident
    that we can design a fix without introducing regressions ;-)

    @ambv
    Copy link
    Contributor

    ambv commented Aug 17, 2017

    I am unable to repro this locally. With the changes to libpython.py test_gdb still passes on CentOS 7 and macOS 10.12.

    @ambv
    Copy link
    Contributor

    ambv commented Aug 18, 2017

    OK, I got the repro. You have to rebuild python for /Tools/gdb/libpython.py to land in /python-gdb.py. The latter is used during test_gdb.py.

    @vstinner
    Copy link
    Member

    You have to rebuild python for /Tools/gdb/libpython.py to land in
    /python-gdb.py. The latter is used during test_gdb.py.

    Ah yes, sorry, that's not obvious. "make" does the copy for you.

    @ambv
    Copy link
    Contributor

    ambv commented Aug 18, 2017

    The issue originally stems from the fact that a non-debug Python build with --enable-shared is inlining PyEval_EvalFrameEx into _PyEval_EvalFrameDefault.

    So, the patch should really become to *replace* discovering PyEval_EvalFrameEx with the discovery of _PyEval_EvalFrameDefault. This might be problematic in the future when there is an actual JIT but currently it's strictly better than the current situation where the gdb bindings simply don't work for shared builds.

    @ambv
    Copy link
    Contributor

    ambv commented Aug 21, 2017

    New changeset 5fe59f8 by Łukasz Langa in branch 'master':
    bpo-30983: [gdb] Fix py-bt, etc. for non-debug shared builds (bpo-3153)
    5fe59f8

    @ambv
    Copy link
    Contributor

    ambv commented Aug 22, 2017

    New changeset 8e57249 by Łukasz Langa in branch '3.6':
    [3.6] bpo-30983: [gdb] Fix py-bt, etc. for non-debug shared builds (GH-3153) (bpo-3192)
    8e57249

    @ambv ambv closed this as completed Aug 22, 2017
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @ambv @vstinner