Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.7.0b2 Interpreter crash in dev mode (or with PYTHONMALLOC=debug) with 'python -X dev -c 'import os; os.fork()' #77186

Closed
jamadden mannequin opened this issue Mar 5, 2018 · 8 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@jamadden
Copy link
Mannequin

jamadden mannequin commented Mar 5, 2018

BPO 33005
Nosy @vstinner, @ericsnowcurrently, @jamadden, @zhangyangyu, @miss-islington
PRs
  • bpo-33005: Fix _PyGILState_Reinit() #6001
  • [3.7] bpo-33005: Fix _PyGILState_Reinit() (GH-6001) #6003
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2018-03-06.14:30:30.166>
    created_at = <Date 2018-03-05.22:40:09.571>
    labels = ['interpreter-core', '3.7', '3.8']
    title = "3.7.0b2 Interpreter crash in dev mode (or with PYTHONMALLOC=debug) with 'python -X dev -c 'import os; os.fork()'"
    updated_at = <Date 2018-03-06.15:01:11.300>
    user = 'https://github.com/jamadden'

    bugs.python.org fields:

    activity = <Date 2018-03-06.15:01:11.300>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2018-03-06.14:30:30.166>
    closer = 'vstinner'
    components = ['Interpreter Core']
    creation = <Date 2018-03-05.22:40:09.571>
    creator = 'jmadden'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 33005
    keywords = ['patch']
    message_count = 8.0
    messages = ['313296', '313297', '313318', '313324', '313325', '313326', '313328', '313329']
    nosy_count = 5.0
    nosy_names = ['vstinner', 'eric.snow', 'jmadden', 'xiang.zhang', 'miss-islington']
    pr_nums = ['6001', '6003']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue33005'
    versions = ['Python 3.7', 'Python 3.8']

    @jamadden
    Copy link
    Mannequin Author

    jamadden mannequin commented Mar 5, 2018

    At the request of Victor Stinner on twitter, I ran the gevent test suite with Python 3.7.0b2 with the new '-X dev' argument and discovered an interpreter crash. With a bit of work, it boiled down to a very simple command:

    $ env -i .runtimes/snakepit/python3.7.0b2 -X dev -c 'import os; os.fork()'
    *** Error in `.runtimes/snakepit/python3.7.0b2': munmap_chunk(): invalid pointer: 0x0000000001c43a80 ***
    ======= Backtrace: 

    =========
    /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f5a971607e5]
    /lib/x86_64-linux-gnu/libc.so.6(cfree+0x1a8)[0x7f5a9716d698]
    .runtimes/snakepit/python3.7.0b2(_PyRuntimeState_Fini+0x30)[0x515d90]
    .runtimes/snakepit/python3.7.0b2[0x51445f]
    .runtimes/snakepit/python3.7.0b2[0x42ce40]
    .runtimes/snakepit/python3.7.0b2(_Py_UnixMain+0x7b)[0x42eaab]
    /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f5a97109830]
    .runtimes/snakepit/python3.7.0b2(_start+0x29)[0x42a0d9]
    ======= Memory map: ========
    00400000-00689000 r-xp 0000000 08:01 177409 //.runtimes/versions/python3.7.0b2/bin/python3.7
    00888000-00889000 r--p 00288000 08:01 177409 //.runtimes/versions/python3.7.0b2/bin/python3.7
    00889000-008f3000 rw-p 00289000 08:01 177409 //.runtimes/versions/python3.7.0b2/bin/python3.7
    008f3000-00914000 rw-p 0000000 00:00 0
    01b84000-01c64000 rw-p 0000000 00:00 0 [heap]
    7f5a96052000-7f5a96068000 r-xp 0000000 08:01 265946 /lib/x86_64-linux-gnu/libgcc_s.so.1
    7f5a96068000-7f5a96267000 ---p 00016000 08:01 265946 /lib/x86_64-linux-gnu/libgcc_s.so.1
    7f5a96267000-7f5a96268000 rw-p 00015000 08:01 265946 /lib/x86_64-linux-gnu/libgcc_s.so.1
    7f5a96268000-7f5a96273000 r-xp 0000000 08:01 268943 /lib/x86_64-linux-gnu/libnss_files-2.23.so
    7f5a96273000-7f5a96472000 ---p 0000b000 08:01 268943 /lib/x86_64-linux-gnu/libnss_files-2.23.so
    7f5a96472000-7f5a96473000 r--p 0000a000 08:01 268943 /lib/x86_64-linux-gnu/libnss_files-2.23.so
    7f5a96473000-7f5a96474000 rw-p 0000b000 08:01 268943 /lib/x86_64-linux-gnu/libnss_files-2.23.so
    7f5a96474000-7f5a9647a000 rw-p 0000000 00:00 0
    7f5a9647a000-7f5a96485000 r-xp 0000000 08:01 268947 /lib/x86_64-linux-gnu/libnss_nis-2.23.so
    7f5a96485000-7f5a96684000 ---p 0000b000 08:01 268947 /lib/x86_64-linux-gnu/libnss_nis-2.23.so
    7f5a96684000-7f5a96685000 r--p 0000a000 08:01 268947 /lib/x86_64-linux-gnu/libnss_nis-2.23.so
    7f5a96685000-7f5a96686000 rw-p 0000b000 08:01 268947 /lib/x86_64-linux-gnu/libnss_nis-2.23.so
    7f5a96686000-7f5a9669c000 r-xp 0000000 08:01 268927 /lib/x86_64-linux-gnu/libnsl-2.23.so
    7f5a9669c000-7f5a9689b000 ---p 00016000 08:01 268927 /lib/x86_64-linux-gnu/libnsl-2.23.so
    7f5a9689b000-7f5a9689c000 r--p 00015000 08:01 268927 /lib/x86_64-linux-gnu/libnsl-2.23.so
    7f5a9689c000-7f5a9689d000 rw-p 00016000 08:01 268927 /lib/x86_64-linux-gnu/libnsl-2.23.so
    7f5a9689d000-7f5a9689f000 rw-p 0000000 00:00 0
    7f5a9689f000-7f5a968a7000 r-xp 0000000 08:01 268938 /lib/x86_64-linux-gnu/libnss_compat-2.23.so
    7f5a968a7000-7f5a96aa6000 ---p 00008000 08:01 268938 /lib/x86_64-linux-gnu/libnss_compat-2.23.so
    7f5a96aa6000-7f5a96aa7000 r--p 00007000 08:01 268938 /lib/x86_64-linux-gnu/libnss_compat-2.23.so
    7f5a96aa7000-7f5a96aa8000 rw-p 00008000 08:01 268938 /lib/x86_64-linux-gnu/libnss_compat-2.23.so
    7f5a96acc000-7f5a96b4c000 rw-p 0000000 00:00 0
    7f5a96b4c000-7f5a96b4e000 r-xp 0000000 08:01 184551 //.runtimes/versions/python3.7.0b2/lib/python3.7/lib-dynload/_heapq.cpython-37m-x86_64-linux-gnu.so
    7f5a96b4e000-7f5a96d4e000 ---p 00002000 08:01 184551 //.runtimes/versions/python3.7.0b2/lib/python3.7/lib-dynload/_heapq.cpython-37m-x86_64-linux-gnu.so
    7f5a96d4e000-7f5a96d4f000 r--p 00002000 08:01 184551 //.runtimes/versions/python3.7.0b2/lib/python3.7/lib-dynload/_heapq.cpython-37m-x86_64-linux-gnu.so
    7f5a96d4f000-7f5a96d51000 rw-p 00003000 08:01 184551 //.runtimes/versions/python3.7.0b2/lib/python3.7/lib-dynload/_heapq.cpython-37m-x86_64-linux-gnu.so
    7f5a96d51000-7f5a96e11000 rw-p 0000000 00:00 0
    7f5a96e11000-7f5a970e9000 r--p 0000000 08:01 133586 /usr/lib/locale/locale-archive
    7f5a970e9000-7f5a972a9000 r-xp 0000000 08:01 268930 /lib/x86_64-linux-gnu/libc-2.23.so
    7f5a972a9000-7f5a974a9000 ---p 001c0000 08:01 268930 /lib/x86_64-linux-gnu/libc-2.23.so
    7f5a974a9000-7f5a974ad000 r--p 001c0000 08:01 268930 /lib/x86_64-linux-gnu/libc-2.23.so
    7f5a974ad000-7f5a974af000 rw-p 001c4000 08:01 268930 /lib/x86_64-linux-gnu/libc-2.23.so
    7f5a974af000-7f5a974b3000 rw-p 0000000 00:00 0
    7f5a974b3000-7f5a975bb000 r-xp 0000000 08:01 268926 /lib/x86_64-linux-gnu/libm-2.23.so
    7f5a975bb000-7f5a977ba000 ---p 00108000 08:01 268926 /lib/x86_64-linux-gnu/libm-2.23.so
    7f5a977ba000-7f5a977bb000 r--p 00107000 08:01 268926 /lib/x86_64-linux-gnu/libm-2.23.so
    7f5a977bb000-7f5a977bc000 rw-p 00108000 08:01 268926 /lib/x86_64-linux-gnu/libm-2.23.so
    7f5a977bc000-7f5a977be000 r-xp 0000000 08:01 268937 /lib/x86_64-linux-gnu/libutil-2.23.so
    7f5a977be000-7f5a979bd000 ---p 00002000 08:01 268937 /lib/x86_64-linux-gnu/libutil-2.23.so
    7f5a979bd000-7f5a979be000 r--p 00001000 08:01 268937 /lib/x86_64-linux-gnu/libutil-2.23.so
    7f5a979be000-7f5a979bf000 rw-p 00002000 08:01 268937 /lib/x86_64-linux-gnu/libutil-2.23.so
    7f5a979bf000-7f5a979c2000 r-xp 0000000 08:01 268932 /lib/x86_64-linux-gnu/libdl-2.23.so
    7f5a979c2000-7f5a97bc1000 ---p 00003000 08:01 268932 /lib/x86_64-linux-gnu/libdl-2.23.so
    7f5a97bc1000-7f5a97bc2000 r--p 00002000 08:01 268932 /lib/x86_64-linux-gnu/libdl-2.23.so
    7f5a97bc2000-7f5a97bc3000 rw-p 00003000 08:01 268932 /lib/x86_64-linux-gnu/libdl-2.23.so
    7f5a97bc3000-7f5a97bdb000 r-xp 0000000 08:01 268929 /lib/x86_64-linux-gnu/libpthread-2.23.so
    7f5a97bdb000-7f5a97dda000 ---p 00018000 08:01 268929 /lib/x86_64-linux-gnu/libpthread-2.23.so
    7f5a97dda000-7f5a97ddb000 r--p 00017000 08:01 268929 /lib/x86_64-linux-gnu/libpthread-2.23.so
    7f5a97ddb000-7f5a97ddc000 rw-p 00018000 08:01 268929 /lib/x86_64-linux-gnu/libpthread-2.23.so
    7f5a97ddc000-7f5a97de0000 rw-p 0000000 00:00 0
    7f5a97de0000-7f5a97e06000 r-xp 0000000 08:01 268928 /lib/x86_64-linux-gnu/ld-2.23.so
    7f5a97e10000-7f5a97fb5000 rw-p 0000000 00:00 0
    7f5a97fb5000-7f5a97fdc000 r--p 0000000 08:01 135047 /usr/lib/locale/C.UTF-8/LC_CTYPE
    7f5a97fdc000-7f5a97fe1000 rw-p 0000000 00:00 0
    7f5a97ffd000-7f5a97ffe000 rw-p 0000000 00:00 0
    7f5a97ffe000-7f5a98005000 r--s 0000000 08:01 529048 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
    7f5a98005000-7f5a98006000 r--p 00025000 08:01 268928 /lib/x86_64-linux-gnu/ld-2.23.so
    7f5a98006000-7f5a98007000 rw-p 00026000 08:01 268928 /lib/x86_64-linux-gnu/ld-2.23.so
    7f5a98007000-7f5a98008000 rw-p 0000000 00:00 0
    7fff79aeb000-7fff79b0c000 rw-p 0000000 00:00 0 [stack]
    7fff79b1d000-7fff79b20000 r--p 0000000 00:00 0 [vvar]
    7fff79b20000-7fff79b22000 r-xp 0000000 00:00 0 [vdso]
    ffffffffff600000-ffffffffff601000 r-xp 0000000 00:00 0 [vsyscall]

    The crash is reproducible on Ubuntu 16.04 with a pyenv-built 3.7.0b2 and on macOS 10.13 with the python.org build.

    Individually setting PYTHONMALLOC=debug also triggers the crash:

    $ PYTHONMALLOC=debug /usr/local/bin/python3.7 -c 'import os; os.fork()'
    Python(16996,0x7fffb1879340) malloc: *** error for object 0x7f90e6d01ff0: pointer being freed was not allocated
    *** set a breakpoint in malloc_error_break to debug

    @jamadden jamadden mannequin added 3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Mar 5, 2018
    @jamadden
    Copy link
    Mannequin Author

    jamadden mannequin commented Mar 5, 2018

    I built a local version of master (6821e73) and was able to get some line numbers (they're off by one for some reason, it appears):

    Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
    0 libsystem_kernel.dylib 0x00007fff78972e3e __pthread_kill + 10
    1 libsystem_pthread.dylib 0x00007fff78ab1150 pthread_kill + 333
    2 libsystem_c.dylib 0x00007fff788cf312 abort + 127
    3 libsystem_malloc.dylib 0x00007fff789cc866 free + 521
    4 python.exe 0x0000000100fba715 _PyRuntimeState_Fini + 37 (pystate.c:90)
    5 python.exe 0x0000000100fb9d73 Py_FinalizeEx + 547 (pylifecycle.c:1231)
    6 python.exe 0x0000000100fddd80 pymain_main + 5808 (main.c:2664)
    7 python.exe 0x0000000100fdec82 _Py_UnixMain + 178 (main.c:2697)
    8 libdyld.dylib 0x00007fff78823115 start + 1

    @vstinner
    Copy link
    Member

    vstinner commented Mar 6, 2018

    This bug is likely a regression on my work on Py_Main(): see bpo-32030. I proposed a fix: see attached PR 6001.

    @vstinner
    Copy link
    Member

    vstinner commented Mar 6, 2018

    New changeset 5d92647 by Victor Stinner in branch 'master':
    bpo-33005: Fix _PyGILState_Reinit() (bpo-6001)
    5d92647

    @miss-islington
    Copy link
    Contributor

    New changeset 31e2b76 by Miss Islington (bot) in branch '3.7':
    bpo-33005: Fix _PyGILState_Reinit() (GH-6001)
    31e2b76

    @vstinner
    Copy link
    Member

    vstinner commented Mar 6, 2018

    Thanks Jason Madden for your bug report! I should now be fixed.

    You might want to try the 3.7 or master branch until the next 3.7 release: PEP-537 ("3.7.0 beta 3: 2018-03-26").

    @vstinner vstinner added the 3.8 only security fixes label Mar 6, 2018
    @vstinner vstinner closed this as completed Mar 6, 2018
    @jamadden
    Copy link
    Mannequin Author

    jamadden mannequin commented Mar 6, 2018

    Thank you! I can confirm that git commit 31e2b76 on the 3.7 branch fixes the issue for me.

    @vstinner
    Copy link
    Member

    vstinner commented Mar 6, 2018

    Thank you! I can confirm that git commit 31e2b76 on the 3.7 branch fixes the issue for me.

    Cool. You can now continue your gevent tests using -X dev ;-)

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants