Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

io.IOBase destructor silence I/O error on close() by default #62948

Closed
ionelmc mannequin opened this issue Aug 15, 2013 · 39 comments
Closed

io.IOBase destructor silence I/O error on close() by default #62948

ionelmc mannequin opened this issue Aug 15, 2013 · 39 comments
Labels
3.8 only security fixes 3.9 only security fixes build The build process and cross-build extension-modules C modules in the Modules dir type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@ionelmc
Copy link
Mannequin

ionelmc mannequin commented Aug 15, 2013

BPO 18748
Nosy @pitrou, @vstinner, @tirkarthi
PRs
  • bpo-18748: io.IOBase destructor now logs close() errors in dev mode #12786
  • bpo-18748: test_io: silence destructor errors #12805
  • bpo-18748: _pyio.IOBase emits unraisable exception #13512
  • bpo-18748: Fix _pyio.IOBase destructor (closed case) #13952
  • Files
  • bug.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-06-12.23:14:43.171>
    created_at = <Date 2013-08-15.13:10:42.326>
    labels = ['extension-modules', 'build', '3.8', '3.9', 'type-crash']
    title = 'io.IOBase destructor silence I/O error on close() by default'
    updated_at = <Date 2021-06-28.23:33:12.533>
    user = 'https://bugs.python.org/ionelmc'

    bugs.python.org fields:

    activity = <Date 2021-06-28.23:33:12.533>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-06-12.23:14:43.171>
    closer = 'vstinner'
    components = ['Build', 'Extension Modules']
    creation = <Date 2013-08-15.13:10:42.326>
    creator = 'ionelmc'
    dependencies = []
    files = ['31301']
    hgrepos = []
    issue_num = 18748
    keywords = ['patch']
    message_count = 39.0
    messages = ['195253', '195319', '195321', '195324', '195325', '195342', '195505', '195507', '195523', '195525', '195527', '195528', '195565', '294551', '294873', '294874', '339502', '339979', '340055', '340056', '340058', '340059', '340062', '340104', '341106', '342483', '342484', '342493', '343270', '343699', '345157', '345160', '345422', '345428', '347240', '347268', '347284', '348515', '396673']
    nosy_count = 9.0
    nosy_names = ['pitrou', 'vstinner', 'ionelmc', 'neologix', 'nikicat', 'zwol', 'Barry Davis', 'xtreak', 'Yhojann Aguilera']
    pr_nums = ['12786', '12805', '13512', '13952']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue18748'
    versions = ['Python 3.6', 'Python 3.8', 'Python 3.9']

    Linked PRs

    @ionelmc
    Copy link
    Mannequin Author

    ionelmc mannequin commented Aug 15, 2013

    Running the file couple of times will make the interpreter fail with: libgcc_s.so.1 must be installed for pthread_cancel to work

    From what I've seen it is triggered from PyThread_delete_key (tries to load libgcc_s.so at that time).

    How does it happen?

    The main thread will close fd 4 (because fh object is getting dereferenced to 0) exactly at the same time libpthread will try to open and read libgcc_s.so with the same descriptor (4)

    It's fairly obvious that the file handling in bug.py is a bug, but the interpreter should not crash like that !

    This doesn't happen on python2.7. Also, python2.7 appears to be linked with libgcc_s.so.1 directly while the 3.x does not (I've tried 3.2 from ubuntu repos, and built 3.3 and 3.4 myself on ubuntu 12.04.2) - at least that's what ldd indicates.

    @ionelmc ionelmc mannequin added build The build process and cross-build extension-modules C modules in the Modules dir type-crash A hard crash of the interpreter, possibly with a core dump labels Aug 15, 2013
    @neologix
    Copy link
    Mannequin

    neologix mannequin commented Aug 16, 2013

    Unfortunately, there's not much we can do about it: if dlsym() fails - which is the case here either because read() fails with EBADF, or because the file descriptor now points to another stream (i.e. not libgcc), the libc aborts (here upon pthread_exit(), not PyThread_delete_key()):
    """
    libgcc_s.so.1 must be installed for pthread_cancel to work

    Program received signal SIGABRT, Aborted.
    [Switching to Thread 0xb7b0eb70 (LWP 17152)]
    0xb7fe1424 in __kernel_vsyscall ()
    (gdb) bt
    #0 0xb7fe1424 in __kernel_vsyscall ()
    #1 0xb7e4e941 in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
    #2 0xb7e51d72 in *__GI_abort () at abort.c:92
    #3 0xb7e8ae15 in __libc_message (do_abort=1, fmt=0xb7f606f5 "%s") at ../sysdeps/unix/sysv/linux/libc_fatal.c:189
    #4 0xb7e8af44 in *__GI___libc_fatal (message=0xb7fc75ec "libgcc_s.so.1 must be installed for pthread_cancel to work\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:200
    #5 0xb7fc4ffa in pthread_cancel_init () at ../nptl/sysdeps/pthread/unwind-forcedunwind.c:65
    #6 0xb7fc509d in _Unwind_ForcedUnwind (exc=0xb7b0edc0, stop=0xb7fc2bf0 <unwind_stop>, stop_argument=0xb7b0e454) at ../nptl/sysdeps/pthread/unwind-forcedunwind.c:126
    #7 0xb7fc2b98 in *__GI___pthread_unwind (buf=<optimized out>) at unwind.c:130
    #8 0xb7fbcce0 in __do_cancel () at pthreadP.h:265
    #9 __pthread_exit (value=0x0) at pthread_exit.c:30
    #10 0x08132ced in PyThread_exit_thread () at Python/thread_pthread.h:266
    #11 0x08137c37 in t_bootstrap (boot_raw=0x8318aa8) at ./Modules/_threadmodule.c:1023
    #12 0xb7fbbc39 in start_thread (arg=0xb7b0eb70) at pthread_create.c:304
    #13 0xb7ef978e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
    """

    So if you're unlucky and end up closing the FD referring to libgcc used by dlopen/dlsym, you're pretty much screwed, and there's no way around this.

    Note that you specific problem (upon PyThread_delete_key()) doesn't occur for python 2.7 because it uses and ad-hoc TLS implementation, whereas Python 3 uses the platform native TLS.
    But as noted above, you can very well get an abort on pthread_exit(), and in likely many other places (pretty much everywhere libgcc can be dlopen'ed).

    Unfortunately, we can't do much against this, so I'm tempted to close this as "wont fix".

    @pitrou
    Copy link
    Member

    pitrou commented Aug 16, 2013

    Perhaps the only thing we could do would be try to "preload" libgcc by calling one of those APIs at startup? But I'm not sure it's a good idea to add such a platform-specific hack (for what is arguably an obscure and rare issue).

    @vstinner
    Copy link
    Member

    Maries Ionel Cristian> ubuntu 12.04.2

    What is the version of your libc library? Try something like "dpkg -l libc6".

    --

    Could this issue be related to this glibc issue?
    http://sourceware.org/bugzilla/show_bug.cgi?id=2644

    I ran your script on Python 3.4 in a shell loop: I'm unable to reproduce the issue. I'm running Fedora 18 with the glibc 2.16.

    @vstinner
    Copy link
    Member

    Oh ok, I'm able to reproduce the issue with the system Python 3.3:

    $ while true; do echo "loop"; python3.3 bug.py || break; done
    loop
    ...
    loop
    libgcc_s.so.1 must be installed for pthread_cancel to work
    Abandon (core dumped)

    @neologix
    Copy link
    Mannequin

    neologix mannequin commented Aug 16, 2013

    Perhaps the only thing we could do would be try to "preload" libgcc by
    calling one of those APIs at startup?

    Yeah, I was thinking about doing this in PyThread_init_thread() but...

    But I'm not sure it's a good idea to
    add such a platform-specific hack (for what is arguably an obscure and rare
    issue).

    then thought the exact same thing.

    Also, this exact problem can possibly show up anywhere dlopen() is
    used (a grep through glibc code shows at least an occurrence in
    nsswitch-related code).

    @ionelmc
    Copy link
    Mannequin Author

    ionelmc mannequin commented Aug 17, 2013

    What is the version of your libc library? Try something like "dpkg -l
    libc6".

    2.15-0ubuntu10.4

    I don't think it's that obscure ... uwsgi has this issue
    https://www.google.com/search?q=libgcc_s.so.1+must+be+installed+for+pthread_cancel+to+work+uwsgi+site:lists.unbit.it-
    they cause it probably different but the point is that it's not
    obscure.

    @ionelmc
    Copy link
    Mannequin Author

    ionelmc mannequin commented Aug 17, 2013

    @vstinner
    Copy link
    Member

    2013/8/17 Maries Ionel Cristian <report@bugs.python.org>:

    I don't think it's that obscure ... uwsgi has this issue
    https://www.google.com/search?q=libgcc_s.so.1+must+be+installed+for+pthread_cancel+to+work+uwsgi+site:lists.unbit.it-
    they cause it probably different but the point is that it's not
    obscure.

    The error message is maybe the same, the reason is completly different:
    http://lists.unbit.it/pipermail/uwsgi/2013-July/006213.html
    "IT WAS A limit-as issue"

    The root cause is an arbitrary limit on the size of the virtual memory.

    Here the bug is that a thread B closes a file descriptor opened in a
    thread B by the glibc.

    @ionelmc
    Copy link
    Mannequin Author

    ionelmc mannequin commented Aug 17, 2013

    Well anyway, is there any way to preload libgcc ? Because in python2.x it wasn't loaded at runtime.

    @vstinner
    Copy link
    Member

    2013/8/18 Maries Ionel Cristian <report@bugs.python.org>:

    Well anyway, is there any way to preload libgcc ? Because in python2.x it wasn't loaded at runtime.

    On Linux, you can try to set the LD_PRELOAD environment variable as a
    workaround.

    LD_PRELOAD=libgcc_s.so.1 python bug.py

    You may need to specify the full path.

    @ionelmc
    Copy link
    Mannequin Author

    ionelmc mannequin commented Aug 17, 2013

    Alright ... would it be a very big hack to preload libgcc in the thread module (at import time) ? There is platform specific code there anyway, it wouldn't be such a big deal would it?

    @neologix
    Copy link
    Mannequin

    neologix mannequin commented Aug 18, 2013

    On Linux, you can try to set the LD_PRELOAD environment variable as a
    workaround.

    LD_PRELOAD=libgcc_s.so.1 python bug.py

    You may need to specify the full path.

    I don't think that'll work.
    Despite its name, using LD_PRELOAD won't "preload" the library. It
    will only be loaded upon dlopen(). It just makes sure that symbols
    will be looked for in this library first, even before the libc.

    Because in python2.x it wasn't loaded at runtime.

    Yes it was. As explained above, you can get the very same crash upon
    pthread_exit().

    Alright ... would it be a very big hack to preload libgcc in the thread module (at import
    time) ?

    IMO yes, but if someone writes a patch, I won't oppose to it :-)

    @BarryDavis
    Copy link
    Mannequin

    BarryDavis mannequin commented May 26, 2017

    I think this is the same issue I'm getting.
    I'm hitting it when compiling python 3.6.2 compiled with gcc-4.8.4.
    This wasn't occasional, it was every time I tried.
    As a feeble workaround I was compiling in parallel, then in serial when it fails, although I'm now hitting the same issue when building uwsgi-2.0.15.

    I'm building on a 56 core system so that might make me more likely to hit the issue.

    @BarryDavis
    Copy link
    Mannequin

    BarryDavis mannequin commented May 31, 2017

    Looks like I was mistaken. My cross compiler was trying to load libgcc_s.so.1 from the standard location and not liking the one it found. Fixed for now by setting LD_LIBRARY_PATH to point at the dir containing the right libgcc_s.so.1

    @BarryDavis
    Copy link
    Mannequin

    BarryDavis mannequin commented May 31, 2017

    I meant my cross compiled python, not my cross compiler.

    @zwol
    Copy link
    Mannequin

    zwol mannequin commented Apr 5, 2019

    I have observed this problem in a production application using Python 3.5 and 3.6 (system-packaged interpreters from Ubuntu) and I would like to mention that this is an effective workaround:

    import ctypes
    libgcc_s = ctypes.CDLL("libgcc_s.so.1")
    

    provided you do it before creating any threads, and the variable libgcc_s remains live until after all threads are terminated.

    @vstinner
    Copy link
    Member

    I looked at this old issue. First of all, "bug.py" filename is accurate: this program contains a bug, but it's non-obvious.

    • the main thread opens a file which gets the file descriptor 3
    • a new thread is spawns which indirectly closes this file descriptor on: sys.stdout.close()
    • the main thread is not aware that the file has been closed, and so will try to close the same file descriptor 3 sometime during Python shutdown
    • then, sometimes, the glibc fails with an internal error because the main thread closed a file descriptor which was just opened by the other thread to implement thread cancellation: that logs "libgcc_s.so.1 must be installed for pthread_cancel to work" and calls abort()

    Note: the Python file object remains alive until the "t" local variable is cleared, since "t.fh" contains a strong reference to the file object.

    Again: it's a bug in the program. "libgcc_s.so.1 must be installed for pthread_cancel to work" is just *one* example of bad thing that can happen.

    What can be do on the Python side? Well, stop to ignore silently I/O errors when a file is closed by its destructor.

    I wrote a shy PR 12786 to only log these exceptions in development mode (-X dev). I'm not sure if we could stop to silence these exceptions *by default*.

    Python has been modified last years to first raise an exception on I/O errors when file.close() is called explicitly, and then a similar change has been done for call to socket.socket.close().

    @vstinner vstinner changed the title libgcc_s.so.1 must be installed for pthread_cancel to work io.IOBase destructor silence I/O error on close() by default Apr 11, 2019
    @vstinner
    Copy link
    Member

    New changeset 4423504 by Victor Stinner in branch 'master':
    bpo-18748: io.IOBase destructor now logs close() errors in dev mode (GH-12786)
    4423504

    @vstinner
    Copy link
    Member

    To debug remaining "Exception ignored in:" issues, I'm using the following patch:

    diff --git a/Lib/unittest/case.py b/Lib/unittest/case.py
    index 972a4658b1..be38af3daa 100644
    --- a/Lib/unittest/case.py
    +++ b/Lib/unittest/case.py
    @@ -710,6 +710,7 @@ class TestCase(object):
     
                 # clear the outcome, no more needed
                 self._outcome = None
    +            import gc; gc.collect()
     
         def doCleanups(self):
             """Execute all cleanup functions. Normally called for you after

    And I run:

    ./python -X dev -u -m test test_io -v 2>&1|tee log

    @vstinner
    Copy link
    Member

    I wrote PR 12805 to silence IOBase destructor exceptions in test_io.

    @vstinner
    Copy link
    Member

    Is there someone interested to debug remaining "Exception ignored:" logs in test_urllib?

    test_invalid_redirect (test.test_urllib.urlopen_HttpTests) ...
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24be0>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    ok
    
    test_read_bogus (test.test_urllib.urlopen_HttpTests) ...
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24cd0>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    ok
    
    test_redirect_limit_independent (test.test_urllib.urlopen_HttpTests) ...
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24cd0>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24c30>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24460>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24a50>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f247d0>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52ea95f0>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52ea99b0>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f2f820>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f2fbe0>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f2fd70>
    Traceback (most recent call last):
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
        super().close() # set "closed" flag
      File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
        self.fp.flush()
    ValueError: I/O operation on closed file.
    ok

    It's unclear to be if it's a bug in http.client, a bug in the test... or something else.

    @vstinner
    Copy link
    Member

    Maybe the fact that close() exceptions are ignored silently in io.IOBase constructor should be better documented, but I'm not sure where it should be documented. If someone has an idea, please go ahead and write a pull request :-)

    @vstinner
    Copy link
    Member

    New changeset 472f794 by Victor Stinner in branch 'master':
    bpo-18748: test_io: silence destructor errors (GH-12805)
    472f794

    @tirkarthi
    Copy link
    Member

    The ValueError warnings in test_urllib noted in msg340059 feels like an issue with the test where FakeSocket calls close by itself once it's internal io_refs counter is 0 and during destructor again close is called on an already closed object causing the ValueError.

    • The test uses fakehttp() creating FakeSocket starting with io_refs as 1 [1]
    • During the test in urlopen, sock.makefile() (io_refs += 1) is called increasing the io_refs to be 2.
    • HTTPConnection.close calls sock.close (fakesocket.close) [3] calling io_refs -= 1 and to fakesocket.io_ref to be 1.
    • This test uses raises status 302 with http_error_302 which calls self.redirect_internal where self.close is again called causing fp.io_refs == 0 and subsequently calling io.BytesIO.close(self) to close the object. [4]
    • During the end of the test destructor is called on the above closed fakesocket object and trying to flush on a closed object causes the ValueError warnings .

    Maybe a check could be added during flush to make sure the object is not closed by testing for self.closed but I am not sure if closed attribute is guaranteed to be present. Removing the manual call to close in fakesocket could help since the destructor should be taking care of it and I could see no test failures or warnings removing the close as io_refs gets to 0.

    [1]

    io_refs = 1

    [2]
    self.io_refs += 1

    [3]
    sock.close() # close it manually... there may be other refs

    [4]
    fp.close()

    @tirkarthi
    Copy link
    Member

    I was wrong so removing the close call when io_refs gets to zero causes error on test_urllib2 [0] where close is asserted to be True and I only tested test_urllib initially. One another way would be to check for fp.closed in http.client before flushing but as mentioned before I am not sure of the attribute being always present and also to modify http.client code just for tests.

    [0]

    self.assertTrue(conn.fakesock.closed,

    @vstinner
    Copy link
    Member

    Karthikeyan Singaravelan: Would you mind to open a separated issue for test_urllib warnings?

    @tirkarthi
    Copy link
    Member

    Okay, opened bpo-36918 . Raising PR against that issue.

    @vstinner
    Copy link
    Member

    New changeset bc2aa81 by Victor Stinner in branch 'master':
    bpo-18748: _pyio.IOBase emits unraisable exception (GH-13512)
    bc2aa81

    @vstinner
    Copy link
    Member

    Related issue: bpo-37054 "Ignored exceptions in test_memoryio".

    @vstinner
    Copy link
    Member

    I created bpo-37223: test_io logs Exception ignored in: <function IOBase.__del__ at 0x7f2f3c73e4b0> warnings.

    @vstinner
    Copy link
    Member

    New changeset 4f6f7c5 by Victor Stinner in branch 'master':
    bpo-18748: Fix _pyio.IOBase destructor (closed case) (GH-13952)
    4f6f7c5

    @vstinner
    Copy link
    Member

    New changeset c15a682 by Victor Stinner in branch '3.8':
    bpo-37223: test_io: silence destructor errors (GH-14031)
    c15a682

    @vstinner vstinner added 3.8 only security fixes 3.9 only security fixes labels Jun 12, 2019
    @vstinner
    Copy link
    Member

    I closed the issue. I consider that the initial issue has been fixed and all related issues have been fixed as well:

    • Python 3.8 now logs close() exception in file finalizer in development mode (-X dev) and debug mode (./configure --with-pydebug)
    • I added sys.unraisablehook to Python 3.8 which allows to decide how these unraisable exceptions are handled
    • Tests no longer log any "unraisable exception" in file finalizers (test_io and test_urllib have been fixed)
    • regrtest now marks a test as ENV_CHANGED if it logs an unraisable exception (like the close() exception)

    --

    For Python 3.7, there is no known workaround.

    It seems like Python 2.7 in debug mode logs an error:

    $ python2.7-dbg bug.py 
    close failed in file object destructor:
    IOError: [Errno 9] Bad file descriptor

    @YhojannAguilera
    Copy link
    Mannequin

    YhojannAguilera mannequin commented Jul 3, 2019

    Same problem using Python 3.6.8 on Ubuntu 18.04 LTS.

    For now, solve this using

    LD_PRELOAD=libgcc_s.so.1 python3 ...

    For more details and pocs: yhojann-cl/wss#2

    @vstinner
    Copy link
    Member

    vstinner commented Jul 4, 2019

    Same problem using Python 3.6.8 on Ubuntu 18.04 LTS.

    To be clear: this issue is NOT a bug in Python, but a bug in your application. You must fix your application. You can try to run it on Python 3.8 with python3.8 -X dev to get a log on the error. Good luck to debug it ;-)

    The change that I made is only to add a log to help developers to find their own bugs ;-)

    @zwol
    Copy link
    Mannequin

    zwol mannequin commented Jul 4, 2019

    To be clear: this issue is NOT a bug in Python

    I don't think that's entirely true. I think CPython needs to be linked against libgcc_s.so, so that this class of application bugs will no longer manifest as interpreter crashes. I filed bpo-37395 for that.

    @YhojannAguilera
    Copy link
    Mannequin

    YhojannAguilera mannequin commented Jul 26, 2019

    I hope that when an error occurs, python tells me what the problem is. The abort core error is a problem at a lower level than python because python is not able to recognize or handle the error.

    The main problem is that I exceeded the maximum number of process threads supported by the kernel.

    What I hope is that python throws an exception when it exceeds this limit or when it cannot access the pointer in memory of the process thread.

    The problem is not if the script is right or wrong, but that Python is not able to recognize and handle the problem. A generic message saying that an error occurred without indicating where and how it occurred is a python bug.

    @vstinner
    Copy link
    Member

    Running the file couple of times will make the interpreter fail with: libgcc_s.so.1 must be installed for pthread_cancel to work

    See also bpo-44434: "_thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work".

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes build The build process and cross-build extension-modules C modules in the Modules dir type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants