Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abort in PyErr_PrintEx() when no memory #75000

Closed
xdegaye mannequin opened this issue Jun 30, 2017 · 9 comments
Closed

Abort in PyErr_PrintEx() when no memory #75000

xdegaye mannequin opened this issue Jun 30, 2017 · 9 comments
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@xdegaye
Copy link
Mannequin

xdegaye mannequin commented Jun 30, 2017

BPO 30817
Nosy @vstinner, @xdegaye, @serhiy-storchaka
PRs
  • bpo-30817: Fix PyErr_PrintEx() when no memory #2526
  • [3.6] bpo-30817: Fix PyErr_PrintEx() when no memory (GH-2526). #4107
  • Files
  • memerr.py
  • pyerr_printex.patch
  • set-last-exception.diff
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2017-10-24.14:44:13.263>
    created_at = <Date 2017-06-30.14:19:31.165>
    labels = ['interpreter-core', '3.7', 'type-crash']
    title = 'Abort in PyErr_PrintEx() when no memory'
    updated_at = <Date 2017-10-24.14:44:13.262>
    user = 'https://github.com/xdegaye'

    bugs.python.org fields:

    activity = <Date 2017-10-24.14:44:13.262>
    actor = 'xdegaye'
    assignee = 'none'
    closed = True
    closed_date = <Date 2017-10-24.14:44:13.263>
    closer = 'xdegaye'
    components = ['Interpreter Core']
    creation = <Date 2017-06-30.14:19:31.165>
    creator = 'xdegaye'
    dependencies = []
    files = ['46982', '46983', '46986']
    hgrepos = []
    issue_num = 30817
    keywords = ['patch']
    message_count = 9.0
    messages = ['297408', '297411', '297414', '297483', '297495', '297508', '304809', '304818', '304913']
    nosy_count = 3.0
    nosy_names = ['vstinner', 'xdegaye', 'serhiy.storchaka']
    pr_nums = ['2526', '4107']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue30817'
    versions = ['Python 3.6', 'Python 3.7']

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jun 30, 2017

    To reproduce the abort (the set_nomemory() function is being added to _testcapi in bpo-30695):

    $ ./python memerr.py
    python: Objects/call.c:89: _PyObject_FastCallDict: Assertion `!PyErr_Occurred()' failed.
    Aborted (core dumped)

    The corresponding backtrace:

    #0 0x00007ffff7131670 in raise () from /usr/lib/libc.so.6
    #1 0x00007ffff7132d00 in abort () from /usr/lib/libc.so.6
    #2 0x00007ffff712a45a in __assert_fail_base () from /usr/lib/libc.so.6
    #3 0x00007ffff712a4d2 in __assert_fail () from /usr/lib/libc.so.6
    #4 0x00000000004628ce in _PyObject_FastCallDict (
    callable=<built-in method excepthook of module object at remote 0x7ffff70ab8d8>,
    args=args@entry=0x7fffffffe330, nargs=nargs@entry=3, kwargs=kwargs@entry=0x0)
    at Objects/call.c:89
    #5 0x0000000000426004 in PyErr_PrintEx (set_sys_last_vars=set_sys_last_vars@entry=1)
    at Python/pythonrun.c:641
    #6 0x00000000004263ac in PyErr_Print () at Python/pythonrun.c:510
    #7 0x0000000000427586 in PyRun_SimpleFileExFlags (fp=fp@entry=0x987620, filename=<optimized out>,
    filename@entry=0x7ffff6febcd0 "memerr.py", closeit=closeit@entry=1,
    flags=flags@entry=0x7fffffffe4cc) at Python/pythonrun.c:403
    #8 0x0000000000427673 in PyRun_AnyFileExFlags (fp=fp@entry=0x987620,
    filename=0x7ffff6febcd0 "memerr.py", closeit=closeit@entry=1, flags=flags@entry=0x7fffffffe4cc)
    at Python/pythonrun.c:82
    #9 0x0000000000437217 in run_file (fp=fp@entry=0x987620, filename=0x91a190 L"memerr.py",
    p_cf=p_cf@entry=0x7fffffffe4cc) at Modules/main.c:341
    #10 0x0000000000437bae in Py_Main (argc=argc@entry=2, argv=argv@entry=0x918020)
    at Modules/main.c:901
    #11 0x000000000041d7fa in main (argc=2, argv=0x7fffffffe6c8) at ./Programs/python.c:102

    @xdegaye xdegaye mannequin added 3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump labels Jun 30, 2017
    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jun 30, 2017

    With pyerr_printex.patch we get the following correct results (rc=120 is set by Py_Main() after Py_FinalizeEx() returns with an error):

    $ ./python memerr.py 5 12
    set_nomemory(0, 5)
    result = _PythonRunResult(rc=1, out=b'', err=b'MemoryError\n\nDuring handling of the above exception, another exception occurred:\n\nMemoryError')

    =============================

    set_nomemory(0, 6)
    result = _PythonRunResult(rc=1, out=b'', err=b'sys.excepthook is missing\nMemoryError\n\nDuring handling of the above exception, another exception occurred:\n\nMemoryError')
    =============================
    set_nomemory(0, 7)
    result = _PythonRunResult(rc=1, out=b'', err=b'sys.excepthook is missing\nMemoryError\n\nDuring handling of the above exception, another exception occurred:\n\nMemoryError')
    =============================
    set_nomemory(0, 8)
    result = _PythonRunResult(rc=1, out=b'', err=b'sys.excepthook is missing\nobject  : MemoryError()\ntype    : MemoryError\nrefcount: 1\naddress : 0x7f7ae1be3eb0\nlost sys.stderr')
    =============================
    set_nomemory(0, 9)
    result = _PythonRunResult(rc=1, out=b'', err=b'sys.excepthook is missing\nobject  : \ntype    : MemoryError\nrefcount: 1\naddress : 0x7f8aead15eb0\nlost sys.stderr')
    =============================
    set_nomemory(0, 10)
    result = _PythonRunResult(rc=1, out=b'', err=b'sys.excepthook is missing\nobject  : \ntype    : MemoryError\nrefcount: 1\naddress : 0x7f7b489d1eb0\nlost sys.stderr')
    =============================
    set_nomemory(0, 11)
    result = _PythonRunResult(rc=1, out=b'', err=b'sys.excepthook is missing\nobject  : \ntype    : MemoryError\nrefcount: 1\naddress : 0x7ff97365eeb0\nlost sys.stderr')
    =============================
    set_nomemory(0, 12)
    result = _PythonRunResult(rc=120, out=b'', err=b"sys.excepthook is missing\nobject  : \ntype    : MemoryError\nrefcount: 1\naddress : 0x7f559c9e0eb0\nlost sys.stderr\nException ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>\nMemoryError")

    =============================

    @vstinner
    Copy link
    Member

    pyerr_printex.patch LGTM, please convert it to a GitHub PR.

    @serhiy-storchaka
    Copy link
    Member

    Is it good to ignore errors of setting sys attributes? Can we prevent these errors?

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jul 1, 2017

    There are just 6 cases left where the return code of _PySys_SetObjectId() is ignored:

    Python/pylifecycle.c|689 col 5| _PySys_SetObjectId(&PyId_stderr, pstderr);
    Python/pylifecycle.c|1211 col 9| _PySys_SetObjectId(&PyId_stderr, pstderr);
    Python/pylifecycle.c|1657 col 5| _PySys_SetObjectId(&PyId_stdin, std);
    Python/pylifecycle.c|1666 col 5| _PySys_SetObjectId(&PyId_stdout, std);
    ./Python/pythonrun.c|104 col 9| _PySys_SetObjectId(&PyId_ps1, v = PyUnicode_FromString(">>> "));
    ./Python/pythonrun.c|109 col 9| _PySys_SetObjectId(&PyId_ps2, v = PyUnicode_FromString("... "));

    Error should be handled there in the same way errors are handled within the functions where they are invoked (returning -1, NULL or aborting). This should be the purpose of another issue as it is acceptable I think that there is no test case for those changes.

    I am leaving the day after tomorrow for few weeks, without internet access (sailing to the mid-atlantic), and will create then a new issue if no one else does that first.

    @serhiy-storchaka
    Copy link
    Member

    I tried to set last_type, last_value and last_traceback to None at startup, and this also fixes a crash. But this changes behavior (these attributes became set), and deleting them doesn't guaranties that following setting will be successful, especially with new compact dict implementation.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Oct 23, 2017

    Removing 2.7 as the problem cannot be reproduced here. PyEval_CallObjectWithKeywords() does not assert on PyErr_Occurred().

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Oct 23, 2017

    New changeset 66caacf by xdegaye in branch 'master':
    bpo-30817: Fix PyErr_PrintEx() when no memory (bpo-2526)
    66caacf

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Oct 24, 2017

    New changeset d5d7954 by xdegaye in branch '3.6':
    [3.6] bpo-30817: Fix PyErr_PrintEx() when no memory (GH-2526). (bpo-4107)
    d5d7954

    @xdegaye xdegaye mannequin closed this as completed Oct 24, 2017
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants