Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Misc fixes and cleanups in error handling C code #73180

Closed
serhiy-storchaka opened this issue Dec 16, 2016 · 15 comments
Closed

Misc fixes and cleanups in error handling C code #73180

serhiy-storchaka opened this issue Dec 16, 2016 · 15 comments
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 28994
Nosy @terryjreedy, @pfmoore, @larryhastings, @tjguk, @ned-deily, @zware, @serhiy-storchaka, @zooba, @ZackerySpytz
PRs
  • bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). #2034
  • bpo-28994: PyErr_NormalizeException() no longer recursive. #2035
  • gh-73180: Chain exceptions rather than dropping the old exception. #2072
  • [3.6] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) #2121
  • [3.5] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) #2122
  • [2.7] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) #2123
  • [2.7] bpo-28994: Remove mistakenly backported atexitmodule.c #9214
    		•
    Files
  • errors.diff
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2016-12-16.23:38:16.853>
labels = ['interpreter-core', 'type-bug', '3.7']
title = 'Misc fixes and cleanups in error handling C code'
updated_at = <Date 2018-09-12.11:28:24.267>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2018-09-12.11:28:24.267>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2016-12-16.23:38:16.853>
creator = 'serhiy.storchaka'
dependencies = []
files = ['45932']
hgrepos = []
issue_num = 28994
keywords = ['patch']
message_count = 14.0
messages = ['283451', '295618', '295743', '295744', '295745', '295747', '295817', '295820', '295821', '295822', '295823', '305588', '325129', '325133']
nosy_count = 9.0
nosy_names = ['terry.reedy', 'paul.moore', 'larry', 'tim.golden', 'ned.deily', 'zach.ware', 'serhiy.storchaka', 'steve.dower', 'ZackerySpytz']
pr_nums = ['2034', '2035', '2072', '2121', '2122', '2123', '9214']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28994'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']
    @serhiy-storchaka
    Copy link
    Member Author

    Proposed patch makes few minor changes related to handling errors in C.

    • Fixed possible NULL dereference in PyErr_NormalizeException().

    • Fixed error checking in atexit._run_exitfuncs().

    • Fixed possible memory leaks in _Py_FatalError_PrintExc().

    • PyErr_NormalizeException() no longer recursive.

    • If an error is occurred during handling other error, the original exception is chained as the __contex__ attribute to the new exception instead of just be dropped.

    • PyTraceBack_Print() no longer fails OverflowError if tracebacklimit is very large negative or positive value.

    • ctype error message now include the name of the initial exception instead of the repr of its class.

    • Py_XDECREFs is replaced with Py_DECREFs if this is safe.

    • Added few asserts.

    • Other minor cleanups.

    @serhiy-storchaka serhiy-storchaka added 3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Dec 16, 2016
    @serhiy-storchaka
    Copy link
    Member Author

    I'm trying to split the patch on smaller parts.

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset 3fd54d4 by Serhiy Storchaka in branch 'master':
    bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (bpo-2034)
    3fd54d4

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset d89dc84 by Serhiy Storchaka in branch '3.6':
    [3.6] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (bpo-2121)
    d89dc84

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset 7d8c1eb by Serhiy Storchaka in branch '3.5':
    [3.5] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (bpo-2122)
    7d8c1eb

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset 0cc43df by Serhiy Storchaka in branch '2.7':
    [2.7] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (bpo-2123)
    0cc43df

    @terryjreedy
    Copy link
    Member

    3.6 no longer compiles for me on Windows. A repeat of pcbuild\build.bat -d gives the same error report as below.

    My previous build was 25 hours ago. atexitmodule.c is the only C file in the 3.6 merge update, so the backport might be to blame. Git status say that Objects/listobject is changed, even though not changed by PR 2121, and that there is an untracked Objects/clinic/listobject.c.h. (but there is one in 3.7, so...?).

    chkdsk F: /F found one unrelated orphaned file but not other specific errors. It ended with this:

    Stage 3: Examining security descriptors ...
    Security descriptor verification completed.
    40783 data files processed.
    CHKDSK is verifying Usn Journal...
    3224 USN bytes processed.
    Usn Journal verification completed.
    An unspecified error occurred (6e74667363686b2e 1475).

    (which it also did a week ago) I don't remember what was done after the USN check. Web search has not helped yet for the 63... error. In any case, I thing someone else should verify building on Windows before 3.6.2 is tagged, if it has not yet been.

    Building heads/3.6-dirty:8399a177de 3.6
    atexitmodule.c
    listobject.c
    f:\dev\36\objects\clinic/listobject.c.h(24): warning C4133: 'function': incompatible type
    s - from 'char [10]' to 'PyObject *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
    f:\dev\36\objects\clinic/listobject.c.h(25): warning C4133: 'function': incompatible type
    s - from 'Py_ssize_t *' to '_PyArg_Parser *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
    f:\dev\36\objects\clinic/listobject.c.h(29): warning C4013: '_PyArg_NoStackKeywords' unde
    fined; assuming extern returning int [f:\dev\36\PCbuild\pythoncore.vcxproj]
    f:\dev\36\objects\clinic/listobject.c.h(112): warning C4133: 'function': incompatible typ
    es - from 'char [7]' to 'PyObject *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
    f:\dev\36\objects\clinic/listobject.c.h(113): warning C4133: 'function': incompatible typ
    es - from 'Py_ssize_t *' to '_PyArg_Parser *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
    f:\dev\36\objects\clinic/listobject.c.h(147): warning C4013: '_PyArg_ParseStackAndKeyword
    s' undefined; assuming extern returning int [f:\dev\36\PCbuild\pythoncore.vcxproj]
    f:\dev\36\objects\clinic/listobject.c.h(198): warning C4133: 'function': incompatible typ
    es - from 'char [13]' to 'PyObject *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
    f:\dev\36\objects\clinic/listobject.c.h(199): warning C4047: 'function': '_PyArg_Parser *
    ' differs in levels of indirection from 'PyObject **' [f:\dev\36\PCbuild\pythoncore.vcxpr
    oj]
    f:\dev\36\objects\clinic/listobject.c.h(199): warning C4024: '_PyArg_ParseStack': differe
    nt types for formal and actual parameter 4 [f:\dev\36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(1979): error C2059: syntax error: '<<' [f:\dev\36\PCbuild\pythonc
    ore.vcxproj]
    ..\Objects\listobject.c(1984): error C2014: preprocessor command must start as first nonw
    hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2015): error C2014: preprocessor command must start as first nonw
    hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2171): error C2143: syntax error: missing ';' before '<<' [f:\dev
    \36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2173): error C2143: syntax error: missing ';' before '==' [f:\dev
    \36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2175): error C2143: syntax error: missing ';' before '>>' [f:\dev
    \36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2175): error C2014: preprocessor command must start as first nonw
    hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2256): error C2143: syntax error: missing ';' before '<<' [f:\dev
    \36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2261): error C2143: syntax error: missing ';' before '==' [f:\dev
    \36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2262): error C2014: preprocessor command must start as first nonw
    hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2268): error C2059: syntax error: 'if' [f:\dev\36\PCbuild\pythonc
    ore.vcxproj]
    ..\Objects\listobject.c(2273): error C2059: syntax error: 'for' [f:\dev\36\PCbuild\python
    core.vcxproj]
    ..\Objects\listobject.c(2273): error C2143: syntax error: missing '{' before '<' [f:\dev\
    36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2273): error C2059: syntax error: '<' [f:\dev\36\PCbuild\pythonco
    re.vcxproj]
    ..\Objects\listobject.c(2273): error C2143: syntax error: missing '{' before '++' [f:\dev
    \36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(2273): error C2059: syntax error: '++' [f:\dev\36\PCbuild\pythonc
    ore.vcxproj]
    ..\Objects\listobject.c(2273): error C2059: syntax error: ')' [f:\dev\36\PCbuild\pythonco
    re.vcxproj]
    ..\Objects\listobject.c(2280): error C2059: syntax error: 'string' [f:\dev\36\PCbuild\pyt
    honcore.vcxproj]
    ..\Objects\listobject.c(2280): error C2040: 'PyErr_Format': 'int ()' differs in levels of
    indirection from 'PyObject *(PyObject *,const char *,...)' [f:\dev\36\PCbuild\pythoncore
    .vcxproj]
    ..\Objects\listobject.c(2281): error C2059: syntax error: 'return' [f:\dev\36\PCbuild\pyt
    honcore.vcxproj]
    ..\Objects\listobject.c(2282): error C2059: syntax error: '}' [f:\dev\36\PCbuild\pythonco
    re.vcxproj]
    ..\Objects\listobject.c(2422): error C2059: syntax error: '<<' [f:\dev\36\PCbuild\pythonc
    ore.vcxproj]
    ..\Objects\listobject.c(2443): error C2014: preprocessor command must start as first nonw
    hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
    ..\Objects\listobject.c(3113): fatal error C1004: unexpected end-of-file found [f:\dev\36
    \PCbuild\pythoncore.vcxproj]
    Generating Code...

    f:\dev\36>git log Modules/atexitmodule.c
    commit d89dc84
    Author: Serhiy Storchaka <storchaka@gmail.com>
    Date: Mon Jun 12 09:02:13 2017 +0300

    [3.6] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (bpo-2121)

The traceback no longer displayed for SystemExit raised in a callback registered by atexit..
(cherry picked from commit 3fd54d4a7e604067e2bc0f8cfd58bdbdc09fa7f4)

    When I reverted the listobject.c change and rebuilt, compilation finished and the test suite ran to SUCCESS.

    @serhiy-storchaka
    Copy link
    Member Author

    Last time Objects/listobject.c was changed 2 months ago. This looks as an error in your workspace. Maybe something wrong happened when you switched from master to 3.6.

    @zware
    Copy link
    Member

    zware commented Jun 12, 2017

    Agreed, looks like something went wrong in your checkout, Terry. Just confirmed that a 3.6 build on a fresh checkout on Windows is fine.

    @zooba
    Copy link
    Member

    zooba commented Jun 12, 2017

    I just updated to the latest 3.6 commit and listobject.c doesn't even use clinic in this branch.

    Serhiy is probably right - this is a workspace issue.

    @terryjreedy
    Copy link
    Member

    Since I fetch, merge, and build with a .bat file that has worked fine at least 10 times, with pushes in between, the glitch is a puzzle. Next time I will just revert, as I did, or reclone and rebuild the 3.6 worktree.

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset cf29653 by Serhiy Storchaka in branch 'master':
    bpo-28994: PyErr_NormalizeException() no longer use C stack for recursion. (bpo-2035)
    cf29653

    @ZackerySpytz
    Copy link
    Mannequin

    ZackerySpytz mannequin commented Sep 12, 2018

    In PR 2123, it was reported that the Modules/atexitmodule.c file was backported to 2.7. PR 9214 addresses this.

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset b36567b by Serhiy Storchaka (Zackery Spytz) in branch '2.7':
    [2.7] bpo-28994: Remove mistakenly backported atexitmodule.c (GH-9214)
    b36567b

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @arhadthedev
    Copy link
    Member

    arhadthedev commented Mar 4, 2023 •
    edited

    The feature is implemented before Python 3.3 (an enhancement of this feature is listed in 3.3's What's New, didn't try to search further) so closing as done.

    @bedevere-bot bedevere-bot mentioned this issue Apr 5, 2023
    Open
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    5 participants
    @zooba @serhiy-storchaka @arhadthedev @zware @terryjreedy