classification
Title: Misc fixes and cleanups in error handling C code
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ZackerySpytz, larry, ned.deily, paul.moore, serhiy.storchaka, steve.dower, terry.reedy, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2016-12-16 23:38 by serhiy.storchaka, last changed 2018-09-12 11:28 by serhiy.storchaka.

Files
File name Uploaded Description Edit
errors.diff serhiy.storchaka, 2016-12-16 23:38 review
Pull Requests
URL Status Linked Edit
PR 2034 merged serhiy.storchaka, 2017-06-09 19:32
PR 2035 merged serhiy.storchaka, 2017-06-09 19:58
PR 2072 open serhiy.storchaka, 2017-06-10 06:58
PR 2121 merged serhiy.storchaka, 2017-06-12 05:32
PR 2122 merged serhiy.storchaka, 2017-06-12 05:34
PR 2123 merged serhiy.storchaka, 2017-06-12 05:55
PR 9214 merged ZackerySpytz, 2018-09-12 08:25
Messages (14)
msg283451 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-16 23:38
Proposed patch makes few minor changes related to handling errors in C.

* Fixed possible NULL dereference in PyErr_NormalizeException().

* Fixed error checking in atexit._run_exitfuncs().

* Fixed possible memory leaks in _Py_FatalError_PrintExc().

* PyErr_NormalizeException() no longer recursive.

* If an error is occurred during handling other error, the original exception is chained as the __contex__ attribute to the new exception instead of just be dropped.

* PyTraceBack_Print() no longer fails OverflowError if tracebacklimit is very large negative or positive value.

* ctype error message now include the name of the initial exception instead of the repr of its class.

* Py_XDECREFs is replaced with Py_DECREFs if this is safe.

* Added few asserts.

* Other minor cleanups.
msg295618 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-10 07:01
I'm trying to split the patch on smaller parts.
msg295743 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-12 05:25
New changeset 3fd54d4a7e604067e2bc0f8cfd58bdbdc09fa7f4 by Serhiy Storchaka in branch 'master':
bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (#2034)
https://github.com/python/cpython/commit/3fd54d4a7e604067e2bc0f8cfd58bdbdc09fa7f4
msg295744 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-12 06:02
New changeset d89dc844d288ee2e403272f3e4552eed6911cafd by Serhiy Storchaka in branch '3.6':
[3.6] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (#2121)
https://github.com/python/cpython/commit/d89dc844d288ee2e403272f3e4552eed6911cafd
msg295745 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-12 06:02
New changeset 7d8c1ebd86ce27b28736c5e97fef58ec60b8ef31 by Serhiy Storchaka in branch '3.5':
[3.5] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (#2122)
https://github.com/python/cpython/commit/7d8c1ebd86ce27b28736c5e97fef58ec60b8ef31
msg295747 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-12 06:10
New changeset 0cc43df05e36655220468953e838169966b94ebd by Serhiy Storchaka in branch '2.7':
[2.7] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (#2123)
https://github.com/python/cpython/commit/0cc43df05e36655220468953e838169966b94ebd
msg295817 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-12 19:42
3.6 no longer compiles for me on Windows.  A repeat of pcbuild\build.bat -d gives the same error report as below.

My previous build was 25 hours ago.  atexitmodule.c is the only C file in the 3.6 merge update, so the backport might be to blame.  Git status say that Objects/listobject is changed, even though not changed by PR 2121, and that there is an untracked Objects/clinic/listobject.c.h. (but there is one in 3.7, so...?).

chkdsk F: /F found one unrelated orphaned file but not other specific errors.  It ended with this:

Stage 3: Examining security descriptors ...
Security descriptor verification completed.
  40783 data files processed.
CHKDSK is verifying Usn Journal...
  3224 USN bytes processed.
Usn Journal verification completed.
An unspecified error occurred (6e74667363686b2e 1475).

(which it also did a week ago) I don't remember what was done after the USN check.  Web search has not helped yet for the 63... error.  In any case, I thing someone else should verify building on Windows before 3.6.2 is tagged, if it has not yet been. 

  Building heads/3.6-dirty:8399a177de 3.6
  atexitmodule.c
  listobject.c
f:\dev\36\objects\clinic/listobject.c.h(24): warning C4133: 'function': incompatible type
s - from 'char [10]' to 'PyObject *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
f:\dev\36\objects\clinic/listobject.c.h(25): warning C4133: 'function': incompatible type
s - from 'Py_ssize_t *' to '_PyArg_Parser *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
f:\dev\36\objects\clinic/listobject.c.h(29): warning C4013: '_PyArg_NoStackKeywords' unde
fined; assuming extern returning int [f:\dev\36\PCbuild\pythoncore.vcxproj]
f:\dev\36\objects\clinic/listobject.c.h(112): warning C4133: 'function': incompatible typ
es - from 'char [7]' to 'PyObject *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
f:\dev\36\objects\clinic/listobject.c.h(113): warning C4133: 'function': incompatible typ
es - from 'Py_ssize_t *' to '_PyArg_Parser *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
f:\dev\36\objects\clinic/listobject.c.h(147): warning C4013: '_PyArg_ParseStackAndKeyword
s' undefined; assuming extern returning int [f:\dev\36\PCbuild\pythoncore.vcxproj]
f:\dev\36\objects\clinic/listobject.c.h(198): warning C4133: 'function': incompatible typ
es - from 'char [13]' to 'PyObject *' [f:\dev\36\PCbuild\pythoncore.vcxproj]
f:\dev\36\objects\clinic/listobject.c.h(199): warning C4047: 'function': '_PyArg_Parser *
' differs in levels of indirection from 'PyObject **' [f:\dev\36\PCbuild\pythoncore.vcxpr
oj]
f:\dev\36\objects\clinic/listobject.c.h(199): warning C4024: '_PyArg_ParseStack': differe
nt types for formal and actual parameter 4 [f:\dev\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(1979): error C2059: syntax error: '<<' [f:\dev\36\PCbuild\pythonc
ore.vcxproj]
..\Objects\listobject.c(1984): error C2014: preprocessor command must start as first nonw
hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2015): error C2014: preprocessor command must start as first nonw
hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2171): error C2143: syntax error: missing ';' before '<<' [f:\dev
\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2173): error C2143: syntax error: missing ';' before '==' [f:\dev
\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2175): error C2143: syntax error: missing ';' before '>>' [f:\dev
\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2175): error C2014: preprocessor command must start as first nonw
hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2256): error C2143: syntax error: missing ';' before '<<' [f:\dev
\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2261): error C2143: syntax error: missing ';' before '==' [f:\dev
\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2262): error C2014: preprocessor command must start as first nonw
hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2268): error C2059: syntax error: 'if' [f:\dev\36\PCbuild\pythonc
ore.vcxproj]
..\Objects\listobject.c(2273): error C2059: syntax error: 'for' [f:\dev\36\PCbuild\python
core.vcxproj]
..\Objects\listobject.c(2273): error C2143: syntax error: missing '{' before '<' [f:\dev\
36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2273): error C2059: syntax error: '<' [f:\dev\36\PCbuild\pythonco
re.vcxproj]
..\Objects\listobject.c(2273): error C2143: syntax error: missing '{' before '++' [f:\dev
\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(2273): error C2059: syntax error: '++' [f:\dev\36\PCbuild\pythonc
ore.vcxproj]
..\Objects\listobject.c(2273): error C2059: syntax error: ')' [f:\dev\36\PCbuild\pythonco
re.vcxproj]
..\Objects\listobject.c(2280): error C2059: syntax error: 'string' [f:\dev\36\PCbuild\pyt
honcore.vcxproj]
..\Objects\listobject.c(2280): error C2040: 'PyErr_Format': 'int ()' differs in levels of
 indirection from 'PyObject *(PyObject *,const char *,...)' [f:\dev\36\PCbuild\pythoncore
.vcxproj]
..\Objects\listobject.c(2281): error C2059: syntax error: 'return' [f:\dev\36\PCbuild\pyt
honcore.vcxproj]
..\Objects\listobject.c(2282): error C2059: syntax error: '}' [f:\dev\36\PCbuild\pythonco
re.vcxproj]
..\Objects\listobject.c(2422): error C2059: syntax error: '<<' [f:\dev\36\PCbuild\pythonc
ore.vcxproj]
..\Objects\listobject.c(2443): error C2014: preprocessor command must start as first nonw
hite space [f:\dev\36\PCbuild\pythoncore.vcxproj]
..\Objects\listobject.c(3113): fatal error C1004: unexpected end-of-file found [f:\dev\36
\PCbuild\pythoncore.vcxproj]
  Generating Code...

f:\dev\36>git log  Modules/atexitmodule.c
commit d89dc844d288ee2e403272f3e4552eed6911cafd
Author: Serhiy Storchaka <storchaka@gmail.com>
Date:   Mon Jun 12 09:02:13 2017 +0300

    [3.6] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (#2121)

    The traceback no longer displayed for SystemExit raised in a callback registered by atexit..
    (cherry picked from commit 3fd54d4a7e604067e2bc0f8cfd58bdbdc09fa7f4)

When I reverted the listobject.c change and rebuilt, compilation finished and the test suite ran to SUCCESS.
msg295820 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-12 20:03
Last time Objects/listobject.c was changed 2 months ago. This looks as an error in your workspace. Maybe something wrong happened when you switched from master to 3.6.
msg295821 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-06-12 20:13
Agreed, looks like something went wrong in your checkout, Terry.  Just confirmed that a 3.6 build on a fresh checkout on Windows is fine.
msg295822 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-06-12 20:14
I just updated to the latest 3.6 commit and listobject.c doesn't even use clinic in this branch.

Serhiy is probably right - this is a workspace issue.
msg295823 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-12 20:28
Since I fetch, merge, and build with a .bat file that has worked fine at least 10 times, with pushes in between, the glitch is a puzzle.  Next time I will just revert, as I did, or reclone and rebuild the 3.6 worktree.
msg305588 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-11-05 09:27
New changeset cf296537f164abeacd83011239881f75f290ed31 by Serhiy Storchaka in branch 'master':
bpo-28994: PyErr_NormalizeException() no longer use C stack for recursion. (#2035)
https://github.com/python/cpython/commit/cf296537f164abeacd83011239881f75f290ed31
msg325129 - (view) Author: Zackery Spytz (ZackerySpytz) * Date: 2018-09-12 08:27
In PR 2123, it was reported that the Modules/atexitmodule.c file was backported to 2.7. PR 9214 addresses this.
msg325133 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-12 11:28
New changeset b36567bef80202f53ebe924dd183270c276497f8 by Serhiy Storchaka (Zackery Spytz) in branch '2.7':
[2.7] bpo-28994: Remove mistakenly backported atexitmodule.c (GH-9214)
https://github.com/python/cpython/commit/b36567bef80202f53ebe924dd183270c276497f8
History
Date User Action Args
2018-09-12 11:28:24serhiy.storchakasetmessages: + msg325133
2018-09-12 08:27:38ZackerySpytzsetnosy: + ZackerySpytz
messages: + msg325129
2018-09-12 08:25:40ZackerySpytzsetpull_requests: + pull_request8646
2017-11-05 09:27:53serhiy.storchakasetmessages: + msg305588
2017-06-12 20:28:51terry.reedysetmessages: + msg295823
2017-06-12 20:14:11steve.dowersetmessages: + msg295822
2017-06-12 20:13:07zach.waresetpriority: release blocker -> normal

messages: + msg295821
2017-06-12 20:03:55serhiy.storchakasetmessages: + msg295820
2017-06-12 19:42:39terry.reedysetpriority: normal -> release blocker
nosy: + terry.reedy, ned.deily, larry, paul.moore, tim.golden, zach.ware, steve.dower
messages: + msg295817

2017-06-12 06:10:55serhiy.storchakasetmessages: + msg295747
2017-06-12 06:02:35serhiy.storchakasetmessages: + msg295745
2017-06-12 06:02:16serhiy.storchakasetmessages: + msg295744
2017-06-12 05:55:40serhiy.storchakasetpull_requests: + pull_request2177
2017-06-12 05:34:45serhiy.storchakasetpull_requests: + pull_request2176
2017-06-12 05:32:51serhiy.storchakasetpull_requests: + pull_request2175
2017-06-12 05:25:07serhiy.storchakasetmessages: + msg295743
2017-06-10 07:01:42serhiy.storchakasetmessages: + msg295618
2017-06-10 06:58:33serhiy.storchakasetpull_requests: + pull_request2135
2017-06-09 19:58:00serhiy.storchakasetpull_requests: + pull_request2100
2017-06-09 19:32:07serhiy.storchakasetpull_requests: + pull_request2099
2016-12-16 23:38:16serhiy.storchakacreate