This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_json.test_recursion.TestPyRecursion.test_endless_recursion stack overflow in AMD64 Windows8.1 Non-Debug 3.x
Type: behavior Stage:
Components: Tests Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: serhiy.storchaka, steve.dower, vstinner, xtreak
Priority: normal Keywords:

Created on 2019-05-20 17:45 by xtreak, last changed 2022-04-11 14:59 by admin.

Messages (6)
msg342937 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-05-20 17:45
test_json results in stack overflow after the commit to implement __repr__ for weakset. This is very much similar to the one consistently occurring on my inspect module PR in Windows tests only for the past two weeks. My change was in changing inspect module and more specifically in importing ast at the top of inspect module. The relevant commit for buildbot failure is in implementing __repr__ for weakset.

Buildbot error : https://buildbot.python.org/all/#/builders/12/builds/2497

Error in my PR that is consistent and same : https://ci.appveyor.com/project/python/cpython/builds/24631185#L1547
msg343022 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-21 12:04
> Buildbot error : https://buildbot.python.org/all/#/builders/12/builds/2497

It's AMD64 Windows8.1 Non-Debug 3.x:

test_endless_recursion (test.test_json.test_recursion.TestPyRecursion) ... ok
Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00000820 (most recent call first):
  File "D:\buildarea\3.x.ware-win81-release.nondebug\build\lib\json\decoder.py", line 83 in py_scanstring
  File "D:\buildarea\3.x.ware-win81-release.nondebug\build\lib\json\decoder.py", line 167 in JSONObject
  File "D:\buildarea\3.x.ware-win81-release.nondebug\build\lib\json\scanner.py", line 37 in _scan_once
  (...)
  File "D:\buildarea\3.x.ware-win81-release.nondebug\build\lib\json\scanner.py", line 37 in _scan_once
  File "D:\buildarea\3.x.ware-win81-release.nondebug\build\lib\json\decoder.py", line 186 in JSONObject
  File "D:\buildarea\3.x.ware-win81-release.nondebug\build\lib\json\scanner.py", line 37 in _scan_once
  File "D:\buildarea\3.x.ware-win81-release.nondebug\build\lib\json\decoder.py", line 186 in JSONObject
  File "D:\buildarea\3.x.ware-win81-release.nondebug\build\lib\json\scanner.py", line 37 in _scan_once
  File "D:\buildarea\3.x.ware-win81-release.nondebug\build\lib\json\decoder.py", line 186 in JSONObject
  ...
test_highly_nested_objects_decoding (test.test_json.test_recursion.TestPyRecursion) ... 
D:\buildarea\3.x.ware-win81-release.nondebug\build>exit /b 0 

command timed out: 1200 seconds without output running [b'Tools\\buildbot\\test.bat', b'-x64', b'-j2', b'+d', b'-j4', b'--timeout', b'900'], attempting to kill
SIGKILL failed to kill process


> https://ci.appveyor.com/project/python/cpython/builds/24631185#L1547

test_endless_recursion (test.test_json.test_recursion.TestPyRecursion) ... ok
test_highly_nested_objects_decoding (test.test_json.test_recursion.TestPyRecursion) ... Fatal Python error: Cannot recover from stack overflow.
Current thread 0x00000e2c (most recent call first):
  File "C:\projects\cpython\lib\json\decoder.py", line 83 in py_scanstring
  File "C:\projects\cpython\lib\json\decoder.py", line 167 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  (...)
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  File "C:\projects\cpython\lib\json\scanner.py", line 37 in _scan_once
  File "C:\projects\cpython\lib\json\decoder.py", line 186 in JSONObject
  ...
C:\projects\cpython>set lastexitcode=-1073740791 
C:\projects\cpython>set  1>C:\Users\appveyor\AppData\Local\Temp\1\tmp4CD0.tmp 
C:\projects\cpython>echo C:\projects\cpython  1>C:\Users\appveyor\AppData\Local\Temp\1\tmp4CD1.tmp 
C:\projects\cpython>exit /b -1073740791
msg343025 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-21 12:11
The mashal also caused similar issues with stack overflow in the past:

commit f6c69e6cc9aac35564a2a2a7ecc43fa8db6da975
Author: Steve Dower <steve.dower@microsoft.com>
Date:   Sat Nov 1 15:15:16 2014 -0700

    #22734 marshal needs a lower stack depth for debug builds on Windows

commit 24e33acf8c422f6b8f84387242ff7874012f7291
Author: Victor Stinner <victor.stinner@gmail.com>
Date:   Sun Jul 7 02:49:07 2013 +0200

    Issue #17206: On Windows, increase the stack size from 2 MB to 4.2 MB to fix
    a stack overflow in the marshal module (fix a crash in test_marshal).
    Patch written by Jeremy Kloth.


History of PyOS_CheckStack:

commit 5299935be5acefae819bcc08a888a9466747d7cb
Author: Kristján Valur Jónsson <kristjan@ccpgames.com>
Date:   Mon Feb 18 17:40:47 2008 +0000

    Perform correct handling of stack overflow for windows: Catch the correct exception code and reset the overflow condition when handled.

commit dc61901dd2b1bcb5467ebc36ef35a51106d9b103
Author: Amaury Forgeot d'Arc <amauryfa@gmail.com>
Date:   Sat Nov 22 20:01:18 2008 +0000

    #3996: On Windows, PyOS_CheckStack is supposed to protect the interpreter from
    stack overflow. But doing this, it always crashes when the stack is nearly full.
    
    Reviewed by Martin von Loewis. Will backport to 2.6.

commit 92e4dd865709f4a4be2b11453ef4de954b8d7b14
Author: Tim Peters <tim.peters@gmail.com>
Date:   Sat Oct 5 01:47:34 2002 +0000

    s/_alloca/alloca/g; Windows doesn't need the former, at least not unless
    __STDC__ is defined (or something like that ...).

commit 399739f79f1d6cef6bc8a8ec63f9a81255464f15
Author: Fred Drake <fdrake@acm.org>
Date:   Thu Aug 31 05:52:44 2000 +0000

    PyOS_CheckStack():  Better ANSI'fy this while we're at it.

commit 2f15b25da2060ab723e0bb82a8f4f713d547b2b8
Author: Fredrik Lundh <fredrik@pythonware.com>
Date:   Sun Aug 27 19:15:31 2000 +0000

    implements PyOS_CheckStack for Windows and MSVC.  this fixes a
    couple of potential stack overflows, including bug #110615.
    
    closes patch #101238
msg343028 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-21 12:13
bpo-25329 was a similar json crash with stack overflow in 2015. The issue gives pointers to:

* bpo-25222: 3.5.0 regression - Fatal Python error: Cannot recover from stack overflow
* bpo-25342: test_json segfault on OpenBSD
msg343029 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-21 12:14
See also:

* bpo-25240: Stack overflow in reprlib causes a core dump
* bpo-28913: "Fatal Python error: Cannot recover from stack overflow." from RecursionError in Python 3.5
* bpo-22583: C stack overflow in the Python 2.7 compiler
msg343033 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-05-21 12:21
I have seen it occurring consistently in my PR 10307 in Travis and AppVeyor. It seems to be reproducible by importing ast at the top in inspect module which seems to have an effect as libregrtest is used to run the test_json. Using unittest module to run test_json doesn't seem to crash the last time I have checked.
History
Date User Action Args
2022-04-11 14:59:15adminsetgithub: 81154
2019-05-21 12:21:28xtreaksetmessages: + msg343033
2019-05-21 12:14:23vstinnersetmessages: + msg343029
2019-05-21 12:13:15vstinnersetnosy: + serhiy.storchaka
2019-05-21 12:13:10vstinnersetmessages: + msg343028
2019-05-21 12:11:35vstinnersetmessages: + msg343025
2019-05-21 12:04:31vstinnersetmessages: + msg343022
2019-05-20 17:45:36xtreakcreate