classification
Title: test_json.test_endless_recursion(): "Fatal Python error: Cannot recover from stack overflow." on x86 XP-4 3.x buildbot
Type: crash Stage:
Components: Extension Modules, Tests Versions: Python 3.5
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: ezio.melotti, rhettinger, serhiy.storchaka, vstinner
Priority: normal Keywords: buildbot, patch

Created on 2014-12-02 21:15 by vstinner, last changed 2015-03-18 10:52 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
test_endless_recursion.patch vstinner, 2014-12-05 01:01 review
Messages (7)
msg232049 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-12-02 21:15
http://buildbot.python.org/all/builders/x86%20XP-4%203.x/builds/11271/steps/test/logs/stdio

[116/390] test_json
Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00000e84 (most recent call first):
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\json\encoder.py", line 250 in iterencode
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\json\encoder.py", line 192 in encode
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_json\test_recursion.py", line 96 in test_endless_recursion
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\case.py", line 577 in run
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\case.py", line 625 in __call__
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 125 in run
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 87 in __call__
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 125 in run
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 87 in __call__
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 125 in run
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 87 in __call__
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 125 in run
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 87 in __call__
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 125 in run
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py", line 87 in __call__
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\runner.py", line 168 in run
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\support\__init__.py", line 1769 in _run_suite
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\support\__init__.py", line 1803 in run_unittest
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\..\lib\test\regrtest.py", line 1279 in test_runner
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\..\lib\test\regrtest.py", line 1280 in runtest_inner
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\..\lib\test\regrtest.py", line 967 in runtest
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\..\lib\test\regrtest.py", line 763 in main
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\..\lib\test\regrtest.py", line 1564 in main_in_temp_cwd
  File "d:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\PCbuild\..\lib\test\regrtest.py", line 1589 in <module>
msg232054 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-02 21:58
JSON encoder uses recursion calls guards but they don't save from an overflow of C stack.

How to reproduce:

import json
sys.setrecursionlimit(1000000000)
json.dumps(5j, check_circular=False, default=lambda o: [o])
msg232170 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-12-05 01:01
Here is a patch to use a lower recursion limit to no stress the C stack.
msg232181 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-05 08:41
I think this patch will break the purpose of the test. We should fix the issue instead of hide it from tests.
msg232187 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-12-05 09:41
> I think this patch will break the purpose of the test. We should fix the issue instead of hide it from tests.

The test ensures that json calls Py_EnterRecursiveCall() which relies on sys.getrecursionlimit(), see the commit below.

There is no generic fix for C stack overflow. The tes failed on Windows XP which is no more supported, I don't think that it's interesting to spend time on it.

changeset:   70021:9557e4eeb291
branch:      3.1
parent:      69997:b7abf0590e1c
user:        Ezio Melotti <ezio.melotti@gmail.com>
date:        Wed May 11 01:02:56 2011 +0300
files:       Lib/json/tests/test_recursion.py Misc/NEWS Modules/_json.c
description:
#12051: Fix segfault in json.dumps() while encoding highly-nested objects using the C accelerations.
msg232202 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-05 18:31
> The tes failed on Windows XP which is no more supported, I don't think that
> it's interesting to spend time on it.

So may be just skip it on Windows XP?

Or as far as this approach doesn't solve the problem and the test may crash on 
other platforms, may be run this test in a subprocess?
msg238406 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-18 10:52
The buildbot is broken, it cannot compile anymore. So I just close the issue.
History
Date User Action Args
2015-03-18 10:52:20vstinnersetstatus: open -> closed
resolution: out of date
messages: + msg238406
2014-12-05 18:37:46serhiy.storchakasetnosy: + ezio.melotti
2014-12-05 18:31:53serhiy.storchakasetmessages: + msg232202
2014-12-05 09:41:16vstinnersetmessages: + msg232187
2014-12-05 08:42:26serhiy.storchakasetnosy: + rhettinger
2014-12-05 08:41:30serhiy.storchakasetmessages: + msg232181
2014-12-05 01:01:50vstinnersetfiles: + test_endless_recursion.patch
keywords: + patch
messages: + msg232170
2014-12-02 21:58:07serhiy.storchakasetmessages: + msg232054
components: + Extension Modules
2014-12-02 21:46:21serhiy.storchakasetnosy: + serhiy.storchaka
2014-12-02 21:15:19vstinnercreate