test_json segfault on OpenBSD #69529

rpointel · 2015-10-08T11:57:40Z

BPO	25342
Nosy	@vstinner, @skrah, @serhiy-storchaka, @matrixise

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2015-10-08.11:57:39.664>
labels = []
title = 'test_json segfault on OpenBSD'
updated_at = <Date 2015-10-10.15:51:58.146>
user = 'https://bugs.python.org/rpointel'

bugs.python.org fields:

activity = <Date 2015-10-10.15:51:58.146>
actor = 'skrah'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2015-10-08.11:57:39.664>
creator = 'rpointel'
dependencies = []
files = []
hgrepos = []
issue_num = 25342
keywords = []
message_count = 12.0
messages = ['252531', '252532', '252533', '252534', '252535', '252537', '252538', '252539', '252540', '252547', '252550', '252721']
nosy_count = 5.0
nosy_names = ['vstinner', 'skrah', 'rpointel', 'serhiy.storchaka', 'matrixise']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue25342'
versions = ['Python 3.6']

matrixise · 2015-10-08T11:59:16Z

What the details of this issue?
On 8 Oct 2015, at 13:57, Remi Pointel wrote:

Changes by Remi Pointel <python@xiri.fr>:

----------
nosy: rpointel
priority: normal
severity: normal
status: open
title: json

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue25342\>

Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/stephane%40wirtel.be

rpointel · 2015-10-08T12:00:28Z

Sorry I clicked on "create" before adding info in comment.

When I run the test_json suite on OpenBSD, I have a segfault:

$ egdb ./python            
GNU gdb (GDB) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-openbsd5.8".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./python...done.
(gdb) set env PYTHONPATH ./Lib
(gdb) set env LD_LIBRARY_PATH .
(gdb) set args ./Lib/test/test_json/
(gdb) r
Starting program: /home/remi/dev/cpython/python ./Lib/test/test_json/
.............................................s......................................................
Program received signal SIGSEGV, Segmentation fault.
0x00001dd119cf9e0d in PyEval_EvalFrameEx (f=<error reading variable: Cannot access memory at address 0x7f7fffbda518>, throwflag=<error reading variable: Cannot access memory at address 0x7f7fffbda514>) at Python/ceval.c:798
(gdb) bt
#0  0x00001dd119cf9e0d in PyEval_EvalFrameEx (f=<error reading variable: Cannot access memory at address 0x7f7fffbda518>, throwflag=<error reading variable: Cannot access memory at address 0x7f7fffbda514>)
    at Python/ceval.c:798
#1  0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11c90, arg=None, exc=0) at Objects/genobject.c:125
#2  0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11c90, arg=None) at Objects/genobject.c:223
#3  0x00001dd119d0267b in PyEval_EvalFrameEx (
    f=Frame 0x1dd32e645c38, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 324, in _iterencode_list (lst=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0, buf='[', newline_indent=None, separator=', ', first=False, value=<complex at remote 0x1dd3de8f74f0>, chunks=<generator at remote 0x1dd35dc11c90>), throwflag=0) at Python/ceval.c:2038
#4  0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11c08, arg=None, exc=0) at Objects/genobject.c:125
#5  0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11c08, arg=None) at Objects/genobject.c:223
#6  0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd39a980c38, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 427, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#7  0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11b80, arg=None, exc=0) at Objects/genobject.c:125
#8  0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11b80, arg=None) at Objects/genobject.c:223
#9  0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd39a980838, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 437, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#10 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11af8, arg=None, exc=0) at Objects/genobject.c:125
#11 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11af8, arg=None) at Objects/genobject.c:223
#12 0x00001dd119d0267b in PyEval_EvalFrameEx (
    f=Frame 0x1dd3b9e32c38, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 324, in _iterencode_list (lst=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0, buf='[', newline_indent=None, separator=', ', first=False, value=<complex at remote 0x1dd3de8f74f0>, chunks=<generator at remote 0x1dd35dc11af8>), throwflag=0) at Python/ceval.c:2038
#13 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11a70, arg=None, exc=0) at Objects/genobject.c:125
#14 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11a70, arg=None) at Objects/genobject.c:223
#15 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd32e645838, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 427, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#16 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc119e8, arg=None, exc=0) at Objects/genobject.c:125
#17 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc119e8, arg=None) at Objects/genobject.c:223
#18 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd39a980438, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 437, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#19 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11960, arg=None, exc=0) at Objects/genobject.c:125
#20 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11960, arg=None) at Objects/genobject.c:223
#21 0x00001dd119d0267b in PyEval_EvalFrameEx (
    f=Frame 0x1dd3b9e32838, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 324, in _iterencode_list (lst=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0, buf='[', newline_indent=None, separator=', ', first=False, value=<complex at remote 0x1dd3de8f74f0>, chunks=<generator at remote 0x1dd35dc11960>), throwflag=0) at Python/ceval.c:2038
#22 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc118d8, arg=None, exc=0) at Objects/genobject.c:125
#23 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc118d8, arg=None) at Objects/genobject.c:223
#24 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd3b9e32038, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 427, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#25 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11850, arg=None, exc=0) at Objects/genobject.c:125
#26 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11850, arg=None) at Objects/genobject.c:223
#27 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd3a36ba438, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 437, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#28 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc117c8, arg=None, exc=0) at Objects/genobject.c:125
#29 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc117c8, arg=None) at Objects/genobject.c:223
#30 0x00001dd119d0267b in PyEval_EvalFrameEx (
    f=Frame 0x1dd32e645438, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 324, in _iterencode_list (lst=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0, buf='[', newline_indent=None, separator=', ', first=False, value=<complex at remote 0x1dd3de8f74f0>, chunks=<generator at remote 0x1dd35dc117c8>), throwflag=0) at Python/ceval.c:2038
#31 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11740, arg=None, exc=0) at Objects/genobject.c:125
#32 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11740, arg=None) at Objects/genobject.c:223
#33 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd3b9e32438, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 427, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#34 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc116b8, arg=None, exc=0) at Objects/genobject.c:125
#35 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc116b8, arg=None) at Objects/genobject.c:223
#36 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd3a36ba038, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 437, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#37 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11630, arg=None, exc=0) at Objects/genobject.c:125
#38 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11630, arg=None) at Objects/genobject.c:223

rpointel · 2015-10-08T12:11:09Z

It fails on test_endless_recursion:

$ LD_LIBRARY_PATH=. PYTHONPATH=./Lib/ ./python ./Lib/test/test_json -v

test_endless_recursion (test.test_json.test_recursion.TestPyRecursion) ... zsh: segmentation fault (core dumped)

skrah · 2015-10-08T12:18:59Z

Try setting the stack size to 8MB (the default on Linux) in login.conf.

rpointel · 2015-10-08T12:32:53Z

It's good when I entered "ulimit -s 8192". Sorry for the noise...

skrah · 2015-10-08T12:43:10Z

It's not noise, we also have problems on Windows with test_json.

In theory we should set Py_DEFAULT_RECURSION_LIMIT in Python/ceval.c
to appropriate values for the default OS stack sizes in order to get
a proper RuntimeError instead of a segfault.

For OpenBSD it's clearly too high for the default. Could you try
to find a safe value (perhaps 500 instead of 1000) for OpenBSD's
default?

vstinner · 2015-10-08T12:55:12Z

Would it be possible to compute the recursion limit depending on the current maximum C stack size? At least estimate it? I guess the sys.getdefaultlimit() is 1000 is an arbitrary value. I also know that the effective limit depends on the memory allocated on the stack by C functions. Calling a Python function and a function implemented in C is different :-/

skrah · 2015-10-08T13:16:26Z

The ratio (1000/8MB) seems to be stable on Linux. Perhaps we could
write a getrlimit program for ./configure to get the stack size and
adjust the recursion limit to keep the ratio the same.

It's just an estimate of course.

skrah · 2015-10-08T13:20:41Z

Or did you want to compute it at runtime? Actually, why not?

vstinner · 2015-10-08T15:52:53Z

Stefan Krah added the comment:

Or did you want to compute it at runtime? Actually, why not?

I'm asking to adjust the limit at _runtime_. getrlimit() is not
something static, it's common to modify them manually for the current
shell process (and child process), or system-wide.

serhiy-storchaka · 2015-10-08T16:03:52Z

Duplicates and related issues: bpo-25329, bpo-25222, bpo-24999, bpo-22984, bpo-18075, bpo-12980.

skrah · 2015-10-10T15:51:58Z

Adjusting the estimate at runtime sounds good to me.

kumaraditya303 · 2022-06-19T10:27:30Z

Closing as OpenBSD is not supported as per PEP 11.

vstinner · 2022-06-20T13:52:26Z

Lib/test/test_json/test_recursion.py now uses with support.infinite_recursion(): which uses a lower recursion limit and so is less likely to crash with a stack overflow. Previously, the test required a large stack.

rpointel mannequin changed the title ~~json~~ test_json segfault on OpenBSD Oct 8, 2015

ezio-melotti transferred this issue from another repository Apr 10, 2022

AlexWaygood added the type-crash A hard crash of the interpreter, possibly with a core dump label Apr 10, 2022

kumaraditya303 closed this as not planned Won't fix, can't repro, duplicate, stale Jun 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_json segfault on OpenBSD #69529

test_json segfault on OpenBSD #69529

rpointel mannequin commented Oct 8, 2015

matrixise commented Oct 8, 2015

rpointel mannequin commented Oct 8, 2015

rpointel mannequin commented Oct 8, 2015

skrah mannequin commented Oct 8, 2015

rpointel mannequin commented Oct 8, 2015

skrah mannequin commented Oct 8, 2015

vstinner commented Oct 8, 2015

skrah mannequin commented Oct 8, 2015

skrah mannequin commented Oct 8, 2015

vstinner commented Oct 8, 2015

serhiy-storchaka commented Oct 8, 2015

skrah mannequin commented Oct 10, 2015

kumaraditya303 commented Jun 19, 2022

vstinner commented Jun 20, 2022

test_json segfault on OpenBSD #69529

test_json segfault on OpenBSD #69529

Comments

rpointel mannequin commented Oct 8, 2015

matrixise commented Oct 8, 2015

rpointel mannequin commented Oct 8, 2015

rpointel mannequin commented Oct 8, 2015

skrah mannequin commented Oct 8, 2015

rpointel mannequin commented Oct 8, 2015

skrah mannequin commented Oct 8, 2015

vstinner commented Oct 8, 2015

skrah mannequin commented Oct 8, 2015

skrah mannequin commented Oct 8, 2015

vstinner commented Oct 8, 2015

serhiy-storchaka commented Oct 8, 2015

skrah mannequin commented Oct 10, 2015

kumaraditya303 commented Jun 19, 2022

vstinner commented Jun 20, 2022