What the details of this issue?
On 8 Oct 2015, at 13:57, Remi Pointel wrote:

> Changes by Remi Pointel <python@xiri.fr>:
>
>
> ----------
> nosy: rpointel
> priority: normal
> severity: normal
> status: open
> title: json
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue25342>
> _______________________________________
> _______________________________________________
> Python-bugs-list mailing list
> Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/stephane%40wirtel.be

Sorry I clicked on "create" before adding info in comment.

When I run the test_json suite on OpenBSD, I have a segfault:

$ egdb ./python            
GNU gdb (GDB) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-openbsd5.8".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./python...done.
(gdb) set env PYTHONPATH ./Lib
(gdb) set env LD_LIBRARY_PATH .
(gdb) set args ./Lib/test/test_json/
(gdb) r
Starting program: /home/remi/dev/cpython/python ./Lib/test/test_json/
.............................................s......................................................
Program received signal SIGSEGV, Segmentation fault.
0x00001dd119cf9e0d in PyEval_EvalFrameEx (f=<error reading variable: Cannot access memory at address 0x7f7fffbda518>, throwflag=<error reading variable: Cannot access memory at address 0x7f7fffbda514>) at Python/ceval.c:798
(gdb) bt
#0  0x00001dd119cf9e0d in PyEval_EvalFrameEx (f=<error reading variable: Cannot access memory at address 0x7f7fffbda518>, throwflag=<error reading variable: Cannot access memory at address 0x7f7fffbda514>)
    at Python/ceval.c:798
#1  0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11c90, arg=None, exc=0) at Objects/genobject.c:125
#2  0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11c90, arg=None) at Objects/genobject.c:223
#3  0x00001dd119d0267b in PyEval_EvalFrameEx (
    f=Frame 0x1dd32e645c38, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 324, in _iterencode_list (lst=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0, buf='[', newline_indent=None, separator=', ', first=False, value=<complex at remote 0x1dd3de8f74f0>, chunks=<generator at remote 0x1dd35dc11c90>), throwflag=0) at Python/ceval.c:2038
#4  0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11c08, arg=None, exc=0) at Objects/genobject.c:125
#5  0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11c08, arg=None) at Objects/genobject.c:223
#6  0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd39a980c38, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 427, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#7  0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11b80, arg=None, exc=0) at Objects/genobject.c:125
#8  0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11b80, arg=None) at Objects/genobject.c:223
#9  0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd39a980838, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 437, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#10 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11af8, arg=None, exc=0) at Objects/genobject.c:125
#11 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11af8, arg=None) at Objects/genobject.c:223
#12 0x00001dd119d0267b in PyEval_EvalFrameEx (
    f=Frame 0x1dd3b9e32c38, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 324, in _iterencode_list (lst=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0, buf='[', newline_indent=None, separator=', ', first=False, value=<complex at remote 0x1dd3de8f74f0>, chunks=<generator at remote 0x1dd35dc11af8>), throwflag=0) at Python/ceval.c:2038
#13 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11a70, arg=None, exc=0) at Objects/genobject.c:125
#14 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11a70, arg=None) at Objects/genobject.c:223
#15 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd32e645838, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 427, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#16 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc119e8, arg=None, exc=0) at Objects/genobject.c:125
#17 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc119e8, arg=None) at Objects/genobject.c:223
#18 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd39a980438, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 437, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#19 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11960, arg=None, exc=0) at Objects/genobject.c:125
#20 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11960, arg=None) at Objects/genobject.c:223
#21 0x00001dd119d0267b in PyEval_EvalFrameEx (
    f=Frame 0x1dd3b9e32838, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 324, in _iterencode_list (lst=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0, buf='[', newline_indent=None, separator=', ', first=False, value=<complex at remote 0x1dd3de8f74f0>, chunks=<generator at remote 0x1dd35dc11960>), throwflag=0) at Python/ceval.c:2038
#22 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc118d8, arg=None, exc=0) at Objects/genobject.c:125
#23 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc118d8, arg=None) at Objects/genobject.c:223
#24 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd3b9e32038, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 427, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#25 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11850, arg=None, exc=0) at Objects/genobject.c:125
#26 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11850, arg=None) at Objects/genobject.c:223
#27 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd3a36ba438, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 437, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#28 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc117c8, arg=None, exc=0) at Objects/genobject.c:125
#29 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc117c8, arg=None) at Objects/genobject.c:223
#30 0x00001dd119d0267b in PyEval_EvalFrameEx (
    f=Frame 0x1dd32e645438, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 324, in _iterencode_list (lst=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0, buf='[', newline_indent=None, separator=', ', first=False, value=<complex at remote 0x1dd3de8f74f0>, chunks=<generator at remote 0x1dd35dc117c8>), throwflag=0) at Python/ceval.c:2038
#31 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11740, arg=None, exc=0) at Objects/genobject.c:125
#32 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11740, arg=None) at Objects/genobject.c:223
#33 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd3b9e32438, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 427, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#34 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc116b8, arg=None, exc=0) at Objects/genobject.c:125
#35 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc116b8, arg=None) at Objects/genobject.c:223
#36 0x00001dd119d0267b in PyEval_EvalFrameEx (f=Frame 0x1dd3a36ba038, for file /home/remi/dev/cpython/Lib/json/encoder.py, line 437, in _iterencode (o=[<complex at remote 0x1dd3de8f74f0>], _current_indent_level=0), 
    throwflag=0) at Python/ceval.c:2038
#37 0x00001dd119dedbe3 in gen_send_ex (gen=0x1dd35dc11630, arg=None, exc=0) at Objects/genobject.c:125
#38 0x00001dd119dee6af in _PyGen_Send (gen=0x1dd35dc11630, arg=None) at Objects/genobject.c:223

It fails on test_endless_recursion:

$ LD_LIBRARY_PATH=. PYTHONPATH=./Lib/ ./python ./Lib/test/test_json -v

test_endless_recursion (test.test_json.test_recursion.TestPyRecursion) ... zsh: segmentation fault (core dumped)

Try setting the stack size to 8MB (the default on Linux) in login.conf.

It's good when I entered "ulimit -s 8192". Sorry for the noise...

It's not noise, we also have problems on Windows with test_json.

In theory we should set Py_DEFAULT_RECURSION_LIMIT in Python/ceval.c
to appropriate values for the default OS stack sizes in order to get
a proper RuntimeError instead of a segfault.

For OpenBSD it's clearly too high for the default.  Could you try
to find a safe value (perhaps 500 instead of 1000) for OpenBSD's
default?

Would it be possible to compute the recursion limit depending on the current maximum C stack size? At least estimate it? I guess the sys.getdefaultlimit() is 1000 is an arbitrary value. I also know that the effective limit depends on the memory allocated on the stack by C functions. Calling a Python function and a function implemented in C is different :-/

The ratio (1000/8MB) seems to be stable on Linux. Perhaps we could
write a getrlimit program for ./configure to get the stack size and
adjust the recursion limit to keep the ratio the same.

It's just an estimate of course.

Or did you want to compute it at runtime?  Actually, why not?

Stefan Krah added the comment:
> Or did you want to compute it at runtime?  Actually, why not?

I'm asking to adjust the limit at _runtime_. getrlimit() is not
something static, it's common to modify them manually for the current
shell process (and child process), or system-wide.

Duplicates and related issues: issue25329, issue25222, issue24999, issue22984, issue18075, issue12980.

Adjusting the estimate at runtime sounds good to me.