classification
Title: json.dumps infinite recurssion
Type: crash Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: MultiSosnooley, ezio.melotti, rhettinger, xtreak
Priority: normal Keywords:

Created on 2019-02-05 16:50 by MultiSosnooley, last changed 2021-09-07 17:31 by iritkatriel. This issue is now closed.

Messages (3)
msg334877 - (view) Author: MultiSosnooley (MultiSosnooley) Date: 2019-02-05 16:50
```
__import__('json').dumps(object(), default=lambda o: repr(o).encode())
```
Produce infinite recursion on `default` function.

Here is more informative example:
```
>>> def f(o):
...     input(f"{o!r} {type(o)}")
...     return repr(o).encode()
... 
>>> import json
>>> json.dumps(object(), default=f)
<object object at 0x7f8c87e987c0> <class 'object'>
b'<object object at 0x7f8c87e987c0>' <class 'bytes'>
b"b'<object object at 0x7f8c87e987c0>'" <class 'bytes'>
b'b"b\'<object object at 0x7f8c87e987c0>\'"' <class 'bytes'>
b'b\'b"b\\\'<object object at 0x7f8c87e987c0>\\\'"\'' <class 'bytes'>
b'b\'b\\\'b"b\\\\\\\'<object object at 0x7f8c87e987c0>\\\\\\\'"\\\'\'' <class 'bytes'>
```
msg334880 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-02-05 18:18
I guess json module expects a return value from default function that is str, int, float, etc. that can be handled by built in encoders and returning a bytes object with repr causes it to keep calling default until a value of one of expected type is returned. With the encoded repr producing new objects circular reference cannot be detected. In the below example I keep returning encoded repr as per the original report and then return a string once length is more than 100. I think this is something similar to https://bugs.python.org/issue21213#msg217203 .

# /tmp/foo.py

import json;

def default(o):
    out = repr(o).encode()
    print(id(out))
    if len(out) > 100:
        print(out)
        return "string"
    return out

print(json.dumps(object(), default=default))


./python.exe /tmp/foo.py
4329334144
4328290960
4328289400
4329073088
4328910624
4329031520
4327246416
4328875288
b'b\'b\\\'b\\\\\\\'b\\\\\\\\\\\\\\\'b\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'b"b\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'<object object at 0x101da6580>\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'\\\\\\\\\\\\\\\'\\\\\\\'\\\'\''
"string"
msg334885 - (view) Author: MultiSosnooley (MultiSosnooley) Date: 2019-02-05 21:51
Oh, I got it. Size is too high growing to reach recurtion limit. I replace repr with slow growing data and now there is good old recursion limit exception.

import json

class F:
    counter = 0
    total = 0
    data = b""

    def __call__(self, o):
        self.counter += 1
        self.data = b"1" + self.data
        self.total += len(self.data)
        print(self.counter, len(self.data), self.total)
        return self.data

json.dumps(object(), default=F())
History
Date User Action Args
2021-09-07 17:31:21iritkatrielsetstatus: open -> closed
resolution: not a bug
stage: resolved
2019-02-05 21:51:07MultiSosnooleysetmessages: + msg334885
2019-02-05 18:18:37xtreaksetnosy: + rhettinger, ezio.melotti, xtreak
messages: + msg334880
2019-02-05 16:50:38MultiSosnooleycreate