This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: calling a _json.Encoder object raises a SystemError in case obj.items() returned a tuple
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Oren Milman, berker.peksag, iritkatriel, rhettinger, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-09-15 17:21 by Oren Milman, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (6)
msg302278 - (view) Author: Oren Milman (Oren Milman) * Date: 2017-09-15 17:21
the following code causes a SystemError:

import json.encoder
class BadDict(dict):
    def items(self):
        return ()

encoder = json.encoder.c_make_encoder(None, None, None, None, 'foo', 'bar',
                                      True, None, None)
encoder(obj=BadDict({'spam': 42}), _current_indent_level=4)

this is because encoder_call() (in Modules/_json.c) passes the 'obj' argument
so that eventually encoder_listencode_dict() calls PyMapping_Items() on it.
encoder_listencode_dict() assumes that PyMapping_Items() returned a list, and
passes it to PyList_Sort().

ISTM that subclassing dict and implementing items() so that it returns a tuple
is not unrealistic.

maybe we should silently convert the tuple that PyMapping_Items() returned to a
msg302290 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-15 18:53
Oh, I was aware of this issue for long time, but didn't have good opportunity for fixing it. And yes, making PyMapping_Items() (and friends) always returning a list looks like a reasonable option to me. This could fix similar bugs in third-party extensions. In Python 2 PyMapping_Items() is documented as returning a list, but actually it can return an arbitrary type. Authors of extensions could be fooled by the documentation and use concrete list API.

The drawback of this solution is some performance degradation in rare case of items() returning a tuple.
msg304712 - (view) Author: Oren Milman (Oren Milman) * Date: 2017-10-21 16:39
ISTM that PR 3840 resolved this issue (as part of bpo-28280).
msg305794 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-11-07 21:57
PR 3840 has been merged and it looks like Oren was correct. I'm getting the following output with current master:

>>> encoder(obj=BadDict({'spam': 42}), _current_indent_level=4)
msg305796 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-11-07 22:35
bpo-28280 fixed this issue only in the master branch. I had left this issue open for fixing 3.6.
msg406717 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-21 14:36
Closing as fixed since it's too late to change 3.6 (if that did not happen yet).
Date User Action Args
2022-04-11 14:58:52adminsetgithub: 75667
2021-11-21 14:36:00iritkatrielsetnosy: + iritkatriel
messages: + msg406717
2021-11-21 14:34:54iritkatrielsetstatus: open -> closed
stage: needs patch -> resolved
resolution: fixed
versions: + Python 3.7, - Python 3.6
2017-11-07 22:35:54serhiy.storchakasetstatus: closed -> open
resolution: out of date -> (no value)
messages: + msg305796

stage: resolved -> needs patch
2017-11-07 21:57:13berker.peksagsetstatus: open -> closed

nosy: + berker.peksag
messages: + msg305794

resolution: out of date
stage: needs patch -> resolved
2017-10-21 17:20:38serhiy.storchakasetstage: needs patch
versions: + Python 3.6, - Python 3.7
2017-10-21 16:39:34Oren Milmansetmessages: + msg304712
2017-09-15 18:53:05serhiy.storchakasetnosy: + rhettinger, serhiy.storchaka
messages: + msg302290
2017-09-15 17:21:39Oren Milmancreate