classification
Title: Python version of JSON decoder does not work with object_pairs_hook
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4, Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ezio.melotti Nosy List: Kuukunen, ezio.melotti, python-dev, r.david.murray, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2013-03-06 16:31 by Kuukunen, last changed 2013-03-12 23:57 by ezio.melotti. This issue is now closed.

Files
File name Uploaded Description Edit
issue17368.diff ezio.melotti, 2013-03-12 05:05 Patch against 3.2.
Messages (6)
msg183600 - (view) Author: Aki (Kuukunen) Date: 2013-03-06 16:31
Specifying any object_pairs_hook makes JSON decoding break when JSONObject from json/decoder.py is used.

Can be emulated easily by setting c_make_scanner = None at json/scanner.py

Example script:
**
import json
test = '{"key": "value", "empty": {}}'

def hook(pairs):
    return dict(pairs)

print(json.loads(test, object_pairs_hook=hook))
**

This test will fail because the return statement on line 166 in http://hg.python.org/cpython/file/cfc777407b03/Lib/json/decoder.py lacks "+ 1" even though the statement on line 170 has it. Basically, any empty JSON object will cause "ValueError: Extra data: [...]".
msg183604 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-03-06 16:45
Both the Python version and the C version are supposed to be subject to the same tests (at least in Python3, I don't remember the state of things in 2.7), so this probably indicates there is a missing test for this case.  Do you have any interest in trying your hand at writing one?  The tests are in Lib/test/json_tests in Python3.
msg184007 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-12 05:05
Here's a patch.
msg184027 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-03-12 14:53
LGTM. Perhaps with object_pairs_hook=tuple or object_pairs_hook=dict this test will look simpler.
msg184060 - (view) Author: Roundup Robot (python-dev) Date: 2013-03-12 23:55
New changeset 4a5ad099b176 by Ezio Melotti in branch '2.7':
#17368: Fix an off-by-one error in the Python JSON decoder that caused a failure while decoding empty object literals when object_pairs_hook was specified.
http://hg.python.org/cpython/rev/4a5ad099b176

New changeset 40c36d873f41 by Ezio Melotti in branch '3.2':
#17368: Fix an off-by-one error in the Python JSON decoder that caused a failure while decoding empty object literals when object_pairs_hook was specified.
http://hg.python.org/cpython/rev/40c36d873f41

New changeset dd2fad6b47c0 by Ezio Melotti in branch '3.3':
#17368: merge with 3.2.
http://hg.python.org/cpython/rev/dd2fad6b47c0

New changeset 885ffda21849 by Ezio Melotti in branch 'default':
#17368: merge with 3.3.
http://hg.python.org/cpython/rev/885ffda21849
msg184062 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-12 23:57
Fixed.  I used OrderedDict to make sure that the object hook is actually used.
History
Date User Action Args
2013-03-12 23:57:19ezio.melottisetstatus: open -> closed
resolution: fixed
messages: + msg184062

stage: patch review -> resolved
2013-03-12 23:55:22python-devsetnosy: + python-dev
messages: + msg184060
2013-03-12 14:53:07serhiy.storchakasetmessages: + msg184027
2013-03-12 05:05:36ezio.melottisetfiles: + issue17368.diff
messages: + msg184007

assignee: ezio.melotti
keywords: + patch
stage: test needed -> patch review
2013-03-06 16:47:51ezio.melottisetnosy: + ezio.melotti, serhiy.storchaka
stage: test needed
type: behavior

versions: - Python 3.1, Python 3.5
2013-03-06 16:45:12r.david.murraysetnosy: + r.david.murray
messages: + msg183604
2013-03-06 16:31:45Kuukunencreate