json.load failure when C optimizations aren't built #53479

freddrake · 2010-07-12T15:47:44Z

BPO	9233
Nosy	@freddrake, @terryjreedy, @etrepum, @ezio-melotti, @alex
Files	test-empty-json-object.diff: test that empty objects are converted correctly issue9233.diff: Patch to skip the tests when _json is not available (2.7)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ezio-melotti'
closed_at = <Date 2011-04-13.04:25:47.408>
created_at = <Date 2010-07-12.15:47:44.428>
labels = ['type-bug', 'library']
title = "json.load failure when C optimizations aren't built"
updated_at = <Date 2011-04-13.04:25:47.405>
user = 'https://github.com/freddrake'

bugs.python.org fields:

activity = <Date 2011-04-13.04:25:47.405>
actor = 'ezio.melotti'
assignee = 'ezio.melotti'
closed = True
closed_date = <Date 2011-04-13.04:25:47.408>
closer = 'ezio.melotti'
components = ['Library (Lib)']
creation = <Date 2010-07-12.15:47:44.428>
creator = 'fdrake'
dependencies = []
files = ['17966', '21628']
hgrepos = []
issue_num = 9233
keywords = ['patch']
message_count = 10.0
messages = ['110099', '110111', '127166', '127355', '133574', '133579', '133597', '133639', '133641', '133642']
nosy_count = 7.0
nosy_names = ['fdrake', 'terry.reedy', 'bob.ippolito', 'dstanek', 'ezio.melotti', 'alex', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue9233'
versions = ['Python 3.1', 'Python 2.7', 'Python 3.2', 'Python 3.3']

freddrake · 2010-07-12T15:47:43Z

The json decoder doesn't pass tests when the C optimizations in the _json module aren't available. The actual test failures appear fairly superficial, but aren't the limit of the problems.

Empty objects are converted to empty lists instead of empty dictionaries when _json isn't available; test attached.

bpo-5723 notes that the pure-Python implementation isn't tested; fixing that might have avoided these problems.

Test failures from Python 2.6.5:

======================================================================
ERROR: test_c_encode_basestring_ascii (json.tests.test_encode_basestring_ascii.TestEncodeBaseStringAscii)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_encode_basestring_ascii.py", line 29, in test_c_encode_basestring_ascii
    self._test_encode_basestring_ascii(json.encoder.c_encode_basestring_ascii)
  File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_encode_basestring_ascii.py", line 32, in _test_encode_basestring_ascii
    fname = encode_basestring_ascii.__name__
AttributeError: 'NoneType' object has no attribute '__name__'

======================================================================
ERROR: test_c_scanstring (json.tests.test_scanstring.TestScanString)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_scanstring.py", line 13, in test_c_scanstring
    self._test_scanstring(json.decoder.c_scanstring)
  File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_scanstring.py", line 17, in _test_scanstring
    scanstring('"z\\ud834\\udd20x"', 1, None, True),
TypeError: 'NoneType' object is not callable

======================================================================
FAIL: test_encode_basestring_ascii (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_speedups.py", line 13, in test_encode_basestring_ascii
    self.assertEquals(encoder.encode_basestring_ascii.__module__, "_json")
AssertionError: 'json.encoder' != '_json'

======================================================================
FAIL: test_scanstring (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/fdrake/src/Python-2.6.5/Lib/json/tests/test_speedups.py", line 9, in test_scanstring
    self.assertEquals(decoder.scanstring.__module__, "_json")
AssertionError: 'json.decoder' != '_json'

Test failures from 2.7:

======================================================================
ERROR: test_c_scanstring (json.tests.test_scanstring.TestScanString)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_scanstring.py", line 13, in test_c_scanstring
    self._test_scanstring(json.decoder.c_scanstring)
  File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_scanstring.py", line 17, in _test_scanstring
    scanstring('"z\\ud834\\udd20x"', 1, None, True),
TypeError: 'NoneType' object is not callable

======================================================================
FAIL: test_encode_basestring_ascii (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_speedups.py", line 12, in test_encode_basestring_ascii
    self.assertEquals(encoder.encode_basestring_ascii.__module__, "_json")
AssertionError: 'json.encoder' != '_json'

======================================================================
FAIL: test_scanstring (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/fdrake/src/Python-2.7/Lib/json/tests/test_speedups.py", line 8, in test_scanstring
    self.assertEquals(decoder.scanstring.__module__, "_json")
AssertionError: 'json.decoder' != '_json'

freddrake · 2010-07-12T16:51:36Z

The attached test only fails for Python 2.7; removing Python 2.6 from this issue.

etrepum · 2011-01-27T00:48:51Z

For what it's worth, I don't think any of the failures mentioned here are in the current version of simplejson. There aren't any tests that check the speedup module name, the speedup tests are skipped when speedups are not detected, and the empty json object bug was fixed in June 2009.

http://code.google.com/p/simplejson/issues/detail?id=57

alex · 2011-01-28T22:03:23Z

Just a note from downstream with PyPy: we've cherry-picked the commit Bob linked into our copy of the 2.7 stdlib since we don't have an _json (yet).

ezio-melotti · 2011-04-12T12:39:29Z

I ran the tests again on 2.7 and got 3 failures:

======================================================================
ERROR: test_c_scanstring (json.tests.test_scanstring.TestScanString)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_scanstring.py", line 13, in test_c_scanstring
    self._test_scanstring(json.decoder.c_scanstring)
  File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_scanstring.py", line 17, in _test_scanstring
    scanstring('"z\\ud834\\udd20x"', 1, None, True),
TypeError: 'NoneType' object is not callable

======================================================================
FAIL: test_encode_basestring_ascii (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_speedups.py", line 14, in test_encode_basestring_ascii
    self.assertEqual(encoder.encode_basestring_ascii.__module__, "_json")
AssertionError: 'json.encoder' != '_json'

======================================================================
FAIL: test_scanstring (json.tests.test_speedups.TestSpeedups)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/wolf/dev/py/issue9233/Lib/json/tests/test_speedups.py", line 11, in test_scanstring
    self.assertTrue(decoder.scanstring is decoder.c_scanstring)
AssertionError: False is not true

The attached patch checks if _json is importable and skip the tests if it's not.

python-dev · 2011-04-12T13:07:29Z

New changeset 500063f6ae5a by Ezio Melotti in branch '2.7':
bpo-9233: skip _json-specific tests when _json is not available.
http://hg.python.org/cpython/rev/500063f6ae5a

terryjreedy · 2011-04-12T18:59:14Z

Do those new check finish this issue and should it be closed?
If not, what is left?

python-dev · 2011-04-13T02:38:17Z

New changeset d58c63ff5bb2 by Ezio Melotti in branch '2.7':
bpo-9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
http://hg.python.org/cpython/rev/d58c63ff5bb2

python-dev · 2011-04-13T04:21:52Z

New changeset 36e676a8541d by Ezio Melotti in branch '3.1':
bpo-9233: skip _json-specific tests when _json is not available.
http://hg.python.org/cpython/rev/36e676a8541d

New changeset 7019fc1a9663 by Ezio Melotti in branch '3.1':
bpo-9233: Fix json to work properly even when _json is not available.
http://hg.python.org/cpython/rev/7019fc1a9663

New changeset a220458179ed by Ezio Melotti in branch '3.1':
bpo-9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
http://hg.python.org/cpython/rev/a220458179ed

ezio-melotti · 2011-04-13T04:25:47Z

Fixed in 2.7 and 3.1 and ported to 3.2 (b279611146d7) and 3.3 (e8e3f2b72a32).
In 3.1 json was completely broken without _json, so I fixed that too.
I also did some minor cleanup in ec6d881f5b02.

freddrake added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Jul 12, 2010

ezio-melotti self-assigned this Apr 12, 2011

ezio-melotti closed this as completed Apr 13, 2011

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

json.load failure when C optimizations aren't built #53479

json.load failure when C optimizations aren't built #53479

freddrake commented Jul 12, 2010

freddrake commented Jul 12, 2010

freddrake commented Jul 12, 2010

etrepum mannequin commented Jan 27, 2011

alex commented Jan 28, 2011

ezio-melotti commented Apr 12, 2011

python-dev mannequin commented Apr 12, 2011

terryjreedy commented Apr 12, 2011

python-dev mannequin commented Apr 13, 2011

python-dev mannequin commented Apr 13, 2011

ezio-melotti commented Apr 13, 2011

json.load failure when C optimizations aren't built #53479

json.load failure when C optimizations aren't built #53479

Comments

freddrake commented Jul 12, 2010

freddrake commented Jul 12, 2010

freddrake commented Jul 12, 2010

etrepum mannequin commented Jan 27, 2011

alex commented Jan 28, 2011

ezio-melotti commented Apr 12, 2011

python-dev mannequin commented Apr 12, 2011

terryjreedy commented Apr 12, 2011

python-dev mannequin commented Apr 13, 2011

python-dev mannequin commented Apr 13, 2011

ezio-melotti commented Apr 13, 2011