classification
Title: test_json crash on memory allocation failure
Type: crash Stage:
Components: Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: python-dev, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2014-09-04 23:27 by vstinner, last changed 2014-09-10 21:37 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
json.patch vstinner, 2014-09-04 23:27 review
Messages (4)
msg226390 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-04 23:27
Using pyfailmalloc, I'm able to reproduce the crash seen on a buildbot.

Attached patch fixes two bugs in error handlers.

http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/8557/steps/test/logs/stdio

[191/390] test_urllib2net
Fatal Python error: Segmentation fault

Current thread 0x0000000000000001 (most recent call first):
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line[192/390] test_json
 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 420 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 430 in _iterencode
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/json/encoder.py", line 317 in _iterencode_list
  ...
Traceback (most recent call last):
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/test/__main__.py", line 3, in <module>
    regrtest.main_in_temp_cwd()
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/test/regrtest.py", line 1564, in main_in_temp_cwd
    main()
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/test/regrtest.py", line 738, in main
    raise Exception("Child error on {}: {}".format(test, result[1]))
Exception: Child error on test_json: Exit code -11
make: *** [buildbottest] Error 1
program finished with exit code 2
msg226636 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-09 14:45
Besides my nitpick on Rietveld the patch LGTM.
msg226724 - (view) Author: Roundup Robot (python-dev) Date: 2014-09-10 21:34
New changeset 3ac9f9576ce6 by Victor Stinner in branch '3.4':
Issue #22338: Fix a crash in the json module on memory allocation failure.
http://hg.python.org/cpython/rev/3ac9f9576ce6

New changeset 135fc23e475c by Victor Stinner in branch 'default':
(Merge 3.4) Issue #22338: Fix a crash in the json module on memory allocation
http://hg.python.org/cpython/rev/135fc23e475c
msg226726 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-09-10 21:37
> Modules/_json.c:1558: item = NULL;
> You can move it outside of the loop.

I prefer to set item to NULL just after PyList_SET_ITEM() to make it explicit that the list now owns the reference.

> But may be it will be better to make this "item" variable local.

"item" is also used below, and I don't want to have two "item" variables.

I commited my to Python 3.4 & 3.5.

Python 2.7 doesn't look to be affected, chunk and item are not cleared (Py_DECREF) in the error handler ("bail:" label).

Thanks for the review.
History
Date User Action Args
2014-09-10 21:37:27vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg226726

versions: - Python 2.7
2014-09-10 21:34:53python-devsetnosy: + python-dev
messages: + msg226724
2014-09-09 14:45:35serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg226636
2014-09-04 23:27:29vstinnersettype: crash
versions: + Python 2.7, Python 3.4, Python 3.5
2014-09-04 23:27:17vstinnercreate