classification
Title: Optimize parsing of JSON numbers
Type: performance Stage: resolved
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: ezio.melotti, jcea, mark.dickinson, pitrou, python-dev, rhettinger
Priority: normal Keywords: patch

Created on 2011-04-16 12:19 by pitrou, last changed 2011-04-25 17:16 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
jsonnumbers.patch pitrou, 2011-04-16 12:19 review
jsonnumbers2.patch pitrou, 2011-04-16 17:30 review
Messages (4)
msg133891 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-04-16 12:19
In Python 3.x, parsing JSON numbers involve calling PyLong_FromUnicode or PyFloat_FromString with an unicode object. These functions are quite costly because they call PyUnicode_TransformDecimalToASCII(). But JSON numbers are always pure ASCII. This patch does the ASCII conversion ourselves.

Small benchmark with integers:
./python -m timeit -s \
  "from json import loads, dumps; d=list(i for i in range(1000)); s=dumps(d)" \
  "loads(s)"

-> without patch: 705 usec per loop
-> with patch: 103 usec per loop
msg133899 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-04-16 17:30
Cleaned up patch.
msg134378 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2011-04-25 11:21
Patch seems OK. Please, commit.
msg134396 - (view) Author: Roundup Robot (python-dev) Date: 2011-04-25 17:16
New changeset d60f9d9983bb by Antoine Pitrou in branch 'default':
Issue #11856: Speed up parsing of JSON numbers.
http://hg.python.org/cpython/rev/d60f9d9983bb
History
Date User Action Args
2011-04-25 17:16:39pitrousetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2011-04-25 17:16:20python-devsetnosy: + python-dev
messages: + msg134396
2011-04-25 11:21:25jceasetnosy: + jcea
messages: + msg134378
2011-04-16 17:30:25pitrousetfiles: + jsonnumbers2.patch

messages: + msg133899
2011-04-16 12:20:02ezio.melottisetnosy: + ezio.melotti
2011-04-16 12:19:20pitroucreate