Author swalker
Recipients bob.ippolito, pitrou, rhettinger, swalker, vkuznet
Date 2010-11-30.23:49:12
SpamBayes Score 1.70974e-14
Marked as misclassified No
Message-id <1291160956.0.0.397581539531.issue6594@psf.upfronthosting.co.za>
In-reply-to
Content
I specifically mentioned *SPARC* as the performance problem area, but the reply about "0.5s to dump" fails to mention on what platform they tested

My problem is not "undiagnosable".  I'll be happy to provide you with even more data files.  But I believe that there is a problem here on some architectures for reasons other than those of simple differences in single-threaded performance that could be accounted to processor architecture.

As an example of something that makes a noticeable difference on SPARC systems I've checked:

178     # could accelerate with writelines in some versions of Python, at
179     # a debuggability cost
180     for chunk in iterable:
181         fp.write(chunk)

Changing that to use writelines() is a significant win.  You go from over a million calls to write (for small bits as simple as a single character such as '{') to one single call to writelines() with an iterable.

The recursive call structure of the json code also adds significant overhead on some architectures.

What's "undiagnosable" here is the response to the issue I reported -- it provides no information about the platform that was tested or how the testing was done.

My testing was done by reading the attached file using json, and then timing the results of writing it back out (to /tmp mind you, which is memory-backed on my OS platform, so no disk I/O was involved, I've also checked writing to a cStringIO object).
History
Date User Action Args
2010-11-30 23:49:16swalkersetrecipients: + swalker, rhettinger, bob.ippolito, pitrou, vkuznet
2010-11-30 23:49:15swalkersetmessageid: <1291160956.0.0.397581539531.issue6594@psf.upfronthosting.co.za>
2010-11-30 23:49:12swalkerlinkissue6594 messages
2010-11-30 23:49:12swalkercreate