Author vstinner
Recipients jstasiak, larry, rhettinger, serhiy.storchaka, vstinner, yselivanov
Date 2016-05-25.14:05:18
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1464185119.8.0.0765486321732.issue26814@psf.upfronthosting.co.za>
In-reply-to
Content
I fixed even more issues with my setup to run benchmark. Results should be even more reliable. Moreover, I fixed multiple reference leaks in the code which introduced performance regressions. I started to write articles to explain how to run stable benchmarks:

* https://haypo.github.io/journey-to-stable-benchmark-system.html
* https://haypo.github.io/journey-to-stable-benchmark-deadcode.html
* https://haypo.github.io/journey-to-stable-benchmark-average.html

Summary of benchmarks at the revision e6f3bf996c01:

Faster (25):
- pickle_list: 1.29x faster
- etree_generate: 1.22x faster
- pickle_dict: 1.19x faster
- etree_process: 1.16x faster
- mako_v2: 1.13x faster
- telco: 1.09x faster
- raytrace: 1.08x faster
- etree_iterparse: 1.08x faster
- regex_compile: 1.07x faster
- json_dump_v2: 1.07x faster
- etree_parse: 1.06x faster
- regex_v8: 1.05x faster
- call_method_unknown: 1.05x faster
- chameleon_v2: 1.05x faster
- fastunpickle: 1.04x faster
- django_v3: 1.04x faster
- chaos: 1.04x faster
- 2to3: 1.03x faster
- pathlib: 1.03x faster
- unpickle_list: 1.03x faster
- json_load: 1.03x faster
- fannkuch: 1.03x faster
- call_method: 1.02x faster
- unpack_sequence: 1.02x faster
- call_method_slots: 1.02x faster

Slower (4):
- regex_effbot: 1.08x slower
- nbody: 1.08x slower
- spectral_norm: 1.07x slower
- nqueens: 1.06x slower

Not significat (13):
- tornado_http
- startup_nosite
- simple_logging
- silent_logging
- richards
- pidigits
- normal_startup
- meteor_contest
- go
- formatted_logging
- float
- fastpickle
- call_simple

I'm now investigating why 4 benchmarks are slower.

Note: I'm still using my patched CPython benchmark suite to get more stable benchmark. I will send patches upstream later.
History
Date User Action Args
2016-05-25 14:05:19vstinnersetrecipients: + vstinner, rhettinger, larry, serhiy.storchaka, yselivanov, jstasiak
2016-05-25 14:05:19vstinnersetmessageid: <1464185119.8.0.0765486321732.issue26814@psf.upfronthosting.co.za>
2016-05-25 14:05:19vstinnerlinkissue26814 messages
2016-05-25 14:05:18vstinnercreate