This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients vstinner
Date 2016-11-05.09:07:43
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1478336865.71.0.400107417809.issue28618@psf.upfronthosting.co.za>
In-reply-to
Content
I ran benchmarks. Globally, it seems like the impact of the patch is positive. regex_v8 and call_simple are slower, but these benchmarks are microbenchmarks impacted by low level stuff like CPU L1 cache. Well, my patch was supposed to optimize CPython for call_simple :-/ I should maybe investigate a little bit more.


Performance comparison (performance 0.3.2):

haypo@smithers$ python3 -m perf compare_to orig.json hot.json -G
Slower (6):
- regex_v8: 40.6 ms +- 5.7 ms -> 47.1 ms +- 0.3 ms: 1.16x slower
- call_simple: 12.6 ms +- 0.2 ms -> 13.2 ms +- 1.3 ms: 1.05x slower
- regex_effbot: 4.58 ms +- 0.07 ms -> 4.70 ms +- 0.05 ms: 1.03x slower
- sympy_integrate: 43.4 ms +- 0.3 ms -> 44.0 ms +- 0.2 ms: 1.01x slower
- nqueens: 239 ms +- 2 ms -> 241 ms +- 1 ms: 1.01x slower
- scimark_fft: 674 ms +- 12 ms -> 680 ms +- 75 ms: 1.01x slower

Faster (32):
- scimark_monte_carlo: 255 ms +- 4 ms -> 234 ms +- 7 ms: 1.09x faster
- chameleon: 28.4 ms +- 3.1 ms -> 27.0 ms +- 0.4 ms: 1.05x faster
- scimark_sor: 488 ms +- 27 ms -> 467 ms +- 10 ms: 1.05x faster
- sqlite_synth: 9.16 us +- 1.03 us -> 8.82 us +- 0.23 us: 1.04x faster
- scimark_lu: 485 ms +- 20 ms -> 469 ms +- 14 ms: 1.03x faster
- xml_etree_process: 226 ms +- 30 ms -> 219 ms +- 4 ms: 1.03x faster
- logging_simple: 29.7 us +- 0.4 us -> 28.9 us +- 0.3 us: 1.03x faster
- pickle_list: 7.99 us +- 0.88 us -> 7.78 us +- 0.05 us: 1.03x faster
- raytrace: 1.26 sec +- 0.08 sec -> 1.23 sec +- 0.01 sec: 1.03x faster
- sympy_expand: 995 ms +- 31 ms -> 971 ms +- 35 ms: 1.03x faster
- deltablue: 17.0 ms +- 0.1 ms -> 16.6 ms +- 0.2 ms: 1.02x faster
- call_method_slots: 16.0 ms +- 0.1 ms -> 15.6 ms +- 0.2 ms: 1.02x faster
- fannkuch: 983 ms +- 12 ms -> 962 ms +- 29 ms: 1.02x faster
- pickle_pure_python: 1.25 ms +- 0.14 ms -> 1.22 ms +- 0.01 ms: 1.02x faster
- logging_format: 34.0 us +- 0.3 us -> 33.4 us +- 1.5 us: 1.02x faster
- xml_etree_parse: 274 ms +- 9 ms -> 270 ms +- 5 ms: 1.02x faster
- sympy_str: 441 ms +- 3 ms -> 433 ms +- 3 ms: 1.02x faster
- genshi_text: 87.6 ms +- 9.2 ms -> 86.0 ms +- 1.4 ms: 1.02x faster
- genshi_xml: 187 ms +- 17 ms -> 184 ms +- 1 ms: 1.02x faster
- django_template: 376 ms +- 4 ms -> 370 ms +- 2 ms: 1.02x faster
- json_dumps: 27.1 ms +- 0.4 ms -> 26.7 ms +- 0.4 ms: 1.02x faster
- sqlalchemy_declarative: 295 ms +- 3 ms -> 291 ms +- 3 ms: 1.01x faster
- call_method_unknown: 18.1 ms +- 0.1 ms -> 17.8 ms +- 0.1 ms: 1.01x faster
- nbody: 218 ms +- 4 ms -> 216 ms +- 2 ms: 1.01x faster
- regex_dna: 250 ms +- 24 ms -> 247 ms +- 2 ms: 1.01x faster
- go: 573 ms +- 2 ms -> 566 ms +- 3 ms: 1.01x faster
- richards: 173 ms +- 4 ms -> 171 ms +- 4 ms: 1.01x faster
- python_startup: 24.6 ms +- 0.1 ms -> 24.5 ms +- 0.1 ms: 1.00x faster
- regex_compile: 404 ms +- 6 ms -> 403 ms +- 5 ms: 1.00x faster
- dulwich_log: 143 ms +- 11 ms -> 143 ms +- 1 ms: 1.00x faster
- pidigits: 290 ms +- 1 ms -> 289 ms +- 0 ms: 1.00x faster
- pickle_dict: 58.3 us +- 6.5 us -> 58.3 us +- 0.7 us: 1.00x faster

Benchmark hidden because not significant (26): 2to3, call_method, chaos, crypto_pyaes, float, hexiom, html5lib, json_loads, logging_silent, mako, meteor_contest, pathlib, pickle, python_startup_no_site, scimark_sparse_mat_mult, spectral_norm, sqlalchemy_imperative, sympy_sum, telco, tornado_http, unpack_sequence, unpickle, unpickle_list, unpickle_pure_python, xml_etree_generate, xml_etree_iterparse

--

More readable output, only display differences >= 5%:

haypo@smithers$ python3 -m perf compare_to orig.json hot.json -G --min-speed=5
Slower (1):
- regex_v8: 40.6 ms +- 5.7 ms -> 47.1 ms +- 0.3 ms: 1.16x slower

Faster (2):
- scimark_monte_carlo: 255 ms +- 4 ms -> 234 ms +- 7 ms: 1.09x faster
- chameleon: 28.4 ms +- 3.1 ms -> 27.0 ms +- 0.4 ms: 1.05x faster

Benchmark hidden because not significant (61): 2to3, call_method, call_method_slots, call_method_unknown, call_simple, chaos, crypto_pyaes, deltablue, django_template, dulwich_log, fannkuch, float, genshi_text, genshi_xml, go, hexiom, html5lib, json_dumps, json_loads, logging_format, logging_silent, logging_simple, mako, meteor_contest, nbody, nqueens, pathlib, pickle, pickle_dict, pickle_list, pickle_pure_python, pidigits, python_startup, python_startup_no_site, raytrace, regex_compile, regex_dna, regex_effbot, richards, scimark_fft, scimark_lu, scimark_sor, scimark_sparse_mat_mult, spectral_norm, sqlalchemy_declarative, sqlalchemy_imperative, sqlite_synth, sympy_expand, sympy_integrate, sympy_str, sympy_sum, telco, tornado_http, unpack_sequence, unpickle, unpickle_list, unpickle_pure_python, xml_etree_generate, xml_etree_iterparse, xml_etree_parse, xml_etree_process
History
Date User Action Args
2016-11-05 09:07:45vstinnersetrecipients: + vstinner
2016-11-05 09:07:45vstinnersetmessageid: <1478336865.71.0.400107417809.issue28618@psf.upfronthosting.co.za>
2016-11-05 09:07:45vstinnerlinkissue28618 messages
2016-11-05 09:07:43vstinnercreate