Author haypo
Recipients Demur Rumed, abarnert, benjamin.peterson, brett.cannon, georg.brandl, haypo, josh.r, ncoghlan, python-dev, rhettinger, serhiy.storchaka, yselivanov
Date 2016-04-13.16:23:44
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1460564626.0.0.816679496508.issue26647@psf.upfronthosting.co.za>
In-reply-to
Content
I ran the Python benchmark suite on wpy6.patch.

* My platform: Linux-4.4.4-301.fc23.x86_64-x86_64-with-fedora-23-Twenty_Three
* My PC: CPU Intel i7-2600 (~2.9 GHz) with 12 GB of RAM
* Benchmark ran on isolated CPU: http://haypo-notes.readthedocs.org/microbenchmark.html
* Command line: ~/bin/taskset_isolated.py time python3 -u perf.py --rigorous "$ORIG_PYTHON" "$PATCHED_PYTHON" -b all 2>&1

It looks like we get more faster benchmarks than slower benchamrks. Faster is up to 11% faster, whereas the worst slowdown is only 4%. The overall results look good to me.

Slower:

* fannkuch: 1.04x slower
* pickle_dict: 1.04x slower
* telco: 1.03x slower
* django_v3: 1.02x slower
* simple_logging: 1.02x slower
* meteor_contest: 1.02x slower

Faster:

* unpack_sequence: 1.11x faster
* etree_parse: 1.06x faster
* call_method_slots: 1.06x faster
* etree_iterparse: 1.05x faster
* call_simple: 1.04x faster
* nbody: 1.04x faster
* float: 1.04x faster
* call_method_unknown: 1.03x faster
* call_method: 1.03x faster
* chaos: 1.03x faster
* mako_v2: 1.03x faster
* richards: 1.02x faster
* silent_logging1: 1.02x faster


Full Output:

Original python: ../wordcode/python
3.6.0a0 (default:ad5b079565ad, Apr 13 2016, 16:30:36) 
[GCC 5.3.1 20151207 (Red Hat 5.3.1-2)]

Patched python: ../wordcode/python
3.6.0a0 (default:c050d203e82b, Apr 13 2016, 16:30:24) 
[GCC 5.3.1 20151207 (Red Hat 5.3.1-2)]

INFO:root:Automatically selected timer: perf_counter
INFO:root:Skipping benchmark slowpickle; not compatible with Python 3.6
INFO:root:Skipping benchmark pybench; not compatible with Python 3.6
INFO:root:Skipping benchmark hg_startup; not compatible with Python 3.6
INFO:root:Skipping benchmark rietveld; not compatible with Python 3.6
INFO:root:Skipping benchmark slowspitfire; not compatible with Python 3.6
INFO:root:Skipping benchmark bzr_startup; not compatible with Python 3.6
INFO:root:Skipping benchmark html5lib_warmup; not compatible with Python 3.6
INFO:root:Skipping benchmark slowunpickle; not compatible with Python 3.6
INFO:root:Skipping benchmark html5lib; not compatible with Python 3.6
INFO:root:Skipping benchmark spambayes; not compatible with Python 3.6
[ 1/43] 2to3...
INFO:root:Running `../wordcode/python lib3/2to3/2to3 -f all lib/2to3`
INFO:root:Running `../wordcode/python lib3/2to3/2to3 -f all lib/2to3` 5 times
INFO:root:Running `../default/python lib3/2to3/2to3 -f all lib/2to3`
INFO:root:Running `../default/python lib3/2to3/2to3 -f all lib/2to3` 5 times
[ 2/43] call_method...
INFO:root:Running `../wordcode/python performance/bm_call_method.py -n 300 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_call_method.py -n 300 --timer perf_counter`
mer. avril 13 16:36:47 CEST 2016
Original python: ../wordcode/python
3.6.0a0 (default:ad5b079565ad, Apr 13 2016, 16:30:36) 
[GCC 5.3.1 20151207 (Red Hat 5.3.1-2)]

Patched python: ../wordcode/python
3.6.0a0 (default:c050d203e82b, Apr 13 2016, 16:30:24) 
[GCC 5.3.1 20151207 (Red Hat 5.3.1-2)]

INFO:root:Automatically selected timer: perf_counter
INFO:root:Skipping benchmark html5lib; not compatible with Python 3.6
INFO:root:Skipping benchmark html5lib_warmup; not compatible with Python 3.6
INFO:root:Skipping benchmark slowpickle; not compatible with Python 3.6
INFO:root:Skipping benchmark slowunpickle; not compatible with Python 3.6
INFO:root:Skipping benchmark slowspitfire; not compatible with Python 3.6
INFO:root:Skipping benchmark rietveld; not compatible with Python 3.6
INFO:root:Skipping benchmark bzr_startup; not compatible with Python 3.6
INFO:root:Skipping benchmark spambayes; not compatible with Python 3.6
INFO:root:Skipping benchmark pybench; not compatible with Python 3.6
INFO:root:Skipping benchmark hg_startup; not compatible with Python 3.6
[ 1/43] 2to3...
INFO:root:Running `../wordcode/python lib3/2to3/2to3 -f all lib/2to3`
INFO:root:Running `../wordcode/python lib3/2to3/2to3 -f all lib/2to3` 5 times
INFO:root:Running `../default/python lib3/2to3/2to3 -f all lib/2to3`
INFO:root:Running `../default/python lib3/2to3/2to3 -f all lib/2to3` 5 times
[ 2/43] call_method...
INFO:root:Running `../wordcode/python performance/bm_call_method.py -n 300 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_call_method.py -n 300 --timer perf_counter`
[ 3/43] call_method_slots...
INFO:root:Running `../wordcode/python performance/bm_call_method_slots.py -n 300 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_call_method_slots.py -n 300 --timer perf_counter`
[ 4/43] call_method_unknown...
INFO:root:Running `../wordcode/python performance/bm_call_method_unknown.py -n 300 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_call_method_unknown.py -n 300 --timer perf_counter`
[ 5/43] call_simple...
INFO:root:Running `../wordcode/python performance/bm_call_simple.py -n 300 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_call_simple.py -n 300 --timer perf_counter`
[ 6/43] chameleon_v2...
INFO:root:Running `../wordcode/python performance/bm_chameleon_v2.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_chameleon_v2.py -n 100 --timer perf_counter`
[ 7/43] chaos...
INFO:root:Running `../wordcode/python performance/bm_chaos.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_chaos.py -n 100 --timer perf_counter`
[ 8/43] django_v3...
INFO:root:Running `../wordcode/python performance/bm_django_v3.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_django_v3.py -n 100 --timer perf_counter`
[ 9/43] etree_generate...
INFO:root:Running `../wordcode/python performance/bm_elementtree.py -n 100 --timer perf_counter generate`
INFO:root:Running `../default/python performance/bm_elementtree.py -n 100 --timer perf_counter generate`
[10/43] etree_iterparse...
INFO:root:Running `../wordcode/python performance/bm_elementtree.py -n 100 --timer perf_counter iterparse`
INFO:root:Running `../default/python performance/bm_elementtree.py -n 100 --timer perf_counter iterparse`
[11/43] etree_parse...
INFO:root:Running `../wordcode/python performance/bm_elementtree.py -n 100 --timer perf_counter parse`
INFO:root:Running `../default/python performance/bm_elementtree.py -n 100 --timer perf_counter parse`
[12/43] etree_process...
INFO:root:Running `../wordcode/python performance/bm_elementtree.py -n 100 --timer perf_counter process`
INFO:root:Running `../default/python performance/bm_elementtree.py -n 100 --timer perf_counter process`
[13/43] fannkuch...
INFO:root:Running `../wordcode/python performance/bm_fannkuch.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_fannkuch.py -n 100 --timer perf_counter`
[14/43] fastpickle...
INFO:root:Running `../wordcode/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle pickle`
INFO:root:Running `../default/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle pickle`
[15/43] fastunpickle...
INFO:root:Running `../wordcode/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle unpickle`
INFO:root:Running `../default/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle unpickle`
[16/43] float...
INFO:root:Running `../wordcode/python performance/bm_float.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_float.py -n 100 --timer perf_counter`
[17/43] formatted_logging...
INFO:root:Running `../wordcode/python performance/bm_logging.py -n 100 --timer perf_counter formatted_output`
INFO:root:Running `../default/python performance/bm_logging.py -n 100 --timer perf_counter formatted_output`
[18/43] go...
INFO:root:Running `../wordcode/python performance/bm_go.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_go.py -n 100 --timer perf_counter`
[19/43] hexiom2...
INFO:root:Running `../wordcode/python performance/bm_hexiom2.py -n 4 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_hexiom2.py -n 4 --timer perf_counter`
[20/43] json_dump_v2...
INFO:root:Running `../wordcode/python performance/bm_json_v2.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_json_v2.py -n 100 --timer perf_counter`
[21/43] json_load...
INFO:root:Running `../wordcode/python performance/bm_json.py -n 100 --timer perf_counter json_load`
INFO:root:Running `../default/python performance/bm_json.py -n 100 --timer perf_counter json_load`
[22/43] mako_v2...
INFO:root:Running `../wordcode/python performance/bm_mako_v2.py -n 1000 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_mako_v2.py -n 1000 --timer perf_counter`
[23/43] meteor_contest...
INFO:root:Running `../wordcode/python performance/bm_meteor_contest.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_meteor_contest.py -n 100 --timer perf_counter`
[24/43] nbody...
INFO:root:Running `../wordcode/python performance/bm_nbody.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_nbody.py -n 100 --timer perf_counter`
[25/43] normal_startup...
INFO:root:Running `../wordcode/python -c ` 2000 times
INFO:root:Running `../default/python -c ` 2000 times
[26/43] nqueens...
INFO:root:Running `../wordcode/python performance/bm_nqueens.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_nqueens.py -n 100 --timer perf_counter`
[27/43] pathlib...
INFO:root:Running `../wordcode/python performance/bm_pathlib.py -n 1000 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_pathlib.py -n 1000 --timer perf_counter`
[28/43] pickle_dict...
INFO:root:Running `../wordcode/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle pickle_dict`
INFO:root:Running `../default/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle pickle_dict`
[29/43] pickle_list...
INFO:root:Running `../wordcode/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle pickle_list`
INFO:root:Running `../default/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle pickle_list`
[30/43] pidigits...
INFO:root:Running `../wordcode/python performance/bm_pidigits.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_pidigits.py -n 100 --timer perf_counter`
[31/43] raytrace...
INFO:root:Running `../wordcode/python performance/bm_raytrace.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_raytrace.py -n 100 --timer perf_counter`
[32/43] regex_compile...
INFO:root:Running `../wordcode/python performance/bm_regex_compile.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_regex_compile.py -n 100 --timer perf_counter`
[33/43] regex_effbot...
INFO:root:Running `../wordcode/python performance/bm_regex_effbot.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_regex_effbot.py -n 100 --timer perf_counter`
[34/43] regex_v8...
INFO:root:Running `../wordcode/python performance/bm_regex_v8.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_regex_v8.py -n 100 --timer perf_counter`
[35/43] richards...
INFO:root:Running `../wordcode/python performance/bm_richards.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_richards.py -n 100 --timer perf_counter`
[36/43] silent_logging...
INFO:root:Running `../wordcode/python performance/bm_logging.py -n 100 --timer perf_counter no_output`
INFO:root:Running `../default/python performance/bm_logging.py -n 100 --timer perf_counter no_output`
[37/43] simple_logging...
INFO:root:Running `../wordcode/python performance/bm_logging.py -n 100 --timer perf_counter simple_output`
INFO:root:Running `../default/python performance/bm_logging.py -n 100 --timer perf_counter simple_output`
[38/43] spectral_norm...
INFO:root:Running `../wordcode/python performance/bm_spectral_norm.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_spectral_norm.py -n 100 --timer perf_counter`
[39/43] startup_nosite...
INFO:root:Running `../wordcode/python -S -c ` 4000 times
INFO:root:Running `../default/python -S -c ` 4000 times
[40/43] telco...
INFO:root:Running `../wordcode/python performance/bm_telco.py -n 100 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_telco.py -n 100 --timer perf_counter`
[41/43] tornado_http...
INFO:root:Running `../wordcode/python performance/bm_tornado_http.py -n 200 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_tornado_http.py -n 200 --timer perf_counter`
[42/43] unpack_sequence...
INFO:root:Running `../wordcode/python performance/bm_unpack_sequence.py -n 100000 --timer perf_counter`
INFO:root:Running `../default/python performance/bm_unpack_sequence.py -n 100000 --timer perf_counter`
[43/43] unpickle_list...
INFO:root:Running `../wordcode/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle unpickle_list`
INFO:root:Running `../default/python performance/bm_pickle.py -n 100 --timer perf_counter --use_cpickle unpickle_list`

Report on Linux smithers 4.4.4-301.fc23.x86_64 #1 SMP Fri Mar 4 17:42:42 UTC 2016 x86_64 x86_64
Total CPU cores: 8

### call_method ###
Min: 0.313558 -> 0.304460: 1.03x faster
Avg: 0.313797 -> 0.304661: 1.03x faster
Significant (t=773.69)
Stddev: 0.00015 -> 0.00014: 1.1084x smaller

### call_method_slots ###
Min: 0.317374 -> 0.300388: 1.06x faster
Avg: 0.317527 -> 0.300701: 1.06x faster
Significant (t=1971.52)
Stddev: 0.00011 -> 0.00010: 1.0595x smaller

### call_method_unknown ###
Min: 0.309548 -> 0.301112: 1.03x faster
Avg: 0.309619 -> 0.301828: 1.03x faster
Significant (t=636.50)
Stddev: 0.00008 -> 0.00020: 2.3452x larger

### call_simple ###
Min: 0.245480 -> 0.235982: 1.04x faster
Avg: 0.246004 -> 0.236310: 1.04x faster
Significant (t=492.66)
Stddev: 0.00023 -> 0.00025: 1.1069x larger

### chaos ###
Min: 0.271012 -> 0.264204: 1.03x faster
Avg: 0.271723 -> 0.264787: 1.03x faster
Significant (t=132.15)
Stddev: 0.00044 -> 0.00028: 1.5564x smaller

### django_v3 ###
Min: 0.544071 -> 0.555346: 1.02x slower
Avg: 0.544697 -> 0.556142: 1.02x slower
Significant (t=-210.46)
Stddev: 0.00036 -> 0.00041: 1.1510x larger

### etree_iterparse ###
Min: 0.215644 -> 0.205198: 1.05x faster
Avg: 0.219440 -> 0.208423: 1.05x faster
Significant (t=53.95)
Stddev: 0.00145 -> 0.00144: 1.0016x smaller

### etree_parse ###
Min: 0.287245 -> 0.271355: 1.06x faster
Avg: 0.288902 -> 0.273051: 1.06x faster
Significant (t=107.60)
Stddev: 0.00106 -> 0.00102: 1.0348x smaller

### fannkuch ###
Min: 0.957137 -> 0.993462: 1.04x slower
Avg: 0.965306 -> 0.995223: 1.03x slower
Significant (t=-42.85)
Stddev: 0.00665 -> 0.00214: 3.1094x smaller

### float ###
Min: 0.258390 -> 0.248217: 1.04x faster
Avg: 0.265902 -> 0.255380: 1.04x faster
Significant (t=17.29)
Stddev: 0.00441 -> 0.00419: 1.0510x smaller

### mako_v2 ###
Min: 0.040757 -> 0.039408: 1.03x faster
Avg: 0.041534 -> 0.040058: 1.04x faster
Significant (t=106.39)
Stddev: 0.00033 -> 0.00029: 1.1548x smaller

### meteor_contest ###
Min: 0.187423 -> 0.192079: 1.02x slower
Avg: 0.188739 -> 0.193440: 1.02x slower
Significant (t=-61.30)
Stddev: 0.00053 -> 0.00056: 1.0503x larger

### nbody ###
Min: 0.227627 -> 0.219617: 1.04x faster
Avg: 0.229736 -> 0.221310: 1.04x faster
Significant (t=23.23)
Stddev: 0.00276 -> 0.00235: 1.1745x smaller

### pickle_dict ###
Min: 0.491946 -> 0.513859: 1.04x slower
Avg: 0.492796 -> 0.515723: 1.05x slower
Significant (t=-158.63)
Stddev: 0.00063 -> 0.00130: 2.0672x larger

### richards ###
Min: 0.159527 -> 0.155970: 1.02x faster
Avg: 0.160603 -> 0.157190: 1.02x faster
Significant (t=36.37)
Stddev: 0.00067 -> 0.00066: 1.0168x smaller

### silent_logging ###
Min: 0.068349 -> 0.067301: 1.02x faster
Avg: 0.069759 -> 0.067481: 1.03x faster
Significant (t=56.73)
Stddev: 0.00038 -> 0.00013: 2.8514x smaller

### simple_logging ###
Min: 0.276149 -> 0.282515: 1.02x slower
Avg: 0.277709 -> 0.283773: 1.02x slower
Significant (t=-53.60)
Stddev: 0.00080 -> 0.00080: 1.0045x smaller

### telco ###
Min: 0.011922 -> 0.012221: 1.03x slower
Avg: 0.011985 -> 0.012283: 1.02x slower
Significant (t=-59.48)
Stddev: 0.00003 -> 0.00004: 1.0912x larger

### unpack_sequence ###
Min: 0.000047 -> 0.000042: 1.11x faster
Avg: 0.000047 -> 0.000042: 1.10x faster
Significant (t=2242.55)
Stddev: 0.00000 -> 0.00000: 1.2134x larger

The following not significant results are hidden, use -v to show them:
2to3, chameleon_v2, etree_generate, etree_process, fastpickle, fastunpickle, formatted_logging, go, hexiom2, json_dump_v2, json_load, normal_startup, nqueens, pathlib, pickle_list, pidigits, raytrace, regex_compile, regex_effbot, regex_v8, spectral_norm, startup_nosite, tornado_http, unpickle_list.
History
Date User Action Args
2016-04-13 16:23:46hayposetrecipients: + haypo, brett.cannon, georg.brandl, rhettinger, ncoghlan, benjamin.peterson, python-dev, serhiy.storchaka, yselivanov, abarnert, josh.r, Demur Rumed
2016-04-13 16:23:46hayposetmessageid: <1460564626.0.0.816679496508.issue26647@psf.upfronthosting.co.za>
2016-04-13 16:23:45haypolinkissue26647 messages
2016-04-13 16:23:44haypocreate