Benchmarks were run like this: python perf.py -r -b all python.orig python.add python perf.py -r -b all python.add python.orig python perf.py -r -b all python.orig python.subscr python perf.py -r -b all python.subscr python.orig python perf.py -r -b all python.orig python.both python perf.py -r -b all python.both python.orig So Slower results in the right-hand column are performance improvements ################################### Add Only ################################## ## Unmodified Reference ## ## Add Reference ## ### chameleon_v2 ### ### call_simple ### Min: 5.932982 -> 5.737039: 1.03x faster Min: 0.253518 -> 0.236977: 1.07x faster Avg: 6.003952 -> 5.794820: 1.04x faster Avg: 0.253937 -> 0.238456: 1.06x faster Significant (t=37.29) Significant (t=201.27) Stddev: 0.04320 -> 0.03576: 1.2079x smaller Stddev: 0.00040 -> 0.00127: 3.1436x larger ### chaos ### ### chameleon_v2 ### Min: 0.306899 -> 0.299243: 1.03x faster Min: 5.724054 -> 5.910359: 1.03x slower Avg: 0.308044 -> 0.300439: 1.03x faster Avg: 5.796646 -> 5.975152: 1.03x slower Significant (t=85.90) Significant (t=-28.93) Stddev: 0.00049 -> 0.00074: 1.5022x larger Stddev: 0.03796 -> 0.04865: 1.2814x larger ### django_v3 ### ### fannkuch ### Min: 0.579438 -> 0.559919: 1.03x faster Min: 1.018536 -> 1.065780: 1.05x slower Avg: 0.580751 -> 0.562284: 1.03x faster Avg: 1.027314 -> 1.071446: 1.04x slower Significant (t=107.02) Significant (t=-68.28) Stddev: 0.00063 -> 0.00161: 2.5400x larger Stddev: 0.00425 -> 0.00487: 1.1437x larger ### etree_generate ### ### nbody ### Min: 0.267283 -> 0.257613: 1.04x faster Min: 0.253961 -> 0.248638: 1.02x faster Avg: 0.269788 -> 0.260302: 1.04x faster Avg: 0.255299 -> 0.249911: 1.02x faster Significant (t=46.17) Significant (t=28.59) Stddev: 0.00147 -> 0.00144: 1.0203x smaller Stddev: 0.00124 -> 0.00142: 1.1484x larger ### etree_parse ### ### normal_startup ### Min: 0.275748 -> 0.284636: 1.03x slower Min: 0.384090 -> 0.384355: 1.00x slower Avg: 0.277660 -> 0.286598: 1.03x slower Avg: 0.432949 -> 0.446583: 1.03x slower Significant (t=-44.52) Significant (t=-2.08) Stddev: 0.00155 -> 0.00128: 1.2144x smaller Stddev: 0.04517 -> 0.04769: 1.0558x larger ### fannkuch ### ### nqueens ### Min: 1.087857 -> 1.001798: 1.09x faster Min: 0.285142 -> 0.300465: 1.05x slower Avg: 1.094680 -> 1.007361: 1.09x faster Avg: 0.286937 -> 0.301331: 1.05x slower Significant (t=127.98) Significant (t=-168.59) Stddev: 0.00470 -> 0.00494: 1.0507x larger Stddev: 0.00065 -> 0.00055: 1.1868x smaller ### formatted_logging ### ### pickle_dict ### Min: 0.345337 -> 0.333157: 1.04x faster Min: 0.533563 -> 0.518039: 1.03x faster Avg: 0.346936 -> 0.335312: 1.03x faster Avg: 0.534322 -> 0.518905: 1.03x faster Significant (t=51.99) Significant (t=203.03) Stddev: 0.00151 -> 0.00165: 1.0908x larger Stddev: 0.00053 -> 0.00054: 1.0092x larger ### go ### ### regex_compile ### Min: 0.559293 -> 0.547249: 1.02x faster Min: 0.332118 -> 0.341098: 1.03x slower Avg: 0.562876 -> 0.550922: 1.02x faster Avg: 0.333081 -> 0.342073: 1.03x slower Significant (t=45.72) Significant (t=-104.86) Stddev: 0.00173 -> 0.00196: 1.1330x larger Stddev: 0.00056 -> 0.00065: 1.1598x larger ### json_load ### ### regex_v8 ### Min: 0.498076 -> 0.454961: 1.09x faster Min: 0.044808 -> 0.043105: 1.04x faster Avg: 0.500558 -> 0.455969: 1.10x faster Avg: 0.047884 -> 0.043441: 1.10x faster Significant (t=460.39) Significant (t=18.47) Stddev: 0.00078 -> 0.00058: 1.3452x smaller Stddev: 0.00206 -> 0.00124: 1.6650x smaller ### nbody ### ### spectral_norm ### Min: 0.246193 -> 0.254998: 1.04x slower Min: 0.299112 -> 0.314866: 1.05x slower Avg: 0.247180 -> 0.256470: 1.04x slower Avg: 0.300035 -> 0.315608: 1.05x slower Significant (t=-54.61) Significant (t=-178.38) Stddev: 0.00053 -> 0.00162: 3.0265x larger Stddev: 0.00068 -> 0.00055: 1.2297x smaller ### normal_startup ### ### unpickle_list ### Min: 0.382287 -> 0.384400: 1.01x slower Min: 0.354309 -> 0.371177: 1.05x slower Avg: 0.423893 -> 0.437052: 1.03x slower Avg: 0.361087 -> 0.373878: 1.04x slower Significant (t=-2.24) Significant (t=-6.67) Stddev: 0.04050 -> 0.04273: 1.0550x larger Stddev: 0.01916 -> 0.00106: 18.0369x smaller ### pickle_dict ### Min: 0.517369 -> 0.533843: 1.03x slower Avg: 0.518309 -> 0.534904: 1.03x slower Significant (t=-195.09) Stddev: 0.00055 -> 0.00065: 1.1904x larger ### pickle_list ### Min: 0.291216 -> 0.309160: 1.06x slower Avg: 0.292597 -> 0.310389: 1.06x slower Significant (t=-142.86) Stddev: 0.00108 -> 0.00062: 1.7367x smaller ### regex_compile ### Min: 0.342482 -> 0.334542: 1.02x faster Avg: 0.343512 -> 0.335310: 1.02x faster Significant (t=96.04) Stddev: 0.00061 -> 0.00059: 1.0319x smaller ### regex_effbot ### Min: 0.049682 -> 0.051017: 1.03x slower Avg: 0.049992 -> 0.051169: 1.02x slower Significant (t=-59.59) Stddev: 0.00015 -> 0.00013: 1.1398x smaller ### simple_logging ### Min: 0.295348 -> 0.289460: 1.02x faster Avg: 0.296715 -> 0.290657: 1.02x faster Significant (t=50.25) Stddev: 0.00081 -> 0.00089: 1.1081x larger ### spectral_norm ### Min: 0.311990 -> 0.297074: 1.05x faster Avg: 0.313024 -> 0.297871: 1.05x faster Significant (t=130.72) Stddev: 0.00097 -> 0.00064: 1.5173x smaller ################################### Subscr Only ################################## ## Unmodified Reference ## ## Subscr Reference ## ### call_simple ### ### 2to3 ### Min: 0.237964 -> 0.243065: 1.02x slower Min: 6.712624 -> 6.787164: 1.01x slower Avg: 0.238617 -> 0.243508: 1.02x slower Avg: 6.753306 -> 6.910494: 1.02x slower Significant (t=-135.41) Significant (t=-3.72) Stddev: 0.00042 -> 0.00046: 1.0837x larger Stddev: 0.02583 -> 0.09076: 3.5137x larger ### chameleon_v2 ### ### call_method_slots ### Min: 5.950643 -> 5.786300: 1.03x faster Min: 0.317343 -> 0.329416: 1.04x slower Avg: 6.012696 -> 5.853764: 1.03x faster Avg: 0.317965 -> 0.330605: 1.04x slower Significant (t=37.17) Significant (t=-302.17) Stddev: 0.02968 -> 0.03078: 1.0371x larger Stddev: 0.00051 -> 0.00052: 1.0178x larger ### chaos ### ### chaos ### Min: 0.310509 -> 0.297379: 1.04x faster Min: 0.298868 -> 0.307761: 1.03x slower Avg: 0.311592 -> 0.298469: 1.04x faster Avg: 0.299778 -> 0.308685: 1.03x slower Significant (t=163.34) Significant (t=-122.23) Stddev: 0.00049 -> 0.00064: 1.3125x larger Stddev: 0.00053 -> 0.00050: 1.0719x smaller ### etree_generate ### ### etree_parse ### Min: 0.269616 -> 0.261554: 1.03x faster Min: 0.283054 -> 0.273544: 1.03x faster Avg: 0.272195 -> 0.263761: 1.03x faster Avg: 0.285265 -> 0.275266: 1.04x faster Significant (t=39.95) Significant (t=54.10) Stddev: 0.00151 -> 0.00148: 1.0202x smaller Stddev: 0.00146 -> 0.00114: 1.2831x smaller ### fannkuch ### ### fannkuch ### Min: 1.062697 -> 1.001734: 1.06x faster Min: 0.955886 -> 1.084675: 1.13x slower Avg: 1.069302 -> 1.005855: 1.06x faster Avg: 0.962642 -> 1.088885: 1.13x slower Significant (t=103.95) Significant (t=-201.26) Stddev: 0.00495 -> 0.00357: 1.3881x smaller Stddev: 0.00452 -> 0.00435: 1.0394x smaller ### fastpickle ### ### formatted_logging ### Min: 0.489767 -> 0.483485: 1.01x faster Min: 0.336150 -> 0.343521: 1.02x slower Avg: 0.519335 -> 0.498537: 1.04x faster Avg: 0.338146 -> 0.345628: 1.02x slower Significant (t=6.24) Significant (t=-31.92) Stddev: 0.03081 -> 0.01271: 2.4232x smaller Stddev: 0.00158 -> 0.00173: 1.0972x larger ### go ### ### go ### Min: 0.559769 -> 0.553110: 1.01x faster Min: 0.544887 -> 0.559069: 1.03x slower Avg: 0.564633 -> 0.582344: 1.03x slower Avg: 0.547762 -> 0.562693: 1.03x slower Significant (t=-5.76) Significant (t=-55.58) Stddev: 0.00228 -> 0.03064: 13.4185x larger Stddev: 0.00181 -> 0.00199: 1.1024x larger ### hexiom2 ### ### hexiom2 ### Min: 128.759568 -> 119.811011: 1.07x faster Min: 120.386410 -> 128.057144: 1.06x slower Avg: 128.884501 -> 119.994238: 1.07x faster Avg: 120.553872 -> 128.285037: 1.06x slower Significant (t=73.92) Significant (t=-43.68) Stddev: 0.13255 -> 0.20072: 1.5143x larger Stddev: 0.16415 -> 0.31360: 1.9104x larger ### json_load ### ### json_dump_v2 ### Min: 0.479604 -> 0.452895: 1.06x faster Min: 3.043143 -> 2.879975: 1.06x faster Avg: 0.480911 -> 0.454755: 1.06x faster Avg: 3.064296 -> 2.889900: 1.06x faster Significant (t=188.16) Significant (t=197.33) Stddev: 0.00096 -> 0.00100: 1.0415x larger Stddev: 0.00749 -> 0.00469: 1.5975x smaller ### mako_v2 ### ### mako_v2 ### Min: 0.040467 -> 0.037949: 1.07x faster Min: 0.035899 -> 0.039279: 1.09x slower Avg: 0.041417 -> 0.038700: 1.07x faster Avg: 0.036759 -> 0.040312: 1.10x slower Significant (t=163.89) Significant (t=-179.49) Stddev: 0.00037 -> 0.00037: 1.0162x smaller Stddev: 0.00043 -> 0.00046: 1.0786x larger ### meteor_contest ### ### meteor_contest ### Min: 0.208277 -> 0.201613: 1.03x faster Min: 0.207239 -> 0.208393: 1.01x slower Avg: 0.209481 -> 0.202763: 1.03x faster Avg: 0.208778 -> 0.213393: 1.02x slower Significant (t=74.29) Significant (t=-3.44) Stddev: 0.00065 -> 0.00063: 1.0236x smaller Stddev: 0.00064 -> 0.01341: 20.9855x larger ### nbody ### ### nbody ### Min: 0.240945 -> 0.226073: 1.07x faster Min: 0.235214 -> 0.246368: 1.05x slower Avg: 0.242988 -> 0.226850: 1.07x faster Avg: 0.236796 -> 0.247879: 1.05x slower Significant (t=78.98) Significant (t=-54.62) Stddev: 0.00200 -> 0.00042: 4.7181x smaller Stddev: 0.00140 -> 0.00147: 1.0441x larger ### pickle_list ### ### normal_startup ### Min: 0.285885 -> 0.364758: 1.28x slower Min: 0.384244 -> 0.385009: 1.00x slower Avg: 0.287357 -> 0.366063: 1.27x slower Avg: 0.428441 -> 0.443600: 1.04x slower Significant (t=-693.43) Significant (t=-2.41) Stddev: 0.00065 -> 0.00093: 1.4427x larger Stddev: 0.04674 -> 0.04226: 1.1060x smaller ### regex_v8 ### ### nqueens ### Min: 0.042516 -> 0.041687: 1.02x faster Min: 0.281015 -> 0.290544: 1.03x slower Avg: 0.043832 -> 0.042255: 1.04x faster Avg: 0.283056 -> 0.291606: 1.03x slower Significant (t=8.58) Significant (t=-65.47) Stddev: 0.00136 -> 0.00124: 1.0976x smaller Stddev: 0.00108 -> 0.00074: 1.4573x smaller ### spectral_norm ### ### pickle_dict ### Min: 0.310979 -> 0.298881: 1.04x faster Min: 0.528784 -> 0.517156: 1.02x faster Avg: 0.312106 -> 0.299629: 1.04x faster Avg: 0.530446 -> 0.518153: 1.02x faster Significant (t=128.40) Significant (t=110.08) Stddev: 0.00081 -> 0.00054: 1.4887x smaller Stddev: 0.00086 -> 0.00071: 1.2115x smaller ### telco ### ### pickle_list ### Min: 0.010738 -> 0.011111: 1.03x slower Min: 0.284315 -> 0.295096: 1.04x slower Avg: 0.010828 -> 0.011257: 1.04x slower Avg: 0.285225 -> 0.296714: 1.04x slower Significant (t=-38.44) Significant (t=-122.87) Stddev: 0.00009 -> 0.00007: 1.1812x smaller Stddev: 0.00051 -> 0.00078: 1.5209x larger ### simple_logging ### Min: 0.288782 -> 0.294265: 1.02x slower Avg: 0.290013 -> 0.296468: 1.02x slower Significant (t=-48.96) Stddev: 0.00084 -> 0.00102: 1.2110x larger ### spectral_norm ### Min: 0.299655 -> 0.310497: 1.04x slower Avg: 0.300715 -> 0.311300: 1.04x slower Significant (t=-111.23) Stddev: 0.00061 -> 0.00073: 1.1832x larger ### telco ### Min: 0.010602 -> 0.011440: 1.08x slower Avg: 0.010703 -> 0.011550: 1.08x slower Significant (t=-93.54) Stddev: 0.00006 -> 0.00007: 1.0652x larger ################################### Both Changes ################################## ## Unmodified Reference ## ## Both Reference ## ### call_simple ### ### chameleon_v2 ### Min: 0.237587 -> 0.242719: 1.02x slower Min: 5.831764 -> 5.960427: 1.02x slower Avg: 0.238111 -> 0.243361: 1.02x slower Avg: 5.879133 -> 6.030118: 1.03x slower Significant (t=-121.65) Significant (t=-40.16) Stddev: 0.00051 -> 0.00055: 1.0882x larger Stddev: 0.02455 -> 0.02847: 1.1600x larger ### chameleon_v2 ### ### chaos ### Min: 5.995550 -> 5.806448: 1.03x faster Min: 0.294183 -> 0.308522: 1.05x slower Avg: 6.047452 -> 5.839800: 1.04x faster Avg: 0.295484 -> 0.309245: 1.05x slower Significant (t=34.40) Significant (t=-157.16) Stddev: 0.05458 -> 0.02578: 2.1169x smaller Stddev: 0.00065 -> 0.00058: 1.1241x smaller ### chaos ### ### etree_generate ### Min: 0.314588 -> 0.296218: 1.06x faster Min: 0.262827 -> 0.269677: 1.03x slower Avg: 0.316190 -> 0.297500: 1.06x faster Avg: 0.265566 -> 0.272138: 1.02x slower Significant (t=212.34) Significant (t=-34.03) Stddev: 0.00060 -> 0.00064: 1.0681x larger Stddev: 0.00137 -> 0.00136: 1.0024x smaller ### etree_generate ### ### etree_process ### Min: 0.265888 -> 0.256898: 1.03x faster Min: 0.221596 -> 0.225477: 1.02x slower Avg: 0.268634 -> 0.259228: 1.04x faster Avg: 0.223674 -> 0.228316: 1.02x slower Significant (t=42.97) Significant (t=-22.85) Stddev: 0.00161 -> 0.00148: 1.0860x smaller Stddev: 0.00137 -> 0.00150: 1.0981x larger ### etree_parse ### ### fannkuch ### Min: 0.274501 -> 0.283342: 1.03x slower Min: 0.973208 -> 1.079941: 1.11x slower Avg: 0.276653 -> 0.285289: 1.03x slower Avg: 0.978633 -> 1.084415: 1.11x slower Significant (t=-43.84) Significant (t=-152.24) Stddev: 0.00139 -> 0.00140: 1.0102x larger Stddev: 0.00504 -> 0.00478: 1.0538x smaller ### etree_process ### ### fastpickle ### Min: 0.226045 -> 0.219913: 1.03x faster Min: 0.501140 -> 0.483467: 1.04x faster Avg: 0.227819 -> 0.222176: 1.03x faster Avg: 0.503016 -> 0.485275: 1.04x faster Significant (t=27.58) Significant (t=122.23) Stddev: 0.00141 -> 0.00149: 1.0551x larger Stddev: 0.00083 -> 0.00119: 1.4386x larger ### fannkuch ### ### float ### Min: 1.087744 -> 0.967702: 1.12x faster Min: 0.282255 -> 0.291807: 1.03x slower Avg: 1.091716 -> 0.974340: 1.12x faster Avg: 0.289957 -> 0.300002: 1.03x slower Significant (t=167.31) Significant (t=-16.66) Stddev: 0.00419 -> 0.00563: 1.3416x larger Stddev: 0.00414 -> 0.00438: 1.0569x larger ### fastunpickle ### ### formatted_logging ### Min: 0.660645 -> 0.647045: 1.02x faster Min: 0.333163 -> 0.343563: 1.03x slower Avg: 0.663346 -> 0.649734: 1.02x faster Avg: 0.334879 -> 0.345540: 1.03x slower Significant (t=64.90) Significant (t=-46.21) Stddev: 0.00131 -> 0.00164: 1.2534x larger Stddev: 0.00169 -> 0.00157: 1.0746x smaller ### float ### ### go ### Min: 0.290251 -> 0.278482: 1.04x faster Min: 0.546839 -> 0.559471: 1.02x slower Avg: 0.298040 -> 0.286213: 1.04x faster Avg: 0.549903 -> 0.563112: 1.02x slower Significant (t=19.51) Significant (t=-51.04) Stddev: 0.00436 -> 0.00421: 1.0363x smaller Stddev: 0.00177 -> 0.00189: 1.0693x larger ### formatted_logging ### ### hexiom2 ### Min: 0.341317 -> 0.333714: 1.02x faster Min: 120.705536 -> 129.059852: 1.07x slower Avg: 0.342848 -> 0.335283: 1.02x faster Avg: 120.812957 -> 129.164489: 1.07x slower Significant (t=34.69) Significant (t=-120.84) Stddev: 0.00154 -> 0.00154: 1.0030x larger Stddev: 0.11002 -> 0.08367: 1.3149x smaller ### hexiom2 ### ### nbody ### Min: 128.849155 -> 121.051296: 1.06x faster Min: 0.235000 -> 0.244803: 1.04x slower Avg: 128.908684 -> 121.208467: 1.06x faster Avg: 0.237193 -> 0.245852: 1.04x slower Significant (t=91.96) Significant (t=-71.38) Stddev: 0.06121 -> 0.15588: 2.5467x larger Stddev: 0.00109 -> 0.00053: 2.0719x smaller ### nbody ### ### normal_startup ### Min: 0.246950 -> 0.235331: 1.05x faster Min: 0.382206 -> 0.385183: 1.01x slower Avg: 0.248454 -> 0.236631: 1.05x faster Avg: 0.418229 -> 0.448886: 1.07x slower Significant (t=66.38) Significant (t=-4.20) Stddev: 0.00154 -> 0.00089: 1.7232x smaller Stddev: 0.03526 -> 0.06392: 1.8127x larger ### nqueens ### ### nqueens ### Min: 0.282850 -> 0.276236: 1.02x faster Min: 0.273482 -> 0.283099: 1.04x slower Avg: 0.283639 -> 0.277317: 1.02x faster Avg: 0.274345 -> 0.283928: 1.03x slower Significant (t=83.18) Significant (t=-134.87) Stddev: 0.00055 -> 0.00053: 1.0421x smaller Stddev: 0.00056 -> 0.00044: 1.2649x smaller ### pathlib ### ### pickle_dict ### Min: 0.086825 -> 0.088899: 1.02x slower Min: 0.554602 -> 0.518523: 1.07x faster Avg: 0.087307 -> 0.089354: 1.02x slower Avg: 0.556059 -> 0.519286: 1.07x faster Significant (t=-99.30) Significant (t=418.98) Stddev: 0.00034 -> 0.00031: 1.0748x smaller Stddev: 0.00073 -> 0.00049: 1.4855x smaller ### pickle_list ### ### pickle_list ### Min: 0.284098 -> 0.301891: 1.06x slower Min: 0.309630 -> 0.284647: 1.09x faster Avg: 0.285180 -> 0.303486: 1.06x slower Avg: 0.311326 -> 0.285466: 1.09x faster Significant (t=-177.88) Significant (t=248.85) Stddev: 0.00055 -> 0.00087: 1.5853x larger Stddev: 0.00086 -> 0.00059: 1.4620x smaller ### regex_v8 ### ### simple_logging ### Min: 0.042903 -> 0.041928: 1.02x faster Min: 0.291289 -> 0.298021: 1.02x slower Avg: 0.043214 -> 0.042204: 1.02x faster Avg: 0.292516 -> 0.299331: 1.02x slower Significant (t=5.78) Significant (t=-47.23) Stddev: 0.00123 -> 0.00124: 1.0158x larger Stddev: 0.00091 -> 0.00112: 1.2390x larger ### spectral_norm ### ### spectral_norm ### Min: 0.311778 -> 0.276145: 1.13x faster Min: 0.278228 -> 0.311677: 1.12x slower Avg: 0.313448 -> 0.276834: 1.13x faster Avg: 0.278897 -> 0.312374: 1.12x slower Significant (t=252.50) Significant (t=-368.99) Stddev: 0.00136 -> 0.00052: 2.6315x smaller Stddev: 0.00069 -> 0.00059: 1.1710x smaller ### unpickle_list ### ### telco ### Min: 0.362565 -> 0.353259: 1.03x faster Min: 0.011857 -> 0.011465: 1.03x faster Avg: 0.364273 -> 0.356519: 1.02x faster Avg: 0.011963 -> 0.011573: 1.03x faster Significant (t=51.14) Significant (t=35.58) Stddev: 0.00084 -> 0.00126: 1.4998x larger Stddev: 0.00006 -> 0.00009: 1.7002x larger