Author vstinner
Recipients larry, rhettinger, serhiy.storchaka, vstinner, yselivanov
Date 2016-04-22.14:56:57
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1461337018.0.0.521048848125.issue26814@psf.upfronthosting.co.za>
In-reply-to
Content
Results of the CPython benchmark suite on the revision 6c376e866330 of  https://hg.python.org/sandbox/fastcall compared to CPython 3.6 at the revision 496e094f4734.

It's surprising than call_simple is 1.08x slower in fastcall. This slowdown is not acceptable and should be fixed. It probable explains why many other benchmarks are slower.

Hopefully, some benchmarks are faster, between 1.02x and 1.09x faster.

IMHO there are still performance issues in my current implementation that can and must be fixed. At least, we have a starting point to compare performances.


$ python3 -u perf.py ../default/python ../fastcall/python -b all
(...)
Report on Linux smithers 4.4.4-301.fc23.x86_64 #1 SMP Fri Mar 4 17:42:42 UTC 2016 x86_64 x86_64
Total CPU cores: 8

[ slower ]

### 2to3 ###
6.859604 -> 6.985351: 1.02x slower

### call_method_slots ###
Min: 0.308846 -> 0.317780: 1.03x slower
Avg: 0.308902 -> 0.318667: 1.03x slower
Significant (t=-464.83)
Stddev: 0.00003 -> 0.00026: 9.8974x larger

### call_simple ###
Min: 0.232594 -> 0.251789: 1.08x slower
Avg: 0.232816 -> 0.252443: 1.08x slower
Significant (t=-911.97)
Stddev: 0.00024 -> 0.00011: 2.2373x smaller

### chaos ###
Min: 0.273084 -> 0.284790: 1.04x slower
Avg: 0.273951 -> 0.293177: 1.07x slower
Significant (t=-7.57)
Stddev: 0.00036 -> 0.01796: 49.9421x larger

### django_v3 ###
Min: 0.549604 -> 0.569982: 1.04x slower
Avg: 0.550557 -> 0.571038: 1.04x slower
Significant (t=-204.09)
Stddev: 0.00046 -> 0.00054: 1.1747x larger

### float ###
Min: 0.261939 -> 0.269224: 1.03x slower
Avg: 0.268475 -> 0.276515: 1.03x slower
Significant (t=-12.22)
Stddev: 0.00301 -> 0.00354: 1.1757x larger

### formatted_logging ###
Min: 0.325786 -> 0.334440: 1.03x slower
Avg: 0.326827 -> 0.335968: 1.03x slower
Significant (t=-34.44)
Stddev: 0.00129 -> 0.00136: 1.0503x larger

### mako_v2 ###
Min: 0.039642 -> 0.044765: 1.13x slower
Avg: 0.040251 -> 0.045562: 1.13x slower
Significant (t=-323.73)
Stddev: 0.00028 -> 0.00024: 1.1558x smaller

### meteor_contest ###
Min: 0.196589 -> 0.203667: 1.04x slower
Avg: 0.197497 -> 0.204782: 1.04x slower
Significant (t=-76.06)
Stddev: 0.00050 -> 0.00045: 1.1111x smaller

### nqueens ###
Min: 0.274664 -> 0.285866: 1.04x slower
Avg: 0.275285 -> 0.286774: 1.04x slower
Significant (t=-68.34)
Stddev: 0.00091 -> 0.00076: 1.2036x smaller

### pickle_list ###
Min: 0.262687 -> 0.269629: 1.03x slower
Avg: 0.263804 -> 0.270789: 1.03x slower
Significant (t=-50.14)
Stddev: 0.00070 -> 0.00070: 1.0004x larger

### raytrace ###
Min: 1.272960 -> 1.284516: 1.01x slower
Avg: 1.276398 -> 1.368574: 1.07x slower
Significant (t=-3.41)
Stddev: 0.00157 -> 0.19115: 122.0022x larger

### regex_compile ###
Min: 0.335753 -> 0.343820: 1.02x slower
Avg: 0.336273 -> 0.344894: 1.03x slower
Significant (t=-127.84)
Stddev: 0.00026 -> 0.00040: 1.5701x larger

### regex_effbot ###
Min: 0.048656 -> 0.050810: 1.04x slower
Avg: 0.048692 -> 0.051619: 1.06x slower
Significant (t=-69.92)
Stddev: 0.00002 -> 0.00030: 16.7793x larger

### silent_logging ###
Min: 0.069539 -> 0.071172: 1.02x slower
Avg: 0.069679 -> 0.071230: 1.02x slower
Significant (t=-124.08)
Stddev: 0.00009 -> 0.00002: 3.7073x smaller

### simple_logging ###
Min: 0.278439 -> 0.287736: 1.03x slower
Avg: 0.279504 -> 0.288811: 1.03x slower
Significant (t=-52.46)
Stddev: 0.00084 -> 0.00093: 1.1074x larger

### telco ###
Min: 0.012480 -> 0.013104: 1.05x slower
Avg: 0.012561 -> 0.013157: 1.05x slower
Significant (t=-100.42)
Stddev: 0.00004 -> 0.00002: 1.5881x smaller

### unpack_sequence ###
Min: 0.000047 -> 0.000048: 1.03x slower
Avg: 0.000047 -> 0.000048: 1.03x slower
Significant (t=-1170.16)
Stddev: 0.00000 -> 0.00000: 1.0749x larger

### unpickle_list ###
Min: 0.325310 -> 0.330080: 1.01x slower
Avg: 0.326484 -> 0.333974: 1.02x slower
Significant (t=-24.19)
Stddev: 0.00100 -> 0.00195: 1.9392x larger

[ faster ]

### chameleon_v2 ###
Min: 5.525575 -> 5.263668: 1.05x faster
Avg: 5.541444 -> 5.281893: 1.05x faster
Significant (t=85.79)
Stddev: 0.01107 -> 0.01831: 1.6539x larger

### etree_iterparse ###
Min: 0.212073 -> 0.197146: 1.08x faster
Avg: 0.215504 -> 0.200254: 1.08x faster
Significant (t=61.07)
Stddev: 0.00119 -> 0.00130: 1.0893x larger

### etree_parse ###
Min: 0.282983 -> 0.260390: 1.09x faster
Avg: 0.284333 -> 0.262758: 1.08x faster
Significant (t=77.34)
Stddev: 0.00102 -> 0.00169: 1.6628x larger

### etree_process ###
Min: 0.218953 -> 0.213683: 1.02x faster
Avg: 0.221036 -> 0.215280: 1.03x faster
Significant (t=25.98)
Stddev: 0.00114 -> 0.00108: 1.0580x smaller

### hexiom2 ###
Min: 122.001408 -> 118.967112: 1.03x faster
Avg: 122.108010 -> 119.110115: 1.03x faster
Significant (t=16.81)
Stddev: 0.15076 -> 0.20224: 1.3415x larger

### pathlib ###
Min: 0.088533 -> 0.084888: 1.04x faster
Avg: 0.088916 -> 0.085280: 1.04x faster
Significant (t=257.68)
Stddev: 0.00014 -> 0.00017: 1.1725x larger


The following not significant results are hidden, use -v to show them:
call_method, call_method_unknown, etree_generate, fannkuch, fastpickle, fastunpickle, go, json_dump_v2, json_load, nbody, normal_startup, pickle_dict, pidigits, regex_v8, richards, spectral_norm, startup_nosite, tornado_http.
History
Date User Action Args
2016-04-22 14:56:58vstinnersetrecipients: + vstinner, rhettinger, larry, serhiy.storchaka, yselivanov
2016-04-22 14:56:57vstinnersetmessageid: <1461337018.0.0.521048848125.issue26814@psf.upfronthosting.co.za>
2016-04-22 14:56:57vstinnerlinkissue26814 messages
2016-04-22 14:56:57vstinnercreate