Author vstinner
Recipients inada.naoki, python-dev, serhiy.storchaka, vstinner
Date 2017-01-26.08:46:55
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1485420416.94.0.0869215775916.issue29259@psf.upfronthosting.co.za>
In-reply-to
Content
performance results on tp_fastcall-6.patch (diff >= 2%):
---
haypo@speed-python$ python3 -m perf compare_to /home/haypo/benchmarks/2017-01-23_14-02-default-cebc9c7ad195.json tp_fastcall-6_ref_cebc9c7ad195.json -G --min-speed=2
Slower (6):
- scimark_monte_carlo: 202 ms +- 5 ms -> 222 ms +- 8 ms: 1.10x slower (+10%)
- telco: 13.0 ms +- 0.2 ms -> 14.2 ms +- 0.4 ms: 1.09x slower (+9%)
- call_method: 11.0 ms +- 0.3 ms -> 11.5 ms +- 0.5 ms: 1.05x slower (+5%)
- scimark_sparse_mat_mult: 8.57 ms +- 0.14 ms -> 8.92 ms +- 0.15 ms: 1.04x slower (+4%)
- fannkuch: 938 ms +- 7 ms -> 974 ms +- 11 ms: 1.04x slower (+4%)
- scimark_fft: 667 ms +- 9 ms -> 690 ms +- 10 ms: 1.03x slower (+3%)

Faster (17):
- scimark_lu: 362 ms +- 13 ms -> 322 ms +- 8 ms: 1.12x faster (-11%)
- chameleon: 23.5 ms +- 0.3 ms -> 21.3 ms +- 0.3 ms: 1.10x faster (-9%)
- xml_etree_process: 179 ms +- 3 ms -> 169 ms +- 3 ms: 1.06x faster (-5%)
- xml_etree_generate: 208 ms +- 4 ms -> 199 ms +- 3 ms: 1.05x faster (-4%)
- logging_silent: 586 ns +- 8 ns -> 562 ns +- 7 ns: 1.04x faster (-4%)
- richards: 147 ms +- 2 ms -> 141 ms +- 3 ms: 1.04x faster (-4%)
- pickle_pure_python: 1.03 ms +- 0.02 ms -> 994 us +- 15 us: 1.04x faster (-4%)
- regex_compile: 379 ms +- 2 ms -> 367 ms +- 4 ms: 1.03x faster (-3%)
- raytrace: 1.08 sec +- 0.01 sec -> 1.04 sec +- 0.02 sec: 1.03x faster (-3%)
- unpickle: 31.0 us +- 0.4 us -> 30.0 us +- 0.4 us: 1.03x faster (-3%)
- pathlib: 42.3 ms +- 0.5 ms -> 41.0 ms +- 0.5 ms: 1.03x faster (-3%)
- scimark_sor: 391 ms +- 7 ms -> 379 ms +- 12 ms: 1.03x faster (-3%)
- genshi_xml: 167 ms +- 2 ms -> 162 ms +- 2 ms: 1.03x faster (-3%)
- django_template: 356 ms +- 5 ms -> 347 ms +- 4 ms: 1.03x faster (-3%)
- regex_dna: 283 ms +- 1 ms -> 276 ms +- 4 ms: 1.03x faster (-3%)
- hexiom: 18.0 ms +- 0.1 ms -> 17.6 ms +- 0.2 ms: 1.02x faster (-2%)
- call_simple: 10.7 ms +- 0.4 ms -> 10.5 ms +- 0.2 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (41): (...)
---

Or less verbose output, only show difference >= 5%:
---
haypo@speed-python$ python3 -m perf compare_to /home/haypo/benchmarks/2017-01-23_14-02-default-cebc9c7ad195.json tp_fastcall-6_ref_cebc9c7ad195.json -G --min-speed=5
Slower (2):
- scimark_monte_carlo: 202 ms +- 5 ms -> 222 ms +- 8 ms: 1.10x slower (+10%)
- telco: 13.0 ms +- 0.2 ms -> 14.2 ms +- 0.4 ms: 1.09x slower (+9%)

Faster (3):
- scimark_lu: 362 ms +- 13 ms -> 322 ms +- 8 ms: 1.12x faster (-11%)
- chameleon: 23.5 ms +- 0.3 ms -> 21.3 ms +- 0.3 ms: 1.10x faster (-9%)
- xml_etree_process: 179 ms +- 3 ms -> 169 ms +- 3 ms: 1.06x faster (-5%)

Benchmark hidden because not significant (59): (...)
---

To be honest, I'm disappointed, I expected more faster benchmarks. Maybe the cost of the temporary tuple/dict to pass arguments to function is not as high as I expected.

telco is also my favorite benchmark, I'm sad that it's 1.09x slower. I should investigate that.

scimark_monte_carlo and scimark_lu benchmarks are still unstable for an unknown reasons, so I don't care too much of these two benchmarks.

* https://speed.python.org/timeline/#/?exe=5&ben=scimark_monte_carlo&env=1&revs=50&equid=off&quarts=on&extr=on
* https://speed.python.org/timeline/#/?exe=5&ben=scimark_lu&env=1&revs=50&equid=off&quarts=on&extr=on
History
Date User Action Args
2017-01-26 08:46:56vstinnersetrecipients: + vstinner, inada.naoki, python-dev, serhiy.storchaka
2017-01-26 08:46:56vstinnersetmessageid: <1485420416.94.0.0869215775916.issue29259@psf.upfronthosting.co.za>
2017-01-26 08:46:56vstinnerlinkissue29259 messages
2017-01-26 08:46:55vstinnercreate