Message378637
Using the following patch:
https://github.com/python/cpython/compare/master...brandtbucher:track-all-heap-types
I got the following pyperformance results (with PGO/LTO and CPU isolation, insignificant rows omitted):
2020-10-13_20-04-master-7992579cd27f.json.gz
============================================
Performance version: 1.0.0
Report on Linux-4.4.0-190-generic-x86_64-with-glibc2.23
Number of logical CPUs: 8
Start date: 2020-10-14 07:49:46.738847
End date: 2020-10-14 08:12:58.050557
2020-10-13_20-37-track-all-heap-types-63d8d25867b5.json.gz
==========================================================
Performance version: 1.0.0
Report on Linux-4.4.0-190-generic-x86_64-with-glibc2.23
Number of logical CPUs: 8
Start date: 2020-10-14 08:29:52.009796
End date: 2020-10-14 08:52:49.230214
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| Benchmark | 2020-10-13_20-04-master-7992579cd27f.json.gz | 2020-10-13_20-37-track-all-heap-types-63d8d25867b5.json.gz | Change | Significance |
+=========================+==============================================+============================================================+==============+=======================+
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| chameleon | 13.7 ms | 13.1 ms | 1.04x faster | Significant (t=7.27) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| chaos | 163 ms | 159 ms | 1.03x faster | Significant (t=5.39) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| crypto_pyaes | 160 ms | 156 ms | 1.02x faster | Significant (t=7.29) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| deltablue | 10.9 ms | 10.7 ms | 1.03x faster | Significant (t=4.36) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| django_template | 73.1 ms | 71.2 ms | 1.03x faster | Significant (t=4.42) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| go | 373 ms | 381 ms | 1.02x slower | Significant (t=-4.71) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| nbody | 206 ms | 200 ms | 1.03x faster | Significant (t=7.52) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| regex_dna | 295 ms | 288 ms | 1.02x faster | Significant (t=5.26) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| richards | 103 ms | 101 ms | 1.03x faster | Significant (t=5.40) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| scimark_sparse_mat_mult | 6.96 ms | 7.17 ms | 1.03x slower | Significant (t=-4.93) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| spectral_norm | 218 ms | 207 ms | 1.05x faster | Significant (t=10.29) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| unpack_sequence | 101 ns | 84.1 ns | 1.20x faster | Significant (t=36.84) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
Note that I used pyperformance==1.0.0, since 1.0.1 won't run on 3.10.
In general, it looks like the patch has no real performance impact. The result for unpack_sequence is *extremely* surprising... I'm quite skeptical of it, although it looks like it is one of the more "micro" benchmarks in the suite.
All things considered, I believe we should fix this... I personally view memory leaks as second only to crashes and incorrect behavior in terms of severity. |
|
Date |
User |
Action |
Args |
2020-10-14 17:17:36 | brandtbucher | set | recipients:
+ brandtbucher, tim.peters, nascheme, pablogsal |
2020-10-14 17:17:36 | brandtbucher | set | messageid: <1602695856.81.0.819985783478.issue41984@roundup.psfhosted.org> |
2020-10-14 17:17:36 | brandtbucher | link | issue41984 messages |
2020-10-14 17:17:36 | brandtbucher | create | |
|