Author brandtbucher
Recipients brandtbucher, nascheme, pablogsal, tim.peters
Date 2020-10-14.17:17:36
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1602695856.81.0.819985783478.issue41984@roundup.psfhosted.org>
In-reply-to
Content
Using the following patch:

https://github.com/python/cpython/compare/master...brandtbucher:track-all-heap-types

I got the following pyperformance results (with PGO/LTO and CPU isolation, insignificant rows omitted):

2020-10-13_20-04-master-7992579cd27f.json.gz
============================================

Performance version: 1.0.0
Report on Linux-4.4.0-190-generic-x86_64-with-glibc2.23
Number of logical CPUs: 8
Start date: 2020-10-14 07:49:46.738847
End date: 2020-10-14 08:12:58.050557

2020-10-13_20-37-track-all-heap-types-63d8d25867b5.json.gz
==========================================================

Performance version: 1.0.0
Report on Linux-4.4.0-190-generic-x86_64-with-glibc2.23
Number of logical CPUs: 8
Start date: 2020-10-14 08:29:52.009796
End date: 2020-10-14 08:52:49.230214

+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| Benchmark               | 2020-10-13_20-04-master-7992579cd27f.json.gz | 2020-10-13_20-37-track-all-heap-types-63d8d25867b5.json.gz | Change       | Significance          |
+=========================+==============================================+============================================================+==============+=======================+
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| chameleon               | 13.7 ms                                      | 13.1 ms                                                    | 1.04x faster | Significant (t=7.27)  |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| chaos                   | 163 ms                                       | 159 ms                                                     | 1.03x faster | Significant (t=5.39)  |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| crypto_pyaes            | 160 ms                                       | 156 ms                                                     | 1.02x faster | Significant (t=7.29)  |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| deltablue               | 10.9 ms                                      | 10.7 ms                                                    | 1.03x faster | Significant (t=4.36)  |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| django_template         | 73.1 ms                                      | 71.2 ms                                                    | 1.03x faster | Significant (t=4.42)  |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| go                      | 373 ms                                       | 381 ms                                                     | 1.02x slower | Significant (t=-4.71) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| nbody                   | 206 ms                                       | 200 ms                                                     | 1.03x faster | Significant (t=7.52)  |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| regex_dna               | 295 ms                                       | 288 ms                                                     | 1.02x faster | Significant (t=5.26)  |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| richards                | 103 ms                                       | 101 ms                                                     | 1.03x faster | Significant (t=5.40)  |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| scimark_sparse_mat_mult | 6.96 ms                                      | 7.17 ms                                                    | 1.03x slower | Significant (t=-4.93) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| spectral_norm           | 218 ms                                       | 207 ms                                                     | 1.05x faster | Significant (t=10.29) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+
| unpack_sequence         | 101 ns                                       | 84.1 ns                                                    | 1.20x faster | Significant (t=36.84) |
+-------------------------+----------------------------------------------+------------------------------------------------------------+--------------+-----------------------+

Note that I used pyperformance==1.0.0, since 1.0.1 won't run on 3.10.

In general, it looks like the patch has no real performance impact. The result for unpack_sequence is *extremely* surprising... I'm quite skeptical of it, although it looks like it is one of the more "micro" benchmarks in the suite.

All things considered, I believe we should fix this... I personally view memory leaks as second only to crashes and incorrect behavior in terms of severity.
History
Date User Action Args
2020-10-14 17:17:36brandtbuchersetrecipients: + brandtbucher, tim.peters, nascheme, pablogsal
2020-10-14 17:17:36brandtbuchersetmessageid: <1602695856.81.0.819985783478.issue41984@roundup.psfhosted.org>
2020-10-14 17:17:36brandtbucherlinkissue41984 messages
2020-10-14 17:17:36brandtbuchercreate