Author vstinner
Recipients rhettinger, vstinner
Date 2020-01-29.21:29:32
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1580333373.26.0.157455576916.issue39489@roundup.psfhosted.org>
In-reply-to
Content
As far as I known, the Fedora package of Python is the most known user of COUNT_ALLOCS special build, and maybe the only user.

The Fedora package of Python 2.7 builds two binaries:

* python2.7: release mode, optimized
* python2.7-debug: debug mode, not optimized, built with COUNT_ALLOCS defined

Sadly, in practice, python2.7-debug is basically unusable since C extensions providing by Fedora are not compatible: the release and the debug modes have a different ABI. I only fixed this recently in Python 3.8.

It uses COUNT_ALLOCS since the following patch (package version 2.6.5-10, in 2010) written by Dave Malcolm.


(1) There is a downstream patch adding --with-count-allocs flag to configure. Patch added to package version 2.6.5-10, in 2010:
https://src.fedoraproject.org/rpms/python2/c/ab11e4c10f6fef4e2e993ef446953df0f0dbb840


(2) Another downstream patch adds PYTHONDUMPCOUNTS environment variable to only dump statistics if the variable is set. Patch added to package version 2.7-8, in 2010:
https://src.fedoraproject.org/rpms/python2/c/5810c5d8b1c876ccc4c547cc71cf20520dd27d85

=> this patch was proposed upstream in bpo-19527. I merged it in Python 2.7.15 (in 2017), but with a different environment variable name: PYTHONSHOWALLOCCOUNT.


(3) Finally, a 3rd downstream patch fix test_abc when COUNT_ALLOCS is defined. Patch added to package version 2.7.1-1, in 2010:
https://src.fedoraproject.org/rpms/python2/c/4b97eebe22c8c5db58ae65cdc7e79c3ccd45b0a4

=> Bohuslav "Slavek" Kabrda created bpo-19527 to propose to make multiple test fixes upstream. Serhiy Storchaka modified the proposed patch by adding @test.support.requires_type_collecting decorator. Change merged in 3.5 and default branches, but not in 2.7.


Another example of Fedora contribution: bpo-31692 reports that test_regrtest fails when Python 2.7.14 is built with COUNT_ALLOCS.

---

Now, the interesting part.

IMHO COUNT_ALLOCS was basically kept between 2010 and 2015 because "it was there" and nobody asked if it still made sense to use it. It wasn't too expensive to maintain in the Fedora package... until someone asked if it's still worth it to maintain it in 2015.

The COUNT_ALLOCS macro has been removed from the Fedora package of Python 3 in 2015:

* https://bugzilla.redhat.com/show_bug.cgi?id=1291325
* "[cleanup] Remove COUNT_ALLOCS patches, see rhbz#1291325" commit:
  https://src.fedoraproject.org/rpms/python3/c/19aade22cbfa57dc500f5fa82229b4b9ed0b4c30

Extract of the bugzilla, "Reply from Dave Malcolm":
"""
My hazy recollection is that at the time I was dealing with lots of
memory leak issues in Python 2, so I was keen to add as much help as
possible in tracking them down to Python 2 and Python 3.

I don't think this patch ever really bought us much, and it sounds like
there are better tools for this now, so feel free to drop the
COUNT_ALLOC patches.
"""

It confirms what I wrote in the initial message of this issue: there are now better tool to track Python memory leaks.

Supporting COUNT_ALLOC in the Fedora package was motivated by Dave Malcolm use case of tracking memory leak, but even Dave wrote: "I don't think this patch ever really bought us much".
History
Date User Action Args
2020-01-29 21:29:33vstinnersetrecipients: + vstinner, rhettinger
2020-01-29 21:29:33vstinnersetmessageid: <1580333373.26.0.157455576916.issue39489@roundup.psfhosted.org>
2020-01-29 21:29:33vstinnerlinkissue39489 messages
2020-01-29 21:29:32vstinnercreate