Author gregory.p.smith
Recipients eelizondo, gregory.p.smith, nascheme, pablogsal, pitrou, tim.peters
Date 2020-04-11.22:37:57
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1586644678.08.0.0845575954584.issue40255@roundup.psfhosted.org>
In-reply-to
Content
Thanks for sharing!  It's good to have a patch implementing for others who might need it to try out.

We experimented with an implementation of this on CPython 2.7 that we called Eternal Refcounts for YouTube many years ago.  For the same reason (saving memory in forked workers by not dirtying pages of known needed objects via refcount changes).

I don't have that old patch handy right now, but looking at yours it was the same concept: A magic refcount value to branch based on.

We wound up not deploying it or pushing for it in CPython because the CPU performance implications of adding a branch instruction to Py_INCREC and Py_DECREF were, unsurprisingly, quite high.  I'd have to go digging to find numbers but it _believe_ it was on the order of a 10% slowdown on real world application code.  (microbenchmarks don't tell a good story, the python performance test suite should)

Given that most people's applications don't fork workers, I do not expect to see such an implementation ever become the default.  It is a very appropriate hack, but the use case is limited to applications that don't use threads, are on POSIX, and always fork() many workers.

Also note that this is an ABI change as those INCREF and DECREF definitions are intentionally public .h file macros or inlines and thus included within any compiledcode rather than being an expensive function call elsewhere (vstinner's new alternate higher level API proposal presumably changes this).  If an interpreter with this loaded a binary using the CPython API built against headers without it, your refcounts could get messed up.
History
Date User Action Args
2020-04-11 22:37:58gregory.p.smithsetrecipients: + gregory.p.smith, tim.peters, nascheme, pitrou, pablogsal, eelizondo
2020-04-11 22:37:58gregory.p.smithsetmessageid: <1586644678.08.0.0845575954584.issue40255@roundup.psfhosted.org>
2020-04-11 22:37:58gregory.p.smithlinkissue40255 messages
2020-04-11 22:37:57gregory.p.smithcreate