Author eric.snow
Recipients eric.snow, jeremy.kloth, jkloth, nanjekyejoannah, ncoghlan, rhettinger, vstinner
Date 2020-02-04.16:24:49
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CALFfu7AJ9sdX1YcEVL6BO7NGgS9u87cfy-rEK+s0PsLR4AR1Jg@mail.gmail.com>
In-reply-to <1580682733.44.0.917064169701.issue39511@roundup.psfhosted.org>
Content
On Sun, Feb 2, 2020 at 3:32 PM Raymond Hettinger <report@bugs.python.org> wrote:
> Random idea (not carefully thought-out):  Would it be simpler to have these objects just
> ignore their refcount by having dealloc() be a null operation or having it set the refcount
> back to a positive number).  That would let sub-interpreters share the objects without
> worrying about race-conditions on incref/decref operations.

This is pretty much one of the two approaches I have been considering. :)

Just to be clear, singletons normally won't end up with a refcount of
0, by design.  However, if there's a incref race between two
interpreters (that don't share a GIL) then you *can* end up triggering
dealloc (via Py_DECREF) and even get negative refcounts (which cause
Py_FatalError() on debug builds).  Here are some mitigations:

* (as noted above) make dealloc() for singletons a noop
* (as noted above) in dealloc() set the refcount back to some positive value
* make the initial refcount sufficiently large such that it is
unlikely to reach 0 even with races
* incref each singleton once during each interpreter initiialization
(and decref once during interp. finalization)

We would have to special-case refleak checks for singletons, to avoid
false positives.

Also note that currently extension authors (and CPython contributors)
can rely on the runtime to identify when then accidentally
incref/decref a singleton too much (or forget to do so).  So it may
make sense to do something in the "singleton dealloc()" in debug
builds to provide similar checks.

>  To make this work, the objects can register themselves as permanent, shared, objects;
> then, during shutdown, we could explicitly call a hard dealloc on those objects.

great point
History
Date User Action Args
2020-02-04 16:24:50eric.snowsetrecipients: + eric.snow, rhettinger, ncoghlan, vstinner, jkloth, jeremy.kloth, nanjekyejoannah
2020-02-04 16:24:50eric.snowlinkissue39511 messages
2020-02-04 16:24:49eric.snowcreate