This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: generator objects can clear their weakrefs before being resurrected
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.8, Python 3.7, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Windson Yang, ncoghlan, pitrou
Priority: low Keywords:

Created on 2012-05-28 13:20 by pitrou, last changed 2022-04-11 14:57 by admin.

Messages (5)
msg161781 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-05-28 13:20
In Objects/genobject.c, gen_dealloc() calls PyObject_ClearWeakRefs() before trying to finalize the generator, during which the generator object can be resurrected. This is probably a bug, since weakrefs are supposed to be cleared (and their callbacks called) only when the object is really being destroyed.

(see also issue14933)
msg222269 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-04 07:50
I'm still not brave enough to take on C code, but could this be handled by someone from the core-mentorship list?
msg338057 - (view) Author: Windson Yang (Windson Yang) * Date: 2019-03-16 07:18
In the docs points out:

> /* Clear weakrefs first before calling any destructors */

So maybe we should also update the docs after fix this bug. Anyway, I will try to understand/fix this bug.
msg338520 - (view) Author: Windson Yang (Windson Yang) * Date: 2019-03-21 02:46
The fixed looks easy, we call `PyObject_CallFinalizerFromDealloc` before PyObject_ClearWeakRefs. But I can't come up with the use case for testing when generator resurrects from `PyObject_CallFinalizer`. Any ideas?
msg339194 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2019-03-30 12:39
If I recall correctly, it's the generator destructor that handles throwing in ``GeneratorExit`` to get the generator to terminate. So this code can resurrect a generator as it's being collected by the GC:

    def resurrecting(resurrected):
        self = yield

    storage = []
    g = resurrecting(storage)
    g.send(g) # Give the generator a reference to itself
    del g # Now the generator is in a cycle with itself
    # Generator has been added to the storage instead of collected
    assert len(storage) == 1
    # Clear the storage to kill it for real this time
    # Weakrefs shouldn't get called until here

Antoine, does that sound right to you?
Date User Action Args
2022-04-11 14:57:30adminsetgithub: 59139
2019-03-30 12:39:58ncoghlansetmessages: + msg339194
2019-03-21 02:46:25Windson Yangsetmessages: + msg338520
2019-03-16 08:42:31SilentGhostsetversions: + Python 3.8, - Python 3.4, Python 3.5, Python 3.6
2019-03-16 07:18:54Windson Yangsetnosy: + Windson Yang

messages: + msg338057
versions: + Python 3.6, Python 3.7
2019-03-16 00:14:22BreamoreBoysetnosy: - BreamoreBoy
2014-07-04 07:50:31BreamoreBoysetnosy: + BreamoreBoy

messages: + msg222269
versions: + Python 3.4, Python 3.5, - Python 3.2, Python 3.3
2012-05-28 13:20:07pitroucreate