This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Handle weakreference callbacks invoked indirectly in the middle of a gc collection
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.9, Python 3.8, Python 3.7
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, nascheme, pablogsal, pitrou, vstinner
Priority: normal Keywords: patch

Created on 2019-09-02 13:56 by pablogsal, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 15645 closed pablogsal, 2019-09-02 13:58
Messages (5)
msg351001 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 13:56
A weak reference may try to invoke a callback object that is being
cleaned (tp_clear) by the garbage collector and it may be in an
inconsistent state. As the garbage collector explicitly does not
invoke callbacks that are part of the same cycle isolate as the
weak reference (pretending that the weak reference was destroyed first),
we should act in the same way when invoking callbacks using Objects/weakrefobject.c:handle_callback.

For example, consider the following scenario:

- F is a function.
- An object O is in a cycle with F.
- O has a weak reference with F as a callback.

When running the garbage collector, is possible to end in Objects/weakrefobject.c:handle_callback if the tp_clear of F decrements the references of O, invoking the weak
reference callback that will try to call F, which is in an incosistent
state as is in the middle of its tp_clear and some internal fields may
be NULL.
msg351002 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 13:57
Antoine, could you check that I am not missing something in this logic?
msg351003 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 14:01
Check for reference on this situation.

I have the suspicion that something in this argument is wrong and the problem of arriving in this situation may be elsewhere, but I wanted to create this issue and show the proposed PR so we can discuss this better with more clarity.
msg351011 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 15:13
I think I am going to close this until we understand better
msg353426 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2019-09-27 23:01
I think the problem with your logic is that the weakref to F is part of the garbage set.  So, handle_finalizers() should detect that and clear the finalizer rather than call it.  Once we get to delete_garbage() and start calling tp_clear(), we can't be running weakref callbacks or finalizers.  The GC logic goes through great pains to ensure that.
Date User Action Args
2022-04-11 14:59:19adminsetgithub: 82190
2019-09-27 23:01:58naschemesetnosy: + nascheme
messages: + msg353426
2019-09-02 21:55:21christian.heimessetnosy: + christian.heimes
2019-09-02 15:13:50pablogsalsetstatus: open -> closed
resolution: not a bug
stage: patch review -> resolved
2019-09-02 15:13:44pablogsalsetmessages: + msg351011
2019-09-02 14:01:01pablogsalsetmessages: + msg351003
2019-09-02 13:58:09pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request15310
2019-09-02 13:57:43pablogsalsetmessages: + msg351002
2019-09-02 13:56:33pablogsalcreate