classification
Title: Handle weakreference callbacks invoked indirectly in the middle of a gc collection
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, nascheme, pablogsal, pitrou, vstinner
Priority: normal Keywords: patch

Created on 2019-09-02 13:56 by pablogsal, last changed 2019-09-27 23:01 by nascheme. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 15645 closed pablogsal, 2019-09-02 13:58
Messages (5)
msg351001 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 13:56
A weak reference may try to invoke a callback object that is being
cleaned (tp_clear) by the garbage collector and it may be in an
inconsistent state. As the garbage collector explicitly does not
invoke callbacks that are part of the same cycle isolate as the
weak reference (pretending that the weak reference was destroyed first),
we should act in the same way when invoking callbacks using Objects/weakrefobject.c:handle_callback.

For example, consider the following scenario:

- F is a function.
- An object O is in a cycle with F.
- O has a weak reference with F as a callback.

When running the garbage collector, is possible to end in Objects/weakrefobject.c:handle_callback if the tp_clear of F decrements the references of O, invoking the weak
reference callback that will try to call F, which is in an incosistent
state as is in the middle of its tp_clear and some internal fields may
be NULL.
msg351002 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 13:57
Antoine, could you check that I am not missing something in this logic?
msg351003 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 14:01
Check https://bugs.python.org/issue38006 for reference on this situation.

I have the suspicion that something in this argument is wrong and the problem of arriving in this situation may be elsewhere, but I wanted to create this issue and show the proposed PR so we can discuss this better with more clarity.
msg351011 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 15:13
I think I am going to close this until we understand https://bugs.python.org/issue38006 better
msg353426 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2019-09-27 23:01
I think the problem with your logic is that the weakref to F is part of the garbage set.  So, handle_finalizers() should detect that and clear the finalizer rather than call it.  Once we get to delete_garbage() and start calling tp_clear(), we can't be running weakref callbacks or finalizers.  The GC logic goes through great pains to ensure that.
History
Date User Action Args
2019-09-27 23:01:58naschemesetnosy: + nascheme
messages: + msg353426
2019-09-02 21:55:21christian.heimessetnosy: + christian.heimes
2019-09-02 15:13:50pablogsalsetstatus: open -> closed
resolution: not a bug
stage: patch review -> resolved
2019-09-02 15:13:44pablogsalsetmessages: + msg351011
2019-09-02 14:01:01pablogsalsetmessages: + msg351003
2019-09-02 13:58:09pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request15310
2019-09-02 13:57:43pablogsalsetmessages: + msg351002
2019-09-02 13:56:33pablogsalcreate