Author vstinner
Recipients M-Reimer, bsteffensmeier, corona10, eric.snow, erlendaasland, graysky, hroncok, miss-islington, ndjensen, petr.viktorin, shihai1991, uckelman, vstinner
Date 2022-01-13.20:13:30
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1642104811.14.0.66475901898.issue46070@roundup.psfhosted.org>
In-reply-to
Content
Victor:
> (*) I made the GC state per-interpreter: commit 7247407c35330f3f6292f1d40606b7ba6afd5700 (Nov 20, 2019)

Eric Snow:
> FYI, this was done by me in an earlier comment which we ended up
reverting.  Later you basically un.reverted that.

Well, I recall that your change had to be reverted 2 or 3 times because there were many crashes on FreeBSD, and no one understood why it crashed. The root cause was bugs related to the GIL and daemon threads. It took me a while (and multiple commits) to identify and fix all of them:
https://vstinner.github.io/gil-bugfixes-daemon-threads-python39.html

I decided to split your work into smaller changes to better debug these crashes. bpo-36854 contains a few changes, but these changes are based on work that I pushed earlier.

For example, there was a tricky bug related to clearing a Python thread state:
https://github.com/python/cpython/commit/9da7430675ceaeae5abeb9c9f7cd552b71b3a93a

Also, once the GC was made per interpreter, we started to discover more and more tricky reference leaks:
https://vstinner.github.io/subinterpreter-leaks.html

I spent a significant time to reorder code of Py_Finalize() and Py_EndInterpreter() to clear objects earlier or in a different order. Recently, I made sure that the free lists can no longer be used after they are cleared. It took some notes at:
https://pythondev.readthedocs.io/finalization.html

One of the hardest fix was the commit 9ad58acbe8b90b4d0f2d2e139e38bb5aa32b7fb6 of bpo-19466. To make this change, first I had to fix a very old bug of PyThreadState_Clear() with commit 5804f878e779712e803be927ca8a6df389d82cdf (bpo-20526).

Well, it was a long journey and it's not done yet :-)
History
Date User Action Args
2022-01-13 20:13:31vstinnersetrecipients: + vstinner, petr.viktorin, eric.snow, ndjensen, hroncok, uckelman, corona10, miss-islington, shihai1991, erlendaasland, graysky, bsteffensmeier, M-Reimer
2022-01-13 20:13:31vstinnersetmessageid: <1642104811.14.0.66475901898.issue46070@roundup.psfhosted.org>
2022-01-13 20:13:31vstinnerlinkissue46070 messages
2022-01-13 20:13:30vstinnercreate