Author ncoghlan
Recipients barry, brett.cannon, eric.snow, ncoghlan, pitrou, serhiy.storchaka
Date 2018-06-09.04:20:56
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1528518056.72.0.592728768989.issue33331@psf.upfronthosting.co.za>
In-reply-to
Content
I think the key concern here is that we *don't* consistently add modules to sys.modules in the order their bodies are executed: we add them in a kind of sawtooth order based on the imports from __main__, and the order of import statements in the imported modules.

For example, given the following import chains:

    __main__ imports A then X
    A imports B which imports C which imports D
    X imports Y which imports B
    X then imports Z which imports C

Then the order in which modules get added to sys.modules will be:

    __main__, A, B, C, D, X, Y, Z

and they'll get cleaned up from left to right

(We're making the assumption here that, for whatever reason, GC hasn't cleaned up A, X, and their dependencies after sys.modules got cleared)

This means that in the status quo, unloading X, Y, and Z can have problems, since B, C, and D will already be gone.

Reversing the order doesn't fix that, and if anything will make things worse, as it means that in the "A -> B -> C -> D" dependency chain, A now gets cleared *last*, instead of getting cleared first as it does today.

So instead of just reversing the order, I wondering if what we may want to consider doing is to:

1. Make sure that we're dropping all the interpreter's internal references to __main__ before clearing sys.modules
2. When running through the module weakref list, force another cyclic GC run after each not-yet-cleared module gets cleaned up
History
Date User Action Args
2018-06-09 04:20:56ncoghlansetrecipients: + ncoghlan, barry, brett.cannon, pitrou, eric.snow, serhiy.storchaka
2018-06-09 04:20:56ncoghlansetmessageid: <1528518056.72.0.592728768989.issue33331@psf.upfronthosting.co.za>
2018-06-09 04:20:56ncoghlanlinkissue33331 messages
2018-06-09 04:20:56ncoghlancreate